Idempotent content migrations: deterministic IDs and content-hash asset dedup
A content migration you can only run once is a migration you will fear. Use createOrReplace with IDs derived from the source, and content-hash asset dedup, so partial failures and iteration are "fix and run again" instead of hand-cleaning production. Idempotency does not catch bad mapping — pair it with a diff you actually read.
Create is the wrong verb
Most CMS client libraries give you create, createIfNotExists, createOrReplace, and patch. The one you reach for by default is create, and it is the one that makes your script single-use. Using createOrReplace with an ID you control means the second run overwrites the first with identical content and nothing changes.
The tradeoff is honest: createOrReplace clobbers editor changes. If someone edited a document in the CMS since the last run and you re-run the seed, their edit is gone. That is the correct behavior during a migration, when the source system is the truth, and the wrong behavior afterward.
So we treat the seed script as a migration tool with a shelf life. It runs during build-out, it runs when we add content in bulk, and once editors are working in the CMS it stops being something you run casually. That transition should be explicit — a note in the README, ideally a confirmation prompt — rather than something everybody is assumed to know.
Derive IDs from the source, not from insertion order
The whole scheme rests on the document ID being a pure function of the source content. A slug works: project.jeffrey-dungan, note.content-that-falls-back. A stable primary key from the old system works. A counter does not, and neither does a random UUID cached in a local file, because the moment two people run the script from different checkouts you get two sets of documents.
Slug-derived IDs have one weakness worth naming. If the slug changes, the ID changes, and you get a new document instead of an updated one. The old document sticks around, unreferenced, and surfaces in the CMS as a confusing duplicate. We have hit this. The mitigation is to treat a slug change as a deliberate operation with a redirect and a deletion, not something that quietly happens because someone edited a spreadsheet.
Prefixing IDs by type helps more than it seems like it should. When you are staring at a document list trying to work out what a script just did, project.watkins-electric tells you everything and a UUID tells you nothing.
Each layer removes one way that running it twice could hurt you.
Assets are where naive migrations get expensive
Documents are cheap to overwrite. Assets are not. Most CMS asset APIs will happily accept the same image twenty times and hand back twenty asset records, each with its own ID and its own storage. Run a naive script five times against a site with a hundred images and you have five hundred assets, most of them identical, and a media library nobody can use.
The fix is to hash the file bytes before uploading and let the hash decide. Compute a SHA of the buffer, check whether an asset with that hash already exists, upload only if it does not, and reference it either way. Some platforms do this for you. Assume they do not until you have verified it, because the failure is silent and only ever shows up as clutter.
On our own site migration this mattered a lot, because we ran the import many times while iterating on the mapping. Without content-hash dedup we would have poisoned the media library on the second run and spent an afternoon deleting assets by hand. With it, the twentieth run uploaded nothing and finished in seconds.
The real payoff is that you stop being careful
Idempotency's benefit is not correctness in the abstract. It is that it changes how you work. A script you can only run once gets run once, at the end, from a branch that has been sitting for a week, with everyone watching. A script you can run any number of times gets run twenty times while you are still figuring out what the mapping should be.
That is the actual value. The migration stops being an event and becomes part of the development loop. Change the mapper, re-run, look at the site, change it again. Mistakes cost a minute instead of an evening.
It also makes partial failure survivable. Halfway through a hundred documents the network drops, or the API rate-limits you, or you hit a record with a field shape you did not anticipate. With a re-runnable script, recovery is: fix it, run again. Without one, recovery starts with working out which of the hundred made it through.
Where idempotency does not save you
It does nothing about deletions. Re-running a script that creates or replaces fifty documents will not remove the three documents you created last week under an older naming scheme. Orphans accumulate silently, and they are the thing that makes a migrated CMS feel messy six months on. If the source is authoritative, the script should also reconcile: list what exists, compare against what should exist, and at minimum report the difference.
It also does not protect the mapping itself. A script can be perfectly re-runnable and still write the wrong values, because the field-by-field mapper it uses forgot a field. We have done exactly that, twice, on the same project. Idempotency means you can fix it and re-run. It does not mean you will notice.
So the pairing we have settled on is idempotent writes plus a diff somebody actually reads. Re-run, then compare what landed in the CMS against the source, field by field, and print the mismatches. It takes an hour to write and it catches the class of bug that idempotency alone leaves wide open.
Questions
- Why is create the wrong default for CMS migration scripts?
- create makes re-runs duplicate documents. createOrReplace with an ID you control overwrites with identical content and no-ops cleanly. Treat that clobber behavior as correct while the source system is truth, and stop casual re-runs once editors own the CMS.
- How should migration document IDs be chosen?
- As a pure function of the source — slug or stable legacy primary key — not counters or random UUIDs cached per machine. Prefix by type. Treat slug changes as deliberate redirects/deletes so you do not orphan old IDs.
- How do you avoid flooding the media library on re-runs?
- Hash file bytes before upload; reuse an existing asset with that hash. Many platforms will happily store the same image twenty times with twenty IDs unless you dedupe yourself.
Sources
- Sanity — ID / document identity
- Sanity JS client mutations
- Sanity — asset pipeline — Verify platform dedup; assume you must hash until proven otherwise.