Mike Cann's avatar
Mike Cann
2 months ago

The secret to migrations

Every non-trivial application eventually needs a database migration. Plans change, someone decides to add a new feature or rework an existing one, and now your well-designed schema needs to change. But how do you do that when you already have hundreds, thousands, or millions of rows in your database? You can't just change the schema. Convex won't allow it.

There's a related question that trips people up just as often. Why can't you drop a column, the way you can in Postgres?

If those questions sound familiar, you're not alone. I had them when I first started using Convex. There are good reasons it works this way. That doesn't make them any less annoying the first time you hit them, and database migration is the topic I get asked about more than anything else right now. The answers get at what makes Convex Convex, which is also why I like it. Let's get into it.

How Convex deployments work

To understand migrations, it helps to back up and look at what makes up a Convex app. Your app, technically a deployment, is made of two things: your functions and your data. Your functions, your queries, mutations, and actions, operate on the data.

Implicit versus explicit schemas

Every Convex table has a schema that defines the shape of your data. You can choose not to define one explicitly. Open the Convex dashboard, create a project, go to the data tab, make a table, and insert data of any shape. Convex still maintains an implicit schema behind the scenes, updating it automatically whenever the data changes. Change a value from a number to a string, and the inferred type becomes a union of float64 and string.

This implicit mode is handy for experimenting, but I wouldn't recommend it in production. Instead, you typically define an explicit schema in code, in schema.ts. Once you do, Convex flips into a more restrictive mode: it won't let you insert or update rows that don't conform to that schema.

Data storage and JSON documents

Even though your data shows up as tables in the dashboard, don't think about it that way, because that's not how it's stored behind the scenes. It's a series of JSON documents in a list, and the explicit schema restricts what can and can't be written into that list. If you self-host Convex locally with the default SQLite backend, you'll see a value column of type JSON in the documents table, and it's just a JSON object sitting there.

That storage model is the reason migrations work the way they do. With an explicit schema, Convex guarantees that every document in the database matches that shape. Functions and schema form a contract. Convex generates a TypeScript interface for you, so TypeScript complains if you try to push something of the wrong shape. You can turn off TypeScript checking, but Convex still catches invalid writes at runtime. Think of your functions and your schema as a single unit, because that's exactly how Convex deploys them, atomically and together.

Atomic deployments and zero downtime

That atomicity is what makes zero-downtime updates possible. Once everything is uploaded and Convex is happy with the schema, it switches all traffic from the old version to the new version without users noticing. It gets trickier with long-running actions, but that's a detail Convex handles for you.

So long as the V1 schema can safely transition to the V2 schema, everything is fine. If it can't safely transition, the function update gets rejected too, because functions and schema move together.

This raises the real question. What does it mean for a schema to "safely transition"? That's what a database migration really is.

The three-step migration process

Take a simple example. Say you have a users table with three rows, and in V1 the schema has name: string and age: number. Now you want to add an email field. If you try this in code with the Convex dev server running, Convex won't let you, because it would require every existing document to already have an email field, and yours don't:

1// convex/schema.ts — adding a required field to a table that already has rows
2users: defineTable({
3  name: v.string(),
4  age: v.number(),
5  email: v.string(),
6}),
7

Convex refuses to apply it and prints a schema validation error: the existing documents in users are missing the required email field. So how do you solve it?

One option is to run a function that iterates through all rows before the new schema is applied and backfills the email address on each one. That would make the schema valid before you flip it over. But that function could take a long time, especially if you have to hit an external service per user to find an email address. Time isn't something you have if you want a zero-downtime deployment. If versions increment atomically, you can't have a moment where some rows are valid and some aren't. The only way to force that would be to bring the whole system down, stop traffic, run the migration, and bring it back up. People used to do that, and some still do, especially in games, but it's not ideal.

The other instinct is to reach for a default value, the way you would in SQL. Convex doesn't have a default concept, and I don't think it's going to get one. What would a default email address even mean? The same question applies to most fields you'd actually want to add.

The practical solution is a three-step process:

  1. Loosen the schema. Make the new field optional, so email can be a string or not exist yet. Now the schema matches your existing data. The trade-off is that your functions have to handle both possibilities, present or absent, until step three.
  2. Migrate the data. Run a migration script that updates old rows to the new shape, for example, backfilling email addresses by calling out to an external service. At the same time, make sure new inserts always provide an email address, so you're not digging the hole deeper while you're filling it in.
  3. Tighten the schema. Once every row has an email, update the schema to make email required again and remove the extra handling from your functions. Convex allows this final change because every document already conforms to the tightened schema.

Three-step schema migrationThree-step schema migration

That loosen-migrate-tighten sequence is what lets you keep zero downtime while still guaranteeing data integrity.

Yes, that's extra work for what looks like a simple schema change. Schema changes are one of the more expensive things in software development, regardless of which database you're using, so it's worth thinking hard before making one that's difficult to reverse. That's especially true now that AI tools are often the ones proposing the change. If an AI edits schema.ts, review that diff carefully. Your server-side functions depend tightly on the schema, and the generated types flow all the way to the frontend, so a careless change can ripple across the whole app.

When migrations are not required

There are situations where you can change the schema without going through any of this:

  • If there's no data in the table yet, change the schema freely.
  • When you're developing locally, just clear your tables and change the schema without a migration.
  • You can make a schema strictly more permissive without a migration, like making a field optional or adding a member to a union, because that new shape matches both the old and new forms of the data.
  • If you've got an optional field you've never populated, delete it safely, since there's no data depending on it.

Changing the type of an existing field is the exception. It follows the same loosen-migrate-tighten pattern: turn the field into a union first, migrate the data, then remove the union once everything conforms.

Why dropping columns is complex

Why can't you just drop a column the way you would in Postgres? Convex is a document database storing data as JSON objects, not columnar tuples. Dropping a field would mean either scanning every document and physically removing that field, or shaping each document on every read to exclude fields the current schema no longer declares. Both are expensive and add complexity that a columnar database with fixed-width rows doesn't have to deal with. Given that trade-off, the loosen-migrate-tighten pattern ends up being the more predictable option.

Tools for database migrations on Convex

Convex has a migrations component that handles the batch-processing side of this for you. You define a function that operates on a single row, and the component iterates through the table in batches, tracking progress as it goes. It has enough options to fit a range of different workflows, so you're not writing the pagination and progress-tracking logic yourself every time.

There's also an agent skill that condenses the pattern described above into something an AI agent can follow. Install it globally with npx skills, or ship it to your project with npx convex ai files install. Combine the component, the skill, and a capable modern coding agent, and you can often just ask the agent to run all three migration steps: open the PRs, run the tests, deploy, run the data migration, and do the final schema tightening. In my experience it works, and it's noticeably less effort than babysitting each step by hand.

Rollback and roll forward

Rollback with migrations is more complicated than a normal deploy. If you haven't yet run the data migration, you can revert the code and schema back to the previous working state without much trouble. But if you're partway through a data migration when something breaks, it's usually best to roll forward rather than back, applying another update that migrates from the broken state to a known-good one. That's more painful to execute, but because Convex's deployments are atomic and zero-downtime, users shouldn't notice either way.

If you want the deeper architectural picture behind all of this, that's what "How Convex Works" covers.

All gas, no breakages

Convex is the reactive backend platform that keeps up with you and your agents. Database, functions, workflow, sync, search, file storage, and more. All TypeScript, zero glue.

Get started