Mike Cann

3 days ago

How hard is it to migrate AWAY from Convex?

(Psst) Don't tell my boss this, but I just spent the past 2 days working on this just so I can show how easy it is to move AWAY from Convex.

Lock-in is something that has always concerned me, and I think you guys too, judging by this comment from Theo's latest video on Convex.

So that sent me down the path of finding out: how hard would it actually be to migrate away from Convex?

Self Hosting

Now as @xWe2s mentioned, Convex is open-source and as such comes with the ability to self-host the entire backend including the dashboard. There's full documentation on it over on GitHub.

https://github.com/get-convex/convex-backend/blob/main/self-hosted/README.md

I think this would certainly be the quickest and easiest way of going about migrating away from the lock-in of the Convex Cloud.

Buuuut I think that would be defeating the purpose for many people. I would presume if you were afraid of the lock-in like the author mentioned, then you would be looking to move to a different database and set of technologies altogether.

So with that in mind, let's go on a journey to see how hard would it REALLY be to move from Convex to something else?

Functions

Alright, so as a starting point I'm going to npm create convex@latest and go with the Tanstack Start template.

If you aren't familiar with this template, it's very simple really. You can click a button that adds numbers to the array that show on screen. It's a great way to showcase a simple schema, query and mutation.

So let's take a look at the Convex code for this

1import { defineSchema, defineTable } from 'convex/server'
2import { v } from 'convex/values'
3
4export default defineSchema({
5  numbers: defineTable({
6    value: v.number(),
7  }),
8})
9

So first we have a very simple schema with just one table numbers with just one field value.

Now let's look at the Convex functions:

1import { v } from 'convex/values'
2import { query, mutation, action } from './_generated/server'
3import { api } from './_generated/api'
4
5export const listNumbers = query({
6  args: {
7    count: v.number(),
8  },
9
10  handler: async (ctx, args) => {
11    const numbers = await ctx.db.query('numbers').order('desc').take(args.count)
12    return numbers.reverse().map((number) => number.value)
13  },
14})
15
16export const addNumber = mutation({
17  args: {
18    value: v.number(),
19  },
20  handler: async (ctx, args) => {
21    const id = await ctx.db.insert('numbers', { value: args.value })
22    console.log('Added new document with id:', id)
23  },
24})
25
26export const myAction = action({
27  args: {
28    first: v.number(),
29  },
30  handler: async (ctx, args) => {
31    const data = await ctx.runQuery(api.myFunctions.listNumbers, {
32      count: 10,
33    })
34    console.log(data)
35    await ctx.runMutation(api.myFunctions.addNumber, {
36      value: args.first,
37    })
38  },
39})
40

Now as you're probably aware, Convex has 3 kinds of functions: Queries, Mutations and Actions. Queries and Mutations can directly access the database and are transactional, and Actions are not.

So I think the first step would be thinking about how we could move these away to something else. The way I think I'm going to tackle this is to convert these into Tanstack Start server functions.

If you aren't familiar with Tanstack Start, it's a really cool new framework for building React-powered Server Side Rendered applications very much like NextJS. I probably could have done this part of the migration with NextJS, but I just got done making a video on Tanstack Start so I am more familiar with it at this point.

Anyway, let's stub out our functions using Tanstack Start:

1import { createServerFn } from '@tanstack/react-start'
2import { z } from 'zod'
3
4let numbers = [69, 99]
5
6export const listNumbers = createServerFn()
7  .validator((input: { count: number }) =>
8    z.object({ count: z.number() }).parse(input),
9  )
10  .handler(async ({ data, context }) => {
11    return numbers.slice(0, data.count)
12  })
13
14export const addNumber = createServerFn()
15  .validator((input: { value: number }) =>
16    z.object({ value: z.number() }).parse(input),
17  )
18  .handler(async ({ data }) => {
19    numbers.push(data.value)
20  })
21
22export const myAction = createServerFn()
23  .validator((input: { first: number }) =>
24    z.object({ first: z.number() }).parse(input),
25  )
26  .handler(async ({ data }) => {
27    const numbers = await listNumbers({ data: { count: 10 } })
28    await addNumber({ data: { value: data.first } })
29  })
30
31

So here we can see that it looks pretty similar to our Convex functions. We have our listNumbers query here at the top that validates our input with zod instead of convex's own validator library.

For now I have just mocked out the numbers to this array at the top, we will look at the database part in a minute. We can see however that our Mutation addNumber again is a TanstackStart server function with a Zod validator writing into our mock "database" numbers array.

And finally our myAction server function is able to call into our query and mutation directly and await the results as with our Convex action from before.

Right so thus far the migration is looking pretty simple. How does it look on the client side?

So before this is how we would run our query:

1const numbers = useQuery(api.myFunctions.listNumbers, { count: 10 })
2

And this is how we would use our mutation:

1const addNumber = useMutation(api.myFunctions.addNumber)
2

And then we would call the mutation like this:

1addNumber({ value: Math.floor(Math.random() * 10) })
2

All pretty simple and clear if you are familiar with Convex.

Now, to call our new Tanstack Start functions we need to swap out our query with something like this:

1 const { data: numbers } = useQuery({
2    queryKey: ['listNumbers', { count }],
3    queryFn: () => listNumbers({ data: { count: 10 } }),
4  })
5

So here we are using React Query with a queryKey and a query function to call our listNumbers server function with a count of 10.

Then to do a mutation we would do:

1 const addNumberMutation = useMutation({
2    mutationFn: (value: number) => addNumber({ data: { value } }),
3  })
4

And call it like this:

1addNumberMutation.mutate(Math.floor(Math.random() * 10))
2

So far so good, this looks quite a lot like our Convex code.

We aren't quite done yet though. You see one of the biggest selling points of Convex is that everything is "reactive" that means that our call to useQuery here is a "live query".

1const numbers = useQuery(api.myFunctions.listNumbers, { count: 10 })
2

So when we add a number down here that query is going to be automatically updated for us.

1addNumber({ value: Math.floor(Math.random() * 10) })
2

Unfortunately Tanstack Start / React Query doesn't have this so instead we need to add a few more lines to our mutation to manually "invalidate" our local query causing it to re-run to get our numbers once again.

1 const queryClient = useQueryClient()
2  const addNumberMutation = useMutation({
3    mutationFn: (value: number) => addNumber({ data: { value } }),
4    onSuccess: () => {
5      queryClient.invalidateQueries({ queryKey: ['listNumbers'] })
6    },
7  })
8

Now when we try it out everything should work as before.

Unfortunately though if I stop my Tanstack Start dev process and start it again we lose our lovely set of numbers. Whoops!

That's because we aren't actually saving our numbers to the database so let's take a look at that part now.

Database

Right so I think so far we are doing pretty well with "de-lock-in-ify-ing" (if that's not a word it should be). We have migrated our server functions to a nice open-source non locked-in version in the form of Tanstack Start.

For the database I want to do the same. I don't want to be locked into another cloud database, so that means Firebase and Supabase are out so I think I'll just go with a regular old Postgres Database running locally.

Don't worry though, I'm not going to call the database with raw SQL like some kind of savage.

I'm going to pull in DrizzleORM for a Convex-like API to access my local Postgres database.

So if we can setup a Drizzle schema like this:

1import {
2  pgTable,
3  serial,
4  text,
5  boolean,
6  numeric,
7  timestamp,
8} from 'drizzle-orm/pg-core'
9
10export const numbersTable = pgTable('numbers', {
11  _id: serial().primaryKey(),
12  createdAt: timestamp().defaultNow().notNull(),
13  value: numeric().array().notNull(),
14})
15
16

Note we had to define the _id and createdAt ourselves, with Convex we get those automatically with every table. And we have our array of numbers.

We have to manually manage our migrations with Postgres so after a quick npx drizzle-kit push

1npx drizzle-kit push
2

we can go ahead and update our server functions like so:

1import { createServerFn } from '@tanstack/react-start'
2import { z } from 'zod'
3import { drizzle } from 'drizzle-orm/node-postgres'
4import { numbersTable } from '~/db/schema'
5import { desc } from 'drizzle-orm'
6
7const db = drizzle(process.env.DATABASE_URL!)
8
9export const listNumbers = createServerFn()
10  .validator((input: { count: number }) =>
11    z.object({ count: z.number() }).parse(input),
12  )
13  .handler(async ({ data, context }) => {
14    const numbers = await db
15      .select()
16      .from(numbersTable)
17      .orderBy(desc(numbersTable.createdAt))
18      .limit(data.count)
19
20    return numbers.map((row) => row.value)
21  })
22
23export const addNumber = createServerFn()
24  .validator((input: { value: number }) =>
25    z.object({ value: z.number() }).parse(input),
26  )
27  .handler(async ({ data }) => {
28    await db.insert(numbersTable).values({ value: [data.value.toString()] })
29  })
30
31export const myAction = createServerFn()
32  .validator((input: { first: number }) =>
33    z.object({ first: z.number() }).parse(input),
34  )
35  .handler(async ({ data }) => {
36    const numbers = await listNumbers({ data: { count: 10 } })
37    console.log(numbers)
38    await addNumber({ data: { value: data.first } })
39  })
40

Now our listNumbers is going to select from our numbersTable order by createdAt descending before returning just the numbers.

Our addNumber mutation is going to insert the value into the database. Note the conversion to string. I think this is something to do with Postgres numeric vs JavaScript number precision issues.

After taking this for a quick spin we can see that as before we can add a random number and again it invalidates our local state and calls our listNumbers query again.

And this time if we restart our dev server.

Then refresh the page.

Our numbers are still there, nice!

Cleanup

Now this is good and all, we have converted our Convex Functions over to Tanstack Start Server Functions and our Convex Database over to DrizzleORM Postgres with not a lot of effort.

Buuuut this code is starting to look less and less like Convex and is starting to look like more and more work if we had a lot of Convex code in our repo this migration might take a while.

Let's see if we can improve this. Me and my buddy Claude just knocked out a little wrapper for Drizzle so that our server functions can now look like this:

1import { createServerFn } from '@tanstack/react-start'
2import { z } from 'zod'
3import { createDrizzleConvexContext } from '~/lib/drizzleConvex'
4
5const ctx = createDrizzleConvexContext(process.env.DATABASE_URL!)
6
7export const listNumbers = createServerFn()
8  .validator((input: { count: number }) =>
9    z.object({ count: z.number() }).parse(input),
10  )
11  .handler(async ({ data, context }) => {
12    // Now using the Convex-like API!
13    const numbers = await ctx.db.query('numbers').order('desc').take(data.count)
14    return numbers.reverse().map((number) => number.value)
15  })
16
17export const addNumber = createServerFn()
18  .validator((input: { value: number }) =>
19    z.object({ value: z.number() }).parse(input),
20  )
21  .handler(async ({ data }) => {
22    // This now handles the string conversion automatically
23    const id = await ctx.db.insert('numbers', { value: data.value })
24    console.log('Added new document with id:', id)
25  })
26
27export const myAction = createServerFn()
28  .validator((input: { first: number }) =>
29    z.object({ first: z.number() }).parse(input),
30  )
31  .handler(async ({ data }) => {
32    const numbers = await listNumbers({ data: { count: 10 } })
33    console.log(numbers)
34    await addNumber({ data: { value: data.first } })
35  })
36
37

So now our database query listNumbers query looks just like our Convex one! The same goes for our addNumber mutation, looks just the same as our Convex one.

I think we can go one step further though by converting those server functions to something a bit more familiar:

1import { createServerFn } from '@tanstack/react-start'
2import { v, query, mutation, action } from '~/lib/tanstackStartConvex'
3
4const listNumbersQuery = query({
5  args: {
6    count: v.number(),
7  },
8  handler: async (ctx, args) => {
9    const numbers = await ctx.db.query('numbers').order('desc').take(args.count)
10    return numbers.reverse().map((number) => Number(number.value))
11  },
12})
13
14const addNumbersMutation = mutation({
15  args: {
16    value: v.number(),
17  },
18  handler: async (ctx, args) => {
19    const id = await ctx.db.insert('numbers', { value: args.value })
20    console.log('Added new document with id:', id)
21    return id
22  },
23})
24
25const myActionAction = action({
26  args: {
27    first: v.number(),
28  },
29  handler: async (ctx, args) => {
30    const numbers = await ctx.runQuery(listNumbersQuery, { count: 10 })
31    console.log('Numbers:', numbers)
32
33    const newId = await ctx.runMutation(addNumbersMutation, {
34      value: args.first,
35    })
36    console.log('Added number with ID:', newId)
37
38    return null
39  },
40})
41

Now we have something that looks almost exactly like our Convex functions from before. We have our query, mutation and action. Each takes an args and a handler.

The args are nicely typed and we are able to access the db on the context and get nicely typed results. Our action even has the ability to run queries and mutations and get typed results back.

So in theory now you should be able to literally just copy / paste some Convex function, change a couple of exports and you should be good to go…

… kinda ...

You may have noticed that these fake Convex functions aren't actually exported.

You have to scroll down a bit to see those:

1export const listNumbers = createServerFn()
2  .validator(listNumbersQuery.validator)
3  .handler(listNumbersQuery.handler)
4
5export const addNumber = createServerFn()
6  .validator(addNumbersMutation.validator)
7  .handler(addNumbersMutation.handler)
8
9export const myAction = createServerFn()
10  .validator(myActionAction.validator)
11  .handler(myActionAction.handler)
12

It turns out that Tanstack Start's bundler expects createServerFn to be at the top level assigned to a const and exported so unfortunately that means we have to do this little bit of boilerplate rather than having nice wrappers.

Oh well it's pretty close and not that much extra boiler.

If you are interested in this glue / translation code then its available here: https://gist.github.com/mikecann/c7cdef776c65c5c7b385e12f34948d2b

Transactions

Right, so now we have an API that looks very similar to Convex's, but there is one other subtle difference that we haven't addressed, and that's Transactions.

If you have watched my "What is Convex" video, then you will know that Convex's Queries and Mutations run in Transactions, that means everything must succeed for any database writes to be accepted.

This is to prevent a very common mistake in app development which leads to corrupted state. Trust me, this is a very easy mistake to make as your project grows.

Now if we head back to our Tanstack server functions and create a little demo to try this out:

1const addNumbersWithErrorMutation = mutation({
2  args: {
3    value: v.number(),
4  },
5  handler: async (ctx, args) => {
6    await ctx.db.insert('numbers', { value: args.value })
7    if (args.value == 69) throw new Error('test')
8    await ctx.db.insert('numbers', { value: 123 })
9  },
10})
11

And then on the frontend add another button and set things up to call this

1<button
2            className="bg-dark dark:bg-light text-light dark:text-dark text-sm px-4 py-2 rounded-md border-2"
3            onClick={() => {
4              addNumberMutation.mutate(69)
5            }}
6          >
7            Add a 69
8          </button>
9

Now when we try it out

We will see that when I add a random number, we get both our random number AND 123 but if I click the "Add a 69" button, then we get just the 69.

Effectively, our data has now entered a corrupted or unexpected state that would be a nightmare to debug. So we need to handle that.

Fortunately, Drizzle does have a way to manage transactions. So I'm going to wrap our query and mutation in a transaction.

So now when we try it again and I try to add 69, bam. Nothing, as expected. The transaction has rolled back and we get an error in our terminal from the server function.

Awesome, no more unexpected state!

Now there are some subtleties here around transactions and serializability, but that's probably going to be a bit out of scope for this video. Just be aware that Drizzle Postgres transactions may not work precisely like the Convex ones.

The Rest

Alright, so at this point I think I have covered what you might think of as the "core" of what Convex is, but I hope you can see at this point it would just be a matter of plugging in those API holes with other alternatives.

• For example, the file storage API could be replaced with S3.
• The full text and vector search could be replaced with Postgres extensions or with another service like Algolia.
• If you want Realtime updates, then maybe have a look at Pusher for that.
• Pagination can be handled by Drizzle cursor-based pagination.
• Auth could be handled either yourself with a library like better-auth or passport.js, or with a third party like Clerk or Auth0.
• Scheduling could be done with a library like PGBoss or via a Redis-based queuing library like BullMQ or a more full-featured queuing system like RabbitMQ.

You get the picture though, at this point it's just a matter of going around plugging the holes and swapping out Convex features for third parties.

I think the only one that might be difficult to replicate is Convex Components. They really are a pretty unique feature with no obvious parallel to swap it out with.

Probably your best bet to replace Convex Components might be some sort of microservice architecture or to simply replace those parts of your app with something you roll yourself.

Data Lock-in

Oh and just finally before we jump into the conclusion, if you are worried about your data being locked into the Convex cloud database, then you need not worry as there are plenty of ways to backup or export your data at any time.

Conclusion

So I think then it's about time to come to a conclusion with all this.

I hope I've demonstrated that API-level lock-in isn't as bad as it might seem. The code modifications required to move away from Convex could be manageable with the right approach.

However, you'd be trading simplicity for complexity by taking on additional infrastructure management and implementing your own solutions for features Convex handles seamlessly for you.

So while it IS possible to write all this yourself, I would still say that if you are worried about lock-in on Convex cloud, your best bet is to migrate to Convex self-hosted.

So one other last point I want to mention here is that the inverse is also true.

If you find yourself writing a bunch of code to create realtime updates, typesafe RPC calls, queueing and scheduling all yourself, consider ditching that complexity and migrating TO Convex instead.

On a personal note, this is exactly why I fell in love with Convex a couple of years ago. I was struggling to implement these features properly in my own projects when I discovered a better way.

Until next time, thanks for reading,

Cheerio!