in 12 hours

I built a component to make high frequency updates work on Convex (kinda)

So, a while back I made a video about a neat little trick that a convex colleague showed me. It's a way of faking really high frequency updates like multiplayer cursors without writing to your database every single time the mouse moves. So, you can build an app like this which has got nice and smooth live drawing that works in multiple tabs with only a few updates a second. [music] And I still think this is such a clever little trick that when somebody asked me in the comments, why did I just turn it into a convex component like the presents one, [music] I said, you know what, that's a great idea. I'll make a note to look into that. Well, that was an embarrassingly long time ago now. [music] New Year's Day, uh, to be exact. Anyways, better late than never, I say, as I've finally gotten around to publishing it as a generic comics component called [music] the batched replay component. So, in this video, I'll show you a bunch of neat little use cases for it as quickly as I can. [music] Oh, and stick around to the end because I've got a bit of a spicy take on whether you should actually use this in production or not. [music] So, make sure you grab yourself a lovely cup of tea, make yourself comfortable, and let's dive into it. [music] So, if you haven't seen the old video, the TLDDW is this. Say you want to see everybody else's cursor moving around in your app. Well, to do that, your mouse fires something like, I don't know, 60 times a second. [music] And if every one of those was a write to the ComX database, then that would be 3,600 writes a minute for every person. And then each one of those updates is then going to get pushed out to everybody because of the way that you know, use query works. So that's going to be a lot of function calls for a transactional database like Convex. So the question is, how do you get around this kind of thing then? Well, the obvious simple thing to do would be just to have, you know, less client events. So let's say I don't know four a second instead [music] of 60 a second. But if we do that then what everybody else sees is the cursor is kind of like jumping around. [music] Now we can use tricks like linear interpolation but if you do that you still get kind of like a a janky behavior. So we need another idea. If instead of throwing away all those updates, we can do what my colleague uh Ian, the absolute goat, by the way, showed me is you can collect all those movements up into a batch and then send them all to the server in one go. Then on everybody else's screen, you basically [music] just get that batch pulled down and then just replay through that batch with the exact same timing as it was recorded with. And if you do that, then you get that nice buttery smooth motion, but [music] still with only four writes to the database per second or whatever you want to set it to. The catch that the eagle-eyed amongst you may already realize is that if you do this, then the replays on the client ends up being a little bit delayed. So at four times a second, it'll be about a third of a secondish delayed. But, you know, I mentioned this in my last video, but for many applications, that doesn't matter so much. I mean, sure, the delay is obvious when you open it up like two tabs side by side like this, but this isn't usually how your user will be using your application. I mean, they'll be far enough apart and thus won't really notice the lag. Oh, and there is one extra benefit of like saving these batches into the comics database like this [music] is it gives us the ability to, you know, time travel to go back in time. [snorts] To show you what I mean, let's take a look at this basic drawing app that I had Opus [music] create for me. So, I've got two copies of the same drawing app open. I can draw on here. And you see that you get a nice smooth pen movement coming across on the other uh window. [music] And again, I can also draw draw on this one. You get it on the other one, except it's delayed a little bit, but you get that nice smooth curves. Now, if I just reload the [music] page, then because everything is stored on the server and not just the latest position, I get all of the the the rights that have happened up to that, including all the stroke information. It also means I can play everything back in a time lapse like this. So, you can see exactly what happened. Oh, that was a bit too quick. Let's play back slower. There we go. So, you can see exactly what happened because each of those move each of those batches [music] was stored on the server. So, you just you just replay through all those batches. Now, you might be thinking it might be expensive [music] to replay all those batches every single time you want to just get like the latest drawing. Like you can imagine if we had lots and lots and lots and lots of drawings on here, then it would [music] have to, you know, load all of the previous batches just to get to the final state. And yes, that would be an expensive way of doing things. That's why the component also lets you optionally save out to your own table after each batch. So you [music] can store, you know, just what the current snapshot of the drawing looks like at [music] any time. Right. Anyway, that's uh been pretty chill. So, let's have a look at something a bit faster. This is a a tiny racing game that I had Opus draw uh create for me. So, you can drive this little uh car around on one tab, and you can see that it shows up on the other tab. Again, this is all done using Convex. And then once you've gone around, you get a ghost uh driving around because we have that same history batch information. And I think ghosts are a really great way of showing this off because, you know, it's already in the past a little bit. So the third of a second extra delay doesn't really matter. So, this is great for, you know, this demo um and for ghosts if you're just doing a single player thing, but maybe for um multiplayer where you want like, you know, millisecond perfect finishes to see who won the race, maybe not such a good application. Now, there is one place where you really will feel the delay. Um let me just show you this demo. If I just open up this QR code on my phone, then now if I click use the tilt sensor allow, I can now uh move this thing on my phone and try and move the uh marble maze on the screen. Oh dear. But because it's delayed, it makes it really kind of tricky to use because there's like a 250 millisecond delay. Oh god. Oh, I was never very good at these games anyway. But I can set this to 50 milliseconds and that should be much more responsive now. Oh yeah, that's way more responsive. So I can go. It's still not going to help me that I'm not very good at these games. But ah that way. That way please. That way. Anyway, you get the picture. But yeah, as we send batches more often, the delay is obviously going to drop. But [music] the more writes that you do, the more expense you're going to have on your convex, you know, transactional database. And we should talk about that in a minute. Actually, [music] let's talk about limits first, actually. So, one question I had for Opus that was helping me out with all this was how many people can you actually realistically fit into one room? You know, one of those drawing sessions or one of those car racing sessions. So, Opus went away and did a bunch of [music] testing and using both the local deployment and the cloud deployment, his conclusion was that you could get like a 100 people or so into one room together, but probably it's probably going to be most comfortable around 50 people [music] or so. And that's just mainly going to be because the way this is going to have to work, you you send the batch to the server and the way that Convex works, it's going to then update all the clients at the same time. [music] So it means that the more clients that connect the more data that has to be [music] sent out every single um batch update. It's like O to the square I think [music] is the uh big big O notation of it. So that 50 number isn't earthshattering if you were thinking about like a big, you know, massively multiplayer game, [music] but you know, 50 people all wiggling their mouse at exactly the same time. It's [music] a lot for like most kind of apps that you're going to want to build. You know, like a like a Google doc with like multiple people in it or like a Figma doc or something like that. It's likely that you're never really going to get near that 50 [music] person limit. And obviously, you don't need to send any batches if the mouse [music] doesn't move at all. So likely you would never get near that limit. Anyway, now I do want to just point out that for each of these demos, [music] it's all using exactly the same component. The only difference really is the kind of data that gets sent [music] in that batch from the client. And that's kind of the beauty of Convex components is that you can just drop these bad boys into like any number of application [music] and leverage all that power without having to reinvent the wheel each time you want to do it. Right. So, just before I let you go, here's that spicy take that I promised at the start of this video. And [music] it's all about the cost of doing this. Now, if you were looking closely and you [music] know hadn't backgrounded me on another tab while you were doing something else, you weren't doing that, were you? [music] Then you may have noticed these little counters on the demos. Uh they basically show you how many function calls estimated your tab has used. [music] So every batch you send obviously is going to be a function call plus another one for every single person it's going to be pushed out to. [music] Now Convex is pretty clever here. You know it's only going to run that query once and then everybody else is going to get the cached result. [music] Sadly though unless you're on the business plan each one of those cache results still is going to count as a function call. So when I had Opus measure it, one person moving for 1 minute sends about, you know, 220 batches, [snorts] but with 20 people watching, [music] that 220 turned into n nearly 5,000 function calls. Now, the free plan does give you a million function calls a month, and after that, it's around, you know, $2 per million. So [music] to put that in perspective, remember those 50 people from before all wiggling their mouse at exactly the same time? [music] That's going to be roughly 10,000 function calls a second or roughly $70 an hour. So, would I recommend doing it exactly like this on Convex in production? Well, [music] I guess it really depends upon your workload. If you're aiming to build a firsterson shoot on Convex, probably not. [music] I mean it might work but I think just the overhead cost you get for using a serverless database like convex might be just too much. [music] Again though if you are on the business plan you get a dedicated deployment and function calls are like 10 times cheaper. So maybe the equation changes there but usually you know right tool for the right job. Of course you can always you know turn that dial down on how many batches get [music] sent. So that's going to mean fewer function calls with a higher delay. But yeah, I mean [music] that really depends upon your use case. Now you might be wondering, can we just skip storing into [music] the convex database entirely and maybe just fire these updates directly over the websocket to the users? [music] Sadly, Convex doesn't give us direct access to the raw websocket, although I have asked for it [music] quite a few times. So, we can't do that just yet. But there is some rumblings internally about something called memory tables [music] that might help us out here. So stay tuned for that. But anyway, I think for small rooms or things where the history is really useful, like in that drawing time lapse I showed you, I still think this is actually a really kind of viable [music] path to doing some of these like real-time aspects directly on on convex without having to go out to another third party service. Okay. So, as I said uh at the start, the com component is called the batched replay component and you can grab it from npm or the convex components directory now. And all the demos you saw in this video are live and on the site too. So, if you want to have a play with them, I've left the link to them all down below. I just want to say a massive big thanks to Ian the Go and SUJ who built the original Curses demo that all of this came from. I'm really curious if you got this far into the video. [music] Can you let me uh know in a comment down below what are you interested in using this for? [music] But anyways, I think that's all I have for you guys for now. Don't forget to drop me a like and sub. And until next time, thanks for watching. Cheerio.

So a while back I made a video about a neat little trick that a Convex colleague showed me. It's a way of faking really high frequency updates, like multiplayer cursors, without writing to your database every single time the mouse moves.

So you can build an app like this, which has got nice and smooth live drawing that works across multiple tabs with only a few updates a second:

0:00 / 0:00

I still think this is such a clever little trick, so when someone in the comments asked why I didn't just turn it into a Convex component, like Presence, I said that's a great idea, I'll make a note to look into that.

Well that was an embarrassingly long time ago now. New Year's Day, to be exact 😅

Anyways, better late than never, as I've finally got around to publishing it as a generic Convex component called Batched Replay. Oh and stick around to the end because I've got a bit of a spicy take on whether you should actually use this in production.

Let's take a look.

The trick

So if you haven't seen the old video, the TLDR is this. Say you want to see everybody else's cursor moving around in your app. Your mouse fires something like 60 times a second, and if every one of those was a write to the Convex database then that would be 3,600 writes a minute for every person. Each one of those is then going to get pushed out to everybody, because of the way useQuery works, so that's a lot of function calls for a transactional database like Convex.

So how do you get around this? Well the obvious simple thing to do would be to just have fewer client events, let's say 4 a second instead of 60. But if we do that then what everybody else sees is the cursor jumping from spot to spot. In these clips the window on the right is what everybody else sees:

0:00 / 0:00

We can use tricks like linear interpolation to glide between those spots, but you still get some janky behaviour, as the cursor moves in straight lines with sharp corners and that's not what the mouse actually did:

0:00 / 0:00

So instead of throwing away all those updates, we can do what my colleague Ian (the absolute goat by the way) showed me, which is to collect all those movements up into a batch and send them to the server in one go. Then on everybody else's screen you pull that batch down and replay through it with the exact same timing it was recorded with:

0:00 / 0:00

If you do that then you get nice buttery smooth motion, but still with only 4 writes to the database a second, or whatever you want to set it to. Noice!

The catch, that the eagle-eyed amongst you may have already spotted in that clip, is that the replay on the other clients ends up being a little bit delayed. At 4 times a second it'll be about a third of a second behind. For many applications that doesn't matter so much tho. Sure, the delay is obvious when you open up two tabs side by side, but this isn't usually how your users will be using your application. They'll each be looking at their own screen somewhere else, so they won't really notice the lag.

Oh and there's one extra benefit of saving these batches into the Convex database, which is that it gives us the ability to go back in time.

Drawing

To show you what I mean, let's go back to that drawing app from the start, which I had Opus create for me. Because every batch is stored on the server, and not just the latest position, when I reload the page I get the whole drawing back. It also means I can play everything back as a timelapse so you can see exactly what happened:

0:00 / 0:00

And that's only a couple of lines:

const replay = useHistory(api.drawing, { room, speed: 10, onSample: draw });
replay.restart();

Super cool!

Now you might be thinking it might be expensive to replay all those batches every single time someone just wants to see the latest drawing, and yes, that would be an expensive way of doing things. That's why the component also lets you optionally save out to your own table after each batch with onAppend.

In the drawing app I use that to build up each stroke, then save it to a strokes table when the pen goes up. That's the Strokes panel you can see in that last clip, and a new window just loads those strokes instead of replaying every batch. Simplified a bit, it looks like this:

onAppend: async (ctx, { room, stream, samples }) => {
  // the stroke this pen is part way through, from our own table
  let stroke = await getCurrentStroke(ctx, room, stream);

  for (const { v: sample } of samples) {
    stroke.points.push({ x: sample.x, y: sample.y });

    if (sample.k === "up") {
      await ctx.db.insert("strokes", { room, stream, points: stroke.points });
      stroke = { points: [] };
    }
  }

  await saveCurrentStroke(ctx, room, stream, stroke);
},

You can see the full version in the example app. It runs in the same transaction as the append, and each person's stroke lives in its own document, which matters. If everybody wrote to one shared document for the whole room then all those appends would conflict with each other and you'd be right back to the problem this component is meant to avoid.

Ghost racer

Right, that's all been pretty chill, so let's have a look at something a bit faster.

This is a tiny racing game that I also had Opus create for me. You can drive this little car around on one tab and it shows up on the other tab, and once you've gone round you get a ghost of your best lap driving around too (that's the see-through glowy car), because we have that same batch history:

0:00 / 0:00

I think ghosts are a really great way of showing this off because a ghost is already in the past, so the extra third of a second doesn't really matter. It's maybe not such a good application for multiplayer tho, where you want millisecond perfect finishes to see who won the race.

Phone controller

Now there is one place where you really will feel the delay. This demo gives you a QR code to open on your phone, then you tilt your phone to roll a marble around this maze on your laptop:

0:00 / 0:00

With batches going out every 250 milliseconds it's really kinda tricky to use, because you tilt the phone and the marble doesn't react until about a third of a second later, so you keep overshooting. Luckily there's a dial for this, which is how often each batch gets sent:

const send = useSender(api.tilt.append, { room, stream: tabId, flushMs: 50 });

At 50 milliseconds it's way more responsive. Not that it helps me much, as I'm not very good at these games 😅

Here's that same dial on the cursors demo, so you can see the trade-off. As it goes down the writes a second go up and the delay on the right drops, and as it goes up you get the opposite:

0:00 / 0:00

So at 50 milliseconds you are only about 130 milliseconds behind, but that's somewhere around 15 to 20 writes a second instead of 4, and the more writes you do the more it's going to cost you. We should talk about that in a minute actually.

How many people can you fit in a room?

So one question I had for Opus, which was helping me out with all this, was how many people can you actually realistically fit into one room? By a room I mean one of those drawing sessions or racing sessions. Opus went away and did a bunch of testing, and its conclusion was that you could get like 100 people or so into one room together, but it's probably most comfortable around 50.

People in one room	Write time (p50)	Each viewer receives
10	about 330 ms	45 KB/s
25	about 360 ms	120 KB/s
50	about 460 ms	240 KB/s
100	about 1.5 s	460 KB/s

That's on a cloud dev deployment with everyone moving at 60 Hz and batches every 250 ms. It also ran from Perth to a US region, so about 300 ms of each write is just the network (the joys of living in Australia).

The limit is mainly down to how this has to work. You send a batch to the server and Convex then updates all the clients, so the more clients that connect, the more data has to go out for every single batch. It's O(n²), I think, if you are into that sort of thing. There were no errors at any of those sizes tho, past 50 everyone just falls further behind.

So 50 isn't earth-shattering if you were thinking about a massively multiplayer game. But 50 people all wiggling their mouse at exactly the same time is a lot for most kinds of apps. Something like a Google Doc or a Figma doc with a few people in it is never really going to get near that, and you don't send any batches at all if the mouse doesn't move.

By the way, every one of these demos is using exactly the same component. The only real difference is what gets sent in the batch. So for the cursors each sample is just a position:

value: v.object({ x: v.number(), y: v.number() }),

And for the drawing it's a position plus whether the pen went down, moved or came up:

value: v.object({
  k: v.union(v.literal("down"), v.literal("move"), v.literal("up")),
  x: v.number(),
  y: v.number(),
}),

That's kind of the beauty of Convex components, you can just drop these bad boys into any number of applications and leverage all that power without having to reinvent the wheel each time.

The cost

Right, here's that spicy take I promised at the start, and it's all about the cost.

If you have a go on the demos you'll see these little counters, which show you roughly how many function calls your tab has used. Here the left window is drawing and the right window is just watching:

The drawing demo in two windows with the function call counters highlighted: 19 writes and 15 updates on the left, 32 updates on the right

Every batch you send is a function call, plus another one for every single person it gets pushed out to. Now Convex is pretty clever here, it's only going to run that query once and then everybody else gets the cached result. Sadly tho, subscription updates count as function calls, so each of those cached results still gets counted.

So when I had Opus measure it, one person moving for one minute sent about 220 batches, but with 20 people watching that turned into nearly 5,000 function calls.

The Free and Starter plans include a million function calls a month (then it's $2.20 per million on Starter), and Professional includes 25 million and then it's $2 per million. So remember those 50 people from before, all wiggling their mouse at exactly the same time? Each of them sends 4 batches a second, and each batch gets pushed out to the other 49. That's roughly 10,000 function calls a second, which is about 36 million an hour, or roughly $70 an hour on Professional 😬

So would I recommend doing it exactly like this on Convex in production? Well I guess it really depends upon your workload. If you're aiming to build a first-person shooter on Convex, probably not. I mean, it might work, but I think the overhead you get for using a serverless database like Convex might be just too much.

If you're on the Business plan with a dedicated deployment then function calls are about 10 times cheaper, at $0.20 per million, so maybe the equation changes there. Usually tho, right tool for the right job. And of course you can always turn that dial down, which means fewer function calls but a longer delay.

A natural question is whether we can just skip the database entirely and fire these updates directly over the WebSocket to the users. Sadly Convex doesn't give us direct access to the raw WebSocket, although I have asked for it quite a few times, so we can't do that.

Anyway, for small rooms, or things where the history is really useful like that drawing timelapse, I still think this is a really viable way of doing some of these realtime things directly on Convex without having to go out to another third party service.

Using it in your app

So first install it and add it to your convex.config.ts:

npm install @convex-dev/batched-replay

// convex/convex.config.ts
import { defineApp } from "convex/server";
import batchedReplay from "@convex-dev/batched-replay/convex.config.js";

const app = defineApp();
app.use(batchedReplay);
export default app;

Then expose an API. A room is a string you pick (a page, a board, a game) and a stream is one sender's key inside it. value is the shape of one sample, which also types the hooks:

// convex/cursors.ts
import { v } from "convex/values";
import { components } from "./_generated/api";
import { exposeApi } from "@convex-dev/batched-replay";

export const { append, latest, history, streams } = exposeApi(
  components.batchedReplay,
  {
    value: v.object({ x: v.number(), y: v.number() }),
    retentionMs: 10 * 60_000,
    authorizeRead: async (ctx) => {
      const identity = await ctx.auth.getUserIdentity();
      if (!identity) throw new Error("Unauthorized");
    },
    authorizeWrite: async (ctx, { stream }) => {
      const identity = await ctx.auth.getUserIdentity();
      if (!identity) throw new Error("Unauthorized");
      return `${identity.tokenIdentifier}:${stream ?? "default"}`;
    },
  },
);

Leave out retentionMs and the batches are kept forever, which you want for something like a drawing but probably not for cursors. Oh and a client can call append as fast as it likes, so for anything public you will want to put the rate limiter component in authorizeWrite too.

Now on the client one hook sends and the other plays back. Here tabId is a key you make for each tab:

import { useSender, usePlayback } from "@convex-dev/batched-replay/react";

const send = useSender(api.cursors.append, { room, stream: tabId, flushMs: 250 });
onPointerMove((e) => send({ x: e.clientX, y: e.clientY }));

const { values, stats } = usePlayback(api.cursors, {
  room,
  self: tabId,
  interpolate: true,
});

values is a map of everyone who has been active in the last minute, and self means you draw your own input locally with no delay. Simple enough, right? The README has the rest.

Conclusion

So just to recap: you collect your fast moving values up into batches, send them about 4 times a second, and everybody else replays them with their original timing about a third of a second behind. You get to keep the batches too, which gives you things like timelapses and ghosts, but the function calls add up really quickly as a room gets bigger, so do the maths for your own app first.

What I like most about this is that it's just a component on plain Convex, with no extra service to set up. All the demos from the video are live if you want to have a play, and you can grab it from npm or the components directory.

A massive thanks to Ian the GOAT and to Sujay, who built the original cursors demo that all of this came from.

Until next time, cheerio! 👋

All gas, no breakages

Convex is the reactive backend platform that keeps up with you and your agents. Database, functions, workflow, sync, search, file storage, and more. All TypeScript, zero glue.

Get started