Ian Macartney

a month ago

Add a collaborative document editor to your app

Picture this: it’s 4pm on a Friday. A product manager pings you. They want to add an editable text area to an existing feature. Users want a shared place to jot down information, to share context with other users. You care about your users, and you like the product manager, so you crank it out: a simple <textarea>, with a new string field in the feature’s existing database table. You’re feeling accomplished, and ready to head home.

Check it out, you say, and send them a link. As you’re both playing with it, you notice that whenever someone makes a change, the other user’s changes get overwritten. Even if you’re submitting a new copy of the text on every keystroke, it’s almost impossible to actually collaborate. The product manager looks at you. You look at your shoes. And sigh. This is not going to be as easy as you thought. Or is it?

If you’re looking to add a collaborative text editor to your product, read on.

This post will walk through:

• Setting up a collaborative text editor that syncs incremental changes.
• Saving the document alongside your existing product data.
• Leveraging the Tiptap editor for markdown-style rich text editing akin to Notion or Dropbox Paper.
• Authorizing the reads and writes to the document on the server.
• Utilizing periodic snapshots to efficiently support features like full text search.

Check out a live demo of it in action here.

Adding a collaborative editor, step by step

We’ll walk through adding the editor, using example code snippets from the project linked above.

Step 0: Use Convex

Convex is a backend platform used to store and sync the data. If you haven’t heard of it before, check out the tutorial. You can add Convex to your existing project, or start fresh with npm create convex@latest.

1npm i convex@latest
2npx convex dev --tail-logs
3

After logging in and setting up your project, this will create a convex/ folder in your project where you write your backend code and define your schema. I like adding --tail-logs so any server logs show up in the terminal. This deploys to a per-developer instance.

You also need to add a Convex React provider, based on your framework and auth setup. Check out the Quickstarts for your flavor. If you’re using Clerk for auth, there’s more specific instructions here. If you’re using Next.js, look here for setup.

Step 1: Configure the sync backend

We will use a Convex Component to keep the document in sync. This will keep the underlying data model of a ProseMirror document, hence the naming. Fun fact: ProseMirror is at the core of both Tiptap and BlockNote.

1npm i @convex-dev/prosemirror-sync
2

Create the file convex/convex.config.ts with the contents:

1// convex/convex.config.ts
2import { defineApp } from "convex/server";
3import prosemirrorSync from "@convex-dev/prosemirror-sync/convex.config";
4const app = defineApp();
5app.use(prosemirrorSync);
6export default app;
7

Watch the terminal where npx convex dev is running and wait for it to succeed. Once it’s done, create the file convex/prosemirror.ts with the contents:

1// convex/prosemirror.ts
2import { ProsemirrorSync } from "@convex-dev/prosemirror-sync";
3import { components } from "./_generated/api";
4
5const prosemirrorSync = new ProsemirrorSync(components.prosemirrorSync);
6export const {
7  getSnapshot,
8  submitSnapshot,
9  latestVersion,
10  getSteps,
11  submitSteps,
12} = prosemirrorSync.syncApi();
13

This exposes the functions getSnapshot, submitSnapshot, … on the server. We’ll use them from our client, which we’ll add next.

Step 2: add the editor to your UI

1npm i @tiptap/react @tiptap/starter-kit
2

Your editor component could be as simple as:

1import { useTiptapSync } from "@convex-dev/prosemirror-sync/tiptap";
2import { EditorContent, EditorProvider } from "@tiptap/react";
3import StarterKit from "@tiptap/starter-kit";
4import { api } from "../convex/_generated/api";
5
6function MyEditor() {
7  const sync = useTipapSync(api.prosemirror, "some-id");
8  return sync.isLoading ? (
9    <p>Loading...</p>
10  ) : sync.initialContent !== null ? (
11    <EditorProvider
12      content={sync.initialContent}
13      extensions={[StarterKit, sync.extension]}
14    >
15      <EditorContent editor={null} />
16    </EditorProvider>
17  ) : (
18    <button onClick={() => sync.create({ type: "doc", content: [] })}>
19      Create document
20    </button>
21  );
22}
23

Note: if you chose a different name than convex/prosemirror.ts use api.path.to.module instead of api.prosemirror.

Now you should be able to make edits in multiple tabs and see them sync. However, the ID is currently hard-coded to "some-id". In the next step we’ll look at using other IDs and explore creating documents server-side, vs. the client-side approach here.

Style points

If you use TailwindCSS, you can install the @tailwindcss/typography plugin and then style the editor with:

1<EditorProvider
2  ...
3	editorProps={{
4    attributes: {
5      class:
6        "prose prose-sm dark:prose-invert sm:prose-base m-5 focus:outline-none",
7    },
8  }}
9

Step 3: create a document

You can create the document client-side or server-side. Client-side has the advantage of allowing for creation while offline or on a spotty internet connection. Server-side has the advantage of transactionally creating all related data, and initializing the document consistently.

IDs

Each document synced is associated with an ID. Generally this can be an ID of a document in one of your existing tables. For instance, if this text field is the description of a task in the tasks table, it could be of type Id<"tasks">. However, for client-side ID generation you won’t have a server-generated document ID, so this can be any string, such as a UUID.

You create the document with JSON that matches your ProseMirror schema, based on your Tiptap extensions. I suggest starting with a predictable, bare document. This reduces the risk that two clients will try to create different initial versions of the same document.

Above we showed how to create a document client-side. To do it server-side, you can export the prosemirrorSync variable, optionally with a custom Id type:

1// in convex/prosemirror.ts
2export const prosemirrorSync = new ProsemirrorSync<Id<"tasks">>(
3  components.prosemirrorSync,
4);
5//...
6
7// in another convex/ file, in a mutation:
8export const createTask = mutation({
9  args: {...},
10  handler: async (ctx, args) {
11    //...
12    const taskId = await ctx.db.insert("tasks", {...});
13    // Creates the document with the ID set to the task's ID.
14    await prosemirrorSync.create(ctx, taskId, { type: "doc", content: [] });
15    //...
16  },
17});
18

Now, once you create a task and load the task in your UI, you can have an editor component like this:

1// in your frontend code, like src/TaskEditor.tsx
2function TaskEditor(props: { id: Id<"tasks"> }) {
3  const sync = useTiptapSync(api.prosemirror, props.id);
4  if (sync.initialContent === null) throw new Error("Document for task not found");
5  //...
6

If they’re both created on the server, the document should always exist when the task does.

Step 4: authorize document reads and writes

It’s important to check who can read and who can write various documents. To do so, we can add some callbacks to the syncApi definition in convex/prosemirror.ts:

1export const {
2  getSnapshot,
3  submitSnapshot,
4  latestVersion,
5  getSteps,
6  submitSteps,
7} = prosemirrorSync.syncApi({
8  checkRead(ctx, id) {
9    // const user = await userFromAuth(ctx);
10    // ...validate that the user can read this document
11  },
12  checkWrite(ctx, id) {
13    // const user = await userFromAuth(ctx);
14    // ...validate that the user can write to this document
15  },
16});
17

You can access the currently logged in user via ctx.getUserIdentity() or a helper like getAuthUserId if you’re using Convex Auth. Read more about working with authentication here.

If you set the Id type parameter above, id here will be your desired type. For the example of a task description could do checks like:

• Check that the user is logged in and on a team associated with the task, or an admin.
• Check that the associated task is in an editable state.
• Check if the task allows being seen by logged-out users.

Step 5: reflect the contents back on snapshots

While editing is happening, changes are being sent immediately. For efficiency, the sync protocol just sends the small changes, and not the full document each time. When there’s a break in editing, a snapshot of the current data is saved. This allows new users or browser tabs to avoid downloading every incremental change. These snapshots are also a good opportunity to do any data denormalizing or logic related to the contents of the document. For instance, we can save a text version of the document into a field with a text search index on it to enable full text search, or generate embeddings to enable vector search of the content.

There is another callback we can use:

1export const { .. } = prosemirrorSync.syncApi({
2  checkRead(ctx, id) { ... },
3  checkWrite(ctx, id) { ... },
4  onSnapshot(ctx, id, snapshot, version) {
5    // ...do something with the snapshot, like store a copy in another table,
6    // save a text version of the document for text search, or generate
7    // embeddings for vector search.
8  },
9});
10

Broadcasting updates to other apps using fetch

If you want to update some third party with the contents every time a snapshot is generated, you can do so with a little code, using the action retrier to retry failures with backoff:

1// in convex/prosemirror.ts
2const retrier = new ActionRetrier(components.actionRetrier);
3
4export const { .. } = prosemirrorSync.syncApi({
5  async onSnapshot(ctx, id, snapshot, version) {
6    await retrier.run(ctx, internal.prosemirror.sendWebhook, {
7      id, snapshot, version
8    });
9  },
10});
11
12export const sendWebhook = internalAction({
13  args: { id: v.id("tasks"), content: v.string(), version: v.number() },
14  handler: async (ctx, args) => {
15    const response = await fetch(process.env.WEBHOOK_URL, {
16      method: "POST",
17      headers: {
18        "Content-Type": "application/json",
19        "Authorization": "Bearer " + process.env.WEBHOOK_AUTH_TOKEN,
20      },
21      body: JSON.stringify(args),
22    });
23    if (!response.ok) {
24      throw new Error(`Webhook failed with status ${response.status}`);
25    }
26  },
27});
28

Bonus: go nuts with extensions

TipTap has a wealth of extensions and a great Discord where you can chat with other developers building interactive editors.

Here’s a concise install command, using brace expansion for brevity:

1npm i @tiptap/extension-{character-count,code-block-lowlight,color,focus,font-family,highlight,horizontal-rule,placeholder,subscript,superscript,table{,-cell,-header,-row},task-{item,list},text-{align,style},typography,underline}
2

I recommend adding all the formatting extensions to a file that you can import on the client or server, like:

1// src/extensions.ts
2import { StarterKit } from "@tiptap/starter-kit";
3import { Typography } from "@tiptap/extension-typography";
4import { Underline } from "@tiptap/extension-underline";
5...
6export const extensions = [
7  StarterKit.configure({
8    codeBlock: false,
9  }),
10  Typography,
11  Underline,
12  ...
13];
14

This allows you to parse out your Schema server-side, in case you want to load or manipulate the document outside of the browser editor.

Tiptap allows you to create your own extensions, or pay for pro extensions. For example, you might want to support file uploads via a user dragging a photo onto the editor. The FileHandler extension gives you just that, with callbacks for how to convert the file into content.

How does it work?

Tiptap is based on ProseMirror, which represents the content in a structured JSON-compatible data model and tracks the data state, transformations, and user interface views. You can learn more about ProseMirror here. Tiptap provides convenient abstractions on top of it and an ecosystem of extensions to provide common editor features. The sync functionality is currently exposed as a Tiptap extension, but could alternatively be packaged as a ProseMirror plugin, if you want to build on ProseMirror directly. Reach out to me if this is interesting to you.

The @convex-dev/prosemirror-sync package handles:

1. Detecting remote changes and applying them to the local ProseMirror document. Each change is stored in a serial order, to allow other clients to efficiently know when they’re behind and fetch exactly what they’re missing.
2. Detecting local changes and submitting them on the server. On the server, if the changes weren’t based on the up-to-date server version, the client then rebases its changes off of the latest version and attempts to re-submit. The sync is single-flighted and each attempt will submit all local changes at once. As a result, as the number of collaborators grow, the longer the period between submissions will increase, and each stored delta will including more steps. One of the benefits of this approach is that the document count will scale sub-linearly with the number of simultaneous collaborators.
3. Submitting snapshots of the document: on creation and periodically when the document has been idle and the current user made the last change. These snapshots allow new clients to get the latest state cheaply, and for the server to have a efficient access to the fully-resolved content for applications that can be slightly stale, like full text search.

Strong guarantees are a joy to build with

It’s worth mentioning all of the complexity that I didn’t need to handle, as a result of implementing this on Convex (instead of traditional API routes & Postgres, say).

1. Subscriptions: Each Convex WebSocket client can have a simple subscription on the latest version of the steps, which is merely a query for the latest step. I didn’t have to add polling, or manage queues or broadcasting all changes.
2. Consistency: When you run a mutation from the client and have an existing query subscription for state related to what the mutation is changing the client will receive the query results before the mutation promise resolves. This makes it easy to reason about keeping the data consistent between code making changes and code getting updated on derived state.
3. Network-flexible sync: When you try to send a mutation while offline, Convex will wait to resolve the promise until the network reconnects. It will retry mutations on transient network errors, and internally implements idempotency to ensure the mutation will be run at most once. Similarly, queries will re-subscribe on reconnect. So my state machine could be blissfully unaware of the network state.
4. Serializable transactions. By default, all Convex mutations are transactions with the highest level of isolation. This podcast does a good job explaining in more detail, but the result is that I can write simple code like fetching the latest version and inserting the next document with that version plus one, and that’s it. Convex will automatically detect conflicts and retry mutations.

What about working offline?

You can continue to edit when offline and changes will sync when the network resumes. However, the current version doesn’t yet have local persistence, so if you refresh your tab or the React component with the sync hook is otherwise unmounted, you will lose those changes.

The current plan for this component will include persistence that:

• Enables refreshing or unmounting the component when there are unsaved changes, and having those sync when next looking at the doc while online, provided you’re on the same browser tab or ReactNative app.
• Enables doing client-side navigation offline and seeing the offline-edited version of the document (again, showing edits from that tab only).
• If you want to support closing the tab, or syncing changes between tabs while offline, look for an upcoming component I’ll be exploring, based on Yjs CRDTs instead of the core ProseMirror model. There may also be a solution leveraging Convex’s local-first object sync engine currently in development.

Where to go from here

I will be exploring more components in this space, including supporting offline cross-tab syncing and more collaborative features, for instance showing the names and text cursors of other users in the document, similar to Google Docs or Notion.

Some extensions you can explore for better UX:

• Store the snapshot version in denormalized data and fetch the latest version from the component, so you can show in the UI if your snapshot is stale, wherever you may be showing the snapshot instead of the full editor.
• Combine the possibly-stale snapshot and deltas server-side to get an up-to-date version, using the ProseMirror schema derived from your extensions.
• Set up a cron to vacuum old document deltas. Note: if an old client comes back online and can’t fetch the steps between where it last synced and the latest version, it will be unable to rebase its changes. So only vacuum steps that you have high confidence are no longer relevant, for instance more than a couple weeks old.
• Gracefully handle sync errors on the client, using the onSyncError callback. For instance, if a user has a document loaded but is no longer allowed to make changes

This current component is still in beta, but is ready for you to try out. You can start by checking out the component and giving feedback as GitHub issues or in Discord.