On 3/1/23 OpenAI released an API for Chat-GPT. In this post, we’ll walk through putting together a full-stack chat app, and add some features. 

We’ll use React and Vite, though any React framework works. We’ll use <QuickConvex /> as the backend, where our server-side functions will run, and where we’ll store our app’s data. 

To see this in action, the code is [here](https://github.com/ianmacartney/convex-chat-gpt), and a running version is [here](https://convex-chat-gpt-f6rq.vercel.app/) (for now).

To run this yourself, you’ll need to make an [OpenAI account](https://platform.openai.com/signup) and [get an API key](https://platform.openai.com/account/api-keys).

If you’re familiar with Convex, you can skip ahead to step 2.

# 0. Bootstrap a Vite React app

If you don’t already have an app, we can make one:

```bash
npm create vite@latest
```

I picked `convex-chatgpt` as the project name, `React` as the framework, and `Javascript` as the variant.

At this point, if we run:

```bash
npm install
npm run dev
```

we have a locally running webapp.

Let’s change `src/App.jsx` to list messages and have a form to submit messages:

<details>
<summary>App.jsx</summary>
    
    ```tsx
    import {useState} from "react";
    import "./App.css";
    
    function App() {
      const messages = [
        {author: "user", body: "Hello, world"},
      ];
      const sendMessage = body =>
        console.log("Trying to send: " + body);
      const [newMessageText, setNewMessageText] =
        useState("");
    
      return (
        <div className="App">
          {messages.map((message, i) => (
            <p key={i}>
              <span>{message.author}: </span>
              <span style={{ whiteSpace: "pre-wrap" }}>
                {message.body ?? "..."}
              </span>
            </p>
          ))}
          <form onSubmit={(e) => {
            e.preventDefault();
            setNewMessageText("");
            sendMessage(newMessageText);
          }}>
            <input
              value={newMessageText}
              onChange={e => setNewMessageText(e.target.value)}
              placeholder="Write a message…"
            />
            <input type="submit" value="Send" disabled={!newMessageText} />
          </form>
        </div>
      );
    }
    
    export default App;
    ```
</details>    

At this point, we have an app that shows a static list of messages and logs to the console when trying to send a message.

# 1. Add Convex

This is similar to the Convex [quickstart](https://docs.convex.dev/quickstart). In a new terminal (leave the other one running `npm run dev`):

```bash
npm install convex
npx convex init
```

If you haven’t used convex before, you’ll be prompted to log in, create an account, etc. I picked `convex-chatgpt` as the project name.

Let’s add the ability to send messages and list messages. We will add functions that run in Convex (on a server) that will read and write to the database with a [query](https://docs.convex.dev/understanding/convex-fundamentals/functions#query-functions) and [mutation](https://docs.convex.dev/understanding/convex-fundamentals/functions#mutation-functions).

Add `convex/messages.ts`:

```jsx
import { query, mutation } from "./_generated/server";

export const list = query(async ({ db }) => {
  return await db.query("messages").take(100);
});

export const send = mutation(async ({ db }, body) => {
  await db.insert("messages", {
    body,
    author: "user",
  });
  const botMessageId = await db.insert("messages", {
    author: "assistant",
  });
});
```

In a separate terminal from the one running `npm run dev`, run `npx convex dev`. This will deploy these functions to your new Convex backend, and as you edit the functions, it will automatically re-deploy. This will ask you to save the deployment URL into your `.env` and `.env.local` files: these point at your convex backends for your production and development deployments. We’ll just be working with the development deployment.

To access Convex from inside React, you’ll need to add a `<ConvexProvider>` context at the top level of your app. Edit `main.jsx`:

```jsx
...
import { ConvexProvider, ConvexReactClient } from "convex/react";

const convex = new ConvexReactClient(import.meta.env.VITE_CONVEX_URL);

ReactDOM.createRoot(document.getElementById("root")).render(
  <React.StrictMode>
    <ConvexProvider client={convex}>
      <App />
    </ConvexProvider>
  </React.StrictMode>,
);
```

We can then use these functions from `App.jsx`:

```jsx
function App() {
  const messages = useQuery("messages:list") || [];
  const sendMessage = useMutation("messages:send");
	...
```

Great! Now we have messages being written to and from the database. Try it out! Those new to Convex will be surprised to see that adding new messages will automatically result in the `useQuery("messages:list")` hook returning new messages. This is part of the magic of Convex. Learn more about it [here](https://docs.convex.dev/understanding/convex-fundamentals/functions#reactivity). 

You can check out your data in the dashboard: `npx convex dashboard`. 

You may note that the query and mutation are named by the `filename:function`. See more about this [here](https://docs.convex.dev/using/writing-convex-functions#defining-convex-functions).

You’ll notice every time we send a message, there’s a “…” message from the assistant. Next let’s update that message from chatGPT. 

# 2. Send a message to the ChatGPT API

So far we’ve been working with a [query](https://docs.convex.dev/understanding/convex-fundamentals/functions#query-functions) and [mutation](https://docs.convex.dev/understanding/convex-fundamentals/functions#mutation-functions), which are Convex functions that interact with the database. In order to interact with external services, we need to do that in an “[action](https://docs.convex.dev/understanding/convex-fundamentals/functions#action-functions)” - which is a Convex function that isn’t inside a deterministic environment and isn’t part of a database transaction. This frees us to do things with side effects, such as making requests to OpenAI’s API. We’ll do this in a few steps.

## Fetching messages to send to ChatGPT

In our `send` mutation, we can grab the latest 10 messages to send to ChatGPT using a database query:

```jsx
export const send = mutation(async ({ db }, body) => {
  //... insert messages to the table
  const messages = await db
    .query("messages")
    .order("desc")
    .filter((q) => q.neq(q.field("body"), null))
    .take(21);
  messages.reverse();
  return { messages, botMessageId };
});
```

This orders messages in descending order (sorted by creation time unless you use a different [index](https://docs.convex.dev/using/indexes)), filters out messages with no body (for instance, the new placeholder system message), and takes the first 21 (which are the 21 most recent messages). It then reverses the list so it’s ordered in ascending time order. I picked 21 so that we’ll have 10 pairs of user/bot messages, followed by the latest prompt. It returns the messages to be used as input to the chat completion API, which we'll add next.

## Creating the action

We’ll use the `openai` npm package. You can install it with:

```bash
npm install openai
```

Make a new file: `convex/actions/openai.js`:

```jsx
import { Configuration, OpenAIApi } from "openai";
import { action } from "../_generated/server";

export const chat = action(async ({ runMutation }, body) => {
  const { messages, botMessageId } = await runMutation("messages:send", body);
  const fail = async (reason) => throw new Error(reason);
  // Grab the API key from environment variables
  // Specify this in your dashboard: `npx convex dashboard`
  const apiKey = process.env.OPENAI_API_KEY;
  if (!apiKey) {
    await fail("Add your OPENAI_API_KEY as an env variable");
  }
  const configuration = new Configuration({ apiKey });
  const openai = new OpenAIApi(configuration);

  const openaiResponse = await openai.createChatCompletion({
    model: "gpt-3.5-turbo",
    messages: [
      {
        role: "system",
        content: instructions,
      },
      ...messages.map(({ body, author }) => ({
        role: author,
        content: body,
      })),
    ],
  });
  if (openaiResponse.status !== 200) {
    await fail("OpenAI error: " + openaiResponse.statusText);
  }
  const body = openaiResponse.data.choices[0].message.content;
	console.log("Response: " + body);
});
```

This will first send the messages with the send mutation we modified, getting in return the list of messages and message ID to update with the bot’s message. It will then make a request to the `gpt-3.5-turbo` model, passing in one system message with instructions (hard-coded for now), followed by each message. We’re turning our body & author fields into “role” and “content”. See their docs [here](https://platform.openai.com/docs/guides/chat) for more details on the API.

## Add your API key to the dashboard

To authenticate requests to OpenAI, you’ll need to add your [API key](https://beta.openai.com/account/api-keys) to the Convex dashboard, so your action (which runs on Convex’s servers) has access to it. Go to your dashboard with `npx convex dashboard` and on the left panel go into Settings and add the key with the name `OPENAI_API_KEY`. While you’re there, you can add it as an env variable to both your production and development deployments by toggling the dropdown in the left panel.

![Dashboard side panel](https://cdn.sanity.io/images/ts10onj4/production/8aa4d6a158df734f3d64e1eb15333528e37c92e5-263x392.png?w=300)


## Triggering the action from the UI

We can change our call to `useQuery("messages:send")` to

```jsx
const sendMessage = useAction("actions/openai:chat");
```

Note the action is addressed by the path to the file (`actions/openai`) and the function (`chat`). The API is the same (taking one argument, `body`), so it’s a drop-in replacement!

At this point, if you run it, it will send a request to ChatGPT and console.log the response. Convex prints Convex function logs to the browser’s console, though you can also see them in the [dashboard](https://dashboard.convex.dev/), to prove to yourself it’s running in the cloud. Now let’s show them in the UI.

## Updating the message with the reply

In order to get the response into the chat message, we’ll want to update the empty placeholder message we added. Treat each call to a mutation as a transaction. Once we called our first mutation, the messages were committed to the database and the UI could show the new messages before doing the slow call to OpenAI. Once we get the response, we update the bot’s message and the UI will update automatically via the `messages:list` query. We’ll add a new mutation in the `convex/messages.js` file:

```jsx
export const update = mutation(async ({ db }, messageId, patch) => {
  await db.patch(messageId, patch);
});
```

This patches the specified message. You could pass in `{body: "hi"}` to change the message’s body to “hi,” for example. Since the first mutation returned the `botMessageId`, we can use that from the action in `convex/actions/openai.js`:

```jsx
export const chat = action(async ({ runMutation }, body) => {
  const { messages, botMessageId } = await runMutation("messages:send", body);

  // Call OpenAI

  await runMutation("messages:update", botMessageId, {
    body: openaiResponse.data.choices[0].message.content,
    // Track how many tokens we're using for various messages
    usage: openaiResponse.data.usage,
    updatedAt: Date.now(),
    // How long it took OpenAI
    ms: Number(openaiResponse.headers["openai-processing-ms"]),
  });
});
```

While we’re here, we can store a bunch of interesting data into the message along with the body. Convex’s database is flexible enough to store [strings, numbers, javascript objects, `Map`s, etc.](https://docs.convex.dev/using/types) while also letting you nail down a [schema](https://docs.convex.dev/using/schemas) later to get typescript types, autocomplete, etc. 

🎉 Now you have chatGPT replying to your messages!

# Summary

In this post, we made a full-stack web app to chat with OpenAI’s ChatGPT API. Let us know [in Discord](https://convex.dev/community) what you think, and if you’d like to see some extensions:

- [Customizing the identity you’re talking to.](https://stack.convex.dev/adding-personality-to-chatgpt-3)
- [Moderating the identities and messages to prevent harmful content.](https://stack.convex.dev/moderating-chatgpt-content-full-stack)
- Create new message threads. (See the [code](https://github.com/ianmacartney/convex-chat-gpt) for how it's done!)

Let's build a full-stack chat app to talk to ChatGPT on its new API!

Building a Full-Stack ChatGPT app

I’ve been using the Dashboard a lot while developing my own apps, so I asked Arnold, the engineer behind the Dashboard, to put together some tips for using it effectively. I hope you find it as useful as I do!

So you’ve got your <QuickConvex /> app set up, eh? Maybe you’ve worked through the [tutorial](https://docs.convex.dev/tutorial/welcome-to-convex) or tried out one of our [demo apps](https://github.com/get-convex/convex-demos). Perhaps you’ve even built your own app already!

Now that you’re familiar with Convex, I’d like to show you some of the tools you can use in the dashboard to level up your development productivity.

## 1. Iterate on and test new functions

To iterate quickly, it’s nice to be able to isolate your code and test backend functionality on its own, without needing to integrate into the frontend. Good news — you can do this straight from the Convex dashboard’s [Functions page](https://docs.convex.dev/using/dashboard#testing-functions). To get started, click the “Run function” button found near your function’s implementation. From there, you can subscribe to live query results and run one-off actions and mutations. If you’re running `npx convex dev` in your code repo, any edits to the functions will sync automatically to your dev instance, so you can quickly edit and re-run your functions as you iterate.

<QuickVid src="https://static.convex.dev/stack/5-dashboard-tricks/run-functions.webm" caption="An example of testing a “list messages” and “send message” Convex function using the dashboard" />

## 2. Seed and prototype data

Before you’ve gotten your functions fully wired up, you can use the dashboard to create tables and documents.

Convex provides two types of editors for editing documents — the default editor provides a simpler view of your data, broken down by field. Clicking the “Edit as JavaScript” button allows you to edit data using the same syntax you would in JavaScript code.

<QuickVid src="https://static.convex.dev/stack/5-dashboard-tricks/prototyping.webm" caption="An example of adding a `channels` table to my chat app project, and associating an existing message with a new channel." />

## 3. Search and destroy — data filters

Got a heap of data to sift through in your tables? Dashboard filters have got you covered. Find this handy button on the [Data page](https://docs.convex.dev/using/dashboard#filtering-documents), and use it to narrow down the results shown to just the documents you’re looking for.

<QuickVid src="https://static.convex.dev/stack/5-dashboard-tricks/filters.webm" caption="An example of filtering a list of chat messages by channel ID" />

## 4. Upload files

Did you know that in addition to documents, Convex allows you to store and retrieve files, such as images and videos? Well now ya do — you can [upload files programmatically](https://docs.convex.dev/using/file-storage) in your Convex functions, or upload individual files using the [File storage tab on the Data page](https://docs.convex.dev/using/dashboard#file-storage).

<QuickVid src="https://static.convex.dev/stack/5-dashboard-tricks/upload-file.webm" caption="Uploading a file to convex" />

## 5. Edit your app’s configuration live

Since Convex queries are [fully reactive](https://docs.convex.dev/understanding/convex-fundamentals/functions#reactivity), updating your data in the dashboard will inform all clients subscribed to the related data that their queries should be re-run. See [this post](https://stack.convex.dev/feature-gating) on using Convex to implement a basic kill switch.

Did you know you can use the Convex Dashboard to run functions, enter seed data, filter & delete data, upload files, and more?

5 Dashboard Tricks

Trying to build an app on top of OpenAI’s Dall-E or ChatGPT? Avoid violating the OpenAI [Usage policies](https://platform.openai.com/docs/usage-policies/platform-policy) or showing inappropriate data to your users by leveraging their [Moderation API](https://platform.openai.com/docs/guides/moderation/overview). In this post, we’ll look at how to use it to flag messages before sending them to Chat-GPT. This will build on the [Building a full-stack ChatGPT app](https://stack.convex.dev/full-stack-chatgpt-app) post, but the approach is generally applicable.

New to Convex? [Convex](https://convex.dev/) is a backend application platform - we make I,t easy to build apps where you need to run code in the server and store data in a database. We run your functions, host your data, and much more. It’s a great fit for working with powerful APIs like OpenAI, as it allows you to build a whole application around powerful AI models without running any servers yourself.

Without further ado, let’s look at how to use OpenAI’s moderation endpoints. 

### **Initializing the API:**

The core piece of code is in the `convex/actions/openai.js` file. This takes less than a millisecond, so we can run it on each request.

```jsx
const apiKey = process.env.OPENAI_API_KEY;
if (!apiKey) {
  await fail(
    "Add your OPENAI_API_KEY as an env variable in the " +
      "[dashboard](https://dasboard.convex.dev)"
  );
}
const configuration = new Configuration({ apiKey });
const openai = new OpenAIApi(configuration);
```

### **Using the moderation API:**

```jsx
// Check if the message is offensive.
const modResponse = await openai.createModeration({
  input: body,
});
const modResult = modResponse.data.results[0];
```

### **Updating the user’s message if the message was flagged:**

```jsx
if (modResult.flagged) {
  await runMutation("messages:update", userMsgId, {
    error:
      "Your message was flagged: " +
      Object.entries(modResult.categories)
        .filter(([, flagged]) => flagged)
        .map(([category]) => category)
        .join(", "),
  });
  return;
}
```

As a reminder, this will call a mutation in `convex/messages.js` that will patch the message, adding an `error` field:

```jsx
export const update = mutation(async ({ db }, messageId, patch) => {
  await db.patch(messageId, patch);
});
```

### Why not check beforehand?

It’s a good point that we could do moderation before inserting the message into the database in the first place. In my case, I wanted to show results immediately and commit the *intent* to send a message. If I waited for moderation, it would take an extra 500ms+ before adding the message and appearing in the UI. For moderating identity creation (see [this post](https://stack.convex.dev/adding-personality-to-chatgpt-3) on adding identities), I moderate first for simplicity. I return an error when the user tries to create a lousy identity. If it felt too long, I could add a spinner while creating the identity. One nice thing about Convex mutations and actions is that you can `await` the results in the UI to know when they succeed/fail, even if you don’t care about their return value. For instance:

```jsx
setLoading(true);
setError(null);
const errorMsg = await addIdentity(
  newIdentityName,
  newIdentityInstructions
);
if (errorMsg) setError(errorMsg);
setLoading(false);
```

### ************************************************************************Refreshing the client’s copy of messages with the new field:************************************************************************

This happens automatically! One of the cool things about Convex is that queries are reactive by default. Here’s how it works:

1. We originally fetched a list of 100 recent messages in the UI with a query called `list` in `convex/messages.js`, which runs in Convex’s cloud. This result is cached automatically, thanks to guarantees provided by Convex’s [deterministic runtime](https://docs.convex.dev/understanding/convex-fundamentals/functions#determinism). 
2. When we sent a new message, we inserted a message into the “messages” table. 
3. Because the `list` query was for the most recent 100 messages, the results were automatically invalidated and re-executed. The new results get pushed to clients over a WebSocket,  triggering a React render for the component(s) that called `useQuery('messages:list')` with the new messages.
4. The same thing happens when we update a message to add an `error` field. As if by magic, the message with the `error` field shows up in the UI and we can render a different UI.

### **Rendering flagged messages in the UI:**

To avoid showing flagged messages, we can do some standard React gating:

```jsx
<span style={{ whiteSpace: "pre-wrap" }}>
  {message.error ? "⚠️ " + message.error : message.body ?? "..."}
</span>
```

Here we show an Error with ⚠️ pre-pended if there is one, otherwise a body if there is one, otherwise “…” for messages we haven’t updated with a bot’s response.

### Filtering flagged messages out of future bot input

We avoided sending the message to ChatGPT when we initially detected bad input. Still, when we send our next set of messages to the API, we don’t want to include the bad input in historical messages either. To do that, we can adjust our filter in `messages:send`:

```jsx
const messages = await db
      .query("messages")
      .filter((q) => q.eq(q.field("error"), null))
      .filter((q) => q.neq(q.field("body"), null))
      .take(21); // 10 pairs of prompt/response and our most recent message.
```

We avoid getting any messages with an “error” field and avoid all the messages without a body, for instance, bot messages that weren’t updated (such as when we bailed due to flagged input).

# Summary

In this post, we looked at how to moderate what content you send to ChatGPT, from the UI to the API calls. I hope it was helpful. Let us know [in Discord](https://convex.dev/community) what you think!

In this post, we’ll look at how to use the moderation API to flag messages before sending them to Chat-GPT, and patterns for handling these errors in a full-stack React app.

Moderating ChatGPT Content: Full-Stack

Building with OpenAI's APIs is fun. By leveraging [Convex](https://convex.dev) to store your data and run server-side functions, you can have a GPT-powered app in no time.

This is a follow-up to [Building a full-stack ChatGPT app](https://stack.convex.dev/full-stack-chatgpt-app), where we set up a Chat-GPT-like chat app using Convex as the backend. In that example, we used a fixed identity string. In this post we’ll look at how to store multiple personalities in the database and provide them in the chat, to enable changing personalities mid-conversation.

See the code [here](https://github.com/ianmacartney/convex-chat-gpt) and a working demo [here](https://convex-chat-gpt-f6rq.vercel.app/).

<BuildWithConvex />

To customize the identity you pass into ChatGPT, you can pass a “system” message to the OpenAI API. See more details on this API [here](https://platform.openai.com/docs/guides/chat). In our case, before having multiple identities, the code looks like this:

```js
const openaiResponse = await openai.createChatCompletion({
  model: "gpt-3.5-turbo",
  messages: [
    {
      role: "system",
      content: instructions,
    },
    ...messages.map(({ body, author }) => ({
      role: author,
      content: body,
    })),
  ],
});
```

Let’s store multiple identities in the Convex database and pick which one to use. 

### Adding and listing identities

Let’s make `convex/identity.js`:

```js
import { query, mutation } from "./_generated/server";

export const list = query(async ({ db }) => {
  const identities = await db.query("identities").collect();
  return identities.map((identity) => identity.name);
});

export const add = mutation(async ({ db }, name, instructions) => {
  const existing = await db
    .query("identities")
    .filter((q) => q.eq(q.field("name"), name))
    .unique();
  if (existing) {
    await db.patch(existing._id, { instructions });
  } else {
    await db.insert("identities", { name, instructions });
  }
});
```

This will run on the server and list the identity names or allow you to add one. Before it adds one, it will check if it already exists and, if so, update it, so there aren’t duplicate names. Because of Convex’s transaction isolation (which is the strictest kind, called “serializable isolation”), we’ll never have duplicate identities with the same name, simply because we queried for that name first. Read more about that [here](https://docs.convex.dev/understanding/deep-dive/occ).

Note that by only returning the identity names, we are hiding the instructions from the clients. This is a big difference between Convex and a platform like Firebase that gives you direct access to databases from the browser. By running code on the server, we can use code to limit what information is returned to clients.

### UI for adding identities

Adding a UI to add an identity is similar to sending messages with an `<AddIdentity />` component:

<details>
<summary>`AddIdentity` component </summary>
```jsx
function AddIdentity() {
  const addIdentity = useMutation("identity:add");
  const [newIdentityName, setNewIdentityName] = useState("");
  const [newIdentityInstructions, setNewIdentityInstructions] = useState("");

  return (
    <form
      onSubmit={async (e) => {
        e.preventDefault();
        await addIdentity(newIdentityName, newIdentityInstructions);
        setNewIdentityName("");
        setNewIdentityInstructions("");
      }}
    >
      <input
        value={newIdentityName}
        onChange={(event) => setNewIdentityName(event.target.value)}
        placeholder="Identity Name"
      />
      <textarea
        value={newIdentityInstructions}
        onChange={(event) => setNewIdentityInstructions(event.target.value)}
        placeholder="GPT3 Instructions"
        rows={2}
        cols={40}
      />
      <input
        type="submit"
        value="Add Identity"
        disabled={!newIdentityName || !newIdentityInstructions}
      />
    </form>
  );
}
```
</details>

Let’s add some identities. Some I like: 

**Rubber Duck**

> You are curious and respond with helpful one-sentence questions.
> 

**Supportive Friend**

> You are a supportive and curious best friend who validates feelings and experiences and will give advice only when asked for it. You give short responses and ask questions to learn more.
> 

**CS Coach**

> You are a highly technically trained coach with expertise in technology and best practices for developing software. Respond with concise, precise messages and ask clarifying questions when things are unclear.
> 

### Using an identity from the UI

To use an identity, we can add a `select` in our form to pick an identity, and pass that identity name in the `messages:send` mutation:

```jsx
function App() {
  const messages = useQuery("messages:list") || [];
  const sendMessage = useMutation("messages:send");
  const [newMessageText, setNewMessageText] = useState("");

  const identities = useQuery("identity:list") || [];
  const [identityName, setIdentityName] = useState("");
  //...
      <form
        onSubmit={(e) => {
          e.preventDefault();
          setNewMessageText("");
          sendMessage(newMessageText, identityName);
        }}
      >
        <select
          value={identityName}
          onChange={(e) => setIdentityName(e.target.value)}
        >
          <option value={""} disabled={identityName}>
            Select an identity
          </option>
          {identities.map((name) => (
            <option key={name} value={name}>
              {name}
            </option>
          ))}
        </select>
    //...
```

### Looking up the identity

In our `messages:send` function, which runs on the server, we can accept the new `identityName` and look up the identity’s instructions to pass to ChatGPT:

```js
export const send = mutation(async ({ db }, body, identityName) => {
  //...
  const identity = await db
    .query("identities")
    .filter((q) => q.eq(q.field("name"), identityName))
    .unique();
  if (!identity) throw new Error("Unknown identity: " + identityName);
  const botMessageId = await db.insert("messages", {
    author: "assistant",
    identityId: identity._id,
  });
  //...
  return { messages, botMessageId, instructions: identity.instructions };
```

Two things to note:

1. We search for the identity by name. If there were a lot of them, we’d use an [index](https://docs.convex.dev/using/indexes), but for small tables doing a filter is just fine.
2. We’re storing the `identityId` into the bot’s message, so we can later know which identity responded to each message.

We can also pass the instructions for each bot message, by looking up the instructions if there is an `identityId`:

```js
await Promise.all(
    messages.map(async (msg) => {
      if (msg.identityId) {
        msg.instructions = (await db.get(msg.identityId)).instructions;
      }
    })
  );
```

We look up each identity by doing `db.get` in a separate async function, and use `Promise.all` to wait for the result of each of the async functions.

<details>
<summary>All together, `send` now looks like:</summary>

```js
export const send = mutation(async ({ db }, body, identityName) => {
  await db.insert("messages", {
    body,
    author: "user",
  });
  const identity = await db
    .query("identities")
    .filter((q) => q.eq(q.field("name"), identityName))
    .unique();
  if (!identity) throw new Error("Unknown identity: " + identityName);
  const botMessageId = await db.insert("messages", {
    author: "assistant",
    identityId: identity._id,
  });
  const messages = await db
    .query("messages")
    .order("desc")
    .filter((q) => q.neq(q.field("body"), null))
    .take(10);
  messages.reverse();
  await Promise.all(
    messages.map(async (msg) => {
      if (msg.identityId) {
        msg.instructions = (await db.get(msg.identityId)).instructions;
      }
    })
  );
  return { messages, botMessageId, instructions: identity.instructions };
});
```
</details>

### Passing the identity to ChatGPT

In our `convex/actions/openai.js` `chat` function, we can now de-structure the `instructions` return instead of hard-coding it:

```js
export const chat = action(async ({ runMutation }, body, identityName) => {
  const { messages, botMessageId, instructions } = 
    await runMutation("messages:send", body, identityName);
  //...
```

Now the response will be a function of what messages have been sent, and what the identity’s instructions are.

To also give GPT context on what its identity was for previous messages, we can sprinkle system messages into the messages we pass to the API:

```js
const gptMessages = [];
let lastInstructions = null;
for (const { body, author, instructions } of messages) {
  if (instructions && instructions !== lastInstructions) {
    gptMessages.push({
      role: "system",
      content: instructions,
    });
    lastInstructions = instructions;
  }
  gptMessages.push({ role: author, content: body });
}
if (instructions !== lastInstructions) {
  gptMessages.push({
    role: "system",
    content: instructions,
  });
  lastInstructions = instructions;
}
const openaiResponse = await openai.createChatCompletion({
  model: "gpt-3.5-turbo",
  messages: gptMessages,
});
```

The goal here is to remind GPT what identity it had for each message it sent. This API is new, so I bet there’s a lot of tweaking we can do with the prompts here to make it work better. For instance, maybe we should be telling it “Ignore previous system instructions for the following messages. Now act like this: (new instructions)”. If you get something working in a better way, let me know [in Discord](https://convex.dev/community)!

### Showing the identity’s name instead of “assistant” in the UI

Currently, all the messages in the UI say “assistant” - let’s show the identity we named by looking up the identities dynamically. Convex supports [relationships](https://docs.convex.dev/using/document-ids#references-and-relationships) in this way.

When listing our messages, we can look up the identity’s name to return. In `convex/messages.ts`:

```js
export const list = query(async ({ db }) => {
  const messages = await db.query("messages").take(20);
  return Promise.all(
    messages.map(async (message) => {
      if (message.identityId) {
        const identity = await db.get(message.identityId);
        message.identityName = identity.name;
      }
      return message;
    })
  );
});
```

We look up each identity by doing `db.get` in a separate async function, and use `Promise.all` to return the result of each of the functions as an array. This is the common pattern for doing join-like behavior with Convex. This way you can execute the reads in parallel but do all your business logic and lookups in JavaScript.

On the client, we can now show the `identityName` if it’s defined instead of just the author in `App.jsx`:

```jsx
<span>{message.identityName ?? message.author}: </span>
```

And we can also default the chat identity to be the last-used identity message, when you load the page with a useEffect:

```jsx
useEffect(() => {
  if (identities.length && !identityName) {
    const lastMessage = messages[messages.length - 1];
    if (lastMessage && identities.indexOf(lastMessage.identityName) !== -1) {
      setIdentityName(lastMessage.identityName);
    } else {
      setIdentityName(identities[0]);
    }
  }
}, [messages, identities, identityName]);
```

### Summary

We’ve now added the ability to change the instructions to ChatGPT on how to respond, so we can have a conversation and be changing the identity mid-stream. See the code [here](https://github.com/ianmacartney/convex-chat-gpt) and a working demo [here](https://convex-chat-gpt-f6rq.vercel.app/). Note, that code & demo has some more features than just what we’ve done in this post.

How to store multiple personalities Convex and provide them to the chatGPT API, enabling changing personalities mid-conversation. This is a follow-up to Building a full-stack ChatGPT app. 

Adding Personality to ChatGPT-3

What do you do when the work you want to perform should happen after a request? 

How can you see incremental progress on workflows that may take a while? Or cancel a job scheduled for the future?

In this post, we’ll look at a pattern for managing asynchronous work that allows you to:

1. Start a long-running job and respond quickly to a client.
2. Track incremental progress on multi-stage tasks.
3. Subscribe to the status and result of a background task from multiple clients.
4. Cancel a request transactionally (i.e., without race conditions).
5. Monitor job timeouts.
6. Implement custom retry logic in the case of transient failures.

As an example of this pattern, I’ll be referencing [a multiplayer game using Dall-E](https://stack.convex.dev/building-a-multiplayer-game). While building it, I found that OpenAI’s image endpoint could sometimes take over 30 seconds, timing out and giving a bad user experience. Rather than have the client wait on a single request, I schedule the work to be run asynchronously using Convex’s [function scheduling](https://docs.convex.dev/using/scheduling). You can see the code [here](https://github.com/get-convex/multiplayer-game-with-dall-e).

# Tracking status in a table

The high-level approach uses a table to keep track of a long-running task’s state. For my example, I made a “submissions” table to track generating an image using OpenAI based on a prompt:

```tsx
// in convex/schema.ts
submissions: defineTable({
    prompt: s.string(),
    authorId: s.id("users"),
    result: s.union(
      s.object({
        status: s.literal("generating"),
        details: s.string(),
      }),
      s.object({
        status: s.literal("failed"),
        reason: s.string(),
        elapsedMs: s.number(),
      }),
      s.object({
        status: s.literal("saved"),
        imageStorageId: s.string(),
        elapsedMs: s.number(),
      })
    ),
  }),
```

Depending on the status of the work, we capture different information.

## 1. Starting a job without waiting for it

To start the process, the client calls a [mutation](https://docs.convex.dev/understanding/convex-fundamentals/functions#mutation-functions), which creates the submission document, schedules the work to start immediately, and returns the ID:

```tsx
// in convex/submissions.tx "start" mutation
const submissionId = await db.insert("submissions", {
  prompt,
  authorId: session.userId,
  result: {
    status: "generating",
    details: "Starting...",
  },
});
// Kick off actions/createImage.ts in the background
// so we don't block this request.
scheduler.runAfter(0, "actions/createImage", prompt, submissionId);
return submissionId;
```

Mutations are transactional, so we could have also checked if there was an ongoing submission for the user or a duplicate request. Importantly, we pass the submission ID to the client and the action that will update the submission.

## 2. Tracking incremental progress

Once the client receives the submission ID, it can update its UI reactively based on the submission status:

```tsx
const Submission = ({ submissionId }) => {
  const result = useSessionQuery("submissions:get", props.submissionId);
  switch (result?.status) {
    case "generating":
      return (
        <figure>
          <article aria-busy="true"></article>
          {result.details}
        </figure>
      );
    case "failed":
      return <p>{result.reason}</p>;
    case "saved":
      return (
        <figure>
          <img src={result.url} />
          Generated in {result.elapsedMs / 1000} seconds.
        </figure>
      );
  }
  return null;
};
```

As a reminder, Convex [queries are re-run automatically](https://docs.convex.dev/understanding/convex-fundamentals/functions#query-functions) whenever the underlying data changes. So as the submission is altered, this React component will re-render with the latest results.

On the server, it can update the status from the action by running the `submissions:update` mutation:

```tsx
// in actions/createImage.ts
runMutation("submissions:update", submissionId, {
  status: "generating",
  details: "Generating image...",
});
```

Which can be as simple as:

```tsx
// in convex/submissions.tx
export const update = mutation(async ({ db }, submissionId, result) => {
  await db.patch(submissionId, { result });
});
```

When the request is done, it’s up to you whether you write to the job table with the results or commit them elsewhere. In my case, I let the user decide whether they like the image before submitting it to the game, so the action is only responsible for generating it.

## 3. Subscribing from multiple clients

One nice side-effect of storing the data in a table and reactively querying it is that you can subscribe to the result from multiple clients. Anyone with the submissionId can wait for results. On a higher level, you can also see real-time statistics about the health of the jobs. Because Dall-E can be so slow, I decided to surface its status in the UI, to manage user expectations.

Here I query the latest five submissions and calculate their average time and average success rate:

```tsx
// in submissions.ts
export const health = query(async ({ db }) => {
  const latestSubmissions = await db
    .query("submissions")
    .order("desc")
    .filter((q) => q.neq(q.field("result.status"), "generating"))
    .take(5);
  let totalTime = 0;
  let successes = 0;
  for (const submission of latestSubmissions) {
    totalTime += submission.result.elapsedMs;
    if (submission.result.status === "saved") successes += 1;
  }
  const n = latestSubmissions.length;
  return [totalTime / n, successes / n];
});
```

## 4. Cancelling a request safely

If you decide to cancel a request, you can check if it’s already started and update its status to “canceled” in the table otherwise. When the job runs, it can query the table and either return early or mark the task as “started.” Because mutations are transactional with serializable isolation by default, you are guaranteed that either the mutation to cancel the job will see that it’s “started” already or the job will see “canceled” - you’ll never think that you canceled a task but find out it ran anyways. 

For example, say you want to make a last-minute change to which email a user will get. It is important to send only one email. You can cancel the current pending email, and if you succeeded in canceling it, send an updated one instead.

## 5. Monitoring timeouts

To mark a job as timed out, you can schedule a follow-up mutation when you’re scheduling the job.

```tsx
// in submissions.ts "start" mutation:
scheduler.runAfter(30, "submissions:timeout", submissionId);
```

The timeout could do something like:

```tsx
export const timeout = mutation(async ({ db }, submissionId) => {
  const submission = await db.get(submissionId);
  if (submission.result.status === "generating") {
    db.patch(submissionId, {
      result: { status: "failed", reason: "Timed out", elapsedMs: 30000 },
    });
  }
});
```

Depending on your application, you might want your background job to not save results if it ends up finishing anyways, so it could check that the status isn’t already “failed" and commit in the same mutation - similar to canceling a request above.

## 6. Implementing retries

Convex functions give you [different guarantees around failure and retries](https://docs.convex.dev/understanding/convex-fundamentals/functions). To summarize, Convex automatically retries your queries and mutations, but cannot automatically retry actions since they may contain side effects that may not be safe.

Sometimes, however, it does make sense to retry an action. Such as fetching resources from an external service that has transient failures. To do this, we can have our actions catch errors and retry themselves:

```tsx
// in actions/createImage
if (!response.ok && retryCount < 3) {
	// call this same function again
	scheduler.runAfter(
    calculateBackoff(retryCount),
    "actions/createImage", prompt, submissionId, retryCount + 1);
}
```

For your use case, `calculateBackoff` will likely implement some [backoff and jitter](https://aws.amazon.com/builders-library/timeouts-retries-and-backoff-with-jitter/) to ensure you don’t exacerbate issues for the service you’re hitting.

# Summary

In this post, we looked at some patterns for using a table to track scheduled functions to achieve a number of common behaviors for background tasks. As always, let us know what you think in [our Discord](https://convex.dev/community)!

Implement asynchronous job patterns using a table to track progress. Fire-and-forget, cancelation, timeouts, and more.

Background Job Management

When building a new app, it can be challenging to build much behavior without the notion of a user. However, getting users to sign up for a new service before seeing any benefits is challenging. Signing up is cumbersome, and many people don’t want to share their personal information with a company until they want a long-standing relationship. In this post, we’ll discuss how to model ephemeral users, leveraging the session “middleware” introduced in [this post](https://stack.convex.dev/sessions-wrappers-as-middleware). We’ll examine two approaches and discuss the challenges with anonymous auth.

# Lightweight: using session IDs as user IDs

To illustrate the session middleware, I made a [demo app](https://github.com/get-convex/convex-demos/tree/main/sessions) in our [convex-demos repo](https://github.com/get-convex/convex-demos).

The app is a clone of our [tutorial demo](https://github.com/get-convex/convex-demos/tree/main/tutorial), which is a basic chat app. The tutorial generates a random user name on the client when the page loads and sends that string whenever it sends a message. This has these downsides, which we’ll aim to remedy:

- Refreshing the page changes your user name.
- An updated name isn’t reflected in past messages.

We can keep your name constant using sessions by storing it in the session data. Instead of initializing the random user name in the client, we initialize it when [creating the initial session data](https://github.com/get-convex/convex-demos/blob/main/sessions/convex/sessions.js#L67):

```jsx
// In convex/sessions.ts, run on server:
export const create = mutation(async ({ db }) => {
  return db.insert("sessions", {
    name: "User " + Math.floor(Math.random() * 10000),
  });
});
```

**********Persisting user name:**********

This way, when the user reloads the page, it will read the existing session ID from the browser (in localStorage or sessionStorage, whichever you configured) and use the existing session name.

**Using anonymous users relationally:**

In the [users-and-auth demo](https://github.com/get-convex/convex-demos/tree/main/users-and-auth), we solve updating names by associating each message with a user ID instead of the user name string. When listing the messages, it would look up the user name on the fly, so the messages always reflected the latest name. The [session demo](https://github.com/get-convex/convex-demos/tree/main/sessions)'s approach is to store the session ID instead of the user ID and look up the name from session IDs. This means updates to the user name will be reflected in old messages:

![chat app in the sessions demo](https://cdn.sanity.io/images/ts10onj4/production/9e4b93acdb6951017419169dbea1e1df373cdd75-960x510.gif?w=450)

## Using the session data

To use the session ID, we store the session ID in the chat message:

```jsx
export default mutationWithSession(async ({ db, session }, body) => {
  const message = { body, sessionId: session._id };
  await db.insert("messages", message);
});
```

And when we return the messages, we look up the sessions on the fly. Note we save one lookup by using the user’s session if they sent the message:

```jsx
export default queryWithSession(async ({ db, session }) => {
  const messages = await db.query("messages").collect();
  return Promise.all(
    messages.map(async message => {
      const { sessionId, ...messageBody } = message;
      const author =
        session && session._id.equals(message.sessionId)
          ? session.name
          : (await db.get(sessionId)).name;
      return { author, ...messageBody };
    })
  );
});
```

You may notice we do all the lookups in parallel, which is wicked fast.

## When to not use this approach

If you want a very lightweight solution, it doesn’t get much simpler than this. However, with its simplicity comes a limitation of what it can represent.

**If your app will have logged-in users who will interact with anonymous users**, it is awkward to have to look in two different places for users and store two different kinds of IDs depending on which type they are.

**************************************************************If you want to keep session IDs private**************************************************************, you should avoid passing around session IDs as user identifiers. For instance, you may want to keep it confidential if you store anything in the session related to a user’s permissions or auth tokens. While the sessions demo only passes down the messages with the session ID replaced with the latest name, it would be too easy to accidentally leak session IDs if they are your app’s user identifier.

To remedy these situations, consider a layer of indirection, where sessions store user IDs pointing to possibly-ephemeral users and are passed around freely.

# Storing anonymous users in the “users” table

While building a [multiplayer Dall-E-based game](https://stack.convex.dev/building-a-multiplayer-game), one goal is to be able to play with the game without having to log in first. However, we also want users to be able to log in eventually, use their profile picture, access old games, or log back into games on a different device or browser tab.

Instead of passing session IDs around for the game, we create a user when the session is created and use that user ID. This way, we can pass around user IDs (which, in general, shouldn’t be treated as secrets) and still have a persistent shared identifier between the client and the server. 

```jsx
// Run on the server on initializing the session
export const create = mutation(async ({ db, auth }) => {
  const userId = await createAnonymousUser(db);
  return db.insert("sessions", { userId });
});
```

We will save the more complex topic of managing the hybrid of logged-in/out users in a future post, but as a teaser, here is how you could avoid creating an anonymous user if the user is logged in:

```jsx
export const create = mutation(async ({ db, auth }) => {
  const identity = await auth.getUserIdentity();
  let userId = identity && (await getOrCreateUser(db, identity));
  if (!userId) {
    userId = await createAnonymousUser(db);
  }
  return db.insert("sessions", {
    userId,
  });
});
```

# Summary

In this post, we looked at a couple of strategies for managing user information without requiring a login. Follow along with the multiplayer game using OpenAI [here](https://stack.convex.dev/building-a-multiplayer-game).

Getting users to sign up for a new service before seeing any benefits is challenging. In this post, we looked at a couple of strategies for managing user information without requiring a login.

Anonymous Users via Sessions

This is part of a series of posts about [building a multiplayer game on Convex](https://stack.convex.dev/building-a-multiplayer-game). In this post, we’ll explore using Dall-E via the OpenAI API. The code is [here](https://github.com/get-convex/convex-demos/tree/main/dall-e-storage-action) if you want to play around with it. We will make a chat app where you can type in an image description as a slash command, and it will generate and insert that image into the chat. On the server, it will:

1. Configure an OpenAI client.
2. Check that the prompt is not offensive using their moderation API.
3. Get the short-lived image URL from Dall-E.
4. Download the image.
5. Upload the image into Convex [File Storage](https://docs.convex.dev/using/file-storage), getting a storage ID.
6. Insert the storage ID into a new message.
7. When serving the message to the client, we will generate a URL based on the storage ID, which they can put in an `<img>` tag directly.

This is now possible using [File Storage](https://docs.convex.dev/using/file-storage) and a Convex [Action](https://docs.convex.dev/using/actions), a serverless function that can have side-effects[^1], comparable to AWS lambda. Read below for how to use OpenAI with Convex, or check out the code [here](https://github.com/get-convex/convex-demos/tree/main/dall-e-storage-action).

# Using OpenAI in Convex

## Set up Convex

Create a Convex app by cloning [the demo](https://github.com/get-convex/convex-demos/tree/main/dall-e-storage-action) and running `npm i; npm run dev`. If you want a fresh project, see the [Tutorial](https://docs.convex.dev/tutorial/welcome-to-convex).

If you’re writing your app from scratch, create an `actions` directory in the `convex/` folder, and add a file. Mine is [`convex/actions/sendDallE.js`](https://github.com/get-convex/convex-demos/blob/main/dall-e-storage-action/convex/actions/sendDallE.js). 

## Configuring OpenAI

1. To use OpenAI, you need an API account. Sign up [here](https://beta.openai.com/signup).
2. Generate an API key [here](https://platform.openai.com/account/api-keys) by clicking “Create new secret key” and copying the text.
3. Store the API key Convex’s [Environment Variables](https://docs.convex.dev/using/environment-variables), where you can store secrets. Go to your project’s dashboard by running `npx convex dashboard` in your project directory or finding it on [dashboard.convex.dev](https://dashboard.convex.dev). Open the project settings on the left and enter it with the key `OPENAI_API_KEY`. I’d recommend saving it in both your Production and Development deployments unless you want the published version of your app to use a different key. There is a toggle to switch between them.
4. Configure the client in your [action](https://docs.convex.dev/using/actions). If you get type errors, `npm install openai`:
    
    ```jsx
    import { Configuration, OpenAIApi } from "openai";
    const configuration = new Configuration({
      apiKey: process.env.OPENAI_API_KEY,
    });
    const openai = new OpenAIApi(configuration);
    ```
    

## Using the Moderation API

The moderation API is useful for checking whether a prompt is obscene or will be rejected by OpenAI. For our purposes, we’ll check if it flags the message:

```jsx
const modResponse = await openai.createModeration({
  input: prompt,
});
const modResult = modResponse.data.results[0];
if (modResult.flagged) {
  throw new Error(
    `Your prompt was flagged: ${JSON.stringify(modResult.categories)}`
  );
}
```

The `categories` object can be used for more fine-grained moderation. It has a score per category. Consult the [moderation documentation](https://platform.openai.com/docs/guides/moderation/overview) for more details.

## Generating an image with Dall-E

You can consult the [image generation documentation](https://platform.openai.com/docs/guides/images) for more details, but for our purposes, the code is:

```jsx
const opanaiResponse = await openai.createImage({
  prompt,
  n: 1,
  size: "256x256",
});
const dallEImageUrl = opanaiResponse.data.data[0]["url"];
```

Now we have a URL to the image. However, this URL [only lasts for an hour](https://platform.openai.com/docs/guides/images/usage?lang=node.js#:~:text=URLs%20will%20expire%20after%20an%20hour). If we want to be able to see this image in the chat longer term, we need to store it.

## Downloading the image

To download the image, we use the [`node-fetch` NPM library](https://www.npmjs.com/package/node-fetch#common-usage). 

```jsx
const imageResponse = await fetch(dallEImageUrl);
const image = Buffer.from(await imageResponse.arrayBuffer());
```

Now we have the image in memory and can store it in Convex.

## Storing the image in Convex

Storing the image in Convex is a two-step process:

1. Generate an upload URL.
2. Upload to that URL, getting a storage ID in return.

### Generating an upload URL

[Upload URLs](https://docs.convex.dev/using/file-storage#uploading-files) are created in a [Convex mutation](https://docs.convex.dev/understanding/convex-fundamentals/functions#mutation-functions). This allows your application to authorize uploads, for instance, if you wanted only to allow certain users to upload. My mutation is very simple:

```jsx
// in convex/sendMessage.js
export const generateUploadUrl = mutation(async ({ storage }) => {
  return await storage.generateUploadUrl();
});
```

One thing to note is that this can’t live in the `convex/actions` directory. Actions are run in a different environment than queries and mutations. You can read more about that [here](https://docs.convex.dev/understanding/convex-fundamentals/functions#action-functions). I put mine in [`convex/sendMessage.js`](https://github.com/get-convex/convex-demos/blob/main/dall-e-storage-action/convex/sendMessage.js).

To use the `generateUploadUrl` from our action:

```jsx
// in sendDallE.js
const postUrl = await runMutation("sendMessage:generateUploadUrl");
```

### Upload to Convex File Storage

It might be counter-intuitive to upload from within a Convex function since that function is already “in” Convex. However, by providing this upload URL, you can have more control over how you upload. For instance, instead of downloading the whole image at once, you could have streamed the data in and streamed it to Convex file storage.

```jsx
const postImageResponse = await fetch(postUrl, {
  method: "POST",
  headers: { "Content-Type": imageResponse.headers.get("content-type") },
  body: image,
});
// Get the storageId for the upload.
const { storageId } = await postImageResponse.json();
```

## Store the storage ID in the database

The storage ID returned from the upload URL is now a permanent identifier of this data. You can store it in your database however you’d like. In our example, we store it in a new chat message.

```jsx
// in sendDallE.js action
await runMutation("sendMessage", storageId, author, "dall-e");
```

And the mutation that stores it, for completeness:

```jsx
// in sendMessage.js
export default mutation(async ({ db }, body, author, format) => {
  // A bit of a hack; we are storing the storage ID in the "body"
  await db.insert("messages", { body, author, format });
});
```

## Serving the image to the client

In Convex, clients subscribe to [queries](https://docs.convex.dev/using/database-queries). For instance, in our frontend React component, we can do:
```jsx
const messages = useQuery("listMessages") || [];
...
{messages.map(message => (
  <Message author={message.author} body={message.body} format={message.format} key={message._id.toString()} />
)};
```
That query maps to a serverless function: the default export of `convex/listMessages.js`. To return the image URL in messages we return to clients, we generate a URL from the storage ID. For our app, we will change each storage ID into a URL. In `convex/listMessages.js`:

```jsx
export default query(async ({ db, storage }) => {
  const messages = await db.query("messages").collect();
  for (const message of messages) {
    if (message.format === "dall-e") {
		  // Replace the storage ID with a URL in the "body"
      message.body = await storage.getUrl(message.body);
    }
  }
  return messages;
});
```
This URL won’t change on each request, so you don’t have to worry about caching the URL on the client. The HTTP response also has cache headers so you don't need to worry about client-side image caching either.

# Performance

One thing I discovered while building this is how slow the OpenAI API calls can be. In my testing, the moderation call took 500ms, and the image generation could take 5+ seconds, sometimes taking over 60s and timing out the request. To make this a good user experience, adding a loading indicator is important. On the client, I used this code:

```tsx
// In React jsx:
const [sending, setSending] = useState(false);
const sendDallE = useAction("actions/sendDallE");
...
setSending(true);
try {
  await sendDallE(prompt, name);
} finally {
  setSending(false);
}
```

# Summary

In this post, we used Convex to fetch an image from OpenAI’s image generation service based on a user-provided prompt. Read about [the game](https://whose-prompt-is-it-anyways.vercel.app/) I've developed on Convex using Dall-E APIs [here](https://stack.convex.dev/building-a-multiplayer-game) for more tips, or [read the code](https://github.com/get-convex/multiplayer-game-with-dall-e) to see how I added error handling and more.


[^1]: This differs from other Convex functions like `query` and `mutation`, which do not have side effects, allowing them to provide features like automatic retries, reactive updates, serializable transaction isolation, and more! Read more about it [here](https://docs.convex.dev/understanding/deep-dive/occ).

Use Convex to fetch an image from OpenAI’s image generation service based on a user-provided prompt.

Using Dall-E from Convex

Building multiplayer games requires a lot of synchronization logic and event systems. However, using Convex, we get a lot of this for free! Follow along here as we build a complex multiplayer game on Convex, leveraging its reactive-by-default queries, transactional mutations, backend storage, and scheduled functions. 

So what’s the game?

We will be building a game where you generate an image by submitting a prompt. We use [Dall-E 2](https://openai.com/dall-e-2/), a service by OpenAI, to turn the prompt into an image. Then, without showing your description, your friends submit alternate prompts that seem plausible. Once everyone has submitted an alternate caption, everyone votes on which prompt they think was the real one and you get points if other people choose your prompt. It’s a ton of fun. Check out the prototype [here](https://whose-prompt-is-it-anyways.vercel.app/) and the code [here](https://github.com/get-convex/multiplayer-game-with-dall-e).

My steps for building this game:

1. Sketch a UX prototype on paper. 
2. Validate any technical challenges with targeted demos. 
3. Identify the actions and state in each phase of the game.
4. Model the server data in the schema.
5. Build the serverless functions, adjusting the data model iteratively.
6. Build a rough UI, wired up to the serverless functions. 
7. Polish the UI and UX iteratively.

# Prototyping

The beauty of low fidelity prototyping is in the speed of iteration. When I use tools that are designed to allow a high level of polish and control, I can lose a lot of time in trying to make it look professional. By committing to throwing away the prototype, it frees you up to cut a lot of corners to answer the more existential questions. Is this idea any good? What is the storyline or user journey? I started with paper and pen, annotating ideas and behaviors, and playing through the experience in my mind. A trick I learned from my friend Bernardo, the most creative designer I know, is the power of using Keynote for rapid prototyping. I know it sounds crazy, but I gave it a shot for this project and am  glad I did. Having something concrete to discuss with engineers and designers grounds the conversation, and having it in low fidelity keeps the discussion away from UI bikeshedding. This is the slideshow I put together:

![Keynote Prototype](https://cdn.sanity.io/images/ts10onj4/production/0f31222584bbfa81d5d26d143ee2013c17938202-720x405.gif)

# Validating technical challenges

## Using the Dall-E API to generate images

The first unknown is working with the OpenAI API to generate an image from a prompt. We tackle this early since there won’t be a game if we can’t get this working. To read more about using Dall-E with Convex, check out the post [here](https://stack.convex.dev/using-dall-e-from-convex). Instead of needing to build the whole game to test this, I just copied the [Convex tutorial](https://github.com/get-convex/convex-demos/tree/main/tutorial) so I could spend most of my time on the critical code.

## Using sessions for anonymous users

Following up on the post on [using sessions in Convex](https://stack.convex.dev/sessions-wrappers-as-middleware), we are enabling users to play this game without logging in by creating ephemeral users when creating a session. This is important, as asking someone to log into a service before they’ve gotten any benefit is a challenge to adoption. Read [this post](https://stack.convex.dev/anonymous-users-via-sessions) to learn more about different approaches to representing anonymous users. In a future post, we’ll tackle the tricky business of transitioning anonymous users to authenticated users when they log in, and keeping data consistent in the process.

![](https://cdn.sanity.io/images/ts10onj4/production/9e4b93acdb6951017419169dbea1e1df373cdd75-960x510.gif)

# Outlining actions and state

As part of building the prototype, I was able to think through the various stages of the game, and by having a rough draft of the UI, I could plan out what the interaction points are and what information needs to be surfaced. This differs from the data in the database, as it is optimized for presentation and hiding information that would allow clients to cheat the game. For instance, when a user is submitting an image with a prompt, they should be able to see who else has finished submitting, but not receive their prompts or images. In hindsight I wish I had gone even deeper here, defining the typescript types for each stage, as a target when implementing the queries and mutations later.

# Modeling data with schema.ts

Once we know what information the client needs, it’s easier to decide a convenient and consistent format to store the data at rest. One decision I made was to have a single game with references to a “round” for each generated image. I planned to re-use UI components to build a public version of the game on the homepage, and decided that was the right level of abstraction to represent both. Convex allows you to store both documents, and relationships between documents. 

When working in typescript, it is very convenient to have auto-complete. I modeled these in `convex/schema.ts` instead of raw typescript types, since that would give me types from all database queries, and warn me if I was inserting incompatible data. By having end to end typing, iterating on the data schema is as easy as modifying `schema.ts` and letting typescript tell me where I needed to update things, both in the server code and in the frontend React code!

# Serverless functions

For my functions, I leveraged [zod validation](https://stack.convex.dev/wrappers-as-middleware-zod-validation) when I received data from the client, and segmented my functions between a few files separated by function. Submissions, rounds, games, users, etc. all get their own namespace for functions. To model the game data, I decided to have a single function that returned different state based on what stage the game was in. Because convex queries are reactive, as the game state updates the new state is automatically pushed to clients.

## Keeping a scheduled jobs table

While building the [demo to generate Dall-E images](https://github.com/get-convex/convex-demos/tree/main/dall-e-storage-action), it became clear that the OpenAI API is slow. Sometimes the requests would take over thirty seconds and time out. In our [demo](https://github.com/get-convex/convex-demos/tree/main/dall-e-storage-action) we waited on the request from the client. In [this post](https://stack.convex.dev/background-job-management) we discuss managing async work by keeping track of a background job’s state in a table. 

![](https://cdn.sanity.io/images/ts10onj4/production/5b6431ba4ee6fb2c13b631dba78eba15c83f2414-1200x852.png?w=600)

## Sharing types between the client and server

While we get types on the client and server by default with Convex, the return type of a function can be very complex, and if you start passing it around, it’s convenient to have it defined as a typescript type you can reference in both places. It’s not a good idea to import server code into the client, as it can expose server logic, so I put types that I want to import from the server and client in to `convex/shared.ts`.

## Composing zod types with unions

For retrieving the round data, the data is pretty different depending on the phase of the game. By using a shared typescript type, I could use a type as props for a page on the frontend and also compose it as a return type on the server. Furthermore, by defining it in zod, I could validate the server’s response like:

```jsx
export const getRound = queryWithSession(
  withZodArgs(
    [zId("rounds")],
    async ({ db, session, storage }, roundId) => {
      ...
    },
    z.union([LabelStateZ, GuessStateZ, RevealStateZ])
  )
);
```

To get the typescript type for the frontend, I have this in `shared.ts` :

```jsx
export type ClientGameState = z.infer<typeof ClientGameStateZ>;
```

# Building a rough UI

In the spirit of iterating quickly, I decided to write zero css to start. I used pico.css so it would look usable for playing with friends and coworkers, and just focused on getting each page wired up to the state coming from the server. If there had been parallel frontend and backend development, I would have probably made stub versions of the serverless functions that just returned static data for each stage, so this work could have happened in parallel.

**Play the prototype [here](https://whose-prompt-is-it-anyways.vercel.app/)!**

Stay tuned for more updates as we polish the game!

Building multiplayer games requires a lot of synchronization logic and event systems. However, using Convex, we get a lot of this for free! Follow along here as we build a complex multiplayer game on Convex, leveraging its reactive-by-default queries, transactional mutations, backend storage, and scheduled functions.

Building a Multiplayer Game

Session tracking is a common practice for application servers. While most of your data is associated with a user or another document, sometimes you have data specific to a user’s tab, or associated with a user who isn’t logged in. Some of this data is stored on the client, such as in the browser’s `sessionStorage` or `localStorage`, while other data is stored on the server. This post will dive into how to implement session storage with Convex using some helper functions we wrote. The code is in the [convex-helpers GitHub repo](https://github.com/get-convex/convex-helpers).

**User data:**

Typically user data is stored on the server to avoid accidental leakage of personal data on public computers. Because this data can exist without a logged-in user, it can enable representing and capturing data about anonymous users. This is great news for building multiplayer experiences where you don’t want to require logging in. In legacy systems, this might also be where you cached the signed-in user. With [Convex, auth is managed automatically](https://docs.convex.dev/using/auth), your serverless functions execute close to the database, and [queries are cached](https://docs.convex.dev/using/auth), so you don’t have to worry about this optimization.

**Ephemeral state:**

Storing session data also provides a more continuous experience for a logged-in user because you can have per-tab information on where they are in the application. Suppose there is a complex multi-step flow, like booking an appointment. In that case, they can book two different appointments simultaneously without losing their progress if they refresh the page and without storing that sensitive data in the browser’s storage.

Typically this data is stored in a separate database because typically databases are slow, far away from the application server, and because this data is frequently queried. Traditional databases use pessimistic locking, which creates a bottleneck when many transactions reference the same rows. With Convex, we can store it in a first-class table, giving us a transactionally consistent view of all our data. Convex also uses optimistic concurrency control, which alleviates the bottleneck concern. Read more on that [here](https://docs.convex.dev/understanding/deep-dive/occ). 

# How to implement sessions with Convex

Continuing the series of Wrappers as “Middleware,” I built some functions to wrap your serverless functions to provide session data. It stores your session data in a “sessions” table in your Convex backend. Because this also requires keeping track of the session ID in the client, I’ve also written some wrappers for `useQuery` and `useMutation` to make it easy.

## Using sessions:

1. In addition to a `ConvexProvider`, wrap your app with a `SessionProvider`: (defined [here](https://github.com/get-convex/convex-helpers/blob/main/src/hooks/session.ts)):
    
    ```tsx
    <ConvexProvider client={convex}>
      <SessionProvider storageLocation={"sessionStorage"}>
        <App />
      </SessionProvider>
    </ConvexProvider>
    ```
    
2. Use `queryWithSession` or `mutationWithSession` as your function (defined [here](https://github.com/get-convex/convex-helpers/blob/main/convex/lib/withSession.ts)):
    
    ```tsx
    export default mutationWithSession(async ({ db, session }, body) => {
      const message = { body, sessionId: session._id };
      await db.insert("messages", message);
    });
    ```
    
3. Use `useSessionQuery` or `useSessionQuery` in your React client (defined [here](https://github.com/get-convex/convex-helpers/blob/main/src/hooks/session.ts)):
    
    ```tsx
    const sendMessage = useSessionQuery('sendMessage');
    ...
    sendMessage(body);
    ```
    
4. [Optional] Write any data that you want to be available in subsequent session
requests to the `sessions` table :
    
    ```tsx
    db.patch(session._id, { anonymousUserId });
    ```
    
5. [Optional] Update the `sessions:create` function (defined [here](https://github.com/get-convex/convex-helpers/blob/main/convex/lib/withSession.ts)) to
initialize the session data. 

# How it works

Under the hood, what it is doing is quite simple.

1. It creates a new session in the `SessionProvider` context. This creates a new document in a “sessions” table and returns its ID to the frontend, which stores the session ID in `sessionStorage` or, optionally, `localStorage`. Notes on that are below. 
2. It passes that session ID as the first parameter in each `query` or `mutation` where you use `useSessionQuery` or `useSessionMutation`. It passes `null` if the session hasn’t been created yet.
3. The serverless functions using `withSession` define an extra first parameter of a session ID and use it to look up the session, passing it in the function’s first argument (that usually has the `db` and `auth` objects), along with the rest of the parameters. It passes `null` if the session ID wasn’t provided.

## `sessionStorage` vs. `localStorage`

If you want the session to be shared between all tabs in a browser. I like the behavior of `sessionStorage` for general use:

- When you refresh a page, the data persists. The data is tied to a specific tab.
- If you open a new tab, you start fresh.
- If you use the “Reopen Closed Tab” feature on Chrome, the data is still there.

Thoughts on `localStorage`:

- When multiple browser tabs are reading and writing the same data, there are a lot of opportunities for bugs and confusion.
- Some data may make sense to be stored at the browser level, such as answering questions like “have I seen this browser before.”
- Keep public computers in mind.

## Initialization behavior

Before the session ID is created, your serverless functions may receive a `null` `session`. You can decide how to gracefully handle this state. In mutations I generally assert the session’s existence, and in queries I fall back gracefully. Thankfully this only happens for brand new sessions, since the ID is stored client-side and read on initialization.

# Summary

In this post we looked at implementing session storage in Convex, using a custom table and some convenience wrappers which make it easy to use session-specific data in your server-side code. We look forward to seeing what you build with it.

Check out the code [here](https://github.com/get-convex/convex-helpers) and submit a pull request if you think of any improvements. 🙏

How to implement session storage with Convex, using some helper functions we wrote. While most of your data is associated with a user or another document, sometimes you have data that is specific to a user’s browser tab, or associated with a user who isn’t logged in. Some of this data is stored on the client, such as in the browser’s `sessionStorage` or `localStorage`, while other data is stored on the server.

Sessions: Wrappers as "Middleware"

Following up on the previous post on using `withUser` to add authentication context to your [Convex functions](https://docs.convex.dev/using/writing-convex-functions), now let’s look at adding function validation using a popular npm package `zod`. Check out the code in action in the [convex-demos](https://github.com/get-convex/convex-demos/tree/main/zod-validation-ts) repo.

Function validation is important for a production app, because you can’t always control which clients are talking to your server. Consider the following code from [our tutorial](https://github.com/get-convex/convex-demos/blob/main/tutorial/convex/sendMessage.js):

```jsx
// convex/sendMessage.js
export default mutation(async ({ db }, body, author) => {
  const message = { body, author };
  await db.insert("messages", message);
});
```

This code runs in the server, and stores a message in the “messages” table. That’s great, assuming the client sends data in the right format. Most of the time, that will be the case. If you use typescript, we’ll even warn you in your frontend React code when the parameters to your server function are the wrong type. See what this looks like in our [typescript demo](https://github.com/get-convex/convex-demos/blob/main/typescript/convex/sendMessage.ts):

```jsx
// convex/sendMessage.js
export default mutation(async ({ db }, body: string, author: string) => {
  const message = { body, author };
  await db.insert("messages", message);
});
```

However, a friendly internet stranger might connect to your backend and send any number of things: wrong types, binary data, nested objects, etc.. Typescript doesn’t enforce types at runtime, it only helps you with static analysis. What does that mean? While Typescript can help catch developer errors where you’re using types incorrectly in code, once the application is running the Typescript types aren’t being enforced by the runtime. For serverless applications that have unauthenticated endpoints, you need to be especially defensive with your function arguments, since a fake client could connect to your backend and pass whatever arguments it wants. Just declaring the type of `body` to be `string` doesn’t make it so. So what can we do?

# Using `withZod` for input validation

Using the popular `zod` library, we can define the types that we expect for our function. When it gets invoked, the inputs will be validated. To make this convenient, I’ve written a `withZod` wrapper so you can type your function arguments, and not have to worry about validating the first `{ db, ... }` argument, which is provided by the query, mutation, or action. So now your code looks like this:

```tsx
export default mutation(
  withZod(
    [z.string(), z.string()],
    async ({ db }, body, author) => {
      const message = { body, author };
      return await db.insert("messages", message);
    }
  )
);
```

Note: For a typescript version of everything in this post, you can look [here](https://github.com/get-convex/convex-helpers/blob/main/convex/lib/withZod.ts). In there are also various helpers for combining with `query`, `mutation`, and `action`, as well as helpers for if you want to pass in a whole custom zod function rather than just the arguments and return type.

### Aside: the above is already valid typescript!

By leveraging zod to give us validation, it is also giving us the types of our parameters. So we can avoid duplicating that definition, while still getting type hints, both in the server code **and in the client**. Convex has always had canonical end-to-end typing: from your data model definition to the client, it’s all in typescript. Now, the typescript types for your functions are generated from your validator, so your code is safe by default!

# Validating returned objects

You can pass an optional third parameter to `withZod` to validate the output of your server code. While you don’t have to worry about malicious clients sending malformed requests, you might still enjoy the peace of mind to ensure you aren’t leaking more data than you want. You can also define a Zod object that has a subset of fields, and let Zod strip your returned object down, so you don’t share more fields than you intended.

```tsx
export default mutation(
  withZod(
    [z.string(), z.string()],
    async ({ db }, body, author) => {
      const message = { body, author };
      return await db.insert("messages", message);
    },
    zId("messages")
  )
);
```

### zId Helper

You’ll note above we used `zId`. This is a helper, meant to resemble `s.id("messages")` in [your `schema.ts` file](https://docs.convex.dev/api/modules/schema), but for zod. Feel free to make any other helpers you’d like and share them with us [in Discord](https://convex.dev/community). 

```jsx
export const zId = (tableName: TableName) =>
  z.custom((val) => val instanceof Id && val.tableName === tableName);
```

# Implementing withZod

For those curious, or who want to copy and extend this pattern, this is what we are doing under the hood:

```jsx
export const withZod = (zodArgs, func, zodReturn?) => {
  const zodFunctionDef = z.function(
    z.tuple(zodArgs),
    z.promise(zodReturn ?? z.unknown())
  );
  return (ctx, ...outerArgs) => {
    const zodFunction = zodFunctionDef.strictImplement((...args) =>
      func(ctx, ...args)
    );
    return zodFunction(...outerArgs);
  };
};
```

We are using a `z.function` and passing in the untrusted arguments, while just passing through the `ctx` argument. We use `strictImplement` here to define a function that adds the `ctx` argument back in. 

Note: For a typescript version, go [here](https://github.com/get-convex/convex-helpers/blob/main/convex/lib/withZod.ts).

# Combining with other wrappers

As with our previous post, you can combine it with other wrappers or one of the [Convex function](https://docs.convex.dev/using/writing-convex-functions) generators to reduce duplicate code, and reduce the indentation of your function definition:

```jsx
const mutationWithZod = (zFunc, func) => mutation(withZod(zFunc, func));
```

```tsx
export default mutationWithZod(
  [z.string(), z.string()],
  async ({ db }, body, author) => {
    const message = { body, author };
    return await db.insert("messages", message);
  },
  zId("messages")
);
```

See [here](https://github.com/get-convex/convex-helpers/blob/main/convex/lib/withZod.ts) for implementations of this and others in typescript.

# A note on args_0:

Unfortunately, you lose the argument names in autocomplete when you use zod functions; they’re replaced with `args_0`, `args_1`, etc. To work around this, you have a couple of options: you can inline the zod function, or pass args as an object.

## Zod without withZod 🤯:

You can do all this yourself, without our fancy `withZod` wrapper, by putting your application code inside an `implements` or `strictImplements` definition for a [zod function](https://zod.dev/?id=functions). 

```tsx
export default mutation(async ({ db }, body: string, author: string) => {
  return z
    .function()
    .args(z.string(), z.string())
    .returns(z.promise(z.object({ _id: zId("messages") })))
    .strictImplement(async (body, author) => {
      const message = { body, author };
      const id = await db.insert("messages", message);

      return (await db.get(id))!;
    })(body, author);
});
```

With this, your function arguments will still show up as “body” and “author”. One downside here is that (for typescript) you need to repeat the definition of the argument types. Once for the mutation arguments, and again for the zod function arguments.

## Using an object arg for an explicit API:

You can also use our `withZodObjectArg` which assumes your function only takes one argument, which is an object. This allows you to write code like:

```tsx
// convex/sendMessage.js
export default mutationWithZodObjectArg(
  { body: z.string(), author: z.string() },
  async ({ db }, { body, author }) => {
    const message = { body, author };
    return await db.insert("messages", message);
  },
  zId("messages")
);

// web client React code
const sendMessage = useMutation("sendMessage");
...
await sendMessage({body, author});
```

The client passes all arguments in a typed object, so every argument has a name. The type hint will now tell you the names of the parameters. If you enjoy wrapping your arguments in objects anyways, you’re welcome. 

# In summary

In this post, we looked at a way to add type validation to [Convex functions](https://docs.convex.dev/using/writing-convex-functions) by using the [`zod` npm](https://www.npmjs.com/package/zod) package. You can grab the library code from [here](https://github.com/get-convex/convex-helpers/blob/main/convex/lib/withZod.ts), or play around with a demo app [here](https://github.com/get-convex/convex-demos/tree/main/zod-validation-ts)! As always, let us know in [our Discord](https://convex.dev/community) what you think!

Function validation is important for a production app because you can’t always control which clients are talking to your server. See how to use zod to validate your Convex functions, using our withZod wrapper.

Zod Validation: Wrappers as “Middleware”

In this post, I’ll introduce a pattern that, like middleware, can add functionality before or after a request but is explicit and granular, unlike middleware. This is the first of a series of posts on pseudo-middleware patterns to help structure your Convex code. Let us know in Discord what else you want to see! If you want to see a typescript version of the code, you can reference it [here](https://github.com/get-convex/convex-helpers/blob/main/convex/lib/withUser.ts).

# The problem

[Setting up auth](https://docs.convex.dev/using/auth) in Convex is easy. However, the resulting code can end up cluttering your functions if you aren’t careful. Consider our [auth demo’s](https://github.com/get-convex/convex-demos/tree/main/users-and-auth) function to [send a message](https://github.com/get-convex/convex-demos/blob/main/users-and-auth/convex/sendMessage.ts):

```tsx
export default mutation(async ({ db, auth }, body) => {
  const identity = await auth.getUserIdentity();
  if (!identity) {
    throw new Error("Unauthenticated call to mutation");
  }
  // Note: If you don't want to define an index right away, you can use
  // db.query("users")
  //  .filter(q => q.eq(q.field("tokenIdentifier"), identity.tokenIdentifier))
  //  .unique();
	const user = await db
    .query("users")
    .withIndex("by_token", q =>
      q.eq("tokenIdentifier", identity.tokenIdentifier)
    )
    .unique();
  if (!user) {
    throw new Error("Unauthenticated call to mutation");
  }

  const message = { body, user: user._id };
  await db.insert("messages", message);
});
```

All of the endpoint logic is in the last few lines! 

# The goal

We want to provide a `user` where we’d normally access the [`auth`](https://docs.convex.dev/api/interfaces/server.Auth) object. 

```tsx
export default mutation(
  withUser(async ({ db, user }, body) => {
    const message = { body, user: user._id };
    await db.insert("messages", message);
  })
);
```

# The `withUser` solution

Our wrapper function, provided below, may look a little complicated, so let’s talk about what it’s doing. Like `mutation` and `query`, `withUser`'s only argument is a function. However, this function wants to be called with the `user` populated in the first parameter. So you can see in the call `func({ ...ctx, user }, ...args)`, we are passing in the user that we looked up. Popping out a layer, `withUser` itself returns an async function that can be passed to `query` or `mutation`. So we define an inline async function that, given the normal `ctx` and some arguments (`...args` captures all of the arguments), will call the passed-in function `func` with the same arguments and the first parameter augmented. If this bends your brain, you’re not alone. Feel free to copy-paste. And for those nervous about how you’d type this in typescript, don’t worry. You can copy it from [here](https://github.com/get-convex/fast5/blob/main/convex/lib/withUser.ts).

```tsx
/**
 * Wrapper for Convex query or mutation functions that provides a user.
 *
 * @param - func Your function that can now take in a `user` in the ctx.
 * @returns A function to be passed to `query` or `mutation`.
 */
export const withUser = (func) => {
  return async (ctx, ...args) => {
    const identity = await ctx.auth.getUserIdentity();
    if (!identity) {
      throw new Error(
        'Unauthenticated call to a function requiring authentication'
      );
    }
    // Note: If you don't want to define an index right away, you can use
    // db.query("users")
    //  .filter(q => q.eq(q.field("tokenIdentifier"), identity.tokenIdentifier))
    //  .unique();
    const user = await ctx.db
      .query('users')
      .withIndex('by_token', (q) =>
        q.eq('tokenIdentifier', identity.tokenIdentifier)
      )
      .unique();
    if (!user) throw new Error('User not found');
    return await func({ ...ctx, user }, ...args);
  };
};
```

## Why extend the first argument with new parameters?

In python, the language I’ve worked with this pattern the most, the common practice for wrapper functions is to pass new parameters as the new first arguments to a wrapped function. However, for Convex we can leverage the fact that the first argument to functions is always [`ctx`](https://docs.convex.dev/generated-api/server#queryctx), and extending it comes with some great ergonomics. Using the fact that it’s an object, we can:

- Flexibly add middleware-like parameters while maintaining the aesthetic of positional arguments matching the positional arguments on the client.
- Add more wrappers in the future without having to keep track of which order the injected parameters are in, or knowing how many parameters they inject.
- Use a middleware wrapper without using its provided value(s).

# Codifying patterns

Those familiar with factories, decorators, or middleware will recognize this pattern. Rather than requiring every user to repeat the same lines of code at the start or end of a function, you can codify that pattern into a function. Beyond saving some time, leveraging patterns like this helps:

- Keep your code [DRY](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself).
- Increase the density of meaningful code.
- Organize code by logical function. See [aspect-oriented programming](https://en.wikipedia.org/wiki/Aspect-oriented_programming) for an extreme perspective on this.
- Give the code reviewer shortcuts.

I’ll expand on this last point. By noticing that it’s using the `withUser` helper, a reviewer can rest assured that this function will only be executed with a logged-in user. This becomes a more powerful reassurance when you compose these functions, such as `withTeamAdmin`, which we’ll see below.

# Composing functions

The exciting part of using this sort of pattern is that it’s easy to compose it with other functions. In a simple example, we can combine `mutation` and `withUser` like so:

```tsx
export const mutationWithUser = (func) => {
  return mutation(withUser(func));
};
```

This has the ergonomic benefit of decreasing the indentation level of your function, if you use [prettier](https://prettier.io/). However, you can imagine much more interesting compositions, like:

```tsx
export const withTeamAdmin = (func) => {
  return withUser(withTeam(async (ctx, ...args) => {
    const {user, team} = ctx;
    const admin = await getTeamAdmin(user, team);
    if (!admin) throw new Error('User is not an admin for this team.')
    return await func({...ctx, admin}, ...args)
  }));
}

export const setTeamName = mutation(
  withTeamAdmin(async ({ db, team, admin }, name) => {
    console.log(`${admin.name} is changing team name`);
    await db.patch(team._id, {name});
  });
);
```

# Why wrap at all?

You might be thinking to yourself that this is a lot of indirection for what could be a series of regular functions. Indeed, this pattern can introduce complexity for someone looking at the code for the first time. Where did the `user` arg come from? Why is my stack trace full of `withX` calls? Sometimes, it is clearer and less surprising to call regular functions. Here are some **scenarios when wrapping is useful**:

- You want to control what the function returns.
- You want to do work before and after the function, such as opening and closing a file.
- You need to clean up if the called function throws an exception.
- Misusing a function (such as forgetting to await it or handle its response correctly) would have serious implications.
- You want to compose your behavior with the above scenarios consistently.
- You will reuse these wrappers frequently.

# Wrapping up

Using wrapper functions like `withUser` can help you organize your code into middleware-like blocks that you can compose to keep your function logic concise. A typescript version of the code is [here](https://github.com/get-convex/convex-helpers/blob/main/convex/lib/withUser.ts), and used by [Fast5](https://fast5.live/) [here](https://github.com/get-convex/fast5/blob/main/convex/lib/withUser.ts). Let us know in [Discord](https://discord.com/channels/1019350475847499849/1066114385543692338) what you think and what you’d like to see next!

Using wrapper functions like withUser can help you organize your code into middleware-like blocks that you can compose to keep your function logic concise.

Authentication: Wrappers as “Middleware”

In this post, I will share some patterns for incorporating presence into a web app. I will be leveraging some features of Convex which makes it easy to implement, and sharing [some utilities I built along the way](https://github.com/convex-stack/convex-presence/) that you’re welcome to use & extend. Check out the code in action in the [convex-demos](https://github.com/get-convex/convex-demos) repo.

# What is presence and why is it important?

Presence, as we’ll use the term here, is about surfacing activity in a UI about other users - surfacing their virtual presence. Some examples you’ve likely seen are the list of people “online” in Messenger, the “…” bubble in Messages when someone is composing a message to you, someone’s cursor in a Google Doc, etc.

The value is a mix of utility and user experience. In a shared document, knowing where someone is typing can help you avoid typing over each other. The more subtle effects, however, tap into our social instincts. Seeing that other people are looking at the same document, seeing active engagement, and gives a sense of aliveness. I personally feel more connected to collaborators than something like a Wiki. In a world where work is increasingly being done in private, I’ll take all the presence I can get.

# Presence in action

![](https://cdn.sanity.io/images/ts10onj4/production/1e4087361b56b4154b2cca3c3627cfe61a6bc431-874x430.png)

## Presence data

Presence data sits in a middle ground between application state and session state. Application data needs to be carefully updated and stored, and is typically read more than it’s written. Presence data is less critical - it’s ok if you skip a few cursor movements, as long as the UI arrives at the correct end state. However, we still care about durability. Session state is ephemeral and can be held in memory & quickly discarded. Some presence state is like this - where your cursor is, whether you’re typing, etc. However, to know when someone last edited a document, or when a user was last online, you need to store longer-term data.

## Presence performance

Presence data is a great candidate for [single-flighting](https://stack.convex.dev/throttling-requests-by-single-flighting) because we care about latency and we want graceful degradation when many users are online at once. It isn’t critical to get every cursor position, but it should show the final cursor position as quickly as possible. Higher throughput can get a higher frame rate, but if we were to decide between getting more data points with more lag or fewer data points more frequently, we’d choose the latter. We aren’t building a 60fps game, we are just conveying basic information. See [the post on single-flighting](https://stack.convex.dev/throttling-requests-by-single-flighting) to see more about how it enables dynamic back-pressure under load.

## `usePresence`

To make it easy to implement presence features, [I wrote a utility](https://github.com/get-convex/convex-presence/blob/main/hooks/usePresence.ts) that saves presence data in a new `presence` table, segmented by “room” and “user”. A room could be a web page, document, chat room, etc. In my example, a user was identified just by a string randomly generated on the client, but you could use authentication data server-side to ensure a user can only modify their own presence data & read presence data in rooms they’re allowed in.

By default, the utility gives you a `useState`-like API but also includes a list of the state for other users in the same room.

```tsx
const [myPresence, othersPresence, updateMyPresence] = usePresence(
  userId,
  roomId,
  initialData
);
```

The main difference is that `updateMyPresence` accepts partial data updates, so you can update your avatar in one component, and set whether you’re typing in another, and the resulting data will be the latest values of each. This is important because it allows us to skip some updates via single-flighting. We know the next update to be sent will have the latest values.

## Online detection

![](https://cdn.sanity.io/images/ts10onj4/production/e888805e48b052cca9e2d7c281c4135d99e09765-362x102.png)

A common way to detect a user’s presence is to periodically send a “heartbeat” message to the server that the client is still there. By checking when a user last updated their presence, you can tell if they’ve gone offline. The more frequently you send it, the faster you can detect that a user is no longer online, but the more resources your app will consume. 

The [`usePresence`](https://github.com/get-convex/convex-presence/blob/main/hooks/usePresence.ts) React hook defaults to 5 seconds, and the [demo app](https://github.com/get-convex/convex-presence/blob/main/pages/index.tsx) considers a user to not be online after 10 seconds. 

```tsx
const online = othersPresence.filter(
    (presence) => Date.now() - presence.updated < 10000
  );
```

If you’re building your own presence utility, remember you can avoid sending heartbeats when you send other messages, as I do [here](https://github.com/get-convex/convex-presence/blob/main/hooks/usePresence.ts):

```tsx
useEffect(() => {
    void updatePresence(room, user, data);
    const intervalId = setInterval(() => {
      void heartbeat(room, user);
    }, heartbeatPeriod);
    return () => clearInterval(intervalId);
  }, [updatePresence, heartbeat, room, user, data, heartbeatPeriod]);
```

## Facepiles

![](https://cdn.sanity.io/images/ts10onj4/production/f2a56c99f00090b2d4a896fcac4c03281613a5fe-572x220.png)

A “facepile” is a popular term for the visual stack of users in a document - whether they’re profile pictures, initials, avatars, or in the case of my demo app, emojis. See my facepile logic [here](https://github.com/get-convex/convex-presence/blob/main/components/Facepile.tsx) or play around with a demo [here](https://github.com/get-convex/convex-demos/tree/main/presence-facepile). Some things to keep in mind when building them:

- You can use the latest heartbeat to segment users into online & offline groups.
- If you sort by the latest update, your pile will jump around as users send their heartbeats. I chose to sort by online/offline, then by their “created” time - when they first were present in that given room. This way it would be stable and active & newer users would show up on top.
- By default React will only re-render when something changes, so if you want to keep re-computing whether a user is online or offline, you can do something like I do [here](https://github.com/get-convex/convex-presence/blob/main/components/Facepile.tsx) and use a `setInterval` to re-compute the list every second. Note that this does not make new network requests, it just re-computes the UI based on the existing data, so you only consume browser resources, not network bandwidth or server compute time.

## Typing indicator

![](https://cdn.sanity.io/images/ts10onj4/production/a1d3e9fffa30209ab3e6e9b82234965fabbf4bcd-960x459.gif)

To make a typing indicator super snappy, you can update presence data to `{typing: true}` when you start typing and explicitly set it back soon after you stop typing. To achieve this, you can use a debounce function from something like `lodash`, or just use a `useEffect` and `setTimeout` as I do [here](https://github.com/get-convex/convex-presence/blob/main/hooks/useTypingIndicator.ts). However, if a user gets disconnected before they can update their presence, they might be stuck in a `typing: true` state, so make sure to take their latest update time into account and exclude offline users.

```tsx
useEffect(() => {
    if (text.length === 0) {
      updateMyPresence({ typing: false });
      return;
    }
    updateMyPresence({ typing: true });
    const timer = setTimeout(() => updateMyPresence({ typing: false }), 1000);
    return () => clearTimeout(timer);
  }, [updateMyPresence, text]);
```
See a working example  [here](https://github.com/get-convex/convex-demos/tree/main/presence-typing-indicator)

## Cursors

![](https://cdn.sanity.io/images/ts10onj4/production/11b6246507267f73121e998aea876fdb2ecc4982-960x459.gif)

Text cursors are an important part of collaborative text editing, where the important piece to get right is how to index your position, given that a pure character offset may be out of sync with the edits you and others are making. You might even embed cursor locations into the data stream of document edits, or pin cursor locations to edits, so new text is always appearing by your cursor on other users’ documents. This is a complex topic worth a post all on its own, so I’ll leave it at that for now.

For mouse cursors, the challenge is giving the illusion of continuous motion when you’re receiving discrete events. Naive implementations will seem very choppy, with the cursor jumping to the latest location immediately. Intermediate implementations will slide around smoothly, though the cursor will always be a little behind. In [my demo](https://spectacular-beijinho-ccf8ab.netlify.app/), I just use a 200ms transition in CSS, which is simple, but still looks a bit jumpy and lags by an extra 200ms (code [here](https://github.com/get-convex/convex-presence/blob/main/components/SharedCursors.tsx)). Advanced implementations may not only smooth between historical points (using bezier or other smoothing algorithms) but also try to anticipate where the cursor is moving. 

Sharing mouse cursor positions are the point at which I’d recommend using a dedicated in-memory service, rather than trying to persist that data to a database, since the data is especially ephemeral.

# How Convex makes it easy

Convex helped make this much easier through its built-in WebSocket reactivity and caching scalability.

## Reactivity

Convex’s data model is [reactive by default](https://docs.convex.dev/understanding/convex-fundamentals/functions#caching) - when you query data, you are automatically subscribed to changes to that data. Because it owns the data retrieval as well as the data mutation, it can intelligently invalidate caches& recompute queries automatically. In this case, querying for data in a given “room” in the presence table meant that every change to presence data in that room resulted in the new data being computed and sent down to clients. Without Convex you’d be either polling or managing a bespoke Pub-Sub / WebSocket system. 

## Caching

Another nice feature of Convex queries is its [caching](https://docs.convex.dev/understanding/convex-fundamentals/functions#caching), and cache invalidation. Its cache primarily uses the function arguments as the key, so our query for all the presence data in a given room will be recomputed once per room, rather than once per user. This means that as the number of users in a room (& their associated mutations) grows, the number of function invocations grows linearly, instead of quadratically, which is a big deal, even for dozens of users.

# Next Steps

Some things that aren’t implemented in [the demo](https://github.com/get-convex/convex-presence), but would be natural extensions, would be:

- Implementing access control so you can’t read or write presence data in rooms you’re not part of.
- Adding a way to clear presence data for a room. Currently, the library merges patch data.
- Check whether the data has changed before sending an update. Currently, all calls to `updateMyPresence` will attempt to update the server.
- Add a parameter to `usePresence` for whether to do heartbeat, since not all applications need to know that a user is still “online”.

Let us know in [our discord](https://convex.dev/community) what you think, and if you implement any of these! PRs welcome: [GitHub](https://github.com/get-convex/convex-presence). ❤️

Some patterns for incorporating presence into a web app leveraging Convex, and sharing some tips & utilities I built along the way.

Implementing Presence with Convex

Building reactive applications can become an obsession. How much fast can I sync state? How can I keep my app feeling responsive? How can I avoid wasting resources?

Let’s say you want a responsive UI and care about how fast your UI can update. A naive approach would just add up the time to send the request, have it processed on the server, and receive a response. And for actions like clicking buttons, this will be a good approximation. However, it’s important to consider requests within the context of other requests. For something like capturing mouse movements or typing, requests could end up being created faster than they can be processed. If requests pile up in an outgoing queue on the client, or are competing for server resources, they may appear much slower as the user waits for previous requests to be processed. Even if your request itself is fast, if it has to wait for hundreds of prior requests to complete, it can seem slow to the end user. In these cases, it can be useful to limit how many requests the client sends. There are many ways to do this: throttling, debouncing, and server side rate limiting are the most common. Can we do better?

In this article we’ll be looking at an approach called “single flighting” or “singleflighting” and what an implementation looks like in React using Convex for the backend.

# Throttle vs. Debounce vs. Rate limit vs. Singleflight

A quick aside on terminology. 

**Debouncing** refers to waiting a specified amount of time before acting on a signal. My first experience with this was in handling a potentially noisy electrical signal. When you press a button, for instance, the voltage may “bounce” for a short period, like so:

![Credit: Geoffrey Hunter https://blog.mbedded.ninja/electronics/circuit-design/debouncing/](https://cdn.sanity.io/images/ts10onj4/production/3037375e7887af09da718a2af77dbe7f53703eb1-1024x721.jpg?w=700)

You don’t want to send “on” and “off” signals in quick succession, you want to wait until it’s settled out and send the value once it’s settled out. You “de-bounce” the signal. In software, this could look like waiting to send a search query until the user has finished typing for some amount of time. Every time the user types another character, it resets the debounce timer. The benefit is avoiding intermediate requests that wouldn’t be used, but the cost is waiting for the debounce period to elapse. If the user keeps typing, they may never see any results!

**Throttling** is the act of spacing out requests that a client sends. In the example with the user continuously typing a search query above, a search could be executed on the first character, and every x seconds after that. So the user could start seeing results from their in-progress query as they continue typing. Under the hood, every time a request is sent, the next request will be held until some time has elapsed. If many keystrokes are issued during that time, only a single request will be sent when the time elapses, with the latest query. This limits the maximum rate at which a single client sends requests.

**Rate limiting** is more commonly referenced on the server side. Rather than clients proactively limiting themselves, this is a way for the server to push back on clients, telling them they’re requesting too much. This is referred to as “back pressure” - the server pushing back on clients when too much is being demanded of it. It can help keep a backend system from being overloaded due to spikes in traffic, though it then relies on clients to handle the request and retry later on. This is a good idea for a reliable system, but should be exceptional, not the only way of limiting client requests.

**Single flighting** is the concept of limiting requests by only ever having one request “in flight.” This is similar to throttling, in that it limits requests from the client. However, the frequency of requests isn’t specified up front, but is a function of how fast the network is, and how fast the request is processed on the server. This second factor also gives it some natural back pressure from the server, which is incredibly valuable. If the server is getting overloaded, that client won’t be sending more requests while it’s waiting for its outstanding one. It also allows us to have gradual performance degradation if many clients are executing requests in parallel. Rather than becoming overwhelmed and failing some subset of requests, the frequency of requests will decrease in each clients as the server hits a bottleneck. The only downside is that your request frequency might be slower than theoretically possible, due to time spent on network transit. Waiting until a response comes back before sending another request is slower than optimistically firing off requests continuously. Alternatively, if your requests return quickly, you might fire off more requests than are necessary and waste CPU & network resources.

As with most things, there are benefits to each, and weighing these strategies is part of the job of the application developer. For this article, we are going to be using single flighting. This plays well with Convex, since the convex client executes mutations serially. With serial execution, debouncing and throttling both risk piling up requests, if they aren’t processed as fast as they’re created. Single flighting helps us avoid this, providing a consistently responsive user experience.

# Implementation: useEffect loop with useLatestValue

One way to achieve single flighting requests is to sit in an infinite loop, waiting for a new value to send and then waiting on the request. This leverages a hook we wrote: `useLatestValue`. This provides two functions: one to update some value, and another that you can await for the latest value, blocking until there’s a newer value than what you’ve already received. It is conceptually similar to a `Promise` where you can keep calling `resolve` with newer values to overwrite the value returned when awaited. Before we talk about how it’s implemented, let’s look at an example of how it might be used:

```tsx
type Pos = {x: number, y: number};

const updatePresence = useMutation("updatePresence");
const [nextPosition, setPosition] = useLatestValue<Pos>();

useEffect(() => {
  let run = true;
  (async () => {
    while (run) {
		  const position = await nextPosition();
			await updatePresence(presenceId, position);
    }
  })();
  return () => { run = false; };
}, [nextPosition, updatePresence]);

return <div onPointerMove={(e) => setPosition({
	x: e.clientX,
  y: e.clientY, 
})}>...</div>
```

This sends position updates whenever a new value is available. The `nextPosition()` promise will resolve when the value is updated. If one or more `setPosition` calls happen before awaiting `nextPosition`, it will immediately return the value from the latest call when it is eventually awaited.

The `useLatestValue` hook helpers are in a working project [here](https://github.com/convex-stack/convex-presence/tree/main/hooks). You can use it as is, but for those who are curious how it works, see below. 
<details>
<summary>`useLatestValue` details...</summary>

    `useLatestValue` uses a `Promise` as a signal that a new value is available. The result of a `Promise` can’t be updated once it’s resolved, so the value is stored separately. When a value is retrieved, the `Promise` is awaited, the latest value returned, and the signal reset. Updating the value just involves updating the value and resolving the `Promise`, relying on the behavior that subsequent calls to `resolve` is a no-op if it’s already been resolved.
    
    ```tsx
    export default function useLatestValue<T>() {
      const initial = useMemo(() => {
        const [promise, resolve] = makeSignal();
        // We won't access data until it has been updated.
        return { data: undefined as T, promise, resolve };
      }, []);
      const ref = useRef(initial);
      const nextValue = useCallback(async () => {
        await ref.current.promise;
        const [promise, resolve] = makeSignal();
        ref.current.promise = promise;
        ref.current.resolve = resolve;
        return ref.current.data;
      }, [ref]);
    
      const updateValue = useCallback(
        (data: T) => {
          ref.current.data = data;
          ref.current.resolve();
        },
        [ref]
      );
    
      return [nextValue, updateValue] as const;
    }
    
    const makeSignal = () => {
      let resolve: () => void;
      const promise = new Promise<void>((r) => (resolve = r));
      return [promise, resolve!] as const;
    };
    ```

</details>

# Implementation: useSingleFlight callback

Another model that avoids the scary infinite loop is using a helper we wrote: `useSingleFlight` which will run a given async function at most once at a time. If no calls are in progress, it will call the function immediately. Otherwise, when the current call finishes, it will call the function again, using the most recent arguments. 

```tsx
const updatePresence = useMutation("updatePresence");
const tryUpdate = useSingleFlight(updatePresence);
return <div onPointerMove={(e) => tryUpdate({
	x: e.clientX,
  y: e.clientY, 
})}>...</div>
```

While it isn’t used here, it’s worth mentioning that `tryUpdate` will always return a `Promise`. If the call isn’t executed, the promise will never resolve or reject. If it is called, the result of the call will be passed through. So you could write code like:

```tsx
console.log('trying to update');
const result = await tryUpdate(pos);
console.log('updated: ' + result);
```

which would log `'trying to update'` for all event callbacks, and only log `'updated: ...'` for requests that were actually sent to the server. And if the call failed, it would throw the exception during the `await`.

The `useSingleFlight` hook helper is in a working project [here](https://github.com/convex-stack/convex-presence/tree/main/hooks). You can use it as is, but for those who are curious how it works, see below. 
<details>
<summary>`useSingleFlight` details...</summary>

    `useSingleFlight` keeps track of whether a request is in flight, using a `useRef` hook to store state. If there is a request in flight, it returns a promise, where it has extracted the `resolve` and `reject` functions to fulfill later if it’s still the latest attempt. It updates the state’s `upNext` to keep track of the arguments and promise functions. If there isn’t a request in flight, it calls the function immediately, and also kicks off an async function to check for `upNext` once the request finishes. It will keep executing the follow-up requests until it finishes without `upNext` being updated.
    
    ```tsx
    export default function useSingleFlight<
      F extends (...args: any[]) => Promise<any>
    >(fn: F) {
      const flightStatus = useRef({
        inFlight: false,
        upNext: null as null | { resolve: any; reject: any; args: Parameters<F> },
      });
    
      return useCallback(
        (...args: Parameters<F>): ReturnType<F> => {
          if (flightStatus.current.inFlight) {
            return new Promise((resolve, reject) => {
              flightStatus.current.upNext = { resolve, reject, args };
            }) as ReturnType<F>;
          }
          flightStatus.current.inFlight = true;
          const firstReq = fn(...args) as ReturnType<F>;
          void (async () => {
            try {
              await firstReq;
            } finally {
              // If it failed, we naively just move on to the next request.
            }
            while (flightStatus.current.upNext) {
              let cur = flightStatus.current.upNext;
              flightStatus.current.upNext = null;
              await fn(...cur.args)
                .then(cur.resolve)
                .catch(cur.reject);
            }
            flightStatus.current.inFlight = false;
          })();
          return firstReq;
        },
        [fn]
      );
    }
    ```

</details>

## Delta challenges

While the approach may seem simple, there are some nuances worth thinking through. If you are limiting how many requests you’ll be sending, that often means some requests will never be executed, or some results won’t get reported. 

Each call to `tryUpdate` above will either:

1. execute `updatePresence` immediately,
2. execute `updatePresence` after some time has elapsed, or
3. never execute `updatePresence`.

In this case, we ignore any intermediate mouse positions. This seems fine for a use case where we’re just sharing cursor location. However, if we were reporting a delta - such as reporting a series of movements rather than absolute positions, then missing intermediate values would be bad news!

Let us know in [our discord](https://convex.dev/community) if you want examples on how to handle those cases.

## Optimistic updates

One thing to keep in mind with our [Optimistic Updates](https://docs.convex.dev/using/optimistic-updates) api is that optimistic updates will only be run as often as your single-flighted function (e.g. your mutation). If you want to update local state faster than the mutations are being executed, you’ll need to manage that state separately. For example:

```tsx
const myMutation = useMutation("myMutation");
const withLocalStore = useCallback((data) => {
  setLocalState(data);
  return myMutation(data);
}, [setLocalState, updatePresence]);
const tryUpdate = useSingleFlight(withLocalStore);
...
```

# Next Steps

We’ve looked at two ways of implementing single-flighting requests, which is a great way of preventing request pile-up and keeping UIs responsive. To see an implementation of this, check out our post on implemeting Presence in Convex (coming soon!). To get the code for our hooks, check it out [here](https://github.com/convex-stack/convex-presence/tree/main/hooks).

For write-heavy applications, use single flighting to dynamically throttle requests. See how we implement this with React hooks for Convex.

Throttling Requests by Single-Flighting

Let’s talk about feature flags.

Have you ever wanted to launch a new feature at an exact time, and not just when some deployment finishes? Or made a mistake on a new feature and had to wait for a “hot fix” deploy to switch back to the old version? Wouldn’t it be great to roll out & roll back a feature with the flick of a switch? Today we’re going to talk about how to flip features on and off remotely using a clever use of the reactive nature of Convex queries. For those who don’t already know, Convex is a backend-as-a-service that allows you to write reactive queries and transactional mutations, all in typescript (or vanilla js if you prefer). Learn more at [docs.convex.dev](http://docs.convex.dev).

# What is a feature flag?

A “feature flag” or “feature gate” or “kill switch” are all terms I’ve heard for similar functionality: deciding what *feature* to show a user based on some configurable state, referred to here as a *flag*. Implementations differ slightly, but there will be some API to get the current value for a given flag  so you can make decisions in the frontend and/or backend code. 

There are companies like LaunchDarkly who have made this their whole business. At  Dropbox, we rolled our own version. There is a lot of value in using an off-the-shelf solution, and a lot of advanced functionality that we won’t replicate here today. The goal today is to see how easy it is to provide a basic implementation. The basic feature set we are targeting is:

1. A feature can be enabled / disabled without re-deploying.
2. A client can get updates to the state without reloading the page.

# Safely adding a new feature

I’m going to be adding an interactive chat to the Convex homepage where there is currently just a static image:

![](https://cdn.sanity.io/images/ts10onj4/production/8ac3c2708ca4d01cc85a8c0c45aeb5c176fe9086-233x403.png?w=300)

... into an interactive chat window:

![](https://cdn.sanity.io/images/ts10onj4/production/23afdc8526353caeaa106d7a5a3fd261ee9791c3-320x355.gif?w=700)

This is the fourth panel in a component showing the code necessary to implement chat, so it seems only fair to see the code in action!

Here’s the React component that’s showing the fourth image:

```jsx
<div className="...">
  <Image
    src="/tabsContent4.png"
    width={210}
    height={388}
    alt="Image of messaging app"
    loading="eager"
  />
</div>
```

What we want is to have some conditional like this:

```jsx
<div className="...">
  {showMessages ? (
    <Messages />
  ) : (
    <Image
      src="/tabsContent4.png"
      width={210}
      height={388}
      alt="Image of messaging app"
      loading="eager"
    />
  )}
</div>
```

But how do we get the value of showMessages?

# Flags table

Let’s keep a table in our backend of what features are on & off, and have the website decide which to show based on that value. Our table in the Convex dashboard after `db.insert("flags", {key: "homepage_chat", value: true})` looks like this:

![](https://cdn.sanity.io/images/ts10onj4/production/f308d225e3233d8aafdee27f41c2d84256477ed8-559x266.png?w=450)

# Reading the value

To wire up the value to the frontend, we will use a server-side query to read the flag’s value. In Convex, we can do this by making a file in `convex/getFlag.ts` in our code repo like this:

```tsx
export default query(async ({ db }, flagName: string) => {
  const flag = await db
    .query("flags")
    .filter(q => q.eq(q.field("key"), flagName))
    .first();
  return flag?.value;
});
```

Once we run `npx convex deploy`, this code will run in Convex’s servers. By using Convex, the return value will automatically be cached based on the function parameters, and the cache automatically invalidates when the flag value changes. Read more about this [here](https://docs.convex.dev/understanding/convex-fundamentals/functions#query-functions). This makes the lookup in the general case wicked fast, and avoids hammering the database. Caching is especially important if it’s a value every client would be fetching, which is the case for config values like this. Traditionally, you’d need to implement your own caching layer with something like Redis, and manually track the cache invalidation. With Convex, it happens by default!

To access it on the client, we just updated our code to:

```tsx
const showMessages = useQuery("getFlag", "homepage_chat");
```

This React hook will return the flag’s value, and will trigger a refresh whenever the value changes. Under the hood it’s using a WebSocket that’s shared with any other Convex queries you might be subscribed to, so it’s not clogging the network with polling requests, and the changes are near-instantaneous!

# In action

To prove that it works in production:

![Enabling the feature in production](https://cdn.sanity.io/images/ts10onj4/production/bad730aad2fcacae636e8b7d1f1c87ad44704e6c-960x583.gif?w=600)

# Default value

One edge case that off-the-shelf solutions provide, is supplying an initial value until you get the latest version from the server. For this, there’s a few strategies:

1. Show a loading indicator until you know what to show. This is a poor UX, but may be necessary when enabling the wrong feature could be catastrophic.
2. Supply a default in code to use until you get a server response. If you go with this approach for a new feature, the typical flow is to ship the code defaulting to “off”, and then once you’ve released the feature, change the default to “on,” to reduce re-rendering. In our code this looks like `useQuery("getFlag", "homepage_chat") ?? DEFAULT` since Convex returns `undefined` until the first response comes in. This is my favorite, as it also serves as documentation to code readers about what the “canonical” value is at a given point in git history.
3. Supply a default based on a recent, but potentially stale state. For SSG, you might read the flag value when the page is being generated, and use that possibly-stale value as the default. For SSR, you could read the flag value when the page is being rendered, and provide that as the default value.
4. Store the last read default in the browser’s localStorage or similar. I like this one the least, since you still have to handle the initial visit default, and the staleness of the value is hard to reason about. If you return something more complex than a boolean, you might end up with a value that is no longer supported!

# Common pitfalls

While feature flagging is great, I can say from experience that if you don’t maintain some discipline, the code can get hard to reason about. In particular, watch out for these pitfalls:

1. Testing: Make sure your tests check all possible values, not just the default path.
2. Ownership: Every feature flag should have a point of contact who knows why it’s there, and when it can be removed. 
3. Documentation: Related to ownership, documenting feature flags can be critical for an oncall rotation to know how to mitigate issues arising from your new feature. What is safe to change? How would I turn it off? I have even linked to feature flags from alerting systems, with instructions about how to turn off features in the case of overload, calling out the expected user impact.
4. Removal: For launching a new feature, you should add removing the feature flag as part of the feature release process. Once it’s been successfully rolled out and you have confidence in it, removing the code helps delete unused code, as well as make the codebase easier to reason about.

# Advanced features

As I mentioned before, there’s good reasons to use off-the-shelf feature flagging solutions. They’ve thought a lot about it, and help you avoid a lot of the above pitfalls, as well as provide rich features that we didn’t even discuss:

- Segmenting the user population. You might want to roll a feature out slowly, only show it to internal users, or A/B test different approaches.
- Metrics. Know how many users have seen a flag in a each state, who they are, and ensuring they’re consistently assigned the same state whenever possible.
- Fancy UIs to manage and audit feature flag changes. In our example we just manually toggled values in the database, which isn’t a very good idea for a production site! These are great places to add documentation and usage graphs.
- Offline caching, which is especially important for mobile clients.

… and the list goes on.

# In summary

![](https://cdn.sanity.io/images/ts10onj4/production/6f755c51e0130a3bf79ba0bda634fd4922aaee18-1985x1468.png?w=450)

Today we made a slick, minimal feature gate for a new feature, allowing us to roll out on our own schedule, and roll back just as fast. Convex allowed us to achieve all of this without having to worry about caches, invalidation, polling, or triggering UI refreshes ourselves. I hope it’s been helpful! And of course, if you have any questions or need help building anything in Convex, please come visit us in [Discord](https://convex.dev/community).

Today we’re going to talk about how to flip features on and off remotely using a clever use of the reactive nature of Convex queries. 

Ian Macartney

All posts by Ian