Code With Antonio

Discord for any problems/errors/bugs: https://www.codewithantonio.com/discord
Github & Live Website: https://www.codewithantonio.com/projects/notes-app
Edgestore: https://edgestore.dev
Trunk: https://tinyurl.com/trunkcheck

Hi all 👋 In this 8 hour tutorial you will learn how to create an end-to-end fullstack notion clone, all with proper notion-style editor, cover images, nested documents, publishing documents to public, real-time database and more.

Fullstack Notion Clone: Next.js 13, React, Convex, Tailwind | Full Course 2023

# Start building faster with `npm create convex`

There’s a new way quickest way to start building a new app on Convex!

You can now use the magical command `npm create convex@latest` to spin up a new Convex app from a preexisting template. This will install and run the latest version of the `create-convex` tool, which will present you with choices about the frontend client & authentication providers you’d like to use in this app.

## Create an app with preconfigured integrations

It’s this simple to generate a new Convex app: run `npm create convex@latest` and follow the prompts! Here’s what it would look like to create an app using React, Vite, and Clerk:

```bash
% npm create convex@latest
Need to install the following packages:
create-convex@0.0.5
Ok to proceed? (y) y
✔ Project name: … my-app
✔ Choose a client: › React (Vite)
✔ Choose user authentication solution: › Clerk

Setting up...

added 454 packages in 9s

✔ Done. Now run:

  cd my-app
  npm run dev
```

Now, in the newly created `my-app` directory you have a complete and ready-to-customize TypeScript React app, with Vite and Clerk configured and a `convex/` directory just waiting for you to fill it with fantastic functions!

## Now make the app your own

To start fleshing out your backend, edit or replace `convex/myFunctions.ts`, which comes pre-populated with examples of `query`, `mutation` , and `action` functions to get you started.

And in this app, which uses React/Vite for the frontend, you can start editing `src/App.tsx` to build out your UI. To help you do that as quickly and effortlessly as possible, the bootstrapped app comes with ready-to-use [shadcn/ui](https://ui.shadcn.com/) components, which you can customize in the `components/` directory and style with TailwindCSS classes.

## Choose from our existing templates

One of the great things about Convex is you’re not restricted to a certain frontend or auth provider, so you can choose your own adventure by answering the prompts. You can currently choose between Next.js and Vite for the client (or none, if you don’t need a frontend or want to provide your own), and between Clerk and Lucia for the authentication provider (or none).

When you run `npm create convex`, the CLI prompts let you walk through those choices one by one, and based on your selections the bootstrapper tool chooses one of several preexisting templates as the basis for your app. 

See the [create-convex README](https://github.com/get-convex/templates/tree/main/create-convex) on Github for a list of the currently supported templates, and stay tuned as we plan to add even more in the future!

## Specify a template with `-t`

If you already know which template you want to use for your app, you can skip the prompts and cut to the chase by specifying the app name and a template directly with the `-t` flag,  using the syntax `npm create convex <app-name-here> -- -t <template-name-here>`, like so:

```bash
$ npm create convex my-next-app -- -t nextjs-lucia-shadcn

Setting up...

added 523 packages in 26s

✔ Done. Now run:

  cd my-next-app
  npm run dev
```

And if you’d rather use some other project as the basis for your app (perhaps one of our many [sample apps](https://www.convex.dev/templates)?), you can specify the owner & name of any Github repo as the template to copy:

```bash
$ npm create convex@latest my-clerk-app -- -t thomasballinger/convex-clerk-users-table
```

For more details, head over to the [get-convex/templates](http://github.com/get-convex/templates) repo on Github, which contains the source of the `create-convex` bootstrapper package as well as all the default templates.

And as always, don’t forget to hop into the Convex [Discord](https://www.convex.dev/community) to share your amazing new apps with the community. Happy building!

There’s a new fastest way to start building a new app on Convex. Use our new magical command, npm create convex, to spin up a new Convex app from a preexisting template.

A New Command to Start Building Faster: npm create convex

Convex now integrates with Fivetran via a [brand new source connector!](https://fivetran.com/docs/databases/convex) This unlocks lots of data engineering use cases for apps built on Convex. Essentially, Convex apps can now replicate data to external data warehouses like Snowflake, Databricks, Google Big Table, and many more from [Fivetran’s destination catalog](https://fivetran.com/docs/destinations).

Check out Fivetran's post on their new integrations and SDKs [here](https://www.fivetran.com/blog/unlocking-third-party-compatibility-with-our-new-sdks).

# OLTP and OLAP Databases

The database at the core of Convex is best suited for Online Transaction Processing (OLTP) use-cases like running a live site and subscribing to realtime queries. Convex is designed to optimize for these use cases so you can build realtime, reactive applications. Data warehouses, on the other hand, are optimized for Online Analytical Processing (OLAP) use-cases like deep analysis on past data. These two types of databases have fundamentally different design.

OLTP databases (like Convex) typically optimize for small, predictable indexed queries. OLTP thrives on point queries (fetching/writing a single document) and indexed range queries over small ranges. OLTP databases often use Optimistic Concurrency Control (OCC) to allow for greater throughput in independent mutations by executing multiple mutations at the same time optimistically, failing them on write if they conflict. OLTP databases often used journaled row-based data structures with auxiliary indexes. This design lends itself to small, fast queries.

OLAP databases (like Snowflake, Databricks, and BigTable) are best suited for large and possibly unpredictable queries like table scans. OLAP databases are typically organized in a column-oriented structure that enables parallel analysis over large amounts of data. High transaction rates and low latency are typically not a priority.

It can even be dangerous to run large analytics queries against OLTP databases. They can result in unpredictable load on the system that could adversely affect your live traffic. Predictability is reliability’s best friend.

# Connecting Convex to Data Warehouses with Fivetran

With the new Convex source connector for Fivetran, you can get the best of both OLTP and OLAP. Build your live application on Convex, then use Fivetran to get your data to your Data Warehouse and Business Intelligence (BI) tools of choice for analysis.

For those who want to just get started quickly, Fivetran with a managed [Google BigQuery offering](https://fivetran.com/docs/destinations/bigquery/managed-bigquery) which works nicely with [Google Looker Studio](https://lookerstudio.google.com/). If you want more control of the setup, you can always choose your own destination.

# Getting Started with Convex+Fivetran

You’ll need a Convex Professional Plan. Learn about the different plans [here](https://www.convex.dev/plans). Then head to your convex dashboard [here](dashboard.convex.dev).

## 1. Grab your Convex Credentials

Grab your Deployment Credentials from the Convex deployment [dashboard](https://dashboard.convex.dev) by going to the Settings section on the bottom of the left nav bar. Your deployment credentials will be the first screen that shows up.

![](https://cdn.sanity.io/images/ts10onj4/production/b999e85353f8832d011755ef3cd761ea7362ba50-2502x1196.png?w=1500)

## 2. Set up your Fivetran Connector

Create a Fivetran account and create a new connector. Select the destination of your choice. Select the “Convex” source connector. From there, you drop your credentials in and get syncing.

![](https://cdn.sanity.io/images/ts10onj4/production/698322cd35cec0636996af11baf162dcac568d38-1228x1244.png?w=1200)

# Next Steps

During the early access period, you’ll have to contact us in our [community Discord](https://convex.dev/community) or through support@convex.dev to get started, but it’ll be freely available soon.

We'd love your feedback on the integration as we continue polishing it with the Fivetran team for a beta release.

Launching our official Convex Fivetran connector.

Adding Data Integration to Your Convex Application with Fivetran

Building ChatGPT-powered experiences feel snappier when the responses show up incrementally. Instead of waiting for the full response before showing the user anything, streaming the text in allows them to start reading immediately. 

OpenAI exposes a streaming API for chat completions. But how do you manage a GPT streaming request when you have a server between the client and OpenAI? You might be tempted to use HTTP streaming end to end - both from the client to the server and the server to OpenAI. However, there’s another way that comes with some big benefits. Spoiler: it’s possible to use a database as a layer of reactivity that separates client request lifecycles from server requests. Don’t worry if that doesn’t make sense yet - we’ll take it one step at a time.

This post will look at working with streams with OpenAI’s Node SDK. Beyond just getting streaming for a single user, we’ll look at an approach that enables:

- **Persisting** the response even if the user closes their browser.
- **Multiplayer chat**, including **streaming multiple ChatGPT messages** at once.
- **Resuming** a stream when a user **refreshes their browser** mid-stream.
- Streaming to **multiple users** at once.
- Implement **custom stream granularity**, such as only updating on full words or sentences, rather than on each token.

To do this, we’ll use <QuickConvex /> to store the messages and make the request to OpenAI. This code is [on GitHub](https://github.com/ianmacartney/streaming-chat-gpt) for you to clone and play with.

![Diagram of browsers talking to Convex, which talks to OpenAI](https://cdn.sanity.io/images/ts10onj4/production/9a7b8865f6cd1cb6748fdb88c986d6ec7bd26bdb-1200x638.png "Data flow overview")


# Persisting messages

Let’s say we have a chat app, like the one pictured in the gif above. We want to store the messages from each user, as well as messages populated by responses from OpenAI. First let’s look at how data is stored (2), assuming a client sends a message (1).

When a user sends a message, we immediately commit it to the database, so they’re correctly ordered by creation time. This code is executed on the server:

```tsx
export const send = mutation({
  args: { body: v.string(), author: v.string() },
  handler: async (ctx, { body, author }) => {
    // Save our message to the DB.
    await ctx.db.insert("messages", { body, author });

    if (body.indexOf("@gpt") !== -1) {
      // ...see below
    }
  }
});
```

This [`mutation`](https://docs.convex.dev/get-started#mutation-functions) saves the message to the database. When the user wants a response from the GPT model (by adding “@gpt” to the message), we will:

1. Store a placeholder message to update later.
2. Make a streaming request to OpenAI in an asynchronous background function. 
3. Progressively update the message as the response streams in.

By running the streaming request asynchronously (versus blocking in a user request), we can interact with ChatGPT and save the data to the database **even if the client has closed their browser**. It also allows us to run many requests in **parallel**, from the same or **multiple users**.

We also run it asynchronously because, in Convex, mutations are pure transactions and as such can’t do non-deterministic things like making API requests. In order to talk to third-party services, we can use an `action`. Actions are non-transactional serverless functions that can talk to third-party services. We trigger the background job to call ChatGPT and update the message body by [scheduling](https://docs.convex.dev/scheduling/scheduled-functions) the [action](https://docs.convex.dev/functions/actions) like so:

```tsx
// ...when the user wants to send a message to OpenAI's GPT model
const messages = // fetch recent messages to send as context
// Insert a message with a placeholder body.
const messageId = await ctx.db.insert("messages", {
  author: "ChatGPT",
  body: "...",
});
// Schedule an action that calls ChatGPT and updates the message.
await ctx.scheduler.runAfter(0, internal.openai.chat, { messages, messageId });
```

We schedule it for zero milliseconds later, similar to doing `setTimeout(fn, 0)` in JavaScript. The message writing and action scheduling happens transactionally in a [mutation](https://docs.convex.dev/functions/mutation-functions), so we will only run the action if the messages are successfully committed to the database.

When the action wants to update the body of a message as the streaming results come in, it can invoke an `update` mutation with the `messageId` from above:

```tsx
export const update = internalMutation({
  args: { messageId: v.id("messages"), body: v.string() },
  handler: async (ctx, { messageId, body }) => {
    await ctx.db.patch(messageId, { body });
  },
});
```

Note: An `internalMutation` is just a mutation that isn’t exposed as part of the public API. Next we’ll look at the code that calls this `update` function.

Convex has end-to-end reactivity, so when we update the messages in the database, the UI automatically updates. See [below](#client-streaming-via-subscriptions) what it looks like to reactively query data.

# Streaming with the OpenAI node SDK

Streaming is currently available in the beta version of OpenAI’s node SDK. To install it:

```bash
npm install openai
```

The `internal.openai.chat` action we referenced above will live in `convex/openai.ts` - see the full code [here](https://github.com/ianmacartney/streaming-chat-gpt). 

```tsx
import { OpenAI } from "openai";
import { internalAction } from "./_generated/server";
//...
type ChatParams = {
  messages: Doc<"messages">[];
  messageId: Id<"messages">;
};
export const chat = internalAction({
  handler: async (ctx, { messages, messageId }: ChatParams) => {
    //...Create and handle a stream request
```

## Creating a stream request

```tsx
// inside the chat function in convex/openai.ts
const apiKey = process.env.OPENAI_API_KEY!;
const openai = new OpenAI({ apiKey });

const stream = await openai.chat.completions.create({
  model: "gpt-3.5-turbo", // "gpt-4" also works, but is so slow!
  stream: true,
  messages: [
    {
      role: "system",
      content: "You are a terse bot in a group chat responding to q's.",
    },
    ...messages.map(({ body, author }) => ({
      role:
        author === "ChatGPT" ? ("assistant" as const) : ("user" as const),
      content: body,
    })),
  ],
});
//...handling the stream
```

Note passing `stream: true`. This changes the return format, which **unfortunately does not currently provide token usage** as the non-streaming version does. I hope this is fixed in a future release, as keeping track of token usage is useful to know how different users or features are affecting your costs.

## Handling the stream

The API exposed by the `openai` SDK makes handling the stream very easy. We use an async iterator to handle each chunk, appending it to the body and updating the message body with everything we’ve received so far:

```tsx
let body = "";
for await (const part of stream) {
  if (part.choices[0].delta?.content) {
    body += part.choices[0].delta.content;
    await ctx.runMutation(internal.messages.update, {
      messageId,
      body,
    });
  }
}
```

Note that here we’re updating the message every time the body updates, but we could implement **custom granularity** by deciding when to call `runMutation`, such as on word breaks or at the end of full sentences.

This action allows us to stream messages from OpenAI to our server function and into the database. But how does this translate to clients updating in real time? Next, let’s see how the client reactively updates as messages are created and updated.

# Client “streaming” via subscriptions

After the previous sections, you might be surprised how little is required to get the client to show live updating messages. I put streaming in quotes since we aren’t using HTTP streaming here - instead, we’re just using the reactivity provided out-of-the-box by Convex.

On the client, we use the `useQuery` hook, which calls the `api.messages.list` server function in the `messages` module, which we’ll see in a second. This hook will give us an updated list of messages every time a message is added or modified. This is a special property of a Convex [query](https://docs.convex.dev/get-started#query-functions): it tracks the database requests, and when any of the data is changed it:

1. Invalidates the query cache (which is managed transparently by Convex).
2. Recomputes the result.
3. Pushes the new data over a WebSocket to all subscribed clients.

```tsx
export default function App() {
  const messages = useQuery(api.messages.list);
  ...
  return (
    ...
    {messages?.map((message) => (
      <article key={message._id}>
        <div>{message.author}</div>
        <p>{message.body}</p>
      </article>
    ))}
```

Because this query is decoupled from the HTTP streaming response from OpenAI, **multiple browsers** can be subscribed to updates as messages change. And if a user **refreshes or restarts their browser**, it will just pick up the latest results of the query.

On the server, this is the [query](https://docs.convex.dev/database/reading-data#querying-documents) that grabs the most recent 100 messages:

```tsx
export const list = query({
  handler: async (ctx): Promise<Doc<"messages">[]> => {
    // Grab the most recent messages.
    const messages = await ctx.db.query("messages").order("desc").take(100);
    // Reverse the list so that it's in chronological order.
    // Alternatively, return it reversed and flip the order via flex-direction.
    return messages.reverse();
  },
});
```

Convex is doing some magic under the hood. If any message is inserted or updated into the database that would match this query - for instance if a new message is added or one of the first 100 messages is edited - then it will automatically re-execute this query (if there are any clients subscribed to it via `useQuery`). If the results differ, it will push the new results over a WebSocket to the clients, which will trigger an update to the components using `useQuery` for that query. 

To give you a sense of performance, `list` takes ~17ms and `update` takes ~7ms for me on the server, so the total latency between a new token coming from OpenAI and a new set of messages being sent to the client is very fast. The gifs in this article are real recordings, not sped up.

![GPT response streaming in quickly](https://cdn.sanity.io/images/ts10onj4/production/2979c6d28d619bffded99c19bca49f6d229a4cdb-960x960.gif)

# Summary

We looked at how to stream ChatGPT responses into Convex, allowing clients to watch the responses, without the flakiness of browser-based HTTP streaming requests. The full code is available [here](https://github.com/ianmacartney/streaming-chat-gpt). Let us know [in Discord](https://convex.dev/community) what you think!

### Extra Credit 🤓

Beyond what’s covered here, it would be easy to extend this demo to:

- Store whether a message has finished streaming by storing a boolean on the message updated at the end of the stream.
- Add error handling, to mark a message as failed if the stream fails. See [this post](https://stack.convex.dev/full-stack-chatgpt-app#creating-the-action) for an example of updating a message in the case of failure.
- Schedule a function to serve as a watchdog, that marks a message as timed out if it hasn’t finished within a certain timeframe, just in case the action failed. See [this post](https://stack.convex.dev/background-job-management#5-monitoring-timeouts) for more details, as well as other patterns for background jobs.
- Organize the messages by thread or user, using [indexes](https://docs.convex.dev/database/indexes/).

Stream GPT responses without brittle browser-based HTTP streaming.
Multiplayer reactivity, persistence, reactivity via Convex. Using OpenAI’s Node SDK server-side, and Convex's useQuery hook client-side.

GPT Streaming With Persistent Reactivity

Implementing user authentication (sign up, log in, log out) is hard! Authentication is an end-to-end feature involving secrets and credentials - the most sensitive kind of app development. For this reason it’s a good idea to outsource this problem to auth solution providers like [Clerk](https://clerk.dev/) or [Auth0](https://auth0.com/), or OAuth providers like Google or Github.

Unfortunately using a third-party comes with its own challenges: User data needs to be synced between the external service and your database. Auth solution providers need to be configured, can come with technical limitations and might charge a hefty sum once your app surpasses their free tiers.

Similarly, OAuth providers can be difficult to set up initially, and they can limit audience if your users do not have an account with a given provider.

For all these reasons you might want to implement authentication yourself. To simplify this, it’s a good idea to use an authentication library such as [Lucia](https://lucia-auth.com/), which will manage the logic around users, sessions and credentials for you.

With Lucia and Convex, we can build APIs reading current user information simply like this:

```jsx
import { queryWithAuth } from "./withAuth";

export const getLoggedInUserEmail = queryWithAuth({
  args: {},
  handler: async (ctx) => {
	  return ctx.session?.user.email;
  }
});
```

No external database or provider required!

We put together a template which includes sign up, log in and log out flows. Check out [the GitHub repo](https://github.com/get-convex/convex-lucia-auth-demo) or follow along with this article for the details.

## Update: Even easier integration

Run `npm create convex@latest -- -t nextjs-lucia-shadcn` to start a new project with a template that includes authentication based on Lucia, using the [convex-lucia-auth](https://www.npmjs.com/package/@convex-dev/convex-lucia-auth) NPM library.

The rest of this article goes over the manual integration of Lucia with Convex, which is still helpful for understanding and in case you want total control over configuring Lucia.

## Overview of the user experience

For this template we implemented a sign up / sign in flow which asks the user for their email and password. Once the user clicks `Sign Up` or `Sign In` their credentials are checked and they are either logged in or an error message is shown.

Once the user is logged-in we show them data associated with their account, in our case that’s their email, and a button that enables them to log out.

Let’s look at how this experience is implemented.

## Step 1: Specify our database schema

Although Convex doesn’t strictly require you to declare your database schema up front, in this case it will be helpful for us to look at what data we’ll be storing, and to get types since we’ll be using TypeScript for end-to-end type safety throughout our auth implementation.

```jsx
// convex/schema.ts

import { defineSchema, defineTable } from "convex/server";
import { Validator, v } from "convex/values";

export default defineSchema({
  users: defineTable({
    id: v.string(),
    email: v.string(),
  }).index("byId", ["id"]),
  sessions: defineTable({
    id: v.string(),
    user_id: v.string(),
    active_expires: v.float64(),
    idle_expires: v.float64(),
  })
    .index("byId", ["id"])
    .index("byUserId", ["user_id"]),
  auth_keys: defineTable({
    id: v.string(),
    hashed_password: v.union(v.string(), v.null()),
    user_id: v.string(),
  })
    .index("byId", ["id"])
    .index("byUserId", ["user_id"]),
});
```

We declare three tables:

- `users` which stores data for each user
- `sessions` which stores data for each session, mainly to do with its expiration
- and `auth_keys`, which represent different user credentials using any sign-in method

This schema will be used by our database adapter. It satisfies Lucia’s requirements. The only caveat to mention is that Lucia is currently architected to allocate IDs itself, while Convex allocates IDs automatically when documents are created. For this reason our users, sessions and auth_keys will have two ID fields: Convex allocated, built-in `_id` and Lucia allocated `id`.

For this demo we added an `email` field for storing the user’s email.

In our [template repo](https://github.com/get-convex/convex-lucia-auth-demo/blob/main/convex/schema.ts) we split the schema between the fields required by Lucia and custom fields, so our schema looks much simpler:

```jsx
export default defineSchema({
  ...authTables({
    user: { email: v.string() },
    session: {},
  }),
});
```

Let’s get on with implementing the database adapter that will enable us to take advantage of Lucia’s features.

## Step 2: Configure Lucia

We’ll want to interact with Lucia from our server queries and mutations. For this we’ll define a function which will instantiate Lucia, with our Convex adapter. This is the `getAuth` function:

```jsx
// lucia.ts
import { lucia } from "lucia";
import { DatabaseWriter } from "./_generated/server";

export function getAuth(db: DatabaseWriter) {
  return lucia({
    adapter: convexAdapter(db),
    // TODO: Set the LUCIA_ENVIRONMENT variable to "PROD"
    // on your prod deployment's dashboard
    env: (process.env.LUCIA_ENVIRONMENT as "PROD" | undefined) ?? "DEV",
    getUserAttributes(user: UserSchema) {
      return {
        _id: user._id,
        _creationTime: user._creationTime,
        email: user.email,
      };
    },
  });
}
```

We can call this function from any query* or mutation, passing in `ctx.db`. `convexAdapter` is the actual database adapter implementation. Check out its [source on Github](https://github.com/get-convex/convex-lucia-auth-demo/blob/main/convex/lucia.ts).

With Convex, implementing the database was straightforward. You can check Lucia’s adapter API docs [here](https://lucia-auth.com/extending-lucia/database-adapters-api).

In the `getUserAttributes` configuration we expose the `_id`, `_creationTime` and `email` documents fields to code loading users via `getAuth`.

*As you can see in the code, `getAuth` accepts `DatabaseWriter`.  This is the database interface exposed to Convex mutations. Convex queries can only read from the database, not write to it, and so they have access to `DatabaseReader` instead. When we call getAuth from a query helper in Step 3, our types won’t perfectly match runtime execution. Lucia’s API is not split between read and write operations, type-wise. We could declare a read-only subset of Lucia’s Auth API and return it from an additional `getReadOnlyAuth` function, but this won’t be necessary as we’ll only use a single, read-only, method in queries: `getSession`.

### Configuring types

Lucia has a cool way of injecting custom database types into its APIs. These are our definitions:

```jsx
// convex/env.d.ts

declare namespace Lucia {
  type Auth = import("./lucia").Auth;
  type DatabaseUserAttributes = {
    _id: import("./_generated/dataModel").Id<"users">;
    _creationTime: number;
    email: string;
  };
  type DatabaseSessionAttributes = {
    _id: import("./_generated/dataModel").Id<"sessions">;
    _creationTime: number;
  };
}
```

If we add more fields to our `"users"` table that we want to access through `getAuth`, we can add them here.

## Step 3: Leverage Lucia in our functions - queries

With the adapter and schema in place, it’s time to use them to implement our authentication logic.

Lucia includes middleware for storing and accessing session IDs on Requests and Responses. Since we want to use Convex’s reactive client which uses a WebSocket, we’ll instead pass the session ID through function arguments and return values.

Our simple “read” “middleware” will use the following function that returns a session if the given session ID is valid:

```jsx
import { getAuth } from "./lucia";
import { QueryCtx } from "./_generated/server";

async function getValidExistingSession(
  ctx: QueryCtx,
  sessionId: string | null
) {
  if (sessionId === null) {
    return null;
  }
  // The cast is OK because we will only expose the existing session
  const auth = getAuth(ctx.db as DatabaseWriter);
  try {
    const session = (await auth.getSession(sessionId)) as Session | null;
    if (session === null || session.state === "idle") {
      return null;
    }
    return session;
  } catch (error) {
    // Invalid session ID
    return null;
  }
}
```

With this helper we can implement a query that relies on the user information attached to a session:

```jsx
import { query } from "./_generated/server";

export const getLoggedInUserEmail = query({
  args: {
    sessionId: v.union(v.null(), v.string()),
    // other argument validators
  },
  handler: async (ctx, args) => {
    const session = await getValidExistingSession(ctx, args.sessionId);
    return session.user.email;
  },
});
```

Having to declare that we take a `sessionId` for every query that needs to use the session information would get old quite quickly. For this reason we implemented a `queryWithAuth` wrapper that does this for us. Check it out in the [repo](https://github.com/get-convex/convex-lucia-auth-demo/blob/main/convex/withAuth.ts). With it, accessing session is as simple as with Convex’s built-in auth. Our query `ctx` gets new `session` field:

```jsx
import { queryWithAuth } from "./withAuth";

export const getLoggedInUserEmail = queryWithAuth({
  args: {},
  handler: async (ctx) => {
	  return ctx.session?.user.email;
  }
});
```

We can access user information, but how does the information get into our DB in the first place?

## Step 4: Leverage Lucia in our functions - mutations

Armed with `getAuth` we can now implement our sign up and sign in mutations:

### Sign up

```jsx
import { v } from "convex/values";
import { getAuth } from "./lucia";

export const signUp = mutation({
  args: {
    email: v.string(),
    password: v.string(),
  },
  handler: async (ctx, { email, password }) => {
    const auth = getAuth(ctx.db);
    const user = await auth.createUser({
      key: {
        password: password,
        providerId: "password",
        providerUserId: email,
      },
      attributes: { email },
    });
    const session = await auth.createSession({
      userId: user.userId,
      attributes: {},
    });
    return session.sessionId;
  }
});
```

Check out the [Auth](https://lucia-auth.com/reference/lucia/interfaces/auth) API reference. Here we use the `createUser` and `createSession` methods to sign the user up and log them in. The `password` field holds the secret provided by the user (in our case, a password), `"password"` is our sign up method (we might add OAuth provider as a different method later), and `providerUserId` in our case is the user-filled out email.
Under the hood this method creates both a document in our `"users"` table and also a key in our `"auth_keys"` table.

We then create a session for the user and return its ID to the client. If an error occurs, such someone trying to sign up with an email that’s already taken, Lucia will throw and the error will propagate to the client.

### Sign in

```jsx
import { v } from "convex/values";
import { getAuth } from "./lucia";

export const signIn = mutation({
  args: {
    email: v.string(),
    password: v.string(),
  },
  handler: async (ctx, { email, password }) => {
    const auth = getAuth(ctx.db);
    const key = await auth.useKey("password", email, password);
    const session = await auth.createSession({
      userId: key.userId,
      attributes: {},
    });
    return session.sessionId;
  }
});
```

Check out the [Auth](https://lucia-auth.com/reference/lucia/interfaces/auth) API reference. Here we use `useKey` and `createSesson` to sign the user in. 

`useKey` takes the sign up method (`"password"` in our case) and the unique `providerUserId` and secret (in our case, the user provided password).

The cool thing is, you are in control of how the sign up / sign in works. The two mutations presented here are the standard approach, but you could for example combine sign up and sign in into a single form with a single mutation, or sign existing users in if they accidentally go through the sign up flow. It’s up to you!

### Other mutations

Just like with queries, we made a [wrapper for mutations](https://github.com/get-convex/convex-lucia-auth-demo/blob/main/convex/withAuth.ts) that makes it easy to check that a user is currently signed in. Since mutations can update the database, we will not only validate the session, but also renew it:

```jsx
import { Auth } from "./lucia";

async function getValidSessionAndRenew(auth: Auth, sessionId: string | null) {
  if (sessionId === null) {
    return null;
  }
  try {
    // Lucia's `validateSession` auto-renews the session
    return await auth.validateSession(sessionId);
  } catch (error) {
    // Invalid session ID
    return null;
  }
}

```

Our wrapper uses this helper under the hood to expose both `auth` and `session` on `ctx`:

```jsx

import { mutationWithAuth } from "./withAuth";

export const doSomethingAsTheUser = mutationWithAuth(
  args: {},
  handler: async (ctx) => {
	  if (ctx.session === null) {
	    throw new Error("Expected a logged-in user");
	  }
	  // do something...
		// ctx.auth is `LuciaAuth` instead of Convex's `Auth`
  },
});
```

You can see in the repo that we used this wrapper to simplify `signIn` and `signUp` mutations as well.

## Step 5: Frontend

### Keeping track of a session

We’ll be storing the secret sessionId in `localStorage`, so we create a simple React Context provider for updating and accessing it:

```jsx
import { createContext, useCallback, useContext, useState } from "react";

const SessionContext = createContext(undefined as any);

export function useSessionId(): string | null {
  return useContext(SessionContext).sessionId;
}

export function useSetSessionId(): (sessionId: string | null) => void {
  return useContext(SessionContext).setSessionId;
}

export function SessionProvider({ children }: { children: React.ReactNode }) {
  const [sessionId, setSessionIdState] = useState(getSavedSessionId());
  const setSessionId = useCallback(
    (value: string | null) => {
      setSavedSessionId(value);
      setSessionIdState(value);
    },
    [setSessionIdState]
  );
  return (
    <SessionContext.Provider value={{ sessionId, setSessionId }}>
      {children}
    </SessionContext.Provider>
  );
}

function getSavedSessionId() {
  return localStorage.getItem("sessionId");
}

export function setSavedSessionId(sessionId: string | null) {
  if (sessionId == null) {
    localStorage.removeItem("sessionId");
  } else {
    localStorage.setItem("sessionId", sessionId);
  }
}
```

And wrap our app in it:

```jsx
// src/main.tsx

ReactDOM.createRoot(document.getElementById("root")!).render(
  <React.StrictMode>
    <SessionProvider>
      <ConvexProvider client={convex}>
        <App />
      </ConvexProvider>
    </SessionProvider>
  </React.StrictMode>
);
```

Now we can use and set session ID from anywhere in our app:

```jsx
// src/App.tsx
import { useSessionId } from "./SessionProvider.tsx";
import { useQuery } from "convex/react";
import { LoginForm } from "./LoginForm";

function App() {
  const sessionId = useSessionId();
  const user = useQuery(api.users.get, {sessionId});

  return (
    <main>
      {user === undefined ? (
        <div>Loading...</div>
      ) : user == null ? (
        <LoginForm />
      ) : (
        <h2>Welcome {user.email}</h2>
      )}
    </main>
  );
}
```

Just like on the backend, we created a [wrapper](https://github.com/get-convex/convex-lucia-auth-demo/blob/main/src/withAuth.tsx) for `useQuery` that passes in the session ID automatically, so that we can we can simplify our code:

```jsx
// src/App.tsx

// Our custom `useQuery` is imported here
import { useQuery } from "./usingSession";

function App() {
  // sessionId is passed in automatically
  const user = useQuery(api.users.get);
  // ...
}
```

### Sign up / sign in form

Let’s see how we can implement the `LoginForm` component for sign up and sign in. This will be calling our `signUp` and `signIn` mutations.

```jsx
import { useState } from "react";
import { useSetSessionId } from "./SessionProvider";
import { useMutation } from "./usingSession";
import { api } from "../convex/_generated/api";

export function LoginForm() {
  const setSessionId = useSetSessionId();
  const [flow, setFlow] = useState<"signIn" | "signUp">("signIn");
  const signIn = useMutation(api.users.signIn);
  const signUp = useMutation(api.users.signUp);

  const handleSubmit = async (event: React.FormEvent<HTMLFormElement>) => {
    event.preventDefault();
    const data = new FormData(event.currentTarget);
    try {
      // Call the corresponding mutation
      const sessionId = await (flow === "signIn" ? signIn : signUp)({
        email: (data.get("email") as string | null) ?? "",
        password: (data.get("password") as string | null) ?? "",
      });
      // Set session ID in the Session React Context and localStorage
      setSessionId(sessionId);
    } catch {
      alert(
        flow === "signIn" ? "Invalid email or password" : "Email already in use"
      );
    }
  };
  return (
    <form onSubmit={handleSubmit}>
      <label htmlFor="username">Email</label>
      <input name="email" id="email" />
      <label htmlFor="password">Password</label>
      <input type="password" name="password" id="password" />
      {flow === "signIn" ? (
        <input type="submit" value="Sign in" />
      ) : (
        <input type="submit" value="Sign up" />
      )}
      <a
        style={{ cursor: "pointer" }}
        onClick={() => {
          setFlow(flow === "signIn" ? "signUp" : "signIn");
        }}
      >
        {flow === "signIn" ? (
          <>Don't have an account? Sign up</>
        ) : (
          <>Already have an account? Sign in</>
        )}
      </a>
    </form>
  );
}
```

Since they are so similar this single form implements both sign in and sign up. You could extend it in any way you like, for example by adding an OAuth sign up method (Google, Github). Lucia can help with this as well, check out [Lucia OAuth docs](https://lucia-auth.com/oauth).

## Conclusion and next steps

Well, that was a wild ride. As you can see even the simplest authentication implementation has a quite a few moving pieces. Good news is that you can start off by cloning our [example repo](https://github.com/get-convex/convex-lucia-auth-demo/blob/main/screenshot.png) with all of these pieces already hooked up.

You now have authentication that talks directly with the Convex database, and without any redirects or need for configuring third party services.

To take your app to production you have several options:

1. Implement optional email verification and password recovery flow. See Lucia’s guide on [email verification](https://lucia-auth.com/guidebook/email-verification-links). You’ll need a third-party (or your own) SMTP server for sending emails (check out [Resend](https://resend.com/)).
2. Switch to [Clerk](https://clerk.dev/) or [Auth0](https://auth0.com/) before you let external users interact with your app, throwing away your existing test signups.
3. Switch to [Clerk](https://clerk.dev/) or [Auth0](https://auth0.com/) later, importing existing signups into their database. See [Clerk import users doc](https://clerk.com/docs/deployments/import-users) or [Auth0 bulk user import doc](https://auth0.com/docs/manage-users/user-migration/bulk-user-imports).

If you’re using SSR with Next.js, Remix, or similar, you can use Lucia’s built-in [middleware](https://lucia-auth.com/basics/handle-requests) to use cookies instead of localStorage for storing sessions. Remember to protect your app against [CSRF attacks](https://github.com/get-convex/convex-lucia-auth-demo#note-on-csrf-protection).

Learn how to build a full stack app with authentication without any third-party auth providers, using Convex and the Lucia library.

Custom Authentication (with Lucia)

If you’re building a full-stack app, chances are you’ll want some of your users to pay you for the service you provide. [Stripe](https://stripe.com/) makes taking payments really easy, and with <QuickConvex /> you can ensure that you deliver when the payment comes in.

We put together a full working app that lets users pay to send a message to a global message board. Check out [the GitHub repo](https://github.com/get-convex/convex-stripe-demo) or follow along with this article that goes over the details.


## Overview of the user experience

Our UX for this example is simple:

1. The user types a message and hits the “Pay $1 and send” button.
2. They are redirected to a checkout site hosted by Stripe, where they fill out their payment details and hit the Pay button.
3. They are then redirected back to our web application, where they can see the message they paid for highlighted.
![Split view of Convex Paid Chat UI and Stripe checkout](https://cdn.sanity.io/images/ts10onj4/production/30a0bc857f303da2f397a20f5082ecfc96b51442-2400x1276.png?w=1200)

How do we know that the payment has been processed and the message can be sent? And how can we tell which message is the one the user just paid for? Read on to find out.

## Step 1: Initiate the checkout flow

To kick off the payment flow, we call a Convex action that does a couple of things for us:

1. It creates a document in our “payments” table so that we can track the progress of the checkout flow.
2. It calls Stripe’s SDK to get the checkout page URL the client should redirect to.
3. It includes the ID of our “payments” document in the `success_url` that Stripe will redirect to after the user has finished paying.
4. Finally, it writes the Stripe checkout session ID into the “payments” document to ensure we only fulfill the “order” once.

```jsx
// convex/stripe.ts

// This pragma is important because Stripe's SDK currently
// only works in the Node Runtime
"use node";

import { v } from "convex/values";
import { action } from "./_generated/server";
import Stripe from "stripe";
import { internal } from "./_generated/api";

export const pay = action({
  // The action takes the message the user composed
  args: { text: v.string() },
  handler: async (ctx, { text }) => {
    // We need to tell Stripe where to redirect to
    const domain = process.env.HOSTING_URL ?? "http://localhost:5173";
    const stripe = new Stripe(process.env.STRIPE_KEY!, {
      apiVersion: "2022-11-15",
    });
    // Here we create a document in the "payments" table
    const paymentId = await ctx.runMutation(internal.payments.create, { text });
    // This is where the Stripe checkout is configured
    const session = await stripe.checkout.sessions.create({
      line_items: [
        {
          // For this example we use dynamic `price_data`,
          // but you could use a predefined `price` ID instead
          price_data: {
            currency: "USD",
            unit_amount: 100,
            tax_behavior: "exclusive",
            product_data: {
              name: "One message of your choosing",
            },
          },
          quantity: 1,
        },
      ],
      mode: "payment",
      // This is how our web page will know which message we paid for
      success_url: `${domain}?paymentId=${paymentId}`,
      cancel_url: `${domain}`,
      automatic_tax: { enabled: true },
    });

    // Keep track of the checkout session ID for fulfillment
    await ctx.runMutation(internal.payments.markPending, {
      paymentId,
      stripeId: session.id,
    });
    // Let the client know the Stripe URL to redirect to
    return session.url;
  },
});
```

This action uses 2 internal mutations that perform writes to our database:

```jsx
// convex/payments.ts

import { v } from "convex/values";
import { internalMutation } from "./_generated/server";

export const create = internalMutation({
  args: { text: v.string() },
  handler: async (ctx, { text }) => {
    return await ctx.db.insert("payments", { text });
  },
});

export const markPending = internalMutation({
  args: {
    paymentId: v.id("payments"),
    stripeId: v.string(),
  },
  handler: async (ctx, { paymentId, stripeId }) => {
    await ctx.db.patch(paymentId, { stripeId });
  },
});
```

All that is left is to call this action when the user clicks the “Pay $1 and send” button and redirect to the returned URL:

```jsx
// src/App.tsx
import { FormEvent, useState } from "react";
import { useAction } from "convex/react";
import { api } from "../convex/_generated/api";

export default function App() {
  // State storing the message text
  const [newMessageText, setNewMessageText] = useState("");
  
  // Reference to our action
  const payAndSendMessage = useAction(api.stripe.pay);

  // We pass this handler to a `form` `onSubmit` in our UI code
  async function handleSendMessage(event: FormEvent) {
    event.preventDefault();
    // Call to our action
    const paymentUrl = await payAndSendMessage({ text: newMessageText });
    // Redirect to Stripe's checkout website
    window.location.href = paymentUrl!;
  }
  // the rest of our UI...
}
```

## Step 2: The payment checkout flow

Nothing to do here! Stripe did this work for us. 😮‍💨

## Step 3: Receiving the payment confirmation

Stripe will notify our server about the confirmed payment via a webhook request. All we need to do is expose an HTTP endpoint that Stripe’s servers can hit whenever a user finishes a payment. That’s a perfect fit for a Convex [HTTP action](https://docs.convex.dev/functions/http-actions). The HTTP action will handle the Stripe webhook request, parsing it, and passing its contents to another Convex action that will confirm the request is valid using the Stripe SDK in the Node.js runtime:

```jsx
// convex/http.ts
import { httpRouter } from "convex/server";
import { httpAction } from "./_generated/server";
import { internal } from "./_generated/api";

const http = httpRouter();

http.route({
  path: "/stripe",
  method: "POST",
  handler: httpAction(async (ctx, request) => {
    // Getting the stripe-signature header
    const signature: string = request.headers.get("stripe-signature") as string;
    // Calling the action that will perform our fulfillment
    const result = await ctx.runAction(internal.stripe.fulfill, {
      signature,
      payload: await request.text(),
    });
    if (result.success) {
      // We make sure to confirm the successful processing
      // so that Stripe can stop sending us the confirmation
      // of this payment.
      return new Response(null, {
        status: 200,
      });
    } else {
      // If something goes wrong Stripe will continue repeating
      // the same webhook request until we confirm it.
      return new Response("Webhook Error", {
        status: 400,
      });
    }
  }),
});

export default http;
```

Our `fulfill` action confirms the webhook request really came from Stripe via its SDK, gets the checkout session ID, and finally “fulfill”s the order by calling a database mutation:

```jsx
// convex/stripe.ts (continued)
export const fulfill = internalAction({
  args: { signature: v.string(), payload: v.string() },
  handler: async (ctx, { signature, payload }) => {
    const stripe = new Stripe(process.env.STRIPE_KEY!, {
      apiVersion: "2022-11-15",
    });

    const webhookSecret = process.env.STRIPE_WEBHOOKS_SECRET as string;
    try {
      // This call verifies the request
      const event = stripe.webhooks.constructEvent(
        payload,
        signature,
        webhookSecret
      );
      if (event.type === "checkout.session.completed") {
        const stripeId = (event.data.object as { id: string }).id;
        // Send the message and mark the payment as fulfilled
        await ctx.runMutation(internal.payments.fulfill, { stripeId });
      }
      return { success: true };
    } catch (err) {
      console.error(err);
      return { success: false };
    }
  },
});
```

The internal mutation makes writes to our database. It’s awesome that this mutation is automatically transactional, so if we made it this far, we are guaranteed to **both** mark the payment as fulfilled (by adding a `messageId` to it) **and** send the message (by adding it to the “messages” table).

```jsx
// convex/payments.ts (continued)
export const fulfill = internalMutation({
  args: { stripeId: v.string() },
  handler: async ({ db }, { stripeId }) => {
    const { _id: paymentId, text } = (await db
      .query("payments")
      .withIndex("stripeId", (q) => q.eq("stripeId", stripeId))
      .unique())!;
    const messageId = await db.insert("messages", { text });
    await db.patch(paymentId, { messageId });
  },
});
```

At this point, the payment and fulfillment flow works end-to-end.

## Bonus: Tying it all together

Our demo includes one more trick for highlighting the message the user just paid for. This could be used to show a unique order confirmation, even if the user made 2 purchases in parallel (from two different browser tabs).

Back in step 1, we added the “payments” document ID to the URL Stripe redirects to after a successful checkout, the `success_url`.

We can use this information from the URL to highlight the relevant message:

```jsx
// src/App.tsx

// A simple React hook that grabs a URL query parameter and erases
// it from the URL.
function useConsumeQueryParam(name: string) {
  const [value] = useState(
    // get the param value
    new URLSearchParams(window.location.search).get(name)
  );

  useEffect(() => {
    const currentUrl = new URL(window.location.href);
    const searchParams = currentUrl.searchParams;
    searchParams.delete(name);
    const consumedUrl =
      currentUrl.origin + currentUrl.pathname + searchParams.toString();
    // reset the URL
    window.history.replaceState(null, "", consumedUrl);
  }, []);
  return value;
}

export default function App() {
  // use the hook to get the "payments" document ID
  const paymentId = useConsumeQueryParam("paymentId");
  // Call a query to exchange the "payments" document ID
  // for the corresponding "messages" document ID
  const sentMessageId = useQuery(api.payments.getMessageId, {
    paymentId: (paymentId ?? undefined) as Id<"payments"> | undefined,
  });
  const messages = useQuery(api.messages.list) || [];
  // render a list of messages
  return (
    <>
      <ul>
        {messages.map((message) => (
          <li
            key={message._id}
            // highlight the message the user just paid for, if any
            className={sentMessageId === message._id ? "sent" : ""}
          >
            <span>{message.text}</span>
          </li>
        ))}
      </ul>
      {* ... *}
    </>
  );
}
```

Refer to the [the Github repo](https://github.com/get-convex/convex-stripe-demo) for the full implementation.

## It was ninety-nine cents!
![Convex $ Stripe from Macklemore's Thrift Shop Music Video](https://cdn.sanity.io/images/ts10onj4/production/331ef4de890fc6db7270439d62592c0d98613726-947x499.jpg)

While most people won’t pay a dollar to send a message to a random message board, our demo app showcases all the necessary pieces for integrating a full-fledged payment flow. Convex and Stripe make this flow feel solid and reliable. Let us know [in Discord](https://convex.dev/community) what your app charges for 🤑

If you’re building a full-stack app, chances are you’ll want some of your users to pay you for the service you provide. How to use Stripe with Convex to get it done.

Wake up, you need to make money! (ft. Stripe)

Looking to implement semantic search or add on-demand context to a GPT prompt so it doesn’t just make shit up (as much)? [Pinecone](https://www.pinecone.io/) and <QuickConvex /> are a good match when you’re looking to build an app that leverages [embeddings](https://stack.convex.dev/the-magic-of-embeddings) and also has user data. Pinecone stores and queries over vectors efficiently, and Convex stores relational & document data with strong transaction guarantees and nifty end-to-end data reactivity.

Let’s walk through how this shakes out in practice. If you want to see some code you can play around with, check out [this GitHub repo](https://github.com/ianmacartney/embeddings-in-convex/) where you can add your own data and compare it and search over it using Convex and Pinecone.

# High-level user flow

To start, what’s an **example**? With Pinecone and Convex, you can have a flow like this:

1. A user submits a **question** and starts **subscribing** to the question’s results. Under the hood, Convex has stored the question in a **document** and kicked off an asynchronous **action**. If this question has been asked before, it might re-use previous results.
2. The **action** creates an **embedding** using a service like OpenAI or Cohere. It can persist this embedding for posterity or to be able to search for similar questions.
3. The action uses the embedding to **query** Pinecone (or any vector store) for **related documents**, **products**, or whatever your embeddings represent.
4. The action stores the **results** in the question **document**, which automatically reflows to update the user’s client with the new data - potentially returning materialized data pulled from other documents in the database associated with the results.
5. If this is part of a broader **chain** of operations, it might use the related documents to compose a **prompt** to an **LLM** like [ChatGPT](https://stack.convex.dev/full-stack-chatgpt-app), using both the related documents and the question to get a more **contextual answer**.

## A word on streaming updates

At every step, updates written to the Convex database will update **all** subscribed clients. Unlike raw HTTP streaming, Convex subscriptions can be trivially consumed by multiple clients in parallel and are resilient to network issues or page refreshes. The data received will be from a consistent snapshot of the database state, making it easier to reason about correctness.

<BuildWithConvex />

Many of the code snippets below can be found in [this GitHub repo](https://github.com/ianmacartney/embeddings-in-convex/) which you’re welcome to play around with using your own data and API keys. If you are desperate for a hosted playground, let me know [in Discord](http://discord.gg/convex)!

# Adding data to Convex and Pinecone

Depending on the application, you may have a large mostly-static corpus of data, or be continually adding data — which I’ll refer to as a **source** below. The process looks something like the following:

## Break up your source into bite-sized chunks.

This helps limit how much data you pass (embedding models have context limits), as well as make the embedding more targeted. You could imagine an embedding of this whole post might not rank as highly against “How do you add data to Convex and Pinecone?” as an embedding that just covered this section. 

To do this, you can split it yourself or use a library like LangChain’s `RecursiveCharacterTextSplitter`:

```tsx
import { RecursiveCharacterTextSplitter } from "langchain/text_splitter";

const textSplitter = new RecursiveCharacterTextSplitter({
  chunkSize: ChunkSize,
});
const splitTexts = await textSplitter.createDocuments([pageContent]);
const chunks = splitTexts.map((chunk) => ({
  text: chunk.pageContent,
  lines: chunk.metadata.loc.lines,
}));
```

You can tune the size and algorithm to your own needs. One tip is to add some overlap, so each chunk has some text from the previous and next sections.

## Store the source in the database

The Convex database is a great place to store all the metadata that your app will want. For embeddings based on text, you’ll likely even want to store the chunk of text in the database, so you can quickly access it to return as part of queries for a client or as part of a pipeline. For larger data, like video, it makes more sense to store the data in file storage.

Importantly, you should **not** store the text chunk directly in Pinecone metadata, as it can quickly fill up the index because **all metadata is indexed by default in Pinecone**.

```tsx
async function addSource(
  db: DatabaseWriter,
  name: string,
  chunks: { text: string; lines: { from: number; to: number } }[]
) {
  const sourceId = await db.insert("sources", {
    name,
    chunkIds: [],
    saved: false,
  });
  const chunkIds = await Promise.all(
    chunks.map(({ text, lines }, chunkIndex) =>
      db.insert("chunks", {
        text,
        sourceId,
        chunkIndex,
        lines,
      })
    )
  );
  await db.patch(sourceId, { chunkIds });
  return (await db.get(sourceId))!;
}
```

There are a few things to note here:

- I’m both saving a back-reference from chunks to sources, as well as a forward reference from a source to many chunks. As discussed in [this post on database relationships](https://stack.convex.dev/relationship-structures-let-s-talk-about-schemas), this is a way to get quick access in both directions without having to define an extra index when you have a small number of relations (8192 technically but &lt;1k is my rule of thumb).
- I’m saving an empty array at first, then patching it with the chunk IDs once I insert them. Convex generates unique IDs on insertion. At this time you can’t pre-allocate or specify custom primary IDs for documents.
- I’m creating the source with `saved: false` - we’ll update this once we’ve saved the embeddings into Pinecone. This allows the client to know the insertion status, as well as help with transient failures, which we’ll see later on.

## Kick off a background action

The Convex `mutation` function is transactional but as a result, we can’t perform a non-transactional operation like talking to a third-party service in the middle of a mutation. A Convex `action` is non-transactional and can talk to the outside world. One trick I like to use is to schedule an action to execute after a mutation commits, ensuring that the communication with the outside world only happens if the mutation has successfully run.

Mutations in Convex are transactions and are prohibited from having non-transactional side effects like calling other cloud services. With actions you can make these sorts of calls, but how do you “call” an action from a mutation if the mutation can’t have side effects? A pattern I really like is to schedule the action from the mutation:

```tsx
await ctx.scheduler.runAfter(0, internal.sources.addEmbedding, {
  source,
  texts: chunks.map(({ text }) => text),
});
```

Thanks to Convex’s strong transaction guarantees, the action is only invoked if the mutation successfully commits, so you’ll never have an action running for a source that doesn’t exist.

## Create an embedding

From our [action](https://docs.convex.dev/functions/actions), we can fetch embeddings. See [this post](https://stack.convex.dev/the-magic-of-embeddings) for more information on what embeddings are. See the [code for fetchEmbeddingBatch here](https://github.com/ianmacartney/embeddings-in-convex/blob/main/convex/lib/embeddings.ts). 

```tsx
const { embeddings } = await fetchEmbeddingBatch(texts);
```

## Upsert into Pinecone

Adding data into Pinecone is a straightforward operation. “Upsert” for those unfamiliar is an update if the specified `id` already exists, otherwise it inserts. 

```tsx
await upsertVectors(
  "chunks", // namespace
  source.chunkIds.map((id, chunkIndex) => ({
    id,
    values: embeddings[chunkIndex],
    metadata: { sourceId: source._id, textLen: texts[chunkIndex].length },
  }))
);
```

**Tips:**

- We aren’t including much metadata here - in general, you should **only store metadata** that you might want to use **to limit Pinecone queries** - such as keywords, categories, or in this case text length[^1].
- We’re **re-using the Convex document ID** for the pinecone vector. This isn’t required—you could make up your own ID and store that in the Convex document—but I find it very handy. Results of Pinecone queries, without returning metadata, can be used directly with `db.get` which is wicked fast. It also means you can fetch or delete the Pinecone vector for a given chunk, without storing an extra ID.
- I used the table name as the Pinecone namespace for convenience, so queries for chunks wouldn’t return vectors for other data. This isn’t required but helped me with organization and naming fatigue.

<details>
<summary>Tip: Use the `@pinecone-database/pinecone` Pinecone client for the best experience in Convex.</summary>

There are two action runtimes in Convex: our optimized runtime, and a generic node environment. When possible I prefer using the optimized runtime, so I can keep the actions in the same file as the queries and mutations, along with some performance benefits. However, our runtime doesn’t support all npm libraries. Thankfully the pinecone package doesn’t depend on any incompatible packages and just uses the `fetch` standard under the hood. This is also why I prefer using `fetch` and the OpenAI HTTP API directly above. See [here](https://docs.convex.dev/functions/runtimes) for more information on runtimes.

</details>

## Mark the source as “saved”

All we need to do to notify the frontend that the data has been saved is to update the source. Any queries that reference the source document will be updated with the latest data automatically.

```tsx
await ctx.runMutation(internal.sources.patch, {
  id: source._id,
  patch: { saved: true, totalTokens, embeddingMs },
});
```

At this point, our data is in Convex and an embedding vector is saved in Pinecone. 

## Extensions

Beyond saving chunks of the source, you might also consider:

- Adding an embedding of a summary of the whole source.
- Add a hierarchy of searches - where you could separately search for a category of documents and then provide that category as a [metadata filter](https://docs.pinecone.io/docs/metadata-filtering) in a later query.
- Namespacing or otherwise segmenting user data so you never accidentally leak context between users.

# Searching for sources

Similarly to inserting data, to do a semantic search over your documents, you can:

1. **Insert the search into a table of searches.** If there’s already an identical search, you could even decide to re-use those results. This is handy for iterating on large pipelines and keeping latency and costs low.
    
    ```tsx
    const searchId = await ctx.db.insert("searches", { input, count });
    ```
    
2. **Kick off an action** transactionally.
    
    ```tsx
    await ctx.scheduler.runAfter(0, internal.searches.search, {
      input,
      searchId,
      topK: count,
    });
    ```
    
3. **Create an embedding** of the search. Aside: I have a hunch there’s a lot of opportunity here for ways of transforming the raw search into a better text input for the embedding.
    
    ```tsx
    const { embedding } = await fetchEmbedding(input);
    ```
    
4. **Use the pinecone query** to find nearby vectors representing chunks. 
    
    ```tsx
    const { matches } = await pinecone.query({
      queryRequest: {
        namespace: "chunks",
        topK,
        vector: embedding,
      },
    });
    ```
    
5. **Update the search results** by running a mutation.
    
    ```tsx
    await ctx.runMutation(internal.searches.patch, {
            id: searchId,
            patch: {
              relatedChunks,
            },
          });
    ```
    
6. Optional: **store the search embedding in Pinecone** if you want to be able to search semantically over searches themselves!

Note: you could just do steps 2-4 directly in an action if you don’t care about keeping a cache and storing the search vector.

## Returning results to the client

The client can subscribe to the search document’s ID:

```tsx
const results = useQuery(api.searches.semanticSearch, { searchId });
```

The query looks for the search and returns the related chunks along with their source’s name:

```tsx
export const semanticSearch = query({
  args: { searchId: v.id("searches") },
  handler: async (ctx, { searchId }) => {
    const search = await ctx.db.get(searchId);
    if (!search) throw new Error("Unknown search " + searchId);
    if (!search.relatedChunks) return null;
    return pruneNull(
      await Promise.all(
        search.relatedChunks.map(async ({ id, score }) => {
          const chunk = await ctx.db.get(id);
          if (!chunk) return null;
          const source = await ctx.db.get(chunk.sourceId);
          return { ...chunk, score, sourceName: source!.name };
        })
      )
    );
  },
});
```

This is parallelized by using `Promise.all` and calls to `db.get` are cached.

# Summary

In this post, we looked at using Pinecone and Embeddings in Convex. One natural extension of this is to then use the sources as part of a GPT prompt template, but I’ll leave that for a future post. Let us know [in discord](https://convex.dev/community) what you think and what you’d like to see next!

[^1]: At the Pinecone hackathon, there was a discussion of issues of semantic rankings sometimes behaving oddly - in my case, when I searched over a corpus of famous poems for “what is the meaning of life?” one of the top hits was a “hello world” dummy text I had added. One participant mentioned that a useful filter—after listing a plethora of sophisticated ranking strategies—was to just exclude text less than 200 characters. Intuitively this makes some sense - the longer something is, as short phrases probably have higher semantic variance.

Pinecone and Convex are a good match when you're looking to build an application that leverages embeddings and also has application data.

Using Pinecone and Embeddings

When you’re starting a project, or adding functionality that requires collecting form data, Typeform can be a valuable tool. It has a beautiful, accessible, ergonomic design out of the box, and can enable you to get results quickly. As your project matures and you want to integrate that data with the rest of your app, you may be left wondering whether you need to rewrite the form yourself, or somehow integrate Typeform with your codebase. This post aims to help you with the latter, allowing you to keep leveraging Typeform, while keeping the data in a powerful transactional application database.

Here’s a [sample app](https://github.com/k8reindeer/typeform-in-convex) demonstrating how to integrate Typeform with Convex! You may want to remix different parts of it depending on what you’re trying to do with Typeform:

- **A.** Embed a Typeform form in your existing app.
- **B.** Import existing Typeform responses into Convex.
- **C.** Collect new Typeform responses into Convex in real-time.

With the approaches in this post, you can use Typeform as a seamless extension of your <QuickConvex/> app. For example:

- A user [logs in](https://docs.convex.dev/auth) to your Convex app.
- They fill out the form, which silently keeps track of their `user_id` in the `data-tf-hidden` attribute.
- Typeform receives the form response and immediately sends a webhook to your Convex app (in my testing, my Convex app heard about the response *faster* than Typeform’s own website!).
- Your Convex webhook handler saves the response data into the database based on the mapping you set up.
- Your reactive front end shows any UI updates as a result of the changed data, based on Convex’s built-in reactivity.

You could also build a UI that uses the `user_id` to show which of your Convex users was the one who filled out the Typeform, restrict users to only see their own responses, etc.

# A. Embed Typeform in your app

Typeform offers a powerful and easy-to-edit form builder product — and they also make it really easy to embed that into your own website, such as one you built with Convex. Typeform itself has great docs on this [here](https://www.typeform.com/help/a/embed-your-typeform-360029249212).

Here’s how it looks in my project:

![Typeform form in a Convex-backed website](https://cdn.sanity.io/images/ts10onj4/production/8889bc06a599e2ea8b464a6ab17e0d37cb6ae871-2880x1532.png?w=900 "Typeform form in a Convex-backed website")

```jsx
import Script from "next/script";
//...
	<button data-tf-slider="vX9noaqM" data-tf-position="right" data-tf-opacity="100" data-tf-iframe-props="title=Spotted a dog" data-tf-auto-close="500" data-tf-transitive-search-params="user_id" data-tf-medium="snippet" data-tf-hidden={`user_id=${userId}`} className="typeform-button">
	  Record a dog sighting
	</button>
	<Script src="//embed.typeform.com/next/embed.js"></Script>
```

This is *almost* what I copied from Typeform’s embed code, with a few notes:

- Typeform provided a bunch of styles for my button in a `style` property -- I just moved those to my existing CSS and customized them.
- I was using Next.js, and it was having issues loading the `<script>` element that Typeform suggested. I fixed this by using `import Script from "next/script";`.
- Finally, I added this attribute to my button, since my form has a `user_id` [hidden field](https://www.typeform.com/help/a/what-are-hidden-fields-360052676372/):

```jsx
data-tf-hidden={`user_id=${userId}`}
```

This keeps track of which Convex user is logged in when they fill out the form. When a user submits the form with this `data-tf-hidden` attribute, the `user_id` is sent to Typeform along with the rest of the data the user entered into the form. The data then ends up in Convex one of two ways: either by importing into Convex as a migration sometime later (section B below) or via a Typeform webhook in near-real-time (section C below). 

# B. Import existing Typeform responses into Convex.

What if you’ve had a Typeform running for a while, and you have form responses that you’d like to incorporate into your Convex app? Here’s how to get all the existing form responses into a table in Convex.

### Setup

First, grab these files from the [example project](https://github.com/k8reindeer/typeform-in-convex):

- `/scripts/setupTypeformMappings.js`
- `/scripts/importTypeform.js`

If you’re using a schema in your project, you can copy over the schema for the `typeform_metadata` table:

```jsx
typeform_metadata: defineTable({
    typeformFormId: v.string(),
    convexTableName: v.string(),
    typeformFieldId: v.string(),
    convexFieldName: v.string(),
  }).index("by_typeform_form_id", ["typeformFormId"]),
```

You’ll also need a Typeform API key ([get one here](https://www.typeform.com/developers/get-started/personal-access-token/)) and run in your terminal:

`export TYPEFORM_API_KEY=<your_api_key>`

Lastly, you’ll need the form ID of the form whose responses you want in Convex! You can find it in the URL of your Typeform — it’s an 8-character string.

### Map your form to database fields

Run 

`node scripts/setupTypeformMappings.js <your_form_id>`

This will create a file `typeformData/mappings_<your_form_id>.json` — open this file up.

Review and update the `convexTableName` and `convexFieldName` values in the json file — by default it just reformats the question text, so you probably want to give each of the fields a short, descriptive name. If you want to skip importing the responses to a question, just delete it at this step. 

### Move the data

Once you have your mappings the way you want them, run:

`node scripts/importTypeform.js <your_form_id>`

This will write a few more files in `typeformData` and print a few `npx convex import` statements, which will look something like 

```jsx
npx convex import typeform_metadata ./typeformData/typeform_metadata.jsonl
npx convex import dogs ./typeformData/dogs.jsonl
```

(My example table is named `dogs` — you’ll see whatever you chose to name your Convex table!) **Copy and paste each command** to import the data.

Now when you check in your [Convex dashboard](https://dashboard.convex.dev), you’ll see your new table with data populated for all the form responses you already have in Typeform, and the fields will be named as specified in your mappings json file. You can start using this to build your features around your new form response data!

# C. Collect new Typeform responses into Convex in real-time.

If you followed the steps in the previous section, you should have all your form responses *so far*… but what if more respondents fill out your form? This section explains how to set things up with a Typeform webhook so that as you get more Typeform responses, they automatically stay up to date in your Convex database. 

### Setup

For starters, you need to do *most* of the steps in item B to set up your Typeform mappings. Do all the setup and mapping steps there, up through 

```jsx
npx convex import typeform_metadata ./typeformData/typeform_metadata.jsonl
```

(you can skip the `npx convex import` for your new table if you don’t want those responses)

### Create webhook

Grab the file `scripts/setupWebhook.js` and run it:

```jsx
node scripts/setupWwebhook.js --form-id [your-typeform-id] --deployment-url https://[some-animal-123].convex.cloud
```

If you prefer, you can set it up manually following the steps [here](https://www.typeform.com/help/a/webhooks-360029573471/).

You can find the deployment url in `.env` (for prod) or `.env.local` (for dev) — this is set up for you when you set up your Convex project. It will look something like `some-animal-123`. 

This script will print out the value of a `TYPEFORM_SECRET_TOKEN` — add this as an [environment variable](https://docs.convex.dev/production/hosting/environment-variables) to your Convex deployment

### Convex functions to handle the webhook

Finally, you need the code! Copy these files into your project from [this repo](https://github.com/k8reindeer/typeform-in-convex):

- `convex/saveTypeformResponse.ts`
- `convex/authenticateTypeformWebhook.ts`
- `convex/http.ts` (if you already have a `http.ts` just combine the contents into your existing file) See [here](https://docs.convex.dev/functions/http-actions) for more info on serving HTTP requests from Convex.

Run `npx convex dev` to get your Convex functions ready!

### Test it out

You can test your webhook setup by submitting a response to your form! But if your form is too long and you want to test faster than that, there’s also a way to test your webhook, from the same page you set it up. Make sure the webhook is turned on on the Typeform side, then click “View Deliveries”, then “Send Test Request”

# Putting it all together

In this post, we walked through the steps for adapting this example Typeform integration to integrate your own Typeform into your own Convex project. Whether you want to embed a form on the front end, migrate response data to your back end, or hook things up to stay in sync going forward, hopefully this helps accelerate your project. Again, [the code is here](https://github.com/k8reindeer/typeform-in-convex). Let us know how you use it!

Leverage Typeform while keeping the data in a powerful transactional application database.

Get Typeform Data Into Convex

Web Dev Cody

GitHub repo: https://github.com/webdevcody/convex-replicate

Building a full stack app to generate images based on a prompt and user-provided sketch

How to Generate AI Images from Scribbles with Next.js

Trying to build an app on top of OpenAI’s Dall-E or ChatGPT? Avoid violating the OpenAI [Usage policies](https://platform.openai.com/docs/usage-policies/platform-policy) or showing inappropriate data to your users by leveraging their [Moderation API](https://platform.openai.com/docs/guides/moderation/overview). In this post, we’ll look at how to use it to flag messages before sending them to Chat-GPT. This will build on the [Building a full-stack ChatGPT app](https://stack.convex.dev/full-stack-chatgpt-app) post, but the approach is generally applicable.

New to Convex? [Convex](https://convex.dev/) is a backend application platform - we make I,t easy to build apps where you need to run code in the server and store data in a database. We run your functions, host your data, and much more. It’s a great fit for working with powerful APIs like OpenAI, as it allows you to build a whole application around powerful AI models without running any servers yourself.

Without further ado, let’s look at how to use OpenAI’s moderation endpoints. 

### **Initializing the API:**

The core piece of code is in the `convex/openai.js` file. This takes less than a millisecond, so we can run it on each request.

```jsx
const apiKey = process.env.OPENAI_API_KEY;
if (!apiKey) {
  await fail(
    "Add your OPENAI_API_KEY as an env variable in the " +
      "[dashboard](https://dasboard.convex.dev)"
  );
}
const configuration = new Configuration({ apiKey });
const openai = new OpenAIApi(configuration);
```

### **Using the moderation API:**

```jsx
// Check if the message is offensive.
const modResponse = await openai.createModeration({
  input: body,
});
const modResult = modResponse.data.results[0];
```

### **Updating the user’s message if the message was flagged:**

```jsx
if (modResult.flagged) {
  await runMutation(internal.messages.update, {
    messageId: userMsgId, 
    patch: {
      error:
        "Your message was flagged: " +
        Object.entries(modResult.categories)
          .filter(([, flagged]) => flagged)
          .map(([category]) => category)
          .join(", "),
    },
  });
  return;
}
```

As a reminder, this will call a mutation in `convex/messages.js` that will patch the message, adding an `error` field:

```jsx
export const update = internalMutation(async ({ db }, { messageId, patch }) => {
  await db.patch(messageId, patch);
});
```

### Why not check beforehand?

It’s a good point that we could do moderation before inserting the message into the database in the first place. In my case, I wanted to show results immediately and commit the *intent* to send a message. If I waited for moderation, it would take an extra 500ms+ before adding the message and appearing in the UI. For moderating identity creation (see [this post](https://stack.convex.dev/adding-personality-to-chatgpt-3) on adding identities), I moderate first for simplicity. I return an error when the user tries to create a lousy identity. If it felt too long, I could add a spinner while creating the identity. One nice thing about Convex mutations and actions is that you can `await` the results in the UI to know when they succeed/fail, even if you don’t care about their return value. For instance:

```jsx
setLoading(true);
setError(null);
const errorMsg = await addIdentity(
  newIdentityName,
  newIdentityInstructions
);
if (errorMsg) setError(errorMsg);
setLoading(false);
```

### ************************************************************************Refreshing the client’s copy of messages with the new field:************************************************************************

This happens automatically! One of the cool things about Convex is that queries are reactive by default. Here’s how it works:

1. We originally fetched a list of 100 recent messages in the UI with a query called `list` in `convex/messages.js`, which runs in Convex’s cloud. This result is cached automatically, thanks to guarantees provided by Convex’s [deterministic runtime](https://docs.convex.dev/understanding/convex-fundamentals/functions#determinism). 
2. When we sent a new message, we inserted a message into the “messages” table. 
3. Because the `list` query was for the most recent 100 messages, the results were automatically invalidated and re-executed. The new results get pushed to clients over a WebSocket,  triggering a React render for the component(s) that called `useQuery(api.messages.list)` with the new messages.
4. The same thing happens when we update a message to add an `error` field. As if by magic, the message with the `error` field shows up in the UI and we can render a different UI.

### **Rendering flagged messages in the UI:**

To avoid showing flagged messages, we can do some standard React gating:

```jsx
<span style={{ whiteSpace: "pre-wrap" }}>
  {message.error ? "⚠️ " + message.error : message.body ?? "..."}
</span>
```

Here we show an Error with ⚠️ pre-pended if there is one, otherwise a body if there is one, otherwise “…” for messages we haven’t updated with a bot’s response.

### Filtering flagged messages out of future bot input

We avoided sending the message to ChatGPT when we initially detected bad input. Still, when we send our next set of messages to the API, we don’t want to include the bad input in historical messages either. To do that, we can adjust our filter in `messages:send`:

```jsx
const messages = await db
      .query("messages")
      .filter((q) => q.eq(q.field("error"), undefined))
      .filter((q) => q.neq(q.field("body"), undefined))
      .take(21); // 10 pairs of prompt/response and our most recent message.
```

We avoid getting any messages with an “error” field and avoid all the messages without a body, for instance, bot messages that weren’t updated (such as when we bailed due to flagged input).

# Summary

In this post, we looked at how to moderate what content you send to ChatGPT, from the UI to the API calls. I hope it was helpful. Let us know [in Discord](https://convex.dev/community) what you think!

In this post, we’ll look at how to use the moderation API to flag messages before sending them to Chat-GPT, and patterns for handling these errors in a full-stack React app.

Moderating ChatGPT Content: Full-Stack

Building with OpenAI's APIs is fun. By leveraging [Convex](https://convex.dev) to store your data and run server-side functions, you can have a GPT-powered app in no time.

This is a follow-up to [Building a full-stack ChatGPT app](https://stack.convex.dev/full-stack-chatgpt-app), where we set up a Chat-GPT-like chat app using Convex as the backend. In that example, we used a fixed identity string. In this post we’ll look at how to store multiple personalities in the database and provide them in the chat, to enable changing personalities mid-conversation.

See the code [here](https://github.com/ianmacartney/convex-chat-gpt) and a working demo [here](https://convex-chat-gpt-f6rq.vercel.app/).

<BuildWithConvex />

To customize the identity you pass into ChatGPT, you can pass a “system” message to the OpenAI API. See more details on this API [here](https://platform.openai.com/docs/guides/chat). In our case, before having multiple identities, the code looks like this:

```js
const openaiResponse = await openai.createChatCompletion({
  model: "gpt-3.5-turbo",
  messages: [
    {
      role: "system",
      content: instructions,
    },
    ...messages.map(({ body, author }) => ({
      role: author,
      content: body,
    })),
  ],
});
```

Let’s store multiple identities in the Convex database and pick which one to use. 

### Adding and listing identities

Let’s make `convex/identity.js`:

```js
import { query, mutation } from "./_generated/server";

export const list = query(async (ctx) => {
  const identities = await ctx.db.query("identities").collect();
  return identities.map((identity) => identity.name);
});

export const add = mutation(async (ctx, { name, instructions }) => {
  const existing = await ctx.db
    .query("identities")
    .filter((q) => q.eq(q.field("name"), name))
    .unique();
  if (existing) {
    await ctx.db.patch(existing._id, { instructions });
  } else {
    await ctx.db.insert("identities", { name, instructions });
  }
});
```

This will run on the server and list the identity names or allow you to add one. Before it adds one, it will check if it already exists and, if so, update it, so there aren’t duplicate names. Because of Convex’s transaction isolation (which is the strictest kind, called “serializable isolation”), we’ll never have duplicate identities with the same name, simply because we queried for that name first. Read more about that [here](https://docs.convex.dev/understanding/deep-dive/occ).

Note that by only returning the identity names, we are hiding the instructions from the clients. This is a big difference between Convex and a platform like Firebase that gives you direct access to databases from the browser. By running code on the server, we can use code to limit what information is returned to clients.

### UI for adding identities

Adding a UI to add an identity is similar to sending messages with an `<AddIdentity />` component:

<details>
<summary>`AddIdentity` component </summary>
```jsx
function AddIdentity() {
  const addIdentity = useMutation("identity:add");
  const [newIdentityName, setNewIdentityName] = useState("");
  const [newIdentityInstructions, setNewIdentityInstructions] = useState("");

  return (
    <form
      onSubmit={async (e) => {
        e.preventDefault();
        await addIdentity({
          name: newIdentityName,
          instructions: newIdentityInstructions,
        });
        setNewIdentityName("");
        setNewIdentityInstructions("");
      }}
    >
      <input
        value={newIdentityName}
        onChange={(event) => setNewIdentityName(event.target.value)}
        placeholder="Identity Name"
      />
      <textarea
        value={newIdentityInstructions}
        onChange={(event) => setNewIdentityInstructions(event.target.value)}
        placeholder="GPT3 Instructions"
        rows={2}
        cols={40}
      />
      <input
        type="submit"
        value="Add Identity"
        disabled={!newIdentityName || !newIdentityInstructions}
      />
    </form>
  );
}
```
</details>

Let’s add some identities. Some I like: 

**Rubber Duck**

> You are curious and respond with helpful one-sentence questions.
> 

**Supportive Friend**

> You are a supportive and curious best friend who validates feelings and experiences and will give advice only when asked for it. You give short responses and ask questions to learn more.
> 

**CS Coach**

> You are a highly technically trained coach with expertise in technology and best practices for developing software. Respond with concise, precise messages and ask clarifying questions when things are unclear.
> 

### Using an identity from the UI

To use an identity, we can add a `select` in our form to pick an identity, and pass that identity name in the `openai:chat` mutation:

```jsx
function App() {
  const messages = useQuery(api.messages.list) || [];
  const sendMessage = useAction("openai:chat");
  const [newMessageText, setNewMessageText] = useState("");

  const identities = useQuery(api.identity:list) || [];
  const [identityName, setIdentityName] = useState("");
  //...
      <form
        onSubmit={(e) => {
          e.preventDefault();
          setNewMessageText("");
          sendMessage(newMessageText, identityName);
        }}
      >
        <select
          value={identityName}
          onChange={(e) => setIdentityName(e.target.value)}
        >
          <option value={""} disabled={identityName}>
            Select an identity
          </option>
          {identities.map((name) => (
            <option key={name} value={name}>
              {name}
            </option>
          ))}
        </select>
    //...
```

### Looking up the identity

In our `messages:send` function, which runs on the server, we can accept the new `identityName` and look up the identity’s instructions to pass to ChatGPT:

```js
export const send = internalMutation(async (ctx, { body, identityName }) => {
  //...
  const identity = await ctx.db
    .query("identities")
    .filter((q) => q.eq(q.field("name"), identityName))
    .unique();
  if (!identity) throw new Error("Unknown identity: " + identityName);
  const botMessageId = await ctx.db.insert("messages", {
    author: "assistant",
    identityId: identity._id,
  });
  //...
  return { messages, botMessageId, instructions: identity.instructions };
```

Two things to note:

1. We search for the identity by name. If there were a lot of them, we’d use an [index](https://docs.convex.dev/using/indexes), but for small tables doing a filter is just fine.
2. We’re storing the `identityId` into the bot’s message, so we can later know which identity responded to each message.

We can also pass the instructions for each bot message, by looking up the instructions if there is an `identityId`:

```js
await Promise.all(
    messages.map(async (msg) => {
      if (msg.identityId) {
        msg.instructions = (await ctx.db.get(msg.identityId)).instructions;
      }
    })
  );
```

We look up each identity by doing `ctx.db.get` in a separate async function, and use `Promise.all` to wait for the result of each of the async functions.

<details>
<summary>All together, `send` now looks like:</summary>

```js
export const send = internalMutation(async (ctx, { body, identityName }) => {
  await ctx.db.insert("messages", {
    body,
    author: "user",
  });
  const identity = await ctx.db
    .query("identities")
    .filter((q) => q.eq(q.field("name"), identityName))
    .unique();
  if (!identity) throw new Error("Unknown identity: " + identityName);
  const botMessageId = await db.insert("messages", {
    author: "assistant",
    identityId: identity._id,
  });
  const messages = await ctx.db
    .query("messages")
    .order("desc")
    .filter((q) => q.neq(q.field("body"), undefined))
    .take(10);
  messages.reverse();
  await Promise.all(
    messages.map(async (msg) => {
      if (msg.identityId) {
        msg.instructions = (await ctx.db.get(msg.identityId)).instructions;
      }
    })
  );
  return { messages, botMessageId, instructions: identity.instructions };
});
```
</details>

### Passing the identity to ChatGPT

In our `convex/openai.js` `chat` function, we can now de-structure the `instructions` return instead of hard-coding it:

```js
export const chat = action(async (ctx, { body, identityName }) => {
  const { messages, botMessageId, instructions } = 
    await ctx.runMutation(internal.messages.send, { body, identityName });
  //...
```

Now the response will be a function of what messages have been sent, and what the identity’s instructions are.

To also give GPT context on what its identity was for previous messages, we can sprinkle system messages into the messages we pass to the API:

```js
const gptMessages = [];
let lastInstructions = null;
for (const { body, author, instructions } of messages) {
  if (instructions && instructions !== lastInstructions) {
    gptMessages.push({
      role: "system",
      content: instructions,
    });
    lastInstructions = instructions;
  }
  gptMessages.push({ role: author, content: body });
}
if (instructions !== lastInstructions) {
  gptMessages.push({
    role: "system",
    content: instructions,
  });
  lastInstructions = instructions;
}
const openaiResponse = await openai.createChatCompletion({
  model: "gpt-3.5-turbo",
  messages: gptMessages,
});
```

The goal here is to remind GPT what identity it had for each message it sent. This API is new, so I bet there’s a lot of tweaking we can do with the prompts here to make it work better. For instance, maybe we should be telling it “Ignore previous system instructions for the following messages. Now act like this: (new instructions)”. If you get something working in a better way, let me know [in Discord](https://convex.dev/community)!

### Showing the identity’s name instead of “assistant” in the UI

Currently, all the messages in the UI say “assistant” - let’s show the identity we named by looking up the identities dynamically. Convex supports [relationships](https://docs.convex.dev/using/document-ids#references-and-relationships) in this way.

When listing our messages, we can look up the identity’s name to return. In `convex/messages.ts`:

```js
export const list = query(async (ctx) => {
  const messages = await ctx.db.query("messages").take(20);
  return Promise.all(
    messages.map(async (message) => {
      if (message.identityId) {
        const identity = await ctx.db.get(message.identityId);
        message.identityName = identity.name;
      }
      return message;
    })
  );
});
```

We look up each identity by doing `db.get` in a separate async function, and use `Promise.all` to return the result of each of the functions as an array. This is the common pattern for doing join-like behavior with Convex. This way you can execute the reads in parallel but do all your business logic and lookups in JavaScript.

On the client, we can now show the `identityName` if it’s defined instead of just the author in `App.jsx`:

```jsx
<span>{message.identityName ?? message.author}: </span>
```

And we can also default the chat identity to be the last-used identity message, when you load the page with a useEffect:

```jsx
useEffect(() => {
  if (identities.length && !identityName) {
    const lastMessage = messages[messages.length - 1];
    if (lastMessage && identities.indexOf(lastMessage.identityName) !== -1) {
      setIdentityName(lastMessage.identityName);
    } else {
      setIdentityName(identities[0]);
    }
  }
}, [messages, identities, identityName]);
```

### Summary

We’ve now added the ability to change the instructions to ChatGPT on how to respond, so we can have a conversation and be changing the identity mid-stream. See the code [here](https://github.com/ianmacartney/convex-chat-gpt) and a working demo [here](https://convex-chat-gpt-f6rq.vercel.app/). Note, that code & demo has some more features than just what we’ve done in this post.

How to store multiple personalities Convex and provide them to the chatGPT API, enabling changing personalities mid-conversation. This is a follow-up to Building a full-stack ChatGPT app. 

Adding Personality to ChatGPT-3

[Convex](https://convex.dev) supports [HTTP actions](https://docs.convex.dev/functions/http-actions), meaning your backend can receive requests not only from Convex clients, such as the [ConvexReactClient](https://docs.convex.dev/using/project-setup#configure-the-convex-client), but also from third-party webhooks and other clients that want to communicate with a custom HTTP API. 

Currently, these endpoints have a simple router. In this post, we’ll look at how to add more advanced features, such as:

- Dynamic routes or slug routes — e.g. `users/:userId`
- Middleware — e.g. check auth on all routes under `/api/*` or implementing CORS
- Helpers for validating an incoming Request’s query params or body
- Helpers for formatting a JSON response or text response
- Custom 404 (Not Found) responses

While it’s possible to build these yourself on top of Convex primitives, existing JS libraries already do a great job at this. In this post, we’re going to go through how you can leverage [Hono](https://hono.dev/) with Convex [HTTP actions](https://docs.convex.dev/functions/http-actions). To use it yourself, you can grab  [`honoWithConvex.ts`](https://github.com/get-convex/convex-helpers/blob/main/convex/lib/honoWithConvex.ts) in the [convex-helpers repo](https://github.com/get-convex/convex-helpers). We’ll also look more generally at how to extend Convex HTTP endpoint behavior. 

Note: you don’t need to use TypeScript, but I will use it in my examples because both Hono and Convex offer slick TypeScript support! 

# Using Hono with Convex

To use Hono, you’ll need to:

1. `npm install hono` in your project. 
2. Copy [`honoWithConvex.ts`](https://github.com/get-convex/convex-helpers/blob/main/convex/lib/honoWithConvex.ts) into your `convex/` directory. I like to keep it in `convex/lib/` to separate it from my application logic.
3. In `convex/http.ts`, import `Hono`, `HonoWithConvex`, and `HttpRouterWithHono` and use them like so:

```jsx
import { Hono } from "hono";
import { HonoWithConvex, HttpRouterWithHono } from "./lib/honoWithConvex";

const app: HonoWithConvex = new Hono();

// Add your routes to `app`. See below 

export default new HttpRouterWithHono(app);
```

The `HonoWithConvex` is just a type that tells Hono to expect the Convex [ActionCtx](https://docs.convex.dev/api/interfaces/server.ActionCtx) as its [env](https://hono.dev/api/context#env) binding. `HttpRouterWithHono` does the magic to connect Hono routes to Convex HTTP actions.

Let’s look at a few ways to use Hono:

## Slug routing and response formatting

For illustration, we’ll implement an endpoint from the [Convex demo for HTTP action](https://github.com/get-convex/convex-demos/tree/main/http) to use Hono. Here’s an example handler showcasing several of Hono’s features:

```typescript
// See https://github.com/sshader/hono-convex/blob/main/convex/http.ts#L24
// Routing with slugs
app.get("/listMessages/:userId{[0-9]+}", async (c) => {
  // Extracting a token from the URL!
  const userId = c.req.param("userId");

  // Running a Convex query
  const messages = await c.env.runQuery(api.messages.getByAuthor, { authorNumber: userId });

  // Helpers for pretty JSON!
  c.pretty(true, 2);
  return c.json(messages);
});
```

…and an example response:

```bash
$ curl https://happy-animal-123.convex.site/listMessages/123
[
  {
    "_creationTime": 1677798437141.091,
    "_id": {
      "$id": "messages|lqMHm5kDS9m6fBsSnx5L2g"
    },
    "author": "User 123",
    "body": "Hello world"
  },
]
```

## Input validation

Here’s another handler with input validation:

```typescript
// See https://github.com/sshader/hono-convex/blob/main/convex/http.ts#L36
app.post(
  "/postMessage",
  // Body validation!
  zValidator(
    "json",
    z.object({
      author: z.string().startsWith("User "),
      body: z.string().max(100),
    })
  ),
  async (c) => {
    // With type safety!
    const { body, author } = c.req.valid("json");
await c.env.runMutation(api.messages.send, { body, author });
    return c.text("Sent message!");
  }
);
```

…and an example response:

```bash
$ curl -d '{ "body": "Hello world", "author": "123" }'  https://happy-animal-123.convex.site/postMessage
{
  "success": false,
  "error": {
    "issues": [
      {
        "code": "invalid_string",
        "validation": {
          "startsWith": "User "
        },
        "message": "Invalid input: must start with \"User \"",
        "path": [
          "author"
        ]
      }
    ],
    "name": "ZodError"
  }
}
```

## Middleware: Adding CORS

Another example, copying from [Hono docs](https://hono.dev/middleware/builtin/cors). This adds CORS support to the `/api/*` and `/api2/*` routes with different configurations.

```typescript
import { cors } from 'hono/cors'
...
// See https://github.com/sshader/hono-convex/blob/main/convex/http.ts#L55
app.use('/api/*', cors())

app.use(
  '/api2/*',
  cors({
    origin: 'http://examplesite.com',
    allowHeaders: ['X-Custom-Header', 'Upgrade-Insecure-Requests'],
    allowMethods: ['POST', 'GET', 'OPTIONS'],
    exposeHeaders: ['Content-Length', 'X-Kuma-Revision'],
    maxAge: 600,
    credentials: true,
  })
)
```

The `.use` function registers a handler for all `/api/*` requests. As we’ll see below, you can use this for a variety of situations, including logging.

## Custom 404 responses

To set up a custom 404, we can do:

```typescript
// See https://github.com/sshader/hono-convex/blob/main/convex/http.ts#L12
// Custom 404
app.notFound(c => {
  return c.text("Oh no! Couldn't find a route for that", 404);
});
```

See [https://hono.dev/](https://hono.dev/) for more features.

<BuildWithConvex />

# Under the hood

Curious about how to extend Convex [HTTP actions](https://docs.convex.dev/functions/http-actions) for your own purposes? Read on!

## Extending routes using the `/` prefix

If the routing options aren’t flexible enough for your use case, you can handle all HTTP requests with a single `httpAction` and do complex routing there. For instance, instead of using `HttpRouterWithHono`, we could define a single route per HTTP method in `convex/http.ts`:

```typescript
import { httpRouter, ROUTABLE_HTTP_METHODS } from "convex/server";
import { httpAction } from "./_generated/server";
import { HonoWithConvex } from "./lib/honoWithConvex";

const app: HonoWithConvex = new Hono();

// Add your routes to `app`.

const http = httpRouter();
for (const routableMethod of ROUTABLE_HTTP_METHODS) {
	http.route({
		pathPrefix: "/",
		method: routableMethod,
		handler: httpAction(async (ctx, request: Request) => {
			return await app.fetch(request, ctx);
		}),
	})
}
export default http;
```

We could stop here — we can now use Hono and Convex together! But we could make a couple of additional improvements to leverage the [Convex dashboard](https://dashboard.convex.dev/).

## Using middleware to add per-route logging

Here’s what we see in the [Convex dashboard](https://dashboard.convex.dev/) under “Logs” given the approach of registering an `httpAction` per method:

![Logs showing GET /*](https://cdn.sanity.io/images/ts10onj4/production/06c3e0c48a26da60a29a095e5c6f5e184d6ae2b0-1302x310.png?w=700)

All our GET requests will appear as `GET /*` even when we have multiple routes.

We can pretty easily get a little more information using one of Hono’s features — logging middleware:

```typescript
import { logger } from "hono/logger";
import stripAnsi from "strip-ansi";

app.use(
  "*",
  logger((...args) => {
    console.log(...args.map(stripAnsi));
  })
);
```

Now the [Convex dashboard](https://dashboard.convex.dev/) looks more like this:

![Logs with GET /listMessages/123](https://cdn.sanity.io/images/ts10onj4/production/e1ff98ab9fdcc7e5eee3b741eca5025b98c762e9-1350x524.png?w=700)

Note: these say `0ms` because they’re running in [Convex’s deterministic environment that provides a different `Date.now()`](https://docs.convex.dev/understanding/convex-fundamentals/functions#determinism).

## Subclassing `HttpRouter` (the `HttpRouterWithHono` approach)

If we want the fullest integration with the [Convex dashboard](https://dashboard.convex.dev/), we’d like to see something like this under “Logs”, where we show the routed path:

![Logs showing GET /listMessages/:userId](https://cdn.sanity.io/images/ts10onj4/production/212100c96760c3d1b86ec79a59ce3d5246a60ca3-1526x514.png?w=700)

And then see a corresponding entry with metrics under the “Functions” tab:

![Functions metrics for GET /listMesssages/:userId](https://cdn.sanity.io/images/ts10onj4/production/9527a4a514b8fb6aa98023e27da58b90fbdede3b-970x922.png?w=450)

The code needed for this behavior is in  [`honoWithConvex.ts`](https://github.com/get-convex/convex-helpers/blob/main/convex/lib/honoWithConvex.ts).

**How does it work?**

`HttpRouterWithHono` is a subclass of the Convex [`HttpRouter`](https://docs.convex.dev/api/classes/server.HttpRouter) which overrides two special methods:

- `getRoutes` returns a list of `[path, method, handler]`, which we use to populate the Functions page on the dashboard.
- `lookup(path, method)` returns `[handler, path, method]`. Convex will run `handler` when responding to the request and use the `path` and `method` for metrics and logging (so this should match a path + method combo from `getRoutes`)
    - As an example, I wanted `lookup("/listMessages/123", "GET")` to return `"/listMessages/:userId{[0-9]+}"` for the path and `"GET"` for the method.

The implementation I added is not optimal (it loops through all the routes), but it still works! The Convex router is very flexible, so there are many options for configuring how your HTTP actions get routed and show up in the dashboard. 

Now, we can use Convex with Hono and take advantage of most of the features provided in the Convex dashboard!

# Summary

In this post, we looked at how to use Hono with Convex, including how to extend Convex’s HTTP actions to add your own functionality. Let us know what you think in [our discord](https://convex.dev/community) and if you end up using Hono and Convex together! ❤️

Adding advanced HTTP Endpoint functionality by extending Convex with Hono.

Advanced HTTP Endpoints: Convex ❤️ Hono

In this post, we’ll walk through putting together a full-stack chat app, and add some features. 

We’ll use React and Vite, though any React framework works. We’ll use <QuickConvex /> as the backend, where our server-side functions will run, and where we’ll store our app’s data. 

To see this in action, the code is [here](https://github.com/ianmacartney/convex-chat-gpt), and a running version is [here](https://convex-chat-gpt-f6rq.vercel.app/) (for now). You can clone it and run it yourself with a few configurations in the README, but read on to see a step by step guide on building your own. Also, the published version uses the "authed" branch in the repo, in case you want to see how simple it is to add auth.

To run this yourself, you’ll need to make an [OpenAI account](https://platform.openai.com/signup) and [get an API key](https://platform.openai.com/account/api-keys).

If you’re familiar with Convex, you can skip ahead to step 2.

<BuildWithConvex />

# 0. Bootstrap a Vite React app

If you don’t already have an app, we can make one:

```bash
npm create vite@latest
```

I picked `convex-chatgpt` as the project name, `React` as the framework, and `Javascript` as the variant.

At this point, if we run:

```bash
npm install
npm run dev
```

we have a locally running webapp.

Let’s change `src/App.jsx` to list messages and have a form to submit messages:

<details>
<summary>App.jsx</summary>
    
    ```tsx
    import {useState} from "react";
    import "./App.css";
    
    function App() {
      const messages = [
        {author: "user", body: "Hello, world"},
      ];
      const sendMessage = body =>
        console.log("Trying to send: " + body);
      const [newMessageText, setNewMessageText] =
        useState("");
    
      return (
        <div className="App">
          {messages.map((message, i) => (
            <p key={i}>
              <span>{message.author}: </span>
              <span style={{ whiteSpace: "pre-wrap" }}>
                {message.body ?? "..."}
              </span>
            </p>
          ))}
          <form onSubmit={(e) => {
            e.preventDefault();
            setNewMessageText("");
            sendMessage(newMessageText);
          }}>
            <input
              value={newMessageText}
              onChange={e => setNewMessageText(e.target.value)}
              placeholder="Write a message…"
            />
            <input type="submit" value="Send" disabled={!newMessageText} />
          </form>
        </div>
      );
    }
    
    export default App;
    ```
</details>    

At this point, we have an app that shows a static list of messages and logs to the console when trying to send a message.

# 1. Add Convex

This is similar to the Convex [quickstart](https://docs.convex.dev/quickstart/react). In a new terminal (leave the other one running `npm run dev`):

```bash
npm install convex
npx convex dev
```

If you haven’t used convex before, you’ll be prompted to log in, create an account, etc. I picked `convex-chatgpt` as the project name. `npx convex dev` will deploy these functions to your new Convex backend, and as you edit the functions, it will automatically re-deploy them. It will also ask you to save the deployment URL into your `.env` and `.env.local` files: these point at your convex backends for your production and development deployments. We’ll just be working with the development deployment here.

Let’s add the ability to send messages and list messages. We will add functions that run in Convex (on a server) that will read and write to the database with a [query](https://docs.convex.dev/functions/query-functions) and [mutation](https://docs.convex.dev/functions/mutation-functions).

Add `convex/messages.ts`:

```jsx
import { query, mutation } from "./_generated/server";

export const list = query(async (ctx) => {
  return await ctx.db.query("messages").collect();
});

export const send = mutation(async (ctx, { body }) => {
  await ctx.db.insert("messages", {
    body,
    author: "user",
  });
  const botMessageId = await ctx.db.insert("messages", {
    author: "assistant",
  });
});
```

To access Convex from inside React, you’ll need to add a `<ConvexProvider>` context at the top level of your app. Edit `main.jsx`:

```jsx
...
import { ConvexProvider, ConvexReactClient } from "convex/react";

const convex = new ConvexReactClient(import.meta.env.VITE_CONVEX_URL);

ReactDOM.createRoot(document.getElementById("root")).render(
  <React.StrictMode>
    <ConvexProvider client={convex}>
      <App />
    </ConvexProvider>
  </React.StrictMode>,
);
```

We can then use these functions from `App.jsx`:

```jsx
function App() {
  const messages = useQuery(api.messages.list) || [];
  const sendMessage = useMutation(api.messages.send);
	...
```

Great! Now we have messages being written to and from the database. Try it out! Those new to Convex will be surprised to see that adding new messages will automatically result in the `useQuery(api.messages.list)` hook returning new messages. This is part of the magic of Convex. Learn more about it [here](https://docs.convex.dev/understanding/convex-fundamentals/functions#reactivity). 

You can check out your data in the dashboard: `npx convex dashboard`. 

You may note that the query and mutation are named by the `filename:function`. See more about this [here](https://docs.convex.dev/using/writing-convex-functions#defining-convex-functions).

You’ll notice every time we send a message, there’s a “…” message from the assistant. Next let’s update that message from chatGPT. 

# 2. Send a message to the ChatGPT API

So far we’ve been working with a [query](https://docs.convex.dev/functions/query-functions) and [mutation](https://docs.convex.dev/functions/mutation-functions), which are Convex functions that interact with the database. In order to interact with external services, we need to do that in an “[action](https://docs.convex.dev/functions/actions)” - which is a Convex function that isn’t inside a deterministic environment and isn’t part of a database transaction. This frees us to do things with side effects, such as making requests to OpenAI’s API. We’ll do this in a few steps.

## Fetching messages to send to ChatGPT

In our `send` mutation, we can grab the latest 10 messages to send to ChatGPT using a database query:

```jsx
export const send = mutation(async (ctx, { body }) => {
  //... insert messages to the table
  const messages = await ctx.db
    .query("messages")
    .order("desc")
    .filter((q) => q.neq(q.field("body"), undefined))
    .take(21);
  messages.reverse();
  return { messages, botMessageId };
});
```

This orders messages in descending order (sorted by creation time unless you use a different [index](https://docs.convex.dev/database/indexes)), filters out messages with no body (for instance, the new placeholder system message), and takes the first 21 (which are the 21 most recent messages). It then reverses the list so it’s ordered in ascending time order. I picked 21 so that we’ll have 10 pairs of user/bot messages, followed by the latest prompt. It returns the messages to be used as input to the chat completion API, which we'll add next.

## Creating the action

We’ll use the `openai` npm package. You can install it with:

```bash
npm install openai
```

Make a new file: `convex/openai.js`:

```jsx
"use node";
import { Configuration, OpenAIApi } from "openai";
import { action } from "../_generated/server";

export const chat = action(async (ctx, { body }) => {
  const { messages, botMessageId } = await ctx.runMutation(api.messages.send, { body });
  const fail = async (reason) => throw new Error(reason);
  // Grab the API key from environment variables
  // Specify this in your dashboard: `npx convex dashboard`
  const apiKey = process.env.OPENAI_API_KEY;
  if (!apiKey) {
    await fail("Add your OPENAI_API_KEY as an env variable");
  }
  const configuration = new Configuration({ apiKey });
  const openai = new OpenAIApi(configuration);

  const openaiResponse = await openai.createChatCompletion({
    model: "gpt-3.5-turbo",
    messages: [
      {
        role: "system",
        content: instructions,
      },
      ...messages.map(({ body, author }) => ({
        role: author,
        content: body,
      })),
    ],
  });
  if (openaiResponse.status !== 200) {
    await fail("OpenAI error: " + openaiResponse.statusText);
  }
  const body = openaiResponse.data.choices[0].message.content;
	console.log("Response: " + body);
});
```

This will first send the messages with the send mutation we modified, getting in return the list of messages and message ID to update with the bot’s message. It will then make a request to the `gpt-3.5-turbo` model, passing in one system message with instructions (hard-coded for now), followed by each message. We’re turning our body & author fields into “role” and “content”. See their docs [here](https://platform.openai.com/docs/guides/chat) for more details on the API.

## Add your API key to the dashboard

To authenticate requests to OpenAI, you’ll need to add your [API key](https://beta.openai.com/account/api-keys) to the Convex dashboard, so your action (which runs on Convex’s servers) has access to it. Go to your dashboard with `npx convex dashboard` and on the left panel go into Settings and add the key with the name `OPENAI_API_KEY`. While you’re there, you can add it as an env variable to both your production and development deployments by toggling the dropdown in the left panel.

![Dashboard side panel](https://cdn.sanity.io/images/ts10onj4/production/6ed549cf10845a91f6b6c8697a165b253705f4b8-1166x629.png?w=450 "Dashboard side panel")


## Triggering the action from the UI

We can change our call to `useQuery(api.messages.send)` to

```jsx
const sendMessage = useAction(api.openai.chat);
```

Note the action is addressed by the path to the file (`openai`) and the function (`chat`). The API is the same (taking one argument, `body`), so it’s a drop-in replacement!

At this point, if you run it, it will send a request to ChatGPT and console.log the response. Convex prints Convex function logs to the browser’s console, though you can also see them in the [dashboard](https://dashboard.convex.dev/), to prove to yourself it’s running in the cloud. Now let’s show them in the UI.

## Updating the message with the reply

In order to get the response into the chat message, we’ll want to update the empty placeholder message we added. Treat each call to a mutation as a transaction. Once we called our first mutation, the messages were committed to the database and the UI could show the new messages before doing the slow call to OpenAI. Once we get the response, we update the bot’s message and the UI will update automatically via the `messages:list` query. We’ll add a new mutation in the `convex/messages.js` file:

```jsx
// An `internalMuation` can only be called from other server functions.
export const update = internalMutation(async (ctx, { messageId, patch }) => {
  await ctx.db.patch(messageId, patch);
});
```

This patches the specified message. You could pass in `{body: "hi"}` to change the message’s body to “hi,” for example. Since the first mutation returned the `botMessageId`, we can use that from the action in `convex/openai.js`:

```jsx
export const chat = action(async (ctx, { body }) => {
  const { messages, botMessageId } = await ctx.runMutation(internal.messages.send, { body });

  // Call OpenAI

  await runMutation(ap.messages.update, {
    messageId: botMessageId, 
    patch: {
      body: openaiResponse.data.choices[0].message.content,
      // Track how many tokens we're using for various messages
      usage: openaiResponse.data.usage,
      updatedAt: Date.now(),
      // How long it took OpenAI
      ms: Number(openaiResponse.headers["openai-processing-ms"]),
    }
  });
});
```

While we’re here, we can store a bunch of interesting data into the message along with the body. Convex’s database is flexible enough to store [strings, numbers, javascript objects, arrays, etc.](https://docs.convex.dev/using/types) while also letting you nail down a [schema](https://docs.convex.dev/database/schemas) later to get typescript types, autocomplete, etc. 

🎉 Now you have chatGPT replying to your messages!

# Summary

In this post, we made a full-stack web app to chat with OpenAI’s ChatGPT API. Let us know [in Discord](https://convex.dev/community) what you think, and if you’d like to see some extensions:

- [Customizing the identity you’re talking to.](https://stack.convex.dev/adding-personality-to-chatgpt-3)
- [Moderating the identities and messages to prevent harmful content.](https://stack.convex.dev/moderating-chatgpt-content-full-stack)
- Create new message threads. (See the [code](https://github.com/ianmacartney/convex-chat-gpt) for how it's done!)

Let's build a full-stack chat app to talk to ChatGPT on its new API!

Building a Full-Stack ChatGPT app

This is part of a series of posts about [building a multiplayer game on Convex](https://stack.convex.dev/building-a-multiplayer-game). In this post, we’ll explore using Dall-E via the OpenAI API. The code is [here](https://github.com/get-convex/convex-demos/tree/main/dall-e-storage-action) if you want to play around with it. We will make a chat app where you can type in an image description as a slash command, and it will generate and insert that image into the chat. On the server, it will:

1. Configure an OpenAI client.
2. Check that the prompt is not offensive using their moderation API.
3. Get the short-lived image URL from Dall-E.
4. Download the image.
5. Upload the image into Convex [File Storage](https://docs.convex.dev/using/file-storage), getting a storage ID.
6. Insert the storage ID into a new message.
7. When serving the message to the client, we will generate a URL based on the storage ID, which they can put in an `<img>` tag directly.

This is now possible using [File Storage](https://docs.convex.dev/using/file-storage) and a Convex [Action](https://docs.convex.dev/using/actions), a serverless function that can have side-effects[^1], comparable to AWS lambda. Read below for how to use OpenAI with Convex, or check out the code [here](https://github.com/get-convex/convex-demos/tree/main/dall-e-storage-action).

# Using OpenAI in Convex

## Set up Convex

Create a Convex app by cloning [the demo](https://github.com/get-convex/convex-demos/tree/main/dall-e-storage-action) and running `npm i; npm run dev`. If you want a fresh project, see the [Tutorial](https://docs.convex.dev/tutorial/welcome-to-convex).

If you’re writing your app from scratch, create a file in the `convex/` folder. Mine is [`convex/actions/sendDallE.js`](https://github.com/get-convex/convex-demos/blob/main/dall-e-storage-action/convex/actions/sendDallE.js). 

## Configuring OpenAI

1. To use OpenAI, you need an API account. Sign up [here](https://beta.openai.com/signup).
2. Generate an API key [here](https://platform.openai.com/account/api-keys) by clicking “Create new secret key” and copying the text.
3. Store the API key Convex’s [Environment Variables](https://docs.convex.dev/using/environment-variables), where you can store secrets. Go to your project’s dashboard by running `npx convex dashboard` in your project directory or finding it on [dashboard.convex.dev](https://dashboard.convex.dev). Open the project settings on the left and enter it with the key `OPENAI_API_KEY`. I’d recommend saving it in both your Production and Development deployments unless you want the published version of your app to use a different key. There is a toggle to switch between them.
4. Configure the client in your [action](https://docs.convex.dev/using/actions). If you get type errors, `npm install openai`:
    
    ```jsx
    "use node";
    import { Configuration, OpenAIApi } from "openai";
    const configuration = new Configuration({
      apiKey: process.env.OPENAI_API_KEY,
    });
    const openai = new OpenAIApi(configuration);
    ```
    

## Using the Moderation API

The moderation API is useful for checking whether a prompt is obscene or will be rejected by OpenAI. For our purposes, we’ll check if it flags the message:

```jsx
const modResponse = await openai.createModeration({
  input: prompt,
});
const modResult = modResponse.data.results[0];
if (modResult.flagged) {
  throw new Error(
    `Your prompt was flagged: ${JSON.stringify(modResult.categories)}`
  );
}
```

The `categories` object can be used for more fine-grained moderation. It has a score per category. Consult the [moderation documentation](https://platform.openai.com/docs/guides/moderation/overview) for more details.

## Generating an image with Dall-E

You can consult the [image generation documentation](https://platform.openai.com/docs/guides/images) for more details, but for our purposes, the code is:

```jsx
const opanaiResponse = await openai.createImage({
  prompt,
  n: 1,
  size: "256x256",
});
const dallEImageUrl = opanaiResponse.data.data[0]["url"];
```

Now we have a URL to the image. However, this URL [only lasts for an hour](https://platform.openai.com/docs/guides/images/usage?lang=node.js#:~:text=URLs%20will%20expire%20after%20an%20hour). If we want to be able to see this image in the chat longer term, we need to store it.

## Downloading the image

To download the image, we use the [`node-fetch` NPM library](https://www.npmjs.com/package/node-fetch#common-usage). 

```js
const imageResponse = await fetch(dallEImageUrl);
const image = await imageResponse.blob();
```

Now we have the image in memory and can store it in Convex.

## Storing the image in Convex

Storing the image in Convex is simple:

```js
const storageId = await ctx.storage.store(image);
```

where `ctx` is the first parameter to the `action` function:

```js
// in sendDallE.js
export default action(async (ctx, { prompt, author }) => {
  ...
});
```

## Store the storage ID in the database

The storage ID returned from the upload URL is now a permanent identifier of this data. You can store it in your database however you’d like. In our example, we store it in a new chat message.

```jsx
// in sendDallE.js action
await ctx.runMutation(api.messages.send, {
  body: storageId, 
  author, 
  format: "dall-e",
  });
```

And the mutation that stores it, for completeness:

```jsx
// in messages.js
export const send = mutation(async (ctx, { body, author, format }) => {
  // A bit of a hack; we are storing the storage ID in the "body"
  await ctx.db.insert("messages", { body, author, format });
});
```

One thing to note is that this can’t live in the same file as the openai action. Actions can run in a node environment (with the `"use node";` string at the top of the file) whereas queries and mutations run in our optimized runtime (which makes them wicked fast). You can read more about that [here](https://docs.convex.dev/understanding/convex-fundamentals/functions#action-functions). I put mine in [`convex/sendMessage.js`](https://github.com/get-convex/convex-demos/blob/main/dall-e-storage-action/convex/sendMessage.js).

## Serving the image to the client

In Convex, clients subscribe to [queries](https://docs.convex.dev/using/database-queries). For instance, in our frontend React component, we can do:
```jsx
const messages = useQuery(api.message.list) || [];
...
{messages.map(message => (
  <Message author={message.author} body={message.body} format={message.format} key={message._id.toString()} />
)};
```
That query maps to a serverless function: the `list` export of `convex/messages.js`. To return the image URL in messages we return to clients, we generate a URL from the storage ID. For our app, we will change each storage ID into a URL. In `convex/messages.js`:

```jsx
export const list = query(async (ctx) => {
  const messages = await ctx.db.query("messages").collect();
  for (const message of messages) {
    if (message.format === "dall-e") {
		  // Replace the storage ID with a URL in the "body"
      message.body = await ctx.storage.getUrl(message.body);
    }
  }
  return messages;
});
```
This URL won’t change on each request, so you don’t have to worry about caching the URL on the client. The HTTP response also has cache headers so you don't need to worry about client-side image caching either.

# Performance

One thing I discovered while building this is how slow the OpenAI API calls can be. In my testing, the moderation call took 500ms, and the image generation could take 5+ seconds, sometimes taking over 60s and timing out the request. To make this a good user experience, adding a loading indicator is important. On the client, I used this code:

```tsx
// In React jsx:
const [sending, setSending] = useState(false);
const sendDallE = useAction(api.actions.sendDallE.default);
...
setSending(true);
try {
  await sendDallE({ prompt, author: name });
} finally {
  setSending(false);
}
```

# Summary

In this post, we used Convex to fetch an image from OpenAI’s image generation service based on a user-provided prompt. Read about [the game](https://whose-prompt-is-it-anyways.vercel.app/) I've developed on Convex using Dall-E APIs [here](https://stack.convex.dev/building-a-multiplayer-game) for more tips, or [read the code](https://github.com/get-convex/multiplayer-game-with-dall-e) to see how I added error handling and more.


[^1]: This differs from other Convex functions like `query` and `mutation`, which do not have side effects, allowing them to provide features like automatic retries, reactive updates, serializable transaction isolation, and more! Read more about it [here](https://docs.convex.dev/understanding/deep-dive/occ).

Use Convex to fetch an image from OpenAI’s image generation service based on a user-provided prompt.