# Convex Ents: Manage your document relationships

Note: This article assumes some familiarity with Convex. If you’re not familiar with it, check out the [Convex tutorial](https://docs.convex.dev/get-started).

[Convex Ents](https://labs.convex.dev/convex-ents) is a library for Convex providing a bunch of useful functionality:

1. Simpler ways to model and query related documents
2. Ability to easily map and filter documents retrieved from the database
3. Enforcing unique document field values
4. Defining default values for easier document shape evolution
5. Propagating deletion to related documents
6. Soft and scheduled document deletion
7. And more

While all of these can be achieved without Convex Ents, the library makes them really easy. If you’re familiar with Prisma or Drizzle ORM, you’ll find yourself at home. Let’s look at each item on the list in more detail.

## Simpler ways to model and query related documents

You can store IDs of other documents in Convex documents, just like in any other relational database. These can represent 1:1 and 1:many relationships between documents, which in the Ents parlance are called “edges”:

In vanilla Convex:

```jsx
// schema.ts
users: defineTable({
  name: v.string(),
}),
messages: defineTable({
  text: v.string(),
  userId: v.id("users")
})
  .index("userId", ["userId"])

// myFunctions.ts
// args: userId
const messages = await ctx.db
  .query("messages")
  .withIndex("userId", (q) => q.eq("userId", userId))
  .collect();
```

In this example we have two tables, users and messages, and messages have a required `userId` field. We also defined an index on this field, so that we can efficiently retrieve just the messages related to a given userId. Which is exactly what we did in the example query.

Now let’s look at the equivalent with Convex ents:

```jsx
// schema.ts
users: defineEnt({
  name: v.string(),
})
  .edges("messages", { ref: true }),
messages: defineEnt({
  text: v.string()
})
  .edge("user")

// myFunctions.ts
// args: userId
const messages = await ctx.table("users")
  .getX(userId)
  .edge("messages");
```

While there are a bunch of differences in the code between this version and the “vanilla” Convex code, the semantics are exactly the same.

First, we define two “ents” (short for “entity”): users and messages. The message ents are declared to have a unique `edge` to the users table. This translates to the exact same code you saw above: a `userId` field, and an associated index. Additionally, the user ents are declared to have 1:many `edges` to the messages table (`ref: true` means that the edge is stored as a “reference” in a field - the field name is inferred). This information doesn’t affect the Convex schema, but it allows you to query the relevant messages “from” the user ent.

And that’s exactly what we do in the example query. Instead of `ctx.db.query` we use `ctx.table`. We then ask for the ent with the given `userId` - but we don’t retrieve it. Instead we immediately ask to traverse the 1:many “messages” edge. This performs the same indexed retrieval as the vanilla code.

### Many to many relationships

So far we have saved a little bit of code, but Convex Ents shine even more when it comes to modeling many to many relationships. Let’s look at vanilla Convex example first:

```jsx
// schema.ts
roles: defineTable({
  name: v.string(),
}),
permissions: defineTable({
  name: v.string(),
})
roles_to_permissions: defineTable({
  rolesId: v.id("roles"),
  permissionsId: v.id("permissions")
})
  .index("rolesId", ["rolesId", "permissionsId"])
  .index("permissionsId", ["permissionsId"])

// myFunctions.ts
// args: roleId
const rolePermissions = await Promise.all(
  await ctx.db
    .query("roles_to_permissions")
    .withIndex("rolesId", (q) => q.eq("rolesId", roleId))
    .collect(),
  (doc) => ctx.db.get(doc.permissionId),
);
// args: roleId, permissionId
const hasPermission = (await ctx.db
  .query("roles_to_permissions")
  .withIndex("rolesId", (q) =>
    q.eq("rolesId", roleId).eq("permissionId", permissionId),
  )
  .first()) !== null;
```

To model a many to many relationship in a relational database, you usually define another table to store the relationship, like the `roles_to_permissions` table in this example. You need 2 indexes on it, one for each “foreign key”, so that you can efficiently retrieve related documents from either “side” of the relationship.

Then when you do this retrieval you have to first find the relevant documents representing the relationship, and then you have to map over them to retrieve the document from the other table, this is how we get `rolePermissions`.

In this example we also showcase how to use one of the indexes to answer the common question: “Does this document have given relationship with this other document?”, to get `hasPermission`.

Now let’s look at the equivalent with Convex ents:

```jsx
// schema.ts
roles: defineEnt({
  name: v.string(),
})
  .edges("permissions"),
permissions: defineEnt({
  name: v.string(),
})
  .edges("roles")

// myFunctions.ts
// args: roleId
const rolePermissions = await ctx.table("roles")
  .getX(roleId)
  .edge("permissions");
// args: roleId, permissionId
const hasPermission = await ctx.table("roles")
  .getX(roleId)
  .edge("permissions")
  .has(permissionId);
```

As before, this code is semantically equivalent to the vanilla Convex code, but is perhaps more clearly aligned with our intent 💡.

Let’s say that you also need to retrieve the role document itself in the previous example. This is easy with Ents:

```jsx
// myFunctions.ts
const role = await ctx.table("roles").getX(roleId)
const rolePermissions = await role.edge("permissions");
```

All we had to do is split the chained call and `await` the result of the `getX` (get or throw) method call.

This brings us to our second item:

## Ability to easily map and filter documents retrieved from the database

You’ve already seen that Convex Ents use chained method calls, similar to the built-in `ctx.db` API. Ents have one trick up their sleeve though: all methods are `await`-able. This makes the API even more fluent:

```jsx
// myFunctions.ts
const allUsers = await ctx.table("users");
const user = await ctx.table("users").getX(userId);
const messages = await ctx.table("users").getX(userId).edge("messages");
```

This is achieved via “lazy” `Promise`s. Unlike normal JavaScript Promises, which kick off work immediately when they’re created, the `ctx.table` method and methods chained to it return a lazy promise, which doesn’t perform any work until it is `await`ed.

This also allows ents to have extra helper methods which help with retrieving documents, performing “joins” and returning filtered data from Convex functions:

```jsx
return await ctx.table("users")
  .getX(userId)
  .edge("messages")
  .map((message) => {
    const attachments = await message.edges("attachments");
    return {
      _id: message._id,
      text: message.text,
      numAttachments: attachments.length,
    };
  });
```

There are two main things happening in this example, using the `map` method:

1. We query the related `attachments` for given message
2. We only return the fields we want to return to the client

This is totally possible with vanilla Convex, it’s just a bit more code:

```jsx
return await Promise.all(
  (
    await ctx.db
      .query("messages")
      .withIndex("userId", (q) => q.eq("userId", userId))
      .collect()
  ).map((message) => {
    const attachments = await ctx.db
      .query("attachments")
      .withIndex("messageId", (q) => q.eq("messageId", message._id))
      .collect();
    return {
      _id: message._id,
      text: message.text,
      numAttachments: attachments.length,
    };
  }),
);
```

We’ll pick up the pace and cover the next two points quickly:

## Unique field values

In databases fields there are often “unique” fields which serve as “secondary” keys by which documents can be retrieved. In Convex we can achieve this by:

1. Defining an index on the field
2. Ensuring that a document with a given value doesn’t already exist, anywhere we write given documents

```jsx
// schema.ts
users: defineTable({
  email: v.string(),
}),
  .index("email", ["email"])

// myFunctions.ts
// Before every insert, patch or replace using the `email` field:
const existing = await ctx.db
  .query("users")
  .withIndex("email", (q) => q.eq("email", email))
  .first();
if (existing !== null) {
  throw new Error(
    `In table "users" cannot create a duplicate document with field "email" of value \`${email}\`, existing document with ID "${
      existing._id as string
    }" already has it.`,
  );
}
```

Convex Ents have a built-in shortcut for this:

```jsx
// schema.ts
users: defineEnt({}),
  .field("email", { unique: true })

// myFunctions.ts
// The uniqueness check is performed automatically
```

No extra code is required when writing to the `users` table.

## Default field values

When you evolve your schema over time you’ll probably add more fields. But existing documents in the database won’t have any values for these fields yet. The easiest approach is to add an optional field:

```jsx
// schema.ts
posts: defineTable({
  // ... other fields
  contentType: v.optional(v.union(v.literal("text"), v.literal("video")))
}),
```

In this example we added a `contentType` field, and made it optional. Everywhere we read posts, we can manually include a default value, in vanilla Convex:

```jsx
// myFunctions.ts
return (await ctx.db.query("posts")).map((post) => ({
  ...post,
  contentType: post.contentType ?? "text",
}));
  
```

Usually you want to always specify the new field when writing the document. It’s not possible to automatically require this with the built-in schema validation, you have to make sure you write the value yourself.

If the default value is just a simple value like in this example, you can achieve this more easily with Convex Ents:

```jsx
// schema.ts
posts: defineEnt({
  // ... other fields
})
  .field(
    "contentType",
    v.union(v.literal("text"), v.literal("video")),
    { default: "text" }
  )

// myFunctions.ts
// The "contentType" is not optional, and defaults to "text"
return await ctx.table("posts");
```

Since `contentType` is not an optional field in the document type, TypeScript can ensure that you’re always providing it when writing to the database.

## Cascading deletes, soft deletion and scheduled deletion

In vanilla Convex, when a document is deleted other documents can still include “references” to it by storing the deleted document’s ID. This is a great, simple and scalable model. When querying the ID Convex will return null, and this can be handled (or ignored) by your code.

However, relationships are often required, and it can be easier to reason about your data model without “dangling references” in your documents. For this reason, Convex Ents do not support dangling references in the edges declared via `edge` and `edges`. Convex already makes this easy when writing data to the database, simply by declaring the field which stores the “foreign key” as NOT optional.

This makes deletion in general more challenging though. You can easily have a scenario where a document’s ID is stored in 1000s or even more other documents. Deleting all of these documents in a single mutation, which is within a single transaction, is simply impossible, as it would require a long-lived transaction, grinding the whole database to a halt (something Convex does not allow, instead failing the mutation).

Convex Ents include 3 deletion behaviors:

1. The default one deletes all related documents that require the existence of a given document - cascading deletions, in a single transaction. This is a fine behavior that preserves the “no dangling references” invariant, as long as you don’t expect to have many related documents.
2. The soft deletion behavior doesn’t actually delete the document, but instead sets a `deletionTime` field on it. It’s up to you to make sure that soft delete documents are not shown when they should not be. For example you might want to show the “group posts” of deleted users, because the posts really belong to the “group”, but you don’t show the user’s “profile”.
3. The scheduled deletion behavior combines the two: First it performs only soft deletion, and then, with an optional delay, performs the cascading delete, over possibly many scheduled mutations to make sure that each individual mutation doesn’t read or write too many documents. The deletion is performed depth first, so that no dangling references are created in the process.

Learn more about the different deletion behaviors in [Cascading Deletes documentation](https://labs.convex.dev/convex-ents/schema/deletes).

## Conclusion

We hope you find the library interesting, both for its own merits and as an example of an abstraction that can be built on top of the powerful Convex base. Notably, Ents is built entirely on top of vanilla Convex, and you can contribute to it or fork it to meet your own needs or preferred API ergonomics. The library is still in its early experimental stage, without the stability or quality guarantees built-in Convex provides. If it does seem promising to you, please give it a try and [let us know your feedback on Discord](https://convex.dev/community).

Check out these links to learn more:
- [Convex Ents docs](https://labs.convex.dev/convex-ents)
- [Convex Ents repo](https://github.com/xixixao/convex-ents)
- [Convex Ents issues](https://github.com/xixixao/convex-ents/issues)
- [Prisma vs Convex examples](https://labs.convex.dev/convex-vs-prisma)

Convex Ents is a library that provides: simpler ways to model and query related documents, ability to easily map and filter documents, enforcing unique field values, default field values, cascading deletes, soft deletion and more!

Convex Ents: Manage your document relationships

AICamp

Michal's talk starts at 13:06.

Relevant Stack posts that go over various methods of building AI-powered chat:
- https://stack.convex.dev/are-vector-databases-dead
- https://stack.convex.dev/ai-chat-with-convex-vector-search
- https://stack.convex.dev/ai-chat-using-langchain-and-convex
- https://stack.convex.dev/ai-chat-using-openai-assistants-api

At AICamp's end-of-year meetup for 2023, Michal goes over why you should build a custom AI-powered chat for your product, what options you have for building one, and which one you should ultimately choose.

Building Context Augmented AI Chat in Practice: AICamp 2023 EOY Meetup

Certain web applications can have a limit to the number of users they can satisfy at a time. To ensure that at least some users can be served, these apps need to protect themselves against surges in demand. A **waitlist** is a user-friendly way to achieve this. Read on to learn how you can add a waitlist to your app!

Pressed for time? Grab the [code on GitHub](https://github.com/get-convex/convex-waitlist).

## Run-through of the user experience

Let’s look at an example. We’re using the demo app functionality from `npm convex create` , which we’ve added a waitlist to. When there aren’t many users, your app should behave normally. 

![standard convex demo app](https://cdn.sanity.io/images/ts10onj4/production/50ce259354cdb7e882ddf665e8182d227bcb813b-1980x782.png)

Once there are too many users, we’ll let the newcomers know that they need to wait. They’ll be shown their estimated position in line and approximately how many people are waiting:

![ui when user is on the waitlist](https://cdn.sanity.io/images/ts10onj4/production/d453315540156b4f54a97a2a4f626df8a33353e5-3254x1140.png)

And once there is “space”, they are let in.

You can play with a live demo [here](https://get-convex.github.io/convex-waitlist/). It’s been configured to allow only 3 users at a time to have access. You can open multiple browser tabs to create multiple user sessions.

## Installation instructions

Can’t wait to get started? Follow the [README](https://github.com/get-convex/convex-waitlist#configuration) to learn how to add a waitlist to your existing app. The README also covers all available configuration. You don’t need to use [Convex](https://www.convex.dev/) for anything else other than the waitlist (*but you really should*).

<GitHubRepo url="https://github.com/get-convex/convex-waitlist" description="An easy to adopt waitlist implementation using Convex." />

## How does it work?

This waitlist implementation has been optimized to:

1. Handle tens of thousands of users per hour
2. Minimize the resources (and hence cost) of running the waitlist

…So it’s got a few tricks in it.

### Waiting and activating

![diagram of waitlist sessions](https://cdn.sanity.io/images/ts10onj4/production/1a40f39a5be67f6dd73e0e67f5741b6daebb487b-1712x1084.png)

Let’s start from the basics:

Clients (web browsers, mobile/desktop apps, etc.) will be talking to our backend (server) where the waitlist implementation is deployed. In this case the backend will be hosted on [Convex](https://www.convex.dev/).

The client identifies each user session with a string `sessionId`, sent to the backend. This string could be a user ID, if the app includes sign-in before users should be put on the waitlist.

The backend sends back to the client the session’s current `status`: `“waiting”` or `“active”`, and its `position`, which is used to show the user’s position in the waiting line, which we’ll cover later. Convex is a reactive backend, so if any of this data changes, it will update the client with the new values.

The backend is configured to only allow a certain number of *active* sessions at one time. Once that limit is reached any new sessions start off with the *waiting* status.

Besides the status and position, the backend stores a `lastActive` timestamp for every session. For *active* sessions, the timestamp can be refreshed every time the user actively uses the app. The implementation provides a `validateSessionAndRefreshLastActive` function for this, which updates the timestamp to the current time. For *waiting* sessions the `lastActive` timestamp is initially `null`, and it only gets set when a user leaves (we’ll also cover this later).

Periodically, say every minute, a cron job runs on the backend that updates the whole waitlist. 

1. First, it checks whether there are any waiting sessions. If there aren’t any, it’s done. 
2. Then it deletes any stale *waiting* sessions.
    1. Stale sessions have a non-null `lastActive` timestamp older than the configured timeout
3. Then it deletes any stale *active* sessions.
4. Then it marks *waiting* sessions and makes them *active* , up to the number of stale sessions it just deleted. 
5. Finally it updates the *active* sessions counter. 

This counter is used when a new session is created to determine the session status (as mentioned above). If the newly created session starts off as *active* the counter is also incremented.

![detailed diagram of updateWaitlist scenario](https://cdn.sanity.io/images/ts10onj4/production/f819babc817d2599350b46bdefbd3f7245b3c57f-1600x786.png)

As a final note, any endpoints reading data (in Convex these are called “queries”) should be protected to only serve *active* sessions by checking the current session status. The implementation provides a `validateSessionIsActive` function that does this and throws an error if the session isn’t active.

### Sidebar: Optimizing performance

The waitlist is optimized to handle a LOT of users. When a new user comes in, the backend needs to create a waitlist session for them, but to create a session for them it needs to know how many active sessions there are already. The implementation could:

1. Count the number of active sessions every time.
2. Keep a count of the active sessions in a separate table.

The first approach works fine for dozens of users, but would be slow for thousands. The second approach is better because it doesn’t waste resources on counting the number of sessions again and again. It does entail duplicating a piece of state though - the number of active sessions - but thanks to Convex’s transactional guarantees, you don’t have to worry about the two getting out of sync, as long as the implementation always updates both in the same transaction (in Convex every mutation is a transaction).

### Showing the position in the line to the user

So far we’ve covered how the backend works efficiently to determine each session’s status. But the implementation also takes care to efficiently update all the clients about their position in the line.

Imagine that the backend sent to each client what their actual position is, say ”1st out of 100”. Every time a new *waiting* session is created, all the clients would need to receive a custom update about the line size! Even if we didn’t show the number of users waiting, every time a session becomes *active*, all the *waiting* sessions behind it in the line would need a tailored update. That’s up to 1000 server function calls when there are 1000 users waiting, on every update!

Clearly this approach is not cheap. We can do better by offloading a little math to the client-side.

We mentioned before that each session receives from the backend the current `status` (`“waiting”` or `“active”`) and its `position`. This is not the position among the currently *waiting* sessions, but a globally incrementing position. The first session ever gets assigned position `0`, the second position `1`, and so on.[^1]

Besides this session specific information, which only changes when the user gets off of the waitlist, the backend also sends to all clients “global” information about the waitlist: `firstWaitingPosition` and `lastWaitingPosition`. This data does change every time a session becomes *active* or when a new *waiting* session is created, but because it doesn’t vary between clients, Convex can cache this result, so we’re only hitting the database once, even when the result gets sent to a 1000 clients.

Finally, given the `firstWaitingPosition`, `lastWaitingPosition` and the current session’s `position` the client can easily compute its position in the waiting line and how many users in total are waiting.

![diagram of the information clients receive](https://cdn.sanity.io/images/ts10onj4/production/62fea9b80884c6d2b77c7a25c3656e4c9f3c433f-1124x464.png)

### Sidebar 2: Forgoing precision

A careful reader such as yourself might spot a little issue with this increment-based approach to estimating the position in the waiting line. What if some people give up and leave while they’re still waiting?

This is another trade-off the implementation makes. To show an accurate position in the line, the backend would have to count (or a keep a count of) how many *waiting* sessions there are in front of each **waiting** session, or recalculate the position of each waiting session every time someone with lower position leaves. But the implementation is designed to support a LOT of users cheaply, especially those on the waitlist. And hence the waitlist forgoes a little precision to keep things efficient. In practice, with a large number of active and waiting sessions, this estimate should still give the user a good idea of where they stand.

### Leaving while waiting

We mentioned before that *active* sessions become stale when the user stops interacting with your app. What about *waiting* sessions though? The waiting users should not have to interact with the app to stay on the waitlist. In fact there should be no additional traffic from the waiting users to keep the cost of the waitlist as low as possible. Instead the implementation takes an “optimistic” approach: The client sends a request when the user navigates away from the app, notifying the server. This is when `lastActive` timestamp gets set. If the user doesn’t come back to the app within the configured timeout period, the *waiting* session becomes stale and is deleted.

## Conclusion

We hope you’ll find this waitlist implementation helpful. It should be easy to adopt, and build upon. For example, you could track the rate at which active sessions become stale to show an estimated waiting time to users in line. Let us know what other features you’d like a waitlist to have and how you’re taking advantage of Convex. Happy waiting.

<GitHubRepo url="https://github.com/get-convex/convex-waitlist" description="An easy to adopt waitlist implementation using Convex." />

[^1]: The implementation uses 64bit floats to represent the integer position, which is perhaps suboptimal, but still allows for over 9 million billion sessions - that’s 30K sessions per second for 10 thousand years. Convex does support 64bit integers though, so this limit can be increased easily. Do let us know if you’re planning to support more than 10^15 sessions, we’d love to hear about your app!

Does your app need to limit the amount of users that can use it at a time? You might need to add a waitlist, which you will learn to build in this post.

Wait a minute, won't you? (Add a waitlist to your product)

# Are vector databases dead?

This year vector databases have sprung up [like mushrooms](https://twitter.com/jobergum/status/1706928630612275309?s=20) to enable applications to retrieve context based on semantic search. A large portion of these applications have used the retrieved context to augment the ability of large language models (LLMs) in a pattern known as RAG. On November 7th OpenAI released its [Assistants API](https://platform.openai.com/docs/assistants/overview), enabling the implementation of AI chat interfaces with context retrieval without needing a separate message store or vector database. Does this new API make vector databases obsolete?

Yes! And no.

This post concludes a series comparing three different implementations of AI chat backends to help answer this question:

1. [Using OpenAI’s Assistants API](https://stack.convex.dev/ai-chat-using-openai-assistants-api)
2. [Using Langchain with our own storage](https://stack.convex.dev/ai-chat-using-langchain-and-convex)
3. [Custom message store and vector search retrieval](https://stack.convex.dev/ai-chat-with-convex-vector-search)
4. Are vector databases dead? (this post)

Read on for the final verdict.



## AI chat - what’s the point?

In the current age of ChatGPT, GPTs, Bards, Bings, and Claudes it might not be obvious why anyone should be building another AI-powered chat, but there are good reasons:

1. Proximity: People’s behavior is driven by convenience, and talking to an AI right in your product (website, app) might be more convenient than opening a separate chat product.
2. Control: Big LLMs have broad knowledge so they need specific prompting to get an answer about a specific topic. Having to provide this prompt again and again can be onerous for end-users. Your product can set up or augment the prompt automatically.
3. Context: Although LLMs retain a lot of knowledge from training, they might lack specific knowledge, either because it didn’t exist, wasn’t public, or wasn’t important enough at training time. Providing specific context as a part of the prompt leads to better accuracy and less hallucination.

Combining all three together can lead to integrations not possible in the frame of existing general chat interfaces. For example, we could provide the LLM with user-specific information, give it precise instructions, and have it take actions directly in our product. Think: going to Amazon and asking to reorder and double the order of an item you bought last week, and having your shopping cart update to reflect this conversation. We can render product information inline and have it be interactive without the user having to navigate between different websites or apps.

## AI chat case study: docs

This post discusses the tradeoffs and details of implementing this kind of AI system. A basic example of a custom AI-powered chat interface is the AI chatbot on the Convex docs site. You can play with it by clicking the chat bubble button at the top of the [page](https://docs.convex.dev).

![docs screenshot with ai chat bot](https://cdn.sanity.io/images/ts10onj4/production/627ee9c0759b8c43b3f43927cb9af70bf638f416-2000x1125.png)

To help answer the titular question, we implemented this AI chat backend in three ways. A guided walkthrough and the source code for each implementation is linked here:

1. [Using the new OpenAI Assistants API](https://stack.convex.dev/ai-chat-using-openai-assistants-api)
2. [Using Langchain with the Convex message and vector DB](https://stack.convex.dev/ai-chat-using-langchain-and-convex)
3. [Entirely on Convex](https://stack.convex.dev/ai-chat-with-convex-vector-search)

This example demonstrates the three reasons for building a custom AI chat:

- The interface is embedded in the docs site where developers look for technical information about Convex.
- The LLM is prompted to answer questions about Convex.
- The implementation passes the LLM the most relevant documentation pertaining to the question.

Even in this simple use case, the context is not static. The Convex team iterates on the docs and the product constantly, and the AI chat needs to reflect these changes. So when designing the implementations we considered how this contextual information will be updated over time.

## Assistants API

### Pros

The new Assistants API tries to provide a cohesive package, and this does have its strengths for us as the developers of the product:

- We don’t have to worry about the context data splitting (chunking) and search implementations.
- OpenAI’s implementation might improve over time without us having to invest in any effort.
- Other tools besides retrieval, currently function calling and code interpreter, are built-in, and OpenAI might add more tools in the future.

### Cons

Yet these come with tradeoffs, broadly around how much we control and how much we still have to do:

- The data splitting and search are a black box. It’s common (and we did so in our implementation) to tune the chunk size to get the best results from semantic search. But we can’t control this with the Assistants API, yet only we understand and can evaluate the quality of the context retrieval.
- The flip side to the API improving over time is that it might produce unexpected results in the future.
- We might want to build more complicated chains of context retrieval and LLM prompting, see [RAG with guardrails](https://www.pinecone.io/learn/fast-retrieval-augmented-generation/#RAG-with-Guardrails). But since we don’t control the implementation this is not possible.
- We still need a server to call the API from, to simplify the uploading and updating of contextual data, and to execute function calls from the LLM.
- It’s likely that we will still want to store message content for analytics, tight integration with our products, and abuse prevention. This leads to the need for syncing the messages to and from OpenAI, which can be more complex than having our server be the source of truth and only interact with OpenAI in a request/response fashion.
- Vendor lock-in: the implementations compared here all use OpenAI, as it performs well on both pricing and performance, especially considering the quality of results from GPT4. But adopting the Assistants API would make it much harder to move to a different LLM provider.

Additionally, the API has some rough edges at the moment. See the [implementation post](https://stack.convex.dev/ai-chat-using-openai-assistants-api) for more details.

Overall we’d suggest waiting before the API moves out of beta before adopting it. Even then, it should be carefully considered what kind of final experience one is trying to achieve, as more custom experiences will likely benefit from a custom implementation.

## LangChain

LangChain is an extremely popular and yet somewhat [controversial library](https://x.com/xixixao/status/1649930481834803200?s=20). It combines two seemingly opposing goals:

1. Make AI development more approachable.
2. Wrap, encapsulate, and expose sophisticated AI use cases.

### Pros

In the [second implementation](https://stack.convex.dev/ai-chat-using-langchain-and-convex), these two goals manifested themselves:

- The implementation is the shortest, as LangChain exposes very high level APIs tailored for exactly our use case. We didn’t even need to write a single system prompt.
- Under the hood the “chain” actually performs summarization before performing the context search and prompting to the LLM.
- Because LangChain abstracts the implementation, it would be trivial to swap in a different embedding or LLM model provider, or different vector database.
- It provides extensive “automatic” debug logging.

### Cons

Achieving these two goals though does have its downsides:

- LangChain defaults can be more complex than what the implementation really needs, and perform worse than simpler alternatives. This is especially true as LangChain tries to be agnostic to which LLM it interfaces with. Different LLMs will perform very differently and therefore might require different approaches to prompting and summarization.
- Once a developer wants to veer off of the default path, they have to understand the ways in which LangChain can be configured. This is tricky, and due to the high amount of abstraction, reading LangChain’s source code to figure out what it does under the hood can be difficult.
- The TypeScript version of LangChain is not perfectly typed, in part because the return values depend on configs nested deep in the chain.

Overall LangChain is great for prototyping and getting a system working quickly, but for a more sophisticated integration it’s a good idea to refactor into a fully custom implementation.

## Pure Convex  implementation

Since Convex includes [vector search](https://docs.convex.dev/vector-search) the entire system can be implemented in it, only using external embedding and an LLM provider. In fact the LangChain implementation already used Convex under the hood for vector search, message storage and embedding caching.

Implementing the system without using a library like LangChain gives us the most control about exactly how the system works: 

- How and when data is split and how it is cached.
- When and how the data is embedded.
- How much context is used.
- In what format the data is passed to the LLM.
- What are the prompts involved.
- How are the responses processed and streamed back to our interface.

All of this is in plain sight in [our code](https://stack.convex.dev/ai-chat-with-convex-vector-search).

We have to admit that this comparison would be different if we weren’t using Convex, and instead had a more traditional server talk to an external relational/document databases and a vector database. With Convex, the custom implementation isn’t that much more complicated than the alternatives. And hence it’s easier to argue for the added up-front cost of developing a custom system to reap the benefits of total control. In turn, we get end-to-end type safety and full transparency.

## Conclusion

To answer the original question, does the new Assistants API make vector databases obsolete? We can argue, yes: If a use case aligns closely with what the API provides, and the alternative is shuffling data to an external vector database, then one might as well shuffle the data to the Assistants API instead.

But we would argue that integrated AI experiences should offer more than standalone ChatGPT to justify their existence, and to build such experiences it's best to retain control over the implementation by leveraging a database that can both store the product data and allow semantically searching it. This contextual data can then be used get the most out of current LLMs.

This year vector databases have sprung up like mushrooms to enable applications to retrieve context based on semantic search. A large portion of these applications have used the retrieved context to augment the ability of large language models (LLMs) in a pattern known as RAG. On November 7th OpenAI released its Assistants API, enabling the implementation of AI chat interfaces with context retrieval without needing a separate message store or vector database. Does this new API make vector databases obsolete?

Are Vector Databases Dead?

# AI Chat with Convex Vector Search

Convex is a full-stack development platform and cloud database, including built-in [vector search](https://docs.convex.dev/vector-search). In this third post in our series, we’ll build an AI-powered chat interface using Convex, with our own message storage and context retrieval.

All code can be found on [GitHub](https://github.com/get-convex/convex-ai-chat/).

<GitHubRepo url="https://github.com/get-convex/convex-ai-chat" description="AI Chat implemented with Convex vector search" />


This post is part of a series where we compare three different implementations of the same AI chatbot and summarize the differences:

1. [Using OpenAI’s Assistants API](https://stack.convex.dev/ai-chat-using-openai-assistants-api)
2. [Using Langchain with our own storage](https://stack.convex.dev/ai-chat-using-langchain-and-convex)
3. Custom message store and vector search retrieval (this post)
4. [Are vector databases dead?](https://stack.convex.dev/are-vector-databases-dead)

## ****Overview of the user experience****

We’ll build an AI chatbot for the [Convex docs site](https://docs.convex.dev/home). You can go to the docs now and interact with the bot there:

![docs screenshot with ai chat bot](https://cdn.sanity.io/images/ts10onj4/production/633b58f26a29b7e3de1a4d402aabc4ca0526b647-2400x1350.png)

## Overview of RAG chat

Here’s a graphical overview of the steps involved. Unlike in our previous posts, we will be implementing all steps ourselves, using Convex for storage and search, and OpenAI for creating embeddings and LLM chat completion.

![rag chat process overview](https://cdn.sanity.io/images/ts10onj4/production/4cb4a308b847724739ea0405f2f13825453efd7d-1730x724.png)

We need to load our data, optionally split it, and then “embed” it (run it through a model that encodes the data into an array of numbers). This step needs to happen at least once, but we probably want to run it periodically as the source data (in our example the Convex docs pages and other information about Convex) changes.

After we ingest data, we can serve traffic, in our case reply to user questions and follow-ups. Each question can be embedded (using the same model that we used during the ingest step), and then the embedding is used to look up the most relevant contextual data. We then pass on this data, the question, and any chat history to the LLM as a prompt, and return back to the user the LLM’s answer.

## Schema

We’ll start with our schema, which gives a good overview of what data will our database store ([source](https://github.com/get-convex/convex-ai-chat/blob/main/convex/schema.ts)):

```jsx
export default defineSchema({
  messages: defineTable({
    // Whether the message is from the AI or the human
    isViewer: v.boolean(),
    // Which conversation this message belongs to
    sessionId: v.string(),
    // Message content
    text: v.string(),
  }).index("bySessionId", ["sessionId"]),
  documents: defineTable({
    // The original page URL for the document
    url: v.string(),
    // The parsed document content
    text: v.string(),
  }).index("byUrl", ["url"]),
  chunks: defineTable({
    // Which document this chunk belongs to
    documentId: v.id("documents"),
    // The chunk content
    text: v.string(),
    // If the chunk has been embedded, which embedding corresponds to it
    embeddingId: v.union(v.id("embeddings"), v.null()),
  })
    .index("byDocumentId", ["documentId"])
    .index("byEmbeddingId", ["embeddingId"]),
  // the actual embeddings
  embeddings: defineTable({
    embedding: v.array(v.number()),
    chunkId: v.id("chunks"),
  })
    .index("byChunkId", ["chunkId"])
    .vectorIndex("byEmbedding", {
      vectorField: "embedding",
      dimensions: 1536,
    }),
});
```

We’ll refer to these tables as we go.

Since we’ll be using the OpenAI SDK, we’ll set our `OPENAI_API_KEY` in the [environment variable settings on Convex dashboard](https://docs.convex.dev/dashboard/deployments/deployment-settings).

## Ingest data: Loading & splitting data

How the data is loaded depends on what it is, whether webpages, PDFs, spreadsheets, scanned documents or any other source. This is the first complexity we encounter when building a system like this, and the choices we make here will have a downstream impact. For example, unless we coerce the LLM to format its answers in specific ways, it is likely to mimic the contextual data we provide it.

In the case of Convex docs, we would like the LLM to reply in markdown, which we can render into a rich text representation in the future. While we have the source of the docs, also in markdown, for our documentation chatbot we still opted to scrape our own website. Our docs include snippets and other content that does not show up until they are rendered into HTML.

You can see the 100 lines of code that perform the parsing from HTML to markdown [here](####). While we could use an existing library, writing this code ourselves gives us the most control over the output - a theme that will repeat throughout this post.

To test this scraping code out we build ourselves an endpoint that takes a single page URL, parses the page, splits it up into chunks, and saves it in the database ([source](https://github.com/get-convex/convex-ai-chat/blob/main/convex/ingest/load.ts)):

```jsx
export const fetchSingle = internalAction({
  args: {
    url: v.string(),
  },
  handler: async (ctx, { url }) => {
    const response = await fetch(url);
    const text = parsePage(await response.text());
    if (text.length > 0) {
      await ctx.runMutation(internal.ingest.load.updateDocument, { url, text });
    }
  },
});

export const updateDocument = internalMutation(
  async (ctx, { url, text }: { url: string; text: string }) => {
    const latestVersion = await ctx.db
      .query("documents")
      .withIndex("byUrl", (q) => q.eq("url", url))
      .order("desc")
      .first();

    const hasChanged = latestVersion === null || latestVersion.text !== text;
    if (hasChanged) {
      const documentId = await ctx.db.insert("documents", { url, text });
      const splitter = RecursiveCharacterTextSplitter.fromLanguage("markdown", {
        chunkSize: 2000,
        chunkOverlap: 100,
      });
      const chunks = await splitter.splitText(text);
      await map(chunks, async (chunk) => {
        await ctx.db.insert("chunks", {
          documentId,
          text: chunk,
          embeddingId: null,
        });
      });
    }
  }
);
```

Some noteworthy details:

- We don’t store pages with no text
- We’re using the `byUrl` index to ensure that we only store a new version of a given page if the content has changed
    - We do keep the existing data in the database since we don’t want to degrade the performance of the AI chat while we’re updating the scraped data. We can delete the stale data after successful update.

We also chunk the document text, for two reasons:

1. We want to make sure our contextual data fits in the LLM’s prompt.
2. Smaller chunks will be semantically more focused, and so we might get better results for specific user questions.

Since we have total control over this setup, we can easily optimize it. For example, instead of storing the full document text in both the `"documents"` table and split-up in the `"chunks"` table, we could store only a hash of the text in the `"documents"` table to determine whether the chunks need updating.

When we’re satisfied with the parsing we can use an action to scrape the whole website. Luckily our docs site has a [site map](https://en.wikipedia.org/wiki/Site_map), which will make it really easy ([source](https://github.com/get-convex/convex-ai-chat/blob/main/convex/ingest/load.ts)).

## Ingest data: Embedding data

For creating the embeddings for our documents we’ll use OpenAI’s API. This is our simple wrapper around the API ([source](https://github.com/get-convex/convex-ai-chat/blob/main/convex/ingest/embed.ts)):

```jsx
export async function embedTexts(texts: string[]) {
  if (texts.length === 0) return [];
  const openai = new OpenAI();
  const { data } = await openai.embeddings.create({
    input: texts,
    model: "text-embedding-ada-002",
  });
  return data.map(({ embedding }) => embedding);
}
```

Since we’ll want to run the embedding periodically, we structured our code to take in a list of document IDs, and we’ll first check which chunks need an embedding ([source](https://github.com/get-convex/convex-ai-chat/blob/main/convex/ingest/embed.ts)):

```jsx
export const chunksNeedingEmbedding = internalQuery(
  async (ctx, { documentId }: { documentId: Id<"documents"> }) => {
    const chunks = await ctx.db
      .query("chunks")
      .withIndex("byDocumentId", (q) => q.eq("documentId", documentId))
      .collect();
    return chunks.filter((chunk) => chunk.embeddingId === null);
  }
);
```

With this function in place, we can write the logic for embedding a list of documents ([source](https://github.com/get-convex/convex-ai-chat/blob/main/convex/ingest/embed.ts)):

```jsx
export const embedList = internalAction({
  args: {
    documentIds: v.array(v.id("documents")),
  },
  handler: async (ctx, { documentIds }) => {
    const chunks = (
      await map(documentIds, (documentId) =>
        ctx.runQuery(internal.ingest.embed.chunksNeedingEmbedding, {
          documentId,
        })
      )
    ).flat();

    const embeddings = await embedTexts(chunks.map((chunk) => chunk.text));
    await map(embeddings, async (embedding, i) => {
      const { _id: chunkId } = chunks[i];
      await ctx.runMutation(internal.ingest.embed.addEmbedding, {
        chunkId,
        embedding,
      });
    });
  },
});
```

With this action, we can again test that the embedding works as expected before we run the action for embedding all documents ([source](https://github.com/get-convex/convex-ai-chat/blob/main/convex/ingest/embed.ts)). And because we made sure to not re-embed text that’s already embedded, we don’t have to worry about how many times we run this action.

## Serving traffic: answering a question

When a user hits send in our little chat box, we’ll kick off our serving answering logic ([source](https://github.com/get-convex/convex-ai-chat/blob/main/convex/messages.ts)):

```jsx
export const send = mutation({
  args: {
    message: v.string(),
    sessionId: v.string(),
  },
  handler: async (ctx, { message, sessionId }) => {
    await ctx.db.insert("messages", {
      isViewer: true,
      text: message,
      sessionId,
    });
    await ctx.scheduler.runAfter(0, internal.aiChat.serve.answer, {
      sessionId,
    });
  },
});
```

The `scheduler.runAfter(0, …)` call is Convex parlance for “run an async job immediately”.

We implement the LLM-backed answering in one 80-line function ([source](https://github.com/get-convex/convex-ai-chat/blob/main/convex/serve.ts)). Let’s break it down:

```jsx
export const answer = internalAction({
  args: {
    sessionId: v.string(),
  },
  handler: async (ctx, { sessionId }) => {
```

We take in the `sessionId` argument which identifies which thread needs an answer. Then:

```jsx
    const messages = await ctx.runQuery(internal.serve.getMessages, {
      sessionId,
    });
    const lastUserMessage = messages.at(-1)!.text;

    const [embedding] = await embedTexts([lastUserMessage]);
```

Next, we find the messages for the thread and embed the last message. Since we expect this action to only be scheduled after a user sends a message, we can assume that the last message is indeed from the user. Then:

```jsx
    const searchResults = await ctx.vectorSearch("embeddings", "byEmbedding", {
      vector: embedding,
      limit: 8,
    });

    const relevantDocuments = await ctx.runQuery(internal.serve.getChunks, {
      embeddingIds: searchResults.map(({ _id }) => _id),
    });
```

We find the 8 most relevant documents for the given message (first performing a search that gives us the embeddings’ IDs, then mapping those to the chunk IDs). Let’s look at the rest of the function:

```jsx
    const messageId = await ctx.runMutation(internal.serve.addBotMessage, {
      sessionId,
    });

    try {
      const openai = new OpenAI();
      const stream = await openai.chat.completions.create({
        model: "gpt-4-32k",
        stream: true,
        messages: [
          {
            role: "system",
            content:
              "Answer the user question based on the provided documents " +
              "or report that the question cannot be answered based on " +
              "these documents. Keep the answer informative but brief, " +
              "do not enumerate all possibilities.",
          },
          ...(relevantDocuments.map(({ text }) => ({
            role: "system",
            content: "Relevant document:\n\n" + text,
          })) as ChatCompletionMessageParam[]),
          ...(messages.map(({ isViewer, text }) => ({
            role: isViewer ? "user" : "assistant",
            content: text,
          })) as ChatCompletionMessageParam[]),
        ],
      });
      let text = "";
      for await (const { choices } of stream) {
        const chunk = choices[0].delta.content;
        if (typeof chunk === "string" && chunk.length > 0) {
          text += choices[0].delta.content;
          await ctx.runMutation(internal.serve.updateBotMessage, {
            messageId,
            text,
          });
        }
      }
    } catch (error: any) {
      await ctx.runMutation(internal.serve.updateBotMessage, {
        messageId,
        text: "I cannot reply at this time. Reach out to the team on Discord",
      });
      throw error;
    }
```

First, we insert an empty message into the thread. This will enable us to show the “thinking” indicator back to the user.

We then call OpenAI’s GPT model. In this implementation, we use the `stream` argument to enable result streaming. This will lead to a better experience for our users, but we could have chosen not to use it as well (perhaps if our UX wasn’t quite the traditional chat app). With Convex, we don’t actually stream the results chunk by chunk to the client, instead, we accumulate the added chunks, store them in the message, and let Convex update the UI every time (via automatic reactivity). The resulting UX is just the same ([see this article for more details](https://stack.convex.dev/gpt-streaming-with-persistent-reactivity)).

Finally, we handle any errors by replacing the message text. We could have also decided to leave any partial message in place. We re-throw the error to get the best debugging experience.

## Serving traffic: listing messages

There’s only one more endpoint left to cover, and that’s the query that returns messages for a given session ([source](https://github.com/get-convex/convex-ai-chat/blob/main/convex/messages.ts)):

```jsx
export const list = query({
  args: {
    sessionId: v.string(),
  },
  handler: async (ctx, args) => {
    return await ctx.db
      .query("messages")
      .withIndex("bySessionId", (q) => q.eq("sessionId", args.sessionId))
      .collect();
  },
});
```

The client subscribes to this query and will get all messages for the session, automatically updating when new messages are added. Learn more about Convex reactivity [here](https://docs.convex.dev/tutorial/reactor).

And that’s it! The backend for our RAG chat is complete.

## Debugging tip

If you’re trying to debug the question-answering logic, it can be helpful to log the retrieved document chunks:

```tsx
// In convex/serve.ts, in the `answer` action:
console.log(lastUserMessage, relevantDocuments);
```

Then check out the [logs page on your Convex dashboard](https://docs.convex.dev/dashboard/deployments/logs) to see the context that was passed to the LLM.

## Frontend Implementation

You can find the front-end implementation of our chat box [here](https://github.com/get-convex/convex-ai-chat/blob/main/src/aiChat/index.tsx). It is only about 250 lines of React and Tailwind CSS, but it does pack a lot of functionality:

1. Realtime reactive updates with the LLM replies.
2. Preserving the session ID in `sessionStorage` (so it survives page reloads).
3. Scrolling to the bottom of the thread.
    1. Stop scrolling if the user scrolls up.
4. Opening the dialog in a React portal.
5. Disabling the send button when the input is empty.
6. Two different UIs: Small in the corner of the page and expanded.
7. Info icon with a tooltip for additional education/disclaimer.
8. Loading indicators
9. Dark mode

If you check out the [repo](https://github.com/get-convex/convex-ai-chat/blob/main/src/App.tsx) you can see that Convex is only used for the chat modal. This is a good example if you want to drop the chat component into an existing website.

## Conclusion

Sticking with Convex, instead of with the [Assistant API](https://stack.convex.dev/ai-chat-using-openai-assistants-api) or [LangChain](https://stack.convex.dev/ai-chat-using-langchain-and-convex) as we did in previous posts, we have full control over our AI system. We also have end-to-end type safety and consistency guarantees. We can log any data flowing through our system, and we can easily inspect the data stored in the database to debug why certain questions lead to given answers. Continue to the final post in this series to read more about the [tradeoffs between different AI product implementations](https://stack.convex.dev/are-vector-databases-dead).

Convex is a full-stack development platform and cloud database, including built-in vector search. In this third post in our [series](https://stack.convex.dev/ai-chat-using-openai-assistants-api), we’ll build an AI-powered chat interface using Convex, with our own message storage and context retrieval.

Build AI Chat with Convex Vector Search

# AI Chat using LangChain and Convex

[Langchain](https://www.langchain.com/) is a very popular library for building AI applications in Python or JavaScript/TypeScript. In this second post in [our series](https://stack.convex.dev/ai-chat-using-openai-assistants-api), we’ll build an AI-powered chat interface using LangChain and its new Convex integration.

All code can be found on [GitHub](https://github.com/get-convex/convex-ai-chat-langchain/).

<GitHubRepo url="https://github.com/get-convex/convex-ai-chat-langchain" description="AI Chat implemented with LangChain and Convex." />


This post is part of a series where we compare three different implementations of the same AI chatbot and summarize the differences:

1. [Using OpenAI’s Assistants API](https://stack.convex.dev/ai-chat-using-openai-assistants-api)
2. Using Langchain with our own storage (this post)
3. [Custom message store and vector search retrieval](https://stack.convex.dev/ai-chat-with-convex-vector-search)
4. [Are vector databases dead?](https://stack.convex.dev/are-vector-databases-dead)


## ****Overview of the user experience****

We’ll build an AI chatbot for the [Convex docs site](https://docs.convex.dev/home). You can go to the docs now and interact with the bot there:

![docs screenshot with ai chat bot](https://cdn.sanity.io/images/ts10onj4/production/633b58f26a29b7e3de1a4d402aabc4ca0526b647-2400x1350.png)

We’re not just exposing the standard LLM model (such as GPT4), but we’re also retrieving the relevant documents for the question and passing them to the LLM as context ([RAG](https://js.langchain.com/docs/use_cases/question_answering/)).

## Overview of RAG chat

Here’s a graphical overview of the steps involved. LangChain will lead to a very short implementation, as almost the entirety of the process will be handled by the library.


![rag chat process overview with langchain](https://cdn.sanity.io/images/ts10onj4/production/7434678c58f8afb661ea5126c15cff68615f2ab7-1714x694.png)

We need to load our data, optionally split it, and then “embed” it (run it through a model that encodes the data into an array of numbers). This step needs to happen at least once, but we probably want to run it periodically as the source data (in our example the Convex docs pages and other information about Convex) changes.

After we ingest data, we can serve traffic, in our case reply to user questions and follow-ups. Each question can be embedded (using the same model that we used during the ingest step), and then the embedding is used to look up the most relevant contextual data. We then pass on this data, the question, and any chat history to the LLM as a prompt, and return back to the user the LLM’s answer.

The LangChain implementation we’ll be using performs a rephrasing step before searching data:

![serve step summarization diagram](https://cdn.sanity.io/images/ts10onj4/production/8288fedee8944ef1edc5a50d8041a37bb6ee0a7e-1338x650.png)

This is helpful for getting more useful documents retrieved for follow-up questions, but it does mean that the LLM is not seeing the conversation as a conversation, but instead is only seeing a single prompt that hopefully summarizes the previous discussion. It is possible to configure LangChain to not do this, but we’ll stick to the defaults in this post.

## Our Server Setup & Schema

We’ll be using the following schema, based on the LangChain Convex integrations docs [for cache](https://js.langchain.com/docs/modules/data_connection/text_embedding/how_to/caching_embeddings#usage-convex), [for documents](https://js.langchain.com/docs/integrations/vectorstores/convex), and [for messages](https://js.langchain.com/docs/integrations/chat_memory/convex) ([source](https://github.com/get-convex/convex-ai-chat-langchain/blob/main/convex/schema.ts)):

```tsx
export default defineSchema({
  // Simple cache to avoid recomputing embeddings
  cache: defineTable({
    // content
    key: v.string(),
    // embedding
    value: v.any(),
  }).index("byKey", ["key"]),
  // one row for each chunk of a document
  documents: defineTable({
    embedding: v.array(v.number()),
    text: v.string(),
    metadata: v.any(),
  }).vectorIndex("byEmbedding", {
    vectorField: "embedding",
    dimensions: 1536,
  }),
  messages: defineTable({
    // Which conversation this message belongs to
    sessionId: v.string(),
    message: v.object({
      // The message author, either AI or human
      type: v.string(),
      data: v.object({
        // The text of the message
        content: v.string(),
        role: v.optional(v.string()),
        name: v.optional(v.string()),
        additional_kwargs: v.optional(v.any()),
      }),
    }),
  }).index("bySessionId", ["sessionId"]),
});
```

We’ll also add `langchain/db.ts` file with the following export:

```tsx
export * from "langchain/util/convex";
```

This declares internal functions used by LangChain to interact with our tables.

Since we’ll be using the OpenAI SDK, we’ll set our `OPENAI_API_KEY` in the [environment variable settings on the Convex dashboard](https://docs.convex.dev/dashboard/deployments/deployment-settings).

## Ingest data: Loading & splitting & embedding data

We’ll lean on LangChain to help us scrape our docs, split them, and embed them for semantic search. We won’t have much control over the exact formatting of the documents using LangChain, but can implement manual parsing which we’ll cover in the [third post](https://stack.convex.dev/ai-chat-with-convex-vector-search) in this series.

Let’s look at the function responsible for ingesting a single page ([source](https://github.com/get-convex/convex-ai-chat-langchain/blob/main/convex/ingest/load.ts)):

```tsx
export const fetchAndEmbedSingle = internalAction({
  args: {
    url: v.string(),
  },
  handler: async (ctx, { url }) => {
    const loader = new CheerioWebBaseLoader(url);
    const data = await loader.load();
    const textSplitter = new RecursiveCharacterTextSplitter({
      chunkSize: 1000,
      chunkOverlap: 200,
    });

    const splitDocs = await textSplitter.splitDocuments(data);

    const embeddings = new CacheBackedEmbeddings({
      underlyingEmbeddings: new OpenAIEmbeddings(),
      documentEmbeddingStore: new ConvexKVStore({ ctx }),
    });

    await ConvexVectorStore.fromDocuments(splitDocs, embeddings, { ctx });
  },
});
```

The `CheerioWebBaseLoader` parses the HTML and returns just the text on the page with no formatting.

Then the `RecursiveCharacterTextSplitter` splits the documents into 1000-character chunks, with 200-character overlap. The size of the chunk depends on the context limits of the model we’re intending to use, we want to make sure that each chunk can fully fit in the context window. If we make chunks smaller we’ll make it more likely that multiple documents will be used as context, which can be better or worse, it really depends on our data and the kinds of questions the system is intended to answer.

Finally, we use `ConvexVectorStore` to run each chunk through OpenAI’s embedding API, caching them using `ConvexKVStore` in the `“cache”` table, and storing the chunks with the embeddings in the `“documents”` table.

## Serving traffic: answering a question

Just like with ingesting data, LangChain really streamlines the process of answering a message ([source](https://github.com/get-convex/convex-ai-chat-langchain/blob/main/convex/serve.ts)):

```tsx
export const answer = internalAction({
  args: {
    sessionId: v.string(),
    message: v.string(),
  },
  handler: async (ctx, { sessionId, message }) => {
    const vectorStore = new ConvexVectorStore(new OpenAIEmbeddings(), { ctx });

    const model = new ChatOpenAI({ modelName: "gpt-4-32k" });
    const memory = new BufferMemory({
      chatHistory: new ConvexChatMessageHistory({ sessionId, ctx }),
      memoryKey: "chat_history",
      outputKey: "text",
      returnMessages: true,
    });
    const chain = ConversationalRetrievalQAChain.fromLLM(
      model,
      vectorStore.asRetriever(),
      { memory }
    );

    await chain.call({ question: message });
  },
});
```

First, we set up the `ConvexVectorStore` using the same OpenAI embeddings API.

Next, we initialize  `ConvexChatMessageHistory` which will both read and write to our `"messages"` table, given the current `sessionId`.

And finally, we run the `ConversationalRetrievalQAChain`, which will write both the user message and the LLM answer to the `“messages”` table.

## Serving traffic: listing messages

The only other function we need is a query for listing current messages given a `sessionId` ([source](https://github.com/get-convex/convex-ai-chat-langchain/blob/main/convex/messages.ts)):

```tsx
export const list = query({
  args: {
    sessionId: v.string(),
  },
  handler: async (ctx, args) => {
    return (
      await ctx.db
        .query("messages")
        .withIndex("bySessionId", (q) => q.eq("sessionId", args.sessionId))
        .collect()
    ).map(({ message: { data, type }, ...fields }) => ({
      ...fields,
      isViewer: type === "human",
      text: data.content,
    }));
  },
});
```

We’re doing a little transformation on each message to return it in the same format as in our [first post](https://stack.convex.dev/ai-chat-using-openai-assistants-api).

## Debugging tip ☝️

If you’re trying to debug a chain, enable the verbose mode like this:

```tsx
    const chain = ConversationalRetrievalQAChain.fromLLM(
      model,
      vectorStore.asRetriever(),
      { memory, verbose: true }
    );
```

Then check out the [logs page on your Convex dashboard](https://docs.convex.dev/dashboard/deployments/logs) to see all the inputs and outputs for each step in the chain.

## Frontend Implementation

You can find the front-end implementation of our chat box [here](####). It is only about 250 lines of React and Tailwind CSS, but it does pack a lot of functionality:

1. Realtime reactive updates with the LLM replies.
2. Preserving the session ID in `sessionStorage` (so it survives page reloads).
3. Scrolling to the bottom of the thread.
    1. Stop scrolling if the user scrolls up.
4. Opening the dialog in a React portal.
5. Disabling the send button when the input is empty.
6. Two different UIs: Small in the corner of the page and expanded.
7. Info icon with a tooltip for additional education/disclaimer.
8. Loading indicators.
9. Dark mode.

If you check out the [repo](https://github.com/get-convex/convex-ai-chat-langchain/blob/main/src/aiChat/index.tsx) you can see that Convex is only used for the chat modal. This is a good example if you want to drop the chat component into an existing website.

## Conclusion

LangChain packs a lot of power into a simple library interface, and with very little code gets us a fairly sophisticated implementation. On the flip side, we do lose some type-safety, and figuring out what exactly the library does under the hood, and how to configure it, can be tricky. Continue on to our [third post](https://stack.convex.dev/ai-chat-with-convex-vector-search) if you’re interested in having total control over how your AI chat app is implemented. Continue to the final post in this series to read more about the [tradeoffs between different AI product implementations](https://stack.convex.dev/are-vector-databases-dead).

In this second post in our series, we’ll build an AI-powered chat interface using LangChain and its new Convex integration.

Build AI Chat with LangChain and Convex

# AI Chat using OpenAI Assistants API

On November 7th OpenAI released its [Assistants API](https://platform.openai.com/docs/assistants/overview), enabling chat bot with context retrieval implementations without needing a messages or vector database. In this post, we’ll cover how to leverage this API to build a fully functioning AI chat interface.

The implementation here can be a basis for a more complex and tailored experience, which ties app-specific and user-specific information together in a single interface, surpassing the capabilities of the standalone ChatGPT.

This post is part of a series where we compare three different implementations of the same AI chatbot and summarize the differences:

1. Using OpenAI’s Assistants API (this post)
2. [Using Langchain with our own storage](https://stack.convex.dev/ai-chat-using-langchain-and-convex)
3. [Custom message store and vector search retrieval](https://stack.convex.dev/ai-chat-with-convex-vector-search)
4. [Are vector databases dead?](https://stack.convex.dev/are-vector-databases-dead)

All code can be found on [GitHub](https://github.com/get-convex/convex-ai-chat-openai/).

<GitHubRepo url="https://github.com/get-convex/convex-ai-chat-openai" description="AI Chat implemented with the OpenAI Assistants API." />

## ****Overview of the user experience****

To compare the three approaches we’ll build an AI chatbot for the [Convex docs site](https://docs.convex.dev/home). You can go to the docs now and interact with the bot there:

![docs screenshot with ai chat bot](https://cdn.sanity.io/images/ts10onj4/production/633b58f26a29b7e3de1a4d402aabc4ca0526b647-2400x1350.png)

We’re not just exposing the standard LLM model (such as GPT4), but we’re also retrieving the relevant documents for the question and passing them to the LLM as context ([RAG](https://js.langchain.com/docs/use_cases/question_answering/)).

## Overview of RAG chat

Here’s a graphical overview of the steps involved. Thanks to the new Assistants API, OpenAI will take care of most of this process.

![process overview for RAG chat](https://cdn.sanity.io/images/ts10onj4/production/0020d6ed2c966add4471a14f084efd7bf9589fbd-1722x708.png)

We need to load our data, optionally split it, and then “embed” it (run it through a model that encodes the data into an array of numbers). This step needs to happen at least once, but we probably want to run it periodically as the source data (in our example the <QuickConvex /> documentation and other information about Convex) changes.

After we ingest data, we can serve traffic. In our case, we reply to user questions about Convex. Each question can be embedded (using the same model that we used during the ingest step), and then the embedding is used to look up the most relevant contextual data. We then pass on this contextual data, the question, and any chat history to the LLM as a prompt, and return the LLM’s answer back to the user.

As we’ll see in the second post, a rephrasing step can be added to enable searching data based on the whole chat history, not just the last user message. The Assistants API might or might not be using this technique, as the search is completely abstracted away from us.

## Our Server Setup & Schema

While the Assistants API is powerful, we still want our own server for a few reasons:

1. We don’t want to expose our OpenAI API key. The OpenAI requests need to be authenticated, but if we send them from the browser, any user could see our key and use it themself.
2. We’ll use the server to pre-process our documentation data before uploading it to OpenAI.
3. Future: Having a server can enable the Assistants API to call functions that we can expose to take some action on behalf of the user.
4. Future: If we want to build a more sophisticated experience, we can directly access our product database during the chat.

We’ll define the following three tables in Convex([source](https://github.com/get-convex/convex-ai-chat-openai/blob/main/convex/schema.ts)):

```jsx
export default defineSchema({
  documents: defineTable({
    // The original page URL for the document
    url: v.string(),
    // The parsed document content
    text: v.string(),
    // The ID returned after uploading to OpenAI
    fileId: v.union(v.string(), v.null()),
  }).index("byUrl", ["url"]),
  messages: defineTable({
    // Whether the message is from the AI or the human
    isViewer: v.boolean(),
    // Which conversation this message belongs to
    sessionId: v.string(),
    // Message content
    text: v.string(),
  }).index("bySessionId", ["sessionId"]),
  threads: defineTable({
    // Client-generated conversation identifier
    sessionId: v.string(),
    // Conversation identifier used by the OpenAI server
    threadId: v.string(),
  }).index("bySessionId", ["sessionId"]),
});
```

Since we’ll be using the OpenAI SDK, we’ll set our `OPENAI_API_KEY` in the [environment variable settings on the Convex dashboard](https://docs.convex.dev/dashboard/deployments/deployment-settings).

## Ingest data: Creating an assistant

We’ll start by creating an [OpenAI assistant](https://platform.openai.com/docs/assistants/overview). You can do this on the OpenAI dashboard, or using the API ([source](https://github.com/get-convex/convex-ai-chat-openai/blob/main/convex/init.ts)):

```tsx
export const createAssistant = internalAction({
  args: {},
  handler: async () => {
    const openai = new OpenAI();
    const assistant = await openai.beta.assistants.create({
      instructions:
        "Answer the user questions based on the provided documents " +
        "or report that the question cannot be answered based on " +
        "these documents. Keep the answer informative but brief, " +
        "do not enumerate all possibilities.",
      model: "gpt-4-1106-preview",
      tools: [{ type: "retrieval" }],
    });
    return assistant.id;
  },
});
```

You can run this action from the [Convex dashboard](https://dashboard.convex.dev) and then save the returned ID as  `ASSISTANT_ID` in your Convex backend’s [environment variables](https://docs.convex.dev/dashboard/deployments/deployment-settings).

## Ingest data: Loading data

We’ll cover the detailed description of scraping in the third post in the series, but here we’ll discuss how to upload data to our OpenAI assistant.

While scraping the docs, we stored the data in the `"documents"` table. We also have a field on each document that stores the `fileId` returned by OpenAI after we upload it. Here’s the function responsible for uploading our files to OpenAI and attaching them to the assistant ([source](https://github.com/get-convex/convex-ai-chat-openai/blob/main/convex/init.ts)):

```tsx
export const uploadDocuments = internalAction({
  args: {
    documentIds: v.array(v.id("documents")),
  },
  handler: async (ctx, { documentIds }) => {
    const openai = new OpenAI();
    await map(documentIds, async (documentId) => {
      const document = await ctx.runQuery(internal.init.getDocument, {
        documentId,
      });
      if (document === null || document.fileId !== null) {
        return;
      }
      const { text, url } = document;
      const blob = new File([text], fileName(url));

      const { id: fileId } = await openai.files.create({
        file: blob,
        purpose: "assistants",
      });
      await openai.beta.assistants.files.create(process.env.ASSISTANT_ID!, {
        file_id: fileId,
      });
      await ctx.runMutation(internal.init.saveFileId, { documentId, fileId });
    });
  },
});
```

This action uses the `openai.files.create` API to upload each document as a Blob, and then the `openai.beta.assistants.files.create` to attach the file to our assistant. You could also manually upload the files on the OpenAI dashboard.

When ready, we can upload and attach all documents from our table in bulk ([source](https://github.com/get-convex/convex-ai-chat-openai/blob/main/convex/init.ts#L30)).

## Ingest data: Splitting & embedding data

Nothing to do here, as OpenAI takes care of this step for us after we upload and attach the files automatically! 👏

## Serving traffic

Here’s a [sequence diagram](https://en.wikipedia.org/wiki/Sequence_diagram) for the traffic-serving portion of our app:

![overview of answer implementation](https://cdn.sanity.io/images/ts10onj4/production/b3d97e0c608ad248bce43b507bb4e83091a10d98-2750x1358.png)

## Serving traffic: answering a question

When a user hits send in our little chat box, we’ll kick off generating the answer ([source](https://github.com/get-convex/convex-ai-chat-openai/blob/main/convex/messages.ts)):

```jsx
export const send = mutation({
  args: {
    message: v.string(),
    sessionId: v.string(),
  },
  handler: async (ctx, { message, sessionId }) => {
    await ctx.db.insert("messages", {
      isViewer: true,
      text: message,
      sessionId,
    });
    await ctx.scheduler.runAfter(0, internal.serve.answer, {
      sessionId,
      message,
    });
  },
});
```

First, we save the message to the `"messages"` table, which will update our UI (if you’re not familiar with Convex and wonder how this works, head over to the [Convex tutorial](https://docs.convex.dev/get-started)).

The `scheduler.runAfter(0, …)` call is Convex parlance for “run an async job immediately”.

Let’s look at the implementation of the `answer` action ([source](https://github.com/get-convex/convex-ai-chat-openai/blob/main/convex/serve.ts)):

```tsx
export const answer = internalAction({
  args: {
    sessionId: v.string(),
    message: v.string(),
  },
  handler: async (ctx, { sessionId, message }) => {
    const openai = new OpenAI();

    const threadId = await getOrCreateThread(ctx, openai, sessionId);

    const { id: lastMessageId } = await openai.beta.threads.messages.create(
      threadId,
      { role: "user", content: message }
    );

    const { id: runId } = await openai.beta.threads.runs.create(threadId, {
      assistant_id: process.env.ASSISTANT_ID!,
    });

    await pollForAnswer(ctx, { threadId, sessionId, lastMessageId, runId });
  },
});
```

The steps we follow are:

1. Get or create an OpenAI thread for the current session
2. Add the user message to the OpenAI thread
3. Create an assistant run
4. Start polling for the answer

Polling is needed because as of the time of this writing, the Assistants API doesn’t support streaming or a single-shot async request.

Here’s the function performing the polling every 500ms[^1] ([source](https://github.com/get-convex/convex-ai-chat-openai/blob/main/convex/serve.ts)):

[^1]: You can also use the scheduler to do this, by re-scheduling the function every 500ms with `scheduler.runAfter(500, `. The while loop approach here is just for simplicity.

```tsx
async function pollForAnswer(
  ctx: ActionCtx,
  args: {
    sessionId: string;
    threadId: string;
    runId: string;
    lastMessageId: string;
  }
) {
  const { sessionId, threadId, runId, lastMessageId } = args;
  const openai = new OpenAI();
  while (true) {
    await sleep(500);
    const run = await openai.beta.threads.runs.retrieve(threadId, runId);
    switch (run.status) {
      case "failed":
      case "expired":
      case "cancelled":
        await ctx.runMutation(internal.serve.addMessage, {
          text: "I cannot reply at this time. Reach out to the team on Discord",
          sessionId,
        });
        return;
      case "completed": {
        const { data: newMessages } = await openai.beta.threads.messages.list(
          threadId,
          { after: lastMessageId, order: "asc" }
        );
        await map(newMessages, async ({ content }) => {
          const text = content
            .filter((item): item is MessageContentText => item.type === "text")
            .map(({ text }) => text.value)
            .join("\n\n");
          await ctx.runMutation(internal.serve.addMessage, { text, sessionId });
        });
        return;
      }
    }
  }
}
```

When the run is `"completed"` we want to store the new message in our `"messages"` table, which will update the UI. We do this using the `{ after: lastMessageId, order: "asc" }` arguments to the `openai.beta.threads.messages.list` API to get only the new messages, and then we save them to our database in the `addMessage` mutation ([source](https://github.com/get-convex/convex-ai-chat-openai/blob/main/convex/serve.ts#L113)).

## Serving traffic: listing messages

There’s only one more endpoint left to cover, and that’s the query that returns all messages for a given session ([source](https://github.com/get-convex/convex-ai-chat-openai/blob/main/convex/messages.ts)):

```jsx
export const list = query({
  args: {
    sessionId: v.string(),
  },
  handler: async (ctx, args) => {
    return await ctx.db
      .query("messages")
      .withIndex("bySessionId", (q) => q.eq("sessionId", args.sessionId))
      .collect();
  },
});
```

The client subscribes to this query and will get all messages for the session, automatically updating when new messages are added. Learn more about Convex reactivity [here](https://docs.convex.dev/tutorial/reactor).

And that’s it! The backend for our RAG chat is complete.

## Frontend Implementation

You can find the front-end implementation of our chat box [here](https://github.com/get-convex/convex-ai-chat-openai/blob/main/src/aiChat/index.tsx). It is only about 250 lines of React and Tailwind CSS, but it does pack a lot of functionality:

1. Realtime reactive updates with the LLM replies.
2. Preserving the session ID in `sessionStorage` (so it survives page reloads).
3. Scrolling to the bottom of the thread.
    1. Stop scrolling if the user scrolls up.
4. Opening the dialog in a React portal.
5. Disabling the send button when the input is empty.
6. Two different UIs: Small in the corner of the page and expanded.
7. Info icon with a tooltip for additional education/disclaimer.
8. Loading indicators
9. Dark mode

If you check out the [repo](https://github.com/get-convex/convex-ai-chat-openai/blob/main/src/App.tsx) you can see that Convex is only used for the chat modal. This is a good example if you want to drop the chat component into an existing website.

## Conclusion

OpenAI’s assistants API makes it quite easy to set up an AI chatbot with context retrieval. The API is in beta and some of its edges are a bit rough at the moment, but these issues will likely be quickly addressed. The Assistants API has additional capabilities we didn’t discuss, mainly the ability for the assistant to run custom functions. This might enable tighter integration with our product database. Alternatively, we can implement context retrieval directly on our server. We cover the first such implementation, using LangChain, in our [next post](https://stack.convex.dev/ai-chat-using-langchain-and-convex). To see how all of the implementations compare, see [this post](https://stack.convex.dev/are-vector-databases-dead).

On November 7th OpenAI released its Assistants API, enabling chat bot with context retrieval implementations without needing a messages or vector database. In this post, we’ll cover how to leverage this API to build a fully functioning AI chat interface.

Build AI Chat with OpenAI's Assistants API

Implementing user authentication (sign up, log in, log out) is hard! Authentication is an end-to-end feature involving secrets and credentials - the most sensitive kind of app development. For this reason it’s a good idea to outsource this problem to auth solution providers like [Clerk](https://clerk.dev/) or [Auth0](https://auth0.com/), or OAuth providers like Google or Github.

Unfortunately using a third-party comes with its own challenges: User data needs to be synced between the external service and your database. Auth solution providers need to be configured, can come with technical limitations and might charge a hefty sum once your app surpasses their free tiers.

Similarly, OAuth providers can be difficult to set up initially, and they can limit audience if your users do not have an account with a given provider.

For all these reasons you might want to implement authentication yourself. To simplify this, it’s a good idea to use an authentication library such as [Lucia](https://lucia-auth.com/), which will manage the logic around users, sessions and credentials for you.

With Lucia and Convex, we can build APIs reading current user information simply like this:

```jsx
import { queryWithAuth } from "./withAuth";

export const getLoggedInUserEmail = queryWithAuth({
  args: {},
  handler: async (ctx) => {
	  return ctx.session?.user.email;
  }
});
```

No external database or provider required!

We put together a template which includes sign up, log in and log out flows. Check out [the GitHub repo](https://github.com/get-convex/convex-lucia-auth-demo) or follow along with this article for the details.

## Update: Even easier integration

Run `npm create convex@latest -- -t nextjs-lucia-shadcn` to start a new project with a template that includes authentication based on Lucia, using the [convex-lucia-auth](https://www.npmjs.com/package/@convex-dev/convex-lucia-auth) NPM library.

The rest of this article goes over the manual integration of Lucia with Convex, which is still helpful for understanding and in case you want total control over configuring Lucia.

## Overview of the user experience

For this template we implemented a sign up / sign in flow which asks the user for their email and password. Once the user clicks `Sign Up` or `Sign In` their credentials are checked and they are either logged in or an error message is shown.

Once the user is logged-in we show them data associated with their account, in our case that’s their email, and a button that enables them to log out.

Let’s look at how this experience is implemented.

## Step 1: Specify our database schema

Although Convex doesn’t strictly require you to declare your database schema up front, in this case it will be helpful for us to look at what data we’ll be storing, and to get types since we’ll be using TypeScript for end-to-end type safety throughout our auth implementation.

```jsx
// convex/schema.ts

import { defineSchema, defineTable } from "convex/server";
import { Validator, v } from "convex/values";

export default defineSchema({
  users: defineTable({
    id: v.string(),
    email: v.string(),
  }).index("byId", ["id"]),
  sessions: defineTable({
    id: v.string(),
    user_id: v.string(),
    active_expires: v.float64(),
    idle_expires: v.float64(),
  })
    .index("byId", ["id"])
    .index("byUserId", ["user_id"]),
  auth_keys: defineTable({
    id: v.string(),
    hashed_password: v.union(v.string(), v.null()),
    user_id: v.string(),
  })
    .index("byId", ["id"])
    .index("byUserId", ["user_id"]),
});
```

We declare three tables:

- `users` which stores data for each user
- `sessions` which stores data for each session, mainly to do with its expiration
- and `auth_keys`, which represent different user credentials using any sign-in method

This schema will be used by our database adapter. It satisfies Lucia’s requirements. The only caveat to mention is that Lucia is currently architected to allocate IDs itself, while Convex allocates IDs automatically when documents are created. For this reason our users, sessions and auth_keys will have two ID fields: Convex allocated, built-in `_id` and Lucia allocated `id`.

For this demo we added an `email` field for storing the user’s email.

In our [template repo](https://github.com/get-convex/convex-lucia-auth-demo/blob/main/convex/schema.ts) we split the schema between the fields required by Lucia and custom fields, so our schema looks much simpler:

```jsx
export default defineSchema({
  ...authTables({
    user: { email: v.string() },
    session: {},
  }),
});
```

Let’s get on with implementing the database adapter that will enable us to take advantage of Lucia’s features.

## Step 2: Configure Lucia

We’ll want to interact with Lucia from our server queries and mutations. For this we’ll define a function which will instantiate Lucia, with our Convex adapter. This is the `getAuth` function:

```jsx
// lucia.ts
import { lucia } from "lucia";
import { DatabaseWriter } from "./_generated/server";

export function getAuth(db: DatabaseWriter) {
  return lucia({
    adapter: convexAdapter(db),
    // TODO: Set the LUCIA_ENVIRONMENT variable to "PROD"
    // on your prod deployment's dashboard
    env: (process.env.LUCIA_ENVIRONMENT as "PROD" | undefined) ?? "DEV",
    getUserAttributes(user: UserSchema) {
      return {
        _id: user._id,
        _creationTime: user._creationTime,
        email: user.email,
      };
    },
  });
}
```

We can call this function from any query* or mutation, passing in `ctx.db`. `convexAdapter` is the actual database adapter implementation. Check out its [source on Github](https://github.com/get-convex/convex-lucia-auth-demo/blob/main/convex/lucia.ts).

With Convex, implementing the database was straightforward. You can check Lucia’s adapter API docs [here](https://lucia-auth.com/extending-lucia/database-adapters-api).

In the `getUserAttributes` configuration we expose the `_id`, `_creationTime` and `email` documents fields to code loading users via `getAuth`.

*As you can see in the code, `getAuth` accepts `DatabaseWriter`.  This is the database interface exposed to Convex mutations. Convex queries can only read from the database, not write to it, and so they have access to `DatabaseReader` instead. When we call getAuth from a query helper in Step 3, our types won’t perfectly match runtime execution. Lucia’s API is not split between read and write operations, type-wise. We could declare a read-only subset of Lucia’s Auth API and return it from an additional `getReadOnlyAuth` function, but this won’t be necessary as we’ll only use a single, read-only, method in queries: `getSession`.

### Configuring types

Lucia has a cool way of injecting custom database types into its APIs. These are our definitions:

```jsx
// convex/env.d.ts

declare namespace Lucia {
  type Auth = import("./lucia").Auth;
  type DatabaseUserAttributes = {
    _id: import("./_generated/dataModel").Id<"users">;
    _creationTime: number;
    email: string;
  };
  type DatabaseSessionAttributes = {
    _id: import("./_generated/dataModel").Id<"sessions">;
    _creationTime: number;
  };
}
```

If we add more fields to our `"users"` table that we want to access through `getAuth`, we can add them here.

## Step 3: Leverage Lucia in our functions - queries

With the adapter and schema in place, it’s time to use them to implement our authentication logic.

Lucia includes middleware for storing and accessing session IDs on Requests and Responses. Since we want to use Convex’s reactive client which uses a WebSocket, we’ll instead pass the session ID through function arguments and return values.

Our simple “read” “middleware” will use the following function that returns a session if the given session ID is valid:

```jsx
import { getAuth } from "./lucia";
import { QueryCtx } from "./_generated/server";

async function getValidExistingSession(
  ctx: QueryCtx,
  sessionId: string | null
) {
  if (sessionId === null) {
    return null;
  }
  // The cast is OK because we will only expose the existing session
  const auth = getAuth(ctx.db as DatabaseWriter);
  try {
    const session = (await auth.getSession(sessionId)) as Session | null;
    if (session === null || session.state === "idle") {
      return null;
    }
    return session;
  } catch (error) {
    // Invalid session ID
    return null;
  }
}
```

With this helper we can implement a query that relies on the user information attached to a session:

```jsx
import { query } from "./_generated/server";

export const getLoggedInUserEmail = query({
  args: {
    sessionId: v.union(v.null(), v.string()),
    // other argument validators
  },
  handler: async (ctx, args) => {
    const session = await getValidExistingSession(ctx, args.sessionId);
    return session.user.email;
  },
});
```

Having to declare that we take a `sessionId` for every query that needs to use the session information would get old quite quickly. For this reason we implemented a `queryWithAuth` wrapper that does this for us. Check it out in the [repo](https://github.com/get-convex/convex-lucia-auth-demo/blob/main/convex/withAuth.ts). With it, accessing session is as simple as with Convex’s built-in auth. Our query `ctx` gets new `session` field:

```jsx
import { queryWithAuth } from "./withAuth";

export const getLoggedInUserEmail = queryWithAuth({
  args: {},
  handler: async (ctx) => {
	  return ctx.session?.user.email;
  }
});
```

We can access user information, but how does the information get into our DB in the first place?

## Step 4: Leverage Lucia in our functions - mutations

Armed with `getAuth` we can now implement our sign up and sign in mutations:

### Sign up

```jsx
import { v } from "convex/values";
import { getAuth } from "./lucia";

export const signUp = mutation({
  args: {
    email: v.string(),
    password: v.string(),
  },
  handler: async (ctx, { email, password }) => {
    const auth = getAuth(ctx.db);
    const user = await auth.createUser({
      key: {
        password: password,
        providerId: "password",
        providerUserId: email,
      },
      attributes: { email },
    });
    const session = await auth.createSession({
      userId: user.userId,
      attributes: {},
    });
    return session.sessionId;
  }
});
```

Check out the [Auth](https://lucia-auth.com/reference/lucia/interfaces/auth) API reference. Here we use the `createUser` and `createSession` methods to sign the user up and log them in. The `password` field holds the secret provided by the user (in our case, a password), `"password"` is our sign up method (we might add OAuth provider as a different method later), and `providerUserId` in our case is the user-filled out email.
Under the hood this method creates both a document in our `"users"` table and also a key in our `"auth_keys"` table.

We then create a session for the user and return its ID to the client. If an error occurs, such someone trying to sign up with an email that’s already taken, Lucia will throw and the error will propagate to the client.

### Sign in

```jsx
import { v } from "convex/values";
import { getAuth } from "./lucia";

export const signIn = mutation({
  args: {
    email: v.string(),
    password: v.string(),
  },
  handler: async (ctx, { email, password }) => {
    const auth = getAuth(ctx.db);
    const key = await auth.useKey("password", email, password);
    const session = await auth.createSession({
      userId: key.userId,
      attributes: {},
    });
    return session.sessionId;
  }
});
```

Check out the [Auth](https://lucia-auth.com/reference/lucia/interfaces/auth) API reference. Here we use `useKey` and `createSesson` to sign the user in. 

`useKey` takes the sign up method (`"password"` in our case) and the unique `providerUserId` and secret (in our case, the user provided password).

The cool thing is, you are in control of how the sign up / sign in works. The two mutations presented here are the standard approach, but you could for example combine sign up and sign in into a single form with a single mutation, or sign existing users in if they accidentally go through the sign up flow. It’s up to you!

### Other mutations

Just like with queries, we made a [wrapper for mutations](https://github.com/get-convex/convex-lucia-auth-demo/blob/main/convex/withAuth.ts) that makes it easy to check that a user is currently signed in. Since mutations can update the database, we will not only validate the session, but also renew it:

```jsx
import { Auth } from "./lucia";

async function getValidSessionAndRenew(auth: Auth, sessionId: string | null) {
  if (sessionId === null) {
    return null;
  }
  try {
    // Lucia's `validateSession` auto-renews the session
    return await auth.validateSession(sessionId);
  } catch (error) {
    // Invalid session ID
    return null;
  }
}

```

Our wrapper uses this helper under the hood to expose both `auth` and `session` on `ctx`:

```jsx

import { mutationWithAuth } from "./withAuth";

export const doSomethingAsTheUser = mutationWithAuth(
  args: {},
  handler: async (ctx) => {
	  if (ctx.session === null) {
	    throw new Error("Expected a logged-in user");
	  }
	  // do something...
		// ctx.auth is `LuciaAuth` instead of Convex's `Auth`
  },
});
```

You can see in the repo that we used this wrapper to simplify `signIn` and `signUp` mutations as well.

## Step 5: Frontend

### Keeping track of a session

We’ll be storing the secret sessionId in `localStorage`, so we create a simple React Context provider for updating and accessing it:

```jsx
import { createContext, useCallback, useContext, useState } from "react";

const SessionContext = createContext(undefined as any);

export function useSessionId(): string | null {
  return useContext(SessionContext).sessionId;
}

export function useSetSessionId(): (sessionId: string | null) => void {
  return useContext(SessionContext).setSessionId;
}

export function SessionProvider({ children }: { children: React.ReactNode }) {
  const [sessionId, setSessionIdState] = useState(getSavedSessionId());
  const setSessionId = useCallback(
    (value: string | null) => {
      setSavedSessionId(value);
      setSessionIdState(value);
    },
    [setSessionIdState]
  );
  return (
    <SessionContext.Provider value={{ sessionId, setSessionId }}>
      {children}
    </SessionContext.Provider>
  );
}

function getSavedSessionId() {
  return localStorage.getItem("sessionId");
}

export function setSavedSessionId(sessionId: string | null) {
  if (sessionId == null) {
    localStorage.removeItem("sessionId");
  } else {
    localStorage.setItem("sessionId", sessionId);
  }
}
```

And wrap our app in it:

```jsx
// src/main.tsx

ReactDOM.createRoot(document.getElementById("root")!).render(
  <React.StrictMode>
    <SessionProvider>
      <ConvexProvider client={convex}>
        <App />
      </ConvexProvider>
    </SessionProvider>
  </React.StrictMode>
);
```

Now we can use and set session ID from anywhere in our app:

```jsx
// src/App.tsx
import { useSessionId } from "./SessionProvider.tsx";
import { useQuery } from "convex/react";
import { LoginForm } from "./LoginForm";

function App() {
  const sessionId = useSessionId();
  const user = useQuery(api.users.get, {sessionId});

  return (
    <main>
      {user === undefined ? (
        <div>Loading...</div>
      ) : user == null ? (
        <LoginForm />
      ) : (
        <h2>Welcome {user.email}</h2>
      )}
    </main>
  );
}
```

Just like on the backend, we created a [wrapper](https://github.com/get-convex/convex-lucia-auth-demo/blob/main/src/withAuth.tsx) for `useQuery` that passes in the session ID automatically, so that we can we can simplify our code:

```jsx
// src/App.tsx

// Our custom `useQuery` is imported here
import { useQuery } from "./usingSession";

function App() {
  // sessionId is passed in automatically
  const user = useQuery(api.users.get);
  // ...
}
```

### Sign up / sign in form

Let’s see how we can implement the `LoginForm` component for sign up and sign in. This will be calling our `signUp` and `signIn` mutations.

```jsx
import { useState } from "react";
import { useSetSessionId } from "./SessionProvider";
import { useMutation } from "./usingSession";
import { api } from "../convex/_generated/api";

export function LoginForm() {
  const setSessionId = useSetSessionId();
  const [flow, setFlow] = useState<"signIn" | "signUp">("signIn");
  const signIn = useMutation(api.users.signIn);
  const signUp = useMutation(api.users.signUp);

  const handleSubmit = async (event: React.FormEvent<HTMLFormElement>) => {
    event.preventDefault();
    const data = new FormData(event.currentTarget);
    try {
      // Call the corresponding mutation
      const sessionId = await (flow === "signIn" ? signIn : signUp)({
        email: (data.get("email") as string | null) ?? "",
        password: (data.get("password") as string | null) ?? "",
      });
      // Set session ID in the Session React Context and localStorage
      setSessionId(sessionId);
    } catch {
      alert(
        flow === "signIn" ? "Invalid email or password" : "Email already in use"
      );
    }
  };
  return (
    <form onSubmit={handleSubmit}>
      <label htmlFor="username">Email</label>
      <input name="email" id="email" />
      <label htmlFor="password">Password</label>
      <input type="password" name="password" id="password" />
      {flow === "signIn" ? (
        <input type="submit" value="Sign in" />
      ) : (
        <input type="submit" value="Sign up" />
      )}
      <a
        style={{ cursor: "pointer" }}
        onClick={() => {
          setFlow(flow === "signIn" ? "signUp" : "signIn");
        }}
      >
        {flow === "signIn" ? (
          <>Don't have an account? Sign up</>
        ) : (
          <>Already have an account? Sign in</>
        )}
      </a>
    </form>
  );
}
```

Since they are so similar this single form implements both sign in and sign up. You could extend it in any way you like, for example by adding an OAuth sign up method (Google, Github). Lucia can help with this as well, check out [Lucia OAuth docs](https://lucia-auth.com/oauth).

## Conclusion and next steps

Well, that was a wild ride. As you can see even the simplest authentication implementation has a quite a few moving pieces. Good news is that you can start off by cloning our [example repo](https://github.com/get-convex/convex-lucia-auth-demo/blob/main/screenshot.png) with all of these pieces already hooked up.

You now have authentication that talks directly with the Convex database, and without any redirects or need for configuring third party services.

To take your app to production you have several options:

1. Implement optional email verification and password recovery flow. See Lucia’s guide on [email verification](https://lucia-auth.com/guidebook/email-verification-links). You’ll need a third-party (or your own) SMTP server for sending emails (check out [Resend](https://resend.com/)).
2. Switch to [Clerk](https://clerk.dev/) or [Auth0](https://auth0.com/) before you let external users interact with your app, throwing away your existing test signups.
3. Switch to [Clerk](https://clerk.dev/) or [Auth0](https://auth0.com/) later, importing existing signups into their database. See [Clerk import users doc](https://clerk.com/docs/deployments/import-users) or [Auth0 bulk user import doc](https://auth0.com/docs/manage-users/user-migration/bulk-user-imports).

If you’re using SSR with Next.js, Remix, or similar, you can use Lucia’s built-in [middleware](https://lucia-auth.com/basics/handle-requests) to use cookies instead of localStorage for storing sessions. Remember to protect your app against [CSRF attacks](https://github.com/get-convex/convex-lucia-auth-demo#note-on-csrf-protection).

Learn how to build a full stack app with authentication without any third-party auth providers, using Convex and the Lucia library.

Custom Authentication (with Lucia)

If you’re building a full-stack app, chances are you’ll want some of your users to pay you for the service you provide. [Stripe](https://stripe.com/) makes taking payments really easy, and with <QuickConvex /> you can ensure that you deliver when the payment comes in.

We put together a full working app that lets users pay to send a message to a global message board. Check out [the GitHub repo](https://github.com/get-convex/convex-stripe-demo) or follow along with this article that goes over the details.


## Overview of the user experience

Our UX for this example is simple:

1. The user types a message and hits the “Pay $1 and send” button.
2. They are redirected to a checkout site hosted by Stripe, where they fill out their payment details and hit the Pay button.
3. They are then redirected back to our web application, where they can see the message they paid for highlighted.
![Split view of Convex Paid Chat UI and Stripe checkout](https://cdn.sanity.io/images/ts10onj4/production/30a0bc857f303da2f397a20f5082ecfc96b51442-2400x1276.png?w=1200)

How do we know that the payment has been processed and the message can be sent? And how can we tell which message is the one the user just paid for? Read on to find out.

## Step 1: Initiate the checkout flow

To kick off the payment flow, we call a Convex action that does a couple of things for us:

1. It creates a document in our “payments” table so that we can track the progress of the checkout flow.
2. It calls Stripe’s SDK to get the checkout page URL the client should redirect to.
3. It includes the ID of our “payments” document in the `success_url` that Stripe will redirect to after the user has finished paying.
4. Finally, it writes the Stripe checkout session ID into the “payments” document to ensure we only fulfill the “order” once.

```jsx
// convex/stripe.ts

// This pragma is important because Stripe's SDK currently
// only works in the Node Runtime
"use node";

import { v } from "convex/values";
import { action } from "./_generated/server";
import Stripe from "stripe";
import { internal } from "./_generated/api";

export const pay = action({
  // The action takes the message the user composed
  args: { text: v.string() },
  handler: async (ctx, { text }) => {
    // We need to tell Stripe where to redirect to
    const domain = process.env.HOSTING_URL ?? "http://localhost:5173";
    const stripe = new Stripe(process.env.STRIPE_KEY!, {
      apiVersion: "2022-11-15",
    });
    // Here we create a document in the "payments" table
    const paymentId = await ctx.runMutation(internal.payments.create, { text });
    // This is where the Stripe checkout is configured
    const session = await stripe.checkout.sessions.create({
      line_items: [
        {
          // For this example we use dynamic `price_data`,
          // but you could use a predefined `price` ID instead
          price_data: {
            currency: "USD",
            unit_amount: 100,
            tax_behavior: "exclusive",
            product_data: {
              name: "One message of your choosing",
            },
          },
          quantity: 1,
        },
      ],
      mode: "payment",
      // This is how our web page will know which message we paid for
      success_url: `${domain}?paymentId=${paymentId}`,
      cancel_url: `${domain}`,
      automatic_tax: { enabled: true },
    });

    // Keep track of the checkout session ID for fulfillment
    await ctx.runMutation(internal.payments.markPending, {
      paymentId,
      stripeId: session.id,
    });
    // Let the client know the Stripe URL to redirect to
    return session.url;
  },
});
```

This action uses 2 internal mutations that perform writes to our database:

```jsx
// convex/payments.ts

import { v } from "convex/values";
import { internalMutation } from "./_generated/server";

export const create = internalMutation({
  args: { text: v.string() },
  handler: async (ctx, { text }) => {
    return await ctx.db.insert("payments", { text });
  },
});

export const markPending = internalMutation({
  args: {
    paymentId: v.id("payments"),
    stripeId: v.string(),
  },
  handler: async (ctx, { paymentId, stripeId }) => {
    await ctx.db.patch(paymentId, { stripeId });
  },
});
```

All that is left is to call this action when the user clicks the “Pay $1 and send” button and redirect to the returned URL:

```jsx
// src/App.tsx
import { FormEvent, useState } from "react";
import { useAction } from "convex/react";
import { api } from "../convex/_generated/api";

export default function App() {
  // State storing the message text
  const [newMessageText, setNewMessageText] = useState("");
  
  // Reference to our action
  const payAndSendMessage = useAction(api.stripe.pay);

  // We pass this handler to a `form` `onSubmit` in our UI code
  async function handleSendMessage(event: FormEvent) {
    event.preventDefault();
    // Call to our action
    const paymentUrl = await payAndSendMessage({ text: newMessageText });
    // Redirect to Stripe's checkout website
    window.location.href = paymentUrl!;
  }
  // the rest of our UI...
}
```

## Step 2: The payment checkout flow

Nothing to do here! Stripe did this work for us. 😮‍💨

## Step 3: Receiving the payment confirmation

Stripe will notify our server about the confirmed payment via a webhook request. All we need to do is expose an HTTP endpoint that Stripe’s servers can hit whenever a user finishes a payment. That’s a perfect fit for a Convex [HTTP action](https://docs.convex.dev/functions/http-actions). The HTTP action will handle the Stripe webhook request, parsing it, and passing its contents to another Convex action that will confirm the request is valid using the Stripe SDK in the Node.js runtime:

```jsx
// convex/http.ts
import { httpRouter } from "convex/server";
import { httpAction } from "./_generated/server";
import { internal } from "./_generated/api";

const http = httpRouter();

http.route({
  path: "/stripe",
  method: "POST",
  handler: httpAction(async (ctx, request) => {
    // Getting the stripe-signature header
    const signature: string = request.headers.get("stripe-signature") as string;
    // Calling the action that will perform our fulfillment
    const result = await ctx.runAction(internal.stripe.fulfill, {
      signature,
      payload: await request.text(),
    });
    if (result.success) {
      // We make sure to confirm the successful processing
      // so that Stripe can stop sending us the confirmation
      // of this payment.
      return new Response(null, {
        status: 200,
      });
    } else {
      // If something goes wrong Stripe will continue repeating
      // the same webhook request until we confirm it.
      return new Response("Webhook Error", {
        status: 400,
      });
    }
  }),
});

export default http;
```

Our `fulfill` action confirms the webhook request really came from Stripe via its SDK, gets the checkout session ID, and finally “fulfill”s the order by calling a database mutation:

```jsx
// convex/stripe.ts (continued)
export const fulfill = internalAction({
  args: { signature: v.string(), payload: v.string() },
  handler: async (ctx, { signature, payload }) => {
    const stripe = new Stripe(process.env.STRIPE_KEY!, {
      apiVersion: "2022-11-15",
    });

    const webhookSecret = process.env.STRIPE_WEBHOOKS_SECRET as string;
    try {
      // This call verifies the request
      const event = stripe.webhooks.constructEvent(
        payload,
        signature,
        webhookSecret
      );
      if (event.type === "checkout.session.completed") {
        const stripeId = (event.data.object as { id: string }).id;
        // Send the message and mark the payment as fulfilled
        await ctx.runMutation(internal.payments.fulfill, { stripeId });
      }
      return { success: true };
    } catch (err) {
      console.error(err);
      return { success: false };
    }
  },
});
```

The internal mutation makes writes to our database. It’s awesome that this mutation is automatically transactional, so if we made it this far, we are guaranteed to **both** mark the payment as fulfilled (by adding a `messageId` to it) **and** send the message (by adding it to the “messages” table).

```jsx
// convex/payments.ts (continued)
export const fulfill = internalMutation({
  args: { stripeId: v.string() },
  handler: async ({ db }, { stripeId }) => {
    const { _id: paymentId, text } = (await db
      .query("payments")
      .withIndex("stripeId", (q) => q.eq("stripeId", stripeId))
      .unique())!;
    const messageId = await db.insert("messages", { text });
    await db.patch(paymentId, { messageId });
  },
});
```

At this point, the payment and fulfillment flow works end-to-end.

## Bonus: Tying it all together

Our demo includes one more trick for highlighting the message the user just paid for. This could be used to show a unique order confirmation, even if the user made 2 purchases in parallel (from two different browser tabs).

Back in step 1, we added the “payments” document ID to the URL Stripe redirects to after a successful checkout, the `success_url`.

We can use this information from the URL to highlight the relevant message:

```jsx
// src/App.tsx

// A simple React hook that grabs a URL query parameter and erases
// it from the URL.
function useConsumeQueryParam(name: string) {
  const [value] = useState(
    // get the param value
    new URLSearchParams(window.location.search).get(name)
  );

  useEffect(() => {
    const currentUrl = new URL(window.location.href);
    const searchParams = currentUrl.searchParams;
    searchParams.delete(name);
    const consumedUrl =
      currentUrl.origin + currentUrl.pathname + searchParams.toString();
    // reset the URL
    window.history.replaceState(null, "", consumedUrl);
  }, []);
  return value;
}

export default function App() {
  // use the hook to get the "payments" document ID
  const paymentId = useConsumeQueryParam("paymentId");
  // Call a query to exchange the "payments" document ID
  // for the corresponding "messages" document ID
  const sentMessageId = useQuery(api.payments.getMessageId, {
    paymentId: (paymentId ?? undefined) as Id<"payments"> | undefined,
  });
  const messages = useQuery(api.messages.list) || [];
  // render a list of messages
  return (
    <>
      <ul>
        {messages.map((message) => (
          <li
            key={message._id}
            // highlight the message the user just paid for, if any
            className={sentMessageId === message._id ? "sent" : ""}
          >
            <span>{message.text}</span>
          </li>
        ))}
      </ul>
      {* ... *}
    </>
  );
}
```

Refer to the [the Github repo](https://github.com/get-convex/convex-stripe-demo) for the full implementation.

## It was ninety-nine cents!
![Convex $ Stripe from Macklemore's Thrift Shop Music Video](https://cdn.sanity.io/images/ts10onj4/production/331ef4de890fc6db7270439d62592c0d98613726-947x499.jpg)

While most people won’t pay a dollar to send a message to a random message board, our demo app showcases all the necessary pieces for integrating a full-fledged payment flow. Convex and Stripe make this flow feel solid and reliable. Let us know [in Discord](https://convex.dev/community) what your app charges for 🤑

If you’re building a full-stack app, chances are you’ll want some of your users to pay you for the service you provide. How to use Stripe with Convex to get it done.

Michal Srb

All posts by Michal