In case you haven’t heard the [buzz](https://news.ycombinator.com/item?id=37128293), some cool new kids just arrived on the block! We’re talking, of course, about the AI-powered virtual residents of [AI Town](http://github.com/get-convex/ai-town), a multiplayer conversational simulation built with [Convex](https://convex.dev) (hey, that’s us!), [OpenAI](https://openai.com), and [Pinecone](https://pinecone.io). 

Visit the [live demo](http://convex.dev/ai-town) to watch in awe as the game’s autonomous agents, each of whom has their own personality, memory, and goals, wander freely around the pixelated map and have eerily lifelike conversations with each other. And check out this video to learn how it all works:

<Video videoId="UoJjeyQR66s" />

If you’re anything like us, you could spend all day spying on these eerily lifelike characters and pondering the nature of consciousness. But you’re probably also wondering how you can make your mark on this virtual town, asking yourself (or the [community](https://discord.gg/PQUmTBTGmT)) questions like:

- What gives these characters their unique interests, attitudes & perspectives?
- How can I change their personalities, or add entirely new characters of my own?
- Do I need to be an expert software developer to make my mark on this virtual town? (spoiler: nope!)

In this post, we’ll answer these questions so that you can customize your own AI Town, where you get to be the wizard behind the curtain! So grab your keyboard & magic wand and join us in hacking the source code of this adorable matrix.

Before we begin: follow the directions in the project [README](https://github.com/get-convex/ai-town/blob/main/README.md) to spin up a local AI Town.

# OpenAI think, therefore I am

Behind the scenes of AI Town, the heavy lifting is done by the increasingly impressive large language models (generally known as LLMs) available through the OpenAI API. They are responsible for the characters’ “thoughts”, helping each agent decide what to say next in the conversation based on text prompts that convey: 

1. the agent’s personality, relationships, and goals
2. general directives about how all conversations should proceed
3. the agent’s memories about their past conversations

In this post, we’ll focus on factor #1, so you can quickly put your own spin on the town’s colorful cast of characters. But in future blog posts, we’ll dive into the other factors, including how agents’ memories work - so stay tuned! 

# Accessing agent personalities

If you spend a little time in AI Town, you’ll quickly notice the distinct perspectives, interests, and dispositions that set the agents apart. These “personalities” are essentially predefined descriptions for each agent, which state in plain text who they are & what they’re into. 

You can find the list of agents and the description for each one in `convex/characterData/data.ts`. In the array called `Descriptions`, you’ll see a list of objects like this:

```json
{
    name: 'Alex',
    character: 'f5',
    memories: [
      {
        type: 'identity' as const,
        description: `You are a fictional character whose name is Alex.  You enjoy painting,
      programming and reading sci-fi books.  You are currently talking to a human who
      is very interested to get to know you. You are kind but can be sarcastic. You
      dislike repetitive questions. You get SUPER excited about books.`,
      },
      {
        type: 'relationship' as const,
        description: 'You like lucky',
        playerName: 'Lucky',
      },
      {
        type: 'plan' as const,
        description: 'You want to find love.',
      },
    ],
    position: { x: 10, y: 10 },
 }
```

Let’s break down these properties: 

- `name` : the character’s first name
- `character`: the [sprite](https://en.wikipedia.org/wiki/Sprite_(computer_graphics)), or set of graphics, corresponding to this character (each refers to a specific subset of the character [spritesheet](https://github.com/get-convex/ai-town/blob/main/convex/characterdata/spritesheets/f5.ts) found at `public/assets/32x32folk.png`)
- `position`: the character’s starting coordinates on the town map
- `memories`: chunks of information about this character,  each with a specific `type`:
    - `identity`: a first- or second-person description of who this character is, what they’re interested in, what kind of attitude they have, etc.
    - `plan`: a second-person description of the character’s main goal in “life”
    - `relationship`: an optional, second-person declaration of how this character feels about someone else in the town, referencing the other character’s `playerName`

# A whole new world

The information in `convex/characterData/data.ts` is used to create a `world` for the town, which includes all the characters and their rich inner lives. So in order to change the world, we have to blow it up and start fresh with a new big bang.

Assuming you’ve already followed the README instructions to spin up a local version of the application, follow these steps to become developer, destroyer (and re-creator) of worlds:

1. If you’re already running `npx run dev` in a terminal, abort with `CTRL-C`
2. Run this command in a terminal to erase the humdrum default characters (and everything else!) from your town’s database and start fresh [WARNING this will delete ALL the data in your Convex project!]: 
`npx convex run testing:debugClearAll`
3. Edit `convex/characterData/data.ts` as needed (see exercises below)
4. Re-populate your town’s database, including the new-and-improved characters, with the command `npm run dev`
5. Visit the URL `localhost:3000` in your browser to take a peek into the world you’ve created. Click on a character and you should see their identity description and the content of their conversations reflect the thoughts you put in their head.

# Exercise: Total recall

If you edit the various `description` texts for a given agent’s memories, you’ll be effectively changing who they are and how they interact with others. (We’ll leave the ensuing philosophical inquiry into the nature of identity as a follow-up exercise for the reader.)

Use your inner creative genius to edit the `memories` in `convex/characterData/data.ts` as you see fit. Leave each memory’s `type` line as-is, and only edit the bits of text to the right of each `description` (and optionally `playerName`, if you’re messing with the character’s relationships too).

Let’s do one together. First, add this to the end of Alex’s description:

> It's very important to you to convince others that Pixar's Wall-E is the greatest movie ever made.
> 

Then, we need to reset the game state as outlined above. And sure enough, within a few minutes, Alex is out there pitching Wall-E:

![Picture of Alex pitching Wall-E](https://cdn.sanity.io/images/ts10onj4/production/73334be925b8a8d8306dea25b6ee63b19bc35d4f-2168x1094.png?w=960 "Alex with the not-so-subtle segue.")

# Exercise: Create a life, without going into labor

To generate even more excitement in town, you can help a newcomer move in by adding a new object to the `Descriptions` array in `convex/characterData/data.ts`. You can use the `Alex` object above as a template, and keep in mind: 

- `name` should be unique within the town
- `character` can duplicate one of the existing `f` values from `f1` to `f8`
- `position` should not have the exact same `x` and `y` as another agent , or they’ll both occupy the same spacetime and unravel the universe! …or just behave strangely

For example, your `Descriptions` array might look something like this:

```json
export const Descriptions = [
  {
    name: 'Newman',
    character: 'f5',
    memories: [
      {
        type: 'identity' as const,
        description: `Your name is Newman. You are a deeply unhappy person whose only joy in life comes from the suffering of others. You have an irrational superiority complex as well as a mysterious phobia of dinosaurs.`,
      },
      {
        type: 'relationship' as const,
        description: 'You despise Jerry and everything he does.',
        playerName: 'Jerry',
      },
      {
        type: 'plan' as const,
        description: 'You want to make Jerry cry.',
      },
    ],
    position: { x: 2, y: 3 },
  },
  {
    name: 'Jerry',
    character: 'f3',
    memories: [
      {
        type: 'identity' as const,
        description: `You are a comedian who constantly uses humour to mask your deep insecurity. You don't have many friends, and you're not always kind to the few friends you do have. You go on a lot of dates with women, but always find something about them that annoys you. You love soup and hate wearing puffy shirts that make you look like a pirate.`,
      },
      {
        type: 'plan' as const,
        description: 'You want to figure out what the deal is.',
      },
    ],
    position: { x: 5, y: 6 },
  },
//.... all the other characters you tweaked before
];
```

After resetting and rerunning the game, watch Newman and Jerry go to work:

![Picture of Newman and Jerry talking in AI Town](https://cdn.sanity.io/images/ts10onj4/production/e176c0f5d0236c14b0dab77979fe5519f663a03d-2270x892.png?w=960 "Newman (who looks suspiciously like Alex) has no qualms about his passion for the misery of others.")

# Do not weep, for there are more worlds to conquer

Congratulations on your first foray into becoming the wonderful wizard of ~~Oz~~ AI Town! Let’s recap your rise to power:

- You learned the secrets of where & how AI Town agents’ personalities & relationships are defined, in `convex/characterData/data.ts`
- You pulled the right strings (err… edited the right `string`s) to turn the default agents into the higher selves you always knew they could become
- You created (artificial) life using nothing but your willpower and web technology

But what does the future hold for your AI Town? What should you tackle next, now that you’ve conquered this realm?

**Share your triumphs:** Are you ready to brag about the virtual children you’ve created? Are they having fascinating chats thanks to the sparkling personalities you gave them? Shout it from the rooftops on the [Convex Discord](https://convex.dev/community).

**Be the Bond villain:** Is your inner evil genius already thinking up fabulous new ways to wield your newfound power? Do you have a diabolical plan to take over the virtual world? Make like every foe of 007 and reveal your secret agenda to the community before you put it into practice! Other [AI Town enthusiasts](https://discord.gg/PQUmTBTGmT) would love to plot and scheme with you.

**Keep your finger on the pulse**: Are you hungry for more world-altering knowledge? Do you want to be the first to know about any new developments in AI Town? Follow Convex on [Twitter](https://twitter.com/convex_dev) and [YouTube](https://www.youtube.com/channel/UCoC_9mdiPwIu1sDxDtGQggQ) to ensure you never miss the latest town gossip.

Learn more about how AI Town works and how to customize it. In this post, we'll dig into how character identities are represented: how to alter them, and how to add entirely new characters.

Building AI Town's Backend: Character Identities

Implementing user authentication (sign up, log in, log out) is hard! Authentication is an end-to-end feature involving secrets and credentials - the most sensitive kind of app development. For this reason it’s a good idea to outsource this problem to auth solution providers like [Clerk](https://clerk.dev/) or [Auth0](https://auth0.com/), or OAuth providers like Google or Github.

Unfortunately using a third-party comes with its own challenges: User data needs to be synced between the external service and your database. Auth solution providers need to be configured, can come with technical limitations and might charge a hefty sum once your app surpasses their free tiers.

Similarly, OAuth providers can be difficult to set up initially, and they can limit audience if your users do not have an account with a given provider.

For all these reasons you might want to implement authentication yourself. To simplify this, it’s a good idea to use an authentication library such as [Lucia](https://lucia-auth.com/), which will manage the logic around users, sessions and credentials for you.

With Lucia and Convex, we can build APIs reading current user information simply like this:

```jsx
import { queryWithAuth } from "./withAuth";

export const getLoggedInUserEmail = queryWithAuth({
  args: {},
  handler: async (ctx) => {
	  return ctx.session?.user.email;
  }
});
```

No external database or provider required!

We put together a template which includes sign up, log in and log out flows. Check out [the GitHub repo](https://github.com/get-convex/convex-lucia-auth-demo) or follow along with this article for the details.

## Overview of the user experience

For this template we implemented a sign up / sign in flow which asks the user for their email and password. Once the user clicks `Sign Up` or `Sign In` their credentials are checked and they are either logged in or an error message is shown.

Once the user is logged-in we show them data associated with their account, in our case that’s their email, and a button that enables them to log out.

Let’s look at how this experience is implemented.

## Step 1: Specify our database schema

Although Convex doesn’t strictly require you to declare your database schema up front, in this case it will be helpful for us to look at what data we’ll be storing, and to get types since we’ll be using TypeScript for end-to-end type safety throughout our auth implementation.

```jsx
// convex/schema.ts

import { defineSchema, defineTable } from "convex/server";
import { Validator, v } from "convex/values";

export default defineSchema({
  users: defineTable({
    id: v.string(),
    email: v.string(),
  }).index("byId", ["id"]),
  sessions: defineTable({
    id: v.string(),
    user_id: v.string(),
    active_expires: v.float64(),
    idle_expires: v.float64(),
  })
    .index("byId", ["id"])
    .index("byUserId", ["user_id"]),
  auth_keys: defineTable({
    id: v.string(),
    hashed_password: v.union(v.string(), v.null()),
    user_id: v.string(),
  })
    .index("byId", ["id"])
    .index("byUserId", ["user_id"]),
});
```

We declare three tables:

- `users` which stores data for each user
- `sessions` which stores data for each session, mainly to do with its expiration
- and `auth_keys`, which represent different user credentials using any sign-in method

This schema will be used by our database adapter. It satisfies Lucia’s requirements. The only caveat to mention is that Lucia is currently architected to allocate IDs itself, while Convex allocates IDs automatically when documents are created. For this reason our users, sessions and auth_keys will have two ID fields: Convex allocated, built-in `_id` and Lucia allocated `id`.

For this demo we added an `email` field for storing the user’s email.

In our [template repo](https://github.com/get-convex/convex-lucia-auth-demo/blob/main/convex/schema.ts) we split the schema between the fields required by Lucia and custom fields, so our schema looks much simpler:

```jsx
export default defineSchema({
  ...authTables({
    user: { email: v.string() },
    session: {},
  }),
});
```

Let’s get on with implementing the database adapter that will enable us to take advantage of Lucia’s features.

## Step 2: Configure Lucia

We’ll want to interact with Lucia from our server queries and mutations. For this we’ll define a function which will instantiate Lucia, with our Convex adapter. This is the `getAuth` function:

```jsx
// lucia.ts
import { lucia } from "lucia";
import { DatabaseWriter } from "./_generated/server";

export function getAuth(db: DatabaseWriter) {
  return lucia({
    adapter: convexAdapter(db),
    // TODO: Set the LUCIA_ENVIRONMENT variable to "PROD"
    // on your prod deployment's dashboard
    env: (process.env.LUCIA_ENVIRONMENT as "PROD" | undefined) ?? "DEV",
    getUserAttributes(user: UserSchema) {
      return {
        _id: user._id,
        _creationTime: user._creationTime,
        email: user.email,
      };
    },
  });
}
```

We can call this function from any query* or mutation, passing in `ctx.db`. `convexAdapter` is the actual database adapter implementation. Check out its [source on Github](https://github.com/get-convex/convex-lucia-auth-demo/blob/main/convex/lucia.ts).

With Convex, implementing the database was straightforward. You can check Lucia’s adapter API docs [here](https://lucia-auth.com/extending-lucia/database-adapters-api).

In the `getUserAttributes` configuration we expose the `_id`, `_creationTime` and `email` documents fields to code loading users via `getAuth`.

*As you can see in the code, `getAuth` accepts `DatabaseWriter`.  This is the database interface exposed to Convex mutations. Convex queries can only read from the database, not write to it, and so they have access to `DatabaseReader` instead. When we call getAuth from a query helper in Step 3, our types won’t perfectly match runtime execution. Lucia’s API is not split between read and write operations, type-wise. We could declare a read-only subset of Lucia’s Auth API and return it from an additional `getReadOnlyAuth` function, but this won’t be necessary as we’ll only use a single, read-only, method in queries: `getSession`.

### Configuring types

Lucia has a cool way of injecting custom database types into its APIs. These are our definitions:

```jsx
// convex/env.d.ts

declare namespace Lucia {
  type Auth = import("./lucia").Auth;
  type DatabaseUserAttributes = {
    _id: import("./_generated/dataModel").Id<"users">;
    _creationTime: number;
    email: string;
  };
  type DatabaseSessionAttributes = {
    _id: import("./_generated/dataModel").Id<"sessions">;
    _creationTime: number;
  };
}
```

If we add more fields to our `"users"` table that we want to access through `getAuth`, we can add them here.

## Step 3: Leverage Lucia in our functions - queries

With the adapter and schema in place, it’s time to use them to implement our authentication logic.

Lucia includes middleware for storing and accessing session IDs on Requests and Responses. Since we want to use Convex’s reactive client which uses a WebSocket, we’ll instead pass the session ID through function arguments and return values.

Our simple “read” “middleware” will use the following function that returns a session if the given session ID is valid:

```jsx
import { getAuth } from "./lucia";
import { QueryCtx } from "./_generated/server";

async function getValidExistingSession(
  ctx: QueryCtx,
  sessionId: string | null
) {
  if (sessionId === null) {
    return null;
  }
  // The cast is OK because we will only expose the existing session
  const auth = getAuth(ctx.db as DatabaseWriter);
  try {
    const session = (await auth.getSession(sessionId)) as Session | null;
    if (session === null || session.state === "idle") {
      return null;
    }
    return session;
  } catch (error) {
    // Invalid session ID
    return null;
  }
}
```

With this helper we can implement a query that relies on the user information attached to a session:

```jsx
import { query } from "./_generated/server";

export const getLoggedInUserEmail = query({
  args: {
    sessionId: v.union(v.null(), v.string()),
    // other argument validators
  },
  handler: async (ctx, args) => {
    const session = await getValidExistingSession(ctx, args.sessionId);
    return session.user.email;
  },
});
```

Having to declare that we take a `sessionId` for every query that needs to use the session information would get old quite quickly. For this reason we implemented a `queryWithAuth` wrapper that does this for us. Check it out in the [repo](https://github.com/get-convex/convex-lucia-auth-demo/blob/main/convex/withAuth.ts). With it, accessing session is as simple as with Convex’s built-in auth. Our query `ctx` gets new `session` field:

```jsx
import { queryWithAuth } from "./withAuth";

export const getLoggedInUserEmail = queryWithAuth({
  args: {},
  handler: async (ctx) => {
	  return ctx.session?.user.email;
  }
});
```

We can access user information, but how does the information get into our DB in the first place?

## Step 4: Leverage Lucia in our functions - mutations

Armed with `getAuth` we can now implement our sign up and sign in mutations:

### Sign up

```jsx
import { v } from "convex/values";
import { getAuth } from "./lucia";

export const signUp = mutation({
  args: {
    email: v.string(),
    password: v.string(),
  },
  handler: async (ctx, { email, password }) => {
    const auth = getAuth(ctx.db);
    const user = await auth.createUser({
      key: {
        password: password,
        providerId: "password",
        providerUserId: email,
      },
      attributes: { email },
    });
    const session = await auth.createSession({
      userId: user.userId,
      attributes: {},
    });
    return session.sessionId;
  }
});
```

Check out the [Auth](https://lucia-auth.com/reference/lucia/interfaces/auth) API reference. Here we use the `createUser` and `createSession` methods to sign the user up and log them in. The `password` field holds the secret provided by the user (in our case, a password), `"password"` is our sign up method (we might add OAuth provider as a different method later), and `providerUserId` in our case is the user-filled out email.
Under the hood this method creates both a document in our `"users"` table and also a key in our `"auth_keys"` table.

We then create a session for the user and return its ID to the client. If an error occurs, such someone trying to sign up with an email that’s already taken, Lucia will throw and the error will propagate to the client.

### Sign in

```jsx
import { v } from "convex/values";
import { getAuth } from "./lucia";

export const signIn = mutation({
  args: {
    email: v.string(),
    password: v.string(),
  },
  handler: async (ctx, { email, password }) => {
    const auth = getAuth(ctx.db);
    const key = await auth.useKey("password", email, password);
    const session = await auth.createSession({
      userId: key.userId,
      attributes: {},
    });
    return session.sessionId;
  }
});
```

Check out the [Auth](https://lucia-auth.com/reference/lucia/interfaces/auth) API reference. Here we use `useKey` and `createSesson` to sign the user in. 

`useKey` takes the sign up method (`"password"` in our case) and the unique `providerUserId` and secret (in our case, the user provided password).

The cool thing is, you are in control of how the sign up / sign in works. The two mutations presented here are the standard approach, but you could for example combine sign up and sign in into a single form with a single mutation, or sign existing users in if they accidentally go through the sign up flow. It’s up to you!

### Other mutations

Just like with queries, we made a [wrapper for mutations](https://github.com/get-convex/convex-lucia-auth-demo/blob/main/convex/withAuth.ts) that makes it easy to check that a user is currently signed in. Since mutations can update the database, we will not only validate the session, but also renew it:

```jsx
import { Auth } from "./lucia";

async function getValidSessionAndRenew(auth: Auth, sessionId: string | null) {
  if (sessionId === null) {
    return null;
  }
  try {
    // Lucia's `validateSession` auto-renews the session
    return await auth.validateSession(sessionId);
  } catch (error) {
    // Invalid session ID
    return null;
  }
}

```

Our wrapper uses this helper under the hood to expose both `auth` and `session` on `ctx`:

```jsx

import { mutationWithAuth } from "./withAuth";

export const doSomethingAsTheUser = mutationWithAuth(
  args: {},
  handler: async (ctx) => {
	  if (ctx.session === null) {
	    throw new Error("Expected a logged-in user");
	  }
	  // do something...
		// ctx.auth is `LuciaAuth` instead of Convex's `Auth`
  },
});
```

You can see in the repo that we used this wrapper to simplify `signIn` and `signUp` mutations as well.

## Step 5: Frontend

### Keeping track of a session

We’ll be storing the secret sessionId in `localStorage`, so we create a simple React Context provider for updating and accessing it:

```jsx
import { createContext, useCallback, useContext, useState } from "react";

const SessionContext = createContext(undefined as any);

export function useSessionId(): string | null {
  return useContext(SessionContext).sessionId;
}

export function useSetSessionId(): (sessionId: string | null) => void {
  return useContext(SessionContext).setSessionId;
}

export function SessionProvider({ children }: { children: React.ReactNode }) {
  const [sessionId, setSessionIdState] = useState(getSavedSessionId());
  const setSessionId = useCallback(
    (value: string | null) => {
      setSavedSessionId(value);
      setSessionIdState(value);
    },
    [setSessionIdState]
  );
  return (
    <SessionContext.Provider value={{ sessionId, setSessionId }}>
      {children}
    </SessionContext.Provider>
  );
}

function getSavedSessionId() {
  return localStorage.getItem("sessionId");
}

export function setSavedSessionId(sessionId: string | null) {
  if (sessionId == null) {
    localStorage.removeItem("sessionId");
  } else {
    localStorage.setItem("sessionId", sessionId);
  }
}
```

And wrap our app in it:

```jsx
// src/main.tsx

ReactDOM.createRoot(document.getElementById("root")!).render(
  <React.StrictMode>
    <SessionProvider>
      <ConvexProvider client={convex}>
        <App />
      </ConvexProvider>
    </SessionProvider>
  </React.StrictMode>
);
```

Now we can use and set session ID from anywhere in our app:

```jsx
// src/App.tsx
import { useSessionId } from "./SessionProvider.tsx";
import { useQuery } from "convex/react";
import { LoginForm } from "./LoginForm";

function App() {
  const sessionId = useSessionId();
  const user = useQuery(api.users.get, {sessionId});

  return (
    <main>
      {user === undefined ? (
        <div>Loading...</div>
      ) : user == null ? (
        <LoginForm />
      ) : (
        <h2>Welcome {user.email}</h2>
      )}
    </main>
  );
}
```

Just like on the backend, we created a [wrapper](https://github.com/get-convex/convex-lucia-auth-demo/blob/main/src/withAuth.tsx) for `useQuery` that passes in the session ID automatically, so that we can we can simplify our code:

```jsx
// src/App.tsx

// Our custom `useQuery` is imported here
import { useQuery } from "./usingSession";

function App() {
  // sessionId is passed in automatically
  const user = useQuery(api.users.get);
  // ...
}
```

### Sign up / sign in form

Let’s see how we can implement the `LoginForm` component for sign up and sign in. This will be calling our `signUp` and `signIn` mutations.

```jsx
import { useState } from "react";
import { useSetSessionId } from "./SessionProvider";
import { useMutation } from "./usingSession";
import { api } from "../convex/_generated/api";

export function LoginForm() {
  const setSessionId = useSetSessionId();
  const [flow, setFlow] = useState<"signIn" | "signUp">("signIn");
  const signIn = useMutation(api.users.signIn);
  const signUp = useMutation(api.users.signUp);

  const handleSubmit = async (event: React.FormEvent<HTMLFormElement>) => {
    event.preventDefault();
    const data = new FormData(event.currentTarget);
    try {
      // Call the corresponding mutation
      const sessionId = await (flow === "signIn" ? signIn : signUp)({
        email: (data.get("email") as string | null) ?? "",
        password: (data.get("password") as string | null) ?? "",
      });
      // Set session ID in the Session React Context and localStorage
      setSessionId(sessionId);
    } catch {
      alert(
        flow === "signIn" ? "Invalid email or password" : "Email already in use"
      );
    }
  };
  return (
    <form onSubmit={handleSubmit}>
      <label htmlFor="username">Email</label>
      <input name="email" id="email" />
      <label htmlFor="password">Password</label>
      <input type="password" name="password" id="password" />
      {flow === "signIn" ? (
        <input type="submit" value="Sign in" />
      ) : (
        <input type="submit" value="Sign up" />
      )}
      <a
        style={{ cursor: "pointer" }}
        onClick={() => {
          setFlow(flow === "signIn" ? "signUp" : "signIn");
        }}
      >
        {flow === "signIn" ? (
          <>Don't have an account? Sign up</>
        ) : (
          <>Already have an account? Sign in</>
        )}
      </a>
    </form>
  );
}
```

Since they are so similar this single form implements both sign in and sign up. You could extend it in any way you like, for example by adding an OAuth sign up method (Google, Github). Lucia can help with this as well, check out [Lucia OAuth docs](https://lucia-auth.com/oauth).

## Conclusion and next steps

Well, that was a wild ride. As you can see even the simplest authentication implementation has a quite a few moving pieces. Good news is that you can start off by cloning our [example repo](https://github.com/get-convex/convex-lucia-auth-demo/blob/main/screenshot.png) with all of these pieces already hooked up.

You now have authentication that talks directly with the Convex database, and without any redirects or need for configuring third party services.

To take your app to production you have several options:

1. Implement optional email verification and password recovery flow. See Lucia’s guide on [email verification](https://lucia-auth.com/guidebook/email-verification-links). You’ll need a third-party (or your own) SMTP server for sending emails (check out [Resend](https://resend.com/)).
2. Switch to [Clerk](https://clerk.dev/) or [Auth0](https://auth0.com/) before you let external users interact with your app, throwing away your existing test signups.
3. Switch to [Clerk](https://clerk.dev/) or [Auth0](https://auth0.com/) later, importing existing signups into their database. See [Clerk import users doc](https://clerk.com/docs/deployments/import-users) or [Auth0 bulk user import doc](https://auth0.com/docs/manage-users/user-migration/bulk-user-imports).

If you’re using SSR with Next.js, Remix, or similar, you can use Lucia’s built-in [middleware](https://lucia-auth.com/basics/handle-requests) to use cookies instead of localStorage for storing sessions. Remember to protect your app against [CSRF attacks](https://github.com/get-convex/convex-lucia-auth-demo#note-on-csrf-protection).

Learn how to build a full stack app with authentication without any third-party auth providers, using Convex and the Lucia library.

Custom Authentication (with Lucia)

If you’re building a full-stack app, chances are you’ll want some of your users to pay you for the service you provide. [Stripe](https://stripe.com/) makes taking payments really easy, and with <QuickConvex /> you can ensure that you deliver when the payment comes in.

We put together a full working app that lets users pay to send a message to a global message board. Check out [the GitHub repo](https://github.com/get-convex/convex-stripe-demo) or follow along with this article that goes over the details.


## Overview of the user experience

Our UX for this example is simple:

1. The user types a message and hits the “Pay $1 and send” button.
2. They are redirected to a checkout site hosted by Stripe, where they fill out their payment details and hit the Pay button.
3. They are then redirected back to our web application, where they can see the message they paid for highlighted.
![Split view of Convex Paid Chat UI and Stripe checkout](https://cdn.sanity.io/images/ts10onj4/production/30a0bc857f303da2f397a20f5082ecfc96b51442-2400x1276.png?w=1200)

How do we know that the payment has been processed and the message can be sent? And how can we tell which message is the one the user just paid for? Read on to find out.

## Step 1: Initiate the checkout flow

To kick off the payment flow, we call a Convex action that does a couple of things for us:

1. It creates a document in our “payments” table so that we can track the progress of the checkout flow.
2. It calls Stripe’s SDK to get the checkout page URL the client should redirect to.
3. It includes the ID of our “payments” document in the `success_url` that Stripe will redirect to after the user has finished paying.
4. Finally, it writes the Stripe checkout session ID into the “payments” document to ensure we only fulfill the “order” once.

```jsx
// convex/stripe.ts

// This pragma is important because Stripe's SDK currently
// only works in the Node Runtime
"use node";

import { v } from "convex/values";
import { action } from "./_generated/server";
import Stripe from "stripe";
import { internal } from "./_generated/api";

export const pay = action({
  // The action takes the message the user composed
  args: { text: v.string() },
  handler: async (ctx, { text }) => {
    // We need to tell Stripe where to redirect to
    const domain = process.env.HOSTING_URL ?? "http://localhost:5173";
    const stripe = new Stripe(process.env.STRIPE_KEY!, {
      apiVersion: "2022-11-15",
    });
    // Here we create a document in the "payments" table
    const paymentId = await ctx.runMutation(internal.payments.create, { text });
    // This is where the Stripe checkout is configured
    const session = await stripe.checkout.sessions.create({
      line_items: [
        {
          // For this example we use dynamic `price_data`,
          // but you could use a predefined `price` ID instead
          price_data: {
            currency: "USD",
            unit_amount: 100,
            tax_behavior: "exclusive",
            product_data: {
              name: "One message of your choosing",
            },
          },
          quantity: 1,
        },
      ],
      mode: "payment",
      // This is how our web page will know which message we paid for
      success_url: `${domain}?paymentId=${paymentId}`,
      cancel_url: `${domain}`,
      automatic_tax: { enabled: true },
    });

    // Keep track of the checkout session ID for fulfillment
    await ctx.runMutation(internal.payments.markPending, {
      paymentId,
      stripeId: session.id,
    });
    // Let the client know the Stripe URL to redirect to
    return session.url;
  },
});
```

This action uses 2 internal mutations that perform writes to our database:

```jsx
// convex/payments.ts

import { v } from "convex/values";
import { internalMutation } from "./_generated/server";

export const create = internalMutation({
  args: { text: v.string() },
  handler: async (ctx, { text }) => {
    return await ctx.db.insert("payments", { text });
  },
});

export const markPending = internalMutation({
  args: {
    paymentId: v.id("payments"),
    stripeId: v.string(),
  },
  handler: async (ctx, { paymentId, stripeId }) => {
    await ctx.db.patch(paymentId, { stripeId });
  },
});
```

All that is left is to call this action when the user clicks the “Pay $1 and send” button and redirect to the returned URL:

```jsx
// src/App.tsx
import { FormEvent, useState } from "react";
import { useAction } from "convex/react";
import { api } from "../convex/_generated/api";

export default function App() {
  // State storing the message text
  const [newMessageText, setNewMessageText] = useState("");
  
  // Reference to our action
  const payAndSendMessage = useAction(api.stripe.pay);

  // We pass this handler to a `form` `onSubmit` in our UI code
  async function handleSendMessage(event: FormEvent) {
    event.preventDefault();
    // Call to our action
    const paymentUrl = await payAndSendMessage({ text: newMessageText });
    // Redirect to Stripe's checkout website
    window.location.href = paymentUrl!;
  }
  // the rest of our UI...
}
```

## Step 2: The payment checkout flow

Nothing to do here! Stripe did this work for us. 😮‍💨

## Step 3: Receiving the payment confirmation

Stripe will notify our server about the confirmed payment via a webhook request. All we need to do is expose an HTTP endpoint that Stripe’s servers can hit whenever a user finishes a payment. That’s a perfect fit for a Convex [HTTP action](https://docs.convex.dev/functions/http-actions). The HTTP action will handle the Stripe webhook request, parsing it, and passing its contents to another Convex action that will confirm the request is valid using the Stripe SDK in the Node.js runtime:

```jsx
// convex/http.ts
import { httpRouter } from "convex/server";
import { httpAction } from "./_generated/server";
import { internal } from "./_generated/api";

const http = httpRouter();

http.route({
  path: "/stripe",
  method: "POST",
  handler: httpAction(async (ctx, request) => {
    // Getting the stripe-signature header
    const signature: string = request.headers.get("stripe-signature") as string;
    // Calling the action that will perform our fulfillment
    const result = await ctx.runAction(internal.stripe.fulfill, {
      signature,
      payload: await request.text(),
    });
    if (result.success) {
      // We make sure to confirm the successful processing
      // so that Stripe can stop sending us the confirmation
      // of this payment.
      return new Response(null, {
        status: 200,
      });
    } else {
      // If something goes wrong Stripe will continue repeating
      // the same webhook request until we confirm it.
      return new Response("Webhook Error", {
        status: 400,
      });
    }
  }),
});

export default http;
```

Our `fulfill` action confirms the webhook request really came from Stripe via its SDK, gets the checkout session ID, and finally “fulfill”s the order by calling a database mutation:

```jsx
// convex/stripe.ts (continued)
export const fulfill = internalAction({
  args: { signature: v.string(), payload: v.string() },
  handler: async (ctx, { signature, payload }) => {
    const stripe = new Stripe(process.env.STRIPE_KEY!, {
      apiVersion: "2022-11-15",
    });

    const webhookSecret = process.env.STRIPE_WEBHOOKS_SECRET as string;
    try {
      // This call verifies the request
      const event = stripe.webhooks.constructEvent(
        payload,
        signature,
        webhookSecret
      );
      if (event.type === "checkout.session.completed") {
        const stripeId = (event.data.object as { id: string }).id;
        // Send the message and mark the payment as fulfilled
        await ctx.runMutation(internal.payments.fulfill, { stripeId });
      }
      return { success: true };
    } catch (err) {
      console.error(err);
      return { success: false };
    }
  },
});
```

The internal mutation makes writes to our database. It’s awesome that this mutation is automatically transactional, so if we made it this far, we are guaranteed to **both** mark the payment as fulfilled (by adding a `messageId` to it) **and** send the message (by adding it to the “messages” table).

```jsx
// convex/payments.ts (continued)
export const fulfill = internalMutation({
  args: { stripeId: v.string() },
  handler: async ({ db }, { stripeId }) => {
    const { _id: paymentId, text } = (await db
      .query("payments")
      .withIndex("stripeId", (q) => q.eq("stripeId", stripeId))
      .unique())!;
    const messageId = await db.insert("messages", { text });
    await db.patch(paymentId, { messageId });
  },
});
```

At this point, the payment and fulfillment flow works end-to-end.

## Bonus: Tying it all together

Our demo includes one more trick for highlighting the message the user just paid for. This could be used to show a unique order confirmation, even if the user made 2 purchases in parallel (from two different browser tabs).

Back in step 1, we added the “payments” document ID to the URL Stripe redirects to after a successful checkout, the `success_url`.

We can use this information from the URL to highlight the relevant message:

```jsx
// src/App.tsx

// A simple React hook that grabs a URL query parameter and erases
// it from the URL.
function useConsumeQueryParam(name: string) {
  const [value] = useState(
    // get the param value
    new URLSearchParams(window.location.search).get(name)
  );

  useEffect(() => {
    const currentUrl = new URL(window.location.href);
    const searchParams = currentUrl.searchParams;
    searchParams.delete(name);
    const consumedUrl =
      currentUrl.origin + currentUrl.pathname + searchParams.toString();
    // reset the URL
    window.history.replaceState(null, "", consumedUrl);
  }, []);
  return value;
}

export default function App() {
  // use the hook to get the "payments" document ID
  const paymentId = useConsumeQueryParam("paymentId");
  // Call a query to exchange the "payments" document ID
  // for the corresponding "messages" document ID
  const sentMessageId = useQuery(api.payments.getMessageId, {
    paymentId: (paymentId ?? undefined) as Id<"payments"> | undefined,
  });
  const messages = useQuery(api.messages.list) || [];
  // render a list of messages
  return (
    <>
      <ul>
        {messages.map((message) => (
          <li
            key={message._id}
            // highlight the message the user just paid for, if any
            className={sentMessageId === message._id ? "sent" : ""}
          >
            <span>{message.text}</span>
          </li>
        ))}
      </ul>
      {* ... *}
    </>
  );
}
```

Refer to the [the Github repo](https://github.com/get-convex/convex-stripe-demo) for the full implementation.

## It was ninety-nine cents!
![Convex $ Stripe from Macklemore's Thrift Shop Music Video](https://cdn.sanity.io/images/ts10onj4/production/331ef4de890fc6db7270439d62592c0d98613726-947x499.jpg)

While most people won’t pay a dollar to send a message to a random message board, our demo app showcases all the necessary pieces for integrating a full-fledged payment flow. Convex and Stripe make this flow feel solid and reliable. Let us know [in Discord](https://convex.dev/community) what your app charges for 🤑

If you’re building a full-stack app, chances are you’ll want some of your users to pay you for the service you provide. How to use Stripe with Convex to get it done.

Wake up, you need to make money! (ft. Stripe)

Building ChatGPT-powered experiences feel snappier when the responses show up incrementally. Instead of waiting for the full response before showing the user anything, streaming the text in allows them to start reading immediately. 

OpenAI exposes a streaming API for chat completions. But how do you manage a streaming request, when you have a server between the client and OpenAI? You might be tempted to use HTTP streaming end to end - both from the client to the server and the server to OpenAI. However, there’s another way that comes with some big benefits. Spoiler: it’s possible to use a database as a layer of reactivity that separates client request lifecycles from server requests. Don’t worry if that doesn’t make sense yet - we’ll take it one step at a time.

This post will look at working with streams with OpenAI’s beta v4.0.0 Node SDK. Beyond just getting streaming for a single user, we’ll look at an approach that enables:

- **Persisting** the response even if the user closes their browser.
- **Multiplayer chat**, including **streaming multiple ChatGPT messages** at once.
- **Resuming** a stream when a user **refreshes their browser** mid-stream.
- Streaming to **multiple users** at once.
- Implement **custom stream granularity**, such as only updating on full words or sentences, rather than on each token.

To do this, we’ll use <QuickConvex /> to store the messages and make the request to OpenAI. This code is [on GitHub](https://github.com/ianmacartney/streaming-chat-gpt) for you to clone and play with.

![Diagram of browsers talking to Convex, which talks to OpenAI](https://cdn.sanity.io/images/ts10onj4/production/9a7b8865f6cd1cb6748fdb88c986d6ec7bd26bdb-1200x638.png "Data flow overview")


# Persisting messages

Let’s say we have a chat app, like the one pictured in the gif above. We want to store the messages from each user, as well as messages populated by responses from OpenAI. First let’s look at how data is stored (2), assuming a client sends a message (1).

When a user sends a message, we immediately commit it to the database, so they’re correctly ordered by creation time. This code is executed on the server:

```tsx
export const send = mutation({
  args: { body: v.string(), author: v.string() },
  handler: async (ctx, { body, author }) => {
    // Save our message to the DB.
    await ctx.db.insert("messages", { body, author });

    if (body.indexOf("@gpt") !== -1) {
      // ...see below
    }
  }
});
```

This [`mutation`](https://docs.convex.dev/get-started#mutation-functions) saves the message to the database. When the user wants a response from the GPT model (by adding “@gpt” to the message), we will:

1. Store a placeholder message to update later.
2. Make a streaming request to OpenAI in an asynchronous background function. 
3. Progressively update the message as the response streams in.

By running the streaming request asynchronously (versus blocking in a user request), we can interact with ChatGPT and save the data to the database **even if the client has closed their browser**. It also allows us to run many requests in **parallel**, from the same or **multiple users**.

We also run it asynchronously because, in Convex, mutations are pure transactions and as such can’t do non-deterministic things like making API requests. In order to talk to third-party services, we can use an `action`. Actions are non-transactional serverless functions that can talk to third-party services. We trigger the background job to call ChatGPT and update the message body by [scheduling](https://docs.convex.dev/scheduling/scheduled-functions) the [action](https://docs.convex.dev/functions/actions) like so:

```tsx
// ...when the user wants to send a message to OpenAI's GPT model
const messages = // fetch recent messages to send as context
// Insert a message with a placeholder body.
const messageId = await ctx.db.insert("messages", {
  author: "ChatGPT",
  body: "...",
});
// Schedule an action that calls ChatGPT and updates the message.
await ctx.scheduler.runAfter(0, internal.openai.chat, { messages, messageId });
```

We schedule it for zero milliseconds later, similar to doing `setTimeout(fn, 0)`. The message writing and action scheduling happens transactionally in a [mutation](https://docs.convex.dev/functions/mutation-functions), so we will only run the action if the messages are successfully committed to the database.

When the action wants to update the body of a message as the streaming results come in, it can invoke an `update` mutation with the `messageId` from above:

```tsx
export const update = internalMutation({
  args: { messageId: v.id("messages"), body: v.string() },
  handler: async (ctx, { messageId, body }) => {
    await ctx.db.patch(messageId, { body });
  },
});
```

Note: An `internalMutation` is just a mutation that isn’t exposed as part of the public API. Next we’ll look at the code that calls this `update` function.

Convex has end-to-end reactivity, so when we update the messages in the database, the UI automatically updates. See [below](#client-streaming-via-subscriptions) what it looks like to reactively query data.

# Streaming with the OpenAI node SDK

Streaming is currently available in the beta version of OpenAI’s node SDK. To install it:

```bash
npm install openai@4.0.0-beta.2
```

This is very similar to previous releases of the `openai` package, with a few nuances you can read about [here](https://github.com/openai/openai-node/discussions/182).

The `internal.openai.chat` action we referenced above will live in `convex/openai.ts` - see the full code [here](https://github.com/ianmacartney/streaming-chat-gpt). One important detail is that it needs to run in the [`node` runtime](https://docs.convex.dev/functions/runtimes) to support some dependencies in the `openai` package, which means it needs to have `"use node";` as the first line in the file.

```tsx
"use node";
import { OpenAI } from "openai";
import { internalAction } from "./_generated/server";
//...
type ChatParams = {
  messages: Doc<"messages">[];
  messageId: Id<"messages">;
};
export const chat = internalAction({
  handler: async (ctx, { messages, messageId }: ChatParams) => {
    //...Create and handle a stream request
```

## Creating a stream request

```tsx
// inside the chat function in convex/openai.ts
const apiKey = process.env.OPENAI_API_KEY!;
const openai = new OpenAI({ apiKey });

const stream = await openai.chat.completions.create({
  model: "gpt-3.5-turbo", // "gpt-4" also works, but is so slow!
  stream: true,
  messages: [
    {
      role: "system",
      content: "You are a terse bot in a group chat responding to q's.",
    },
    ...messages.map(({ body, author }) => ({
      role:
        author === "ChatGPT" ? "assistant" : "user",
      content: body,
    })),
  ],
});
//...handling the stream
```

The main difference here from previous `openai` versions (aside from their new simplified client configuration) is passing `stream: true`. This changes the return format, which **unfortunately does not currently provide token usage** as the non-streaming version does. I hope this is fixed in a future release, as keeping track of token usage is useful to know how different users or features are affecting your costs.

## Handling the stream

The API exposed by the `openai` SDK makes handling the stream very easy. We use an async iterator to handle each chunk, appending it to the body and updating the message body with everything we’ve received so far:

```tsx
let body = "";
for await (const part of stream) {
  if (part.choices[0].delta?.content) {
    body += part.choices[0].delta.content;
    await ctx.runMutation(internal.messages.update, {
      messageId,
      body,
    });
  }
}
```

Note that here we’re updating the message every time the body updates, but we could implement **custom granularity** by deciding when to call `runMutation`, such as on word breaks or at the end of full sentences.

This action allows us to stream messages from OpenAI to our server function and into the database. But how does this translate to clients updating in real time? Next, let’s see how the client reactively updates as messages are created and updated.

# Client “streaming” via subscriptions

After the previous sections, you might be surprised how little is required to get the client to show live updating messages. I put streaming in quotes since we aren’t using HTTP streaming here - instead, we’re just using the reactivity provided out-of-the-box by Convex.

On the client, we use the `useQuery` hook, which calls the `api.messages.list` server function in the `messages` module, which we’ll see in a second. This hook will give us an updated list of messages every time a message is added or modified. This is a special property of a Convex [query](https://docs.convex.dev/get-started#query-functions): it tracks the database requests, and when any of the data is changed it:

1. Invalidates the query cache (which is managed transparently by Convex).
2. Recomputes the result.
3. Pushes the new data over a WebSocket to all subscribed clients.

```tsx
export default function App() {
  const messages = useQuery(api.messages.list);
  ...
  return (
    ...
    {messages?.map((message) => (
      <article key={message._id}>
        <div>{message.author}</div>
        <p>{message.body}</p>
      </article>
    ))}
```

Because this query is decoupled from the HTTP streaming response from OpenAI, **multiple browsers** can be subscribed to updates as messages change. And if a user **refreshes or restarts their browser**, it will just pick up the latest results of the query.

On the server, this is the [query](https://docs.convex.dev/database/reading-data#querying-documents) that grabs the most recent 100 messages:

```tsx
export const list = query({
  handler: async (ctx): Promise<Doc<"messages">[]> => {
    // Grab the most recent messages.
    const messages = await ctx.db.query("messages").order("desc").take(100);
    // Reverse the list so that it's in chronological order.
    // Alternatively, return it reversed and flip the order via flex-direction.
    return messages.reverse();
  },
});
```

Convex is doing some magic under the hood. If any message is inserted or updated into the database that would match this query - for instance if a new message is added or one of the first 100 messages is edited - then it will automatically re-execute this query (if there are any clients subscribed to it via `useQuery`). If the results differ, it will push the new results over a WebSocket to the clients, which will trigger an update to the components using `useQuery` for that query. 

To give you a sense of performance, `list` takes ~17ms and `update` takes ~7ms for me on the server, so the total latency between a new token coming from OpenAI and a new set of messages being sent to the client is very fast. The gifs in this article are real recordings, not sped up.

![GPT response streaming in quickly](https://cdn.sanity.io/images/ts10onj4/production/2979c6d28d619bffded99c19bca49f6d229a4cdb-960x960.gif)

# Summary

We looked at how to stream ChatGPT responses into Convex, allowing clients to watch the responses, without the flakiness of browser-based HTTP streaming requests. The full code is available [here](https://github.com/ianmacartney/streaming-chat-gpt). Let us know [in Discord](https://convex.dev/community) what you think!

### Extra Credit 🤓

Beyond what’s covered here, it would be easy to extend this demo to:

- Store whether a message has finished streaming by storing a boolean on the message updated at the end of the stream.
- Add error handling, to mark a message as failed if the stream fails. See [this post](https://stack.convex.dev/full-stack-chatgpt-app#creating-the-action) for an example of updating a message in the case of failure.
- Schedule a function to serve as a watchdog, that marks a message as timed out if it hasn’t finished within a certain timeframe, just in case the action failed. See [this post](https://stack.convex.dev/background-job-management#5-monitoring-timeouts) for more details, as well as other patterns for background jobs.
- Organize the messages by thread or user, using [indexes](https://docs.convex.dev/database/indexes/).

Stream GPT responses without brittle browser-based HTTP streaming.
Multiplayer reactivity, persistence, reactivity via Convex. Using OpenAI’s beta v4.0.0 Node SDK server-side, and Convex's useQuery hook client-side.

GPT Streaming With Persistent Reactivity

Simon Grimm

In this video, you will learn to build a powerful realtime chat with React Native and use Convex as the backend. From creating groups to uploading images and styling messages you will learn all the skills you need to build a React Native chat!

React Native Realtime Chat (Expo, File Upload, Typescript)

Jack Herrington

Check out Convex, an incredibly well-designed Firebase alternative that provides a realtime database, object storage, server functions, http API hosting, cron jobs and more!

The Next Level Firebase for Modern Developers

Jamie Turner

So you have a pretty good grasp on web development, but you want to learn more about building with generative AI? You're not alone. In this live-coding course, follow along as we build a full-stack generative AI app using LangChain.js, Replicate, and OpenAI.

Building a full-stack AI storybook app with LangChain, Replicate, and OpenAI

Yet Another Infrastructure Group

Hosted by Ian & Alex, with Alana (Readyset), Hojjat (Deltastream), Jamie (Convex).

Convex and the future of streaming infrastructure

Web Dev Cody

Convex is the sponsor of the hackathon I'm running, but technically did not sponsor this video specifically.  I just wanted to make some resources to help people start learning about convex so they can feel confident in contributing in the hackathon.

💬 Discord: https://discord.gg/4kGbBaa
🔔 Newsletter: https://newsletter.webdevcody.com/
📁 GitHub: https://github.com/webdevcody
📺 Twitch: https://www.twitch.tv/webdevcody
🤖 Website: https://webdevcody.com
🐦 Twitter: https://twitter.com/webdevcody

A quick start guide for using Convex with Next.js

Using Convex with Next.js Quickstart