Alex Cole

a year ago

End-to-end TypeScript with Convex

Convex provides automatic type safety all the way from your database schema to your React app. How does it work? Hint: we use some wild TypeScript.

A Convex app consists of:

1. [Optional] A schema.ts file defining the tables in your project.
2. Query and mutation functions that read and write documents in your tables.
3. React components that invoke mutation functions and subscribe to query functions for realtime updates.

In Convex, all 3 of these layers are pure TypeScript. No query languages or configuration formats required.

Furthermore, every layer is completely type-safe given the previous one! The types in your query and mutation functions reflect your schema. The types of your React hooks reflect your query and mutation functions.

On top of that, most changes don’t require updating generated code. We use code generation to automatically create convenience utilities to build your app, but the types update automatically.

Under the hood, this uses a pattern I call “types as data structures”. Large TypeScript types pass around metadata about the schema and functions of the app.

Let’s start with some examples and then we’ll see how the magic works. You can also play with it yourself in a TypeScript playground.

The developer experience

Say we define the schema for a basic chat app like:

// convex/schema.ts
import { defineSchema, defineTable } from "convex/server";
import { v } from "convex/values";

export default defineSchema({
  messages: defineTable({
    author: v.string(),
    body: v.string(),
  }),
});

This defines a single table, messages, that contains documents with an author field and a body field.

Now if we write a query function to load messages; the db knows what tables we have and what types they store:

// convex/messages.ts
import { query } from "./_generated/server";

export const list = query(async ({ db }) => {
  const messages = await db.query("messages").take(10);
  return messages;
});

Writing a query function

Furthermore, when we use this query in our React app, the React useQuery hook knows the query’s name and return type!

import { useQuery } from "../convex/_generated/react";

export default function App() {
  const messages = useQuery(api.messages.list)
}

`useQuery` knows the types of our functions

Lastly, let’s update our schema and add a channels table:

// convex/schema.ts
import { defineSchema, defineTable } from "convex/server";
import { v } from "convex/values";

export default defineSchema({
  messages: defineTable({
    author: v.string(),
    body: v.string(),
    channel: v.id("channels")
  }),
  channels: defineTable({
    name: v.string()
  }),
});

Immediately, the type in our React code updates with no code regeneration! You can see below that the messages now have a channel property.

The type of `useQuery` updates automatically

That’s what end-to-end type safety is all about!

The rest of the post explains how Convex achieves this.

How is this possible?

This is built using a pattern I call “types as data structures”.

The idea is to construct TypeScript types to store information about the Convex app’s data model and API. These types are passed around as type parameters and accessed to make all of the methods type-safe.

For example, a type describing the data model of this chat app could look like this:

type MyDataModel = {
  messages: {
    author: string;
    body: string;
    channel: Id<"channels">
  };
  channels: {
    name: string;
  }
};

Here MyDataModel is an object type mapping table names to the type of object in each table. Documents in the "messages" table have author, body, and channel properties. Channels just have a name.

The strange thing about this type is that we never construct values that match it! There will be messages and channels, but never a MyDataModel object. It’s only used as a data structure to pass around type-level metadata about your schema that disappears completely at runtime.

We can pass around MyDataModel as a type parameter to make all of your interactions with Convex type-safe.

Let’s dig into how this data model type works. This is a simplified explanation of Convex’s TypeScript types, but it’ll give you a feel for how it’s built.

Using data structure types

Once we have a DataModel type, it’s possible to define a ton of useful type helpers. For example, we can use the keyof type operator to get a union of the string literal table names in the data model. An index access type extracts the document type.

/**
 * The table names in a data model.
 * 
 * This is a union of string literal types like `"messages" | "channels"`.
 * 
 * @typeParam DataModel - A data model type.
 */
type TableNames<DataModel> = keyof DataModel;

/**
 * The type of a document in a data model.
 * 
 * @typeParam DataModel - A data model type.
 * @typeParam TableName - A string literal type of the table name (like "messages").
 */
type Doc<DataModel, TableName extends TableNames<DataModel>> = DataModel[TableName]

Now let’s build up the types used in our original example. Recall that we want to make this code block type safe:

export default query(async ({ db }) => {
  const messages = await db.query("messages").take(10);
	return messages;
});

We’ll do this from the inside outwards. First, we need a type for the .take(10) method within the database query:

interface DatabaseQuery<Document> {
  take(n: number): Promise<Document[]>;
}

This is parameterized over some document type Document.

Then, we can use that to build the type of db. Note that this code is parameterized over a generic DataModel. It uses TableNames to extract the names of the tables from it and Doc to get the document type and pass it to DatabaseQuery.

interface Database<DataModel> {
  query<TableName extends TableNames<DataModel>>(
    tableName: TableName
  ): DatabaseQuery<Doc<DataModel, TableName>>;
}

Lastly, we can define the type of the query wrapper itself.

query is a function that takes in an inner function and produces some RegisteredQuery. The inner function’s first argument is a context object with a db:

type QueryBuilder<DataModel> = (
  func: (ctx: { db: Database<DataModel> }, args?: any) => any
) => RegisteredQuery;

class RegisteredQuery {}

declare const query: QueryBuilder<MyDataModel>

And that’s it! Now when you define a Convex query function with query(...), Convex will take your schema into account. If you want to see where this happens in real Convex apps, check out the generated code and our open-source npm package.

Building data structure types

Type parameters are cool, but how do we get a DataModel type?

We could code generate it, but then we’d need to regenerate the code every time the schema changes. Instead, we can infer the data model straight from the schema definition. That way, when you update your schema the TypeScript types will update automatically!

To recap, we want to take a schema definition like:

const schema = defineSchema({
  messages: defineTable({
    author: v.string(),
    body: v.string(),
    channel: v.id("channels")
  }),
  channels: defineTable({
    name: v.string()
  }),
});

and turn this into a type like:

type MyDataModel = {
  messages: {
    author: string;
    body: string;
    channel: Id<"channels">
  };
  channels: {
    name: string
  }
};

To do this we’ll need to:

1. Define the type of the validator builder, v.
2. Define the types of defineTable and defineSchema.
3. Lastly, do the magic to convert the type of the schema into a data model type.

v

The validator builder v needs to keep track of the TypeScript type of each property in the schema. We can do this by constructing Validator<T> objects that store the underlying TypeScript type, T, in its type property. For example, v.string() returns a Validator<string>, v.id returns an Id for the table wrapped in a Validator, etc.

To get the underlying TypeScript type of a Validator, you can index into it:

const validator = v.string();
type ValidatedType = typeof validator["type"];

In Convex’s actual implementation, there’s additional information passed around with Validator, such as the field paths available for indexing.

We can build a basic version like this:

class Id<TableName extends string> {
  public tableName: TableName;
  public id: string;

  constructor(tableName: TableName, id: string) {
    this.tableName = tableName;
    this.id = id;
  }
}

class Validator<T> {
  type!: T;
}

declare const v: {
  string(): Validator<string>;
  id<TableName extends string>(tableName: TableName): Validator<Id<TableName>>;
};

Okay, v.string() and v.id() were simple. For fun, let’s implement one of the tricky ones too: v.union.

v.union takes in a variable number of arguments, each of which is a Validator like v.union(v.string(), v.number()). We can represent this using a rest parameter like ...innerTypes: T where T is an array of Validators or a Validator<any>[].

It needs to return a Validator of a union of the underlying TypeScript types. How do we construct this?

T is an array type, so indexing into T like T[0] gives us the type of the first element. If we index instead with number like T[number], this gives us a union type of all the values in the array. That’s a union of Validators. Now if we index "type" into that, we get a union of all the underlying TypeScript types!

declare const v: {
  // ...
  union<T extends Validator<any>[]>(
    ...innerTypes: T
  ): Validator<T[number]["type"]>;
};

defineTable and defineSchema

Comparatively, defineTable and defineSchema are quite straightforward.

A TableDefinition is just an object mapping property names to their types. defineTable can just be an identity function of those. We’re using a type parameter here so that the original, more specific type of the table is preserved.

type TableDefinition = Record<string, Validator<any>>;

declare function defineTable<TableDef extends TableDefinition>(
  tableDefinition: TableDef
): TableDef;

Similarly, a SchemaDefinition is just an object mapping table names to their TableDefinitions. defineSchema just needs to take in and return SchemaDefinitions.

type SchemaDefinition = Record<string, TableDefinition>;

declare function defineSchema<SchemaDef extends SchemaDefinition>(
  schema: SchemaDef
): SchemaDef;

DataModelFromSchemaDefinition

Now that we have the type of a schema, we need to convert it into a data model.

First, we can convert a table definition, or Record<string, Validator<T>>, into the type of document in that table, or Record<string, T>. This uses a mapped type to iterate over the properties in the table definition and gets the TypeScript type of each one.

type DocFromTableDefinition<TableDef extends TableDefinition> = {
  [Property in keyof TableDef]: TableDef[Property]["type"];
};

Then we need to do the same thing with SchemaDefinitions. For each table in the definition, we can call DocFromTableDefinition to get the document type:

type DataModelFromSchemaDefinition<SchemaDef extends SchemaDefinition> = {
  [TableName in keyof SchemaDef]: DocFromTableDefinition<SchemaDef[TableName]>;
};

Then we can produce MyDataModel from our schema definition:

type MyDataModel = DataModelFromSchemaDefinition<typeof schema>; 

Unfortunately, TypeScript refuses to simplify the result giving us an ugly type in our editor. Gross!

Ugly data model

Making it pretty

To solve this, let’s introduce the Expand type. Expand is functionally an identity type. It takes in some object type and maps all the original keys to the original values. It wraps this in a conditional type which helps to convince the TypeScript compiler to simplify the original type.

type Expand<ObjectType extends Record<any, any>> =
  ObjectType extends Record<any, any>
    ? {
        [Key in keyof ObjectType]: ObjectType[Key];
      }
    : never;

Now, if we add Expand into DataModelFromSchemaDefinition, we see that we’ve correctly constructed our data model type!

type DataModelFromSchemaDefinition<SchemaDef extends SchemaDefinition> = {
  [TableName in keyof SchemaDef]: Expand<
    DocFromTableDefinition<SchemaDef[TableName]>
  >;
};

Pretty data model

Putting it all together

Now we have all the pieces to build our end-to-end type safety. To recap, we now can:

1. Define a database schema in TypeScript.
2. Turn that schema into a type describing the app’s data model.
3. Use the data model type to write type-safe query functions.

To see this in action, check out this TypeScript playground with all of our code!

Convex also uses this same “types as data structures” pattern to make our React hooks like useQuery type safe by constructing an API type from all of the Convex functions.

Parting thoughts

If you thought this was interesting, check out Convex!

If you want to learn how the real Convex types work, check out our open-source code! validator.ts, data_model.ts, registration.ts, and api.ts are particularly interesting.

Thanks to my ex-coworkers at Asana, the maintainers of Zod and tRPC, and all of the other TypeScript enthusiasts I've met online and offline for teaching me about all the wild things TypeScript can do.