Sarah Shader

8 months ago

Running tests using a local open-source backend

Note: This article was written before we released the convex-test library along with the testing guidance to use it for unit tests. We generally recommend writing unit tests using convex-test over using a local backend, and this section lists some of the pros and cons for testing against a local backend.

Convex recently released an open source version of the backend. We can use this to unit test Convex functions by running them with a local backend. These tests will work by running a backend locally, pushing your code, and using the ConvexClient to execute queries and mutations from your favorite testing framework and asserting on the results.

Here’s an example of a simple test we might want to write if we were building a notes app:

  test("can only read own notes", async () => {
    const personASessionId = "Person A";
    await t.mutation(api.notes.add, {
      sessionId: personASessionId,
      note: "Hello from Person A"
    });

    await t.mutation(api.notes.add, {
      sessionId: personASessionId,
      note: "Hello again from Person A"
    });

    const personBSessionId = "Person B";
    await t.mutation(api.notes.add, {
      sessionId: personBSessionId,
      note: "Hello from Person B"
    });

    const personANotes = await t.query(api.notes.list, {
      sessionId: personASessionId
    });
    expect(personANotes.length).toEqual(2);
  });

Setting up a local backend

First, we need to make sure we are able to run a local backend by following the instructions here. Make sure just run-local-backend runs successfully.

To make it easier to interact with the local backend from our project, we can add a Justfile supporting the following commands: just run-local-backend, just reset-local-backend, and just convex [args] to run Convex CLI commands against the local backend. These can be copied from the open source backend's Justfile.

We want a command like npm run testFunctions that does the following:

• Ensures a fresh local backend is running
• Sets the IS_TEST environment variable, allowing us to run testing only functions (just convex env set IS_TEST true)
• Pushes code to the local backend (just convex deploy)
• Runs the tests (e.g. npx vitest)
• Finally, tears down the local backend

One way to do this is to write a script, backendHarness.js, that handles setting up the local backend, runs a command, and finally tears down the backend.

We can add commands to our package.json that look something like this:

// package.json
"scripts": {
	// Test assuming there's an existing local backend running
	"testFunctionsExistingBackend": "just convex env set IS_TEST true && just convex deploy && vitest",
	// Test handling local backend creation + cleanup
	"testFunctions": "node backendHarness.js 'npm run testFunctionsExistingBackend'"
}

Writing tests

In this set up, all our tests will be running against the same local backend, since running a backend for each test would be too slow. To keep tests isolated from each other, we want to configure our testing framework to run tests one at a time (e.g. maxWorkers: 1 in vitest.config.mts) and also call a function to clear out all our data after each test.

We’ll add a clearAll mutation that looks something like this:

export const clearAll = testingMutation(async (ctx) => {
  for (const table of Object.keys(schema.tables)) {
    const docs = await ctx.db.query(table as any).collect();
    await Promise.all(docs.map((doc) => ctx.db.delete(doc._id)));
  }
  const scheduled = await ctx.db.system.query("_scheduled_functions").collect();
  await Promise.all(scheduled.map((s) => ctx.scheduler.cancel(s._id)));
  const storedFiles = await ctx.db.system.query("_storage").collect();
  await Promise.all(storedFiles.map((s) => ctx.storage.delete(s._id)));
})

It clears data from all tables listed in the schema, cancels any scheduled jobs, and deletes any stored files.

The testingMutation wrapper uses customMutation from convex-helpers to ensure all functions check that the IS_TEST environment variable is set to prevent clearAll from being called on Production or Development deployments.

export const testingMutation = customMutation(mutation, {
  args: {},
  input: async (_ctx, _args) => {
    if (process.env.IS_TEST === undefined) {
      throw new Error(
        "Calling a test only function in an unexpected environment"
      );
    }
    return { ctx: {}, args: {} };
  },
});

Putting all of this together, we can write a test for our Convex functions which we can run via npm run testFunctions:

import { api } from "./_generated/api";
import { ConvexTestingHelper } from "convex-helpers/testing";

describe("testingExample", () => {
  let t: ConvexTestingHelper;

  beforeEach(() => {
    t = new ConvexTestingHelper();
  });

  afterEach(async () => {
    await t.mutation(api.testingFunctions.clearAll, {});
    await t.close();
  });

  test("can only read own notes", async () => {
    const personASessionId = "Person A";
    await t.mutation(api.notes.add, {
      sessionId: personASessionId,
      note: "Hello from Person A"
    });

    await t.mutation(api.notes.add, {
      sessionId: personASessionId,
      note: "Hello again from Person A"
    });

    const personBSessionId = "Person B";
    await t.mutation(api.notes.add, {
      sessionId: personBSessionId,
      note: "Hello from Person B"
    });

    const personANotes = await t.query(api.notes.list, {
      sessionId: personASessionId
    });
    expect(personANotes.length).toEqual(2);
  });
});

A PR setting up tests for a Chess app built on Convex, following this guide, can be found here. It includes some examples of more complex tests.

It includes testing authenticated functions (this uses the same mechanism as the dashboard function tester to act as a particular user)

 const sarahIdentity = t.newIdentity({ name: "Sarah" });
 const asSarah = t.withIdentity(sarahIdentity);
 
 const gameId = await asSarah.mutation(api.games.newGame, {
	 player1: "Me",
	 player2: null,
 });
 let game = await t.query(api.games.get, { id: gameId });
 expect(game.player1Name).toEqual("Sarah");

It also uses test only mutations to set up data (setting up a chess game 2 moves from the end):

// Two moves before the end of the game
const gameAlmostFinishedPgn = "1. Nf3 Nf6 2. d4 Nc6 3. e4 Nxe4" /* ... */

const gameId = await t.mutation(api.testing.setupGame, {
	player1: sarahId,
	player2: leeId,
	pgn: gameAlmostFinishedPgn,
	finished: false,
});

Limitations

Scheduled functions (ctx.scheduler and crons) will run in this local backend as time progresses, and while it’s possible to check the state of a scheduled job using ctx.db.system, there’s no built in way to run a scheduled function for testing or to mock out time and manually advance it in tests.

There is also no built in way to mock out or control randomness in the tested functions.

See the documentation for a comparison of this testing strategy with unit tests using the convex-test library.

Summary

Convex functions can be unit tested using a locally running Convex backend.

Check out the open source backend or see an example of these tests in action!