Jamie Turner

2 years ago

Convex gets Rusty with Santa

At Convex we like Rust. After all, we’ve used Rust in production for almost ten years, and the Convex backend is even built in Rust! So when we began to add support for new languages beyond JavaScript/TypeScript, Rust was near the top of the list.

We’re very pleased to announce our Rust client is now up on GitHub and crates.io! This means you can write Rust programs that read, write, and subscribe to records in your Convex deployments, just as with our original JavaScript/TypeScript library.

So what might you do with this new Rust crate?

Haskell helps Convex onto the nice list

I know, I know—this article is about using Rust with Convex. So why is the Haskell programming language getting involved? Well, Convex’s overall architectural principles are heavily influenced by ideas from Haskell. In particular, Convex’s decision to represent transactions as side-effect-free code is an idea that Haskell advanced beautifully with its Software Transactional Memory (STM) implementation and library.

In 2007, in the book Beautiful Code, Haskell creator Simon Peyton-Jones contributed a wonderful chapter called “Beautiful Concurrency.” Inside, he introduced STM as a way to model concurrent systems that desire atomic changes to shared state without using pessimistic locking.

The chapter contained a Haskell program demonstrating STM in practice, implementing a contrived scenario Simon dubbed “The Santa Claus problem.” The full chapter provides a great amount of detail on this problem, but here’s a summary:

1. At the North Pole, nine reindeer (Simon generously included Rudolph), ten elves, and Santa all need to work together following some rules
2. If all nine reindeer are ready to work, Santa gathers them up and delivers presents with them
3. Otherwise, if at least three elves are ready to work, Santa goes into his study to build presents with exactly three of the elves
4. If neither group is ready, Santa waits. He doesn't work alone!
5. After a shift working with Santa, each elf and reindeer goes alone on vacation for a while, unavailable to work until they return

In Beautiful Code, Simon implemented each of these characters as an actor in a separate Haskell thread, using STM to coordinate their shared state. I decided to recreate my own solution in a multi-process, multi-host distributed system, using the Convex backend to coordinate state. Read on to see how!

Making a flight plan

It will help to understand our scenario's rules if we make some algorithmic pseudocode. First, let’s write down the logic for the elves and reindeer. They both have the same routine, so this routine will be for a generic “worker.”

1register new ("elf" | "reindeer") as state "ready"
2loop {
3  wait until Santa says our state is "working"
4  wait (aka "work") until Santa says our state is "vacationing"
5  sleep for a random amount of time -- on vacation
6  set our own state to "ready"
7}
8

Now, let’s write down Santa’s logic:

1loop {
2  do {
3    poll everyone with state as "ready"
4  } until all 9 reindeer or at least 3 elves are "ready"
5
6  if 9 reindeer are "ready" {
7    the work group = all nine reindeer
8  } else {
9    the work group = any three elves who are "ready"
10  }
11
12  atomically set every member of the work group to "working"
13  sleep a random amount of time -- the group is working
14  atomically set every member of the work group to "vacationing" 
15}
16

Polar architecture

Based on our pseudocode exploration above, we can model the shared state of this problem as a table of records where each record represents an individual worker. We have exactly two requirements for these records:

1. We need to distinguish between “reindeer” and “elf” workers, since Santa’s selection behavior is different for each (namely, preferring reindeer, and working with the whole set vs. a subset)
2. We need to track the current state of each of these workers as either “ready”, “working”, or “vacationing”. This is effectively how Santa and the workers will communicate with each other.

Time to get coding! First, let’s lock down the Convex schema in convex/schema.ts:

1import { defineSchema, defineTable } from "convex/schema";
2import { v } from "convex/values";
3
4export default defineSchema({
5	workers: defineTable({
6    workerType: v.union(v.literal("reindeer"), v.literal("elves")),
7    state: v.union(
8      v.literal("ready"),
9      v.literal("working"),
10      v.literal("vacationing")
11    ),
12  }),
13});
14

Now, in a file called convex/worker.ts let’s implement the Convex query and mutation functions that will soon service our worker actor:

1import { Doc, Id } from "./_generated/dataModel";
2import {
3  DatabaseReader,
4  DatabaseWriter,
5  mutation,
6  query,
7} from "./_generated/server";
8
9// Initial call. Create a new worker with an initially ready-to-work state.
10export const insertReadyWorker = mutation(
11  async (
12    { db }: { db: DatabaseWriter },
13    { workerType }: { workerType: "reindeer" | "elves" }
14  ) => {
15    const newId = await db.insert("workers", { workerType, state: "ready" });
16    return newId;
17  }
18);
19
20// Check if Santa has told us it's time to work.
21export const isTimeToWork = query(
22  async ({ db }: { db: DatabaseReader }, { id }: { id: Id<"workers"> }) => {
23    return (await db.get(id))!.state == "working";
24  }
25);
26
27// Check if Santa has told us our work is done.
28export const isTimeToVacation = query(
29  async ({ db }: { db: DatabaseReader }, { id }: { id: Id<"workers"> }) => {
30    return (await db.get(id))!.state == "vacationing";
31  }
32);
33
34// After we're done vacationing, mark ourselves ready to work again.
35export const markBackFromVacation = mutation(
36  async ({ db }: { db: DatabaseWriter }, { id }: { id: Id<"workers"> }) => {
37    await db.patch(id, {
38      state: "ready",
39    });
40  }
41);
42

Each of these functions provides the server-side capabilities for the Rust app’s worker routine, which we’ll get to momentarily. Notice these functions are only about maintaining one actor’s state. There is no collective querying or mutation–the worker only cares about their own tasks.

Santa, meanwhile, is the one who decides what the group does, so his module of Convex functions (convex/santa.ts) will contain queries and mutations that span all worker records.

According to our pseudocode, he’ll need some kind of query to check if there is a group ready to work:

1export const newGroupReady = query(async ({ db }): Promise<"reindeer" | "elves" | null> => {
2  // Quick invariant check. No one should be working if we don't expect it.
3  const workersBusy = await anyoneWorking({ db });
4  if (workersBusy) {
5    throw "Busy workers before Santa said to go?";
6  }
7  // No one working. Do we have another workgroup?
8
9	// Helper function that queries workerType = reindeer + "ready"
10  const reindeer = await waitingReindeer({ db });
11  if (reindeer.length == 9) {
12    return "reindeer";
13  }
14	// Helper function that queries workerType = elves + "ready"
15  const elves = await waitingElves({ db });
16  if (elves.length >= 3) {
17    return "elves";
18  }
19  return null;
20});
21

Once a group is ready, Santa needs a mutation to schedule that group to work and one more to finish that work and put them on vacation:

1// Grab the appropriate group and mark them working. The given `work`
2// was returned as ready from the `newGroupReady` query.
3export const dispatchGroup = mutation(
4  async ({ db }, { work }: { work: "reindeer" | "elves" }) => {
5    if (await anyoneWorking({ db })) {
6      throw "should never try to kick off a workgroup when work is already happening";
7    }
8    if (work == "reindeer") {
9      const reindeer = await waitingReindeer({ db });
10      assert(reindeer.length === 9);
11      for (const r of reindeer) {
12        await db.patch(r._id, { state: "working" });
13      }
14    } else if (work == "elves") {
15      // Then, kick off elves to work.
16      const elves = await waitingElves({ db });
17      assert(elves.length >= 3);
18      const wakeElves = elves.slice(0, 3);
19      for (const e of wakeElves) {
20        await db.patch(e._id, { state: "working" });
21      }
22    } else {
23      throw "Uh, what kind of job is this?";
24    }
25  }
26);
27
28// Release the current workgroup. Put them on vacation.
29export const releaseGroup = mutation(async ({ db }) => {
30  const workers = await currentWorkers({ db });
31  for (const w of workers) {
32    await db.patch(w._id, {
33      state: "vacationing",
34    });
35  }
36});
37

And that’s basically everything we need. Time to build the actor routines that use these Convex functions in a Rust app!

Rusty Routines

Rust apps communicate with Convex using a client from the Convex crate. Creating a new Rust client is easy:

1let mut convex = ConvexClient::new(&cli.deployment_url).await.unwrap();
2

Invoking mutations is a client method that takes a name and arguments just like JavaScript/TypeScript. It returns a future since the Convex client is 100% async.

Subscriptions are exposed as a futures::Stream, yielding new values whenever Convex pushes them over the wire when dependent backend data changes.

In Rust, arguments to all Convex functions are provided as a BTreeMap, where each value is a Convex-specific Value type that represents all the types you can pass to Convex functions.

And that’s all we need to know about the Convex crate API. With those basic ingredients covered, let’s now walk through the Rust function for Santa's implementation line by line. We’ll use these few Convex client methods to invoke the Convex functions in santa.ts to create our actor.

First, the preamble…

1async fn santa(deployment_url: String, max_work_ms: Arc<AtomicU64>) {
2    let mut convex = ConvexClient::new(&deployment_url).await.unwrap();
3    let mut rng = rand::rngs::StdRng::from_entropy();
4    loop {
5

Our pseudocode outlined the beginning of Santa's algorithm as:

1  do {
2    poll everyone with state as "ready"
3  } until all 9 reindeer or at least 3 elves are "ready"
4  if 9 reindeer are "ready" {
5    the work group = all nine reindeer
6  } else {
7    the work group = any three elves who are "ready"
8  }
9

You may recall this logic is already in our deployment’s query function newGroupReady. This will return null if no group is ready, or a group name if so. All that remains is to write this bit of Rust, which blocks on a subscription until a group type is returned:

1    // Wait for a new group of workers to be ready!
2    let mut sub = convex
3        .subscribe("santa:newGroupReady", maplit::btreemap! {})
4        .await
5        .unwrap();
6    let mut job = String::new();
7    while let Some(result) = sub.next().await {
8        if let FunctionResult::Value(Value::String(given_job)) = result {
9            job = given_job;
10            break;
11        }
12    }
13    drop(sub);
14

And to recap the rest of our Santa algorithm pseudocode:

1  atomically set every member of the work group to "working"
2  sleep a random amount of time -- working
3  atomically set every member of the work group to "vacationing"
4

Right. So, atomically (which we get for free in Convex mutations) set everyone in the workgroup to state “working”:

1    // Kick off work.
2    convex
3        .mutation(
4            "santa:dispatchGroup",
5             maplit::btreemap! {
6                 "work".to_owned() => Value::String(job),
7             },
8        )
9        .await
10        .unwrap();
11

Our Santa routine will now sleep for a random duration to simulate the time it takes to finish the work.

1    // Let the group "work" with Santa.
2    tokio::time::sleep(
3        Duration::from_millis(
4            rng.gen_range(0..max_work_ms.load(Ordering::Relaxed)),
5        )
6    )
7

Then, we need to call a mutation to atomically release the current workers so they stop working and go on vacation:

1    // Release them from their duties.
2    convex
3        .mutation("santa:releaseGroup", maplit::btreemap! {})
4        .await
5        .unwrap();
6   } // go back to the top of the loop and wait for the next group!
7}
8

We’ll skip a detailed examination of the worker routine’s Rust code. In brief, it uses the Convex functions from workers.ts in a loop similar to Santa:

1. It uses subscriptions to wait until Santa puts them to work and then again until he releases them
2. It calls a mutation to re-mark itself as ready to work after a random delay for “vacation”

The Arctic in Action

Let’s see this simulation run!

The running Rust program on the left side of this video contains all 20 actors in separate threads with separate Convex connections–19 worker threads (nine reindeer, ten elves) and one Santa thread. Their only shared state is in the Convex deployment they access over the network.

On the right, we can see the Convex dashboard update as the collective state machine cycles:

And since Convex is great at web stuff, I built a tiny bonus React app. The app provides a nifty visual depiction of the North Pole state and additionally lets you adjust two Convex-backed values that parameterize how long the Rust program sleeps during vacation and work periods:

Want to run it yourself, or go deeper into the code? The whole codebase is up on GitHub.

Ho! Ho! Workflow!

Some may argue this Santa situation is pretty silly. What’s the point?

Well, what we’ve really created is a distributed workflow system where different programs depend on each other’s state to advance a collective process. And because of the Convex foundation, this workflow utilizes ACID semantics to guarantee appropriate coordinated behavior.

This scenario represents one common pattern where we’re excited to see folks use Rust (and Python!) with Convex. After all, there are numerous background, batch, command line, data-intensive programs out there quietly intermingling with and enhancing the UI-centric web and mobile apps we interact with and love.

And now with Convex, you can use a single, powerful, consistent set of abstractions to build these hybrid systems with ease.