a year ago

Why Convex Queries are the Ultimate Form of Derived State

derived state is one of the most powerful Concepts in all of programming but I think it's one that's not appreciated as much as it should be for example let's take a look at the state so here we have a simple variable as simple as it can get uh name equals mik we can easily update the name by just setting name equal to something else now what was if we didn't want to change name but wanted to adapt it to be something else that's where derived State comes in so here we have name and age then we have a message which is derived from the name and age simple enough right but what was if we wanted name and age to be changeable themselves but still reflect that relationship that that message is well that's where functions come in so this get message function is name takes a name and an age and returns that that relationship the message name is age we actually do this kind of thing in the react World all the time so here's a standard react functional component takes a name and age and it returns back some derived State some uh react elements using reacts used State we can contain that state locally within the component itself so that when we click a button for example we can reduce we can change the state but then the derived state which is the react elements is always going to stay up to date because of the magic of how your state works now the problem start when we want to do more than just have our local users see our lovely State we want to be able to share share that with many clients so this generally means moving our state to the server for distribution to various clients so here we have our react component again um except this time we're now loading the state from the server when this component mounts so we're fetching and then we are returning some Json and we're setting the name and age uh local state from what is returned um if it's load if it's undefined means it's loading um and then we have just having the state derived state of dating now the obvious issue here is that when we click the button to set the age this is only going to set the age locally it's not going to set the age for all other people who we want to see this website or app we can fix this though if we change the button so that it's now uh when it's clicked sending a message to the server to update the state and then we reload the entire page which then causes it to be F the state to be fetched again updated and then we get our D drive State again just looking at this a bit more closer though I think we can probably simplify this a little bit more if we combine these two together and then we simplify this use effect we'll have to change this loading here to be message then we can go down here and we can now update this so what's happening now is that our message our d a derived State message is being given to us from the server directly and we're not calculating that derived State locally great so now our Drive State comes from the server on request but what happens when another client updates that state so what's going to happen is we're going to have one client that is seeing that the age is 21 and we're going to have another client that is seeing that the age is 20 um this is obviously not ideal because ideally both clients at the same time are going to be able to see the same state and the reason why this happens is because we have no way of the server notifying the other clients when that state has changed now I could start talking here about web sockets or server sent events or uh server side rendering or anything like that but let me just cut to the Chase and just show you how convec solves this problem in a really really elegant and neat and Powerful way so first and foremost convex is a database it's a very powerful reactive database but it's database and as such we Define a schema for our database this is where our this is the structure that our state is going to have and it's going to be stored persisted on the server so we can see we have a Define a schema we Define a table and we Define our two bits of State here name and age clients then access that data by using queries on the server so here is a a standard convex query called get message and we can see the first line here is where we actually pull the data from our database we're pulling down the first row um of my state if it's not there we're going to throw an error it should be there uh for the purpose of this demo we're going to assume that it is there and then we do the magic part we do our derived State and similarly to update the state we have to write a mutation so here we have our reverse aging mutation and as before we grab the state from the database the first row assuming that it's going to be there and then we patch it with whatever the current value of the age is and then subtract one from it okay so now from the client side we can update our react component to use convex so the first thing we need to do is pull in our react client here uh pull in the dependency for that then we can pull in our API which is automatically generated for us when we run convex dev then we need to replace where we pull our state from the server so we just delete that stick this in here so now we can see that we when we're mounting we we're calling convex do query to pull down our get message and then we're setting the message up here and just to note here that um this this API is entirely type safe and so if I misspell something types scpt is going to yell at me which is very handy and so the final thing we need to do here is just scroll down here and replace this bit which which is um what happens when we click the button so we're now calling our convex mutation and then once that returns we then reload the page like we were doing before and this would work um I mean we're Place replacing one for one what we had before but this is convex we can do one better than that so what we can do next is pull in our special hooks here use mutation and use Query and then all we need to do is replace all of this code with this and what this is going to do is effectively is us just stating that we're interested in that piece of derived State the state that is derived from the age and name and it's up to convex to worry about um what happens when either of those values change and keep us updated with that later State and then same for the mutation side we can simplify things if we use pull in our used mutation we then have our reverse aging function here and and we can just stick that in here and note we no longer need to reload the page because our used query is going to automatically keep us up to date so once we change the state we don't need to reload the page we don't get any Jank it's just immediately the state is updated for us and just to illustrate that derived State via queries applies just about everything in convex let's take a look at a little bit more complicated example so if I just uh paste in here here so we now have a uh table that uh represents list of messages in a chat application for example um it has a channel it has a body and it has uh from the user it came from and we have an index on there and then from the query side we can just replace this with this function uh this query get messages in channel uh it takes the channel that we're interested in and then it queries the messages table that we just defined um with that index that we defined passing in the channel that we're interested in and what this is going to do is it's going to only obviously only going to return back the messages that are related to that channel and don't forget this is uh just derived State and it's reactive derive state so take for example if we have a look at this diagram here we have our list of messages in our table um belonging to different channels General random standup General whatever this query is only going to to um return us ones that belong to let's say the general channel so that if we have another message that suddenly pops into the bottom um message six here if we are only subscribed to random for example then that message is not going to it's not going to update that query it's only messages that are related to um the queries that the client is interested in so yeah this is really powerful it means that um the reactivity is fine grained enough that it it responds to the very specific um filtered query that we are interested the very specific derive State we're interested in and we can combine that state from multiple tables if we wish so what ends up happening is because it's so easy to write these queries um we can basically just write queries that um are very specifically needed for our frontend so for example like here a query get the state needed to render my sidebar on the settings page for example and then on our client side we just um we just use our use Query and then we pointed to that um and it just makes it effectively like your server is your derived State and it's reactive so you know that it's always going to keep all your clients updated um yeah and this concept is I think is called backend for front end um and it's it's a really powerful and and very quick way to develop your apps and know that all the state is going to be kept in sync with everything and all users are going to get a really nice experience doing this so I hope through this I have shown that I consider convex queries to be the ultimate form of derive State um it's super powerful and I'm only just barely scratching the surface of what you can do here um there's much more I could talk about um but I will save those for future videos thanks for watching

Let's talk about state, specifically Derived State, and how I believe people are overlooking what I think is the end-game of derived state: Convex Queries.

State

First we should all know what state is but just so we are clear what I mean when I talk about state in this post lets look at this example:

1let name = "Mike";
2

Pretty obvious. Its just a variable that holds some data. Where this data is stored doesn't really matter, it could be held in memory, in a JS runtime, or it could be stored in your favorite database in the cloud.

We can easily manipulate this state:

1let name = "Mike";
2name = "Cann"
3

In so doing we have altered the original state.

I know this is obvious stuff but stick with me as its going to get interesting soon I promise.

Derived State

Now, what if we want to keep that original state around but also adapt it to make some more state? This is what Derived State is.

For example:

1const name = "Mike";
2const age = 21;
3
4const message = `${name} is ${age}`;
5

Here, message is derived state because it's calculated from name and age. It doesn't exist independently, it depends entirely on these two variables.

Simple enough, right?

But what if we want name and age to be changeable while keeping the derived state reflecting their relationship?

That's where functions come in:

1const getMessage = (name: string, age: number) => `${name} is ${age}`;
2
3console.log(getMessage("mike", 21)); // writes "mike is 21"
4

We actually do this sort of thing all the time in the React world using Function Components:

1export const Message: React.FC<{ name: string; age: number }> = ({ name, age }) => {
2  return (
3    <div>
4      {name} is {age}
5    </div>
6  );
7};
8

The elements returned from this component are derived from the input props.

Using React, we can contain state locally within the component. By leveraging the useState hook, we ensure the derived message stays up to date whenever the local state changes:

1import { useState } from "react";
2
3export const MessageLabel: React.FC = ({  }) => {
4  const [name, setName] = useState("mike");
5  const [age, setAge] = useState(21);
6
7  return (
8    <div>
9      <button onClick={() => setAge(age - 1)}>Reverse Aging</button>
10      {name} is {age}
11    </div>
12  );
13};
14

Server State

The problems start when we want more than just you to be able to see your lovely state. We need a way to share it between many clients, this generally means moving that state to a server for distribution:

Now the server holds the state (typically in a database) and when the client loads, it can request this state from the server:

1import { useEffect, useState } from "react";
2
3export const MessageLabel: React.FC = ({  }) => {
4  const [name, setName] = useState<string>();
5  const [age, setAge] = useState(0);
6
7  // Runs when the component is mounted lets load the initial state
8  useEffect(() => {
9	  // Just pretend this is defined on a server somewhere
10    fetch("https://myserver.com/getState")
11      .then(resp => resp.json())
12      .then(data => {
13        setName(data.name);
14        setAge(data.age);
15      })
16  }, []);
17
18  // Not loaded yet? Lets show a loading indicator
19  if (name === undefined) return <div>loading..</div>
20
21  return (
22    <div>
23      <button onClick={() => setAge(age - 1)}>Reverse Aging</button>
24      {name} is {age}
25    </div>
26  );
27};
28

There's an obvious issue here. When we click the button, the age only changes locally. As soon as we refresh the page, that age state resets to whatever is stored on the server.

We can fix this by moving the setState operation to the server and calling it from the button:

1export const MessageLabel: React.FC = ({  }) => {
2  ...
3  return (
4    <div>
5      <button
6        onClick={() =>
7          fetch("https://myserver.com/reverseAging", {
8            method: "POST",
9          }).then(() => window.location.reload())
10        }
11      >
12        Reverse Aging
13      </button>
14      ...
15    </div>
16  );
17};
18

After setting the state, we reload the page to view the updated data.

Not a great user experience but let's just go with it for now.

Looking at the component one more time, I think we could streamline things a bit more. The client only really cares about the message, not name and age separately. So we could have the server just send down the "derived" message instead:

1export const MessageLabel: React.FC = ({  }) => {
2  const [message, setMessage] = useState<string>();
3
4  useEffect(() => {
5    fetch("https://myserver.com/getMessage")
6      .then(resp => resp.text())
7      .then(setMessage)
8  }, []);
9
10  if (message === undefined) return <div>loading..</div>
11
12  return (
13    <div>
14      <button
15        onClick={() =>
16          fetch("https://myserver.com/reverseAging", {
17            method: "POST",
18          }).then(() => window.location.reload())
19        }
20      >
21        Reverse Aging
22      </button>
23      {message}
24    </div>
25  );
26};
27

Great, now our derived state comes from the server on request. But what happens when another client updates the state?

Now our view of the state differs between the two clients until one of these refreshes the page. This happens because we lack a mechanism to notify everyone when the state changes.

This creates a poor user experience and can lead to frustrating consequences. For example, if Client B reverses aging and then Client A does the same, Client A will see the value 19 when they expected 20.

I could show you how to solve these problems using WebSocket's, Server Sent Events or Server Side Rendering, but just trust me, you don't want to deal with these complexities unless absolutely necessary.

So instead, let's cut to the chase and see how Convex solves this and why I think it's so powerful.

State on Convex

The first thing we want to do is define the “schema” or the shape we want our state to take:

1// convex/schema.ts
2
3import { defineSchema, defineTable } from "convex/server";
4import { v } from "convex/values";
5
6export default defineSchema({
7  myState: defineTable({
8    name: v.string(),
9    age: v.number(),
10  }),
11});
12
13

Now we can write the server side function to access the data like so:

1// convex/myState.ts
2
3import { query } from "./_generated/server";
4
5export const getMessage = query(async (context) => {
6  const state = await context.db.query("myState").first();
7  if (!state) throw new Error("Could not find myState");
8
9  return `${state.name} is ${state.age}`;
10});
11

And a mutation to update the state like so:

1// convex/myState.ts
2
3import { mutation, query } from "./_generated/server";
4
5...
6
7export const reverseAging = mutation(async (context) => {
8  // Grab the current state first
9  const currentState = await context.db.query("myState").first();
10  if (!currentState) throw new Error("Could not find myState");
11
12  // Then update it
13  await context.db.patch("myState", currentState._id, {
14    age: currentState.age + 1,
15  });
16});
17

We are going to assume that the initial value for the data has been previously entered elsewhere in these examples

Naive querying via useEffect

We can then update our client to use this new query:

1import { useConvex } from "convex/react";
2import { useEffect, useState } from "react";
3import { api } from "../convex/_generated/api";
4
5export const MessageLabel: React.FC = ({}) => {
6  const [message, setMessage] = useState<string>();
7
8  // Grab the client we can use to make a type-safe query to the server with
9  // the React provider for this exists higher up in the tree
10  const convex = useConvex();
11
12  // On mount
13  useEffect(() => {
14    // Get the message from the server and set it
15    // NOTE: this isnt the best way to use Convex, we'll upgrade this in a minute
16    convex.query(api.myState.getMessage).then(setMessage);
17  }, []);
18
19  if (message === undefined) return <div>loading..</div>;
20
21  return (
22    <div>
23      <button
24        onClick={() =>
25          convex
26            .mutation(api.myState.reverseAging)
27            .then(() => window.location.reload())
28        }
29      >
30        Reverse Aging
31      </button>
32      {message}
33    </div>
34  );
35};
36
37

This is cleaner and works the same as before but we aren't yet harnessing the true power of Convex, lets simplify things a bit more by using the useQuery and useMutation hooks:

Idiomatic querying via useQuery

1import { useMutation, useQuery } from "convex/react";
2import { api } from "../convex/_generated/api";
3
4export const MessageLabel: React.FC = ({}) => {
5  // We simply declare we want the message and let Convex worry about keeping it updated
6  const message = useQuery(api.myState.getMessage);
7
8  // A nice type-safe way of updating our state
9  const reverseAging = useMutation(api.myState.reverseAging);
10
11  if (message === undefined) return <div>loading..</div>;
12
13  return (
14    <div>
15      <button onClick={() => reverseAging()}>Reverse Aging</button>
16      {message}
17    </div>
18  );
19};
20

Now we're cooking!

This code is much cleaner. We simply declare that we want the message with useQuery(api.myState.getMessage) without any messy useEffect logic.

Notice we also removed the page reload when we changed the button click to use useMutation(api.mike.reverseAging);. What happens now when we click the button? This is where the power of the Convex sync engine shines.

When we call reverseAging, it updates the state in our database. Since Convex knows we've declared our interest in api.mike.getMessage, it automatically re-runs that query and syncs the result to the client.

Super cool! And we didn't need to modify our server-side code at all to enable this.

So our server-side query has effectively become the source of our "derived state"—and better yet, it automatically stays in sync whenever the underlying state changes.

Lists

To make this a bit more concrete lets take a look at a more realistic example from the docs. Starting with the following schema:

1// convex/shema.ts
2
3import { defineSchema, defineTable } from "convex/server";
4import { v } from "convex/values";
5
6export default defineSchema({
7
8  // Define a messages table with two indexes.
9  messages: defineTable({
10    channel: v.id("channels"),
11    body: v.string(),
12    user: v.id("users"),
13  })
14    .index("by_channel", ["channel"])
15    .index("by_channel_user", ["channel", "user"])
16    
17});
18

We can then write a query like so:

1// convex/chat.ts
2
3import { query } from "./_generated/server";
4import { v } from "convex/values";
5
6export const getMessagesInChannel = query({
7  args: { channel: v.id("channels") },
8  handler: async (ctx, args) => {
9    return await ctx.db
10		  .query("messages")
11		  .withIndex("by_channel", (q) =>
12		    q
13		      .eq("channel", channel)
14		      .gt("_creationTime", Date.now() - 2 * 60000)
15		      .lt("_creationTime", Date.now() - 60000),
16		  )
17		  .collect();
18  },
19});
20

When any message within the specified channel and time window is added, removed, or modified, the query automatically re-executes and synchronizes with all subscribed clients.

This is really powerful because our derived state can automatically depend not just on one document, but on many documents across a table or even multiple tables!

State in a Convex World

Because Convex queries are able to provide derived state on the server side which is also able to stay fresh when your state changes you can just start to think of the server state as your single “source of truth”.

It turns out you don't need local state management libraries like Redux or MobX to “cache” your server-side state. You can simply write a query that is highly targeted towards the use-case your client needs:

1export const getTheStateINeedToRenderMySidebarOnTheSettingsPage = query(async () => {
2  return {
3    someState: ...,
4    someOtherState: ...,
5  }
6});
7

This is sometimes known as Backend for Frontend and is one of the things that allows you as a developer to move so quickly when building on Convex.

Conclusion

I hope I've shown why Convex queries are powerful when viewed as derived state. You can subscribe to this derived state, just like with MobX or other reactive local state libraries, and it updates automatically whenever the underlying state changes.

Best of all, this synchronization works seamlessly across countless clients on different devices simultaneously!

I'm only scratching the surface of what's possible. You should definitely check out the comprehensive docs if you want to learn more.

Build in minutes, scale forever.

Convex is the backend platform with everything you need to build your full-stack AI project. Cloud functions, a database, file storage, scheduling, workflow, vector search, and realtime updates fit together seamlessly.

Get started