InfoQ

Alex Cole discusses dynamic web apps, how serverless solutions compare to in-house stacks, and how product development changes when individual engineers can own features end-to-end.


Everyone Can Be a Full-Stack Engineer

Convex

Dynamic application developers need global state management, not a database.

James (@jamesacowling) presents his talk from the HYTRADBOI 2022 database conference on how Convex is rethinking how app developers manage global state.

Data Interactivity in the Serverless Future (HYTRADBOI 2022)

While using one of your favorite apps, have you ever run into something like this?

<QuickVid src="https://static.convex.dev/stack/search-sanity/bad-search.m4v" caption="Old content is searchable, but new content is… not?" />

It’s pretty reasonable to expect that once you add a document to your Notion workspace, the search box will find it! What’s going on here?

# Unsavory search secrets

You may be surprised to learn that there’s nothing particularly unusual about what Notion is doing. In most systems, backend search architectures are structured just so:

<QuickVid src="https://static.convex.dev/stack/search-sanity/search-arch-animation.m4v" use_poster caption="The search query can arrive after the document is saved, but before the search index is updated" />

In this common design, the system that *stores* your records is separate from the system that *indexes* them for search. Specialized systems are chained together, such as MySQL for the database and Elasticsearch for full-text search.

So there is inherently a delay in the pipeline that connects storage and search, and your users might experience a temporary inconsistency–a stored document not yet searchable, just as we saw with Notion. This inconsistency is often only a few seconds, but sometimes can be minutes or hours.

While the ease of leveraging several ready-to-go systems for each task may feel like a pragmatic decision for a software team, it really sucks that it “leaks” internal architectural choices in the form of a confusing user experience.

# Fancy concepts, simple expectations

At <QuickConvex />, we love deep infrastructure concepts like strong consistency and [ACID](https://stack.convex.dev/dont-drop-acid). But not because distributed systems papers are riveting beach reads. Instead, because we believe the mark of a great platform is facilitating the easy creation of software that Just Works. Convex projects should automatically do what the developer and user expect.

And that’s why Convex’s search is *transactional:* search indexes in Convex guarantee that the very moment a document is committed to the database, that document shows up in search results. User expectations met!

And while we’re at it, Convex developers expect that everything in our platform is seamlessly reactive. So we made search reactive, too.

# What “just works” looks like

Let’s use Convex to search through a sample of Wikipedia articles!

As a prerequisite, we retrofitted the Convex [tutorial chat app](https://docs.convex.dev/tutorial/welcome-to-convex) by loading 100,000 random Wikipedia articles with the [command line import tool](https://docs.convex.dev/using/cli#import-a-file-into-convex). We assigned the page body to the `body` field and the title to the `author` field of the messages table.

Time to get searching! First, we need to create the search index using [Convex’s schema definition](https://docs.convex.dev/using/schemas). Let’s create an index called “search_body” that is indexing the `body` field. We’ll do this by using the `searchIndex` method on the schema object:

```tsx
import { defineSchema, defineTable } from "convex/schema";
import { v } from "convex/values";

export default defineSchema({
  messages: defineTable({
    body: v.string(),
    author: v.string(),
  }).searchIndex("search_body", {
    searchField: "body",
  }),
});
```

Then, we can easily use this search index in our app’s [query functions](https://docs.convex.dev/using/database-queries) by calling our query’s `withSearchIndex` method.

```tsx
export default query(async ({ db }, { bodyQuery }) => {
  const results = await db
    .query("messages")
    .withSearchIndex("search_body", q => q.search("body", bodyQuery))
    .take(5);
  return results;
});
```

Finally, as usual in React + Convex apps, we’ll leverage the Convex library’s `useQuery` hook to [attach this backend query function](https://docs.convex.dev/using/writing-convex-functions#defining-convex-functions) to a component rendering a list of results:

<QuickVid src="https://static.convex.dev/stack/search-sanity/good-search.m4v" caption="Instantly reactive, fully consistent ACID-compliant search." />

This principled take on search is just one of the many ways in which Convex nudges your projects toward the [pit of success](https://blog.codinghorror.com/falling-into-the-pit-of-success/). So start a new project today, try out transactional search, and [come show us what you built](https://convex.dev/community)!

Your document is safely stored, but search is coming up empty? Inconsistent search systems can really confuse users! Don't worry–in Convex, transactional search indexing ensures that users always get what they expect and your apps just work.

Searching for Sanity

SQL has been a part of computing for a very long time, and SQL-based database systems underly most of the world's applications. But recently, developers have been clamoring for something better, and it's time to move on.

It's not you, it's SQL

[Convex 0.5.0](https://blog.convex.dev/announcing-convex-0-5-0/) introduces built-in support for *actions* — arbitrary lambda functions that run in a Node.js environment on the Convex cloud. We use the term *action* to differentiate these functions from Convex *mutation* and *query* functions, which enable [transactional writes and dynamic subscriptions that update whenever data changes](https://www.youtube.com/watch?v=AdrO6IvHjRQ).

Queries and mutations are the core building blocks in Convex, but they’re required to be deterministic and free of side-effects. We leverage this determinism to efficiently schedule transactions, dynamically track changes to data, and automatically cache and distribute query results.

Actions are Convex’s first foray into executing code that has side-effects or isn’t purely oriented around reading or writing data. Data is the center of your application, and Convex has a custom-built and highly scalable database at its core, but Convex is not just a database. Convex is a platform designed to make backend engineering a thing of the past. We have a vision for the future where the vast majority of developers focus on building application code and leverage a small convenient set of abstractions that obviate the need for thinking about storage, execution, sync, queueing, or workflow.

There are a bewildering array of tools and technologies in the Infrastructure-as-a-Service landscape. These tools eliminate inconveniences but [they don’t make problems go away](https://blog.convex.dev/missing-link/)—developers still end up having to compose these tools into a platform that runs their application. Convex isn’t a bag of tools. We have a lot of features coming soon, but these aren’t  standalone building blocks that introduce another component to an already complex infrastructure. All features in Convex are designed to work together seamlessly: tying actions, mutations, queries, subscriptions and end-to-end reactivity together without developers dealing with glue code or data synchronization.

We think you’re going to like what’s coming soon but there’s a very good chance that Convex already meets your needs. [Give it a try](https://convex.dev) and say farewell to thoughts about infra overhead and data management.

Convex 0.5.0 introduces built-in support for actions — arbitrary lambda functions that run in a Node.js environment on the Convex cloud. We use the term action to differentiate these functions from Convex mutation and query functions, which enable transactional writes and dynamic subscriptions that update whenever data changes.

Ready for Actions

Jamstack TV

Today, a new wave of innovation is making databases approachable for developers without requiring a specialized skillset, opening the door to a whole new category of ambitious web experiences. In this panel, you’ll hear from the people pushing the boundaries of database technology and learn what’s possible — and what’s coming.

Jamstack Conf 2022 Panel: The Future of Databases

I was recently on a [Future of Databases panel](https://youtu.be/yIoFpfr4vA0) and asked what technological shifts spurred the new wave of serverless databases. I believe this shift has been largely sociological: a desire to make problems go away and instead focus on product. However, there’s a looming technical challenge we need to solve if the serverless revolution is to reach its goals.

This looming challenge is *end-to-end data management*. The serverless movement has eliminated so many problems but it has left application developers with the hardest one of all: managing distributed state.

# Building products not scaffolding

The “serverless revolution” is a marked shift in how developers build their apps, driven heavily by platforms like Netlify and Vercel. Once upon a time, building a scalable, performant web application meant deploying web servers, distributing static assets to a CDN, dealing with request routing, managing release process. These issues are largely a thing of the past for developers in the serverless ecosystem. Developers build apps in a framework like Next.js, put up a PR on GitHub, test out automatic previews while their code is reviewed, merge their changes, and then their app gets automatically deployed around the world. It’s a tremendously convenient workflow that [you should](https://www.netlify.com/) [try out](https://vercel.com/) if you haven’t already.
CDNs and Platforms as a Service aren't new technologies but what is new is how developers think about their applications. It wasn’t long ago that the vast majority developers wanted to own their stack. In the days of the [LAMP stack](https://en.wikipedia.org/wiki/LAMP_(software_bundle)), it was a rite of passage to firewall a Linux machine, configure MySQL, set up replication, etc. If a company outsourced these to a third party, that was usually considered a sign of low standards, that it wasn’t worth it to invest in a faster, cheaper, secure, and reliable self-hosted backend. (Even I was responsible for [migrating Dropbox off of S3](https://www.wired.com/2016/03/epic-story-dropboxs-exodus-amazon-cloud-empire/), but that’s a story for another time.)

These days using a platform for hosting your web app is a sign of *high* standards. It’s extremely difficult for a developer to replicate the performance, scale, and reliability of a third-party serverless backend, let alone approach the productivity gain from not having to build their infrastructure. Use of third-party platforms is also often a sign of better priorities. I can’t remember the last time I heard people passionately arguing about Red Hat vs Debian. These days most application developers just want to focus their time on building actual products that benefit actual end users.

# Making problems go away

If we were to differentiate “serverless” (Netlify/Vercel) from “Infrastructure as a Service” (AWS/Google Cloud), I think the key is that serverless isn’t just about providing tools that simplify infrastructure, it’s about making problems go away entirely. EC2, S3, ECR, ELB and Route53 are all amazing technologies, but they generally require a lot of work to combine into an actual solution to a problem. AWS hasn’t made infra teams go away, they just build infra on AWS now. Modern serverless platforms make problems go away, to the point that there’s a new generation of developers who don’t even know that some of these problems existed.

<BuildWithConvex />

# The missing link is data

Serverless platforms have improved life for application developers… up until they have to deal with managing data. Reliable hosted databases exist now, which eliminate a lot of operational concerns, but this has never been the real challenge when dealing with databases. The real challenge is a distributed systems one: keeping data on one host (the browser) consistent with another (the database). Gone is `postgresql.conf`; in is WebSockets, polling, API servers, SWR, caching, server components.

This challenge is perhaps best illustrated via this 2-minute demo:

<Video videoId="AdrO6IvHjRQ" />

The missing link in the serverless vision, the vision of truly making problems *go away*, is data. There are a lot of companies out there making databases more convenient to *run*, but not that much more convenient to *use*. These platforms are useful but they don’t eliminate this class of data management problems. The serverless revolution deserves a counterpart that solves the problem of data management and we believe that solution is Convex.

The serverless movement has eliminated so many problems but it has left application developers with the hardest one of all: managing distributed state.

The serverless revolution deserves more than just a serverless database

The future of databases is Convex. This video originally aired at Jamstack 2022, highlighting the next evolution of modern web development architecture.

The future of databases is not just a database

One of our goals here at Convex is to provide the right level of functionality at the right time in your application’s journey. From the prototype where you want to side step binding decisions to a mature high usage application with many developers, we strive to provide the right tools at each step of your journey.

We’ve all used tools that are easy to get started with, but after a certain level of complexity their tooling or abstractions break down on some critical dimensions (think untyped languages for building large applications). Or we’ve used tools that feel like they require a PhD to even understand what they’re talking about (what’s a transclusion anyways?)

We hope that Convex is easy to get started, and then provides easy to use “power tools” as your application grows both in terms of developers and complexity.

Here are a few areas where this philosophy shows up in Convex:

## Schema (less-ish)

When you are first getting started, you may not fully know the schema of your application. You’ll just be stuffing data into tables. You’ll throw together different screens, you’ll test your application with your users, and only over time will find product market fit, that blissful state when users are clamoring to use your product. By this time you’ll learn what your application is truly about and only then want to formalize your schema. This is particularly helpful as your team grows larger, and your application gets a bit more sophisticated.

Turns out Convex has been tracking your schema all along! You can see it in the Convex dashboard by clicking on the Schema button next to each table. You can then take that Schema and decide to enforce it. Using schemas in Convex will only get better, stay tuned for some new features in this space soon!

![image](https://cdn.sanity.io/images/ts10onj4/production/7cc06b8db9a68d1da9053a13f27848207062edac-2000x1143.png?w=450)

## Indexes

Similar to schemas, indexes are more useful as you learn the performance characteristics of your app. For example, let’s say you have a Users table. But turns out you’re often querying it by email rather than by `Id` (the default index). Once you get enough users (yay! congrats!), you may find that this query by email is a performance bottleneck. At this point you’ll probably want to add a `by_email` index to speed up that query.

We’re actively working on adding custom indexes to Convex. Watch this space for an update in the near future. This is another great way we’re ensuring that you have all the tools you need to work with your data as your app scales.

## Caching

All Convex functions are [deterministic](https://docs.convex.dev/understanding/convex-fundamentals/functions#determinism). One great side effect of this is that it’s very easy to cache a query’s output. Convex does this automatically for you! If you call the same query again with the same set of inputs, you will get the data back faster because it’s already cached. Even better, if you’re subscribed to a query then we update that query only when the dependencies change, and then cache the results. There is no excess computation or data fetches due to polling or multiple requests for the same query.

Put another way, you don’t have to worry about whether your cache is valid or not, you just get the benefits without any extra work. This means as more people use your app the more the cache is used. Convex’s performance scales to your usage.

We hope this makes Convex the right product to chose for your application’s data needs. You don’t have to worry about the system locking you into a decision that seems convenient at first but is a headache to deal with later. We’d love to get your feedback on how we can make these decisions easier, [join our Discord](https://convex.dev/community).

You don’t have to worry about the platform locking you into a decision that seems convenient at first but is a headache to deal with later.

The platform you need, when you need it

While the TIOBE language rankings might place traditional functional programming languages solidly outside the top 10, the paradigms advanced by those languages are foundational to the popular emerging technologies engineers are using today.

Ideas around immutability, purity, and dataflow programming are quietly saturating our technologies like Bitcoin and React. And while these early inroads are enabling exciting new ways to build large and dependable applications, there is still much to do to realize the full potential of this revolution.

But first, let’s go back to the advent of some of these functional patterns through a comparison of the very different ways Haskell and C achieve a common programming task.

# **Getting beyond imperative**

Given both resource and compiler limitations, most early mainstream programming languages were very close analogs to the underlying machine code. "Take this value from here, then do this to it, then put it here." These are imperative programming languages, which command the computer to do fairly concrete things as if moving boxes around a warehouse.

As an example, lets double an array of values imperatively in a language similar to C:

```c
for (int i = 0; i < 5; i++) {
	my_list[i] *= 2;
}

```

Examined critically, this code expresses much beyond the programmer’s desire to simply double the list. It's not just what the programmer wants done, it's explicitly how the machine should do it. Perhaps, as this code instructs, the right way to do this is to walk through each item in the list. And yes, perhaps there should be an incrementing variable that represents the index as we go. But those are "how" not "what" choices.

In contrast, later functional languages leveraged more powerful compilers and fewer resource limitations to provide new abstractions that were closer to mathematical functions.  Compare our C code above to the equivalent Haskell:

```haskell
let newList = map (* 2) myList
```

This more directly says: create a new list which is this existing list doubled.

In this manner, functional languages introduced higher level abstractions that were akin to declarative transforms of values rather than literal commands to the machine. Functional idioms are less "do this and then do that" and more "data like this can transform in this way to create data like that."

Another way to think of this: you're not getting and setting data, you're just making rules about how data can *flow*.

# **Immutability and purity**

One cool property of these functional languages is they could explore the benefits of making variables immutable by default, because a variable no longer intrinsically mapped to a specific memory address.

And Haskell did. Recall our list doubler:

```haskell
let newList = map (* 2) myList
```

At the language level, everything here is immutable. `myList` never changes, and `newList` also can never change. Even if the compiler opts to have the underlying memory change in place in the final machine code, the compiler still knows that the *programmer* has no specific need for this data to change in place.

In addition to default immutability of variables, Haskell functions by default are *pure*. Their behavior does not and cannot read or write any dynamic global or environmental state like I/O, time, or random number generators. Pure functions have the consequence, then, of always producing the same output for the same input. In fact, it’s impossible for them to produce a different result: they depend on nothing besides their arguments, and computers don't wiggle bits around just for fun! [^1]

This property means you only have to run them once for a given set of arguments, since the result will be the same every time. This is clearly great for automatic caching and [memoization](https://en.wikipedia.org/wiki/Memoization). But it's also very useful for reproducibility: you know that anyone, on any computer can also run these pure functions and replicate your result, because your algorithm doesn't depend on any environmental factors.

Known immutability has an important relationship to functional purity: It’s the necessary ingredient to create “real purity” instead of merely “fingers-crossed purity." If you know that both the arguments and the global state are immutable, then you also know the entire function is guaranteed pure. Absent an environment that can enforce these constraints, when we utilize pure patterns, we’re merely achieving “fingers-crossed purity”. The programmer has to be careful that their functions don’t violate any of the rules!

*Quick note: we're going to use the terms "purity" and "determinism" interchangeably from here on out, but generally speaking, purity is a stricter form of determinism. Impure deterministic functions can also make a consistent change to global (non-return value) state; but if you consider those effects part of the return value, the ideas are essentially equivalent. [This stackoverflow answer is a good summary.](https://stackoverflow.com/a/40296536)*

## **Mainstream purity: Safe HTTP methods**

An example of purity you're probably already familiar with is HTTP GET, the most important [safe method](https://datatracker.ietf.org/doc/html/rfc7231#section-4.2.1) in HTTP.

Every dutiful REST practitioner knows that your GET needs to return the same result for the same URL and ETag[^2]  every time and have no side effects. And what benefits do we get out of that? Massive global CDN networks that can serve data out of caches! Faster websites, energy savings, lower hosting bills.

However, in practice this is almost always merely “fingers-crossed purity”, since our GET controllers are not typically written in languages and environments that truly prohibit state changes.

# **Chained purity and deterministic dataflow**

If you pass the output of one pure function as the input of another, you create a long deterministic chain[^3] that retains all the useful reproducible/verifiable properties of any individual link. You can append new pure links over time to a previous chain to extend it.

When you chain pure functions, you’re creating a kind of [dataflow programming](https://en.wikipedia.org/wiki/Dataflow_programming) system. You can think of dataflow programing as a graph of deterministic computation that can be recomputed consistently by replaying it.

As you probably know, some of the world's most popular types of pure functions to chain are hash functions. Chains of hash functions have been used to create things like git, and some crypto stuff you may have heard about...

## **Mainstream deterministic dataflow: Blockchain, Ethereum and the EVM**

Back in 2008, Satoshi Nakamoto's [famous paper](https://bitcoin.org/bitcoin.pdf) emerged, laying out the ideas behind bitcoin and blockchain generally. Like git on steroids, if we have a sequence of checksummed and signed payloads, we can append a new checksummed and signed payload to it, and everyone else can re-run the same[^4] pure functions we did to collectively reproduce and validate our new payload. In bitcoin specifically, these payloads were monetary transfers of a new virtual coin, creating a system with an immutable, fully verifiable ledger where ownership of coins cannot be faked and history cannot be rewritten.

Bitcoin was a collection of known ideas, but composed together clearly and cleverly with striking implications for the future of decentralized systems. But very soon after, people starting contemplating other non-monetary payloads that could be put onto blockchain systems. Like domain names, or real estate titles, or... units of computation! If instead of coins, the payloads were code that everyone could run, you could make a big distributed computer that could be used to create *any kind* of other thing.

This is a great idea, right? Well, not if that code reads some file on the original developer's machine, or uses randomness, or tries to run an HTTP request to a web site that stopped paying its hosting bill. If it does anything like this, everyone can't generate the same ledger by running the same payload code, and the underlying verifiable principles behind blockchain are violated.

So computation on blockchain, too, requires computational determinism.

In Ethereum's case, currently the most popular general-computational blockchain, whatever code you write eventually gets compiled into byte code that runs on the Ethereum Virtual Machine, the EVM. And sure enough, from [the EVM docs](https://ethereum.org/en/developers/docs/evm/):

> The EVM behaves as a mathematical function would: Given an input, it produces a **deterministic** output. It therefore is quite helpful to more formally describe Ethereum as having a state transition function:
> 

Therefore, the code you write in your input program won't be allowed to do any non-deterministic things in order to be able to compile to EVM's intentionally limited instruction set. Thus the EVM enforces “real determinism." This is essential, as the whole point of Ethereum's big distributed computer is to be resistant to manipulation and abuse, so “fingers-crossed determinism” doesn’t go far enough.

# **Functional paradigms and pure dataflow power the modern web**

In our opinion, the most exciting recent emergence of functional paradigms is in web development. About ten years ago, everyone’s favorite reactive framework appeared on the scene and changed everything.

We’re talking about [Elm](https://en.wikipedia.org/wiki/Elm_(programming_language))... of course!

For many developers, Elm was the first whiff of something like *functional reactive programming*. Generally, in this pattern there is some *declarative* way you express how your app maps underlying data into a visual form. These declarations are deterministic. When any mutation changes the underlying source data, that mutating code isn't directly responsible for fanning-out updates to all interested UI components. Instead, a runtime knows what components depend on this data and re-runs the appropriate declarative transforms over the new data to update the associated UI.

This pattern ends up being a very nice way to structure applications with UI. It really addresses one of the hardest parts about the traditional, ad-hoc way visual applications often manage the connections between visual UI and underlying data. Without a deterministic-subscription exchange like FRP provides, every mutation has to maintain a group of one-to-many mappings that represent "who cares about this change?" And even in many cases "how should they change?"

Managing this problem without a principled framework like FRP tends to lead to fragile spaghetti-like code expressing data dependencies all over your codebase. Keeping all these dependencies in sync as the app evolves is a *huge* headache for developers and slows down progress on the product. So, kudos to Elm for showing the web world a better, functional way.

Oh yeah, and then a year after Elm, Facebook released React. It did *okay*.

React basically allowed you to get a decent first taste of that happy FRP life without completely switching your web programming language. In particular, you had a way to set component-local state and specify a reactive pure transform that would turn that state into DOM that the browser can render.

Here's a silly app called "Fruit Trader" that will illustrate how the modern web utilizes pure dataflow, starting with how far React alone takes us:

![image](https://cdn.sanity.io/images/ts10onj4/production/f2d6780b99965d5f696e6b2ad0c910c5fdaee0e9-1000x563.jpg?w=700)

## **Redux / MobX and the state management gang**

Granted, React helps you achieve the very end of the deterministic reactive dataflow puzzle–the place where a single component’s underlying state is updated and that component is re-rendered. But it doesn’t help with data exchange between more complex shared state which *many* components depend on.

So very soon after the release of React, projects like Redux, MobX, and others showed up to take on the completing-the-dataflow-graph problem which the web ecosystem has labeled "state management". Instead of the state graph just having a single edge with connecting a component with its specific data, these libraries let you create complex graphs with relationships between dependent data, any number of intermediate transformations, and then finally that react component-local state.  Each of these hops along this graph must be–you guessed it–deterministic:

![image](https://cdn.sanity.io/images/ts10onj4/production/2db01ce9f7f48e5de8055a4a86e57fa564c58612-1000x563.jpg?w=700)

Redux even has a page on its [three principles](https://redux.js.org/understanding/thinking-in-redux/three-principles#changes-are-made-with-pure-functions) that emphasize the functional underpinnings of Redux, of which two are:

> State is read-only
Changes are made with pure functions
> 

Mutations in these systems involve modifying only the root object(s), and then letting the dataflow graph react, transform, and finally re-render:

![image](https://cdn.sanity.io/images/ts10onj4/production/ccce2e4d5726ee7694519cdccb97fe217d2a0b26-1000x563.jpg?w=700)

[](https://redux.js.org/understanding/thinking-in-redux/three-principles#changes-are-made-with-pure-functions)And with these libraries at hand, client-local state is pushing pure dataflow around, and we have clear principles about where data changes and how all the reactions cascade out. State management in web programming is solved, right? Not so fast.

# **The bummer about backends**

With React and its rich ecosystem of state management libraries, building powerful web apps with clean abstractions is becoming easier and easier–as long as all the state necessary to service those applications is already in the browser.

The problem is those darn servers and that critical state that is stored and distributed on them. They're in the state graph, but they're very poor participants compared to our powerful browser-side libraries.

At Convex, we believe there are two main flaws that make them particularly underpowered for what modern apps need:

1. The have poor support for the reasoning about immutability vs. mutability and therefore for purity/determinism
2. They have very limited semantics around state subscription and invalidation

Let's talk about each.

## **Backend bummer: fragile server-side determinism**

As we’ve explored, we already have safe GET! But really, that only restates the goal, because backends today are in the “fingers-crossed purity” zone. It’s still up to us to ensure the controllers backing our GET endpoints only read state and do not change state.

So the server-side help for this objective amounts to little more than *be careful*. As long as we're very careful, we can probably cache our GET requests and trust our ETags.

At Convex, we believe that backend systems should provide real determinism, and therefore enable anxiety-free GET safety and perfect caching. And they should do so without requiring we completely replace our backend programming languages and library ecosystems.

## **Backend bummer: limited subscription semantics**

Even if our backends start providing real deterministic guarantees, this makes our GETs requests powerful and robust *only* against misbehaving GETs.

Eventually, one of our POST endpoints that *is* allowed to change state... does its thing and changes some backend data. Presumably some rows in some tables on the backend are new or updated, and therefore some GET requests need to be recomputed and some browsers' state updated:

![image](https://cdn.sanity.io/images/ts10onj4/production/148a467a9c207be852cbf17d51a43d3d882bcc0a-1000x563.jpg?w=700)

Keeping track of these mappings is a very tricky problem, especially as our applications grow. The server acts as a kind of black-box where state interchanges without helping the application know what needs updating. It's a giant, ugly interruption in our beautiful deterministic dataflow state graph!

In theory, subscriptions should solve this problem. But backend systems today do not have sufficiently powerful subscription semantics. They’re built on databases that only allow you to do row-level or table-level subscriptions. But what our components are *actually* dependent on is a controller function running, which may fetch records from multiple tables, and might only look at some subset of those records, and then may transform that data into a nice JSON object for the application's model. It might do all kinds of things to create that object.

But that object is really what they depend on.  And today, there's not really a great way to subscribe to... whatever all of that is. We just want to know when that final JSON object would change.

And so it often falls to our app to somehow keep track of which POSTs imply what table updates which map to the outputs of our GET endpoints. The app has to stay in lock-step with what's happening within the controllers and the backing database so it can create subscriptions in the database's language instead of the app's language. [^5]

# **Enter Convex**

Our mission at Convex is to solve this exact problem. We believe frontend engineers deserve to have the backend and the database manage state in the app's language. We think it's time for the backend to center its design around the needs of the app instead of the other way around.

And this is what our product is. Convex is a backend that supports automatic, pervasive ideas of immutability and mutability, and it leverages that information to provide controller and function-level subscriptions to backend state.

Basically: Convex completes your app's deterministic dataflow graph. The backend feels like a seamless, persistent extension of your state management.

Want to see for yourself? [Sign up for our beta wait list.](https://convex.dev/) And if you’re excited to help us build Convex, [we’re hiring!](https://convex.dev/jobs)


[^1]: Well, bit flips do happen, but we have little idea how much *fun* the machine finds them to be.

[^2]: One way to think about ETags is they version the "global" read-only state for a given URL and its params.

[^3]: Or tree, or DAG, or whatever. The dataflow graph doesn't have to be linear, but git's happens to be. And (eventually) blockchain's is as well, given the property that the majority of actors "vote" to kill off invalid forks via cryptographic verification and consensus.

[^4]: Technically, the signed aspect of blockchain does not use the "same" algorithm as the originator of the payload, but is the asymmetric public analog of that algorithm that can verify (but not forge) the private provenance

[^5]: Or, let's be honest, we just throw our hands up in the air, and invalidate and re-run everything when data changes *just in case.*

Ideas around immutability, purity, and dataflow programming are quietly saturating our technologies like Bitcoin and React. And while these early inroads are enabling exciting new ways to build large and dependable applications, there is still much to do to realize the full potential of this revolution.

Perspectives