Web Development at Berkeley

a year ago

Building an Application Portal on Convex

Hey there! We're Web Development at Berkeley (WDB), UC Berkeley's premier web development and design-focused organization that loves bringing full-stack web development education and hands-on industry development and design experience to UC Berkeley students. These past few months we've been building a full-stack web app with Convex, a backend-as-a-service that replaces your database, servers, cache, and realtime message queue with a simple JavaScript/TypeScript interface. Convex has made our developer experience pretty amazing, and we're excited to share how we've integrated it within our own workflow!

Overview of Our Project

This semester, our student organization wanted to develop an application portal that would unify the process for applying to student organizations on one convenient website, providing unique features like automatically-saved student applications and powerful collaborative tools for organization admins to view application submissions. We have a working demo version of the app along with the associated GitHub repo here if you'd like to get a look behind the scenes!

To implement this project, we decided to use Convex due its potential for super fast development speed and low barrier to entry for getting a backend up and running. Because Convex runs in the cloud, we didn't have to manage our own servers, and we also didn't have to do much configuration to set up the data tables that we needed. Convex abstracts away a lot of these database-level issues such as thread safety and data synchronization, allowing us to develop more efficiently than ever before.

For some context, here's a quick overview of how we structured our application. We started by creating a few data schemas in Convex which you can see at our schema.ts file:

• users: to store all users that log into the application website.
• applications: stores the information about each club's application.
• submissions: represents the submission of an individual user for a specific application (i.e. a user/application pair).
• admins: stores a user/application pair indicating that the user is an admin for that application.

Convex provides the ability to create query and mutation functions that run in the cloud; queries return data which is automatically updated and synced (similar to an HTTP GET request) and mutations modify the database (like an HTTP POST request). We created a couple queries and mutation functions to go along with our schema, all of which you can view in the convex folder of our repo:

• createUser (mutation): creates a user based on the auth entity currently logged in
• getSubmission (query), updateSubmission (mutation): gets/updates the submission associated with the provided application and the optionally provided user or the user currently logged in
• getApplication (query): gets an application by string identifier

We separated some functionality into utility functions which are used across multiple queries and mutations, including:

• getUser({db, auth}) which returns the user object currently logged in
• isAdmin(db, user, application) which returns if a specific user is an admin for an application

Why We're Loving Convex

The useQuery and useMutation Hooks

In the past when we've built a traditional backend with technologies such as Express, hooking up React components with the backend usually ended up being quite a challenging task. In order to communicate with the backend, we need to send HTTP requests, and write additional logic to make sure we're updating our application with new data as it comes in. Convex eliminated this completely, as the useQuery and useMutation hooks provided a layer of abstraction from directly communicating with the backend. Throughout our entire frontend, we utilized these hooks in place of the HTTP requests that we would normally need to use:

export default function Application({ id }: ApplicationProps) {
  const [currentStep, setCurrentStep] = useState(0);
  const [latestCompletedStep, setLatestCompletedStep] = useState(0);
  const application = useQuery(api.applications.get, id);
  const submission = useQuery(api.submissions.get, id);
  // we can now just use the queried application and submission directly
  // no need to worry about stale or out-of-sync data!
  if (!submission?.submitted) {
    // return the application frontend view (omitted)
  }
}

Another benefit of this query/mutation model is that there was no parsing required to utilize the data returned from the backend. We could directly use the returned data, as it was already a TypeScript object. The application variable declared above is already typed with all the proper fields we declared in our schema, so it was easy to make sure that our frontend was always consistent with the data format our backend was returning.

Reactivity

Convex's reactive-by-default design was another aspect which made developing many of our differentiating features a breeze. Traditional relational databases are not built with reactivity in mind, which means that we would have needed to implement our own solution to ensure that the frontend has data updated in real-time (potentially with some pretty complicated WebSockets logic). However, Convex is reactive by default, so all queries made with useQuery receive real-time updates. This made it incredibly simple during our development process to propagate changes from the database to the frontend instantly. We leveraged this functionality by implementing an autosave feature which allows multiple sessions of the same application to be active simultaneously. We also synced submission data across our admin views so that multiple admins can view and edit a single applicant's data together in real-time, allowing for close collaboration!

export default function SingleSubmissionPage({ params }: ParamsProp) {
  // ...

  // The application, submission, and associated notes are all updated in real-time across everyone's web client by default
  // No need for any extra logic!
  const application = useQuery(api.applications.get, {
    applicationId: params.applicationId
  });
  const submission = useQuery(
    api.submissions.get, {
      applicationId: params.applicationId,
      submissionId: params.submissionId,
      deleting
    }
  );
  const notes = useQuery(api.notes.get, {
    submissionId: params.submissionId,
    deleting
  });
  
  // ...

  return (
    // render the frontend for the single submission page
  );

No race conditions

When working with Convex, we didn't have to worry about things like transactions, allowing the distributed aspect to be abstracted away. Mutation functions are automatically thread-safe. For example, one of our first tasks was to implement a function which creates a new user in our DB. With other backend, this might produce unsafe behavior if implemented like this pseudocode:

// not thread-safe!
if (user does not exist in database):
    // another thread could run here!
    create user
}

The user could be created twice if another thread runs in the commented section. However, Convex uses Optimistic Concurrency Control, so this code is completely safe using Convex. Meaning, when trying to create a user, we could simply check for the existence of the user in the database and create it if nothing was returned - no need to worry about a user being created between time of check and time of use! Our function became as simple as this (some parts omitted for brevity, see the repo for the full file)

export default mutation(async ({ db, auth }): Promise<Id<"users">> => {
  // Check if we've already stored this identity before.
  const identity = await auth.getUserIdentity();
  if (!identity) {
    throw new Error("Called getUser without authentication present");
  }

  const user: Document<"users"> | null = await db
    .query("users")
    .withIndex("by_tokenIdentifier", (q) =>
      q.eq("tokenIdentifier", identity.tokenIdentifier)
    )
    .first();

  if (user !== null) {
    // ... Patch not shown
    return user._id;
  }
  // If it's a new identity, create a new `User`.
  return db.insert("users", {
      // ... Arguments not shown
  });

This made working with the database (as well as writing helper functions) much simpler.

TypeScript instead of a database query language

Another aspect of Convex that stood out for us was the ability to skip most of the learning curve for a database-specific query language (or a wrapper for one), which we would have faced if we were to use something like MongoDB or another traditional database. Convex has a small core database API which allows for powerful database-level performance when making queries. Along with this, however, the fact that Convex functions and the database are tightly coupled means that additional filtering and sorting on your data can be done directly in TypeScript, with no new syntax or APIs to learn. This meant we could focus on writing our application logic with a language we already all knew, all while knowing that our database logic would still be performant. The benefits are clear: no limits on what you can do with regards to filtering, sorting, selecting, etc. (think what you can do with just an array of objects in TypeScript — anything!)

For instance, as we were designing our admin table view for viewing a list of submissions, we wanted to incorporate server-side filtering and sorting on our data. Writing this logic involved using Convex's database indexing for the core data filtering, and then for further refining the result we could simply use TypeScript's Array.sort() and Array.filter():

const submissions: Document<"submissions">[] = (
      await db
        .query("submissions")
        .withIndex("by_application", (q) =>
          q.eq("application", application._id)
        )
        .collect()
    ).filter((submission) => {
      let matchesFilters = true;
      // keyword matching and filtering logic here!
      return matchesFilters;
    });

    if (argumentsModifiers?.sorting) {
      submissions.sort((a, b) => {
        // sorting logic here!
      });
    }

Authentication

Setting up a proper secure authentication flow can be quite challenging, as we've seen in our previous Express-based projects. Thankfully, Convex has had great integration with Auth0, making the process of setting up a user signup flow extremely easy. Integrating authentication-based logic was also made seamless by the auth argument being automatically passed in to every Convex query and mutation function. We used this object to automatically associate the submission being updated/fetched with the user currently logged in if no user was provided:

// convex/common.ts
export const getUser = async ({
  db,
  auth,
}: {
  db: DatabaseReader;
  auth: Auth;
}): Promise<Document<"users"> | null> => {
  // Use the auth identity from Auth0 to get the user's logged-in state
  const identity = await auth.getUserIdentity();
  if (!identity) {
    throw new Error("Called getUser without authentication present");
  }

  return await db
    .query("users")
    .withIndex("by_tokenIdentifier", (q) =>
      q.eq("tokenIdentifier", identity.tokenIdentifier)
    )
    .first();
};


// convex/getSubmission.ts
export default query({
  args: {
    applicationId: v.string(),
    submissionId: v.optional(v.string()),
    deleting: v.optional(v.boolean())
  },
  handler: async (
    { db, auth }, {applicationId, submissionId, deleting}
  ): Promise<Document<"submissions"> | null> => {
    // ...

    // Get the current user with a helper function
    const user: Document<"users"> | null = await getUser({ db, auth });

    // ...

    const submission: Document<"submissions"> | null = await db
      .query("submissions")
      // We can now use the user ID from the authentication data to fetch the correct submission!
      .withIndex("by_userAndApplication", (q) =>
        q.eq("user", user._id).eq("application", application._id)
      )
      .first();

    return submission;
  }
});

Convex handled the core logic for interfacing with Auth0 for us, meaning we could focus on actually building the authentication-based logic.

Final Thoughts

Working with Convex has made our development workflow so much more streamlined—we no longer have to worry about the complexities of traditional frontend-backend communication, enabling us to build out the core of our new application portal at a blazing-fast speed. If the stuff we've shown here seems interesting, you should definitely try out Convex by heading over to their website. And, of course, check us out at Web Development at Berkeley for the latest updates about web dev at Cal!