Ian Macartney

3 months ago

Supercharge `npm run dev` with package.json scripts

npm run dev is the standard for "run my website locally," but how does it work? How can we expand its functionality? In this post we'll look at:

• How to configure what npm run dev does.
• How to decompose complex commands into granular units.
• How to run multiple commands in parallel.
• How to run pre-requisites without losing normal Ctrl-C behavior.
• How to add seed data (if none exists) when starting up a Convex backend.

As a motivating example, here are some scripts defined in the convex-helpers example app. We'll cover what each piece does

  "scripts": {
    "dev": "npm-run-all --parallel dev:backend dev:frontend",
    "build": "tsc && vite build",
    "dev:backend": "convex dev",
    "dev:frontend": "vite",
    "predev": "convex dev --until-success",
    "test": "vitest"
  },

How and where they're defined

npm run runs commands that are defined in your package.json in your project's workspace. These commands are often pre-configured when you start your repo from a command like npm create vite@latest with commands for:

• dev: Run a development environment. This often includes autor-reloading the UI when files change. For Vite this is vite and Next.js is next dev.
• build: Build the website for deployment. This will generally compile and bundle all your html, css, and javascript. For Vite this is vite build and Next.js is next build.
• test: Run tests - if you're using Jest, it's just "test": "jest" or vitest for Vitest.

Here's a basic example from Next.js:

// in package.json
{
// ...
  "scripts": {
    "dev": "next dev",
    "build": "next build",
    "start": "next start",
    "lint": "next lint"
  },
//...

Here you can run npm run dev or npm run lint etc.

You can learn more about npm run in the docs.

Why use scripts?

It's a fair question why one would put commands that already so simple into package scripts. Why not just call jest or vite or next build? There's a few good reasons:

1. You can save the default parameters for commands so you don't have to remember or document the "standard" way of starting something. We'll see below how you can configure it to chain commands and run others in parallel.
2. It allows you to easily run commands that are installed by npm but not globally accessible from your shell (terminal).¹ When you install things like npm install -D vitest, it installs vitest into node_modules/.bin.² You can't run vitest directly in your shell,³ but you can have a config like: "scripts": { "test": "vitest" } and npm run test will run vitest.
3. It always runs with the root of the package folder as the "current directory" even if you're in a subdirectory. So you can define a script like "foo": "./myscript.sh" and it will always look for myscript.sh in the package root (in the same directory as package.json). Note: you can access the current directory where it was called via the INIT_CWD environment variable.
4. You can reference variables in the package.json easily when the script is run from npm run. For instance, you can access the "version" of your package with the npm_package_version environment variable, like process.env.npm_package_version in js or $npm_package_version in a script.
5. If you have multiple workspaces (many directories with their own package.json configured into a parent package.json with a "workspaces" config), you can run the same command in all workspaces with npm test --workspaces or one with npm run lint --workspace apps/web.

Does npm run dev work with yarn / pnpm / bun?

Yes! Even if you install your dependencies with another package manager, you can still run your package scripts with npm.

yarn # similar to `npm install`
npm run dev # still works!

You don't have to remember that npm run dev maps to yarn dev (or yarn run dev). The same goes for npx: npx convex dev works regardless of what package manager you used to install things.

Running commands in parallel

There are a couple packages you can use to run commands concurrently:⁴

1. npm-run-all
2. concurrently

We'll just look at npm-run-all here. Consider our example:

  "scripts": {
    "dev": "npm-run-all --parallel dev:backend dev:frontend",
    "dev:backend": "convex dev",
    "dev:frontend": "vite",
  },

This defines three scripts.

1. npm run dev:backend runs convex dev.
2. npm run dev:frontend runs vite.
3. npm run dev runs both convex dev and vite in parallel via npm-run-all.

Both outputs are streamed out, and doing Ctrl-C will interrupt both scripts.

predev? postbuild?

You can specify commands to run before (pre) or after (post) another command (say, X) by naming your command preX or postX. In the example:

  "scripts": {
    "dev": "npm-run-all --parallel dev:backend dev:frontend",
    "dev:backend": "convex dev",
    "dev:frontend": "vite",
    "predev": "convex dev --until-success",
  },

This will run convex dev --until-success, before the "dev" command of npm-run-all --parallel dev:backend dev:frontend.

Chaining with "&&"

For those used to shell scripting, you can run two commands in sequence if the previous one succeeds with commandA && commandB. This works on both Windows and Unix (mac / linux).

However, there's a couple advantages to just using pre-scripts:

1. You can run either command with npm run dev --ignore-scripts to not do the "predev" script, or npm run predev to explicitly only do the "predev" step.
2. The Ctrl-C behavior is more predictable in my experience. In different shell environments, doing Ctrl-C (which sends an interrupt signal to the current process) would sometimes kill the first script but still run the second script. After many attempts we decided to switch to "predev" as the pattern.

Run interactive steps first

For Convex, when you first run npx convex dev (or npm run dev with the above scripts), it will ask you to log in if you aren't already, and ask you to set up your project if one isn't already set up. This is great, but interactive commands that update the output text don't work well when the output is being streamed by multiple commands at once. This is the motivation for running npx convex dev --until-success before npx convex dev.

• convex dev syncs your functions and schema whenever it doesn't match what you have deployed, watching for file changes.
• The --until-success flag syncs your functions and schema only until it succeeds once, telling you what to fix if something is wrong and retrying automatically until it succeeds or you Ctrl-C it.
• By running npx convex dev --until-success, we can go through the login, project configuration, and an initial sync, all before trying to start up the frontend and backend.
• The initial sync is especially helpful if it catches issues like missing environment variables which need to be set before your app can function.
• This way the frontend doesn't start until the backend is ready to handle requests with the version of functions it expects.

Seeding data on startup

If you change your "predev" command for Convex to include --run it will run a server-side function before your frontend has started.

  "scripts": {
	  //...
    "predev": "convex dev --until-success --run init",
		//...
  },

The --run init command will run a function that is the default export in convex/init.ts. You could also have run --run myFolder/myModule:myFunction. See docs on naming here. See this post on seeding data but the gist is that you can define an internalMutation that checks if the database is empty, and if so inserts a collection of records for testing / setup purposes.

tsc?

If you use TypeScript, you can run a type check / compile your typescript files with a bare tsc. If your tsconfig.json is configured to emit types, it will write out the types. If not, it will just validate the types. This is great to do as part of the build, so you don't build anything that has type errors. This is why the above example did:

    "build": "tsc && vite build",

How to pass arguments?

If you want to pass arguments to a command, for instance passing arguments to your testing command to specify what test to run, you can pass them after a -- to separate the command from the argument. Technically you don't need -- if your arguments are positional instead of --prefixed, but it doesn't hurt to always do it in case you forget which to do it for.

npm run test -- --grep="pattern"

Summary

We looked at some ways of using package.json scripts to simplify our workflows. Who knew how much power could rest behind a simple npm run dev? Looking at our original example:

  "scripts": {
    "dev": "npm-run-all --parallel dev:backend dev:frontend",
    "build": "tsc && vite build",
    "dev:backend": "convex dev",
    "dev:frontend": "vite",
    "predev": "convex dev --until-success",
    "test": "vitest"
  },

• dev runs the frontend and backend in parallel, after predev.
• build does type checking via tsc before building the static site.
• dev:backend continuously deploys the backend functions to your development environment as you edit files.
• dev:frontend runs a local frontend server that auto-reloads as you edit files.
• predev runs before dev and does an initial deployment, handling login, configuration, and an initial sync as necessary.
• test uses Vitest to run tests. Note: npm test is shorthand for npm run test along with other commands, but they're special cases. npm run test is the habit I suggest.