Mike Cann
3 hours ago

Me and my son made a vibe coding app for kids!

This is Let's Build It. It's a Convex powered iPhone and iPad app designed to let kids build apps and games using just their voice.

Josh's project shelf in Let's Build It, with the apps and games he has madeJosh's project shelf in Let's Build It, with the apps and games he has made

I've been building it with my son Josh for the past year or so and I'm really quite proud of it. Josh can't actually read yet, so the whole idea is that he makes his own apps and games just by talking to his iPad. He's always got plenty of ideas, and I wanted to give him a way to exercise that creativity without me having to type something into Claude or Codex for him every single time.

And if he has an idea or wants to make a change later on he just says so, and then he's off playing again while it's building in the background:

The assistant in Let's Build It saying "Adding multiplication questions to the math challenge. Let me build it!"The assistant in Let's Build It saying "Adding multiplication questions to the math challenge. Let me build it!"

The best way to see what the app actually does is the little video Josh and I put together when we launched it, and it's free on the App Store if you fancy trying it out with your own kids.

Now along the way I learnt a bunch about building an agentic application that I think would be worth sharing with you guys. So I thought it would be good to take you through what's going on a little bit behind the scenes, and then give you a few quick lessons, so hopefully you can learn from my fails.

Let's take a look.

How it works behind the scenes

So the way it works is that the kid, Josh in this case, talks to the friendly assistant. When it's got enough to go on it says "Let me build it!", and that kicks off a job on Convex which runs his idea through a few different AI models, one after the other:

An iPad with the assistant saying "Let me build it!", and an arrow to Convex running four steps: Designer, Mockup, Design notes and CoderAn iPad with the assistant saying "Let me build it!", and an arrow to Convex running four steps: Designer, Mockup, Design notes and Coder

First the conversation goes into a designer step, which is running on GPT-6 Sol at the moment. It takes everything Josh said and turns it into a little brief for the game.

I didn't pick Sol on a whim by the way. I ran it in a blind test against various other models and to be totally honest I just couldn't tell them apart, so I went with Sol because it's the right price and it's also quick.

After the designer step an image model draws a quick mockup of what the game could or should look like. So it's kind of like an inverse screenshot of the finished thing:

The mockup the image model drew of a cake stacking game on the left, and the game that got built from it on the rightThe mockup the image model drew of a cake stacking game on the left, and the game that got built from it on the right

Sol then writes a few design notes to go with it. I found that giving the coding model that image helps it write the code better, particularly when combined with those notes. You see, often when a kid describes the thing they want they don't use enough language for the coding model to do a good job, particularly the cheaper models that I've been using. Here's everything that was asked for in one of the test games on the left, and the brief the designer wrote from it on the right:

The five short lines asking for a cake stacking game in a test, next to the much longer brief the designer step wrote from themThe five short lines asking for a cake stacking game in a test, next to the much longer brief the designer step wrote from them

Then all of that gets fed in, along with the conversation, to the coder model, which goes ahead and builds the game or app. That's Sol again. I'll talk about this a bit more later, but I went through a lot of different models and right now I've found Sol has got the right trade-off between cost, speed and quality.

As for the output from the coding model, it's just HTML, CSS and JS right now. So it's pretty simple for the job to save it to Convex file storage, and then it gets pushed down to the phone when you tap to load one of these apps:

File Storage in the Convex dashboard, with a game's HTML file and its mockup imageFile Storage in the Convex dashboard, with a game's HTML file and its mockup image

The whole process is pretty quick too. It takes about a minute from when the kid has finished talking to them being able to play it.

The UI was a bit of a nightmare

Now I do want to say that getting the UI to look like this was actually quite a painful process. I'm pretty happy with how it looks now, but getting there was a bit of a nightmare.

You see, back in June when I was doing a lot of this UI work I was using Codex a lot of the time, and I asked it to restyle the whole app to match some mockups that I had generated. It went off and opened something like 20 pull requests:

A list of closed draft pull requests from June, each one called "Apply Warm Maker styling to" another part of the appA list of closed draft pull requests from June, each one called "Apply Warm Maker styling to" another part of the app

And when I asked it how close it actually was, it reckoned it was maybe 35 to 45 percent of the way there:

Codex's reply: "I'd call the current auth slice maybe 35-45% there visually. It has the palette and some surface language, but not the composition."Codex's reply: "I'd call the current auth slice maybe 35-45% there visually. It has the palette and some surface language, but not the composition."

So I just binned the whole lot šŸ˜…

I think the lesson at the time was that coding models were just not good enough at design, or at least the OpenAI ones weren't. It would be interesting to see how well Opus or the current batch of models does, because I think they would do way, way better.

Anyways, what I ended up doing was taking one screen, one bit of the UI at a time, and putting it together in Storybook. Then I could iterate on smaller chunks like that, with a mockup from an image model to show what I actually wanted at the end of it:

My message to Codex with a mockup of the voice screen attached, next to the speech bubble component it built in StorybookMy message to Codex with a mockup of the voice screen attached, next to the speech bubble component it built in Storybook

And eventually, after a lot of iteration and time, I got it to a state I was happy with.

Lesson 1: BDD

Now I promised a few lessons, and the first one is to write down what your app is supposed to do in plain English, and then test that.

You see, when I started this project about a year ago I was finding that the agents were often drifting away from what I originally asked them to build. So I decided to pull out an old tool from my toolbox called BDD.

BDD, or behaviour driven development if you haven't heard of it, is a process where every feature you want in your app starts life as a story written in semi-structured natural language. I could probably do a whole video just on this topic alone, but the idea is that you write down what you expect your app to do, and then you have tests that enforce each line of it. Here's one of those stories from Let's Build It:

Feature: Project Editing
  As a parent helping a kid update a project
  I want to describe changes with voice or text
  So the project can be updated without losing the current version too early

  Scenario: Editing keeps the current version live until the new version is ready
    Given a parent is authenticated
    And a kid "Luna" exists for the parent
    And a project "Space Game" exists with 1 ready current version
    When the parent opens the edit screen for "Space Game"
    And the parent submits an edit request
    Then the project keeps the current version live while the next version builds
    And the new version is offered to the parent

So that's the requirement that when Josh asks for a change to his game, he's able to keep playing the old one until the new one is ready.

And here's the test behind that "Then" line (the app uses playwright-bdd for these):

// e2e/steps/project-editing.steps.ts
Then(
  "the project keeps the current version live while the next version builds",
  async ({ page, scenarioState }) => {
    await expect(page).toHaveURL(
      new RegExp(
        `/project/${getProjectId(scenarioState)}/${getOriginalVersionId(scenarioState)}$`,
      ),
    );
    await expect(
      page.frameLocator('[data-testid="project-html-iframe"]').getByText("Original version"),
    ).toBeVisible();
  },
);

It checks that we're still on the original version's page and that the original game is still the one showing (the stand-in game in the test just says "Original version" on it). Put the lines together and that story becomes a test that clicks its way through the app in a browser, so an end-to-end test:

The scenario ticked off line by line on the left. On the right the test's browser still shows the original Space Game, with a "Your new version is ready!" messageThe scenario ticked off line by line on the left. On the right the test's browser still shows the original Space Game, with a "Your new version is ready!" message

Now you might have bells going off in your head here, because it's kind of like a prompt that we give to an AI. Except you can deterministically enforce it, so you can have it run on the CI or wherever, to make sure your features and requirements are still being adhered to much further down in your development process.

A commit on main with all its checks passed: the unit tests, Storybook and seven shards of end-to-end testsA commit on main with all its checks passed: the unit tests, Storybook and seven shards of end-to-end tests

Now you as a human should be writing those behaviours, but the tests can be written by agents. So the question is, do those tests actually test what they should be testing to enforce your behaviour? And the answer is most of the time, but not always.

For example, we had a scenario with this line in it:

When the parent creates a kid with name "Max" and avatar "robot"

And it passed. But if you actually looked at it, the kid it made was a unicorn:

The home screen that scenario ended on, with kids called Luna and Max who both have the unicorn avatarThe home screen that scenario ended on, with kids called Luna and Max who both have the unicorn avatar

The agent had written the test so that it ignored the bit about the robot, so it was happily passing without actually checking the avatar at all.

To fix that we now have a check that runs before every commit, and if a test ignores part of its own story, it fails:

$ bun run test:bdd-check
BDD feature / step verification failed:

e2e/steps/_common.steps.ts:269 When /^the parent creates a kid with name "(.*)" and avatar "(.*)"$/ declares 2 capture(s), but the callback only binds 1 step argument(s), so 1 captured value(s) are dropped. Bind the missing parameter(s) and use them, or bind intentionally ignored values with a leading underscore and add a comment explaining why.
error: script "test:bdd-check" exited with code 1

So the line asks for a name and an avatar, the test's code only took the name, and now that gets caught before it's ever committed. Noice.

Lesson 2: State machines

The second lesson is something I've talked about a little bit before, in How I go FAST with AI and Your code is not readable, here's how to fix it, but it's worth talking about again, and that's the power of state machines.

One of the things I kept running into in the early development of Let's Build It was that the projects a kid was building were getting stuck, like forever in a spinning loop. A lot of the time it came down to how the agents were storing things in the Convex database. They would end up adding these optional fields and flags that could disagree with each other, so you'd end up in this half state between two different things happening.

Here's what the projects table looked like back in June:

// convex/projects/schema.ts in June, without its indexes
export const projectsTable = defineTable({
  kidId: v.id("kids"),
  title: v.string(),
  lifecycle: projectLifecycleValidator,
  currentVersionId: v.optional(v.id("projectVersions")),
  currentVersionStatusKind: v.optional(projectCurrentVersionStatusKindValidator),
  pendingVersionId: v.optional(v.id("projectVersions")),
  pendingVersionStatusKind: v.optional(projectCurrentVersionStatusKindValidator),
  nextVersionNumber: v.number(),
  icon: v.optional(projectIconValidator),
});

So a project had an optional version id, and then a separate optional flag for what state that version was in. There's nothing there to stop one being set without the other, or the two of them disagreeing.

And the solution, as it often is, is to use state machines, or in this case discriminated unions:

// convex/projects/schema.ts now, without its indexes
export const projectVersionSlotValidator = v.union(
  v.object({ kind: v.literal("none") }),
  v.object({
    kind: v.literal("generating"),
    versionId: v.id("projectVersions"),
  }),
  v.object({
    kind: v.literal("ready"),
    versionId: v.id("projectVersions"),
  }),
  v.object({
    kind: v.literal("error"),
    versionId: v.id("projectVersions"),
  }),
);

export const projectsTable = defineTable({
  kidId: v.id("kids"),
  title: v.string(),
  lifecycle: projectLifecycleValidator,
  currentVersion: projectVersionSlotValidator,
  pendingVersion: projectVersionSlotValidator,
  nextVersionNumber: v.number(),
  // ...
});

So now a game is either building, ready or failed (or there's nothing in that slot yet), and it can only ever be in one of those states at any one time. It can never be in a kind of failed but building state, for example.

Each build of a game is a version, and a version's own status is a union too, so the data sits with the state it belongs to. Only a ready version has the HTML file in file storage, and only an error one has a message:

// convex/projectVersions/schema.ts
export const projectVersionStatusValidator = v.union(
  v.object({ kind: v.literal("generating") }),
  v.object({
    kind: v.literal("ready"),
    htmlStorageId: v.id("_storage"),
  }),
  v.object({
    kind: v.literal("error"),
    message: v.string(),
  }),
);

And because it's right there in the schema, Convex won't let the agent save something in between, and it also won't let us push a schema change that would break the rows that are already there. Here's a little throwaway project with the same kind of union to show you. It has a row in the error state, and the schema being pushed has dropped error from the union:

$ npx convex dev
āœ– Schema validation failed.
Document with ID "j571c6x5233fhbqyh3fvxz2gnd8j3fcj" in table "projectVersions" does not match the schema: Value does not match validator.
Path: .status
Value: {kind: "error", message: "Build timed out"}
Validator: v.union(v.object({kind: v.literal("generating")}), v.object({html: v.string(), kind: v.literal("ready")}))

So Convex is going to yell at you before anything gets broken.

The other thing I really like about state machines is that you can get your agent to generate a diagram of your actual system, and where the data is inside it, because the diagram just is the code:

A state diagram an agent drew from the schema: create or edit goes to Generating, then to Ready when the HTML is saved or to Error when generation failsA state diagram an agent drew from the schema: create or edit goes to Generating, then to Ready when the HTML is saved or to Error when generation fails

Oh and I'm working on a state machine component for Convex at the moment, so you guys can leverage a lot of the power of state machines without suffering many of the pains I ran into along the way. Hopefully I'll have a lot more to share on that soon.

Lesson 3: Picking a model

Okay, the last thing I want to share is how I picked which AI model actually builds the games.

I hinted at this earlier, but getting the cost right was high up on my priority list. The app runs on credits, and I didn't want kids building lots of games that cost a lot of money and then running out of credits very quickly:

The card a kid sees when the credits run out: "Time for a recharge! Your building battery is empty. A grown-up can recharge it."The card a kid sees when the credits run out: "Time for a recharge! Your building battery is empty. A grown-up can recharge it."

But I also didn't want the quality of the generated projects to suffer either. So getting that balance between cost, quality and build time just right was very important to me.

The trouble is it feels like there's a new model coming out every week or so, and back in July I think I changed the model something like seven times in one week:

The git log for the model config in early July, with the seven commits that switched model highlightedThe git log for the model config in early July, with the seven commits that switched model highlighted

It was getting a bit ridiculous. So in the end I set up a big comparison, where I took a bunch of game ideas and pitted the different models against each other. And I did it in a blind way, so I had my agent kick out an HTML page where I could play the games side by side as A and B, without knowing which one was which:

A page asking "Which game is better?", with two builds of the same drawing app labelled A and B and buttons for A is better, About the same and B is betterA page asking "Which game is better?", with two builds of the same drawing app labelled A and B and buttons for A is better, About the same and B is better

What I ended up with, and what's running right now, is GPT-6 Sol. It costs about 16 cents per project at the moment, but I'm always keeping an eye on that as new models come out.

Conclusion

So just to recap: a kid talks to the assistant, a job on Convex runs their idea through a designer, a mockup, some design notes and a coder, and about a minute later there's a game sat in file storage ready to play. And the three lessons were to write down what your app should do in plain English and then test that, to keep your state in a union so it can only ever be one thing at a time, and to pick your model blind.

Let's Build It on the App Store, where it's free with in-app purchases, for iPhone and iPadLet's Build It on the App Store, where it's free with in-app purchases, for iPhone and iPad

I'm a builder at heart, and I just wanted Josh to have a little bit of that joy of building things himself. If you want to give it a go with your own kids, Let's Build It is free on the App Store and there's more about it over at letsbuildit.app.

Until next time, cheerio! šŸ‘‹

All gas, noĀ breakages

Convex is the reactive backend platform that keeps up with you and your agents. Database, functions, workflow, sync, search, file storage, and more. All TypeScript, zero glue.

Get started