Kate Rudolph

10 months ago

Get Typeform Data Into Convex

When you’re starting a project, or adding functionality that requires collecting form data, Typeform can be a valuable tool. It has a beautiful, accessible, ergonomic design out of the box, and can enable you to get results quickly. As your project matures and you want to integrate that data with the rest of your app, you may be left wondering whether you need to rewrite the form yourself, or somehow integrate Typeform with your codebase. This post aims to help you with the latter, allowing you to keep leveraging Typeform, while keeping the data in a powerful transactional application database.

Here’s a sample app demonstrating how to integrate Typeform with Convex! You may want to remix different parts of it depending on what you’re trying to do with Typeform:

• A. Embed a Typeform form in your existing app.
• B. Import existing Typeform responses into Convex.
• C. Collect new Typeform responses into Convex in real-time.

With the approaches in this post, you can use Typeform as a seamless extension of your Convex app. For example:

• A user logs in to your Convex app.
• They fill out the form, which silently keeps track of their user_id in the data-tf-hidden attribute.
• Typeform receives the form response and immediately sends a webhook to your Convex app (in my testing, my Convex app heard about the response faster than Typeform’s own website!).
• Your Convex webhook handler saves the response data into the database based on the mapping you set up.
• Your reactive front end shows any UI updates as a result of the changed data, based on Convex’s built-in reactivity.

You could also build a UI that uses the user_id to show which of your Convex users was the one who filled out the Typeform, restrict users to only see their own responses, etc.

A. Embed Typeform in your app

Typeform offers a powerful and easy-to-edit form builder product — and they also make it really easy to embed that into your own website, such as one you built with Convex. Typeform itself has great docs on this here.

Here’s how it looks in my project:

Typeform form in a Convex-backed website

import Script from "next/script";
//...
	<button data-tf-slider="vX9noaqM" data-tf-position="right" data-tf-opacity="100" data-tf-iframe-props="title=Spotted a dog" data-tf-auto-close="500" data-tf-transitive-search-params="user_id" data-tf-medium="snippet" data-tf-hidden={`user_id=${userId}`} className="typeform-button">
	  Record a dog sighting
	</button>
	<Script src="//embed.typeform.com/next/embed.js"></Script>

This is almost what I copied from Typeform’s embed code, with a few notes:

• Typeform provided a bunch of styles for my button in a style property -- I just moved those to my existing CSS and customized them.
• I was using Next.js, and it was having issues loading the <script> element that Typeform suggested. I fixed this by using import Script from "next/script";.
• Finally, I added this attribute to my button, since my form has a user_id hidden field:

data-tf-hidden={`user_id=${userId}`}

This keeps track of which Convex user is logged in when they fill out the form. When a user submits the form with this data-tf-hidden attribute, the user_id is sent to Typeform along with the rest of the data the user entered into the form. The data then ends up in Convex one of two ways: either by importing into Convex as a migration sometime later (section B below) or via a Typeform webhook in near-real-time (section C below).

B. Import existing Typeform responses into Convex.

What if you’ve had a Typeform running for a while, and you have form responses that you’d like to incorporate into your Convex app? Here’s how to get all the existing form responses into a table in Convex.

Setup

First, grab these files from the example project:

• /scripts/setupTypeformMappings.js
• /scripts/importTypeform.js

If you’re using a schema in your project, you can copy over the schema for the typeform_metadata table:

typeform_metadata: defineTable({
    typeformFormId: v.string(),
    convexTableName: v.string(),
    typeformFieldId: v.string(),
    convexFieldName: v.string(),
  }).index("by_typeform_form_id", ["typeformFormId"]),

You’ll also need a Typeform API key (get one here) and run in your terminal:

export TYPEFORM_API_KEY=<your_api_key>

Lastly, you’ll need the form ID of the form whose responses you want in Convex! You can find it in the URL of your Typeform — it’s an 8-character string.

Map your form to database fields

Run

node scripts/setupTypeformMappings.js <your_form_id>

This will create a file typeformData/mappings_<your_form_id>.json — open this file up.

Review and update the convexTableName and convexFieldName values in the json file — by default it just reformats the question text, so you probably want to give each of the fields a short, descriptive name. If you want to skip importing the responses to a question, just delete it at this step.

Move the data

Once you have your mappings the way you want them, run:

node scripts/importTypeform.js <your_form_id>

This will write a few more files in typeformData and print a few npx convex import statements, which will look something like

npx convex import typeform_metadata ./typeformData/typeform_metadata.jsonl
npx convex import dogs ./typeformData/dogs.jsonl

(My example table is named dogs — you’ll see whatever you chose to name your Convex table!) Copy and paste each command to import the data.

Now when you check in your Convex dashboard, you’ll see your new table with data populated for all the form responses you already have in Typeform, and the fields will be named as specified in your mappings json file. You can start using this to build your features around your new form response data!

C. Collect new Typeform responses into Convex in real-time.

If you followed the steps in the previous section, you should have all your form responses so far… but what if more respondents fill out your form? This section explains how to set things up with a Typeform webhook so that as you get more Typeform responses, they automatically stay up to date in your Convex database.

Setup

For starters, you need to do most of the steps in item B to set up your Typeform mappings. Do all the setup and mapping steps there, up through

npx convex import typeform_metadata ./typeformData/typeform_metadata.jsonl

(you can skip the npx convex import for your new table if you don’t want those responses)

Create webhook

Grab the file scripts/setupWebhook.js and run it:

node scripts/setupWwebhook.js --form-id [your-typeform-id] --deployment-url https://[some-animal-123].convex.cloud

If you prefer, you can set it up manually following the steps here.

You can find the deployment url in .env (for prod) or .env.local (for dev) — this is set up for you when you set up your Convex project. It will look something like some-animal-123.

This script will print out the value of a TYPEFORM_SECRET_TOKEN — add this as an environment variable to your Convex deployment

Convex functions to handle the webhook

Finally, you need the code! Copy these files into your project from this repo:

• convex/saveTypeformResponse.ts
• convex/authenticateTypeformWebhook.ts
• convex/http.ts (if you already have a http.ts just combine the contents into your existing file) See here for more info on serving HTTP requests from Convex.

Run npx convex dev to get your Convex functions ready!

Test it out

You can test your webhook setup by submitting a response to your form! But if your form is too long and you want to test faster than that, there’s also a way to test your webhook, from the same page you set it up. Make sure the webhook is turned on on the Typeform side, then click “View Deliveries”, then “Send Test Request”

Putting it all together

In this post, we walked through the steps for adapting this example Typeform integration to integrate your own Typeform into your own Convex project. Whether you want to embed a form on the front end, migrate response data to your back end, or hook things up to stay in sync going forward, hopefully this helps accelerate your project. Again, the code is here. Let us know how you use it!