Who's on Call? Learn to Sync Pagerduty with Slack
If you use PagerDuty and Slack you could probably benefit from automatically changing the topic for an #oncall Slack channel to display who’s currently on call. You can get this running in a manner of minutes using this repo.
All you need to do is clone the repository, deploy the application to Convex, then enter your API keys and configuration entries.
git clone https://github.com/get-convex/pagerduty-slack-sync.git
cd pagerduty-slack-sync
npm install
npx convex deploy
# Set up the PagerDuty and Slack API keys on the dashboard. See README.md for details.
npx convex dashboard
And that’s it!
Read on to see how it’s built.
Sync PagerDuty to Slack in minutes with Convex
At Convex, we use PagerDuty to handle our alerting needs so engineers can keep Convex running smoothly. PagerDuty has a powerful interface for creating multiple schedules letting engineers add temporary overrides, but this means that in the heat of an incident, it can be hard to quickly figure out who is actually on-call.
PagerDuty’s scheduling interface
Ideally, we wanted to see the current primary and secondary on-call engineer at a glance in Slack — where most incidents are discussed and resolved. While PagerDuty doesn’t have a built-in integration for this, we found this repository from the folks at PagerDuty which sets up a function in AWS Lambda to update your Slack channel topic with the current on-call status. It looked perfect for our needs — we followed the instructions to set up the CloudFormation deployment in AWS, set up the API Keys in AWS Systems Manager Parameter Store, stored the configuration in AWS DynamoDB, and a few hours later had PagerDuty updates streaming into our Slack channel.
Architecture diagram from the PagerDuty repository
Wait. A few hours? Surely we can do better than that!
The task at hand sounds simple enough: on a schedule, check who is on-call via the PagerDuty API. Send a request via the Slack API to update the channel topic with the latest state. This is a perfect fit for Cron Job Scheduled Functions on Convex!
We can re-use the ideas from the original PagerDuty repository and write simple Convex functions to accomplish our task.
Let’s start with a function to fetch the current on-call user for a given PagerDuty schedule:
getOncallUser
// convex/sync.ts
async function getOncallUser(schedule: string): Promise<string> {
const token = process.env.PAGERDUTY_API_KEY;
if (token === undefined || token == "") {
throw "PagerDuty API Key not set";
}
const headers = {
Accept: "application/vnd.pagerduty+json;version=2",
Authorization: `Token token=${token}`,
};
const normal_url = `https://api.pagerduty.com/schedules/${schedule}/users`;
const override_url = `https://api.pagerduty.com/schedules/${schedule}/overrides`;
const now = new Date();
let since = new Date(now);
// The PagerDuty API gets all users on-call for a schedule within a time bound. The easiest way to get the current on-call is to set a tight bound.
since.setSeconds(since.getSeconds() - 5);
let payload: Record<string, string> = {};
payload["since"] = since.toISOString();
payload["until"] = now.toISOString();
let parameters = new URLSearchParams(payload);
let query_string = `?${parameters.toString()}`;
let normal_schedule = await fetch(normal_url + query_string, {
headers: headers,
});
if (normal_schedule.status == 404) {
throw `Invalid schedule ${schedule}`;
}
const response = await normal_schedule.json();
const users = response["users"];
const user = users[0];
if (!user) {
return "No one :panic:";
}
let username = user["name"];
if (!username) {
return "Deactivated user :panic:";
}
let override_schedule = await fetch(override_url + query_string, {
headers: headers,
});
if ((await override_schedule.json())["overrides"].length > 0) {
username += " (Override)";
}
return username;
}
We’ve chosen to store the PagerDuty API key in an environment variable so it doesn’t need to be committed to your repository.
Similarly, we’ll need to write a function that can set the topic of a given Slack channel. For extra style points, we’ll first get the current topic and preserve its contents when setting a new on-call user.
getSlackTopic and updateSlackTopic
// convex/sync.ts
async function getSlackTopic(channel: string): Promise<string> {
const token = process.env.SLACK_API_KEY;
if (token === undefined || token == "") {
throw "Slack API Key not set";
}
const payload = {
token: token,
channel: channel,
};
let response = await fetch("https://slack.com/api/conversations.info", {
method: "POST",
body: new URLSearchParams(payload).toString(),
headers: {
"Content-Type": "application/x-www-form-urlencoded; charset=utf-8",
},
}).then((r) => r.json());
try {
const topic = response["channel"]["topic"]["value"];
return topic;
} catch (e) {
console.error(e);
console.info(response);
throw `Could not find channel ${channel} on Slack; ensure the bot is in this channel`;
}
}
const updateSlackTopic = async function (
channel: string,
proposed_update: string
): Promise<void> {
const token = process.env.SLACK_API_KEY;
if (token === undefined || token == "") {
throw "Slack API Key not set";
}
let payload: Record<string, string> = {
token: token,
channel: channel,
};
let current_topic = await getSlackTopic(channel);
if (current_topic == "") {
current_topic = ".";
}
const pipe = current_topic.indexOf("|");
if (pipe !== undefined) {
let oncall_message = current_topic.substring(0, pipe).trimEnd();
current_topic = current_topic.substring(pipe + 1).trimStart();
if (oncall_message == proposed_update) {
console.log("No topic update required");
return;
}
}
proposed_update = proposed_update + " | " + current_topic;
payload["topic"] = proposed_update;
if (process.env.DRY_RUN !== undefined) {
console.log("Would update topic with request", payload);
console.log("Exiting without sending request");
return;
}
const response = await fetch("https://slack.com/api/conversations.setTopic", {
method: "POST",
headers: {
"Content-Type": "application/x-www-form-urlencoded; charset=utf-8",
},
body: new URLSearchParams(payload).toString(),
});
if (response.status != 200) {
const error = await response.text();
throw `Failed to update topic: ${error}`;
}
};
Note that we also added a dry-run mode for testing without actually updating the topic.
Next, we need a way to store our configuration such that it’s easy to add and remove Slack channels and PagerDuty schedules. Turns out Convex is pretty good at storing documents!
Let’s write a simple schema for our configuration documents:
// convex/schema.ts
import { defineSchema, defineTable } from "convex/server";
import { v } from "convex/values";
export default defineSchema({
// Each `config` defines a list of PagerDuty schedules (with names)
// to fetch, and a Slack channel ID to update.
configs: defineTable({
channel: v.string(),
schedules: v.array(v.object({ schedule: v.string(), name: v.string() })),
}),
});
Lastly, we can wrap this all up in an Action that iterates over the configs and performs the updates, then set up a cron job to call our function every minute.
Action and Cron
// convex/sync.ts
export const getConfig = internalQuery({
handler: async (ctx) => {
return await ctx.db.query("configs").collect();
},
});
async function performUpdate(config: Doc<"configs">) {
console.log("Updating for config:", config);
let topic = "";
for (const schedule of config.schedules) {
const oncall = await getOncallUser(schedule.schedule);
if (topic != "") {
topic += ", ";
}
topic += `${schedule.name}: ${oncall}`;
}
await updateSlackTopic(config.channel, topic);
}
export default action({
handler: async (ctx) => {
const configs = await ctx.runQuery(internal.sync.getConfig);
await Promise.all(configs.map(performUpdate));
},
});
// convex/crons.ts
import { cronJobs } from "convex/server";
import { api } from "./_generated/api";
const crons = cronJobs();
crons.interval("Update Slack topic", {minutes: 1}, api.sync.default);
export default crons;
You can find the completed code for this integration here.
Sync PagerDuty to Slack in minutes with Convex
And that’s it — deploying it is as simple as cloning this repository, setting up your Convex project, and configuring your API keys. You can add or remove configurations from the Convex dashboard and you’ll see changes immediately reflect in the linked Slack channel. No advanced degrees in AWS services required!
Convex is the sync platform with everything you need to build your full-stack project. Cloud functions, a database, file storage, scheduling, search, and realtime updates fit together seamlessly.