Hamza Saleem

7 hours ago

Keeping Users in Sync: Building Real-time Collaboration with Convex

Keeping Users in Sync: Building Real-time Collaboration with Convex is a guest Stack Post from Convex Champion Hamza Saleem.

Sticky my startup, is inspired by the idea of making collaboration as natural as doing it in person. Building features for Stick was both challenging and exciting experience. As I worked through it, I gained alot of insights on how to keep users in sync, handle data efficiently and to make sure everything works smoothly.

Why Real Time Collaboration?

Realtime collaboration is important for modern workflow. Either for brainstorming some ideas, editing documents or debugging your code. Immediate feedback keeps the momentum alive. It lets you bounce ideas of off each other just like you will do in person.

Apps like Google Docs or Figma have done a great job making this possible. I tried to bring the same effortless feel to brainstorming ideas using sticky notes,

Challenges making Real Time Collaboration work:

Its not just about syncing data but also about creating a seamless and engaging experience. Here's how i approached some of the problems using Convex:

Sync and Optimistic Updates Syncing data in real time is a common challenge in a collaborative app. In Sticky, each user's change needs to be reflected across all the active sessions instantly the traditional approach would require creating a complex system using Pollling and WebSockets. Which requires ton of custom logic and backend setup.

How Convex Solves This: Convex simplifies this with a native real-time data sync mechanism. By using the useQuery hook, you can easily fetch and sync data, like sticky notes, with minimal setup. Convex handles the underlying complexities of WebSocket connections, keeping all clients in sync automatically. This lets developers focus on creating features instead of managing the socket connections to keep everything connected in real time

Optimistic Updates: When collaborating in real time, you'd expect changes to appear instantly, however network latency can cause delays in reflecting these updates making the user experience sluggish. to address this properly. I used optimistic updates to instantly reflect changes in the UI even before server save changes in the backend.

What Optimistic Updates Help Solve: Optimistic updates solve the problem of UI delay caused by network latency. When user creates a new note the change is immediately shown in the Board even though the mutation maybe still be pending. The change is rolled back if mutation fails ensuring data consistency without effecting the user experience.

1const createNote = useMutation(api.notes.createNote).withOptimisticUpdate(
2  (localStore, args) => {
3    const existingNotes =
4      localStore.getQuery(api.notes.getNotes, { boardId: actualBoardId }) || [];
5    const tempId = `temp_${Date.now()}` as Id<"notes">;
6    const now = Date.now();
7    localStore.setQuery(api.notes.getNotes, { boardId: actualBoardId }, [
8      ...existingNotes,
9      { _id: tempId, _creationTime: now, ...args },
10    ]);
11  }
12);
13const createNote = useMutation(api.notes.createNote).withOptimisticUpdate(
14  (localStore, args) => {
15    const existingNotes =
16      localStore.getQuery(api.notes.getNotes, { boardId: actualBoardId }) || [];
17    const tempId = `temp_${Date.now()}` as Id<"notes">;
18    const now = Date.now();
19    localStore.setQuery(api.notes.getNotes, { boardId: actualBoardId }, [
20      ...existingNotes,
21      { _id: tempId, _creationTime: now, ...args },
22    ]);
23  }
24);
25
26

By immediately showing the new note on the board, users can continue collaborating without waiting for the server's response. Only if the mutation completes with a different result will the UI be updated accordingly.

Real-time Presence Tracking Tracking who is online and where they are on the board is essential for real-time collaboration. In Sticky, I needed to show users cursor positions, indicating where others are working on the board. Real-time presence tracking also ensures that the team knows who is active and engaged.

What I Learned: Implementing real-time presence required handling frequent updates of user cursor positions, which can overwhelm the system if not managed efficiently.

How Convex Solves This: I used debounced updates to efficiently manage the frequency of presence updates. This function ensures the database isn't overloaded with too many requests by limiting the frequent updates.

Here’s a quick explanation of debouncing: debouncing is a technique where the function only triggers after a specified delay, avoiding redundant or unnecessary calls. In this case, it helps prevent sending excessive updates for small, rapid movements of the cursor.

1export function usePresence(boardId: Id<"boards">, isShared: boolean) {
2  const updatePresence = useMutation(api.presence.updatePresence);
3  const removePresence = useMutation(api.presence.removePresence);
4  const activeUsers = useQuery(api.presence.getActiveUsers, { boardId });
5  const cursorPositionRef = useRef({ x: 0, y: 0 });
6  const [localCursorPosition, setLocalCursorPosition] = useState({ x: 0, y: 0 });
7
8  const debouncedUpdatePresence = useCallback(
9    debounce((position: { x: number; y: number }) => {
10      if (isShared) {
11        updatePresence({
12          boardId,
13          cursorPosition: position,
14          isHeartbeat: false
15        });
16      }
17    }, PRESENCE_UPDATE_INTERVAL, { maxWait: PRESENCE_UPDATE_INTERVAL * 2 }),
18    [boardId, updatePresence, isShared]
19  );
20
21  useEffect(() => {
22    if (!isShared) return;
23
24    const heartbeatInterval = setInterval(() => {
25      updatePresence({
26        boardId,
27        cursorPosition: cursorPositionRef.current,
28        isHeartbeat: true
29      });
30    }, HEARTBEAT_INTERVAL);
31  
32    return () => {
33      clearInterval(heartbeatInterval);
34      removePresence({ boardId });
35    };
36  }, [boardId, updatePresence, removePresence, isShared]);
37
38  return {
39    activeUsers: isShared ? activeUsers : [],
40    updateCursorPosition,
41    localCursorPosition
42  };
43}
44

By debouncing the updates, the system can efficiently tracks user activity without overwhelming the backend or causing performance issues.

Schema Design and Indexing As real-time data grows, efficient database design becomes crucial. I had to ensure that data could be retrieved quickly as the board and user base expanded.

How Convex Helps: Convex's schema design allows for easy definition of tables and indexes.

For example, the presence table schema enables efficient queries based on user and board, as well as tracking the last update time for each user.

1presence: defineTable({
2  userId: v.id("users"),
3  boardId: v.id("boards"),
4  lastUpdated: v.number(),
5  cursorPosition: v.object({
6    x: v.number(),
7    y: v.number(),
8  }),
9})
10  .index("by_board", ["boardId"])
11  .index("by_user_and_board", ["userId", "boardId"])
12  .index("by_board_and_lastUpdated", ["boardId", "lastUpdated"]);
13
14

These indexes ensure that as the database grows, data retrieval remains fast, even with a large number of active users or boards.

4. End-to-End Type Safety A major advantage of using Convex is its seamless integration with TypeScript. The type safety that TypeScript offers across the stack is a huge benefit when building real-time systems. With type-safe queries, mutations, and schema definitions, I was able to catch potential bugs during development, making the entire process smoother and more predictable.

Lesson Learned

I learned was how important is is for state management to be simple. Real time systems are naturally complex but with right tools it can be make a huge difference. With Convex, I can focus on creating a great user experience instead of worrying about the technicals.

A Final Thought

Building Sticky was a rewarding experience. The process taught me a lot about real-time systems and the value of tools that simplify complex tasks. For anyone looking to implement similar features, my advice is to start small, leverage existing tools, and iterate as you learn.

Check out Sticky to see these features in action or explore the source code for more implementation details.

hamzasaleem2/sticky

About Organize thoughts, collaborate in real-time, and access your ideas from anywhere.