Skip to content
Tutorials Center
My learning

Topics

Lesson 49 of 65 ยท Case Studies: Classic Designs

Design a Chat System

Design a messaging app: WebSocket connections, message routing, ordering with sequence numbers, offline delivery, group chat fan-out, presence and storage, with Python.

A chat system such as WhatsApp or Messenger moves short messages between people in real time. It sounds like "just send a message", but it hides hard problems: millions of long-lived connections, messages that must arrive once and in order, users who are offline, and groups with thousands of members.

Requirements

  • One-to-one and group chat (up to a few hundred members).
  • Real-time delivery when the recipient is online; storage and delivery later when offline.
  • Delivery receipts (sent, delivered, read) and presence (online, last seen).
  • Message history synced across a user's devices.
  • Scale: 500 million daily users, 40 messages per user per day.

Estimate: 500M x 40 = 20 billion messages per day, about 230,000 per second on average. At 100 bytes each that is 2 TB per day of new data, so storage needs partitioning and a plan for old data.

Connections: why WebSockets

Plain HTTP request and response makes the client poll ("any new message?"), which wastes bandwidth and adds delay. A WebSocket is a long-lived two-way connection, so the server can push a message the moment it arrives. Each connection is held by a chat server; one server may hold several hundred thousand idle connections.

flowchart LR
    A["Alice's phone"] <--> CS1["Chat server 1"]
    B["Bob's phone"] <--> CS2["Chat server 2"]
    CS1 --> MQ["Message routing / queue"]
    MQ --> CS2
    CS1 --> DB[("Message store")]
    CS1 --> PS["Presence service"]
    MQ --> PN["Push notification service"]

The sender's chat server stores the message, then routes it to whichever server holds the recipient's connection.

A session registry (a fast key-value store) records which chat server each online user is connected to, so routing is a lookup: user -> server.

The message flow

  1. Alice sends a message with a client-generated ID to her chat server.
  2. The server assigns a per-conversation sequence number and stores the message.
  3. It acknowledges to Alice ("sent").
  4. It looks up Bob in the registry. If online, it forwards to Bob's server, which pushes it over his socket. Bob's app acknowledges ("delivered").
  5. If Bob is offline, the message waits in his inbox and a push notification wakes his phone. On reconnect Bob asks for everything after his last seen sequence number.
Python Routing, ordering and offline delivery

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Two ideas carry the weight here. A client-generated message ID makes retries safe (idempotency): a flaky network can resend without creating duplicates. A per-conversation sequence number gives a total order inside each chat, and a reconnecting client simply asks for "everything after N".

Group chat

For small groups, the server fans out on write: it stores the message once and pushes a copy to each member's connection or inbox. For very large groups or channels, storing one copy and letting members pull ("fan out on read") saves work. Limiting group size keeps the common case simple.

Python Fan-out cost for groups

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Presence

Presence is "online" or "last seen". Each client sends a small heartbeat every few seconds. The presence service stores the timestamp and treats a user as offline after a few missed heartbeats. Do not push every change to everyone: only notify a user's currently open chats, or let the client fetch status when a chat screen opens, otherwise one popular user flapping online and offline creates millions of events.

Storage

Chat history is append-heavy and read by recent range, which fits a wide-column or key-value store (HBase, Cassandra-style) keyed by (conversation_id, sequence). Keep the latest messages hot, archive old ones, and search through a separate index. For end-to-end encryption (as in Signal-style protocols) the servers only see ciphertext, so server-side search and moderation have to move to the client.

Other considerations: media files go to object storage with a CDN and only a link is sent in the message; multiple devices each keep their own cursor; a heartbeat-based cleanup removes stale registry entries when a server crashes.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why use WebSockets instead of polling over HTTP?

    A persistent two-way connection removes the delay and waste of repeated polling.
  2. 2. What is the purpose of a client-generated message ID?

    If the same ID arrives twice, the server recognizes it and stores the message once.
  3. 3. A user reconnects after a day offline. How do they catch up?

    Sequence numbers per conversation make 'give me everything after N' cheap and exact.
  4. 4. How does a server find which chat server holds Bob's connection?

    A fast key-value registry maps each online user to a chat server.

Summary

  • Use WebSockets for pushes; each chat server holds many idle connections.
  • Route with a session registry; store messages with sequence numbers per conversation.
  • Make sends idempotent with client IDs; sync offline users by cursor and wake them with push notifications.
  • Fan out on write for small groups, consider pull for huge ones.
  • Presence uses heartbeats and should be delivered sparingly.

Next: Design a News Feed