System Design - Real-Time Chat (WhatsApp / Slack)

Designing a real-time messaging system (such as WhatsApp, Slack, Discord, or WeChat) is one of the most frequently asked questions in senior and staff-level system design interviews. It tests your ability to handle persistent bidirectional network connections, low-latency event fan-out, horizontal scaling of stateful servers, and high-volume append-only storage.

Requirements & Scope

Functional Requirements

One-on-One Chat: Low-latency (< 100ms) private messaging between two users.
Group Chat: Multi-user group conversations (supporting up to a few thousand members).
User Presence: Real-time online/offline status indicators.
Message Status: Sent, Delivered, and Read receipts.
Media Support: Sharing images, audio, and videos.
Offline Delivery: Queuing and delivering messages when disconnected users return online.

Non-Functional Requirements

Ultra-Low Latency: Real-time delivery with minimal perceptible delay.
High Availability & Fault Tolerance: 99.99% uptime with zero data loss for committed messages.
Data Durability & Message Ordering: Messages must be delivered and displayed in strict chronological sequence without duplicates.
Massive Scale: 500 million Daily Active Users (DAU), 50 billion messages exchanged per day.

Scale & Capacity Estimations

Average Message QPS: $$\text{Average QPS} = \frac{50,000,000,000 \text{ messages}}{86,400 \text{ seconds}} \approx 580,000 \text{ QPS}$$
Peak Message QPS (Factor of 2): $1.16 \text{ million QPS}$.
Daily Storage Ingress:
- Average text message: 100 bytes of text + 100 bytes of metadata = 200 bytes.
- Daily text storage: $50 \times 10^9 \times 200 \text{ bytes} \approx 10 \text{ TB / day}$.
- Over 5 years: $10 \text{ TB} \times 365 \times 5 \approx 18.25 \text{ PB}$ (excluding media attachments).
Concurrent Open Connections: 500M DAU with 10% active at any instant = 50 million concurrent persistent TCP/WebSocket connections.

Communication Protocol Selection

Why not standard HTTP request/response?

HTTP Polling: Client periodically polls the server (GET /messages). Causes enormous empty response overhead and unacceptably high latency.
HTTP Long Polling: Client holds the connection open until a message arrives. Better, but still incurs repeated HTTP header parsing and connection handshakes for high-frequency chat.
Server-Sent Events (SSE): Efficient server-to-client streaming, but unidirectional. The client must still send messages over separate HTTP POST requests.
WebSockets (The Standard Choice):
- Initiated via an HTTP Upgrade handshake, transitioning into a persistent, bidirectional, full-duplex TCP stream.
- Minimal framing overhead (only 2 to 10 bytes per frame).
- Perfect for high-frequency, bidirectional communication.

High-Level System Architecture

                                [ Client A ]
                                     │ (WebSocket)
                                     ▼
                           [ WebSocket Gateway ] ──► [ User Session Map ]
                           (Stateful Chat Pods)         (Redis Cluster)
                                     │
                     ┌───────────────┴───────────────┐
                     ▼                               ▼
            [ Message Broker ]              [ Presence Service ]
              (Kafka / Pulsar)                (Heartbeat / TTL)
                     │                               │
       ┌─────────────┴─────────────┐                 │
       ▼                           ▼                 ▼
[ Message Storage Engine ]  [ Notification Svc ] [ Redis Presence ]
(ScyllaDB / Cassandra)      (Apple APNs / FCM)
                                   │
                                   ▼
                              [ Client B ]
                           (Offline -> Push)

Core Architecture Components

WebSocket Gateway (Chat Servers): Stateful servers that maintain persistent WebSocket connections with active clients. They terminate TLS and route raw payloads.
Session / Connection Registry: A distributed in-memory store (Redis Cluster) mapping user_id to the specific chat_server_id holding the user's active WebSocket connection.
Presence Service: Manages user liveness. Clients send heartbeats every 30 seconds over the WebSocket. If no heartbeat arrives within 60 seconds, the user is marked offline.
Message Broker (Kafka / Redis Streams): Decouples chat servers, acts as an asynchronous buffer, and distributes incoming messages to the proper destination gateways.
Push Notification Service: Triggers external mobile pushes (APNs for iOS, FCM for Android) when messages are addressed to offline recipients.

Data Storage Architecture

A chat system requires two fundamentally different storage layers:

1. Relational Metadata Store (PostgreSQL / Aurora)

Stores user profiles, credentials, friend graphs, group membership rosters, and channel permission settings. These require strong consistency and relational lookups:

users (user_id, username, email, created_at)
group_members (group_id, user_id, role, joined_at)

2. Message History Store (Apache Cassandra / ScyllaDB)

Chat history is write-heavy (millions of inserts/sec), append-only, and queried sequentially by time ranges. Wide-column NoSQL stores built on LSM-Trees are ideal:

CREATE TABLE messages (
    chat_id bigint,
    message_id bigint,
    sender_id bigint,
    content text,
    media_url text,
    created_at timestamp,
    PRIMARY KEY ((chat_id), message_id)
) WITH CLUSTERING ORDER BY (message_id DESC);

Partition Key (chat_id): Colocates all messages belonging to a 1-on-1 conversation or group on the same storage node for fast range scans.
Clustering Key (message_id): Sorts messages chronologically within the partition. Using a distributed time-sortable 64-bit ID (like Twitter Snowflake) ensures zero-collision global ordering.

Step-by-Step Message Flows

1. One-on-One Message Flow

User A types a message and sends a WebSocket frame containing { recipient_id: B, content: "Hello", client_seq: 101 }.
Chat Server 1 (holding A's socket) receives the message, generates a Snowflake message_id, and acknowledges receipt back to User A (displaying a single checkmark: "Sent").
Chat Server 1 persists the message to the Message Store and publishes an event to the Message Broker.
The router queries the Session Registry to determine User B's whereabouts:
- Case 1: User B is Online. Session Registry reveals User B is connected to Chat Server 4. The message is routed to Chat Server 4, which pushes it down User B's active WebSocket. User B's device responds with an ack ("Delivered" double checkmark).
- Case 2: User B is Offline. No active connection exists. The message router forwards the event to the Push Notification Service, sending a notification via APNs/FCM. When User B opens the app later, the client pulls unread messages from the Message Store via cursor pagination.

2. Group Chat Message Flow (Fan-Out Strategies)

Group chats introduce significant fan-out amplification:

Fan-Out on Write (Push Model):
- For small groups (< 500 members): When User A sends a message, the server fetches the group member list, creates a delivery task for all members, and pushes the message into each member's personal inbox queue.
- Advantage: Fast read access; each user's client only monitors their own incoming stream.
Fan-Out on Read (Pull Model):
- For massive channels (Slack #general with 50,000 members, Discord servers): Duplicating messages 50,000 times on every write creates massive write amplification.
- Solution: The message is written only once to the group channel's timeline. Members pull new messages from the channel partition using their client sync cursor (last_synced_message_id).

Handling Tricky Edge Cases

1. Message Ordering & Concurrent Writes

What if two messages arrive at nearly identical milliseconds?

Relying on client wall-clock timestamps (Date.now()) leads to ordering errors due to device clock skew.
Relying on server timestamps can lead to collisions.
Solution: Use Snowflake IDs or a per-channel sequential counter (seq_id). The storage clustering key guarantees messages render in strict chronological sequence across all devices.

2. Duplicate Messages & Idempotency

Network disconnects cause mobile clients to retry sending messages:

The client generates a unique client-side client_msg_id (UUID) for each outbound message draft.
The chat server checks client_msg_id in Redis or database unique indexes. If the ID was already recorded within the last 5 minutes, the duplicate write is dropped and success is returned to the client.

3. Media Upload Pipeline

Never stream raw video/image binary files through the WebSocket Gateway!

Client requests an upload token: POST /api/media/upload-url.
API Gateway generates a pre-signed Amazon S3 / GCS URL with a 15-minute expiration.
Client uploads the media directly to S3 via HTTP PUT.
A background transcoding worker compresses the media and creates preview thumbnails.
Client sends the resulting media_url in the lightweight WebSocket message payload.