Theory
Chat System Architecture
📋 Overview
A real-time chat system enables instant communication between users across various platforms (mobile, web). Designing a chat system is a classic high-scale problem that requires managing persistent connections, message ordering, delivery guarantees (push notifications), and presence tracking (online/offline status). The architecture must transition from traditional HTTP to persistent protocols like WebSockets to achieve sub-second latency.
🏗️ Core Principles & Characteristics
- Stateful Connections: Uses WebSockets (or MQTT/SSE) to maintain a persistent bi-directional pipe between the client and the server.
- Message Sequencing: Ensuring messages appear in the correct order using monotonically increasing IDs (Snowflake IDs) or sequence numbers per conversation.
- Delivery Guarantees: Implementing "At-least-once" delivery with acknowledgments (ACKs) from the client.
- Decoupled Services: Separating "Chat Service" (message logic) from "Presence Service" (status) and "Notification Service" (offline delivery).
- Pub/Sub Mechanism: Using a message broker (Redis Pub/Sub or Kafka) to route messages to the specific server holding the recipient's connection.
⚖️ Trade-offs: Pros & Cons
- Pros:
- Real-time Experience: Minimal latency for message delivery and typing indicators.
- Efficiency: WebSockets reduce the overhead of repeated HTTP headers.
- Rich Interaction: Supports read receipts, reactions, and multi-device sync.
- Cons:
- Connection Management: Keeping millions of open TCP connections requires specialized load balancing and "Sticky Sessions."
- Scalability Complexity: Sharding connections across multiple servers requires a complex "routing" layer to find which server a user is connected to.
- Battery Drain: Persistent connections on mobile devices can be power-hungry (solved by using Push Notifications when the app is backgrounded).
🌍 Real-World Implementation
- WhatsApp/Discord: Use customized protocols (like Erlang/XMPP or WebSockets) to handle billions of concurrent connections.
- Slack: Utilizes a highly available edge network to reduce the latency of the initial WebSocket handshake.
- Storage Strategy:
- Relational (PostgreSQL): For user profiles and settings.
- NoSQL (Cassandra/HBase): For massive message history due to high write-throughput and easy horizontal scaling.
- Redis: For transient data like "Who is online" and "Who is typing."
💡 Interview "Gotchas" & Tips
- The "Fan-out" Problem: In a group chat with 10,000 members, sending one message requires 10,000 writes/pushes. Solution: Limit group size or use a lazy-loading approach for history.
- Presence Heartbeats: Users don't always "log out" (they just lose signal). Use a Heartbeat mechanism—if the server hasn't heard from a client in 30 seconds, mark them offline.
- Mobile Push (FCM/APNs): If a user is offline, don't keep trying WebSockets; immediately hand the message to the Push Notification Service.
- Message ID Generation: Don't use DB auto-increment; it doesn't scale across shards. Use a distributed ID generator like Snowflake.
📐 Suggested Architecture Primitives
- WebSocket Gateway: Manages the open connections and handles the handshake.
- Message Broker: (Redis/Kafka) to broadcast messages between WebSocket servers.
- Presence Store: (Redis) A key-value store mapping
user_id -> {status, server_id, last_active}. - Push Service: (FCM for Android, APNs for iOS) for background delivery.
- Chat DB: (Cassandra) Optimized for high-volume time-series writes.
Canvas