News Feed System Design

📋 Overview

Designing a News Feed (Facebook, Twitter, Instagram) is one of the most common system design interview questions. It requires balancing a massive write volume (users posting content) with an even larger read volume (users scrolling through their feed). The core challenge is minimizing the latency of the "Read" operation while ensuring that new updates appear near-instantly across the social graph.

---

🏗️ Core Principles & Characteristics

• Fanout Strategy: The method of delivering a post to all followers.
  • Push (Fanout-on-Write): When a user posts, we proactively write that post into the pre-computed "Feed Cache" of every follower. (Fast reads, slow writes).
  • Pull (Fanout-on-Read): The feed is generated only when a user opens the app by querying all followees. (Fast writes, slow reads).
• Hybrid Approach: Use Push for regular users and Pull for "Celebrities" (users with millions of followers) to avoid the "Celebrity Problem" (where one post triggers millions of database writes).
• Feed Ranking: Modern feeds aren't just chronological. They use ML models to rank content based on engagement, relevance, and "freshness."

---

⚖️ Trade-offs: Pros & Cons

• Push Model:
  • Pros: User gets their feed in O(1) from Redis.
  • Cons: A user with 10M followers causes 10M writes to the cache. If they post 10 times a day, that's 100M writes for one user.
• Pull Model:
  • Pros: Posting is just a single DB insert.
  • Cons: Generating a feed for a user following 2,000 people requires fetching and merging data from 2,000 sources at runtime.

---

🌍 Real-World Implementation

• Twitter: Famously uses a hybrid approach. They maintain a "Timeline Cache" (Redis) for every active user. Celebrities are "pulled" into timelines at query time.
• Facebook: Uses a sophisticated graph database called Tao to store the social graph and an edge-computing layer to assemble feeds.
• Instagram: Leverages "Content Delivery Networks" (CDNs) heavily for the media-heavy feed, using pre-signed URLs to ensure security.

---

💡 Interview "Gotchas" & Tips

• The "Celebrity Problem": Always bring this up. It shows you understand scaling limits. Suggest the hybrid model as the solution.
• Pagination: Don't just say "I'll fetch the feed." Mention Cursor-based Pagination (using last_id) rather than OFFSET/LIMIT to avoid missing or duplicating posts as the feed updates.
• Active vs. Inactive Users: Don't pre-compute feeds for users who haven't logged in for 30 days. This saves massive amounts of cache memory.

---

📐 Suggested Architecture Primitives

• Redis Clusters: To store the pre-computed "Feed/Timeline" for every active user.
• Message Queues (Kafka): To handle the Fanout process asynchronously so the "Post" API returns immediately.
• Social Graph Service: A dedicated service (often backed by Neo4j or optimized SQL) to manage "Follow" relationships.
• Notification Service: To trigger Push/Email alerts when a relevant post enters the fanout pipeline.