Caching

📋 Overview

Caching is the process of storing copies of data in a high-speed storage layer (typically RAM) so that future requests for that data can be served significantly faster than fetching it from the original source. It is the single most effective way to improve application performance, reduce user-perceived latency, and decrease the operational load on backend databases.

---

🏗️ Core Principles & Characteristics

• Cache-Aside (Lazy Loading): The application checks the cache first. On a "miss," it fetches data from the DB, stores it in the cache, and returns it. This is the most common pattern.
• Write-Through: Data is written to the database and the cache simultaneously. This ensures the cache is never stale but adds latency to write operations.
• Write-Back (Write-Behind): Data is written to the cache immediately, and the database update happens asynchronously in the background. High performance for writes, but risks data loss if the cache crashes.
• Read-Through: The application only communicates with the cache library. If data is missing, the cache library itself handles the DB fetch and updates its internal state.

---

⚖️ Trade-offs: Pros & Cons

• Pros:
  • Sub-millisecond Speed: Reading from RAM is orders of magnitude faster than Disk I/O.
  • Cost Efficiency: Offloading traffic from a managed DB (like RDS) to a cache (like ElastiCache) is often cheaper at scale.
  • Availability: In some patterns, the cache can serve stale data if the main database is temporarily offline.
• Cons:
  • Data Inconsistency: The "Stale Data" problem—the cache might contain old data after the DB has been updated.
  • Complexity: Managing cache invalidation, eviction policies, and "Thundering Herd" scenarios adds significant logic to the codebase.
  • Memory Cost: RAM is expensive; caching every piece of data is rarely cost-effective.

---

🌍 Real-World Implementation

• Content Delivery Networks (CDNs): Caching static assets (images, JS, CSS) at edge locations close to the user (e.g., Cloudflare, Akamai).
• Session Management: Storing user login sessions in Redis for fast validation on every request.
• API Response Caching: Buffering the results of expensive, slow API calls or complex SQL joins.
• Database Internal Caching: MySQL and Postgres have internal buffer pools to cache frequently accessed disk pages in memory.

---

💡 Interview "Gotchas" & Tips

• TTL (Time to Live): Never cache without a TTL! It is your "safety net" against permanent data staleness.
• The Invalidation Problem: "There are only two hard things in Computer Science: cache invalidation and naming things." Discuss using versioning or event-driven invalidation.
• Cache Penetration: Requests for data that doesn't exist (hitting the DB every time). Solution: Cache the null result or use a Bloom Filter.
• Thundering Herd: When a hot key expires and thousands of requests hit the DB at once. Solution: Use Lease Locking or Probabilistic Early Re-computation.

---

📐 Suggested Architecture Primitives

• Distributed Cache: Redis (supports complex data types) or Memcached (simple, multi-threaded).
• Edge Cache: AWS CloudFront or Varnish.
• Storage Tiering: RAM (Fastest) -> SSD (Fast) -> HDD/Object Store (Slower).
• Invalidation: Use CDC (Change Data Capture) from the DB to trigger cache invalidation via Kafka.