Theory
Latency: Definitions and Impact
📋 Overview
Latency is the time interval between a stimulus and its response. In system design, it is the most critical factor for user experience—studies show that a 100ms increase in latency can decrease conversion rates by 7%. It is a cumulative value made up of network travel time, server processing time, and database query execution.
🏗️ Core Principles & Characteristics
- Network Latency: The time spent "on the wire." Limited by the speed of light and the number of router "hops."
- Disk Latency: The time to read/write to storage (HDDs have high seek latency; NVMe has very low).
- Execution Latency: The time the CPU spends running code.
- Queuing Latency: The time a request sits in a buffer waiting for a resource (CPU, DB Connection, etc.) to become available.
⚖️ Trade-offs: Pros & Cons
- Lowering Latency:
- Pros: Snappy UI, higher user retention, better SEO rankings.
- Cons: Expensive (requires CDNs, high-performance hardware, and multi-region deployment).
- Latency vs. Correctness:
- Strong Consistency: High latency (wait for all replicas to agree).
- Eventual Consistency: Low latency (return result immediately).
🌍 Real-World Implementation
- Edge Computing: Running logic (Lambda@Edge) at the CDN level to eliminate the trip back to the origin server.
- Database Indexing: Reducing "Query Latency" by avoiding full table scans.
- Caching (Redis): Replacing 10ms disk latency with 0.1ms RAM latency.
💡 Interview "Gotchas" & Tips
- The Speed of Light: It takes ~67ms for light to travel halfway around the earth. This is the "theoretical floor" for global latency that no software can fix.
- Query Latency vs. Execution Time: Latency includes the time the request spent in the network; Execution Time is only the time the DB engine spent working.
- P99 vs. Average: Never quote "Average Latency." Always talk about P99 (99th percentile) to show you understand that outliers are what destroy user trust.
📐 Suggested Architecture Primitives
- CDN / Edge Cache: For reducing network distance.
- In-Memory Store (Redis): For reducing data access latency.
- Read Replicas: For reducing database contention.
- Protocols (UDP/HTTP3): For reducing transport-layer handshaking.
Canvas