Theory
Load Balancing
📋 Overview
Load Balancing is the strategic distribution of incoming network traffic across a group of backend servers (the server pool). It serves as the "traffic cop" of the data center, ensuring high availability, reliability, and optimal performance by preventing any single server from becoming a bottleneck or a single point of failure.
🏗️ Core Principles & Characteristics
- Health Monitoring: Continuous "Liveness" and "Readiness" checks to ensure traffic only flows to operational nodes.
- OSI Layer Differentiation:
- Layer 4 (L4): Operates at the Transport Layer (TCP/UDP). Routes based on IP/Port. High performance due to lack of packet inspection.
- Layer 7 (L7): Operates at the Application Layer (HTTP/HTTPS). Routes based on URLs, Headers, or Cookies. Allows for "Content-Aware" routing.
- Persistence (Stickiness): Mechanisms (like session cookies or IP hashing) to ensure a client stays connected to the same server for the duration of a session.
⚖️ Trade-offs: Pros & Cons
- Pros:
- Fault Tolerance: Automatic failover when a server goes down.
- Elasticity: Seamlessly add/remove capacity during traffic spikes.
- SSL Offloading: Simplifies certificate management by handling encryption at the LB level.
- Cons:
- Latency: Every request must pass through the LB, adding a small overhead.
- Configuration Complexity: Sticky sessions and SSL passthrough vs. termination require careful tuning to avoid "unbalanced" load.
🌍 Real-World Implementation
- High-Performance Software: HAProxy is the gold standard for high-throughput L4/L7 balancing; NGINX is preferred for L7 balancing and static caching.
- Cloud Native: AWS ALB (Application) for microservices and NLB (Network) for millions of requests per second with ultra-low latency.
- Service Mesh: Envoy proxies act as "sidecar" load balancers in Kubernetes environments, handling service-to-service traffic.
💡 Interview "Gotchas" & Tips
- The "Thundering Herd" Problem: When a server comes back online, an LB might overwhelm it with traffic. Mention "Slow Start" or "Warm-up" periods as a solution.
- Stickiness Pitfalls: Explain how IP-based stickiness fails for users behind a shared NAT (e.g., a corporate office appearing as one IP). Recommend Cookie-based stickiness for L7.
- Dynamic vs Static: Know when to use Least Connections (long-lived sessions like WebSockets) vs Round Robin (short, uniform requests).
📐 Suggested Architecture Primitives
- Active-Passive LB Pair: Using VRRP or Keepalived to ensure the LB itself isn't a SPOF.
- Global Server Load Balancing (GSLB): Using DNS-based routing (e.g., AWS Route53) to distribute traffic across different geographical regions.
- Weighted Round Robin: Crucial for heterogeneous clusters where servers have different CPU/RAM specs.
Canvas