Theory
Deployment Strategies: Blue/Green & Canary
📋 Overview
Deployment strategies are patterns used to release new software versions while minimizing risk, reducing downtime, and ensuring the ability to roll back quickly if issues arise. The two most prominent industry standards are Blue/Green (switching between two identical environments) and Canary (gradual rollout to a subset of users).
🏗️ Core Principles & Characteristics
- Blue/Green Deployment:
- Maintains two identical production environments.
- "Blue" is live; "Green" receives the new deployment.
- Traffic is swapped at the Load Balancer level once Green is verified.
- Canary Deployment:
- Rolls out the new version to a small percentage of users (the "canaries").
- Monitors health metrics (error rates, latency).
- Increases traffic incrementally (5% → 25% → 100%) if metrics remain stable.
- Automated Rollbacks: Monitoring tools trigger an immediate revert if performance degrades beyond a threshold.
⚖️ Trade-offs: Pros & Cons
- Blue/Green:
- Pros: Instant rollback, simple traffic routing, reduced downtime.
- Cons: Very expensive (requires double infrastructure), complexity in DB synchronization.
- Canary:
- Pros: Lowest risk, real-world testing with live traffic, cost-efficient (same infra).
- Cons: Complex routing logic, slower rollout process, requires advanced monitoring.
🌍 Real-World Implementation
- AWS CodeDeploy: Supports both Blue/Green and Canary deployments natively.
- Kubernetes (K8s): Uses RollingUpdates by default, but Service Meshes like Istio or Linkerd enable advanced Canary traffic splitting.
- Netflix: Uses Spinnaker for automated, metric-driven Canary analysis (Kayenta).
💡 Interview "Gotchas" & Tips
- Database Migrations: What happens if the Blue/Green switch fails but the DB schema has already changed? (Answer: Use backward-compatible changes or Expand/Contract pattern).
- Session Stickiness: How do you handle users in the middle of a session when the traffic switches?
- Metric Selection: For Canaries, which metrics matter most? (Answer: Error rates and latency, not just CPU/Memory).
- Feature Flags: Often used alongside Canary deployments to decouple "Deployment" from "Release."
📐 Suggested Architecture Primitives
- Load Balancers / Ingress Controllers: To handle traffic shifting.
- Service Mesh (Istio): For fine-grained traffic splitting and observability.
- Observability Stack (Prometheus/Grafana): To monitor deployment health in real-time.
Canvas