SYS ARCHITECTLearning Platform

Phase 1

  • Interview Framework
  • Back-of-the-Envelope Estimation
  • ACID
  • CAP
  • Consistency vs Integrity
  • Idempotency
  • NFR
  • SPOF
  • Stateless vs Stateful
  • SOLID
  • SRP
  • OCP
  • LSP
  • DIP vs LSP
  • Clear Architecture Principles
  • 12-Factor App

    • Phase 2

  • SQL vs NoSQL
  • Normalization vs Denormalization
  • Partition vs Sharding
  • Sharding
  • Partition Tolerance
  • Serializable Transactions
  • MVCC
  • Optimistic vs Pessimistic Locking
  • Row Level Locking
  • Distributed Locks
  • Redis Lock
  • Replication
  • Primary Replica Setup
  • Eventual Consistency
  • Data Consistency
  • Quorum
  • Write Ahead Logs
  • Redo Logs
  • DB Durability
  • Data Recovery
  • Two Phase Commit
  • Saga Pattern
  • Cassandra
  • S3
  • HDFS
  • Timeseries DB

    • Phase 3

  • Caching
  • Cache Eviction Policies
  • Hot Keys
  • Hot Reads and Writes
  • Latency
  • Latency vs Throughput
  • Tail Latency
  • Low Latency Patterns
  • Backpressure
  • Sudden Traffic Spike
  • Buffer Burst

    • Phase 4

  • Replication
  • Consistent Hashing
  • Leader Election
  • Distributed Locking
  • Fault Tolerance
  • Resiliency
  • Raft Consensus
  • Paxos
  • Event Driven Architecture
  • Event Sourcing
  • Outbox Pattern
  • CQRS

    • Phase 5

  • How HTTP Works
  • HTTP vs HTTPS
  • TCP vs UDP
  • TLS
  • Long Polling vs WebSockets
  • Server Sent Events
  • Load Balancer
  • L4 vs L7
  • Load Balancer vs Reverse Proxy vs API Gateway
  • CDN
  • Geo DNS

    • Phase 6

  • JWT
  • OAuth2
  • Opaque vs JWT Token
  • TLS
  • OWASP
  • Secrets Management

    • Phase 7

  • Creating Docker Image
  • Kubernetes ConfigMap
  • Kubernetes Ingress
  • Persistent Volume
  • Prometheus and Grafana

    • Phase 8

  • Monolith vs Microservices
  • API Gateway
  • Unique ID Generator
  • Retry Mechanism
  • Deployment Strategy
  • Rate Limiting

    • Phase 9

  • URL Shortener
  • Chat System
  • News Feed
  • Web Crawler
  • Notifications Design
  • System Design Problems
    • Settings
    Theory

    Prometheus & Grafana: Monitoring & Visualization

    📋 Overview

    In a distributed microservices environment, visibility into system health is paramount. Prometheus and Grafana form the industry-standard "Observability Stack." Prometheus acts as the time-series database and metrics aggregator, while Grafana provides the visualization layer to transform raw metrics into actionable, real-time dashboards and alerts.

    🏗️ Core Principles & Characteristics

    1. Prometheus (The Collector)

    • Pull-Based Model: Prometheus "scrapes" metrics from applications via HTTP endpoints (e.g., /metrics) at regular intervals.
    • Time-Series Data: Stores data as (metric_name, label_set, timestamp, value).
    • Multi-Dimensional Data: Uses "Labels" (key-value pairs) to filter and group metrics (e.g., http_requests_total{method="POST", service="payments"}).

    2. Grafana (The Visualizer)

    • Data Source Agnostic: Can visualize data from Prometheus, Elasticsearch, CloudWatch, and SQL databases.
    • Dynamic Dashboards: Supports variables to switch between different environments (Dev/Prod) or services quickly.

    ⚖️ Trade-offs: Pros & Cons

    • Pros: Highly scalable; PromQL (Query Language) is extremely powerful for math/aggregations; massive ecosystem support (exporters for almost everything).
    • Cons: Prometheus is not designed for long-term "durable" storage (usually capped at 15-30 days); the Pull-model requires service discovery to find pods in a dynamic K8s cluster.

    🌍 Real-World Implementation

    • Node Exporter: Installed on every server to monitor CPU, RAM, and Disk I/O.
    • Kube-State-Metrics: To monitor Kubernetes-specific data like "Pods Pending" or "Container Restarts."
    • Alertmanager: A Prometheus component that sends Slack/Email notifications if a metric (e.g., Error Rate > 5%) stays high for too long.
    • Custom Business Metrics: A Spring Boot app exposing "Total Orders Processed" or "Payment Latency" via the Micrometer library.

    💡 Interview "Gotchas" & Tips

    • Metrics vs. Logs: Metrics are for "What is happening now" (numbers); Logs are for "Why did it happen" (text). Mentioning the difference shows architectural maturity.
    • High Cardinality: Warning: Adding too many labels (like user_id) to a metric can crash Prometheus. Labels should have a limited set of possible values.
    • Pull vs. Push: Prometheus "Pulls." For short-lived batch jobs (which may finish before a scrape), use the Pushgateway.

    📐 Suggested Architecture Primitives

    • Exporters: Intermediate agents that translate 3rd party metrics (e.g., mysql_exporter) into Prometheus format.
    • PromQL: Use functions like rate() or histogram_quantile() for calculating throughput and latency percentiles (P99).
    • Dashboards-as-Code: Storing Grafana JSON definitions in Git to ensure consistent dashboards across clusters.
    Canvas