Redo Logs: Ensuring Durability

📋 Overview

A Redo Log is a critical component of transactional databases used to guarantee the Durability (D in ACID) of committed changes. It is a disk-based data structure that records all modifications made to the database in a sequential, append-only fashion. In the event of a system crash, the database uses the Redo Log to "replay" or "redo" committed transactions that were in memory but hadn't yet been written to the main data files.

---

🏗️ Core Principles & Characteristics

• Write-Ahead Logging (WAL): Redo logs are the primary implementation of the WAL principle: "Changes must be logged to disk before they are applied to the data files."
• Sequential I/O: Writing to a log is a sequential append operation, which is significantly faster than the random I/O required to update various tables and indexes across the disk.
• Circular Buffers: Many databases (like MySQL) use fixed-size circular redo logs; once the log is full, the database must flush old changes to the main data files before it can overwrite old log entries.
• Recovery: During startup after a crash, the database scans the redo log and reapplies any committed changes that are missing from the tables.

---

⚖️ Trade-offs: Pros & Cons

• Pros: Dramatic performance boost (turns slow random writes into fast sequential writes); ensures data integrity; enables recovery from power failures or OS crashes.
• Cons: Adds a small latency to every transaction (the "log flush"); disk space overhead; if the redo log file is corrupted, recovery becomes impossible.

---

🌍 Real-World Implementation

• MySQL (InnoDB): Uses the ib_logfile0 and ib_logfile1 files as its circular redo logs.
• PostgreSQL: Implements this concept via WAL files in the pg_wal directory.
• Oracle Database: Uses a complex set of Online Redo Log groups to ensure zero data loss.
• Cassandra: Uses a "Commit Log" for the same purpose before writing to Memtables.

---

💡 Interview "Gotchas" & Tips

• Redo vs. Undo: This is a classic trap. Redo logs are for recovery (replaying committed changes). Undo logs are for rollback (reversing uncommitted changes if a transaction fails).
• Double Buffering: Mention that data is first written to a "Redo Log Buffer" in RAM and then flushed to disk. The frequency of this flush (e.g., innodb_flush_log_at_trx_commit) defines the risk of data loss vs. performance.
• Log Sequencing: Each log entry has a unique LSN (Log Sequence Number) which the database uses to keep track of where it is in the recovery process.

---

📐 Suggested Architecture Primitives

• Append-Only Storage: Optimized for write-heavy sequential access.
• Check-pointing: The process where the database periodically flushes dirty pages to disk to shorten the time required for crash recovery.
• ACID Compliance: Redo logs are the "heart" of the Durability guarantee.