Lecture 05. Coordination, failure detection and leader election
Week 9 · Syllabus and preparation from the course outline; no study date or personal lecture notes have been recorded.
Business scenario
Three Delivery Coordinator instances are running. The service should assign couriers through one active coordinator and reject stale leaders.
Topics to cover in this class
- Heartbeats, lease expiry, incomplete failure detection and false suspicion
- Bully election or a scoped term-based election under a crash-only model
- Epochs / fencing tokens, atomic database ownership checks and stale leadership
- Split brain in partitions; safe assignment and unique active courier constraints
- Controlled container failures, recovery logs and declared safety assumptions
Learning objectives
- Demonstrate coordinator election and failover.
- Prevent a stale coordinator from committing a new assignment.
- Describe limitations of heartbeat-only election.
Live demonstration checklist
- Start three coordinator containers with distinct IDs and expose role/epoch status.
- Stop leader, observe election, restart old leader and reject stale write.
Preparation and optional written reflection (ungraded)
Reading focus: Read introductory Raft leader election discussion; compare consensus with a simple bully election. Optional individual reflection: Explain why two nodes can both believe they are leader and what a fencing token can prevent.
Study materials: official docs, websites and YouTube
MIT 6.5840 | Course schedule and readings | https://pdos.csail.mit.edu/6.824/schedule.html Raft/coordination readings
Website | Raft visualization | https://raft.github.io/ Accessible animated intuition and links
Official | PostgreSQL INSERT on conflict | https://www.postgresql.org/docs/current/sql-insert.html Atomic conflict resolution and uniqueness
Website | Arpit Bhayani distributed systems lessons | https://arpitbhayani.me/topics/distributed-systems/ Leader-election algorithms and explanations
YouTube | Raft Algorithm Explained | https://www.youtube.com/watch?v=TKz7DLDw_hc Terms, leadership and log agreement overview