Assignment 05. Leader-Elected Delivery Coordinator with Fencing

Week 10 · Assignment brief from the course outline; submission and evaluation results have not been recorded.

Dependency: Extends A4 reliable checkout

Scenario and assignment objective

Delivery assignments use three coordinator replicas. After the active one fails, a replacement should take over without allowing stale coordinators to write.

Required system features

1. Add three coordinator processes/containers with unique ID and status endpoint.
2. Implement educational bully election OR a scoped term-based election using messages/heartbeats.
3. Track monotonically increasing leader epoch, stored centrally and verified atomically at write time.
4. Only a coordinator with the current fencing epoch may assign a courier.
5. Enforce no duplicate courier assignment through DB constraints/transactions.
6. Log candidate->leader/follower transitions, election rounds and observed failures.

Implementation guide - suggested steps

1. Introduce Coordinator interface: Heartbeat, ElectionMessage, LeaderAnnouncement, AssignCourier.
2. Model follower/candidate/leader states, peer list and a bounded timeout; separate election from job logic.
3. On leadership acquisition, obtain a strictly larger fencing epoch from shared durable state.
4. Use a conditional SQL write: assignment accepted only if requested epoch equals current epoch.
5. Run three instances with IDs; test ordinary assignment, leader crash, and old leader returning.
6. Record which failure cases are covered: crash-stop and delayed messaging, not arbitrary partition consensus.

Hints (not complete solutions)

• A process claiming leadership is insufficient; the shared store must authorize its write.
• Use a transaction with row lock or CAS-style version update to guard current fencing epoch.
• A network timeout means suspicion, not proof of process death.
• Use deterministic simulated couriers and independently verify unique active assignments.

API / data contracts

• GET /coordinator/status -> {node_id,role,epoch,last_heartbeat}.
• POST /deliveries/{delivery_id}/assign -> {courier_id,leader_epoch,status}, guarded by fencing.

Mandatory verification and failure experiments

1. Leader container stops; remaining connected nodes elect a replacement.
2. Old leader resumes and its previous epoch is denied when assigning a courier.
3. Concurrent assignments do not allocate one courier to two active deliveries.

Required individual submission

• Election diagram, three-node Compose environment, epoch enforcement code, logs.
• Automated failover/fencing tests and explicit model assumptions.

Acceptance checklist

• Leader failover observed in stated failure model.
• Stale coordinator cannot commit courier assignment.
• Concurrent assignment safety is enforced by durable state.

Marks: 7 / 40 lab marks. Use the common rubric: implementation 40%, tests 25%, fault experiment 20%, documentation/viva 15%.