Lab 12. Reproduced results, validity threats and thesis paths
Part 4 · Scheduled 24 Jan 2027 · 2-hour lab
Upcoming course outline. This class has not been recorded as studied.
Lab brief
Peer reproduction only: team B follows team A's README, recomputes the headline number from frozen raw data and reruns one task end to end; fills the review form; each team A member answers one code question (about 3 min). Team A writes a one-page decision memo for a manager: cost per accepted task, risk, recommendation.
What to build and measure
- Team B runs team A's README: recompute the headline number from the frozen raw data; rerun one task end to end.
- Fill the review form; each member of team A answers one question on the code (3 min); team A writes a one-page decision memo for a manager (cost per accepted task, risk, recommendation).
- Pass check: the headline number is reproduced within the stated interval; the rerun finishes; the draft has a falsifiable question, baseline, k >= 3, intervals, error taxonomy and threats.
- Final paper due Sun 31 Jan.
Why this class matters
- Industry: A README that a stranger can run is the standard for any internal experiment or benchmark.
- Academic: Peer reproduction is the one step most student projects skip; it makes the 4-page paper defensible.
- Elsewhere: Students experience review from both sides.
In-room pass check
Peer team reproduces the headline number within the stated interval; the rerun finishes; the draft has a falsifiable question, baseline, k >= 3, intervals, error taxonomy and threats; the memo exists.
Required materials
- Drafts in by Fri 22 Jan; frozen data tagged in class 11.
- Review form (ready by 3 Jan).
- LAN server for the reruns.
These are materials the draft assigns to the course team. They are not bundled with these notes. See the course guide for the test-first verification log and the deliverables for part boundaries.
Results ledger
[Measurements, test output, run-record paths, and verification log]
Sources
See the paired lecture and the reference library.