Part deliverables and assessment
The course direction is a draft, not a confirmed grading contract. Dates below are its planned dates; January dates fall in 2027.
Part 1 — The Model as a Measured Component
Classes 1–3 · 8–22 November 2026 · Currently in progress
Build an LM wrapper with cache, cost ledger, and budget cap; version a structured-output schema; then grow the provided loop into a three-tool agent over the Class 2 notices.
The Class 3 handoff is a tested three-tool agent, a one-command 20-task evaluation with k = 3, pass^3 per task, and JSONL run records. Scripted-LM tests cover a malformed reply, a tool exception, and the step cap. Tag the repository part1-ref.
Part 2 — The Agent as Tested Subsystems
Classes 4–6 · 29 November–13 December 2026 · Deliverable due 19 December
Add retrieval, an MCP server and client, context policy, memory, and a resumable progress file. Record an architectural decision with a C4 container diagram, trust boundaries, the quality attribute traded, and at least one measurement.
Class 6 compares no policy, observation masking, and summary compaction. Run one long task per arm in class; then 5 tasks × 3 arms × k = 3 on the LAN server after class. The handoff is agent + MCP + C4 ADR, tagged part2-ref.
Part 3 — The Harness as a Control Layer
Classes 7–9 · 20 December 2026–3 January 2027
Start from part2-ref. Diagnose seeded faults, write configuration fitness checks, evaluate a skill with distractors and held-out prompts, and deliver trajectory matchers under specifications, locked tests, and layer gates. Harden both the coding agent and the team's own tool policy.
Class 9 requires zero bypasses on 10 public and 5 held-out probes on both agents, a dependency gate that rejects seeded invented/typosquatted packages, and a report distinguishing vendor-provided controls from student-built controls. The reference handoff is part3-ref.
Part 4 — The Architecture as an Evaluated Claim
Classes 10–12 · 10–24 January 2027 · Final paper due 31 January
- Class 10: Calibrate a local judge against 30 hand labels, revise it once, report both kappa values, test trajectory matchers, add a pass^k CI gate, and submit a one-page analysis plan naming variable, metric, k, and test.
- Class 11: Compare bare, harnessed, and one extension arm on three tasks, k = 3, recording pass^3, tokens, cost, and minutes per accepted task. Review three passing patches by hand and tag
data-frozen. Paper draft due Friday, 22 January. - Class 12: A peer team recomputes the headline number from frozen raw data and reruns one task end to end. Each team member answers a roughly three-minute code question. Write a one-page decision memo with cost per accepted task, risk, and recommendation.
The final output is a four-page paper with a falsifiable question, baseline, k ≥ 3, intervals, error taxonomy, and threats to validity. The peer reproduction must agree within the stated interval.
Proposed assessment
The draft asks whether the academic committee accepts these weights:
| Component | Proposed weight |
|---|---|
| Final | 30 |
| Midterm | 10 |
| In-lab checks | 24 |
| Part deliverables | 24 |
| Paper | 12 |
| Total | 100 |
Exam dates, detailed marking rules, and approval are not supplied. Class-priority scores and ranks in the PDF are curriculum-planning metadata, not student marks.
Research pathways
Class 12 identifies five thesis pathways arising from course data:
- Bangla tokenizer fertility: tokenization cost differences across Bangla and English.
- Context policy: observation masking versus summary compaction.
- Specification versus prompt: free-form versus specification-guided delivery under fixed tests.
- Judge calibration: agreement with human labels and change after revision.
- Harness cost: success, cost, and time per accepted task across architectures.
The draft mentions MSR, LLM4Code, and BLP as research venues.