Lecture 11. Harness architectures compared on cost, time and success

Part 4 · The Architecture as an Evaluated Claim · Scheduled 17 Jan 2027 · 1-hour lecture

Upcoming course outline. This class has not been recorded as studied.

Learning objectives

SWE-agent, mini-SWE-agent and Pi as reference architectures, scored on the class 1 quality attributes.
HarnessTax: harness choice moved success about 2-5 points but cost up to 5x; Jain: a harness took a cheap model from 3/10 to 10/10.
Autonomy levels and the run contract; single agent first; worktrees isolate changes, not behaviour; MAST failure taxonomy.
Review with agents: read the test diffs first.

Industry: 'Does the harness pay?' is the question every team asks before buying tooling; students answer it with cost per accepted task.
Academic: Bare vs harnessed, k = 3, with cost: a complete experiment and the core of the Part 4 paper.
Elsewhere: Students know when a multi-agent setup is a trap and when it helps.

Stanford CS329Z L5 and L8 slides; L16, L18 when published.
arena-harnesstax; Jain, Are harnesses worth the cost?; Osmani on autonomy and review.
The paper (Yang et al. 2024) as pre-reading.

These are preparation requirements in the draft; staff materials are not bundled with this site.

Stanford CS329Z L5: Frameworks and orchestration (slides)
Stanford CS329Z L8: Multi-agent systems (slides)
Stanford CS329Z L16: Guest lecture: SWE-bench (schedule entry, slides not published)
Stanford CS329Z L18: Coding and software agents (schedule entry, slides not published)
Arena.ai: HarnessTax
Osmani: Agentic Autonomy Levels
Yang et al., SWE-agent (NeurIPS 2024)

The linked reference library contains the source URLs extracted from the course PDF. Availability notes above reflect the draft of 4 November 2026.

Continue with Lab 11: Harness architectures compared on cost, time and success. Every lab contributes an artifact to the same repository and a measurement to the results ledger.

[Lecture notes and observations]