Lecture 03. The agent loop as a tested, evaluated component

Part 1 · The Model as a Measured Component · Scheduled 22 Nov 2026 · 1-hour lecture

Part 1 is currently in progress. This page records the supplied class plan; personal notes and results have not been added.

Learning objectives

A tool call is structured output that your code runs; the schema is the contract.
The loop: step cap, errors returned as observations, stop conditions; one run record per task (steps, tool calls, tokens, cost, wall-clock) that classes 6, 8, 10 and 11 read.
A scripted LM makes the loop unit-testable without a model.
One run is not an evaluation: task, trial, grader, transcript; pass@k vs pass^k; code graders.
Recap, 5 min maximum: 70-toolcall and 80-agentloop.

Industry: Students test an agent loop like any other service and report pass^k instead of a demo.
Academic: The eval habit starts here; every later class and the Part 4 paper reuse this 4-tuple.
Elsewhere: Students can judge vendor claims that say 'it works' with one run.

These are preparation requirements in the draft; staff materials are not bundled with this site.

Stanford CS329Z L1: Introduction: what are agentic systems? (slides)
Stanford CS329Z L4: Tool use and function calling (slides)
Stanford CS329Z L13: Evaluation fundamentals and benchmark design (schedule entry, slides not published)
Anthropic: Demystifying evals for AI agents
Yao et al., ReAct (ICLR 2023)

The linked reference library contains the source URLs extracted from the course PDF. Availability notes above reflect the draft of 4 November 2026.

Continue with Lab 03: The agent loop as a tested, evaluated component. Every lab contributes an artifact to the same repository and a measurement to the results ledger.

[Lecture notes and observations]