Lecture 02. Inference capacity and structured output as a contract

Part 1 · The Model as a Measured Component · Scheduled 15 Nov 2026 · 1-hour lecture

Part 1 is currently in progress. This page records the supplied class plan; personal notes and results have not been added.

Learning objectives

The decoder block as a concept; prefill vs decode; why the first token is slow and the rest are fast.
KV-cache arithmetic in 10 min: layers x heads x head-dim x 2 x bytes x tokens; memory at 8k and 32k.
Latency budget: time to first token, tokens per second, prompt caching; what each costs in money and seconds.
Temperature, top-k, top-p; effort as test-time compute, the mechanism behind the effort setting.
Post-training in one slide: where the tool-call format comes from.
Constrained decoding as a logit mask; structured output as a versioned interface contract.

Industry: Students size GPU memory, set decoding parameters with a reason, and ship JSON output that does not break parsers.
Academic: Builder-depth model internals is the 'new knowledge' a university course must add to training; it links the course to an ML thesis.
Elsewhere: Students can read a model card and an inference-server config and know what each number does.

These are preparation requirements in the draft; staff materials are not bundled with this site.

The linked reference library contains the source URLs extracted from the course PDF. Availability notes above reflect the draft of 4 November 2026.

Continue with Lab 02: Inference capacity and structured output as a contract. Every lab contributes an artifact to the same repository and a measurement to the results ledger.

[Lecture notes and observations]