Lab 02. Inference capacity and structured output as a contract
Part 1 · Scheduled 15 Nov 2026 · 2-hour lab
Part 1 is currently in progress. This page records the supplied class plan; personal notes and results have not been added.
Lab brief
Calculate the KV-cache size of Qwen2.5-7B at 8k and 32k. Extract 50 domain notices into a versioned JSON schema, prompt-only vs grammar-constrained (Ollama format or llama.cpp json_schema), 3 runs each, priced through the class 1 ledger. The notices are the corpus for classes 3-6.
What to build and measure
- Compute the KV-cache size of Qwen2.5-7B at 8k and 32k tokens; compare with the answer key.
- Extract 50 domain notices into a versioned JSON schema: prompt-only vs grammar-constrained, 3 runs each, priced through the class 1 ledger. The notices are the corpus for classes 3-6.
- Write a contract test that accepts an additive schema change and fails a breaking one.
- Report invalid rate and field accuracy per method, with spread.
Why this class matters
- Industry: Schema contracts and constrained decoding are how structured LM output goes to production; a spread of 3 runs is the minimum honest report.
- Academic: The prompt-only vs grammar comparison is a small controlled experiment with a known paper to compare against.
- Elsewhere: Students learn to version an interface that a model, not a programmer, fills.
In-room pass check
Invalid rate, field accuracy and cost per method with spread; a contract test accepts an additive change and fails a breaking one; KV numbers within 10% of the key.
Required materials
- Ollama format or llama.cpp json_schema set up and piloted (not yet piloted).
- 50 notices and the answer key for KV numbers.
- Starter repo with schema version 1 and the contract-test skeleton.
These are materials the draft assigns to the course team. They are not bundled with these notes. See the course guide for the test-first verification log and the deliverables for part boundaries.
Results ledger
[Measurements, test output, run-record paths, and verification log]
Sources
- Ollama docs