HALBACH

· MD

Unit prices and work per result: evidence audit

Research date: 22 October 2026. Audience: Knauf Digital technology leadership. Scope: public evidence for a 23-minute AI cost optimisation talk. No claim concerns Knauf’s measured spending. No deck or Git state was changed.

A lower token price does not determine the cost of an accepted result.

Use the historical price decline to establish that efficiency gains are real. Then show that one problem can be solved through one generated answer or many candidate answers plus selection. The relevant business measurement remains the full workflow: input, output and reasoning tokens; calls and tool use; failures and retries; infrastructure; and human review per accepted result.

This is an accounting implication and a design choice, not a claim that every enterprise bill is increasing. A cheaper token can reduce a bill when the workload is held fixed. Larger volumes, longer contexts, additional reasoning or more verification can offset that saving. Whether they do so at Knauf requires measurement.

Fact 1: a fixed benchmark threshold became more than 280 times cheaper per token

For models meeting MMLU >= 64.8, Stanford’s 2025 report records a fall from USD 20 per million tokens in November 2022 to USD 0.07 in October 2024. The underlying Epoch table supplies exact dates, 30 November 2022 and 3 October 2024. The model labels are GPT-3.5 and Gemini-1.5-Flash-8B.

The denominator is a 3:1 input/output weighted API token price, not cost per answered question or accepted business task. The threshold is benchmark performance, not identical overall capability. This is a changing-model frontier, not a price cut for the same model. The data compare non-reasoning models; the methodology explicitly excludes reasoning models because longer generated traces can make per-token comparisons misleading.

Primary sources:

Correction to preserve

Stanford’s summaries describe these endpoints as 18 months, while the chapter says about 1.5 years. Those durations disagree with the reported dates. The exact Epoch endpoints are 673 days apart, about 22.1 average months. Use the explicit dates or say under two years. Do not repeat the 18-month description.

The ratio 20 / 0.07 is about 286, but the input prices are rounded. The presentation should retain more than 280 times, rather than imply an exact 285.7-fold estimate.

Chart treatment

A two-endpoint slope or a stepped frontier with seven observed points is defensible. Use a logarithmic price axis if showing the full series, label it clearly, and show both dollar endpoints. A continuous curve would imply unobserved interpolation; use a step chart or points. Do not extrapolate past October 2024. Mark this as historical benchmark-level API pricing, not October 2026 pricing.

Suggested on-slide claim: GPT-3.5-level benchmark capability became over 280 times cheaper per token.

Suggested subtitle: MMLU >= 64.8; USD per 1M tokens; 3:1 input/output blend; Nov 2022–Oct 2024.

2026 update check

The official 2026 AI Index PDF was downloaded from Stanford and text-extracted locally after the web reader rejected the 37.9 MB file. The complete extracted text was searched for inference price, inference cost, price per, per-token and test-time terms. Its section 1.4, printed p.36, discusses energy and carbon estimates per query, not an updated fixed-capability token-price series.

I found no replacement for the 2025 price series in the checked 2026 report. This is a scoped report inspection, not a claim that no newer price research exists. The 2026 energy estimates must not be substituted for an API price or an enterprise bill.

Downloaded PDF SHA-256: fff5ac1df971cee7daf847fa5ed00dd7444db693092821c3e691debc1eaada09. The temporary PDF was inspected, not copied into the presentation deliverable.

Fact 2: the same benchmark problem can consume one answer or many candidates

In OpenAI’s September 2024 experiment, the research o1 model scored 74.4% pass@1 on AIME 2024 and 83.3% with consensus over 64 generated samples. This is 8.9 percentage points more accuracy, with a different inference and aggregation procedure. The article also reports a rounded 93% after reranking 1,000 candidates; keep that third configuration as optional backup, since it changes both sample count and the selector.

Primary source: OpenAI, Learning to reason with LLMs, Evals and Appendix A. Publication: 12 September 2024. Local data: 2026-10-22-o1-sampling-chart.csv.

This is a vendor-reported experiment on competition mathematics. The article expresses average scores against 15 problems per exam; it does not provide per-question token counts, a full candidate-cost ledger, confidence intervals for this comparison, or an enterprise acceptance test. It says evaluations generally use maximal test-time compute. One sample can itself contain substantial reasoning. The exact research o1 configuration is not interchangeable with launch o1-preview or current products.

Do not label the horizontal axis dollars, FLOPs or exact cost multiples. Sixty-four candidate solutions do not establish a 64-fold total bill: trace lengths, prompt reuse, batching, caching and aggregation overhead matter. Label it candidate solutions per problem, with the selection method alongside it. Do not compare the rounded main-text values to the more precise appendix figures without explaining the rounding.

Suggested on-slide claim: One problem can use 64 candidate answers to improve its success rate.

The leadership implication is to treat inference strategy as a budgeted quality decision. Measure whether additional attempts improve accepted outcomes enough to justify their end-to-end cost. This benchmark does not tell us that 64 samples are appropriate for Knauf.

Supporting research: allocation can matter as much as brute force

Snell et al., 2024, sections 3–6 and figures 4 and 8, compare verifier search and revision strategies using a fine-tuned PaLM 2-S* model on the MATH 500 test questions. Their reported low-budget search comparison uses 16 versus 64 generations; their revision comparison uses 64 versus 256. Adaptive allocation can match or surpass a parallel best-of-N baseline with approximately a quarter of the generation budget in these comparisons.

Crucial limits: difficulty estimation itself uses expensive sampling that is excluded from their accounting; capabilities were specifically fine-tuned; compute is a proxy rather than a measured API bill; difficult questions show limited gains, and stronger optimisation can harm easy-question performance. The results support testing adaptive budgets, not promising a 75% production saving.

Use this study as technical backup for spend effort where it changes the answer. Do not recreate a numerical accuracy frontier by reading approximate y-values from a low-resolution chart. The local CSV stores the explicitly stated generation budgets only, with accuracy deliberately blank.

Selection for the main talk

Recommended main-stage evidence:

  1. The two price endpoints, with the fixed benchmark and historical dates visible.
  2. The two o1 configurations, if the narrative needs one concrete illustration of work per problem.

The two charts answer different questions and should have separate axes and footnotes. They are not an empirical model of Knauf’s future expenditure. A bridge sentence can be: The unit got cheaper. We still decide how much work to spend on each accepted result.

Avoid:

  • Universal or forecast claims that AI bills must rise.
  • Treating MMLU equivalence as equal performance on product-data extraction.
  • Calling the historical price frontier current pricing.
  • Conflating API price with supplier compute cost.
  • Presenting additional samples as a guaranteed accuracy improvement on every task.
  • Promising production savings from the research compute proxy.

Files and verification

  • 2026-10-22-claim-ledger.json: scoped claims, verdicts, source locators and exclusions.
  • 2026-10-22-fixed-threshold-price-chart.csv: seven observed historical frontier points.
  • 2026-10-22-o1-sampling-chart.csv: primary-source AIME configurations, with candidate counts and methods.
  • 2026-10-22-adaptive-budget-comparisons.csv: research generation-budget comparisons, not dollar savings.
  • 2026-10-22-source-manifest.json: retrieval and provenance, including the full 2026 report check.

Arithmetic was independently recomputed for the price ratio, date interval and AIME percentage-point difference. Public sources only. No source passages were quoted verbatim.