HALBACH

· MD

AI cost economics — research for Knauf Digital

Research date: 22 October 2026. Session: 29 October, 14:30–15:00, with 20–25 minutes of content by Geoffrey Kasenbacher and Tebjan Halm. This memo supports the outline; it is not a provider recommendation or an assessment of Knauf’s current estate.

Executive conclusions

  1. Optimise cost per accepted business outcome. A cheap model call can become an expensive workflow if it requires retries, review or correction. This is our analytical framing, not a claim that any particular model is better.
  2. The model bill is a component of cost. Production architecture also requires data preparation, retrieval, application infrastructure, guardrails and operations. AWS’s production guidance explicitly includes token rates, traffic patterns and infrastructure in the cost model. C12 — AWS
  3. The largest available saving depends on the workload. Model choice, workflow shape, repeated context, latency tolerance and hosting demand are different decision variables. Do not present a universal priority order without Knauf’s measurements.
  4. Self-hosting converts some variable spend into capacity and operating obligations. Its economics depend on demand and qualified throughput. AWS recommends establishing a demand profile and right-sizing to performance needs before optimising hosting. C11 — AWS
  5. List price comparisons belong in backup material. They are useful to explain billing mechanics; they do not establish task quality, actual tokens, enterprise terms or total cost.

1. Define the decision before comparing prices

Use three views together. These are proposed management definitions for this talk.

View Question Include Useful for
Marginal cash cost What does the next unit of work cost? Incremental model/tool usage, additional review, burst capacity Routing, retries, scaling a live workflow
Avoidable cost What spend disappears if we stop or change this workload? Cancellable services, released capacity, actual labour changes Scale/optimise/stop decisions
Fully loaded cost What resources sustain this service? Runtime plus allocated platforms, people, maintenance, amortised implementation Portfolio economics and build/buy choices

A shared engineer’s time may be a real opportunity cost without creating an immediate cash saving. An already purchased GPU can have low incremental compute cost while still consuming scarce capacity. Do not count sunk implementation spend as a reason to keep a poor workflow, or treat all historical investment as recoverable savings.

Recommended primary unit: cost per outcome that meets a defined acceptance criterion within the required service level. Examples for discussion only: a product record accepted into the catalogue; a source-grounded answer accepted by a support agent; an invoice exception correctly resolved. These are illustrative, not confirmed Knauf deployments.

Report the denominator explicitly: attempted records, completed records and accepted records are different. Record rejected outputs, human touches, rework and turnaround time alongside cost. A single metric cannot settle the business case; throughput, adoption and the operational value of the outcome also matter.

2. A practical full cost stack

The table is a proposed accounting model, synthesised for the visit rather than a measured industry cost breakdown. AWS identifies production cost categories including query volumes, prompt/completion tokens, infrastructure, vector stores and guardrails; FinOps provides date-aware request-level attribution guidance. C12 — AWS, C10 — FinOps Foundation

Layer Typical meter What can be missed Decision implication
SaaS and embedded AI Seats, credits, actions, usage add-ons Inactive users, duplicated licences, mandatory base product, bundled allowances Improve adoption or contract fit; token optimisation may not reduce the invoice
Managed model inference Input, output, cache reads/writes, modality Reasoning, repeated context, multiple agent steps, premium geography/tier Measure actual billed categories per workflow
Tools and data processing Searches, OCR/pages, reranking, embeddings, executions Tool calls can generate more prompts; reruns of ingestion Budget the entire transaction
Retrieval and application services Storage, database queries, CPUs, queues, network Index refresh, document versions, logging and backups Include steady and change-driven costs
Dedicated capacity Accelerator/instance hours, provisioned throughput Idle hours, redundancy, peak headroom, warm-up, fragmentation Use qualified demand curves and capacity steps
Build and run Engineering, platform operations, integration, support Evaluation datasets, upgrades, on-call, security work Include operating obligations in build/buy choices
Quality and process Human review, rework, escalations, failed cases Cheap calls that shift work downstream Compare accepted-outcome economics

Scope boundary: generic deterministic automation, search or a small specialised model may meet a narrowly defined task more economically than a general-purpose generative model. This is an option to test, not a presumed result. Do not make this session solely about LLM tokens when the brief covers AI costs more broadly.

3. Training, inference and SaaS require different models

These are analytical cost-model distinctions:

  • Inference is running an existing model for a task. Forecast volume, traffic shape, token/modality mix, calls per task, tools, retries and the required latency.
  • Fine-tuning or adaptation adds data preparation, training runs, experiment evaluation, versioning and deployment. Compare the extra lifecycle cost with demonstrated task improvements or runtime savings over a realistic lifetime. An adaptation technique is not itself evidence of cheaper inference.
  • Pretraining is a different investment category with large experiment and infrastructure implications. It need not become the centre of this enterprise cost talk unless the audience actually operates it.
  • SaaS/embedded AI may offer bundled capabilities and contractual usage units. Optimising a prompt can improve experience while leaving committed licence spend unchanged. Measure adoption and unit value at the contract level.

Always separate development/evaluation consumption from production use. Treat necessary evaluation as a deliberate expense that prevents a misleading cost reduction from entering production.

4. What a token bill actually measures

A useful generic invoice identity is:

model cost = ordinary-input units × input rate + cached-read units × read rate + cache-write units × write rate + billed-output units × output rate

workflow cost = sum(model calls + tools + data services) + review/rework + attributable operating cost

Apply provider-specific usage-field semantics; token categories must not overlap. For OpenAI reasoning models, reasoning is part of billed output, including cases that hit the output limit before giving a visible answer. Do not add the reported reasoning subtotal to output again. C06 — OpenAI

The number of calls per business transaction is a design variable: routing, agents, reflection, validation and retry loops can each add calls. Track cost at both call and transaction level. A short user question can invoke a long system prompt, retrieved documents and several tool turns.

Current representative rate evidence

The machine-readable file provider-pricing.csv is a 22 October 2026 list-price snapshot, in USD per one million model tokens, covering representative OpenAI, Anthropic and Google rates, cache modes and processing tiers. It contains no quality ranking. No EUR conversion is assumed. Tax, negotiated contracts, committed-use offers and additional regional premiums are excluded. Empty fields mean not transcribed/not applicable, never zero.

Opened official pages were used because search-cache snippets contained older model names and terms. Google labels its page updated 19 October. Its Argon price is promotional through 30 November, so it is explicitly dated. Current pricing is evidence of terms available to inspect, not a guarantee of Knauf account access. C01 — OpenAI, C02 — Anthropic, C03 — Google

The API changelog explicitly places the short-context boundary for GPT-6.1 Sol and Luna at 272,000 input tokens; Haiku 5.5 has a different 100,000-token price boundary. Rate tables must carry model, tier and context conditions rather than a single provider price. C13 — OpenAI, C02 — Anthropic

Worked billing illustration, not a deployment forecast

Assume 100,000 independent monthly requests; each has 3,000 ordinary input tokens and 500 total billed output tokens; no cache writes, tools, retries or premium tier. At GPT-6.1 Sol Standard short-context rates of $2 input/$10 output per million:

  • Input: 100000 × 3000 / 1000000 × $2 = $600.
  • Output: 100000 × 500 / 1000000 × $10 = $500.
  • Model usage: $1,100/month, or $0.011 per request.

Arithmetic is ours; rates are from C01. Output length includes reasoning under the assumption. This is an isolated model-meter example, not TCO or cost per accepted outcome.

If only 80% pass the acceptance rule without additional work, the same $1,100 model bill divided by 80,000 accepted items becomes $0.01375 per accepted item, before review or rework. Acceptance rate is hypothetical; it is not a result for this model.

5. Caching, batching and reasoning: benefits with eligibility rules

Prefix caching

An unchanged prompt prefix can reuse model state. OpenAI’s current GPT-5.6+ rules include a 1,024-visible-token minimum, write fees and retention choices; GPT-6.1 Sol’s read rate differs from other family members. Confirm billed read/write counters, not just a cache-enabled flag. C04 — OpenAI

Our algebra: for a repeated prefix processed N times, let ordinary processing cost P, write multiplier w and read multiplier r. Without caching: N × P. With one write and N−1 complete reads: P × [w + (N−1)r]. Saving exists when N > (w−r)/(1−r), provided the entries remain valid and the prefix matches. This excludes changed input, output, tools and implementation effort.

Illustration with the transcribed Sol rates: w=1.25, r=0.05 and N=10 gives 1.70P rather than 10P: 83% lower cost for that repeated prefix only. Do not label this an 83% application saving. Low reuse, expiry, changing instructions or writes for unused prefixes can erode the benefit.

Google’s explicit caching also bills storage time; Argon supports implicit caching only. Therefore a generic cache hit percentage cannot be applied across every model or cache mode. C07 — Google

Batch processing

OpenAI’s asynchronous Batch API publishes a 50% model cost discount with a 24-hour processing window and explicit expired-job handling. It fits work that can wait, such as an overnight catalogue update; it does not establish suitability for an interactive product. Separate provider batch processing from batching requests inside a self-hosted inference engine. C05 — OpenAI

Our arithmetic: if 60% of otherwise eligible model spend moves to a tier priced 50% lower, model spend falls 30% before other changes. If model spend is 20% of total service cost, the direct TCO reduction is 6%, before scheduling and support costs. Both shares are hypothetical.

Reasoning and output control

Set effort and output limits against task difficulty and acceptance tests. Shortening the visible answer alone does not establish reduced reasoning consumption. An aggressive cap can produce incomplete results and retries. Use observed output usage and failure rates. C06 — OpenAI

6. API versus self-hosting: an explicit break-even model

This section is our conditional economic model, not a claim that a particular open model matches a proprietary API. Establish equivalent task acceptance, data handling, latency, availability and operational coverage first. If equivalence fails, cost arithmetic cannot settle the choice.

Let Q = monthly accepted outcomes, A = API variable cost per accepted outcome, v = self-host variable cost per accepted outcome, F_api/F_self = monthly avoidable fixed costs.

C_api(Q) = F_api + A × Q

C_self(Q) = F_self + v × Q

Q* = (F_self − F_api) / (A − v) when A > v and additional fixed self-host cost is positive.

Capacity is a separate constraint:

Q <= n × H × 3600 × r × u

Here n is serving replicas, H is paid hours per replica, r is measured accepted outcomes/second that meet the SLA, and u is the productive fraction across paid hours. u includes idle time and demand variation; it is not GPU utilisation reported by a hardware monitor. r already includes the acceptance criterion, so do not multiply by acceptance again.

F_self should include committed capacity, required standby replicas, platform and on-call cost, licences, and amortised migration/setup as appropriate to the chosen decision view. Include API integration fixed cost too. If existing platform cost is shared, report the allocation separately from cash avoidability.

Synthetic sensitivity example

Assume F_api=$0, F_self=$10,000/month, A=$0.020/accepted outcome, v=$0.002, H=730 paid hours, n=1 and r=1 accepted outcome/second. These are teaching assumptions, not quotes, measurements or a recommendation.

Break-even is 555,556 accepted outcomes/month, about 21.1% of the assumed 2,628,000-outcome monthly ceiling. The sensitivity file gives:

Productive fraction Accepted outcomes/month Self-host cost/accepted outcome API cost/accepted outcome
10% 262,800 $0.04005 $0.02000
25% 657,000 $0.01722 $0.02000
50% 1,314,000 $0.00961 $0.02000
75% 1,971,000 $0.00707 $0.02000

The 100% ceiling exists in the CSV for auditability, not as a planning recommendation. Bursty demand, maintenance, cold starts or an availability requirement can add replicas and increase F_self in steps. Twice the fixed cost doubles Q* under these assumptions; half the qualified throughput doubles the productive fraction needed for the same Q*. If A falls or v rises, break-even moves away; if A <= v, increased volume cannot recover higher fixed cost in this model.

For scale context only, Lambda’s specific one-GPU H100 PCIe rate is $3.29/hour: 730 hours would be $2,401.70 before tax. That is a rental meter, not an all-in serving cost, an EU quote or proof one GPU can serve the workload. The source does not establish geography or production SLA for this plan. The synthetic sensitivity deliberately does not infer throughput from that price. C09 — Lambda

7. Benchmark evidence: what leadership should demand

NVIDIA’s methodology separates queue, prefill and decode and emphasises input/output lengths, concurrency, throughput and latency. Accuracy must be validated separately. C08 — NVIDIA

MLCommons defines benchmarks through datasets, quality targets and serving scenarios. Its new end-to-end RAG benchmark starts with an Offline scenario; offline results should not be recast as online latency evidence. C14 — MLCommons, C15 — MLCommons

Our proposed comparison checklist:

  • Same representative documents, languages and acceptance rules, including hard cases.
  • Actual model snapshot, tokenizer, prompt, precision, engine and hardware configuration.
  • Realistic input/output distribution, tool use, cache reuse and peak arrivals.
  • End-to-end p50/p95 or p99 latency, plus time to first token where relevant.
  • Accepted outcomes per paid hour and fully specified cost boundary.
  • Failure, retry and human-review rates, with identical accounting on both sides.

Avoid a provider league table based on price per token alone, or a GPU chart with maximum throughput and no response-time constraint. A high cache-hit benchmark on repeated synthetic prompts does not predict a low-reuse document workflow.

Main body: one cost-stack diagram, one outcome-based example, one practical eligibility matrix and one workload/demand-based hosting decision. Keep provider rates and break-even arithmetic in backup material unless the audience asks.

Suggested line for Geoffrey: “The useful question is what one accepted result costs — including the work around the model.”

Suggested line for Tebjan: “A GPU hour only becomes cheap per task when the system can use it productively and still meet the response-time requirement.”

For this leadership audience, finish with a concrete measurement assignment: pick one high-volume workflow, name the cost owner, define acceptance and latency, capture one representative period, then trial the lowest-effort eligible changes. The ordering is a proposed working method and should be adjusted when Knauf provides workload evidence.

9. What remains unknown

No Knauf consumption, contracts, model mix, document distribution, residency requirements, staffing allocation, workload peaks or acceptance benchmarks were supplied. There is no basis for a quantified Knauf saving, build/buy verdict or savings guarantee. Recheck mutable rate cards shortly before the visit; preserve the dated CSV when updating rather than silently overwriting the evidence trail.