# Optimise the cost of a completed task, not a token in isolation

**Recommendation:** Build the financial model around a defined business task, then decompose its cost into workflow demand, service consumption, provisioned capacity and operating effort.

**Why:** Token prices measure one input to delivery. They omit failed attempts, retrieval, other models, human review and the cost of maintaining the system. The FinOps unit-economics framework explicitly distinguishes resource-efficiency metrics from business-outcome metrics. [S02]

## 1. Separate three financial questions

| Question | Appropriate measure | Common mistake |
| --- | --- | --- |
| What changes if we process one more task? | Marginal cost | Allocating every fixed cost to a token and treating it as avoidable |
| What does the service cost to operate? | Fully loaded cost over a defined period | Counting only the LLM invoice |
| Should we invest or change architecture? | Lifecycle incremental costs and benefits | Ignoring migration, integration, evaluation and operating effort |

A dedicated GPU's idle hours are part of the service's cost. They are not necessarily caused by the next request. Conversely, a small API charge can trigger substantial review or rework.

## 2. Map costs without double-counting

| Cost family | Typical items | Measurement basis |
| --- | --- | --- |
| Employee AI tools | Provisioned seats, included allowances, incremental overages | Actual contracts and invoices; unused seats still incur cost |
| Purchased model consumption | Uncached input, cache writes/reads, output, reasoning, audio/image units | Provider-reported billable units and applicable rate schedule |
| Internally operated inference | Rented GPUs, CPU, storage, networking, replicas | Paid resource time and contracted rates |
| Supporting services | Retrieval, embeddings, reranking, OCR, vector storage, tool APIs | Component-level usage |
| Operating the product | Observability, evaluations, support, security, maintenance | Infrastructure bills plus explicitly scoped labour |
| Human handling | Review, exception resolution, corrections | Observed time per task and loaded cost assumptions |
| Establishing/changing the service | Integration, data preparation, tuning, migration, training | One-off investment and agreed amortisation period |

For rented hardware, electricity is normally embedded in the rental price: do not add an owned-datacentre energy model to the same rental charge. Report direct consumption, shared allocations and one-off investments separately before producing a total.

For coding assistants, explicitly include additional CI runs, preview environments and review effort induced by the workflow. Compare provisioned with active seats over a defined window before a renewal decision. A vendor's API-equivalent usage valuation under a subscription is not an extra invoice charge. These are accounting recommendations prompted by the Navigara review, not measured Knauf savings. See chapter 10 [local reference omitted].

## 3. Model API consumption using exclusive billing buckets

For a single request, an illustrative model is:

```text
request_cost = uncached_input × input_rate
             + cache_write_units × write_rate
             + cache_read_units × read_rate
             + billable_output_units × output_rate
             + other_billable_units × their_rates
```

All rates must use the same denominator, such as price per million tokens. Some providers report totals that include cached or reasoning subsets. Convert those fields into the provider's billable categories before adding them. An output total that already includes reasoning must not have reasoning added again. The current OpenTelemetry convention treats billed counts as authoritative where available; Langfuse documents overlapping-bucket errors. [S08] [S07]

Prompt caching is distinct from response caching. The former reuses processing of a matching input prefix while still generating a new answer. The latter reuses a previous answer, requiring a separate validity policy. [S10]

## 4. Forecast workloads, not just monthly invoices

**Proposed forecasting model:**

```text
monthly_cost = fixed_operating_cost
             + Σ(workload_task_volume × expected_variable_cost_per_task)
             + scheduled_change_or_experiment_cost
```

Estimate expected task cost from the observed distribution of model calls, lengths, retries, tool calls and human handling. Include unsuccessful tasks in spend even when they contribute nothing to the successful-outcome denominator.

Create low/base/high scenarios across independent drivers:

- Adoption: eligible users, active users and tasks per active user.
- Workload mix: simple versus difficult tasks and model escalation rates.
- Execution: context growth, output/reasoning length, retries and branching.
- Commercial terms: provider rates, region, service tier and discounts.
- Capacity: replica count, minimum uptime, peak demand and redundancy.

FinOps forecasting guidance supports collaborative demand assumptions and review of forecast variance; the detailed driver model above is our proposed implementation. [S31]

## 5. Agent consumption can grow faster than task volume

**Author derivation:** suppose each of K turns re-sends an initial prefix of P tokens and adds d new tokens per preceding turn. Without cached billing:

```text
total_input_tokens = K × P + d × K × (K - 1) / 2
```

This is a simplified model, not a universal law of agent cost. Summarisation, retrieval, bounded histories, provider caching and different tool outputs change it. Its management implication is robust: track calls and context growth per task, not merely task count. Budgeted token prices alone cannot bound an unconstrained loop.

## 6. Explain why efficiency may not lower the total bill

```text
total_spend = delivered_volume × unit_cost + fixed_cost
```

If useful volume doubles while unit cost falls by 30%, variable expenditure rises by 40%. That can be an economically good result if business value grows accordingly. Report volume, unit cost and total spend together.

Likewise, improving runtime efficiency on an unchanged, continuously rented fleet does not immediately reduce its invoice. Financial benefit arrives through increased useful capacity, fewer rented GPUs, shorter operating hours, or avoided expansion.

## Presentation implication

For the owned-hardware lifecycle model, the rented-GPU alternative and the inputs still needed for actual provider task-cost comparisons, see 09_provider_eligibility_tokens_and_tco.md [local reference omitted]. Keep cash acquisition and depreciation in separate financial views to avoid double-counting.

Use a single end-to-end task example. Place token costs inside the total-cost picture and show the decision affected by each component. Keep the detailed equations in the appendix.

**Questions for Knauf:** What task counts already exist? Which systems own the outcome data? Are workloads API-backed, subscription-based or running on dedicated capacity? Who can change the resources after an efficiency improvement?

## Primary sources

- [S02 — FinOps Unit Economics](https://www.finops.org/framework/capabilities/unit-economics/): business versus resource units; live guidance.
- [S07 — Langfuse token and cost tracking](https://langfuse.com/docs/observability/features/token-and-cost-tracking): exclusive buckets, supplied versus inferred costs.
- [S08 — OpenTelemetry inference token metrics](https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/gen-ai-token-metrics.md): billed units, counters and token categories.
- [S10 — Claude prompt caching](https://platform.claude.com/docs/en/build-with-claude/prompt-caching): cache mechanics and separate rates.
- [S31 — FinOps Forecasting](https://www.finops.org/framework/capabilities/forecasting/): collaborative planning and variance management.

[S02]: https://www.finops.org/framework/capabilities/unit-economics/
[S07]: https://langfuse.com/docs/observability/features/token-and-cost-tracking
[S08]: https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/gen-ai-token-metrics.md
[S10]: https://platform.claude.com/docs/en/build-with-claude/prompt-caching
[S31]: https://www.finops.org/framework/capabilities/forecasting/