Connect operational telemetry to invoices and business outcomes
Recommendation: Maintain three linked records: request/task telemetry for diagnosis, provider/resource expenditure for financial reconciliation, and business outcomes for value assessment. No single record substitutes for the other two.
1. Use a shared workload identity through the pipeline
Proposed data architecture:
Business task record
↳ task ID, workload ID, owner, outcome, review effort
Request/component records
↳ task ID, provider request ID, model/version, usage, latency, retries
Financial records
↳ account/project, period, contract/rate, invoiced amount, resource spend
Propagate IDs through orchestration, gateways, retrieval, model serving and human review. Gateway architecture can check requests and record model consumption, but task identity must originate in the application. LiteLLM documents budget checks before requests and spend logging in asynchronous post-response tasks. This matters for concurrency and stale accounting; it does not establish a strict financial ceiling or business attribution by itself. S06
2. Capture the minimum useful fields
| Level | Fields | Purpose |
|---|---|---|
| Attribution | Workload, team/cost centre, environment, application, task/trace ID | Ownership and allocation |
| Execution | Requested and actual model/version, provider/platform, region, service tier | Reproduce price and performance decisions |
| Usage | Billable input/output, cache writes/reads, reasoning subset, modality units | Calculate direct variable consumption |
| Behaviour | Attempts, retries, fallbacks, agent steps, tool calls, cache reuse | Explain task-cost variation |
| Service | Queue time, time to first token, output-token interval, end-to-end latency, failures | Identify useful capacity under service constraints |
| Outcome | Accepted/rejected, business task completed, human review and rework | Evaluate quality and cost per outcome |
| Accounting | Price definition/version, estimated versus reported cost, invoice period/currency | Reconciliation and auditability |
This is a proposed schema. Do not assume all providers supply all fields, particularly actual model identity, detailed reasoning usage or per-request prices.
3. Avoid subtle token-accounting errors
The retrieved OpenTelemetry GenAI specification distinguishes usage counters from per-operation histograms. It recommends counters for total usage; histograms describe distributions. Billed counts take precedence when providers supply both used and billable counts. Cached input and reasoning output can be included in their respective totals. S08
Practical consequences:
- Do not add a subset to its already-inclusive total.
- Do not add p95 values across components or modalities to estimate total p95.
- Keep exact provider usage where supplied; flag tokenizer estimates.
- Account for failures and retries at the attempt level, but aggregate value at the task level.
- Separate operational token categories from the exclusive price categories used for costing.
Illustrative arithmetic: a provider reports 10,000 input tokens including 8,000 cached tokens. If the uncached rate is r and the cache-read rate is c, cost is 2,000r + 8,000c, not 10,000r + 8,000c.
Langfuse gives precedence to ingested values over inferred values, allows custom price definitions, and notes that price-definition changes apply to subsequent generations. Its estimated totals are useful telemetry; they are not automatically the contracted invoice amount. S07
4. Reconcile the operational and financial views
FOCUS 1.4, released 10 June 2026, adds invoice-related datasets and rules for financial reconciliation and cost-data integrity. Provider exports can still implement older specification versions; the presence of a standard does not imply every supplier exposes the same fields. S04
Proposed reconciliation process:
- Aggregate request estimates by provider account, model/service, period and currency.
- Compare with billed cost and explain differences: negotiated rates, credits, commitments, taxes, rounding, late charges and missing telemetry.
- Preserve consumption at a consistent rate basis for engineering comparisons.
- Report actual payable cost separately, including credit effects.
- Record corrections and the reconciliation status instead of silently overwriting history.
Cloud credits can make the invoice temporarily small while the underlying consumption is economically expensive. Maintain both views to avoid designing a system around a subsidy that will expire.
5. Monitor rented GPU systems beyond GPU utilisation
Recommended metrics: successful requests per paid GPU-hour, useful tokens per paid GPU-hour, request/queue latency distributions, resident model and KV-cache memory, cache reuse, preemption/recomputation, failures, replica-hours and idle capacity.
High GPU utilisation is not proof of useful efficiency: repeated work or an overloaded queue can drive it upward. vLLM documents that KV-cache shortages cause preemption and recomputation, and that concurrency and parallelism changes have latency and synchronisation trade-offs. S13
For vision or speech models, retain native units such as images processed or audio duration, alongside accepted business outcomes. A token-only dashboard would miss parts of a multimodal pipeline.
6. Design dashboards around decisions
| Viewer | Main view | Decision enabled |
|---|---|---|
| Leadership | Workload spend, forecast range, accepted outcomes, quality, benefit evidence | Fund, scale, change or retire |
| Business owner | Volume, adoption, cost/task, review effort and exception rate | Improve workflow and demand |
| Engineering | Stage-level latency, calls/task, context length, cache reuse, resource efficiency | Diagnose and optimise |
| Finance | Payable cost, commitments, credits, allocations and reconciliation | Validate and plan expenditure |
Report weighted totals: aggregate cost per successful task is total relevant cost divided by total successful tasks, not the unweighted mean of each team’s ratio. Compare like workloads; a changing task mix can otherwise make an efficient system appear worse.
7. Keep cost accounting complete without tracing everything
Engineering recommendation: Retain complete low-volume usage/accounting records while sampling expensive diagnostic detail. Sampled distributed traces alone cannot reconstruct exact expenditure. Link diagnostic records to the complete ledger and explicitly identify missing coverage.
Cost attribution rarely requires full prompt bodies. Use metadata-first collection, controlled access and a defined retention policy for sensitive content. This is an architectural recommendation, not a legal compliance opinion.
Implementation appendix: source usage from the actual interface
Capture raw response usage before normalising it. OpenAI Responses exposes input/output totals plus cache and reasoning details; Anthropic Messages separates uncached input, cache creation and cache reads; Bedrock Converse exposes input/output and optional cache counters/details. Similar names do not imply identical inclusivity or pricing semantics. S36 S10 S37
Anthropic documents total input as input_tokens + cache_creation_input_tokens + cache_read_input_tokens. For an interface that instead includes cached or reasoning subsets in its totals, do not add those subsets again. Capture streaming usage with the provider-specific event schema; mark incomplete usage explicitly. Local tokenisation remains a labelled estimate. No provider adapter or reconciliation pipeline has been implemented or validated as part of this research pack.
OpenAI documents organisation-level Usage and Costs APIs for financial aggregation; these do not replace business-task IDs. S38 The detailed field table, adapter requirements and reconciliation procedure are in 09_provider_eligibility_tokens_and_tco.md [local reference omitted].
Presentation implication
Extend the ledger to coding-tool telemetry without mistaking it for billing
Navigara documents OTLP ingestion from coding tools and proxies. S43 Use this as an integration example, not a validated adapter. Add objective/operating mandate to owner, workload and task IDs; report unexplained linkage separately from proven waste. Prefer team/workload views for management decisions.
Anthropic labels Claude Code cost telemetry approximate and documents streaming overcounts before v2.1.214. S47 Record collector/client versions, check streaming behaviour, deduplicate overlapping client/proxy records, and reconcile with provider charges. Keep provisioned seats, entitlement and incremental overage distinct from API-equivalent usage valuations. The native billing counters and invoice remain the authority for their respective consumption and financial records.
See chapter 10 [local reference omitted] for this extension and its evidence limits.
Show a request moving from a workload owner to its components, then to a completed business task and its financial record. The leadership message is simple: monitoring becomes useful when someone can connect a cost change to a cause and a decision.