AI Cost Optimization
AI Cost Optimization. Geoffrey Kasenbacher and Tebjan Halm. 29 October 2026 / Merantix AI Campus
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
AI Cost Optimization
Geoffrey Kasenbacher and Tebjan Halm
29 October 2026 / Merantix AI Campus
Who we are
. Geoffrey Kasenbacher. Mathematics PhD ML research. Tebjan Halm. Applied mathematics Real-time GPU systems. 02
Geoffrey
Kasenbacher
Mathematics
PhD ML research
Tebjan Halm
Applied mathematics
Real-time GPU systems
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
02
AI keeps changing. So should the workflow.
Halbach. AI keeps changing. So should the workflow.. Understand. See the whole task. →. Monitor. Track cost + performance. →. Improve. Test, adopt, repeat. Halbach / AI & ML operations as a service. 03
Halbach
AI keeps changing. So should the workflow.
Understand
See the whole task
→
Monitor
Track cost + performance
→
Improve
Test, adopt, repeat
Halbach / AI & ML operations as a service
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
03
Illustrative department budget
Illustrative department budget. A planning example for the complete operating budget. 40%. Talent + upskilling. 30%. Compute + infrastructure. 20%. Data architecture. 10%. Governance + maintenance. Then attribute costs to accepted tasks. Illustrative allocation; not an industry benchmark. 04
Illustrative
department budget
A planning example for the complete operating budget
40%
Talent +
upskilling
30%
Compute +
infrastructure
20%
Data
architecture
10%
Governance +
maintenance
Then attribute costs to accepted tasks
Illustrative allocation; not an industry benchmark
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
04
The cost of the complete task
The cost of the complete task. Cost per accepted task. Total workflow cost. ÷. Accepted tasks. Model work. People + operation. Quality evaluation. 05
The cost of the complete task
Cost per accepted task
Total workflow cost
÷
Accepted tasks
Model work
People + operation
Quality evaluation
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
05
Klarna: cost per completed transaction
Klarna: cost per completed transaction. Klarna / adjusted service & operations cost per completed transaction. Q1 2023. $0.32. Q1 2024. $0.25. Q1 2025. $0.19. $0. Payment transactions, not chats · company-reported; AI contribution not isolated. Source: Klarna · Q1 2025 results, p. 7. 06
Klarna: cost per completed transaction
Klarna / adjusted service & operations cost per completed transaction
Q1 2023
$0.32
Q1 2024
$0.25
Q1 2025
$0.19
$0
Payment transactions, not chats · company-reported; AI contribution not isolated
Source: Klarna · Q1 2025 results, p. 7
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
06
What role does AI play?
What role does AI play?. A tool. Improve an existing process. Invest for task value. Core product value. The experience depends on AI. Invest for differentiation. 07
What role does AI play?
A tool
Improve an existing process
Invest for task value
Core product value
The experience depends on AI
Invest for differentiation
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
07
AI is already affecting business economics
AI is already affecting business economics. Klarna. ~US$59m. Reported assistant savings. 2025 / company estimate. DBS. ~S$1bn. Economic value from analytics + AI/ML. 2025 / 430+ use cases. Different currencies and measures; company-reported, not comparable net ROI. Source: Klarna · 2025 Form 20-F · p. 184. Source: DBS · 2025 Annual Report. 08
AI is already affecting
business economics
Klarna
~US$59m
Reported assistant savings
2025 / company estimate
DBS
~S$1bn
Economic value from analytics +
AI/ML
2025 / 430+ use cases
Different currencies and measures; company-reported, not comparable net ROI
Source: Klarna · 2025 Form 20-F · p. 184
Source: DBS · 2025 Annual Report
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
08
A quality failure can trigger repayment
A quality failure can trigger repayment. Deloitte / Australian employment department. ~A$98k. Repayment requested. AI-generated citation errors. Quality and disclosure failure. 17 September 2025 brief / A$97,587.11 incl. GST requested, not confirmed repaid. Source: DEWR · ministerial brief · page 108. 09
A quality failure
can trigger repayment
Deloitte / Australian employment department
~A$98k
Repayment requested
AI-generated
citation errors
Quality and disclosure failure
17 September 2025 brief / A$97,587.11 incl. GST requested, not confirmed repaid
Source: DEWR · ministerial brief · page 108
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
09
Two goals for optimization
Two goals for optimization. Same useful result. Lower complete cost. Less repeated work. New possibilities. Meet a new requirement. Klaus interaction · 8K video. 10
Two goals for optimization
Same useful result
Lower complete cost
Less repeated work
New possibilities
Meet a new requirement
Klaus interaction · 8K video
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
10
Meet Klaus
Meet Klaus. Duplex AI interaction. Listen. ↔. Respond. AI reconstruction of Klaus Halbach. Portrait: Prok4477 / CC BY-SA 4.0. 11
Meet Klaus
Duplex AI interaction
Listen
↔
Respond
Live demonstration during the session
AI reconstruction of Klaus Halbach
Portrait: Prok4477 / CC BY-SA 4.0
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
11
8K generation on a 16 GB GPU
8K generation on a 16 GB GPU. Offline generation · 8,192 × 4,096 output · 1080p playback copy. 12
8K generation on a 16 GB GPU
Video demonstration shown during the session
Offline generation · 8,192 × 4,096 output · 1080p playback copy
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
12
Everyday optimization levers
Everyday optimization levers. Cost per accepted task. Total workflow cost. ÷. Accepted tasks. Model work. People + operation. Quality evaluation. 13
Everyday optimization levers
Cost per accepted task
Total workflow cost
÷
Accepted tasks
Model work
People + operation
Quality evaluation
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
13
Caching cut API cost by about 48%
Caching cut API cost by about 48%. USD per claim · synthetic insurance task · Opus 5 (high effort). Without cache. $0.2906. With cache. $0.1505. 10/10 correct in each run. 10 synthetic claims · cached bar = mean of two runs. Source: Anthropic cookbook · 9 August 2026 · model API cost only. 14
Caching cut API cost
by about 48%
USD per claim · synthetic insurance task · Opus 5 (high effort)
Without cache
$0.2906
With cache
$0.1505
10/10 correct in each run
10 synthetic claims · cached bar = mean of two runs
Source: Anthropic cookbook · 9 August 2026 · model API cost only
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
14
Same context: $0.20 instead of $4.00
Same context: $0.20 instead of $4.00. Shared instructions + stable context. Task A. Shared instructions + stable context. Task B. Shared instructions + stable context. Task C. Reusable prefix. Task-specific suffix. Opus 5.5 cache reads: $0.20 vs $4.00 / million qualifying input tokens. Source: Claude API docs · checked 28 October 2026 · diagram is schematic. 15
Same context: $0.20
instead of $4.00
Shared instructions + stable context
Task A
Shared instructions + stable context
Task B
Shared instructions + stable context
Task C
Reusable prefix
Task-specific suffix
Opus 5.5 cache reads: $0.20 vs $4.00 / million qualifying input tokens
Source: Claude API docs · checked 28 October 2026 · diagram is schematic
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
15
Best practices for model use
Best practices for model use. A suitable model. Route simple tasks when quality holds. Reuse shared context. Cache stable instructions and context. Group non-urgent work. Combine and process later when deadlines allow. Keep the evaluation fixed and measure the complete task after each change. 16
Best practices for model use
A suitable model
Route simple tasks when quality holds
Reuse shared context
Cache stable instructions and context
Group non-urgent work
Combine and process later when deadlines
allow
Keep the evaluation fixed and measure the complete task after each change
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
16
Choose how much of the stack to operate
Choose how much of the stack to operate. Managed API. Buy a service. Rented server. Operate the model. Owned hardware. Operate the infrastructure. More control brings more operating responsibility. 17
Choose how much of
the stack to operate
Managed API
Buy a service
Rented server
Operate the model
Owned hardware
Operate the
infrastructure
More control brings more operating responsibility
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
17
Operate it when control is worth the responsibility
Operate it when control is worth the responsibility. Why control?. Data boundary · product behavior. What capacity?. Useful demand · existing hardware. Who operates it?. People · energy · maintenance. Compare the full cost at the same quality and service level. 18
Operate it when control
is worth the responsibility
Why control?
Data boundary · product behavior
What capacity?
Useful demand · existing hardware
Who operates it?
People · energy · maintenance
Compare the full cost at the same quality and service level
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
18
Evaluate, measure, improve
Evaluate, measure, improve. Evaluate. Can AI do useful work?. Baseline. Define KPIs; count all costs. Improve. Test one change. Scale what works. Revise what does not. 19
Evaluate, measure, improve
Evaluate
Can AI do useful work?
Baseline
Define KPIs; count all
costs
Improve
Test one change
Scale what works
Revise what does not
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
19
Lower technical prices create new options
Lower technical prices create new options. Token price. −20%. Opus 5 → 5.5 Input + output unit prices. September 2026. GPU-hour price. −44%. AWS P5 / H100 On-Demand pricing. June 2025 price cut. Benchmark task cost. −47%. Per quarter at fixed scores Across five benchmarks. 2026 study / historical rate. Source: Anthropic · Opus 5.5. Source: AWS · P5 price update. Source: Epoch · Sep 2026. 20
Lower technical prices
create new options
Token price
−20%
Opus 5 → 5.5
Input + output unit prices
September 2026
GPU-hour price
−44%
AWS P5 / H100
On-Demand pricing
June 2025 price cut
Benchmark task cost
−47%
Per quarter at fixed scores
Across five benchmarks
2026 study / historical rate
Source: Anthropic · Opus 5.5
Source: AWS · P5 price update
Source: Epoch · Sep 2026
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
20
Slides and sources
Slides and sources. Questions. halbach.ai/ cost-optimization. Slides and references
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
Slides
and sources
Questions
halbach.ai/
cost-optimization
Slides and references
The minimum measurement record
The minimum measurement record. Identity. Workflow, task, owner, period. Consumption. Model, tools, retries, compute. Outcome. Accepted, rejected, review time. Cost basis. Invoice charges and allocated costs. A1
The minimum measurement record
Identity
Workflow, task, owner, period
Consumption
Model, tools, retries, compute
Outcome
Accepted, rejected, review time
Cost basis
Invoice charges and allocated costs
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
A1
Dedicated capacity needs enough eligible work
Dedicated capacity needs enough eligible work. Fully loaded capacity cost. Quality-qualified results delivered. Include idle time, redundancy, evaluation and ongoing operations. A2
Dedicated capacity needs enough eligible work
Fully loaded capacity cost
Quality-qualified results delivered
Include idle time, redundancy, evaluation and ongoing operations
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
A2
Provider eligibility before price
Provider eligibility before price. Data classification. What may leave the boundary?. Processing conditions. Residency, retention and agreements. Approved route. Access, tools and audit trail. Response policy. Owner, limits and escalation. A3
Provider eligibility before price
Data classification
What may leave the boundary?
Processing conditions
Residency, retention and agreements
Approved route
Access, tools and audit trail
Response policy
Owner, limits and escalation
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
A3
The illustrative cost model
The illustrative cost model. 980 accepted. Review: 20% / 30% / 15%, two minutes each, valued at €60/hour.. A4 Per 1,000 attempts, Baseline, Cheaper model, Improved workflow. Model / tools, €40, €10, €20. Review capacity value, €400, €600, €300. Platform / runtime, €10, €10, €15. Total attributed cost, €450, €620, €335. Per accepted record, €0.459, €0.633, €0.342
The illustrative cost model
980 accepted. Review: 20% / 30% / 15%, two minutes each, valued at €60/hour.
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
A4
Per 1,000 attempts
Baseline
Cheaper model
Improved workflow
Model / tools
€40
€10
€20
Review capacity value
€400
€600
€300
Platform / runtime
€10
€10
€15
Total attributed cost
€450
€620
€335
Per accepted record
€0.459
€0.633
€0.342
Four times the pixels
Twice the dimensions. Four times the pixels.. 1920 × 1080. 3840 × 2160. 4×. Pixel count is exact. Memory and runtime depend on the pipeline.. A5
Twice the dimensions.
Four times the pixels.
1920 × 1080
3840 × 2160
4×
Pixel count is exact. Memory and runtime depend on the pipeline.
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
A5
Specialization can change inference economics
Specialization can change inference economics. Pinterest / reported relative inference transaction cost. Closed-model comparator. 100. Post-trained open models. <8. 0. <8 is an upper bound · full training and engineering cost not disclosed. Source: NVIDIA · Pinterest case study · accessed 27 October 2026. A6
Specialization can change
inference economics
Pinterest / reported relative inference transaction cost
Closed-model
comparator
100
Post-trained
open models
<8
0
<8 is an upper bound · full training and engineering cost not disclosed
Source: NVIDIA · Pinterest case study · accessed 27 October 2026
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
A6
AI is an investment decision
AI is an investment decision. Holcim / building materials. ~CHF 20m. Annual growth investment. CHF 200m. Recurring EBIT target by 2028. Company target: savings and cost avoidance; not realised benefits. Source: Holcim · Q1 2026 results · AI programme. A7
AI is an investment decision
Holcim / building materials
~CHF 20m
Annual growth investment
CHF 200m
Recurring EBIT target
by 2028
Company target: savings and cost avoidance; not realised benefits
Source: Holcim · Q1 2026 results · AI programme
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
A7
Financial benefits are uneven
Financial benefits are uneven. CEOs reporting AI outcomes over the previous 12 months. Neither revenue gain nor cost reduction. 56%. Both benefits. 12%. 0. Selected categories, not an exhaustive split · total survey n = 4,454. Source: PwC · 29th Global CEO Survey · January 2026. A8
Financial benefits are uneven
CEOs reporting AI outcomes over the previous 12 months
Neither revenue gain
nor cost reduction
56%
Both benefits
12%
0
Selected categories, not an exhaustive split · total survey n = 4,454
Source: PwC · 29th Global CEO Survey · January 2026
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
A8
AI leaders redesign workflows
AI leaders redesign workflows. Respondents fundamentally redesigning workflows. AI high performers. ~75%. Other respondents. ~25%. 0. Approximate shares · survey association, not a causal result. Source: McKinsey · State of AI, August 2026 · total survey n = 1,719. A9
AI leaders redesign workflows
Respondents fundamentally redesigning workflows
AI high
performers
~75%
Other
respondents
~25%
0
Approximate shares · survey association, not a causal result
Source: McKinsey · State of AI, August 2026 · total survey n = 1,719
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
A9
Selective caching reduced peak memory in a retained test
Memory headroom creates operating options. Separate avatar-generation test / sampled peak GPU memory. Full cache. 77.7 GiB. Selective cache. 48.4 GiB. 0. One A100. Fixed input. 129 frames at 768 × 512. Two warm calls per mode.. A10
Memory headroom
creates operating options
Separate avatar-generation test / sampled peak GPU memory
Full cache
77.7 GiB
Selective cache
48.4 GiB
0
One A100. Fixed input. 129 frames at 768 × 512. Two warm calls per mode.
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
A10
Planning scenarios, not forecasts
Planning scenarios, not forecasts. Fixed-capability price index / 2026 = 100. 0. 50. 100. 2026. 2027. 2028. Flat 100. −50% / year 25. Historical rate 0.62. Historical rate extrapolated mechanically; no forecast of future prices or total spend. Source for historical rate: Epoch AI · 22 September 2026. A11
Planning scenarios, not forecasts
Fixed-capability price index / 2026 = 100
0
50
100
2026
2027
2028
Flat
100
−50% / year
25
Historical rate
0.62
Historical rate extrapolated mechanically; no forecast of future prices or total spend
Source for historical rate: Epoch AI · 22 September 2026
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
A11
Compare the model and how it is served
Compare the model and how it is served. Model. Can it do the task?. Quality and useful response time. Provider. Under what conditions?. Price, data terms, region and runtime configuration. The same model can behave differently across serving configurations. A12
Compare the model
and how it is served
Model
Can it do the task?
Quality and useful response time
Provider
Under what conditions?
Price, data terms, region
and runtime configuration
The same model can behave differently across serving configurations
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
A12
Take control where the requirements justify it
Take control where the requirements justify it. Data boundary. The workload must stay inside it. Differentiation. Control matters to the product. Capacity. Existing hardware can do useful work. Own the capability you need; owning all the hardware is optional. A13
Take control where
the requirements justify it
Data boundary
The workload must stay inside it
Differentiation
Control matters to the product
Capacity
Existing hardware can do useful work
Own the capability you need; owning all the hardware is optional
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
A13
Acceptance keeps the KPI honest
Acceptance keeps the KPI honest. Cost per accepted task. Total workflow cost. ÷. Accepted tasks. Model work. People + operation. Quality evaluation. A14
Acceptance keeps the KPI honest
Cost per accepted task
Total workflow cost
÷
Accepted tasks
Model work
People + operation
Quality evaluation
Halbach wordmark in paper — working artwork
Halbach wordmark in black — working artwork
A14
Back
Next
Sources
Appendix
Dark mode
Fullscreen
Close recording
Recorded demonstration
Arrows / Space to advance
·
N sources
·
T theme
·
A appendix
·
F fullscreen
Close sources