Every chapter in this book measures one thing. This chapter puts those measurements side by side and ranks them, because the ordering is not what most teams assume — and the gap between the top and the bottom of the list is roughly two orders of magnitude.
Every figure below is carried from a chapter that sources it primarily. Nothing here is new data; the value is the ordering.
The ranking
| # | Lever | Measured | Effort | Where |
|---|---|---|---|---|
| 1 | Serving stack — naive loop → paged attention | up to 24× | Deployment change | Continuous batching |
| 2 | Managed endpoint vendor — same open weights | 6.7× | Base-URL change | Managed endpoints |
| 3 | Where the GPU runs — full stack, compute + egress | 4.5× | Migration | Hyperscaler vs neocloud |
| 4 | Accelerator hour — same H100, different logo | 4.7× | Vendor switch | What an H100-hour costs |
| 5 | Prompt caching — cache read vs input | 10× on cached prefix | Config + prompt order | What you pay for |
| 6 | Output volume — output vs input rate | 5× per token saved | Prompt engineering | The output premium |
| 7 | Chip choice — H100 → H200 | 1.9× | Instance type | Memory bandwidth |
| 8 | Batch mode — async vs sync | 2× | Config change | The output premium |
| 9 | Egress placement — GCP vs AWS first tier | 1.33× on transfer | Vendor/architecture | The bill around the GPU |
Four things this ordering says
The biggest lever is a deployment change, not a negotiation. Serving a model through a naive transformers loop instead of a paged-attention stack costs up to 24× — the largest single multiple measured anywhere in this book. It requires no vendor conversation, no commitment, and no architecture rewrite. A surprising number of internal deployments are still on the naive path.
Ranks 2, 3 and 4 are all "which vendor," and they compound. 6.7× on endpoints, 4.5× on hosting, 4.7× on the raw hour. These are not independent — they are the same question asked at three layers — but the point stands: vendor selection dominates this list, and it is the work almost nobody does, because pricing one workload across four vendors is tedious and pricing it on the incumbent is easy.
The two levers teams reach for first are ranks 5 and 6. Prompt caching and prompt trimming are real — 10× on a cached prefix is excellent, and it is a config change. But they operate inside a vendor choice that may already be costing you 4–6×. Optimising the prompt on the wrong endpoint is polishing a rounding error.
Rank 7 is where the attention goes and it is seventh. Chip selection — H100 versus H200, which generation, how many — attracts more discussion than anything else on this list and is worth 1.9×. Real, and eighth-largest.
The lever that is not on this list
Switching to a cheaper model tier is absent, deliberately. The output premium shows why: Anthropic's ladder is a pure scalar — output is exactly 5× input at every tier, from Haiku 4.5 to Fable 5 across a 10× price range. Downgrading scales your entire bill and changes nothing about its shape.
That makes it a discount, not a lever. It is the single most common first response to a high AI bill, and it is the one action on this page that provably cannot rebalance anything. If the model is good enough at the cheaper tier, use it — but do not mistake it for optimisation.
How to actually use this
Work top-down, and stop when the effort exceeds the multiple:
- Are you on a paged-attention serving stack? vLLM, SGLang, TensorRT-LLM, or a managed endpoint built on one. If not, that is up to 24× and nothing else matters yet.
- Have you priced your exact workload at four vendors in the last quarter? Not "checked the incumbent's page." Four.
- Is your cached-prefix hit rate non-zero? Cache writes with no reads means an unstable prefix — you pay the 25% write premium and collect none of the 90% discount.
- Is anything asynchronous still running synchronously? That is 2× for a config flag.
- Only then tune prompts, chips and regions.
Most teams run this list backwards, which is why most AI cost work produces disappointing results.
What this ranking is not
These are measured maxima under specific conditions, not guarantees. The 24× is vLLM's published benchmark against HuggingFace transformers on particular models and hardware; if you are already on a modern stack, your remaining headroom there is 2–3.5×, not 24×. The 6.7× is one open-weights model across four vendors on one day. The 4.5× assumes an egress-heavy workload — for pure text inference, rank 9 is worth approximately nothing to you.
The ordering is robust. The magnitudes are situational, and the only way to know yours is to measure — which is what reading your first bill is for, and why it comes before this chapter rather than after it.