What an H100-hour actually costs

Once you stop buying tokens and start buying compute, the unit of account changes. You are no longer paying per million tokens; you are renting accelerators by the hour, and your cost per token becomes a function of how busy you keep them.

So the first question is what an hour costs. Here is the identical machine — 8× NVIDIA H100 80GB SXM, on-demand, US region, Linux — from published list rates on 2026-08-05:

Same silicon. Same count. Same week. Azure costs 79% more than AWS for it.

That is the finding most teams never see, because almost nobody prices the same workload on three clouds — they price it on the cloud they are already on and negotiate from there.

The full ladder

Add discount mechanisms and the specialist providers, and the range gets wider:

A 4.7× spread on one chip. No amount of serving-stack optimisation beats picking the right row.

Five things this table says that are not obvious

AWS is the cheap hyperscaler, by a lot. The reflex is to treat the big three as roughly interchangeable on price and choose on ecosystem. On H100 capacity they are not close: AWS on-demand undercuts GCP by 38% and Azure by 44%. If you are on Azure and GPU-heavy, that gap is likely the largest single line item you can move.

GCP's three-year commitment is more expensive than Lambda's no-commitment rate. $4.86 committed for three years, against $3.99 walk-up. You would be signing a three-year term to land above a price available this afternoon on a credit card. Any commitment business case has to clear that bar, and this one does not.

AWS spot is the cheapest option on the board — cheaper than every neocloud. $2.62 beats RunPod's $2.99. "Leave the hyperscaler to save money" is not automatically right; if your workload tolerates interruption, the cheapest H100 in this table is on the cloud you are probably already using, with no migration and no new vendor.

AWS three-year reserved ($2.97) and RunPod on-demand ($2.99) are the same price. One of them locks you in for three years. The other does not.

Region matters nearly as much as vendor. Azure's ND96isr_H100_v5 runs $98.32/hr in East US and up to $186.61/hr in its most expensive region — a 1.9× spread inside a single cloud. Before you migrate vendors, check whether you are simply in an expensive region.

The break-even that decides commitments

The reserved-versus-on-demand question has an arithmetic answer, and most teams never compute it. Divide the committed rate by the on-demand rate to get the utilisation at which the commitment pays for itself:

• AWS: $2.97 ÷ $6.88 = 43%
• GCP (3-year CUD): $4.86 ÷ $11.06 = 44%

Both land in the same place, which is useful: keep the accelerator busy more than ~43% of the hours across the whole term and committing is cheaper. Below that, you are pre-paying for idle.

The question to test is not "will we still be doing AI in three years." It is "will this specific accelerator, at this quantity, be busy 43% of the time in year three" — in a market where the hardware generation turns over in roughly two. Those are very different questions and only the second one matters.

Run the division on your own quotes. One line of arithmetic converts a strategic-sounding argument into a utilisation forecast you can actually check.

What the cheap number leaves out

Everything above is list rate for the accelerator. A neocloud can be a third of Azure's price mostly because it is selling less, and some of what it omits will matter:

• Reliability and replacement. A failed node on a hyperscaler is an API call. On a smaller provider it can be a support ticket and a wait. An annoyance for checkpointed training; an availability problem for production inference.
• Egress. Lambda publishes no egress fees. All three hyperscalers charge for data leaving, and that line is not in the GPU rate.
• Capacity guarantees. "On-demand" at a neocloud can mean first-come, first-served. Community-tier capacity is someone else's spare machine, priced accordingly.
• Compliance posture. Specific certifications, data residency, or a signed BAA shorten the eligible list fast, and the comparison changes with it.
• The rest of the bill. Storage, networking, and the engineering time to run your own serving stack are not in the hourly rate. Teams moving off a token API to save on compute often discover they traded a variable bill for a fixed one plus an on-call rotation.

None of this means the expensive option is correct. It means the honest comparison is total cost of the workload — and a 4.7× gap is wide enough that it usually survives an honest accounting, which is exactly why the accounting is worth doing.

Where the rest of the fleet sits

Published rates the same day, per GPU/hour:

Two things. First, the A100 spread between two neoclouds is 87% — $1.49 against $2.79 for the same part. Provider choice matters more than most architectural decisions teams agonise over.

Second, the ladder from A6000 to B300 spans roughly 14×, and the top of it is frequently the wrong answer. The right accelerator is the cheapest one whose memory holds your model and whose bandwidth meets your latency target — a question about your workload, not about which chip is newest. That is the silicon layer, Part 3.