Every chapter before this one produces a number with a denominator attached: dollars per GPU-hour, dollars per million tokens, dollars per megawatt-year. Those denominators are useful for comparing two options. None of them survives a budget conversation, because none of them is a thing the business sells.
The move from cost to unit economics is the move to a denominator someone outside engineering already thinks in. It sounds like a formatting exercise. It is not. The denominator you choose determines which optimisations look worthwhile, and the industry publishes at least three competing ones for the same underlying spend.
Three layers, three denominators
The same dollar of inference cost is described three different ways depending on who is filing.
At the silicon layer it is cost per token. NVIDIA's FY2026 annual report describes the Rubin platform as delivering "up to a 10x reduction in cost per token compared to Blackwell", and describes Blackwell Ultra as delivering a reduction in cost per token relative to Hopper. Note what kind of claim that is: a forward-looking vendor statement about a part that had not yet shipped in volume at filing. Cost per token is the denominator in which accelerators are sold.
At the hyperscaler layer it becomes margin. Microsoft's FY2026 annual report puts it in the risk factors, not the marketing: the cost structure for AI products and services is subject to significant uncertainty "with respect to model training and inference costs", and if those costs stay elevated while AI pricing declines through competition or commoditisation, margins and results "could be adversely affected". The denominator has changed from tokens to gross margin, because that is what a shareholder prices.
At the application layer it disappears into an accounting line. Doximity's FY2026 annual report defines cost of revenue as cloud hosting, customer-success staff, third-party platform access, information technology, and "software costs, including generative AI platform usage and inference costs", plus amortisation and allocated overhead. Inference is in there. It is not separable from the outside, and possibly not from the inside either.
That progression is the whole problem in three sentences. The layer that reports cost per token does not pay your bill, and the layer that pays your bill does not report cost per token.
What the application layer's numbers actually show
Doximity is a useful worked example precisely because it is not an AI company. It is a profitable SaaS business that added AI to an existing product, which is the situation most readers are in.
From the FY2026 income statement, all figures as filed, in thousands:
| Fiscal year ended 31 March | 2024 | 2025 | 2026 |
|---|---|---|---|
| Revenue | $475,422 | $570,399 | $644,863 |
| Cost of revenue | $50,669 | $55,874 | $70,326 |
| Gross profit | $424,753 | $514,525 | $574,537 |
| Gross margin (computed) | 89.34% | 90.20% | 89.09% |
Two things fall out of that table that the table does not say.
Cost of revenue grew 25.87% while revenue grew 13.05% — a ratio of 1.98, almost exactly two to one. In FY2025 the same comparison was 10.27% against 19.98%: costs grew at roughly half the rate of revenue. The relationship inverted.
Gross margin gave back 1.11 points, from 90.20% to 89.09%, landing below where it started in FY2024. Expressed as a unit metric a CFO would recognise: the business earned $10.21 of revenue per dollar of cost of revenue in FY2025 and $9.17 in FY2026.
Now the honest part, and it is the reason this chapter exists rather than a chart. None of that is attributable to inference. Cost of revenue on that line also contains cloud hosting, headcount, third-party platform access, amortisation, and overhead, and any of them could have driven the increase. What the filing establishes is narrower and still useful: this is the line inference lives in, this is the direction it moved, and no external reader can decompose it. If it is not decomposable from outside, the question is whether it is decomposable from inside — and for most companies, on the first attempt, it is not.
Resource metrics are not business metrics
The FinOps Foundation framework makes a distinction worth stealing, because it names the failure directly. Among the activities under its Unit Economics capability is to "distinguish resource efficiency unit metrics from business unit metrics, and clarify how they relate."
Cost per token, cost per GPU-hour, and cost per request are resource efficiency metrics. They tell you whether the machine is being used well. Cost per closed ticket, per document processed, per active customer, per completed booking are business metrics. They tell you whether the machine is worth running.
The two diverge in a specific and expensive way: a resource metric can improve while the business metric gets worse. Halve your cost per token by moving to a cheaper model, lose accuracy, and now three agent turns are needed where one used to do — cost per token down, cost per resolved case up. The same inversion is why FLOPs, MFU, and the utilisation trap exists as a chapter: a busy accelerator and a productive one are different claims, and only the business denominator can tell them apart.
A resource metric that is not paired with a business metric is a way to be efficiently wrong.
Choosing the denominator
Four rules, in order of how often they are broken.
- Pick the denominator the business already counts. If revenue is reported per seat, the unit is per seat. Do not invent "cost per inference" for a company that has never once discussed inferences. A metric nobody already tracks needs a champion, and champions leave.
- The numerator must include everything. Tokens plus serving plus retrieval plus storage plus egress plus the human review step. A per-unit cost that counts only the model call is the number that makes an AI feature look free until the invoice arrives — see Egress, storage, and the bill around the GPU.
- Carry both metrics, always. One resource metric to detect waste, one business metric to detect whether the waste mattered. Reporting either alone produces a predictable class of confident mistake.
- Fix the denominator before you optimise. A unit cost that improves because the definition changed is the most common false win in this discipline, and it is nearly undetectable a quarter later.
The honest limit of this chapter
The three filings above establish how the industry describes AI cost at three layers. They do not establish anyone's actual cost per token, per query, or per customer, because no company in this comparison discloses one — Doximity does not break inference out of cost of revenue, Microsoft does not break it out of Microsoft Cloud, and NVIDIA's cost-per-token claims are about its own silicon relative to its own previous silicon, not about any buyer's realised cost.
That absence is itself the finding. If three public companies at three layers of this market all decline to publish a unit cost for AI, the reasonable inference is not that they are hiding a good number. It is that the number is genuinely hard to construct, and that a team producing one internally is already ahead of the disclosure practice of the entire industry.