# The AI Bill: Itemised — Episode 1

## The model got cheaper. Why didn't the bill?

AI-presented hosts: Alex and Jordan. All case details and prices are fictional teaching assumptions.

## Opening

**[00:00] Alex:** You switch to an AI model at half the price. A month later, the bill is higher. Do you switch back?

**[00:06] Jordan:** Before I do that, tell me what I got for the extra money.

**[00:10] Alex:** Welcome to The AI Bill: Itemised. I'm Alex.

**[00:14] Jordan:** And I'm Jordan.

**[00:16] Alex:** If you use AI to get work done, this show helps you decide what deserves your money. We look at models, tools and the way you use them: what they cost, what they can do, and whether the results are worth paying for.

**[00:30] Jordan:** Today, we're looking at a bill that went up after a price cut. We'll work out what changed, then decide what to check before cutting usage or changing models again. Because a bigger bill could mean more useful work. It could also mean you're paying for work nobody needs.

## The bill on the table

**[00:51] Alex:** We've built a fictional example so we can follow the whole bill. The numbers are invented and the calculation is in the episode notes. This isn't a customer result.

**[01:02] Jordan:** What's the job?

**[01:03] Alex:** Summarising documents for a team. Someone sends in a document and gets back a summary. They're supposed to be able to use that summary in their work. Last month the invoice was one hundred and forty dollars. The team switches to a model whose input and output prices are both half as much. Same amount of work, same length of answer, and you'd expect seventy dollars.

**[01:28] Jordan:** Then the invoice arrives.

**[01:30] Alex:** One hundred and sixty.

**[01:32] Jordan:** Twenty dollars more than before. That's annoying, but I wouldn't change a working product over that alone.

**[01:39] Alex:** Neither would I. The useful question is why the expected saving disappeared. If the team doesn't understand the change, they can't sensibly budget for the next one.

**[01:49] Jordan:** So let's see whether the work stayed the same.

## What changed

**[01:52] Alex:** It didn't. The team processed ten thousand documents before the switch. Afterward, sixteen thousand.

**[01:59] Jordan:** That could be good. More people found a use for it.

**[02:03] Alex:** It could. And it explains part of the bill. But the new summaries also got longer. Twice as long, on average. The documents going in stayed the same size.

**[02:14] Jordan:** Did somebody ask for longer summaries?

**[02:17] Alex:** We haven't established that. We know the length changed. We don't yet know whether people wanted it.

**[02:23] Jordan:** Okay. More documents, and more writing for each document. That explains how a cheaper model could produce a bigger bill. But was each summary actually cheaper?

**[02:34] Alex:** Yes, in this example. Generating a thousand summaries used to cost fourteen dollars. With the new prices and longer answers, it costs ten.

**[02:44] Jordan:** So the unit cost came down. The total went up because the team did more work.

**[02:50] Alex:** And the longer answers used up part of the saving. If you follow the calculation in the notes, the extra documents and extra output explain the whole gap between the expected bill and the actual one. We haven't hidden a retry charge or a fee in there.

**[03:06] Jordan:** Then switching back could cost more for the volume we now have.

**[03:10] Alex:** It could. Looking only at the invoice total might push you toward the wrong decision.

## Cheaper at what?

**[03:16] Jordan:** There's still something I don't like about our comparison. You said a thousand summaries cost less. Are they a thousand useful summaries?

**[03:26] Alex:** We don't know yet.

**[03:27] Jordan:** Because I can imagine both versions of this. Maybe the longer answer includes something people were missing. Or maybe people now have to dig through twice as much text to find the point.

**[03:39] Alex:** And those lead to different decisions. For this team, a useful summary needs to capture the main point and the important caveat, without inventing anything. That's the standard we'd use to judge the result.

**[03:52] Jordan:** Then I'd want someone to look at the summaries against that standard. I wouldn't call the extra output waste just because it's longer.

**[04:00] Alex:** Nor would I call it valuable just because people generated it. The provider can charge us for an answer whether anyone uses it or not.

**[04:09] Jordan:** Can the bill tell us which answers were useful?

**[04:11] Alex:** Usually not by itself. We need some evidence from the work: a review result, a completed task, or feedback from the person using the summary. If all we have is the invoice, usefulness stays unknown.

**[04:26] Jordan:** So we've answered why the money moved. We haven't answered whether it was well spent.

**[04:31] Alex:** Right. Those are separate questions. And we haven't priced human review time either. The dollar figures we've discussed are model charges, not the whole cost of running the product.

## The next decision

**[04:43] Jordan:** My first instinct was to leave it alone because the increase was small. Now I'd keep the model for the moment, but check what the longer answer is doing for us.

**[04:53] Alex:** How would you check?

**[04:55] Jordan:** I'd take a small set of documents that resemble the work people actually send in. Include a few difficult ones. Run those through the current setup, then through a version that asks for a shorter summary. Keep the model and the documents the same.

**[05:11] Alex:** And decide what counts as a good summary before looking at which version costs less.

**[05:16] Jordan:** Yes. Does it capture the point? Does it keep the important caveat? Can someone use it without fixing it? If it drops an important detail, I wouldn't ship it.

**[05:27] Alex:** Count every billed attempt, too. If someone regenerates an answer, include that cost even if they discard it. Some retries succeed; some don't. Either way, they're part of what it took to finish the job.

**[05:40] Jordan:** I'd also note whether the reviewer has to spend longer fixing one version. I might save on the model and create more work for a person.

**[05:50] Alex:** That gives us a useful first test. A small sample can reveal problems; it won't tell us exactly how every production task will behave. If the shorter setup looks promising, try it on a limited share of real work and check again.

**[06:05] Jordan:** And if the longer version is clearly more useful, keep it and budget for it. The point of the check is to make that a decision we can explain.

## Try it on one piece of work

**[06:15] Alex:** You can start this on your own stack without investigating everything at once. Pick one recurring job: answering a support question, writing a document summary, extracting information from a form. Something where you can describe a useful result.

**[06:32] Jordan:** Then pick comparable periods. Don't compare a quiet week with a launch week and assume the model caused the difference. If the work changed, write that down.

**[06:43] Alex:** Look for what you spent, how many original jobs you tried to complete, and how many results met your standard. Count the original job once, even if getting it done took several model calls. Keep all of those calls in the cost.

**[06:58] Jordan:** And if the usage dashboard only gives me requests, not completed jobs?

**[07:03] Alex:** Mark the task count as missing. Ask whoever owns the workflow whether they can connect requests to jobs. You can still investigate the spend; you just can't honestly calculate cost per useful job yet.

**[07:16] Jordan:** That's a better first finding than dividing by the wrong thing.

**[07:20] Alex:** Once you can count useful results, divide the model spend, including discarded calls, by the original jobs that produced a useful result. If none did, report that directly. Write down what you know, what's missing and the next check. You can start there even before you have enough data for a calculation.

## Close

**[07:41] Jordan:** Back to the opening: the model got cheaper, but the bill went up. Do we switch back?

**[07:47] Alex:** Not on that evidence alone. In our example, each summary became cheaper to generate. The team processed more documents, and longer answers consumed some of the saving. The next check is whether those answers are worth the extra output.

**[08:03] Jordan:** That's the question I'd take into a budget meeting: what useful work did we get for this spend, and what would we change next?

**[08:11] Alex:** That's what we'll keep investigating on The AI Bill. When a model, a tool or the way you use it changes, we want you to be able to judge the tradeoff. For this episode, pick one workflow and write down what a useful result looks like. Then check whether your cost data can tell you what it takes to produce one.

**[08:32] Jordan:** Start with one piece of work you actually care about. Thanks for listening.