The Cursor incident is the clearest case study in what happens when you price a flat plan on top of a cost structure that isn't flat. A developer posted a $7,225 invoice from a single day of usage. It got 797,000 views. The CEO apologized and issued refunds. Nobody on that team was being reckless — they just hadn't accounted for the variance in output token counts, which is where your budget assumptions go to die.
Most teams building AI features have the input side roughly under control. You know your system prompt length. You can count your RAG chunks. You can estimate conversation history. But output tokens? Those are determined at inference time, by the model, on every single request. And they cost more — often several times more per token than input. That asymmetry is where surprise bills come from.
The Compounding Problem Nobody Draws on the Whiteboard
Here's the mechanics that make output variance dangerous in agentic systems specifically.
An agent doesn't make one call. It loops. And every turn re-sends the entire conversation as input — including all previous tool results and reasoning. A six-step investigation with 3,000 tokens of genuinely unique content bills you for roughly 30,000 input tokens, because turn six carries turns one through five on its back. That's before you account for output variance.
Now layer in the output side. A model that decides to reason verbosely — or gets stuck in a retry loop — generates far more output tokens than your estimate assumed. Anthropic measured agents using roughly 4x the tokens of a chat session, and multi-agent systems roughly 15x, with token usage alone explaining roughly 80% of cost variance. The same task can cost wildly different amounts across runs.
The recursive loop failure mode is the worst version of this. An agent calls a tool, gets a weak result, feeds its own output back in, and calls the tool again. And again. Every functional dashboard shows success — the spend shows up only on the bill. Provider guardrails don't catch it: OpenAI project budgets are alerts, not hard caps.
Measure Before You Optimize
The instinct when bills spike is to reach for a cheaper model. That's usually the wrong first move.
Datadog's 2026 State of AI Engineering report found that token usage per request more than doubled for median customers year over year, and quadrupled for 90th-percentile power users. That's not a model pricing problem — that's a measurement and attribution problem. You can't control what you can't see at the right granularity.
The four-layer framework that actually works: track costs accurately at the token level, attribute them to users and features, set alerts on thresholds, then optimize. Most surprise bills come from a short list of tracking pitfalls: the same call recorded twice through a gateway and an SDK, overlapping token buckets that double-count cached tokens, model names that fail to match a price entry, and tiered pricing computed at a flat rate. Fix measurement first. Every downstream decision depends on it.
On the instrumentation side, most providers expose a token-counting endpoint you can wire into your test suite. Anthropic's is POST /v1/messages/count_tokens — it prices a request without running it. Use it to assert on your static prefix size in CI. Tool schemas creep. System prompts grow. A test that fails the build when your static prefix bloats catches the problem before it hits production billing.
For attribution, cost needs to live at the unit-economics level — tagged to the feature, environment, and outcome — not buried in a monthly invoice. "What did this run cost, and was it worth it?" should be answerable with data, not a shrug and a spreadsheet.
The Three Levers That Actually Move the Bill
Once you can see costs clearly, the optimization order matters. The three levers that actually move spend are: cache the part of the prompt that never changes, stop pouring raw tool output into context, and route cheap work to a cheap model. Everything else is rounding error.
Prompt caching cuts input costs significantly on repeated context — most providers offer cached-input discounts, but they only help if your workload can actually use them. Tool output truncation is often the highest-leverage change: if your agent is dumping raw API responses into context, you're paying to re-send that noise on every subsequent turn. Summarize tool results before they enter the context window.
Model routing is the third lever, and the key insight is that the model should never pick its own tier. A deterministic router — classify the task type, check severity, assign the model — keeps cheap work cheap without sacrificing quality on the calls that actually need it.
The broader point: output token variance is a runtime governance problem, not a pricing negotiation problem. Budget assumptions built on average output length will be wrong on the requests that matter most — the long-running agents, the retry loops, the edge cases that your p50 estimate never captured. Instrument at the span level, attribute to features, and enforce limits from a control loop that sits outside the agent. The bill is the last place you want to learn about a runaway loop.
