The Tokens Your Agent Spent Arguing With Itself

Every profession develops its own way of reading trouble before trouble announces itself. Radiologists see shadows. Structural engineers hear resonance. Forensic accountants notice the transaction that doesn't fit the pattern.

Dara Okafor left forensic accounting eighteen months ago to become something that didn't have a job title yet. She's a FinOps analyst at a mid-size company running agent deployments across customer service, internal research, and procurement. Two years ago, 31% of FinOps practitioners managed AI spend. Today, 98% do, but three in four enterprises still can't prove AI outcomes to the CFO.¹ The job emerged faster than the tools, the vocabulary, or the org-chart conventions to support it.

Okafor builds the dashboards that track where inference spending actually goes. More interestingly, she's started reading cost patterns the way a mechanic reads engine sounds: a spike in a workflow that nobody changed means the model is struggling with something new. A steady decline might mean the agent learned a shortcut, or it might mean it stopped trying hard cases. She doesn't think of herself as a finance person. She thinks of herself as someone who notices things.

Dara Okafor is a fictional character assembled from the operational reality of a job that appeared faster than anyone could write a description for it. Her frustrations are real. Her spreadsheets are imaginary. Her opinions are her own, which is easy when you don't exist.

When an inference bill arrives, what are you actually looking at?

Dara: A number. Basically just a number. You get aggregate token consumption across the account, maybe broken down by API key if you're lucky. The provider doesn't know your org chart. They don't know your workflows. They just know you consumed tokens.

So my first six months was building the instrumentation to make the bill legible. We deployed a proxy layer that tags every request with a workflow ID, an agent ID, a run ID. Without that, you're staring at a lump sum. Like getting a credit card statement that just says "food: $4,000." Technically accurate. Operationally useless.

And even then, the token bill is one of maybe nine cost buckets.² The surrounding infrastructure—vector database queries, embedding generation, orchestration compute—can run 40 to 60 percent of total feature spend, and none of it shows up on the model provider's invoice.³ I run two parallel data pipelines and join them in a warehouse. Neither one alone tells you what happened. Together they tell you most of what happened, which is the best I've managed so far.

You've started using the phrase "compensatory tokens" internally. Where did that come from?

Dara: Staring at traces. Thousands of them. You develop opinions.

You look at an agent run and you can see tokens doing the work: reading context, reasoning about which tool to call, generating output. Those are productive. Then you see the other kind. The system arguing with itself. Retrying a failed tool call. Rebuilding context after an error. Re-processing information the model already saw ten steps ago but has effectively forgotten because the context window got too crowded. Practitioners call that context rot.⁴ Or my personal favorite: the model spending 200 tokens elaborating on its own mistakes in the reasoning trace before spending another 200 correcting itself. Very thorough. Very expensive.

I started calling those compensatory tokens because they're compensating for something going wrong. And here's what makes it financially interesting: naive agent loops grow token costs quadratically.⁵ Every step re-bills the entire conversation history. A 20-step loop doesn't cost 20 times a single call. Without prompt caching, that loop sends roughly 730,000 input tokens to produce 12,000 output tokens.⁶ Read that ratio again. That's a 60:1 ratio of reading to writing.

Now add retries. An agent with 8,000 tokens of context hits a rate limit, retries three times naively: 32,000 input tokens to achieve nothing.⁷ In multi-step workflows where each step triggers tool calls, it compounds. And the part that keeps me up at night: tool mistakes are worse than wasted tokens, because tool mistakes do things. A bad API call can file a ticket, change a record, send a message. You can't un-send an email. Wasted tokens are wasted money. Bad tool calls are wasted money that also created a cleanup project for someone who doesn't know it's coming yet.

The ratio between productive and compensatory tokens is my primary diagnostic for workflow health. But no standard tooling logs this distinction natively. I construct it from traces, and it's partly interpretive. I'm making judgment calls about which tokens were load-bearing. It's not clean. It's not automated. It's me, reading traces, developing an opinion. I keep meaning to write a classifier for it. I keep not having time. The cobbler's children, et cetera.

How do you tell the difference between a workflow that's genuinely complex and one that's just unreliable?

Dara: Tool-call failure rate. That's the separator.

Complex workflows have high cost-per-attempt but reasonable cost-per-successful-outcome, because they almost always succeed. Unreliable workflows can have low cost-per-attempt but terrible cost-per-successful-outcome, because they fail 60 percent of the time and you're paying for all the failures.⁸ The FinOps Foundation draws this distinction explicitly: cost per verified outcome, not cost per run.

The counterintuitive thing I've learned to watch for: unusually cheap, unusually smooth completions.

There's a time dimension too. A spike in a workflow that hasn't changed means the model is struggling with something new. A steady cost decline without any declared optimization? Check whether human escalations are rising. If they are, the agent is offloading difficulty, not reducing it. The dashboard looks great. The support queue tells a different story.

Is there a standard job description for what you do?

Dara: No. I wrote my own. HR was confused. They're still confused, honestly, but the CFO stopped asking questions after I saved him from approving a renewal based on per-run costs that didn't account for a 40% failure rate. Context helps.

What happens when you show engineers their cost-per-successful-outcome for the first time?

Dara: One of two reactions. Either the number is way worse than expected, because failed runs were never counted in their unit economics, or the model inference cost looks fine but the harness costs are enormous by comparison. People don't realize how much they're spending on the scaffolding around the model. The model is the celebrity. The scaffolding is the entourage. The entourage is expensive.

There's a political consequence too. Ownership becomes visible. A team that was anonymous behind an aggregate bill now has a number attached to its workflow. That's not always welcome. Nobody minds transparency in the abstract.

The recommendation I end up making over and over is model right-sizing. Engineers selected the best model available; the workflow doesn't require it. A frontier model can cost 50 to 100 times more per token than a smaller one from the same provider.⁹ A routing layer that sends simple requests to cheaper models can cut average cost 60 to 80 percent. But getting engineers to accept that their workflow doesn't need the best model? That's a conversation about pride as much as architecture. I've learned to bring the numbers and then wait. The numbers usually win, but they need a few days.

What do you wish the systems logged that they don't?

Dara: Why a run was expensive. I can see that it was. But "this run cost $1.20 instead of $0.30 because the agent hit a wall on step 7, rebuilt its context three times, and discarded two drafts" requires interpretation that current logging doesn't support without custom work.

I want retry reason codes. Was it a tool error, a model hallucination, a transient API issue, a rate limit? Compensatory tokens from flaky vendor APIs are a vendor problem. Compensatory tokens from model errors are a prompt problem. Right now the log just says "retried." Thanks. Very helpful.

And the big one: downstream human work.

FOCUS 1.5 is supposed to standardize some of the token-level fields across providers by December, which is progress.¹⁰ But it standardizes the token view, not the outcome view. Not the causal view. Not the "what happened after the agent said it was done" view.

What can't you see from the cost data?

Dara: Whether the output is good.

I can tell you a workflow is expensive, or getting more expensive, or suspiciously cheap. I can tell you its compensatory-token share is rising, which means something changed. But when an engineer says "we're adding a memory retrieval layer to improve success rates," I can price the marginal cost with some confidence. The embedding calls, the storage, the retrieval overhead. I can't price the quality benefit. And I've seen memory layers where the retrieval costs show up clearly in my attribution but the success-rate improvement... doesn't really materialize. The cost is legible. The value is somebody else's spreadsheet.

The success definition itself is the deeper problem. Engineering, product, and operations define "successful outcome" differently for the same workflow. My denominator is someone else's judgment call.¹ I'm building financial instrumentation on top of a definitional foundation I don't control.

I've avoided committing on volume forecasts for procurement twice now because I didn't trust my own numbers. That's not a great feeling. But the honest answer is better than the confident wrong one. I learned that in forensic accounting, actually. The moment you start trusting a number you can't verify is the moment you stop being useful. It's the moment you become a very expensive rubber stamp.

Footnotes

1. Ajay Kumar, "FinOps for AI in 2026: 98% Manage Token Spend, 3 in 4 Can't Prove the Value," October 2026. https://www.ajayk.sh/blog/finops-for-ai-token-costs-focus-2026/ ↩ ↩²

2. FinOps X 2026 identified nine distinct cost buckets for agent workloads; Mavvrik AI recap. https://www.mavvrik.ai/blog/finops-x-2026-ai-token-economics/ ↩

3. Compresr AI, "AI FinOps 2026: The Definitive Guide to Costs and Value." https://compresr.ai/blog/ai-finops-definitive-guide-costs-and-value ↩

4. "Less Context, Better Agents: Efficient Context Engineering for Long-Horizon Tool-Using LLM Agents," 2026. https://arxiv.org/pdf/2606.10209 ↩

5. Augment Code, "AI Agent Loop Token Costs: How to Constrain Context," April 2026. https://www.augmentcode.com/guides/ai-agent-loop-token-cost-context-constraints ↩

6. Ajay Kumar, "FinOps for AI in 2026." https://www.ajayk.sh/blog/finops-for-ai-token-costs-focus-2026/ ↩

7. Tian Pan, "The Retry Storm Problem in Agentic Systems," April 2026. https://tianpan.co/blog/2026/04/10/retry-storm-agentic-systems-cascading-failure ↩

8. FinOps Foundation, "Tokenomics: Managing AI Value in SaaS Model Token Costs," June 2026. https://www.finops.org/wg/token-economics-saas/ ↩

9. FinOps Foundation, "Tokenomics." https://www.finops.org/wg/token-economics-saas/ ↩

10. Finout, "FinOps for AI: Pricing, Ownership, Maturity, and KPIs." https://www.finout.io/blog/finops-for-ai-the-definitive-overview ↩