CURRENT

Inference accounts for roughly 85% of enterprise AI spend. The price spread is staggering: Claude Opus 4.5 runs $25 per million output tokens; GPT-5 nano costs $0.05 per million input. Same model, different provider? Expect 10x variance. Choosing where to run is a budget decision disguised as a technical one.

Gartner puts the agentic multiplier at 5–30x more tokens per task compared to standard chat. Complex multi-agent workflows regularly blow past a million tokens on a single job. Per-token costs keep dropping. Volume is rising faster. The bill grows anyway.

Every token in the context window gets billed on every LLM call. One agent resending a 25,000-token document across five reasoning steps burns 125,000 input tokens on a single task. MCP tool metadata alone can eat 40–50% of available context. RAG and prompt caching cut this waste by 60–90%.

When agents hand off to other agents, the coordination layer becomes its own cost surface. A single edge case can trigger retry chains costing 50x the normal path. Framework choice compounds this: in benchmarks, some orchestration frameworks consumed nearly double the tokens of others running identical tasks.

Kubernetes provisioning, vector databases, memory backends, retry queues, load balancers. All separate line items, all easy to forget. Self-hosting above 100K requests per day saves 60–80% versus API pricing. Below that threshold, fixed GPU costs make self-hosting the more expensive option.

Agent observability runs $25–$500/month at startup scale, $2K–$10K+ for enterprise volumes. The billing math gets adversarial fast: a 10-span trace with one evaluation score costs 12 billable units. Some teams have resorted to sampling 0.1% of traffic just to keep observability affordable.

Plan-and-Execute splits the expensive reasoning from the cheap labor. A frontier model architects the strategy; distilled or smaller models carry out each step. Production deployments report 80% cost cuts, stretching toward 90% with full distillation. The catch: distilled executors fail on rare edge cases outside their training data. Precisely the cases where failure costs the most.

Monolithic agent architectures stuff every subagent with the full conversation history. Scope isolation walls off each agent's context to only what it needs. LangChain's multi-agent documentation shows a 67% token reduction per request through this boundary enforcement. The cost: each handoff between isolated agents adds 500ms–1.5s of summarization latency and risks dropping context that mattered.

A frontier reasoning call can cost 190x more than a small model handling equivalent work. Routing each agent step by complexity, sending simple extraction to cheap models and reserving heavy inference for hard decisions, yields 30–70% savings across practitioner reports. The fragility: you now depend on a routing classifier that adds its own latency, and a single provider outage can break a multi-model pipeline in ways a single-model system never would.

Agents hammer the same system prompts, tool schemas, and few-shot examples on every invocation. Provider-level prompt caching (Anthropic discounts cached tokens by 90%) eliminates that redundancy. Practitioners report 45–80% cost reductions on cache-eligible workloads. The caveat worth remembering: when tool schemas update, stale cache hits silently degrade output quality unless you manage invalidation explicitly.

The strongest evidence in this set comes from a peer-reviewed preprint: caching and adapting prior execution plans across five agent workloads cut costs by 50% and latency by 27%, retaining 96.6% accuracy. This goes beyond prompt-prefix caching to reuse the planner's actual reasoning. Requires a Plan-Act architecture to work, and effectiveness drops sharply on genuinely novel tasks where no prior plan applies.