Share

Reliability As Shape

The Exponent Borrows Two Things Nobody Has Measured

By Sevda Polat— July 29, 2026

Feature image for article: The Exponent Borrows Two Things Nobody Has Measured

Every agent deployment estimate runs the same arithmetic. Per-step success rate, raised to the number of steps, watch the floor open. Directionally right, and nobody should be argued out of it. But it borrows two things the evidence hasn't supplied: one per-step rate to raise, and steps that fail independently of each other. Five runs of the same task, same agent, same instruction, different bill. Horizon is doing most of the damage in the reliability results, and it is the variable those studies measure least directly.

Reliability As Shape

The Exponent Borrows Two Things Nobody Has Measured

By Sevda Polat— July 29, 2026

Every agent deployment estimate runs the same arithmetic. Per-step success rate, raised to the number of steps, watch the floor open. Directionally right, and nobody should be argued out of it. But it borrows two things the evidence hasn't supplied: one per-step rate to raise, and steps that fail independently of each other. Five runs of the same task, same agent, same instruction, different bill. Horizon is doing most of the damage in the reliability results, and it is the variable those studies measure least directly.

Where State Lives Now

Somebody Has to Hold the Basket

By Sevda Polat— July 29, 2026

Feature image for article: Somebody Has to Hold the Basket

A basket, a second item, a card that only the person at the keyboard can produce, and a refund that may take a day to land. Four calls, one story.

The 2026-07-28 revision removed the connection that used to hold that story together. Nothing was actually subtracted. The handshake, the session, the durable job each have a successor, and every one of those successors now sits on your side of the line. What follows is what a shopping cart looks like once the protocol stops keeping it for you.

Where State Lives Now

Somebody Has to Hold the Basket

By Sevda Polat— July 29, 2026

A basket, a second item, a card that only the person at the keyboard can produce, and a refund that may take a day to land. Four calls, one story.

The 2026-07-28 revision removed the connection that used to hold that story together. Nothing was actually subtracted. The handshake, the session, the durable job each have a successor, and every one of those successors now sits on your side of the line. What follows is what a shopping cart looks like once the protocol stops keeping it for you.

Primary Sources

MCP "Key Changes: 2026-07-28" Changelog

Read it for the negative space. Sessions and handshakes are gone; what institutions must now design themselves is the real story.

Rabanser et al., "Towards a Science of AI Agent Reliability" (ICML 2026)

Twelve metrics across four dimensions. Calibration — whether the agent knows it's uncertain — is the one most teams quietly skip. This makes the omission visible.

Albayaydh, Zhao, and Flechais, "Beyond the Leaderboard" (July 2026 preprint)

Zhou et al., "Authentication Security in Real-World Remote MCP Servers" (May 2026)

Deliberate pair: METR Uplift Update (February 2026) and Murphy-Hill et al., Command-Line Coding Agent Adoption (July 2026 preprint)

Staufer et al., "The 2025 AI Agent Index" (FAccT 2026)

Primary Sources

MCP "Key Changes: 2026-07-28" ChangelogRead it for the negative space. Sessions and handshakes are gone; what institutions must now design themselves is the real story.

Rabanser et al., "Towards a Science of AI Agent Reliability" (ICML 2026)Twelve metrics across four dimensions. Calibration — whether the agent knows it's uncertain — is the one most teams quietly skip. This makes the omission visible.

Quick links

Albayaydh, Zhao, and Flechais, "Beyond the Leaderboard" (July 2026 preprint)

Zhou et al., "Authentication Security in Real-World Remote MCP Servers" (May 2026)

Deliberate pair: METR Uplift Update (February 2026) and Murphy-Hill et al., Command-Line Coding Agent Adoption (July 2026 preprint)

Staufer et al., "The 2025 AI Agent Index" (FAccT 2026)

The Horizon Ladder

A wrong parameter on a single tool call is annoying. A lost constraint propagating through a five-step chain is dangerous. A compressed summary that quietly drops the one fact preventing a catastrophic action twenty steps later is something else entirely. The failure modes change character as tasks get longer, not just frequency.

Recent research quantifies what practitioners already feel in their debugging sessions: capability gains have outpaced reliability gains by roughly seven to one, and failures compound nonlinearly with step count. Scaffolding that fixes one band's problems can introduce new failure surfaces at the next. These six cards map each step-count band to its dominant failure mode and the specific instrumentation check worth running this week.

The Horizon Ladder

A wrong parameter on a single tool call is annoying. A lost constraint propagating through a five-step chain is dangerous. A compressed summary that quietly drops the one fact preventing a catastrophic action twenty steps later is something else entirely. The failure modes change character as tasks get longer, not just frequency.

Recent research quantifies what practitioners already feel in their debugging sessions: capability gains have outpaced reliability gains by roughly seven to one, and failures compound nonlinearly with step count. Scaffolding that fixes one band's problems can introduce new failure surfaces at the next. These six cards map each step-count band to its dominant failure mode and the specific instrumentation check worth running this week.

Single Call

One Tool Call, One Schema, One Chance

Dominant failure: malformed arguments, wrong parameters, silent type coercion. Validate every argument against the tool's declared schema before execution. Then run the identical prompt five times. If tool selection or argument values drift between runs, you have a consistency problem that pass-at-one scoring will never reveal.

Short Chain

Three to Seven Steps, Constraints Quietly Dropped

Dominant failure: a constraint established at step one gets silently violated by step four. Agents pick similar actions across runs but reorder them unpredictably. Log the provenance of every argument, tracing which prior output populated it. Compare final states across five runs, not just whether the chain completed.

Full Session

Dozens of Steps, Nonlinear Compounding Takes Over

Dominant failure: degradation that compounds faster than linearly. A thirty-step session doesn't fail three times more often than ten steps. It can fail an order of magnitude more. Track the step number where the first unexpected action appears. Assert final state against the authoritative system of record, never the agent's own summary of what it did.

Context Reset

The Agent Summarizes Away Its Own Constraints

Dominant failure: lossy compression at context boundaries. When history gets summarized, constraints from earlier steps vanish, and nothing downstream knows they ever existed. After every context reset, inject a probe question only answerable from pre-reset information. Consistent failure on that recall test means the compression layer is discarding critical state.

Multi-Agent

Multiple Agents, One Broken Handoff Between Them

Dominant failure: every agent completes its piece correctly, but the handoff loses scope or constraints. Log which agent made each tool call and under what delegated authority. Compare what the orchestrator sent against what the subagent understood as its task. Divergence here is a coordination failure even when everything technically "succeeds."

Long-Running

Hours to Days, Where Errors Become Irreversible

Dominant failure: error severity grows with elapsed time. A mistake you could reverse in thirty minutes becomes permanent after forty-eight hours of committed state changes. Track elapsed time on every unresolved approval gate. Assert final state against the authoritative record at task close. Replay golden tasks after any model or prompt change, because recoverability degrades silently.

Where The State Went

The MCP 2026-07-28 revision removes protocol-level sessions from Streamable HTTP. Everything that used to ride implicitly on a persistent connection now needs to be declared, carried, or echoed back. Otherwise it simply vanishes between requests.

Where things landed varies. Some moved into per-request headers. Others became opaque handles the client passes as tool arguments. A few live in entirely new extension mechanisms. But the pattern holds: ambient context is gone, and explicit declaration is the only thing left.

Each card below maps one implicit dependency to its new location and the concrete breakage a team hits by leaving it unaddressed.

Where The State Went

The MCP 2026-07-28 revision removes protocol-level sessions from Streamable HTTP. Everything that used to ride implicitly on a persistent connection now needs to be declared, carried, or echoed back. Otherwise it simply vanishes between requests.

Where things landed varies. Some moved into per-request headers. Others became opaque handles the client passes as tool arguments. A few live in entirely new extension mechanisms. But the pattern holds: ambient context is gone, and explicit declaration is the only thing left.

Each card below maps one implicit dependency to its new location and the concrete breakage a team hits by leaving it unaddressed.

Capability Negotiation

Capabilities Ride Every Request Now, Not Once

Used to be settled in the initialize handshake and remembered. Now declared via _meta per request, with server/discover replacing the old capability exchange. Behind a load balancer, nodes that never saw the handshake treat every client as unknown and silently fall back to conservative defaults. Behavior drifts across retries.

Version Agreement

Protocol Version Moved from Handshake to Header

Established once during session init. Now MCP-Protocol-Version is a required header on every Streamable HTTP POST, alongside Mcp-Method and Mcp-Name. In mixed deployments where some instances have upgraded and others haven't, identical requests get processed under different semantic rules with zero indication anything is wrong.

Application State

Workflow Context Lives in Server-Minted Handles

Was keyed to Mcp-Session-Id, stored in-process or in a shared store. Now servers mint explicit handles (basket_id, browser_id) as tool results; clients pass them back as arguments. Skip this and a rebalanced request quietly spins up a fresh workflow. Your user's cart empties itself mid-checkout.

Task Resumption

Resuming Long Tasks Requires the Tasks Extension