CURRENT | Foundations

The Determinism Budget

The Non-Determinism Budget

By Nora Kaplan— May 21, 2026

Feature image for article: The Non-Determinism Budget

A system with 95% reliability at each model-directed step delivers 36% reliability over twenty steps. The math is just multiplication. But the agent ecosystem has no standard way to measure this decay, and new research shows why it's so insidious: failed runs are statistically indistinguishable from successful ones through most of their execution. The budget is spent long before anyone can tell. Experienced teams navigate this constraint well. They navigate it by instinct, without a number, making consequential architecture decisions about a resource they've never quantified.

The Determinism Budget

The Non-Determinism Budget

By Nora Kaplan— May 21, 2026

A system with 95% reliability at each model-directed step delivers 36% reliability over twenty steps. The math is just multiplication. But the agent ecosystem has no standard way to measure this decay, and new research shows why it's so insidious: failed runs are statistically indistinguishable from successful ones through most of their execution. The budget is spent long before anyone can tell. Experienced teams navigate this constraint well. They navigate it by instinct, without a number, making consequential architecture decisions about a resource they've never quantified.

The Autonomy Dial

Where Intelligence Lives in Each Step

By Rina Takahashi— May 21, 2026

Feature image for article: Where Intelligence Lives in Each Step

A login form needs zero intelligence. A Playwright selector finds the username field, fills it, clicks "Sign In" in milliseconds. Two steps later, the same workflow hits a CAPTCHA rendered as a bitmap inside a canvas element. Now you need a vision model looking at a screenshot. Same workflow, wildly different requirements per step. Browser automation has three layers of intelligence available at every step, from free and instant to expensive and flexible. How a system moves between them mid-run is the design choice that actually holds up or doesn't.

The Autonomy Dial

Where Intelligence Lives in Each Step

By Rina Takahashi— May 21, 2026

A login form needs zero intelligence. A Playwright selector finds the username field, fills it, clicks "Sign In" in milliseconds. Two steps later, the same workflow hits a CAPTCHA rendered as a bitmap inside a canvas element. Now you need a vision model looking at a screenshot. Same workflow, wildly different requirements per step. Browser automation has three layers of intelligence available at every step, from free and instant to expensive and flexible. How a system moves between them mid-run is the design choice that actually holds up or doesn't.

Further Reading

Checkpoints Are Not Durable Execution: Why LangGraph, CrewAI, Google ADK Fall ShortSaving state and guaranteeing completion are structurally different problems. Most agent frameworks quietly conflate them.

Beyond pass@1: A Reliability Science Framework for Long-Horizon LLM AgentsFormalizes what production teams already suspect: reliability decays with task length, and single-attempt benchmarks hide it.

Quick links

AI Agent Orchestration: LLM vs Code-Driven Patterns

Production-Ready LLM Agents: A Comprehensive Framework for Offline Evaluation

MCP's Biggest Growing Pains for Production Use Will Soon Be Solved

The Determinism Spectrum

Every interesting design decision in the agent ecosystem sits on an axis between determinism and autonomy. How much does the code control? How much does the model decide? Where does reliability end and flexibility begin?

These six distinctions map the territory where those tradeoffs get made. Each one defines a boundary that's easy to blur in conversation and expensive to blur in production. The point of drawing them sharply here is practical: confusing a checkpoint for a durability guarantee, or a single success for consistent reliability, leads to systems that fail in ways their architecture promised they wouldn't.

The Determinism Spectrum

Every interesting design decision in the agent ecosystem sits on an axis between determinism and autonomy. How much does the code control? How much does the model decide? Where does reliability end and flexibility begin?

These six distinctions map the territory where those tradeoffs get made. Each one defines a boundary that's easy to blur in conversation and expensive to blur in production. The point of drawing them sharply here is practical: confusing a checkpoint for a durability guarantee, or a single success for consistent reliability, leads to systems that fail in ways their architecture promised they wouldn't.

System Control

Code Picks the Steps, or the Model Does

When code dictates every step, you have a workflow, even if an LLM handles individual steps. When the model picks its own next action and selects its own tools, that's agentic control. The distinction matters because flexibility and risk arrive together. Anthropic's guidance is clear: use workflows where predictability matters, agents where the problem space genuinely demands adaptation.

Browser Architecture

DOM Agents Are Reliable, Vision Agents Reach Further

DOM-driven browser agents (Playwright plus an LLM) hold a 12–17 percentage point reliability edge on common tasks. Vision-driven agents, working from screenshots with mouse and keyboard, get into places DOM can't: canvas apps, image-rendered UIs, anti-bot screens. The capable systems in 2026 run both, choosing per task.

Evaluation Metrics

Succeeding Once Tells You Less Than You Think

pass@1 asks whether a system can do something. pass^k asks whether it does it reliably across k attempts. The gap widens fast with complexity. In tau-bench retail scenarios, GPT-4o managed respectable pass@1 numbers while its pass^8 cratered below 25%. For production, consistency is the metric that pays rent.

Infrastructure Guarantees

Saving State and Guaranteeing Completion Are Different Things

A checkpoint is a snapshot. You're responsible for knowing when to take one, triggering it, and wiring up recovery when something breaks. Durable execution guarantees a workflow finishes regardless of infrastructure failure, with no developer-managed recovery logic. Most agent frameworks ship checkpointing and sell it as durable execution. The difference surfaces at 3 AM.

Orchestration Layers

Code Runs the Outer Loop, LLMs the Inner

Production agentic systems almost always blend both. Code-driven orchestration handles the known business process: predictable inputs, defined sequences, clear exit conditions. LLM-driven execution handles the parts where reasoning actually helps, at contained scope. Where you draw that boundary between the two layers is the design decision everything else follows from.

Protocol Layers

MCP Connects Tools, A2A Coordinates Between Agents

MCP handles what an agent can use: data sources, resources, prompts, executable functions. A2A handles what agents do together: discovery, delegation, task handoffs across vendor boundaries. Two complementary layers solving different problems, now both governed by the Linux Foundation's Agentic AI Foundation and its 146 member organizations.

The Stack, May 2026

Three months is a long time in agent infrastructure. Protocols that launched without governance features are now scrambling to add them. Frameworks are hitting production limits that no benchmark anticipated. Security researchers are quantifying attack surfaces that were hand-waved away a year ago.

Six cards, six snapshots. Adoption numbers, version status, and the one thing practitioners should actually register about each. We'll save the analysis for other pieces. This is just where the ground is right now.

The Stack, May 2026

Three months is a long time in agent infrastructure. Protocols that launched without governance features are now scrambling to add them. Frameworks are hitting production limits that no benchmark anticipated. Security researchers are quantifying attack surfaces that were hand-waved away a year ago.

Six cards, six snapshots. Adoption numbers, version status, and the one thing practitioners should actually register about each. We'll save the analysis for other pieces. This is just where the ground is right now.

Connection Protocol

MCP Hits 97 Million Downloads, Auth Still Optional

Over 10,000 active public servers. 97 million monthly SDK downloads. Donated to the Linux Foundation's Agentic AI Foundation in December 2025. The 2026 roadmap targets stateless transport scalability, task lifecycle management, and enterprise auth. That last item matters because authentication and authorization remain optional in the spec. Gartner has flagged the missing enterprise governance features.

Agent Protocol

A2A Reaches Version 1.2 With Signed Agent Cards

Version 1.2, with 150-plus organizations in production. Governed by AAIF under the Linux Foundation. Backed by Google, Microsoft, AWS, Salesforce, SAP, ServiceNow, Workday, and IBM. The headline feature: signed agent cards with cryptographic signatures for domain verification, which starts to close the trust gap in agent-to-agent discovery.

Agent Framework

LangGraph Leads on Stars, Checkpoints Have Gaps

Passed CrewAI in GitHub stars early 2026. Offers stateful workflow infrastructure with checkpoint-based persistence. Worth knowing: checkpointers save state between nodes, not inside them. A mid-node crash loses all intermediate work. AWS published a DynamoDB integration for durable agent state in January 2026.

Browser Infrastructure

Cloudflare Browser Run Gives Agents Raw CDP

Renamed from Browser Rendering during Agents Week 2026. The key move: exposing Chrome DevTools Protocol directly to agents, skipping higher-level library abstractions for tighter control and better token efficiency. Full browser sessions run on Cloudflare's global network with session recording, replay, real-time debugging, and human-in-the-loop intervention built in.

Security Surface

MCP Tool Poisoning Hits 72 Percent Attack Success

The MCPTox benchmark tested 45 live MCP servers and 353 tools. Attack success rates topped 60% for many popular agents, peaking at 72%. An uncomfortable finding: the most capable models often fared worst. Better instruction-following made them more compliant with malicious tool metadata. Root cause is structural. Tool descriptions get reviewed once at connect-time, but tool responses flow straight into LLM context unchecked.

Eval Frameworks

Agent Benchmarks Finally Measure More Than Single Runs

ReliabilityBench evaluates three dimensions: consistency under repeated execution (k-trial pass rates), robustness to task perturbations, and fault tolerance when infrastructure breaks. Tau-bench's pass^k metric found state-of-the-art agents below 25% consistency on retail scenarios. METR's "50% success horizon" has doubled roughly every seven months since 2019. Standard benchmarks, meanwhile, still report only pass@1 on short tasks.

Past Articles

Every Agent Framework Is an Argument About What's Hard

Framework comparison tables show you what's available. The interesting part starts when a container restarts mid-workflo...

What a Benchmark Score Actually Contains

Claude Opus 4 scores 64.9% on GAIA in one scaffold and 57.6% in another. Same model, same benchmark, same questions. The...

What Happens Between MCP and A2A

MCP and A2A handle different halves of a multi-agent workflow. Most explanations cover each protocol separately and leav...

The Seam Between Tools and Agents

MCP and A2A are becoming the communication layer for autonomous software. MCP assumes the thing on the other end is a to...

Past Articles

Every Agent Framework Is an Argument About What's Hard

Framework comparison tables show you what's available. The interesting part starts when a container restarts mid-workflo...

What a Benchmark Score Actually Contains

Claude Opus 4 scores 64.9% on GAIA in one scaffold and 57.6% in another. Same model, same benchmark, same questions. The...

What Happens Between MCP and A2A

MCP and A2A handle different halves of a multi-agent workflow. Most explanations cover each protocol separately and leav...

The Seam Between Tools and Agents

MCP and A2A are becoming the communication layer for autonomous software. MCP assumes the thing on the other end is a to...