--- title: "Foundations" canonical: "https://current.tinyfish.ai/issue/30/foundations" publication: "TinyFish" publisher: "TinyFish" language: "en-US" guide: "https://current.tinyfish.ai/llms.txt" permalink: "https://current.tinyfish.ai/issue/30/foundations" issue: 30 section: "foundations" description: "Conceptual clarity earned from building at scale" authors: ["Nora Kaplan","Rina Takahashi"] published: 2026-05-25 updated: 2026-05-21 articles: 19 --- # TinyFish > CURRENT is a journal about AI agents in production, for the engineering and product leaders who have to make them work. Drawing on experience building enterprise web agents, it explains why the demo succeeded and the deployment didn't, and what that means for choosing vendors, briefing the board, and avoiding mistakes others have already made. [Read on the website]() ## Issue 30 — Foundations Conceptual clarity earned from building at scale ### The Determinism Budget - [The Non-Determinism Budget]() — By [Nora Kaplan]() · Published 2026-05-21 · Format: The Determinism Budget A system with 95% reliability at each model-directed step delivers 36% reliability over twenty steps. The math is just multiplication. But the agent ecosystem has no standard way to measure this decay, and new research shows why it's so insidious: failed runs are statistically indistinguishable from successful ones through most of their execution. The budget is spent long before anyone can tell. Experienced teams navigate this constraint well. They navigate it by instinct, without a number, making consequential architecture decisions about a resource they've never quantified. ### The Autonomy Dial - [Where Intelligence Lives in Each Step]() — By [Rina Takahashi]() · Published 2026-05-21 · Format: The Autonomy Dial A login form needs zero intelligence. A Playwright selector finds the username field, fills it, clicks "Sign In" in milliseconds. Two steps later, the same workflow hits a CAPTCHA rendered as a bitmap inside a canvas element. Now you need a vision model looking at a screenshot. Same workflow, wildly different requirements per step. Browser automation has three layers of intelligence available at every step, from free and instant to expensive and flexible. How a system moves between them mid-run is the design choice that actually holds up or doesn't. ### Further Reading - **Checkpoints Are Not Durable Execution: Why LangGraph, CrewAI, Google ADK Fall Short** — [Source]() — Published 2026-05-21 Saving state and guaranteeing completion are structurally different problems. Most agent frameworks quietly conflate them. - **Beyond pass@1: A Reliability Science Framework for Long-Horizon LLM Agents** — [Source]() — Published 2026-05-21 Formalizes what production teams already suspect: reliability decays with task length, and single-attempt benchmarks hide it. - **AI Agent Orchestration: LLM vs Code-Driven Patterns** — [Source]() — Published 2026-05-21 The correction arc most production teams travel: start with too much model autonomy, then harden the outer loop. - **Production-Ready LLM Agents: A Comprehensive Framework for Offline Evaluation** — [Source]() — Published 2026-05-21 Deterministic assertion-based testing breaks for non-deterministic systems. Build eval from real production failures instead. - **MCP's Biggest Growing Pains for Production Use Will Soon Be Solved** — [Source]() — Published 2026-05-21 Tool calls look structured at the schema layer but stay fragile where infrastructure reliability actually lives. ### The Determinism Spectrum Every interesting design decision in the agent ecosystem sits on an axis between determinism and autonomy. How much does the code control? How much does the model decide? Where does reliability end and flexibility begin? These six distinctions map the territory where those tradeoffs get made. Each one defines a boundary that's easy to blur in conversation and expensive to blur in production. The point of drawing them sharply here is practical: confusing a checkpoint for a durability guarantee, or a single success for consistent reliability, leads to systems that fail in ways their architecture promised they wouldn't. - **Code Picks the Steps, or the Model Does** — [Source]() — Published 2026-05-21 Tag: System Control When code dictates every step, you have a workflow, even if an LLM handles individual steps. When the model picks its own next action and selects its own tools, that's agentic control. The distinction matters because flexibility and risk arrive together. Anthropic's guidance is clear: use workflows where predictability matters, agents where the problem space genuinely demands adaptation. - **DOM Agents Are Reliable, Vision Agents Reach Further** — [Source]() — Published 2026-05-21 Tag: Browser Architecture DOM-driven browser agents (Playwright plus an LLM) hold a 12–17 percentage point reliability edge on common tasks. Vision-driven agents, working from screenshots with mouse and keyboard, get into places DOM can't: canvas apps, image-rendered UIs, anti-bot screens. The capable systems in 2026 run both, choosing per task. - **Succeeding Once Tells You Less Than You Think** — [Source]() — Published 2026-05-21 Tag: Evaluation Metrics pass@1 asks whether a system can do something. pass^k asks whether it does it reliably across k attempts. The gap widens fast with complexity. In tau-bench retail scenarios, GPT-4o managed respectable pass@1 numbers while its pass^8 cratered below 25%. For production, consistency is the metric that pays rent. - **Saving State and Guaranteeing Completion Are Different Things** — [Source]() — Published 2026-05-21 Tag: Infrastructure Guarantees A checkpoint is a snapshot. You're responsible for knowing when to take one, triggering it, and wiring up recovery when something breaks. Durable execution guarantees a workflow finishes regardless of infrastructure failure, with no developer-managed recovery logic. Most agent frameworks ship checkpointing and sell it as durable execution. The difference surfaces at 3 AM. - **Code Runs the Outer Loop, LLMs the Inner** — [Source]() — Published 2026-05-21 Tag: Orchestration Layers Production agentic systems almost always blend both. Code-driven orchestration handles the known business process: predictable inputs, defined sequences, clear exit conditions. LLM-driven execution handles the parts where reasoning actually helps, at contained scope. Where you draw that boundary between the two layers is the design decision everything else follows from. - **MCP Connects Tools, A2A Coordinates Between Agents** — [Source]() — Published 2026-05-21 Tag: Protocol Layers MCP handles what an agent can use: data sources, resources, prompts, executable functions. A2A handles what agents do together: discovery, delegation, task handoffs across vendor boundaries. Two complementary layers solving different problems, now both governed by the Linux Foundation's Agentic AI Foundation and its 146 member organizations. ### The Stack, May 2026 Three months is a long time in agent infrastructure. Protocols that launched without governance features are now scrambling to add them. Frameworks are hitting production limits that no benchmark anticipated. Security researchers are quantifying attack surfaces that were hand-waved away a year ago. Six cards, six snapshots. Adoption numbers, version status, and the one thing practitioners should actually register about each. We'll save the analysis for other pieces. This is just where the ground is right now. - **MCP Hits 97 Million Downloads, Auth Still Optional** — [Source]() — Published 2026-05-21 Tag: Connection Protocol Over 10,000 active public servers. 97 million monthly SDK downloads. Donated to the Linux Foundation's Agentic AI Foundation in December 2025. The 2026 roadmap targets stateless transport scalability, task lifecycle management, and enterprise auth. That last item matters because authentication and authorization remain optional in the spec. Gartner has flagged the missing enterprise governance features. - **A2A Reaches Version 1.2 With Signed Agent Cards** — [Source]() — Published 2026-05-21 Tag: Agent Protocol Version 1.2, with 150-plus organizations in production. Governed by AAIF under the Linux Foundation. Backed by Google, Microsoft, AWS, Salesforce, SAP, ServiceNow, Workday, and IBM. The headline feature: signed agent cards with cryptographic signatures for domain verification, which starts to close the trust gap in agent-to-agent discovery. - **LangGraph Leads on Stars, Checkpoints Have Gaps** — [Source]() — Published 2026-05-21 Tag: Agent Framework Passed CrewAI in GitHub stars early 2026. Offers stateful workflow infrastructure with checkpoint-based persistence. Worth knowing: checkpointers save state between nodes, not inside them. A mid-node crash loses all intermediate work. AWS published a DynamoDB integration for durable agent state in January 2026. - **Cloudflare Browser Run Gives Agents Raw CDP** — [Source]() — Published 2026-05-21 Tag: Browser Infrastructure Renamed from Browser Rendering during Agents Week 2026. The key move: exposing Chrome DevTools Protocol directly to agents, skipping higher-level library abstractions for tighter control and better token efficiency. Full browser sessions run on Cloudflare's global network with session recording, replay, real-time debugging, and human-in-the-loop intervention built in. - **MCP Tool Poisoning Hits 72 Percent Attack Success** — [Source]() — Published 2026-05-21 Tag: Security Surface The MCPTox benchmark tested 45 live MCP servers and 353 tools. Attack success rates topped 60% for many popular agents, peaking at 72%. An uncomfortable finding: the most capable models often fared worst. Better instruction-following made them more compliant with malicious tool metadata. Root cause is structural. Tool descriptions get reviewed once at connect-time, but tool responses flow straight into LLM context unchecked. - **Agent Benchmarks Finally Measure More Than Single Runs** — [Source]() — Published 2026-05-21 Tag: Eval Frameworks ReliabilityBench evaluates three dimensions: consistency under repeated execution (k-trial pass rates), robustness to task perturbations, and fault tolerance when infrastructure breaks. Tau-bench's pass^k metric found state-of-the-art agents below 25% consistency on retail scenarios. METR's "50% success horizon" has doubled roughly every seven months since 2019. Standard benchmarks, meanwhile, still report only pass@1 on short tasks. ## Other sections in Issue 30 - [Daily Brief]() - [Market Pulse]() - [Practitioner's Corner]() - [Echoes]() - [Vision]() ## More - [Publication guide]() - [Issue archive]()