--- title: "Foundations" canonical: "https://current.tinyfish.ai/issue/31/foundations" publication: "TinyFish" publisher: "TinyFish" language: "en-US" guide: "https://current.tinyfish.ai/llms.txt" permalink: "https://current.tinyfish.ai/issue/31/foundations" issue: 31 section: "foundations" description: "Conceptual clarity earned from building at scale" authors: ["Nora Kaplan","Rina Takahashi"] published: 2026-06-01 updated: 2026-05-28 articles: 22 --- # TinyFish > CURRENT is a journal about AI agents in production, for the engineering and product leaders who have to make them work. Drawing on experience building enterprise web agents, it explains why the demo succeeded and the deployment didn't, and what that means for choosing vendors, briefing the board, and avoiding mistakes others have already made. [Read on the website]() ## Issue 31 — Foundations Conceptual clarity earned from building at scale ### Context Over Instruction - [Where Agent Behavior Actually Gets Decided]() — By [Nora Kaplan]() · Published 2026-05-28 · Format: Context Over Instruction A developer's coding agent worked perfectly for thirty minutes, then started contradicting its own decisions. The prompt was the same. The model was the same. Everything else in the context window had been accumulating. His fix was radical: stop carrying context forward entirely. It worked. Across production agent systems, the same asymmetry kept surfacing — a carefully engineered prompt drowning in accumulated tool outputs performed worse than a simple prompt in a clean, focused context. The field needed a word for the thing practitioners were actually optimizing, because "prompt" didn't describe it. ### The Perception Layer - [What Browser Agents See Before They Think]() — By [Rina Takahashi]() · Published 2026-05-28 · Format: The Perception Layer Before a browser agent reasons about a task, it needs a representation of the page. Raw HTML, a screenshot, or the accessibility tree. That choice, made once and early, sets the token budget, failure modes, and reliability ceiling for everything downstream. OpenAI, Microsoft, and Perplexity built their agent systems independently and all three landed on the same primary perception layer: infrastructure originally designed for people who can't see screens. The convergence is worth understanding. ### Further Reading - **Demystifying Evals for AI Agents** — [Source]() — Published 2026-05-28 Anthropic's engineering team frames evals as compounding investments in behavioral judgment. Each eval built makes the next behavioral change easier to assess, and the returns accumulate. The corollary is what earns this piece its place: without that compounding, teams hit a breaking point where they're shipping changes to agent behavior with no way to know whether things got better or worse. "Flying blind" is the phrase they use, and it's precise. - **State of Agent Engineering** — [Source]() — Published 2026-05-28 The number that should keep people up at night: 89% of practitioner teams have observability in place, but only 52% run evals. Nearly nine in ten teams can watch what their agents do. Barely half have built the capacity to judge whether any of it was appropriate. That asymmetry describes an industry that instrumented the cockpit but hasn't agreed on what the gauges should read. - **Model Context Protocol Threat Modeling and Analysis** — [Source]() — Published 2026-05-28 Tool poisoning ranks first on OWASP's LLM Top 10 and scores critical in this paper's DREAD analysis, which puts a fine point on something easy to overlook: the protocol layer shapes agent behavior before the agent decides anything. When tool descriptions carry malicious instructions, the attack surface is the infrastructure the agent trusts, not the model itself. - **The State of AI Agents in 2026: A Practitioner's Guide** — [Source]() — Published 2026-05-28 This piece names a shift the industry is still absorbing: from content risk (the model says something harmful) to action risk (the system does something harmful in the world). The distinction matters because the entire safety apparatus built around language models was designed for the first category. The second one operates on different timescales, with different consequences, and the tooling hasn't caught up. - **How AI Agents Are Making Accessibility a Business-Critical Development Priority** — [Source]() — Published 2026-05-28 When agents can't parse a site's structure, they can't transact on it. This piece traces the business consequence: accessibility quality is becoming a prerequisite for participation in agent-driven commerce, which means development teams are reprioritizing it for revenue reasons that have nothing to do with compliance. The pressure to build accessible sites is arriving from a direction nobody in the accessibility community anticipated. ### Vocabulary That Shapes Decisions The agent ecosystem borrows vocabulary freely and pays for it later. Terms drift in from adjacent fields, pick up new connotations, and settle into usage where two words that sound interchangeable actually point at different architectures, different risk surfaces, different engineering bets. These five distinctions keep showing up because getting them wrong has traceable consequences. A team optimizes the wrong metric. A security review scopes against the wrong threat model. Someone builds retrieval infrastructure where they needed persistence, or vice versa. The vocabulary here is load-bearing. Swap one term for its near-neighbor and you end up building the wrong thing, confidently. - **Context Fills the Window, Memory Outlives It** — [Source]() — Published 2026-05-28 Tag: Core Concepts Context is what the model sees right now: the prompt, retrieved documents, tool outputs, prior steps in this turn. Memory is what the system decides to persist and may surface later, across sessions, through retrieval logic you have to build. Conflating them is how teams end up stuffing everything into the context window and wondering why their agent has no durable sense of anything it learned last Tuesday. - **Who Picks the Next Step Changes Everything** — [Source]() — Published 2026-05-28 Tag: System Design If your code decides every step, you've built a workflow. It might call a model at each node, but the control flow is yours. If the model decides what to do next, you're in agent territory. The engineering, the eval strategy, and especially the failure modes are different animals. Treating an agent like a workflow means your error handling assumes predictability you don't have. - **Succeeding Once Tells You Almost Nothing Useful** — [Source]() — Published 2026-05-28 Tag: Eval Metrics Pass@1 asks: did the agent get it right on the first try? Pass^k asks: does it keep getting it right across k repeated attempts? Tau-bench found agents scoring above 60% on pass@1 that crater below 25% at pass^8 in retail scenarios. If your production system needs to work reliably, not just occasionally, pass@1 is the wrong scoreboard. - **Reaching Tools and Coordinating Agents Are Different** — [Source]() — Published 2026-05-28 Tag: Agent Protocols MCP gives an agent access to tools and data sources. A2A lets agents talk to other agents. They're complementary layers in a stack, but the risk surfaces barely overlap. Expanding what an agent can touch raises one set of security questions. Expanding who it can delegate to raises a genuinely different set. Scoping a security review against the wrong protocol means missing the threats that actually apply. - **Watching Failures Pile Up Won't Prevent Them** — [Source]() — Published 2026-05-28 Tag: Production Readiness Observability tells you what happened: which tools fired, what errors surfaced, how state changed. Evals tell you whether what happened was correct. A LangChain survey found 89% of teams had adopted observability but only 52% were running evals. That gap explains a lot. Teams can see their agents failing in granular detail and still have no systematic way to catch the failures before users do. ### Landscape Snapshot While most of the industry took a long weekend, the agent stack kept hardening. Infrastructure choices are quietly becoming defaults. Security surfaces are getting named and ranked. Tooling ecosystems are consolidating before the practices they support have fully matured. These six snapshots cover the layers where agent behavior actually gets shaped: markets, failure modes, architectural bets, security boundaries. What connects them is a timing problem. Decisions being made right now at the infrastructure level will constrain what's possible later at the governance level, and the governance vocabulary is still catching up. - **Observability Platforms Hit $2.69 Billion in 2026** — [Source]() — Published 2026-05-28 Tag: Market Signal LLM observability grew 36% year-over-year and is projected to reach $9.26B by 2030. Gartner expects half of all GenAI deployments to invest in observability by 2028, up from roughly 15% today. Worth noting: agent observability is a different animal than model monitoring. Failures show up in multi-step causal chains, and diagnosing them requires full-session traces, not just token-level metrics. - **Durable Execution Returns Without the Custom Infrastructure** — [Source]() — Published 2026-05-28 Tag: Infrastructure Pattern The OpenAI Agents SDK integrated with Temporal went GA in March 2026, turning agent reasoning steps into discrete activities with automatic retry and crash recovery. Microsoft's Agent Framework ships a parallel implementation. The idea underneath both: checkpoint state at every transition so a process can pause, crash, and pick up where it left off. - **Tool Poisoning Ranks First Among MCP Threats** — [Source]() — Published 2026-05-28 Tag: Security Risk Tool poisoning scores Critical at 46.5 out of 50 in DREAD analysis and tops OWASP's LLM Top 10. Peer-reviewed research published in MDPI this May tested seven major MCP clients and found most lack sufficient static validation. Tool descriptions get reviewed once at connection time. After that, tool responses flow into agent context unchecked, and in multi-agent chains, the poisoning propagates. - **Infinite Handoff Loops Break Multi-Agent Systems First** — [Source]() — Published 2026-05-28 Tag: Orchestration Risk The failure everyone hits first: Agent A hands off to B, B to C, C back to A, and nobody owns the task. Five agents can mean 25x the management complexity of one. Princeton NLP research found single agents matched or beat multi-agent setups on 64% of benchmarked tasks given identical tools and context. Practitioner consensus is settling: most teams reach for multi-agent orchestration well before a single agent hits its ceiling. - **Agent Evaluation Consolidates Around Eight Core Platforms** — [Source]() — Published 2026-05-28 Tag: Eval Landscape Five commercial platforms (LangSmith, Braintrust, Helicone, Phoenix, Promptfoo) and three open-source frameworks (OpenAI Evals, DeepEval, Inspect AI) now define the eval category. OpenAI's March 2026 acquisition of Promptfoo introduced an uncomfortable wrinkle: eval tooling owned by a model vendor carries different incentives than eval tooling that isn't. Non-OpenAI teams are already noticing. - **Accessibility Tree Now Leads Production Browser Agents** — [Source]() — Published 2026-05-28 Tag: Browser Perception OpenAI Atlas, Microsoft Playwright MCP, and Perplexity Comet all chose accessibility data as their primary perception layer. The economics are stark: roughly 4,000 tokens per page versus 50,000 for screenshots. Three major platforms, building independently, converged on a data structure originally designed for screen readers. ### Past Articles - [Where Intelligence Lives in Each Step]() — By [Rina Takahashi]() · Published 2026-05-21 · Format: The Autonomy Dial A login form needs zero intelligence. A Playwright selector finds the username field, fills it, clicks "Sign In" in milliseconds. Two steps later, the same workflow hits a CAPTCHA rendered as a bitmap inside a canvas element. Now you need a vision model looking at a screenshot. Same workflow, wildly different requirements per step. Browser automation has three layers of intelligence available at every step, from free and instant to expensive and flexible. How a system moves between them mid-run is the design choice that actually holds up or doesn't. - [The Non-Determinism Budget]() — By [Nora Kaplan]() · Published 2026-05-21 · Format: The Determinism Budget A system with 95% reliability at each model-directed step delivers 36% reliability over twenty steps. The math is just multiplication. But the agent ecosystem has no standard way to measure this decay, and new research shows why it's so insidious: failed runs are statistically indistinguishable from successful ones through most of their execution. The budget is spent long before anyone can tell. Experienced teams navigate this constraint well. They navigate it by instinct, without a number, making consequential architecture decisions about a resource they've never quantified. - [Every Agent Framework Is an Argument About What's Hard]() — By [Rina Takahashi]() · Published 2026-05-13 · Format: Choosing the Scaffold Framework comparison tables show you what's available. The interesting part starts when a container restarts mid-workflow at 3 a.m. LangGraph, CrewAI, the Claude Agent SDK, and Microsoft Agent Framework 1.0 each embed a different argument about what the hard problem in agent orchestration actually is — control flow, task delegation, permission boundaries, enterprise interoperability. Picking one means committing to a control philosophy that shapes how your system encounters problems you haven't anticipated yet. A better approach: name your dominant constraint first, then find the framework whose core abstraction matches it. - [What a Benchmark Score Actually Contains]() — By [Rina Takahashi]() · Published 2026-05-13 · Format: Reading the Leaderboard Claude Opus 4 scores 64.9% on GAIA in one scaffold and 57.6% in another. Same model, same benchmark, same questions. The seven-point gap comes entirely from the orchestration wrapping the model, and it's larger than many consecutive frontier releases produce. Agent benchmark scores carry information about scaffold and token budget alongside model capability, often in roughly equal measure. The numbers tell you where engineering leverage lives, if you're reading all three variables. ## Other sections in Issue 31 - [Daily Brief]() - [Market Pulse]() - [Practitioner's Corner]() - [Echoes]() - [Vision]() ## More - [Publication guide]() - [Issue archive]()