--- title: "Issue 33" canonical: "https://current.tinyfish.ai/issue/33" publication: "TinyFish" publisher: "TinyFish" language: "en-US" guide: "https://current.tinyfish.ai/llms.txt" permalink: "https://current.tinyfish.ai/issue/33" issue: 33 description: "Benchmarks up, context windows longer, a thousand subagents running in parallel. The capacity to verify any of it is quietly disappearing." authors: ["Sable Whitford","Nora Kaplan","Rina Takahashi"] published: 2026-06-15 updated: 2026-11-05 articles: 101 --- # TinyFish > CURRENT is a journal about AI agents in production, for the engineering and product leaders who have to make them work. Drawing on experience building enterprise web agents, it explains why the demo succeeded and the deployment didn't, and what that means for choosing vendors, briefing the board, and avoiding mistakes others have already made. [Read on the website]() ## Issue 33 Benchmarks up, context windows longer, a thousand subagents running in parallel. The capacity to verify any of it is quietly disappearing. ### [Daily Brief]() Your daily digest of web developments through the lens of enterprise agent deployment - [Google Lets Schools Keep Search Without Gemini. The AI Overviews Come Along Anyway.]() — By [Sable Whitford]() · Published 2026-11-04 On Tuesday, which was also Election Day, Google announced a K-12 policy that switches Gemini, Gemini Notebook and **Search off by default** for schools using its education software. A Google education VP told the New York Times that "Search and AI are deeply integrated," so districts that ban AI would lose Search as well. Bernie Sanders called it **"extortion,"** and Sen. Richard Blumenthal and NYC Mayor Zohran Mamdani joined in. Within hours Google **partly walked it back**: schools can now turn on Search by itself, without Gemini. That Search still puts AI Overviews on the results page, though, so **the fight now is over what "AI off" means.** - **Proxy Traffic Leaving the White House Network?** — [Source]() — Published 2026-11-04 Layer3 Intel says residential proxy traffic exited AS2650, the Executive Office of the President's network, 236 times in 30 days through 28 proxy services. It says it wasn't even looking. Its guess is that someone working late has a device carrying a proxy SDK. Treat that as a theory: no device has been named, and six public IPs can sit in front of a lot of machines. - **DreamHost Had Uninvited Guests Since August** — [Source]() — Published 2026-11-04 DreamHost confirmed that attackers used a file manager bug to steal session tokens, customer emails and files, and that they got in back in August. That's a long stay at a host with about 400,000 paying customers. DreamHost says it's unrelated to PHP2Shell. If you're a customer, rotating credentials today is a sensible move. - **PHP2Shell's Hangover Gets a Sequel** — [Source]() — Published 2026-11-04 PHP2Shell was a pair of 9.2-severity bugs that put NASA, Wikipedia and the White House at risk. Two weeks after the patches, Kaspersky counts nearly 30 million servers still unpatched, plus a third, still-open flaw (CVE-2026-115560). Working exploits keep multiplying, though there's no confirmed mass-breach wave yet. Tolmo's Matt Suiche says AI bug hunting is exposing 40 years of legacy code, and that's today's big argument. AI can find a bug in minutes. Someone still has to book maintenance windows for 30 million servers. - **Can DNS Tell You It's a Bot?** — [Source]() — Published 2026-11-04 Singapore's IMDA is reportedly exploring, with GoDaddy, Cloudflare and Google, whether DNS could identify AI agents. That would give the internet's 1983 phone book a second job checking IDs at the door. Every ops person already knows where the postmortem will land: it's always DNS. - **Meta Buys Conductor, Cloud Users Get Notice** — [Source]() — Published 2026-11-04 Meta bought Conductor, the app for running several coding agents in parallel, and the team is joining Meta Superintelligence Labs. Terms weren't disclosed. Conductor Cloud goes read-only Nov. 13 and shuts off Nov. 20. The local app gets model updates and critical fixes until early 2027. Conductor ran agents from rival labs too, so everyone wants to know whether that survives the move. - **OpenAI Shops Too: Pierre Computer** — [Source]() — Published 2026-11-04 Not to be outdone, OpenAI told Axios it plans to buy Pierre Computer, which stores and manages AI-generated code. It's OpenAI's second coding pickup after the Ona deal announced in June. Agents are producing mountains of code nobody typed by hand, and someone has to keep track of it. OpenAI would like to be that someone. - **Kimi K3.1 Pro Weights Drop: Bring Disks** — [Source]() — Published 2026-11-04 Moonshot launched Kimi K3.1 Pro in October, and today the open weights landed on Hugging Face. It has 2.8 trillion parameters with 104B active, and reportedly ships as 96 shards totaling about 1.56 TB under a custom non-OSI licence. Self-hosting starts with a purchase order for storage. Before the release, Artificial Analysis said it would cut the gap between open and proprietary models from 12 points to 3. - **Ripple-65B Crowns Itself America's Coding Champ** — [Source]() — Published 2026-11-04 Ricursive calls its Ripple-65B Preview "U.S. Open-Weight SOTA on Agentic Coding," meaning best in its class. That's the company grading its own homework. The "U.S." qualifier is carrying extra weight today, since a Chinese lab just released 1.5 terabytes of open weights. - **Go Gets a Brand-New JSON API** — [Source]() — Published 2026-11-04 The Go team introduced encoding/json v2, a new official JSON API, and Gophers have plenty to say about it. Nearly every Go service reads or writes JSON somewhere, so a second version of the standard package lands on the desk of anyone shipping APIs. - **Apple Puts Engineers on an AI Diet** — [Source]() — Published 2026-11-04 Bloomberg reports that in October, Apple cut some engineers' daily AI spend caps from $300 to $200. Some could drop to $100, and certain non-engineering teams could be capped at $25. Some teams are being nudged toward cheaper or on-device models. The punchline: few people ever hit $300, and Apple's R&D rose 32% in fiscal 2026. - **Texas Voted, and Data Centers Flinched** — [Source]() — Published 2026-11-04 Last brief, the data center backlash was winning local ballot fights. Now it has the Texas governor's mansion, in a state holding 25%+ of planned US capacity. Gina Hinojosa ran on an immediate moratorium, tougher water and power rules, and no tax breaks, and AI data center stocks fell on the result. GLJ Research says about 13 GW is exposed. Altimeter's Brad Gerstner says 10–15 GW might never get built. Abbott had already paused permits. Musk, who spent big on the losing side, now likes "data centers in space." - **Big Tech Pulls Out the AI Credit Card** — [Source]() — Published 2026-11-04 Oracle filed to sell $20B of new stock and bonds for its AI buildout, and Meta is seeking about $25B in bonds. The timing is bold. They're raising money to build on the same day the state with the most planned capacity elected a governor who ran on a moratorium. - **ChatGPT Search Is 20% of Google's?** — [Source]() — Published 2026-11-04 OpenAI's Adam Fry said Monday that ChatGPT search volume is about 20% of Google's, and that weekly users reportedly went from 1B to 1.2B in 60 days. Check the counting rules first. Any prompt that triggers a web search counts, and OpenAI didn't disclose the Google baseline or its method. Ahrefs puts ChatGPT at 0.36% of web referral traffic. - [The Runnable Paper Needs an Identity Report]() — Published 2026-10-30 Paper2Agent, published in Nature in September, turns research papers into AI agents you can run. Hand over your data and the method executes, with no rebuilding it from prose. The authors want papers to carry an "agent availability" section saying whether a runnable version exists. - [Who Pays While the Agent Waits]() — Published 2026-10-30 Google's new service for hosting AI agents starts from a fact anyone who has watched one run already knew: most of the time, the agent is waiting, for the model to answer or for a person to reply. So Google suspends idle agents and promises you won't pay for idle machines. Fair enough, but the wait still gets paid for. It moves into approval queues nobody times, and then onto the plan, which comes back exactly as it left, even if the stock it counted on has since sold. - [Faria Huq and the Reason Behind a Human Step]() — Published 2026-10-30 When a person takes control from an AI agent driving their web browser, the system records a human step, but not why. Faria Huq and her Carnegie Mellon collaborators have built tools around these moments: an agent that can't find the right forum, a shopper who realized the brand mattered only after seeing the agent's pick. Their system learns to predict when someone will step in. The reason it sets aside would tell a team whether to fix the agent, the website, or its understanding of the person. - [Your Agent's Questions Need a Recipient and a Timer]() — Published 2026-10-30 When an AI agent stops mid-task to ask a human something, it often reads as a failure of autonomy. In one study, letting coding agents ask about bug reports stripped of key details won back most of the lost ground, lifting solve rates from about 55% to 69.4%. But the "human" answering was another model, holding the full report and standing by. Real people are in meetings, or asleep. The design work is in the ask: who gets it, what it says, and what happens while it waits. ### [Market Pulse]() Reading the agent ecosystem through a practitioner's lens - [The Test Suite Was the Hero]() — By [Nora Kaplan]() · Published 2026-06-12 · Format: Verification at Scale Anthropic's Dynamic Workflows can now orchestrate a thousand subagents in parallel. The showcase was a Bun runtime port: 750,000 lines of Zig to Rust, 99.8% test pass rate, eleven days. Impressive. But research on Claude's failure patterns finds that 71% of failures are consistent misinterpretation, not random error. A thousand agents inheriting the same flawed specification converge, confidently, on the same wrong answer. The Bun port had a deterministic test suite to catch that. Most enterprise domains don't. - **MCP Goes Stateless in Largest Revision Since Launch** — [Source]() — Published 2026-06-12 The May 21 release candidate for the 2026-07-28 spec strips out protocol-level sessions entirely. Any request can land on any server instance. A new Extensions framework introduces a formal 12-month deprecation lifecycle, with the final spec shipping July 28. Stateless transport turns horizontal scaling from an infrastructure specialty problem into a straightforward ops decision. - **Five Eyes Issues First Coordinated Agentic AI Guidance** — [Source]() — Published 2026-06-12 Six agencies across all Five Eyes nations published joint guidance on May 1, carving out five risk categories. The core directive: assume agents may behave unexpectedly, and prioritize resilience over efficiency. Prompt injection gets flagged as potentially unsolvable. Forrester already treats this as the de facto procurement floor for regulated-sector vendor evaluations, which means it sets the bar whether or not anyone intended it to. - **Google Declares the Agentic Gemini Era at I/O** — [Source]() — Published 2026-06-12 Gemini 3.5 Flash hit GA alongside Gemini Spark (always-on agents running on dedicated Cloud VMs), Antigravity 2.0 as an agent-first dev platform, and a WebMCP proposed standard co-developed with Microsoft. Capex for 2026 reaches $180-190B. The raw throughput numbers tell their own story: 3.2 quadrillion monthly tokens processed, 19 billion per minute via API. That kind of capacity commits Google to a world where agents are high-volume consumers of inference, not occasional callers. - **Microsoft Foundry Ships GPT-5.5 With Governance Wrapping** — [Source]() — Published 2026-06-12 GPT-5.5 reached general availability in Foundry alongside an 11,000+ model catalog, Agent Framework 1.0 GA, and Hosted Agents with per-agent Entra ID arriving by end of June. Build 2026 added OpenTelemetry tracing and an Agent Control Specification. The pitch is unusually explicit: the platform layer is governance wrapping frontier models, not the models themselves. Microsoft is selling the envelope, not just the letter inside it. - **Snowflake Bets Agents Belong Where Data Lives** — [Source]() — Published 2026-06-12 Cortex Agents now supports MCP integration, with 9,100+ customers using Snowflake AI products weekly. The strategic argument is pointed: agents operating inside Snowflake's governance boundary can access live data without shipping it to external stores. This positions agent control at the data layer rather than the identity layer, a structural counter to Microsoft's OS-level play. Where you govern the data, you govern the agent. - **EU AI Act Enforcement Powers Activate August Second** — [Source]() — Published 2026-06-12 Commission enforcement powers over GPAI model providers activate August 2, 2026, bringing authority to request documentation, conduct evaluations, and impose fines. High-risk AI system requirements kick in on the same date. The deployer/provider boundary stays murky for agent builders doing extensive fine-tuning. Coming three months after the Five Eyes guidance, summer 2026 marks the point where regulatory pressure goes from advisory to binding. - **Towards a Science of AI Agent Reliability** — [Source]() — Published 2026-06-12 Princeton's SAgE group tests 15 models across two benchmarks and finds capability gains barely register as reliability improvements. They propose 12 metrics, borrowed from safety-critical engineering, decomposing reliability into consistency, robustness, predictability, and safety. - **Beyond pass@1: A Reliability Science Framework for Long-Horizon LLM Agents** — [Source]() — Published 2026-06-12 Extends reliability measurement to task duration across 10 models and 23,392 episodes. Reliability decay is universal but sharply uneven. Software engineering tasks collapse at longer horizons while structured document processing holds nearly steady. - [When Agents Spawn Agents, the EU AI Act's Two-Role Framework Has No Answer]() — Published 2026-06-12 The EU AI Act sorts accountability into two buckets: providers build AI systems, deployers operate them. Article 3(1) treats every agent as an AI system. Straightforward enough when a human decides what gets deployed. - **Introducing Claude Opus 4.8 — Anthropic** — [Source]() — Published 2026-06-12 Primary source for Dynamic Workflows, including the hard limits and the Bun-to-Rust case study. The caveats are as revealing as the headline numbers. - **Towards a Science of AI Agent Reliability — Princeton (arXiv:2602.16666)** — [Source]() — Published 2026-06-12 The paper behind the pass^k findings, proposing twelve reliability metrics across consistency, robustness, predictability, and safety. The interactive dashboard at hal.cs.princeton.edu/reliability lets you explore the data across 15 models yourself. - **Careful Adoption of Agentic AI Services — CISA + Five Eyes Agencies** — [Source]() — Published 2026-06-12 First coordinated governmental security guidance targeting agentic AI. Thirty pages, five risk categories, and an unusually candid admission that evaluation methods are still evolving. Already being treated as a procurement floor. - **MCP 2026-07-28 Specification Release Candidate — Model Context Protocol Blog** — [Source]() — Published 2026-06-12 The largest revision since launch: stateless core, an Extensions framework, formal feature lifecycle policy. Every production MCP server will need a migration before the July 28 final spec. - **Google Is Pitching an AI Agent Ecosystem to Consumers Who May Not Buy It — TechCrunch** — [Source]() — Published 2026-06-12 A consumer-beat counterpoint to Google's I/O developer messaging. Useful for the gap it surfaces between platform ambition and what actual users can access today. - [What the Bill Can't Measure]() — By [Rina Takahashi]() · Published 2026-06-03 · Format: The Meter Drops GitHub Copilot moved to metered billing on June 1, and organizations can finally see where their AI development spend is going. That's a real step forward. The natural parallel is cloud FinOps, which saved billions by making waste visible. An idle VM is idle, and the bill reflects it. A confident, systematically wrong code suggestion consumes exactly the same credits as a correct one. Cost now has real infrastructure behind it, so organizations will build discipline around consumption. Consumption, meanwhile, has no reliable relationship to correctness. - [The Data Perimeter Split Both Architectures in Half]() — By [Rina Takahashi]() · Published 2026-05-27 · Format: The Perimeter Problem Last week, Anthropic and OpenAI both responded to the same brute constraint: regulated enterprise data cannot leave the building. Each vendor split the agent to accommodate that boundary. Anthropic moved tool execution inside the customer perimeter but kept the reasoning loop on its own infrastructure. OpenAI announced an on-premises partnership with Dell without specifying where inference actually runs. Neither architecture co-locates governance with the consequential decisions. When something goes wrong, the person who needs to intervene can kill the connection but has no way to inspect or redirect the reasoning that got there. - [What a Required Field Knew]() — By [Rina Takahashi]() · Published 2026-05-20 · Format: The Headless Shift A validation rule in Salesforce fires when a rep moves a deal to "Closed Lost" — won't save without a reason. Nobody calls that governance. It's just how the system works. But as enterprise software goes headless so agents can bypass the UI entirely, that quiet enforcement starts to matter. The interface was encoding what "correct" looked like, thousands of times a day, often without documentation. Now that specification work has to live somewhere else. Most organizations haven't started figuring out where. - [When Infrastructure Writes the Spec]() — By [Rina Takahashi]() · Published 2026-05-13 · Format: Specification by Default A support platform prices its AI agents per automated resolution: no escalation, ticket not reopened within 72 hours. Clean metric — one where a customer who gives up counts the same as one who's satisfied. Most organizations deploying agents haven't specified what "good" means in operational terms. Infrastructure is filling that vacuum. Pricing models, protocol defaults, eval frameworks each encoding their own answer. And the specification decision already happened, whether anyone noticed or not. ### [Foundations]() Conceptual clarity earned from building at scale - [The Number You Didn't Ask About]() — By [Nora Kaplan]() · Published 2026-06-11 · Format: Reliability Lens An agent succeeds at a task 70% of the time. Frame the results one way and you can report 97%. Frame them differently and you report 34%. Same system, same math, a 63-point gap created entirely by which question the benchmark decided to answer. Nearly every major agent benchmark chooses the generous framing. The number that captures what production actually feels like, consecutive successes across a full workload, is one that almost nobody reports. - [The Accessibility Tree Wasn't Built for AI Agents]() — By [Rina Takahashi]() · Published 2026-06-11 · Format: Perception Walkthrough Accessibility trees were built for screen readers, not AI agents. But both share the same constraint: neither can see a web page. Both need it translated into structured, sequential text. Playwright MCP builds its entire browser automation architecture around that convergence, using accessibility snapshots that cost a fraction of what screenshots require per interaction. The token savings are real and they compound across multi-step workflows. So does the downside: on 95.9% of the web's top million pages, the underlying markup has detectable failures. The agent inherits every one of them. - **The Three Architectures of Browser Agents** — [Source]() — Published 2026-06-11 Maps the full perception design space the walkthrough occupies one corner of. - **Demystifying Evals for AI Agents** — [Source]() — Published 2026-06-11 A layered eval methodology for teams ready to move from measuring reliability to building around it. - **The Reliability Gap: Agent Benchmarks for Enterprise** — [Source]() — Published 2026-06-11 Extends pass^k into category-level verdicts on which agent tasks are production-ready today. - **How AI Agents See Your Website: The Accessibility Tree Explained** — [Source]() — Published 2026-06-11 The web-quality reality check behind accessibility-tree agents, with data on how broken the substrate actually is. - **pass@k and pass^k Measure Completely Different Things** — [Source]() — Published 2026-06-11 Confused: treating "succeeded in k tries" and "succeeded every try" as variations of the same score. They aren't. An agent with 70% per-trial success hits ~97% on pass@3 (capability) but just 34% on pass^3 (reliability). Same agent, opposite conclusions. - **Injection and Poisoning Enter Through Different Doors** — [Source]() — Published 2026-06-11 Confused: lumping both under "prompt attacks." Prompt injection is an input-validation problem where malicious instructions slip through unsanitized user content. Tool poisoning is a supply-chain problem where server-side metadata the agent trusts was authored by an attacker. Defending the input channel does nothing about the supply chain, and vice versa. - **Who Decides the Next Step: Code or Model** — [Source]() — Published 2026-06-11 Confused: calling anything that uses an LLM an "agent." A system that calls GPT-4 ten times in a fixed sequence is a workflow if code controls the routing. A system where the model picks its own tools and loops is an agent. The dividing line is control architecture, not model size or capability. - **MCP Standardizes Plumbing, Not Policy or Governance** — [Source]() — Published 2026-06-11 Confused: assuming a standard integration protocol handles authorization and security. MCP defines how agents discover and call tools. It does not enforce least privilege, scope tokens, define audit policy, or verify tool quality. Authorization remains optional in the spec. Trend Micro found 492 MCP servers exposed with zero authentication. - **Durable Execution Survives Crashes, Regular Execution Doesn't** — [Source]() — Published 2026-06-11 Confused: assuming a workflow that runs will keep running. Execution means the code ran. Durable execution means it checkpointed progress, survived a crash, and resumed without duplicating side effects. A server restart at 3am either loses the customer's half-finished task or retries and doubles the order. That's the gap. - **Browser Agents Inherit the User's Full Privilege Surface** — [Source]() — Published 2026-06-11 Confused: treating browser-based and API-based agents as interchangeable approaches. API-first agents call documented endpoints with typed inputs and narrow contracts. Browser-first agents operate through the user's most privileged interface, crossing app boundaries and acting with whatever credentials the session holds. The privilege surface is everything a logged-in user can reach. - **SWE-bench Verified: Real GitHub Bug Fixes** — [Source]() — Published 2026-06-11 Agents tackle human-validated software engineering tasks drawn from actual GitHub issues. Leading score hit 74.4% by end of 2025, though scores have climbed further since, accompanied by growing contamination concerns. Reports pass@k only. Check the primary leaderboard for current numbers. - **GAIA Pushes Multi-Step Reasoning With Tools** — [Source]() — Published 2026-06-11 Tests real-world assistant tasks that chain together web browsing, tool use, and multi-step reasoning across three difficulty tiers. Orchestrated agent systems reached ~90% by end of 2025. Bare models score far lower. Reports pass@k only. - **τ-bench Exposes What Repeated Runs Reveal** — [Source]() — Published 2026-06-11 Policy-adherent tool use in retail and airline customer service. Agents hit 80% on a single attempt, but pass^8 drops below 25% in original experiments. The only benchmark here reporting both capability (pass@k) and reliability (pass^k). Worth sitting with that gap. - **WebArena Navigates Realistic E-Commerce and SaaS** — [Source]() — Published 2026-06-11 Multi-step web navigation across e-commerce sites, forums, and content management systems. GPT-4's original baseline sat at 14.41%. Recent agents clear 60%, with top entries approaching 69% by mid-2026. Reports pass@k only. - **BFCL Scores Tool Calling Across Languages** — [Source]() — Published 2026-06-11 Function-calling accuracy evaluated via AST matching across Python, Java, JavaScript, and REST APIs. Leading score around 77.5% at end of 2025, now updated to V4 with agentic evaluation categories. Reports pass@k only. - **TheAgentCompany: Full Workplace Autonomy, Humbling Results** — [Source]() — Published 2026-06-11 Simulated workplace tasks that blend web browsing, coding, program execution, and communication with simulated coworkers. The best agent completed 30% of tasks in the NeurIPS 2025 version. Comfortably the hardest benchmark in this set. Reports pass@k only. - [What a Website Chooses to Tell an Agent]() — By [Rina Takahashi]() · Published 2026-06-03 · Format: WebMCP Walkthrough A new browser protocol lets websites publish structured tool definitions that agents can call directly, skipping the page entirely. A site registers a product search with four categories in the enum. The actual catalog has fifty. The agent receives that simplified view as the complete picture and has no mechanism to ask what else exists. When a website decides what to tell an agent about itself, it's also deciding what disappears. - [Where You Can Afford to Be Surprised]() — By [Rina Takahashi]() · Published 2026-06-03 · Format: The Control Question "Agent" means at least three different things right now, and the confusion isn't just semantic. Underneath the terminology sits a design decision with real operational consequences: who picks the next step, your code or the model? Code-determined paths produce bounded, explainable failures. Model-determined paths produce creative ones you didn't plan for. Most production systems need both. The hard part is drawing the line between them, and it's a risk-tolerance decision that happens to look like an architecture one. - [What Browser Agents See Before They Think]() — By [Rina Takahashi]() · Published 2026-05-28 · Format: The Perception Layer Before a browser agent reasons about a task, it needs a representation of the page. Raw HTML, a screenshot, or the accessibility tree. That choice, made once and early, sets the token budget, failure modes, and reliability ceiling for everything downstream. OpenAI, Microsoft, and Perplexity built their agent systems independently and all three landed on the same primary perception layer: infrastructure originally designed for people who can't see screens. The convergence is worth understanding. - [Where Agent Behavior Actually Gets Decided]() — By [Nora Kaplan]() · Published 2026-05-28 · Format: Context Over Instruction A developer's coding agent worked perfectly for thirty minutes, then started contradicting its own decisions. The prompt was the same. The model was the same. Everything else in the context window had been accumulating. His fix was radical: stop carrying context forward entirely. It worked. Across production agent systems, the same asymmetry kept surfacing — a carefully engineered prompt drowning in accumulated tool outputs performed worse than a simple prompt in a clean, focused context. The field needed a word for the thing practitioners were actually optimizing, because "prompt" didn't describe it. ### [Practitioner's Corner]() Lessons from the field—what we see building at scale - [APIs for Websites That Don't Have APIs]() — By [Rina Takahashi]() · Published 2026-06-11 · Format: Builder Profile Suchintan Singh describes Skyvern as building "APIs for websites that don't have APIs." It sounds like a pitch deck line. But the phrase does real work if you've spent years watching ML platforms get measured by marketplace revenue, not model accuracy, and Selenium scripts break every time a frontend developer renames a CSS class. His team built a harder benchmark, scored below 50% on it, and published the results anyway. The architecture and the measurement both point the same direction: web agents are an infrastructure problem that the AI framing keeps obscuring. - [The Wall That Used to Be There]() — By [Rina Takahashi]() · Published 2026-06-11 · Format: Enforcement Gap An agent running on an 8K-context model that entered a loop used to crash. The window filled, the API errored, the run died. That was an accidental safety mechanism, and it worked. Context windows now stretch past a million tokens, and the accidental wall is gone. A recent multi-agent loop ran for eleven days and $47,000 before a human noticed the billing dashboard. The team had monitoring. They could see it happening. They couldn't stop it programmatically. Watching and stopping, it turns out, require entirely different infrastructure. - [The Crash That Stopped Coming and the Engineer Who Had to Replace It]() — Published 2026-06-11 *Dag Halvorsen is a platform engineer at a mid-size logistics company who, until recently, was sleeping through the night. He spent six years on Kubernetes autoscaling before moving into agent orchestration, a transition he describes as "going from herding cattle to herding cats that occasionally hallucinate." We spoke over video call. A Post-it note stuck to his monitor read "RAN ≠ SUCCEEDED" in red marker. It had clearly been there a while.* - [The Compound Failure Rate: Why 85% Per-Step Accuracy Means 80% Workflow Failure]() — Published 2026-06-11 P(success) = a^m. Per-step accuracy raised to the number of sequential steps. At 85% accuracy across 10 steps, you land at 19.7% end-to-end success. Bump to 95% per step and a 10-step workflow still only clears 60%. - **Checkpoints Are Not Durable Execution: Why LangGraph, CrewAI, Google ADK Fall Short** — [Source]() — Published 2026-06-11 Why agent framework save points can't be incrementally improved into durable execution - **AI Agent Token Budget Enforcement** — [Source]() — Published 2026-06-11 A $47,000 runaway loop shows why budget alerts without runtime enforcement are a category error - **AI Agents Burn 50x More Tokens Than Chats** — [Source]() — Published 2026-06-11 Audit of 30 engineering teams revealing a 20x cost spread between developers on the same tools - **State of Agent Engineering (2026 Survey)** — [Source]() — Published 2026-06-11 Primary-source data from 1,300+ practitioners on the observability-without-evals gap in production - **AI Agent Retries and Idempotency: Tool Failure Patterns** — [Source]() — Published 2026-06-11 How standard HTTP retry logic creates duplicate writes in agent tool calls and what fixes it - [Shuchang Zheng and the Architecture That Notices]() — By [Rina Takahashi]() · Published 2026-06-03 · Format: Builder Profile Skyvern's architecture compiles AI-learned browser workflows into deterministic scripts, then calls the model back only when a script breaks. The rationale is cost and reliability — compiled paths cut token consumption by 57× at scale. But each time the model gets called back, it timestamps something else: an environment event. The website changed enough to break a path that previously worked. Co-founder Shuchang Zheng spent years at Lyft building infrastructure that told engineers when something changed and whether it mattered. His browser automation architecture produces the same kind of signal as a byproduct, pointed at the web rather than the agent. - [Governing the Hypothetical]() — By [Nora Kaplan]() · Published 2026-06-03 · Format: Governance Before Catastrophe Airworthiness rules carry the fingerprints of specific crashes. Banking regulations followed specific crises. The Five Eyes agentic AI guidance and NSA's MCP security advisory, both published in May 2025, have no crashes to study. They govern systems that barely exist in production, and both documents say so plainly — evaluation methods aren't mature enough to certify what's being built. Critical safety decisions, like whether an operation that runs twice produces the same result, are left to implementers. When regulators publish hypotheses instead of prohibitions, the compliance machinery that receives them may not notice the difference. - [Between Complete and Correct]() — By [Rina Takahashi]() · Published 2026-05-28 · Format: Builder Profile An agent asked to add a specific sneaker to a cart navigated Amazon, found a shoe, added it, and reported success. It was the wrong shoe. Every step completed. The task failed. Browser Use co-founders Gregor Zunic and Magnus Müller have made two moves that trace this gap between "complete" and "correct": a migration to raw Chrome DevTools Protocol for real-time browser visibility, and an unsolved problem listed quietly on their hiring page in five words. One reaches for more observation. The other, reproducibility. Neither closes the gap. - [Retries Were Built for a Deterministic World]() — By [Rina Takahashi]() · Published 2026-05-27 · Format: Reliability vs. Correctness A request times out. You send it again. Retry logic is so fundamental to distributed systems that most frameworks ship it by default. It works because the retried operation is identical to the original. Replace the executor with an LLM, and that assumption dissolves. The agent may reason differently on the second attempt, choose different tools, take a different path. Deterministic side effects, produced by a nondeterministic executor. Reliability infrastructure, doing exactly what it was designed to do, starts producing incorrect outcomes. ### [Echoes]() Past infrastructure decisions echoing in today's production realities - [The Confused Deputy Keeps Getting Promoted]() — By [Rina Takahashi]() · Published 2026-06-11 · Format: The Oldest Deputy In 1977, a Fortran compiler overwrote a billing file because a user asked it to. Nobody hacked anything. The compiler just couldn't tell its own authority apart from the user's request. Norm Hardy named this the "confused deputy" in 1988. The web recreated the pattern with session cookies. OAuth recreated it with delegated tokens. Each time, the fix worked: teach the deputy to distinguish instruction from data. Now LLM agents carry the same structural problem, and the proposed fix is four decades old. This time, the deputy may not be capable of learning. - [Durable Execution: The Rare Case Where Old Knowledge Traveled With the People Who Built It]() — Published 2026-06-11 Most threads in this section trace knowledge that got lost or reinvented blind. Durable execution is the counter-example. - [The Answer That Kept Losing — An Imagined Interview with Norm Hardy]() — Published 2026-06-11 *Norm Hardy died before the industry caught up with him. The computer scientist who named the confused deputy problem in 1988 spent decades advocating capability-based security as the fix. He never saw Google DeepMind name a framework after capabilities, or watched the Model Context Protocol independently derive audience-bound tokens as though nobody had thought of them before.* - [The Operational Inheritance]() — By [Rina Takahashi]() · Published 2026-06-11 · Format: Two Kinds of Debt Browser automation spent twenty years learning to distrust the gap between issuing a command and the command doing what was intended. Selenium engineers built workarounds. Playwright encoded them into architecture. Web agents now operate on the same substrate, facing the same race conditions, the same invisible overlays, the same intercepted clicks. They inherited all of it. The hard-won instinct to doubt when something reports success stayed behind with the community that earned it. - [The Organizational Inheritance]() — By [Rina Takahashi]() · Published 2026-06-11 · Format: Two Kinds of Debt RPA's governance problems didn't arrive all at once. They accumulated quietly: orphaned bots running on departed employees' credentials, exception-handling logic that lived in one person's head, audit trails nobody maintained. The questions were always answerable. Who owns this? What happens when it fails? Organizations often didn't ask until something broke. AI agents face the same questions now. The teams deploying them are rarely the teams that lived through the answers. - **Confused Deputy Attacks on Autonomous AI Agents** — [Source]() — Published 2026-06-11 CSA traces Hardy's 1988 ambient-authority flaw through to AI credential delegation. The debt compounds with each layer of autonomy. - **Prompt Injection May Never Be Fixed, Warns NCSC** — [Source]() — Published 2026-06-11 NCSC compares prompt injection to SQL injection, then delivers the worse news: this time there's no parameterized query waiting in the wings. - **The Future of RPA Ties to AI Agents** — [Source]() — Published 2026-06-11 IDC projects $8.2B in RPA spending through 2028. AI agents aren't replacing that stack. They're inheriting it. - **Shadow AI Morphs Into Shadow Operations** — [Source]() — Published 2026-06-11 Shadow IT took a decade to produce a governance playbook. Shadow AI agents act on live systems before anyone thinks to write one. - **How the Confused Deputy Problem Has Made a Comeback** — [Source]() — Published 2026-06-11 The 1988 confused deputy was contained, never resolved. AI agent teams are rediscovering it without the institutional memory of why it mattered. - **After the Identity Fix: MCP's Confused Deputy Problem** — [Source]() — Published 2026-06-11 Thirty-eight years separate Hardy's paper from OWASP's agentic security classification. The latest identity patch still doesn't touch the structural flaw underneath. - [Consequences as Architecture]() — By [Nora Kaplan]() · Published 2026-06-03 · Format: Frames and Fixes If voluntary coordination almost never closes a severed feedback loop, what has? HTTPS adoption and GDPR enforcement are the clearest cases, and they share a specific shape: someone occupied a chokepoint, reduced the cost of compliance beforehand, and escalated consequences gradually rather than delivering them as a single absorbable event. Most broken feedback loops lack all three conditions. Most institutions would rather convene a working group. - [The Frame That Wins]() — By [Nora Kaplan]() · Published 2026-06-03 · Format: Frames and Fixes When ai.txt was proposed as a successor to robots.txt, two companies signed on. The three that mattered didn't. The proposal was well-designed. It was also a coordination mechanism applied to a power asymmetry. The same pattern shaped RPA governance a few years earlier. In both cases, the winning frame was held by people insulated from the failure it described. That insulation is what keeps the frame in place. - [The Distance Between Breaking and Fixing]() — By [Rina Takahashi]() · Published 2026-06-03 · Format: The Severed Loop When a browser agent can't click the right date on a booking page, the failure log blames the model. The page was already broken for screen reader users, years before agents existed. Screen readers learned to compensate so well that developers never got the signal to fix anything. Now agents have plugged into the same infrastructure and inherited two decades of unrepaired damage. Each workaround, built by the people closest to the pain, quietly widened the gap between breaking and fixing. - [The Session That Assumed You Were There]() — By [Nora Kaplan]() · Published 2026-05-27 · Format: Two Workarounds Agents Straddle In 1994, a Netscape engineer built cookies to remember your shopping cart and specifically rejected proposals to turn browsers into universal identity beacons. Thirty years later, the session token does exactly what he refused to build — it passes for proof of who you are. The browser's security model still assumes a human is making choices about where to go. That assumption is quietly breaking. ### [Vision]() Where human-AI collaboration is heading - [The Quiet Part]() — By [Nora Kaplan]() · Published 2026-06-11 · Format: The Core Problem In a controlled trial, experienced developers using AI tools estimated they'd gotten about 20% faster. Measured result: 19% slower. What lingers is that completing the work, experiencing the full arc of it, didn't correct their perception. Meanwhile, 74% of teams running agent systems in production rely on human reviewers as their primary quality gate. Both findings are solid. Together, they describe something uncomfortable about where organizational AI deployment is actually headed. - [What Aviation Built After It Noticed the Problem]() — By [Nora Kaplan]() · Published 2026-06-11 · Format: Lessons and Designs Aviation solved the automation-complacency problem decades ago with simulator checks, mandatory manual flying, and crew resource management. Those countermeasures rest on a foundation enterprise AI lacks: a stable, agreed-upon definition of what correct human performance looks like. A pilot's hand-flown approach can be graded against known parameters. An analyst's judgment on a market entry recommendation cannot. That structural difference determines whether countermeasures are even possible. - [Monitoring the Monitors]() — By [Nora Kaplan]() · Published 2026-06-11 · Format: Lessons and Designs Organizations are building increasingly sophisticated systems to monitor whether their AI agents perform well. Almost none are checking whether the humans evaluating those agents still can. Evidence from the few domains where measurement exists — developer productivity, medical diagnostics — reveals a durable gap between perceived and actual human capability. The instrument to detect oversight degradation needs to be built now, while the people who could design it still have the expertise to know what it should measure. - [The Woman Who Breaks Her Hospital's AI on Purpose]() — Published 2026-06-11 *Noor Breakpoint is not a real person, though if she were, she'd probably object to us pointing that out before she'd had a chance to make her case. Her title, Director of AI Resilience Testing at a mid-size healthcare network, describes a role that barely existed two years ago and that most health systems still haven't created. Her small team, which she calls "Red Cell" after military adversarial testing units, does something that sounds simple and turns out to be deeply uncomfortable: they deliberately plant subtle errors into AI-assisted clinical and operational workflows, then watch to see if anyone catches them.* - [The Evidence for AI-Induced Deskilling Is No Longer Theoretical]() — Published 2026-06-11 Polish researchers studying AI-assisted colonoscopy weren't looking for deskilling. They found it anyway. After endoscopists gained routine access to AI polyp detection, their unassisted adenoma detection rate fell from 28.4% to 22.4% across four centres and nearly 1,500 procedures. The degradation only surfaced when the tool was taken away. With it, performance held steady. - **What Developer Oversight of AI Agents Actually Looks Like** — [Source]() — Published 2026-06-11 Empirical research finds four distinct forms of oversight work. Most governance frameworks assume one. - **The Guardrail That Belongs Around the Human, Not the Model** — [Source]() — Published 2026-06-11 Makes the case that human attention is a finite organizational resource nobody is actually managing. - **The AI Adoption J-Curve and Why the Dip Looks Like Normal** — [Source]() — Published 2026-06-11 MIT Sloan-backed research on why AI's temporary productivity losses can pass as routine friction for months. - **What Aviation Learned About Oversight That Enterprise AI Hasn't** — [Source]() — Published 2026-06-11 A sharp distinction between placing someone in the loop and training them to function there. - **The Governance Gap Executives Can't See** — [Source]() — Published 2026-06-11 Nearly two-thirds of organizations apply weaker controls to AI agents than to human employees. - [What the Six Percent Are Doing Differently]() — By [Nora Kaplan]() · Published 2026-06-04 · Format: Two Consequences While the apprenticeship pipeline thins, a different question is surfacing at the organizational level: what does work look like when you design it around agent capabilities instead of bolting agents onto legacy processes? Six percent of leaders report progress on this. The other 94 percent are watching gains plateau. What separates the two groups is something more mundane than technology: someone asked what the work actually is before automating it. - [Where Does Judgment Come From Now?]() — By [Nora Kaplan]() · Published 2026-06-04 · Format: Two Consequences Entry-level job postings have fallen 35 percent since early 2023. The tasks vanishing from junior roles—debugging, data reconciliation, document review—look like busywork from the outside. They were also how professionals learned what "normal" looks like in their field. The development pipeline hollowed out one rational automation decision at a time. Now the experiment in whether judgment can form without apprenticeship is already running, undesigned. - [The Boring Turn]() — By [Nora Kaplan]() · Published 2026-06-04 · Format: The Boring Future A year ago, the enterprise AI conversation was about capability — whether agents could navigate websites without hallucinating. That conversation has moved on. The top budget line is now observability tooling. Over half of enterprises have a formal agent operations lead. The questions sound like infrastructure management because they are. And once something becomes operational, the difficulty changes character entirely. The organizational questions that emerge are ones most deployment playbooks aren't built to detect. - [When Smooth Work Feels Like Good Work]() — By [Nora Kaplan]() · Published 2026-05-28 · Format: Window and Mechanism Experienced developers predicted AI made them 24% faster. Actual result: 19% slower. After completing the work, with their own timing data in hand, they still believed AI had helped. That 39-point gap between perceived and actual performance points to something structural about how humans read fluency as accuracy. For organizations building AI-augmented workflows right now, it's also a design problem with concrete solutions. ## More - [Publication guide]() - [Issue archive]()