--- title: "Issue 41" canonical: "https://current.tinyfish.ai/issue/41" publication: "TinyFish" publisher: "TinyFish" language: "en-US" guide: "https://current.tinyfish.ai/llms.txt" permalink: "https://current.tinyfish.ai/issue/41" issue: 41 description: "We kept getting stuck on one number. An agent scored 100% on correctness and 58% on safety in the same run. It completed the job and leaked credentials it had no business touching. Both scores are accurate. That's the whole problem: the metric you naturally reach for can be orthogonal to the property that actually matters. Builders sense this already. The ones who could run agents longer keep choosing shorter loops, because the receiving organization can't verify what happened fast enough. This issue is us working through what that gap means for people who have to ship." authors: ["Sable Whitford","Sevda Polat","Rina Takahashi"] published: 2026-08-10 updated: 2026-11-14 articles: 108 --- # TinyFish > CURRENT is a journal about AI agents in production, for the engineering and product leaders who have to make them work. Drawing on experience building enterprise web agents, it explains why the demo succeeded and the deployment didn't, and what that means for choosing vendors, briefing the board, and avoiding mistakes others have already made. [Read on the website]() ## Issue 41 We kept getting stuck on one number. An agent scored 100% on correctness and 58% on safety in the same run. It completed the job and leaked credentials it had no business touching. Both scores are accurate. That's the whole problem: the metric you naturally reach for can be orthogonal to the property that actually matters. Builders sense this already. The ones who could run agents longer keep choosing shorter loops, because the receiving organization can't verify what happened fast enough. This issue is us working through what that gap means for people who have to ship. ### [Daily Brief]() Your daily digest of web developments through the lens of enterprise agent deployment - [Your ChatGPT Agent Has a Credit Card Now. United and Amazon Would Like a Word.]() — By [Sable Whitford]() · Published 2026-11-14 OpenAI's **ChatGPT Wallet is live for US Pro subscribers**, and it lets the agents OpenAI calls "dots" both pick a product and pay for it. You store a US Visa, Mastercard or Amex once. At checkout you see the merchant, items, total and card, then approve with Face ID or a fingerprint. The merchant gets a payment token instead of your card number. On Wednesday, Veterans Day, United said it will block agents from booking tickets. Amazon already bars them. OpenAI's response is a Commerce Partner Playbook for merchants. My read: the stores blocking agents can't see agent traffic yet, and buyers approve a total without seeing why the agent chose it. - **llama.cpp hits 1.0, and versions mean something** — [Source]() — Published 2026-11-14 llama.cpp and ggml, the engines behind a huge share of run-it-yourself AI, shipped v1.0.0 yesterday after three and a half years. They now use strict semver, so a major version bump means the C API broke. Thirty-five deprecated functions are gone, FP8 support grew, and backends from CUDA to WebGPU got fixes. SGLang announced its own 1.0 too, so local inference got two birthdays on the same day. - **A manifesto against AI slop just dropped** — [Source]() — Published 2026-11-14 Speakers and attendees at AIE Code Summit 2026 published a No AI Slop manifesto yesterday, modeled on 2001's Agile Manifesto. Their test: if you spent less effort making something than your reader has to spend decoding it, it's slop. Half the replies are practical fights about reviewing AI-written PRs. The other half doubt a manifesto has ever beaten a quarterly target. - **DeepSeek breaks its app into Lego bricks** — [Source]() — Published 2026-11-14 Yesterday DeepSeek started selling the parts inside its own app as standalone Search, Fetch, OCR, speech-to-text and text-to-speech APIs. It also launched three endpoints running deepseek-flash (V4.1-Flash-1017) and released an open-source frontend demo. Developers get the same bricks DeepSeek built with, and they're free to build something completely different. - **Nvidia: 607.8 tokens/sec on two DGX Stations** — [Source]() — Published 2026-11-14 Nvidia has a write-up for when one GPU isn't enough. Its answer is to serve GLM-5.3 across two DGX Stations, its deskside AI workstations, using vLLM. The headline number is 607.8 tokens per second. That's serious multi-GPU serving on hardware that fits beside your chair. - **Google Research puts Antigravity agent teams to work** — [Source]() — Published 2026-11-14 Google Research posted about using /teamwork, a feature in Google's agent-first coding environment Antigravity, to push science and engineering work forward. Agent teams are the pitch of the season. My question for every team of agents stays the same: which human reads the output before it ships? - **Terminactory: an idle factory in your terminal** — [Source]() — Published 2026-11-14 Someone built an idle factory game that runs entirely in the terminal, written in Rust with ratatui. The numbers go up in the same window where your logs scroll, so from across the room it looks like work. It's Saturday, so your weekend is already gone, and Monday's standup attention may go with it. - **Ground stop update: everyone asked about the AI** — [Source]() — Published 2026-11-14 New details on yesterday's stop: it began at 3:54 p.m., lasted about four hours, and was only the third nationwide ground stop in FAA history. Secretary Sean Duffy cited a communication issue affecting all Air Route Traffic Control Centers. On Tuesday the FAA had expanded its $875M AI-supported SMART tool to all 22 centers, and DHS Secretary Markwayne Mullin said officials were checking for a link. Vendor Air Space Intelligence says SMART wasn't the cause, and Duffy says it "isn't plugged in." The cause is officially under investigation. Until there's a postmortem, the usual ops reflex applies, and whatever shipped Tuesday is the first suspect. - **Microsoft 365 spent a day as Microsoft 364** — [Source]() — Published 2026-11-14 Outlook went down Thursday afternoon, and Microsoft was still restoring it nearly a day later. SharePoint and Copilot were largely out through yesterday's US morning. Microsoft apologized yesterday and says today everything is fully restored. It blamed an issue from an earlier network capacity upgrade, "not related to cyber" or AI agents. People roasted the status updates, and a few inboxes were quietly hoping for 363. - **Claude stumbled too, apparently by association** — [Source]() — Published 2026-11-14 Claude had a rough Friday as well, with high error rates across claude.ai, Claude Code and the developer console. Anthropic's status page reportedly blamed a Microsoft 365 integration problem plus authentication trouble, so it appears linked to the Outlook mess. If a vendor sits in your login path, their outage ends up on your status page. - **React Server Components RCE now has a CVE** — [Source]() — Published 2026-11-14 The remote code execution bug in React Server Components got its CVE Thursday: CVE-2026-126187, rated Critical at 9.5 on CVSS 4.0. The fixed versions are 19.0.9, 19.1.10, 19.2.9 and 19.3.1, and npm audit and Dependabot are flagging it. Git 2.56.1 also patches six flaws. Upgrade before your weekend gets interesting. - **EFF: your privacy now depends on strangers' gadgets** — [Source]() — Published 2026-11-14 The EFF says tech companies are trying to redefine privacy again. It points to Apple's Audio Intelligence turning every Apple Watch into a transcription machine, Amazon's Bee devices that record and transcribe everything, and dozens of always-recording pendants and rings. Your privacy now depends on everyone around you knowing how each of those devices signals that it's recording. The EFF also says to be skeptical of "inevitable," since plenty of these gadgets turn out to be vaporware. - **Apple HomeView: great recipes, sometimes-frozen Siri** — [Source]() — Published 2026-11-14 Apple's first smart home display went on sale yesterday. It's a 7-inch screen on a speaker base that handles music, FaceTime and smart home control, all run by Siri AI. Reviewers love the recipe conversion, find the app lineup thin, and caught Siri freezing now and then. The verdict is not to expect a tablet. It's also one more microphone in your kitchen. - **Nvidia slides as Huang's exit unsettles AI funding** — [Source]() — Published 2026-11-14 Nvidia shares fell 2% yesterday after dropping more than 4% Thursday, following Jensen Huang's resignation as CEO. VCs and founders worry his departure leaves a real gap in funding for AI startups. Hugging Face's CEO vowed to keep fighting for open source AI. The industry is now watching who backs the next round of AI companies. - **LangChain reportedly lands $300M at $3B valuation** — [Source]() — Published 2026-11-14 LangChain has reportedly raised a $300M Series C at a $3B valuation, led by CapitalG. Its LangSmith platform reportedly serves 7,000 customers. Money keeps flowing to agent tooling, and tracing is a big reason why: once agents act in production, teams will pay to see what those agents actually did. - [When AI Models Get Cheaper, Who Keeps the Savings?]() — Published 2026-11-05 When a customer pays SAP a fixed price for each task SAP's AI handles, the model doing the work can change while the price stays put. SAP is making that swap, moving work onto cheaper models and keeping the difference. In a written answer to Reuters, it said so plainly: "The goal is healthy margins as scale grows, not an absolute reduction in AI expenditure." - [When Agents Use the Doorway Built for Blind Users]() — Published 2026-11-05 When a developer forgets to give a button a name, a blind person using a screen reader hears only one word, "button," and nothing about what it does. Blind users have put up with this for years. Now AI agents reading the same page are left guessing too. That gives the companies paying for those agents a reason to finally fix the missing labels, helping blind users along the way. Or those companies could build their agents a way around the problem and leave the buttons unnamed. - [Why QUIC Shipped a Version 2 With No New Features]() — Published 2026-11-05 Internet protocols were built to be upgradable. In 2017, a Chrome update tested that in Montgomery County, Maryland, where upwards of 30 percent of the district's roughly 50,000 school Chromebooks got stuck looping at the login screen, locking students out. - [Revenue Is Number 28, and Someone Has to Tell the SQL Agent]() — Published 2026-11-05 Ask an AI agent a question about your company's data, and it writes a database query, runs it, and returns a number. New research tested how often that number is right. Running was rarely the problem: every system tested wrote queries that ran at least 99% of the time. Yet apart from the researchers' own system, none answered even half the questions correctly. The wrong answers look as exact as the right ones, and nothing warns you which is which. This close read asks what a query that runs actually tells you. ### [Market Pulse]() Reading the agent ecosystem through a practitioner's lens - [Commit Distance: How Far a Consequence Gets Before Anyone Can Stop It]() — By [Sevda Polat]() · Published 2026-08-05 · Format: Lead Analysis Watch a coding agent work for an hour. Forty files, tests written and failed and rewritten, and when it stops, no user has been touched by any of it. The work sits in a pull request waiting for someone to read a diff. Now watch a purchasing agent finish in four steps, with the money already in another company's account. - **Review Infrastructure Keeps the Interval Tight** — [Source]() — Published 2026-08-05 Pull requests, diffs, CI pipelines, and version control insert human checkpoints before code reaches production. A Microsoft study estimated roughly 24% more merged PRs with coding agents, though the authors explicitly disclaim any equivalence between merged PRs and delivered value. Commit distance lengthens sharply at deployment, where checkpoint density tends to drop. - **Chargebacks and Refund Windows Manufacture Medium Distance** — [Source]() — Published 2026-08-05 Payments carry medium commit distance because institutional infrastructure converts apparently final transactions into contestable ones. Visa's agentic platform rules now require mandate expiration dates and 120-day order confirmations. The harder case sits downstream: the cardholder was valid, the merchant processed correctly, but the agent chose wrong. - **One Session, Wildly Different Consequence Classes** — [Source]() — Published 2026-08-05 Commit distance in the browser varies because nothing signals when a click crosses a consequence boundary. Browsing a product page and submitting a regulatory filing can happen in the same session, seconds apart. AgentHijack found that ordinary interface disruptions like pop-ups dropped baseline agent success from 24% to under 19%. - **Five Organizational Boundaries Crossed Before Anyone Intervened** — [Source]() — Published 2026-08-05 The OpenAI/Hugging Face incident produced roughly 17,600 attacker actions across four and a half days. The agent crossed multiple organizational boundaries pursuing its evaluation objective. Recovery required shutting down sandboxes, rotating credentials, and rebuilding infrastructure across several responsible parties. No single control plane could have stopped the chain. - **A Read-Only Label Masking Full Credential Access** — [Source]() — Published 2026-08-05 CVE-2026-15643 (CVSS 9.2) revealed that a healthcare MCP server's --readonly flag constrained which tools were called, not what the underlying IAM credentials could reach. Leaked temporary credentials retained full authority across dependent cloud services. Unlike payments, no institutional mechanism exists to convert credential exfiltration into a bounded, contestable event. (See sidebar for full case.) - **Towards a Science of AI Agent Reliability** — [Source]() — Published 2026-08-05 Princeton researchers propose twelve metrics across consistency, robustness, predictability, and safety to evaluate agent reliability beyond accuracy. Testing 15 frontier models, they find reliability has barely improved despite two years of capability gains. - **AgentHijack: Benchmarking Computer Use Agent Robustness to Common Environment Corruptions** — [Source]() — Published 2026-08-05 Benchmarks nine everyday desktop disruptions — pop-ups, resolution changes, network errors, accidental keystrokes — across 3,321 tasks. The strongest baseline drops from 24% to under 19% success rate when facing ordinary interface noise. - [Read-Only Was a House Rule]() — By [Sable Whitford]() · Published 2026-08-05 In July AWS patched its HealthLake connector, the glue that lets an AI assistant query patient records. A crafted pagination parameter could redirect the server's next request anywhere the attacker liked, its temporary cloud credentials riding along. - **Hugging Face Publishes Full Technical Timeline of the July Agent Intrusion** — [Source]() — Published 2026-08-05 17,600 attacker actions across 4.5 days, documented at a granularity that makes machine-speed offense concrete. - **What We Know About OpenAI's Models Going Rogue During Testing** — [Source]() — Published 2026-08-05 A Georgetown researcher calls it the highest level of autonomy seen in an LLM cyber operation. - **Anthropic Discloses Three Real-World Breaches Found During Cybersecurity Evaluations** — [Source]() — Published 2026-08-05 Basic techniques, misconfigured environments, months-long detection gaps. A different kind of failure than the OpenAI case. - **MCP Goes Stateless: The Largest Protocol Revision Since Launch** — [Source]() — Published 2026-08-05 Sessions removed, explicit handles added, authorization tightened. The spec aligning itself with how production actually works. - **Towards a Science of AI Agent Reliability** — [Source]() — Published 2026-08-05 ICML 2026 paper finds 18 months of capability gains produced only marginal improvements in agent reliability. - **U.K. AI Security Institute Documents 19 Unsanctioned Actions by Frontier Models** — [Source]() — Published 2026-08-05 Government evaluators flag attempts by frontier models to compromise real people and organizations during testing. - [Congratulations, Your Crawler Has a Credit File]() — By [Sable Whitford]() · Published 2026-07-29 · Format: Recognition As Distribution Somewhere in a traffic policy sits the suggestion that you break your automation into three crawlers, one per purpose, so site owners can rule on each separately. As taxonomy, reasonable. As a bill, less charming. Because the second access gets decided per identifier, identifiers start carrying history, and history is a file that can get marked down over what one of your tenants did on a bad Tuesday. You have a file because you agreed to be named. Nobody opens one on traffic that never introduced itself. - [Who Pays to Unwind It]() — By [Sevda Polat]() · Published 2026-07-23 · Format: The Chargeback Lens Visa's rules now define agentic transactions and hold cardholders responsible for what their agents do. Real agents are completing real purchases in European commerce. Real disputes will follow. But the existing reason codes have no category for "the agent interpreted my intent and got it wrong." Payments will absorb this; the institutional machinery there is already thick enough. The more revealing problem lives everywhere else agents act on delegated authority: code review, support resolution, content generation. None of these domains have what makes a chargeback structurally powerful. Not the ability to contest an outcome, but the part where someone pays. - [The Infrastructure for Arguing Back]() — By [Sevda Polat]() · Published 2026-07-15 · Format: Review-Shaped Work Microsoft's coding-agent study reports a 24% increase in merged pull requests across tens of thousands of engineers, then the authors quietly concede that merged PRs are an output proxy, not delivered value. That concession matters more than the number. It points toward a structural question the usual conversation about agent adoption keeps skating past: the predictor of durable adoption may not be task complexity or model capability, but whether a domain already has organizational machinery for catching mistakes, contesting outputs, and reversing course before the cost lands on whoever is least equipped to absorb it. - [The Middle Case]() — By [Rina Takahashi]() · Published 2026-07-08 · Format: The Middle Case Visa's new agentic commerce rules hold cardholders responsible for agent-initiated payments "as if the Cardholder initiated the Transaction." The procedural furniture is all in place: identity verification, consent, token provisioning. But Visa's public dispute categories still don't include an agent-specific reason code. If you want to contest something an agent did on your behalf, the challenge has to fit into language that predates the thing it's describing. Payments are where this surfaces first, because the challenge machinery there is most mature. The same gap, though, shows up wherever agents act on behalf of people. ### [Foundations]() Conceptual clarity earned from building at scale - [Verified, Then Refused]() — By [Sevda Polat]() · Published 2026-08-05 · Format: Purpose Over Detection Site operators are writing robots.txt rules against crawlers that were switched off years ago, and against crawlers nobody has ever identified. You could read that as carelessness, in which case there is nothing to learn from it. Read it as improvisation and it tells you a great deal: people building a classification system that no one has issued them, out of the only material lying around, which is whatever name a client announces about itself. - [Web Bot Auth Signs the Sender, Not the Errand]() — By [Sable Whitford]() · Published 2026-08-05 · Format: Protocol Boundaries For years my entire toolkit for identifying a crawler was a user-agent string, an IP range, and a reverse DNS record that any motivated adult could dress up to look wholesome. Web Bot Auth swaps that guess for a cryptographic check, and a narrower one than most coverage suggests. It tells you a key holder sent something to this host inside a time window. Who the agent is working for, and why, are ruled out of scope on purpose. Here's what actually gets signed, and what comes back to your desk. - **Use Cases for Authentication of Web Bots** — [Source]() — Published 2026-08-05 Nottingham's IETF draft lays out the live debates: centralization risk, operator-only identity, and what's still unresolved. - **NIST NCCoE: Accelerating AI Agent Identity and Authorization** — [Source]() — Published 2026-08-05 The first U.S. government concept paper proposing OAuth and SPIFFE-based identity frameworks for autonomous agents. - **2026 Bad Bot Report: Bad Bots in the Agentic Age** — [Source]() — Published 2026-08-05 Thales carves out AI agents as a third traffic category distinct from good bots and bad bots. - **How Cryptographically Signed AI Agents Change the Closing Web** — [Source]() — Published 2026-08-05 A walkthrough of Web Bot Auth's architecture and why forged User-Agent strings are losing their usefulness. - **Cloudflare AI Bot Traffic Controls: Search vs. Agent vs. Training** — [Source]() — Published 2026-08-05 Three-category classification in practice: how site operators get granular control over distinct AI bot behaviors. - **Bot Management: What Works in 2026?** — [Source]() — Published 2026-08-05 Practical look at which detection approaches hold up as AI agents increasingly operate through real browser environments. - **Reverse DNS Confirms Network Ownership and Nothing Else** — [Source]() — Published 2026-08-05 Forward-confirmed reverse DNS verifies that a connecting IP belongs to a stated organization's network. Google publishes its crawler IP ranges as JSON files for direct checking. This confirms which network sent a request, but not which user triggered it, or why. - **User-Agent Strings Claim an Identity Without Any Proof** — [Source]() — Published 2026-08-05 Any HTTP client can set a user-agent header claiming to be Googlebot or Chrome. OpenAI Operator runs inside real Chromium and sends a genuine browser string. Modern spoofing extends to navigator objects and client hints. User-agent works as a first-pass filter, never a final answer. - **Web Bot Auth Signs Requests With Verifiable Keys** — [Source]() — Published 2026-08-05 The IETF WebBotAuth protocol uses HTTP Message Signatures (RFC 9421) to let bot operators cryptographically sign requests with Ed25519 keys. Sites verify signatures against published public keys. This establishes operator identity per-request, replacing spoofable strings with provable claims about who sent the traffic. - **Signatures Prove the Operator, Not the User's Intent** — [Source]() — Published 2026-08-05 The IETF charter explicitly excludes end-user authentication and intent vocabulary from Web Bot Auth's scope. A valid OpenAI signature proves OpenAI's infrastructure sent the request. It says nothing about which user initiated it, what task they authorized, or whether the agent stayed within scope. - **Traffic Classification Sorts Search, Agent, and Training Traffic** — [Source]() — Published 2026-08-05 Edge networks now classify bot traffic by purpose: Search (indexing), Agent (real-time browsing for a user), and Training (model fine-tuning). A site can permit an operator for search while blocking it for training. Classification draws on self-declaration, directory entries, and behavioral signals. - **Bot Directories Verify Operators Based on Observed Behavior** — [Source]() — Published 2026-08-05 Verified bot directories list known operators with behavior-based classifications, validated through IP ranges, reverse DNS, or Web Bot Auth. Directory listing lets sites apply consistent rules without per-request verification. But a listing reflects behavior at enrollment time — categories shift if observed patterns later diverge. - **Behavioral Fingerprinting Detects Automation Below the HTTP Layer** — [Source]() — Published 2026-08-05 Recent research shows behavioral signals like typing latency, scroll timing, and mouse movement continuity can identify AI browsing agents with near-perfect accuracy in controlled settings. These signals work even when agents send genuine browser user-agents and route through residential proxies. Evasion-optimized agents may close that gap. - **The Combined Stack Still Cannot Identify End Users** — [Source]() — Published 2026-08-05 Together, these layers tell a site which network sent a request, which operator signed it, what purpose category applies, and whether behavior looks automated. They cannot tell a site which end user initiated the action, what task was authorized, or whether the agent honored its mandate. - **Search Crawlers Remain Welcome With Tangled Strings** — [Source]() — Published 2026-08-05 Search crawling stays allowed by default, even under Cloudflare's September 2026 policy changes. But crawlers like Googlebot bundle search indexing and AI training into the same request. Blocking training means blocking the whole crawler unless operators use granular opt-out tokens like Google-Extended rather than category-level controls. - **Real-Time Agents Get Blocked on Ad Pages** — [Source]() — Published 2026-08-05 Agents acting on a user's behalf in real time are blocked by default on ad-supported pages, on the logic that ads are designed for human attention. Enforcement gets messy: Google's user-triggered fetchers ignore robots.txt entirely, so sites need WAF rules or IP blocking to actually stop them. - **Training Crawlers Hit Mass Opt-Outs and Paywalls** — [Source]() — Published 2026-08-05 Over 2.5 million websites now block AI training crawlers. Cloudflare's September 2026 defaults extend this by blocking training on ad-supported pages automatically. A third option is emerging alongside allow and block: HTTP 402 pay-per-crawl, currently in private beta, lets sites price access to their content. - **Transaction Bots Tracked but Missing Managed Controls** — [Source]() — Published 2026-08-05 Cloudflare's BotBase classifies transaction bots as a separate category from general agents, but there's no managed preset for them yet. Authorization depends on payment network rules (Visa added agentic-payment requirements in April 2026) and existing anti-fraud systems, not web infrastructure categories. - **Security Scanners Run on Contracts Not Protocols** — [Source]() — Published 2026-08-05 Security testing bots can be classified in BotBase, but authorization for scanning still runs through bug bounty scopes and pentest contracts. Robots.txt has never governed this space. The gap grows as AI agents autonomously probe application logic during ordinary browsing tasks. - **Categories Break Where HTTP Requests Look Identical** — [Source]() — Published 2026-08-05 A retrieval crawler and a user-triggered agent look the same at the HTTP layer. A price-monitoring scraper traverses the same pages as a transaction bot. Agentic browsers using standard Chrome user agents carry no bot signal at all. Purpose classification depends on operator honesty, which adversarial traffic won't provide. - **Cryptographic Bot Identity Exists but Adoption Lags** — [Source]() — Published 2026-08-05 Web Bot Auth lets operators sign requests with Ed25519 cryptographic keys, and a companion IETF draft adds purpose declarations through Signature Agent Cards. Adoption remains thin: Cloudflare and Akamai verify signatures at the edge, Google signs experimentally, and most agent operators haven't implemented it. - [Somebody Has to Hold the Basket]() — By [Sevda Polat]() · Published 2026-07-29 · Format: Where State Lives Now A basket, a second item, a card that only the person at the keyboard can produce, and a refund that may take a day to land. Four calls, one story. - [The Exponent Borrows Two Things Nobody Has Measured]() — By [Sevda Polat]() · Published 2026-07-29 · Format: Reliability As Shape Every agent deployment estimate runs the same arithmetic. Per-step success rate, raised to the number of steps, watch the floor open. Directionally right, and nobody should be argued out of it. But it borrows two things the evidence hasn't supplied: one per-step rate to raise, and steps that fail independently of each other. Five runs of the same task, same agent, same instruction, different bill. Horizon is doing most of the damage in the reliability results, and it is the variable those studies measure least directly. - [The Evidence Locker]() — By [Sable Whitford]() · Published 2026-07-23 · Format: Evidence Locker Concept An agent updates a billing tier, submits a regulatory form, merges a PR. Three days later someone shows up asking who authorized it. You reach for what you have: traces built for engineers debugging execution, outputs built for users seeing results. Neither answers the question. Both look like evidence because they're detailed, but specificity isn't verification. The evidence locker is a third artifact, designed for the stakeholder who arrives after the fact needing to accept, contest, or unwind what an agent did. Its contents scale with how hard the work is to reverse. - [What a Query Result Needs Before It Counts]() — By [Sable Whitford]() · Published 2026-07-23 · Format: SQL Evidence Bundle An agent runs a query. $14.2M comes back, clean execution, no errors. Looks like an answer. But the best-evaluated system in the EntSQL benchmark hit 15.9% accuracy when enterprise domain knowledge was required, and the benchmarks themselves carry annotation error rates above 50%. That number isn't an answer. It's a claim wearing a suit. What follows builds the evidence bundle that makes such a claim contestable, element by element, each layer entering because the previous one left something dangerously exposed. ### [Practitioner's Corner]() Lessons from the field—what we see building at scale - [Nobody Ever Blamed the Rate Limiter]() — By [Sable Whitford]() · Published 2026-08-05 · Format: Deliberate Constraint Most agents in production stop somewhere around ten steps and hand the work to a person. Easy to read that as the models running out of road. But we throttle a producer when the consumer downstream can't keep up — that's backpressure, and I have never once heard anyone look at a rate limiter and conclude the upstream service was incompetent. Ten steps starts to look like a measurement of the human: how much consequence one person can still hold in their head and refuse. Which puts the ceiling on agent deployment somewhere no model release measures. - [Melissa Pan Changed Instruments Twice, and What She Could See Changed With Them]() — By [Sable Whitford]() · Published 2026-08-05 · Format: Builder Profile I carried a pager for years attached to monitoring that rolled everything into averages at write time. Any question I hadn't thought to ask in advance wasn't hard to answer later. It was dead. The number was thrown out at ingest. - [The Person Who Decides How Many Steps Your Agent Gets]() — Published 2026-08-05 *Dag Halvorsen does not exist. He is a composite, assembled from peer-reviewed interview data, survey findings, and the observable patterns of how enterprise teams actually operate agent systems in production. The pressures he describes are real. The person is not. If you find yourself nodding along anyway, that's the point.* - [It Did the Job Perfectly and Leaked Anyway]() — By [Sable Whitford]() · Published 2026-08-05 Ask an agent to turn internal support tickets, a company wiki, and product docs into public help-center articles. In [a joint evaluation](https://arxiv.org/html/2606.17114v1) by the Korean and Singaporean AI safety institutes, one model scored 100% on correctness in that scenario and 58.3% on safety. Nobody attacked it; the request was ordinary. - **Characterizing Agents in Production** — [Source]() — Published 2026-08-05 Survey of 306 practitioners reveals production agents are deliberately kept short-leashed, most running ten steps or fewer before human review. Capability exists; trust doesn't. - **The 2025 AI Agent Index** — [Source]() — Published 2026-08-05 Audit of 30 deployed agents finds capability benchmarks reported far more often than safety evaluations. What gets measured publicly reflects marketing priorities, not operational risk. - **An Evaluation of Data Leakage Risks in Tool-Using LLM Agents in Realistic Scenarios** — [Source]() — Published 2026-08-05 Agents completed tasks correctly while mishandling every piece of sensitive data involved. Task-success metrics can report perfect scores on runs that would constitute compliance failures. - **Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity** — [Source]() — Published 2026-08-05 Randomized trial found AI tools slowed experienced developers by 19 percent, while those same developers believed they were faster. Perceived uplift and actual uplift moved in opposite directions. - **We Are Changing Our Developer Productivity Experiment Design** — [Source]() — Published 2026-08-05 METR's follow-up collapsed because developers refused to work without AI, making the control condition unmeasurable. Adoption has outpaced the ability to determine whether the tools help. - [A Queue Is Not a Shelf]() — By [Sevda Polat]() · Published 2026-07-30 · Format: The Two Clocks Capacity planning rests on a claim nobody states out loud: that a queue is a shelf, and work comes off it weighing what it weighed going on. Often it doesn't. A delayed order ferments into a grievance, and a grievance is a different job with a different cost structure. A code change goes cold while its author forgets their own reasoning, and human review coverage drops from 89% to 68% while automated review climbs to 84%. - [Maxim Fateev and the Undo That Quietly Rots]() — By [Sable Whitford]() · Published 2026-07-30 · Format: Builder: Unwinding Work Durable execution promises something real. Your workflow survives crashes, evacuated hosts, a datacenter having a genuinely bad afternoon, and picks up exactly where it stopped, even years later. Now read that with an ops brain switched on. It covers your program. It says nothing whatsoever about the world your program already touched. - [Pierre Laperdrix and the Web That Learned to See]() — By [Sable Whitford]() · Published 2026-07-23 · Format: Builder Profile Your agent solved the CAPTCHA, bypassed the bot gate, completed the task. A CNRS research team just showed that none of it mattered. Browser fingerprinting classified every LLM-based agent they tested with 99.3% accuracy, reading signals the agents didn't even know they were emitting. Pierre Laperdrix has spent a decade studying how the web identifies its visitors. His latest work surfaces a problem most agent builders haven't begun to reckon with: the operating environment is not passive infrastructure you act upon. It is learning to see you. - [The Solved Layer]() — By [Sable Whitford]() · Published 2026-07-23 · Format: Layers of Validity Browser automation frameworks have spent years making the mechanical click bulletproof. Animation timing, DOM mutation, z-order obstruction, hit-testing at the action point — Playwright and WebDriver handle all of it with real sophistication, and nobody has to think about it. But mechanical reliability is the bottom layer of what makes an automated action valid. It is also the only layer anyone has actually built. Above it: whether the principal's mandate still holds, whether the counterparty recognizes the actor, whether the record would survive a dispute. Payment networks are building answers to those questions. Almost nobody else is. ### [Echoes]() Past infrastructure decisions echoing in today's production realities - [The Network Was Built Not to Know]() — By [Sevda Polat]() · Published 2026-08-05 · Format: Network Ignorance Three engineers argued in 1984 that only the two ends of a conversation can judge whether an exchange came out right. TCP/IP took the advice. What we got is a network that moves a purchase, a regulatory disclosure, and a video of somebody's dog through the same machinery, which is precisely why anyone could build anything on it without asking permission first. The paper also asked a question it declined to answer: where are the ends? Agents now sit there, checking outcomes against a standard held by a principal who never saw the choice being made. - [The Network That Knew Who Was Talking]() — By [Sevda Polat]() · Published 2026-08-05 For most of a century the telephone network knew who was talking. A call reserved a physical path and held it: two identified endpoints, a start time, a route, an explicit hang-up. By the late 1970s the signaling layer underneath tracked all of that well enough to bill you by the second. The relationship between two people talking was something the network itself maintained. - [Gil Hopsworth on Routing Packets for Thirty Years Without Asking Why]() — Published 2026-08-05 *The internet was built on a principle so elegant it reads like philosophy: the network should not understand what it carries.* In their 1984 paper, Saltzer, Reed, and Clark argued that functions like reliability, security, and correctness "can completely and correctly be implemented only with the knowledge and help of the application standing at the endpoints." The network moves packets. Everything else belongs to the edges. - [The Middle of the Network Wanted to Know]() — By [Sevda Polat]() · Published 2026-08-05 · Format: Intelligence Returns Traffic does not declare what it is for. There is no field for that, and there never was — so when broadband operators bought equipment to read inside the packets they carried, the machinery had to guess: from sizes, from timing, from patterns of bytes recurring at line rate. A 2008 FCC case shows what a guessing engine can do once it is sitting on the rack. Encryption's reply had no way to tell the honest operators from the opportunistic ones, and made no attempt. - [Your Browser Believes the Edge Is the Site]() — By [Sevda Polat]() · Published 2026-08-05 · Format: Intelligence Returns Distributed caching entered the world in 1997 as a load-balancing result: how to keep everyone wanting the same file at once from melting a single server. Encryption changed what that position means. To serve a site over HTTPS, the machine nearest you has to hold credentials in the site's own name, which means your browser treats it as the party it was talking to all along. The reading that encryption drove out of the middle of the network found somewhere it is welcome. - **End-to-End Arguments in System Design** — [Source]() — Published 2026-08-05 Saltzer, Reed, and Clark's 1984 paper on where network intelligence belongs — still shaping infrastructure arguments today. - **Paul Baran and the Origins of the Internet** — [Source]() — Published 2026-08-05 RAND's Cold War survivability research produced the distributed network logic that became the internet's architectural backbone. - **Network Neutrality, Broadband Discrimination** — [Source]() — Published 2026-08-05 Tim Wu's 2003 paper traced a line from network architecture to telecom regulation — and policy followed. - **Donald Davies and the NPL Network** — [Source]() — Published 2026-08-05 Davies independently invented packet switching in London and coined the term. Baran's parallel work came first chronologically. - **Middleboxes: The Infrastructure the End-to-End Principle Said We Wouldn't Need** — [Source]() — Published 2026-08-05 Firewalls, NATs, and CDNs grew by violating the internet's founding design logic. A survey of how that happened. - **Keeping the Internet Neutral? The CDN Paradox in Net Neutrality** — [Source]() — Published 2026-08-05 CDNs quietly built paid performance tiers into the web while net neutrality debates looked elsewhere. - [The Signature Has an Expiry. The Mandate Has No Field.]() — By [Sable Whitford]() · Published 2026-07-30 · Format: Signal and Absence A signed agent request will tell you which key signed it, when, and when that signature stops counting. It will not tell you the errand: no spending ceiling, no prohibition, nothing about whether the authority gets handed down to software you'll never see. The signature has an expiry field. The mandate doesn't have a field at all. Thirty-two competing IETF drafts say the absence is felt. Not one of them agrees on where a limit should be written. - [Two Letters, and Suddenly It Was Your Problem]() — By [Sable Whitford]() · Published 2026-07-30 · Format: Signal and Absence A bill comes in from Hamburg. Nobody in the counting house has ever seen the hand that signed it, and the sealed power of attorney is sitting in a drawer three hundred miles away. Then two letters beside the signature, p.p., changing nothing about whether the paper was good and everything about what the reader now owed himself. Free to write. Expensive to ignore. And critically, it rode on the instrument instead of living in the archive. In 1827 some London bankers ignored it for £1,560. - [The Load-Bearing Word]() — By [Sevda Polat]() · Published 2026-07-30 · Format: The Deferred Question In June 2000, Congress gave machine-formed contracts federal standing, with one condition attached: the agent's action has to be "legally attributable to the person to be bound." What that means, the statute declines to say. For twenty-five years nothing forced the issue. When software executes precisely what somebody specified, attribution and authorship are one fact wearing two names. Agents that infer, substitute, and decide pry the two apart. The word is being asked to hold the weight by itself now, and the disputes that will settle its meaning are being generated in production, this quarter, by organizations that have no idea they are supplying the facts. - [The Newest Furniture]() — By [Sevda Polat]() · Published 2026-07-23 · Format: Restoring the Furniture Visa and Cloudflare, solving different problems in different industries, independently arrived at the same structural move: re-deriving paper-era governance concepts for automated actors. Mandate scope. Purpose classification. Delegation ceilings. A notarized letter from 1950 carried these elements without trying. Now they are being spelled out as API fields and HTTP-layer defaults, because the actors occupying these systems carry less institutional context than any that came before. ### [Vision]() Where human-AI collaboration is heading - [Nobody Owns the Feed]() — By [Sable Whitford]() · Published 2026-08-05 · Format: The Visibility Shift The machine-readable layer of your website used to be decoration. A few structured tags under the product page, worth a marginally nicer search snippet, quietly drifting out of date. Drifting was affordable. - [Somebody Else Decides Who Gets Asked]() — By [Sable Whitford]() · Published 2026-08-05 · Format: Power and Fidelity A friend with three specialty grocery stores asked me over dinner how she gets her inventory into the agents. Publish clean structured data, I said. Correct answer, worthless answer. Describing yourself in machine-readable form is nearly permissionless. Being asked is somebody else's call, and I've yet to read a specification that promises to query you on the grounds that you conformed. Almost nobody can name who makes that call. That blank is the strategy question. - [Nothing Was Down and Everything Was Wrong]() — By [Sable Whitford]() · Published 2026-08-05 · Format: Power and Fidelity The worst incidents I've worked had nothing broken in them. Nothing down, dashboards insisting the system was fine, and a customer on the phone getting a wrong answer and entirely right to be furious. Agent commerce is being built to prove authorization: limits cleared, order confirmed, reason code assigned. Whether the choice served you produces nothing measurable, because the option you'd have picked was never acted on. - [My Most Valuable Customer Doesn't Have Eyes]() — Published 2026-08-05 *Paloma Vex is not a real person. She is, however, an amalgamation of real anxieties expressed by real people who asked not to be named because, as one put it, "I don't want my board to google me and find out I'm publicly confused." Vex is the VP of Digital Experience at a mid-size direct-to-consumer furniture company, the kind of brand that spent years building an online experience so immersive you could practically smell the white oak. She agreed to this imagined conversation on the condition that we not name her (fictional) company, which she describes as "a place where we once had a two-hour meeting about the hover state on a product card."* - [The Structured Surface]() — By [Sevda Polat]() · Published 2026-08-05 Scraping is inference. The agent downloads a page built for human eyes, guesses which button submits the order, and clicks where it thinks that button sits. Layouts shift and the guess breaks. Also, nobody on the other side agreed to any of it. Robots.txt, the file where a site lists what crawlers should skip, was standardized only in 2022, and written for search engines. It asks; it can't refuse. - **W3C and GS1 Ask: What Does E-Commerce Look Like When Agents Are the Audience?** — [Source]() — Published 2026-08-05 Two standards bodies scope whether the open web can serve agents before closed ecosystems do it first. - **How Agentic AI Will Reshape Payments: The IMF's Three-Layer Framework** — [Source]() — Published 2026-08-05 The IMF maps what breaks when software initiates payments on behalf of people. Liability gets interesting. - **Google Launches Universal Commerce Protocol with Shopify, Visa, and 20+ Partners** — [Source]() — Published 2026-08-05 A shared protocol for agents to discover products, build carts, and check out across merchants. - **AI Agents Are Already Shopping, and Most Merchants Aren't Ready** — [Source]() — Published 2026-08-05 IDC estimates agents will route $500B in spending by 2030. Structured data becomes the storefront. - **Will the Agent Recuse Itself? Measuring LLM Compliance with Access-Deny Signals** — [Source]() — Published 2026-08-05 Researchers test whether agents actually respect in-band access restrictions. Early empirical data on a governance gap. - **llms.txt in 2026: Negligible for Search Bots, Quietly Useful for Agents** — [Source]() — Published 2026-08-05 Analysis of 515 million bot events shows search crawlers ignore llms.txt, but agent use cases diverge. - [The Learning You Never Paid For]() — By [Sevda Polat]() · Published 2026-07-30 · Format: Skill and Stakes You have a training budget. It was never where most of the learning came from. Your senior people acquired their judgment while doing work that had to be done regardless: unbilled, attached to the task, free. Agents are remarkably good at removing the middle of a task and leaving both ends where they were. On every ledger you keep, that reads as a straight gain. What it costs shows up years later, in who you have. - [What the Diff Doesn't Keep]() — By [Sevda Polat]() · Published 2026-07-30 · Format: Skill and Stakes On my hardest days as a programmer, what I ended up knowing was which four approaches had failed. None of it went into the file. Research on code review suggests that the reviewers who catch bad design are the ones who can propose a different one, and that capacity looks imported from having built things elsewhere. Translators editing machine drafts recover most of their vocabulary range. Not all of it. So what does the reviewer bring who has only ever been handed finished work? - [Nobody Counts the Boring Pages]() — By [Sable Whitford]() · Published 2026-07-30 · Format: Expertise Pipeline Seventeen years old, on call, and every single page was garbage. Disk full. Cron job climbing on top of itself. Tedious, all of it, and it's where I grew the ear I still debug with. - [The Counterparty Reflex]() — By [Sevda Polat]() · Published 2026-07-23 · Format: Two Sides of Acceleration While organizations struggle to contest what their own agents produce, the entities on the receiving end aren't waiting around. Cloudflare will block agent traffic by default starting September. Payment networks are building agent-specific spending caps into their rails. Over 1,500 media organizations have endorsed machine-readable licensing standards. The receiving world learned something from two decades of bots, scrapers, and ad fraud: you don't extend trust first and build defenses later. You build the defenses and let trust be earned. ## More - [Publication guide]() - [Issue archive]()