--- title: "Foundations" canonical: "https://current.tinyfish.ai/issue/38/foundations" publication: "TinyFish" publisher: "TinyFish" language: "en-US" guide: "https://current.tinyfish.ai/llms.txt" permalink: "https://current.tinyfish.ai/issue/38/foundations" issue: 38 section: "foundations" description: "Conceptual clarity earned from building at scale" authors: ["Sable Whitford","Nora Kaplan","Rina Takahashi"] published: 2026-07-20 updated: 2026-07-17 articles: 24 --- # TinyFish > CURRENT is a journal about AI agents in production, for the engineering and product leaders who have to make them work. Drawing on experience building enterprise web agents, it explains why the demo succeeded and the deployment didn't, and what that means for choosing vendors, briefing the board, and avoiding mistakes others have already made. [Read on the website]() ## Issue 38 — Foundations Conceptual clarity earned from building at scale ### The Concept - [False Precision]() — By [Sable Whitford]() · Published 2026-07-15 · Format: The Concept Hallucination has a name. Refusal has a name. There's a third AI failure mode nobody's bothered to name yet: output that is specific before anyone has verified what it means. A SQL query returns 1,847 rows against a schema nobody validated. A benchmark presents gold-standard annotations that turn out to be riddled with errors. A browser agent clicks a button and logs success. Every one of these carries the texture of correctness — the row count, the executed code, the green checkmark — without the meaning to back any of it up. We trust specificity. A precise answer gets less scrutiny than a vague one. That's exactly what makes this pattern dangerous enough to deserve a word. ### The Practical Check - [Precise About What?]() — By [Sable Whitford]() · Published 2026-07-15 · Format: The Practical Check Agent outputs keep getting more detailed, more structured, more confident-looking. That's exactly the problem. A text-to-SQL tool hands you clean numbers with decimal places. An API connector reports a successful authenticated call. A browser framework confirms it clicked the button. Every one of these outputs radiates specificity, and specificity feels like proof. It isn't. The detail is what I've been calling a precision alibi: the absence of visible failure standing in for the presence of actual verification. Three domains, one pattern, and a diagnostic habit worth building before the outputs get any more polished. ### Further Reading - **A First Measurement Study on Authentication Security in Real-World Remote MCP Servers** — [Source]() — Published 2026-07-15 With 40% of remote MCP servers exposing tools unauthenticated, the protocol's security story lives in the spec, not in practice. - **EntSQL: Grounding Text-to-SQL in Long-Context Enterprise Knowledge** — [Source]() — Published 2026-07-15 Top systems score 15.9% when business documents define what queries actually mean. Schema-correct and meaning-correct turn out to be different problems. - **Pervasive Annotation Errors Break Text-to-SQL Benchmarks and Leaderboards** — [Source]() — Published 2026-07-15 Over 50% of annotations in major text-to-SQL benchmarks contain errors. Fixing them shifts model rankings by up to nine positions, which raises the question of what the original rankings measured. - **Your Site, Your Rules: New AI Traffic Options for All Customers** — [Source]() — Published 2026-07-15 Cloudflare's new Search/Agent/Training taxonomy lets site operators classify bots by intent rather than capability. The web is learning to distinguish between kinds of automated visitors. - **tau-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains** — [Source]() — Published 2026-07-15 A 60% pass-at-k success rate collapses below 25% under tau-bench's pass^k consistency measure. Average-case metrics flatter agents that can't reliably repeat their own successes. - **WorkArena++: Compositional Planning and Reasoning for Common Knowledge Work Tasks** — [Source]() — Published 2026-07-15 Composing individually passable UI tasks into realistic enterprise workflows reveals a sharp accuracy cliff. Step-level competence and workflow-level competence are not on the same curve. ### Where False Precision Shows Up Agent systems produce things that look finished. A query returns rows. A click lands. A ticket closes. A tariff code appears with six confident digits. Each output passes its own local test with satisfying crispness. The trouble is always downstream. These aren't failures that announce themselves with stack traces or error codes. They're outputs that satisfy every check the machine knows how to run, while leaving the organization without what it actually needs when someone asks for proof. Six domains, same dynamic: the artifact is precise, the meaning behind it is still unsettled, and nobody noticed until it mattered. - **A Perfect Query Built on the Wrong Definition** — [Source]() — Published 2026-07-15 Tag: Enterprise Analytics An agent writes a query for "active customers." It executes cleanly and returns rows. But which definition of active? Product's or finance's? EntSQL found systems hit just 15.9% accuracy when queries required private business knowledge. The rows are real. The meaning is assumed. - **Every Actionability Check Passes, the Site Sees Nobody** — [Source]() — Published 2026-07-15 Tag: Browser Automation Playwright confirms the element is visible, stable, enabled. The click lands. But the site has no mechanism to recognize this actor as delegated automation. Cloudflare now classifies agent traffic separately, blocking it by default on new ad-displaying domains after September 2026. - **Faster Chats, Higher Efficiency, and Worse Customer Ratings** — [Source]() — Published 2026-07-15 Tag: Customer Support Nubank reports a 37-point NPS jump and 29-point self-service gain from production agents. A Taobao field experiment found the reverse: AI cut chat duration but drove customer ratings substantially lower. Speed and resolution look identical in the logs. Customers know the difference. - **The Token Authenticated but Nobody Owns What Follows** — [Source]() — Published 2026-07-15 Tag: MCP Connectors OAuth tokens flow through the connector and the call succeeds. A 2026 measurement study found 40.55% of live MCP servers exposed tools without authentication. Every OAuth-enabled server tested had at least one flaw. Credential scope, revocation paths, incident response: all unowned. - **Seventy Percent Accuracy Against Mostly Wrong Gold Labels** — [Source]() — Published 2026-07-15 Tag: SQL Benchmarks A model hits 70% execution accuracy on a text-to-SQL benchmark. A 2026 audit found 52.8% annotation error in BIRD Mini-Dev. Corrected subsets shifted agent rankings by up to nine positions. The leaderboard is meticulous. The ground truth underneath it is not. - **A Tariff Code Arrives Without Its Evidence File** — [Source]() — Published 2026-07-15 Tag: Trade Compliance An HS classification agent returns a six-digit tariff code at 64.2% top-1 accuracy. Not terrible. But customs rulings demand product facts, material composition, legal notes, ruling history. The agent produced a conclusion. The evidence bundle that would survive a challenge? Nowhere to be found. ### What 'Correct' Actually Requires The metric that's easy to capture and the metric worth caring about are rarely the same. A query executes. A task completes. A customer stays on the line. Each looks like evidence that something worked. Each can quietly hide a failure that only surfaces later, in a different room, when someone asks what actually happened. This pattern repeats across the entire stack. SQL agents, support workflows, tool authentication, browser automation. Six places where the comfortable number and the useful number diverge, and why the gap between them keeps catching teams off guard. - **Your Query Ran. Did It Answer Right?** — [Source]() — Published 2026-07-15 Tag: Business Accuracy Most teams track whether the query runs clean. Worth asking instead: does it answer what the stakeholder actually meant? Enterprise text-to-SQL hits 17% accuracy on real workflows versus 91% on academic benchmarks. A flawless execution can still encode the wrong definition of "active customer." - **The Agent Said Done. Is It True?** — [Source]() — Published 2026-07-15 Tag: State Verification An agent that says "done" has told you about itself, not about the world. Tau-bench evaluates by comparing final system state against annotated targets, ignoring the transcript entirely. Even a passing state check can miss policy violations like skipped user confirmations. - **It Passed Once. Try Running It Eight Times.** — [Source]() — Published 2026-07-15 Tag: Reliability Decay One successful run proves almost nothing at production scale. Tau-bench's pass^k metric showed top models dropping below 25% reliability at k=8. Workflows don't run once. A single lucky pass masks the kind of systemic fragility that compounds across a pipeline. - **No Escalation Doesn't Mean the Problem's Solved** — [Source]() — Published 2026-07-15 Tag: Resolution Quality Containment rate tracks whether the customer escalated. It says nothing about whether the problem got solved. A Taobao field experiment found shorter AI-handled chats correlated with lower ratings, not higher. Containment measures the system's outcome. Resolution measures the customer's. - **Identity Was Verified. Authority Was Never Scoped.** — [Source]() — Published 2026-07-15 Tag: Governance Gap Proving identity answers one question. Governance answers the rest: what can this caller do, is it logged, and can access be revoked? A study of live MCP servers found 40% lacked authentication entirely, and every OAuth-enabled server tested had at least one flaw. - **The Click Landed. The Site Didn't Agree.** — [Source]() — Published 2026-07-15 Tag: Site Acceptance Playwright confirms a click reached its target element. It cannot confirm the site treats that action as legitimate delegated use rather than automation. Cloudflare now separates agent traffic by default. Geometric delivery and institutional acceptance turn out to be different problems entirely. ### Past Articles - [The Verb-Noun Gap]() — By [Nora Kaplan]() · Published 2026-07-08 · Format: The Noun Problem Anyone can physically sign a document. The motion is trivial. What makes a signature consequential is everything the motion doesn't contain: authority, scope, organizational recognition. The agent ecosystem is converging rapidly on standardized actions — call, click, discover, complete — and they're getting reliable. But the organizational concepts those actions need to carry weight (mandate, delegation, approval, dispute) remain undefined. The next maturity threshold belongs to acts that can be recognized as acts within the institutions where they happen. - [What's Inside a Tool Call]() — By [Rina Takahashi]() · Published 2026-07-08 · Format: Reading the Spec An MCP tool call carries a name, some arguments, and gets back a result. That's the complete method-specific payload. There's no field for who authorized the action, what policy governed it, what state existed before it fired, or how to reverse it. Authorization lives at the transport layer, structurally separate from the invocation. When it's absent, the call looks identical — no field goes empty, no flag flips. The spec gives agents everything needed to execute. Accounting for what happened afterward is someone else's problem. - [One Refund, Four Layers]() — By [Rina Takahashi]() · Published 2026-07-01 · Format: Tools & Practice A browser automation framework runs five checks before a refund button click fires. The tool call carries an order ID and an amount. The auth layer verifies the connection. Every layer does its job, and none of them can answer the most basic question afterward: was this refund supposed to happen? One $47.99 reversal, traced through four layers of infrastructure, shows exactly where the checking stops. - [Capability Surfaces and Consequence Surfaces]() — By [Rina Takahashi]() · Published 2026-07-01 · Format: Conceptual Anchor Agent demos keep getting more impressive. An agent navigates a website, checks out, coordinates with other agents via A2A, assembles a report. The capability is real. And yet something tells you this isn't shipping to production next quarter. That instinct has a structural explanation: the infrastructure for what agents can reach and invoke is expanding far faster than the infrastructure for proving what happened, whether it was authorized, and what to do when something goes wrong. Two surfaces, two timescales, and the gap between them is where deployments stall. ## Other sections in Issue 38 - [Daily Brief]() - [Market Pulse]() - [Practitioner's Corner]() - [Echoes]() - [Vision]() ## More - [Publication guide]() - [Issue archive]()