Share

The Concept

False Precision

By Sable Whitford— July 15, 2026

Feature image for article: False Precision

Hallucination has a name. Refusal has a name. There's a third AI failure mode nobody's bothered to name yet: output that is specific before anyone has verified what it means. A SQL query returns 1,847 rows against a schema nobody validated. A benchmark presents gold-standard annotations that turn out to be riddled with errors. A browser agent clicks a button and logs success. Every one of these carries the texture of correctness — the row count, the executed code, the green checkmark — without the meaning to back any of it up. We trust specificity. A precise answer gets less scrutiny than a vague one. That's exactly what makes this pattern dangerous enough to deserve a word.

The Concept

False Precision

By Sable Whitford— July 15, 2026

Hallucination has a name. Refusal has a name. There's a third AI failure mode nobody's bothered to name yet: output that is specific before anyone has verified what it means. A SQL query returns 1,847 rows against a schema nobody validated. A benchmark presents gold-standard annotations that turn out to be riddled with errors. A browser agent clicks a button and logs success. Every one of these carries the texture of correctness — the row count, the executed code, the green checkmark — without the meaning to back any of it up. We trust specificity. A precise answer gets less scrutiny than a vague one. That's exactly what makes this pattern dangerous enough to deserve a word.

The Practical Check

Precise About What?

By Sable Whitford— July 15, 2026

Feature image for article: Precise About What?

Agent outputs keep getting more detailed, more structured, more confident-looking. That's exactly the problem. A text-to-SQL tool hands you clean numbers with decimal places. An API connector reports a successful authenticated call. A browser framework confirms it clicked the button. Every one of these outputs radiates specificity, and specificity feels like proof. It isn't. The detail is what I've been calling a precision alibi: the absence of visible failure standing in for the presence of actual verification. Three domains, one pattern, and a diagnostic habit worth building before the outputs get any more polished.

The Practical Check

Precise About What?

By Sable Whitford— July 15, 2026

Agent outputs keep getting more detailed, more structured, more confident-looking. That's exactly the problem. A text-to-SQL tool hands you clean numbers with decimal places. An API connector reports a successful authenticated call. A browser framework confirms it clicked the button. Every one of these outputs radiates specificity, and specificity feels like proof. It isn't. The detail is what I've been calling a precision alibi: the absence of visible failure standing in for the presence of actual verification. Three domains, one pattern, and a diagnostic habit worth building before the outputs get any more polished.

Further Reading

A First Measurement Study on Authentication Security in Real-World Remote MCP ServersWith 40% of remote MCP servers exposing tools unauthenticated, the protocol's security story lives in the spec, not in practice.

EntSQL: Grounding Text-to-SQL in Long-Context Enterprise KnowledgeTop systems score 15.9% when business documents define what queries actually mean. Schema-correct and meaning-correct turn out to be different problems.

Quick links

Pervasive Annotation Errors Break Text-to-SQL Benchmarks and Leaderboards

Your Site, Your Rules: New AI Traffic Options for All Customers

tau-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains

WorkArena++: Compositional Planning and Reasoning for Common Knowledge Work Tasks

Where False Precision Shows Up

Agent systems produce things that look finished. A query returns rows. A click lands. A ticket closes. A tariff code appears with six confident digits. Each output passes its own local test with satisfying crispness.

The trouble is always downstream. These aren't failures that announce themselves with stack traces or error codes. They're outputs that satisfy every check the machine knows how to run, while leaving the organization without what it actually needs when someone asks for proof. Six domains, same dynamic: the artifact is precise, the meaning behind it is still unsettled, and nobody noticed until it mattered.

Where False Precision Shows Up

Agent systems produce things that look finished. A query returns rows. A click lands. A ticket closes. A tariff code appears with six confident digits. Each output passes its own local test with satisfying crispness.

The trouble is always downstream. These aren't failures that announce themselves with stack traces or error codes. They're outputs that satisfy every check the machine knows how to run, while leaving the organization without what it actually needs when someone asks for proof. Six domains, same dynamic: the artifact is precise, the meaning behind it is still unsettled, and nobody noticed until it mattered.

Enterprise Analytics

A Perfect Query Built on the Wrong Definition

An agent writes a query for "active customers." It executes cleanly and returns rows. But which definition of active? Product's or finance's? EntSQL found systems hit just 15.9% accuracy when queries required private business knowledge. The rows are real. The meaning is assumed.

Browser Automation

Every Actionability Check Passes, the Site Sees Nobody

Playwright confirms the element is visible, stable, enabled. The click lands. But the site has no mechanism to recognize this actor as delegated automation. Cloudflare now classifies agent traffic separately, blocking it by default on new ad-displaying domains after September 2026.

Customer Support

Faster Chats, Higher Efficiency, and Worse Customer Ratings

Nubank reports a 37-point NPS jump and 29-point self-service gain from production agents. A Taobao field experiment found the reverse: AI cut chat duration but drove customer ratings substantially lower. Speed and resolution look identical in the logs. Customers know the difference.

MCP Connectors

The Token Authenticated but Nobody Owns What Follows

OAuth tokens flow through the connector and the call succeeds. A 2026 measurement study found 40.55% of live MCP servers exposed tools without authentication. Every OAuth-enabled server tested had at least one flaw. Credential scope, revocation paths, incident response: all unowned.

SQL Benchmarks

Seventy Percent Accuracy Against Mostly Wrong Gold Labels

A model hits 70% execution accuracy on a text-to-SQL benchmark. A 2026 audit found 52.8% annotation error in BIRD Mini-Dev. Corrected subsets shifted agent rankings by up to nine positions. The leaderboard is meticulous. The ground truth underneath it is not.

Trade Compliance

A Tariff Code Arrives Without Its Evidence File

An HS classification agent returns a six-digit tariff code at 64.2% top-1 accuracy. Not terrible. But customs rulings demand product facts, material composition, legal notes, ruling history. The agent produced a conclusion. The evidence bundle that would survive a challenge? Nowhere to be found.

What 'Correct' Actually Requires

The metric that's easy to capture and the metric worth caring about are rarely the same. A query executes. A task completes. A customer stays on the line. Each looks like evidence that something worked. Each can quietly hide a failure that only surfaces later, in a different room, when someone asks what actually happened.

This pattern repeats across the entire stack. SQL agents, support workflows, tool authentication, browser automation. Six places where the comfortable number and the useful number diverge, and why the gap between them keeps catching teams off guard.

What 'Correct' Actually Requires

The metric that's easy to capture and the metric worth caring about are rarely the same. A query executes. A task completes. A customer stays on the line. Each looks like evidence that something worked. Each can quietly hide a failure that only surfaces later, in a different room, when someone asks what actually happened.

This pattern repeats across the entire stack. SQL agents, support workflows, tool authentication, browser automation. Six places where the comfortable number and the useful number diverge, and why the gap between them keeps catching teams off guard.

Business Accuracy

Your Query Ran. Did It Answer Right?

Most teams track whether the query runs clean. Worth asking instead: does it answer what the stakeholder actually meant? Enterprise text-to-SQL hits 17% accuracy on real workflows versus 91% on academic benchmarks. A flawless execution can still encode the wrong definition of "active customer."

State Verification

The Agent Said Done. Is It True?

An agent that says "done" has told you about itself, not about the world. Tau-bench evaluates by comparing final system state against annotated targets, ignoring the transcript entirely. Even a passing state check can miss policy violations like skipped user confirmations.

Reliability Decay

It Passed Once. Try Running It Eight Times.

One successful run proves almost nothing at production scale. Tau-bench's pass^k metric showed top models dropping below 25% reliability at k=8. Workflows don't run once. A single lucky pass masks the kind of systemic fragility that compounds across a pipeline.

Resolution Quality

No Escalation Doesn't Mean the Problem's Solved

Containment rate tracks whether the customer escalated. It says nothing about whether the problem got solved. A Taobao field experiment found shorter AI-handled chats correlated with lower ratings, not higher. Containment measures the system's outcome. Resolution measures the customer's.