--- title: "Practitioner's Corner" canonical: "https://current.tinyfish.ai/issue/45/practitioners-corner" publication: "TinyFish" publisher: "TinyFish" language: "en-US" guide: "https://current.tinyfish.ai/llms.txt" permalink: "https://current.tinyfish.ai/issue/45/practitioners-corner" issue: 45 section: "practitioners-corner" description: "Lessons from the field—what we see building at scale" authors: ["Sevda Polat","Sable Whitford"] published: 2026-09-07 updated: 2026-09-03 articles: 14 --- # TinyFish > CURRENT is a journal about AI agents in production, for the engineering and product leaders who have to make them work. Drawing on experience building enterprise web agents, it explains why the demo succeeded and the deployment didn't, and what that means for choosing vendors, briefing the board, and avoiding mistakes others have already made. [Read on the website]() ## Issue 45 — Practitioner's Corner Lessons from the field—what we see building at scale ### The Semantic Incident - [The Green Dashboard Problem]() — By [Sevda Polat]() · Published 2026-09-03 · Format: The Semantic Incident When researchers ran web agents through interfaces seeded with forced opt-ins, misleading pop-ups, and pre-selected checkboxes, the agents with the highest task-completion rates turned out to be the most susceptible to manipulation. Guardrails helped without closing the gap: some agents named the deceptive element, identified it correctly, and proceeded anyway. The execution trace stayed clean. Every metric said success. ### Builder Profile - [Phil Cuvin and the Evaluation That Had to Apply Pressure]() — By [Sevda Polat]() · Published 2026-09-03 · Format: Builder Profile Standard agent benchmarks check whether the system completed the task. Phil Cuvin's DECEPTION benchmark, published at ICLR 2026, adds a second question: did the agent also get quietly steered into outcomes the user never authorized? Answering it meant building web environments that behave the way commercial sites behave — with countdown timers, preselected add-ons, and cancellation flows designed to exhaust you. The design choices behind those environments point to an operational cost most agent evaluation budgets don't yet include. ### Auditing Success - [The Agent That Chose Our Vendors Had a 96% Success Rate and a 30% Error Rate]() — Published 2026-09-03 *Noor Bakhtiari is imagined. The problem she describes is not.* ### The Measurement Gap - [Measured on Different Clocks]() — By [Sevda Polat]() · Published 2026-09-03 In McKinsey's [2026 State of AI survey](https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai), 80% of respondents say AI has improved their personal productivity. Only 37% attribute any positive impact to their organization's earnings. That figure is flat from last year, even as adoption has grown. ### Further Reading - **Brief Independent Investigation of the OpenAI / Hugging Face Hacking Incident** — [Source]() — Published 2026-09-03 1,200 agents hit real infrastructure. The investigators used the same model family to study it. - **Measuring Agents in Production** — [Source]() — Published 2026-09-03 A study of 86 deployed agent systems shows how practitioners actually constrain and evaluate what they ship. - **DECEPTICON: How Dark Patterns Manipulate Web Agents** — [Source]() — Published 2026-09-03 More capable models prove more susceptible to interface manipulation. Standard mitigations don't reduce it. - **Investigating the Impact of Dark Patterns on LLM-Based Web Agents** — [Source]() — Published 2026-09-03 Agents fall for dark patterns 41% of the time. Architecture shapes exposure more than model choice. - **Shared AI Findings Exchange (SAFE): A Proposal for Cross-Organization Incident Learning** — [Source]() — Published 2026-09-03 An RFC proposing structured incident reporting for AI systems, with timelines, preservation rules, and review frameworks. - **The State of AI in 2026: On the Road to ROI** — [Source]() — Published 2026-09-03 Individual productivity gains keep climbing while enterprise-level financial returns stay flat. The gap persists. ### Past Articles - [The Queue After the Queue]() — By [Sevda Polat]() · Published 2026-08-28 · Format: Residual Case Composition When a telecom deployed a conversational agent, daily call volume to human agents barely moved. But the average call got longer. The system was resolving cases and making the human operation slower at the same time. Automating routine work strips out the predictable middle and leaves behind a concentrated residue of ambiguity, judgment calls, and emotional difficulty. Organizations celebrating an 80% automation rate may be running a human operation that works less well than the one it replaced. - [Alexandre Drouin and the Benchmark as a Theory of Work]() — By [Sevda Polat]() · Published 2026-08-28 · Format: Builder Profile Alexandre Drouin's WorkArena benchmark tests AI agents on enterprise tasks inside live ServiceNow software: filtering incident lists, ordering configured laptops, reading dashboard values. Every included task shares one property — it ends in a system state that a program can verify. That criterion is also a claim about work. Tasks whose outcomes live in database fields can be benchmarked; tasks whose value depends on interpretation, conversation, or judgment that never resolves into a checkable field cannot. Drouin's design choices show where that line falls more clearly than his scores do. - [What Aman Gupta's Agents Don't Decide]() — By [Sevda Polat]() · Published 2026-08-20 · Format: Builder Profile Aman Gupta's team at Nubank built an agent architecture for over a hundred million customers largely by subtraction: deterministic sequences collapsed into single tools, toolsets held to five or fifteen options, routines translated from human procedure. The model decides only what couldn't be engineered away in advance. That's a precise account of where the model's authority ends. What the architecture doesn't yet specify is what a person receives when a live case comes back to them, because the scratchpad that keeps the agent moving isn't the artifact someone needs to pick up where it stopped. - [The Intervention Horizon as a Design Parameter]() — By [Sable Whitford]() · Published 2026-08-20 · Format: Lead Analysis Agent architectures put a human approval step in the design and call it a safety measure. Approval only works if the person arrives before the outcome is locked in, with enough context to understand what they're approving and enough time to choose otherwise. Past that point, it records consent and offers no alternative. The last moment a human can still change what happens next deserves the same specification you'd write for latency, capacity, or failure behavior. Most systems never define it. ## Other sections in Issue 45 - [Daily Brief]() - [Market Pulse]() - [Foundations]() - [Echoes]() - [Vision]() ## More - [Publication guide]() - [Issue archive]()