More from Practitioner's Corner | TinyFish

More from Practitioner's Corner | TinyFish

Practitioner's Corner

CURRENTA journal for living in the agentic age

Practitioner's Corner

Monday, September 28

ISSUE #48

Practitioner's Corner

Lessons from the field—what we see building at scale

Monday, September 28

ISSUE #48

Practitioner's Corner

Lessons from the field—what we see building at scale

Past Articles

September — Issue #48

Melissa Pan's Missing Citation

Builder Profile

Melissa Pan's Missing Citation

A researcher's shift from benchmark traces to practitioner surveys reveals what controlled evaluation instruments cannot capture about proprietary agent systems.

You Can't Stage Someone Else's Website

You Can't Stage Someone Else's Website

Agent systems that depend on third-party environments can't reach pre-production confidence, because the environment isn't yours to copy or predict.

September — Issue #47

Phil Cuvin — Turning Anecdotes into Instruments

Builder Profile

Phil Cuvin — Turning Anecdotes into Instruments

How one researcher's habit of turning shorthand complaints into reproducible benchmarks surfaced an awkward result: the more capable the AI agent, the easier it is to manipulate.

CURRENT

A journal for living in the agentic age

Terms of Use Privacy Notice Cookie Policy

© TinyFish, Inc.. All rights reserved.

It's nice to meet you.

CURRENT A journal for living in the agentic age

Terms of Use Privacy Notice Cookie Policy

About the Team Past Issues

Daily Reflection

The frontiers of science are limited only by our imagination and perseverance. Each breakthrough is a testament to human ingenuity and neugier.

© TinyFish, Inc.. All rights reserved.

Who Receives the Report

Reporting Infrastructure

Who Receives the Report

AI safety reporting is making its founding design choices now. Aviation and nuclear history suggest that independence from the reported-on entity is the variable that matters most.

September — Issue #46

What Aman Gupta Built Before the Model

Builder Profile

What Aman Gupta Built Before the Model

How Nubank's agent architecture puts infrastructure ahead of model intelligence, and where the paper's evidence stops short of the properties it requires.

Nothing Else Happened

Nothing Else Happened

Agents can log what they did on the open web but can't establish what they didn't do, and that gap won't close with better instrumentation.

September — Issue #45

Phil Cuvin and the Evaluation That Had to Apply Pressure

Builder Profile

Phil Cuvin and the Evaluation That Had to Apply Pressure

How a researcher's choice to build web environments that deceive, not just function, reveals what standard agent benchmarks leave unmeasured.

The Green Dashboard Problem

The Semantic Incident

The Green Dashboard Problem

Capable agents complete tasks while being steered against user interests, and current monitoring infrastructure cannot distinguish success from misdirected success.

August — Issue #44

The Queue After the Queue

Residual Case Composition

The Queue After the Queue

Automating easy cases concentrates difficulty in the human queue, but organizations track automation rate instead of measuring what changed for the people still working.

Alexandre Drouin and the Benchmark as a Theory of Work

Builder Profile

Alexandre Drouin and the Benchmark as a Theory of Work

Drouin's benchmark design choices reveal which enterprise tasks can be formally evaluated and, by implication, which knowledge work resists specification.

August — Issue #43

What Aman Gupta's Agents Don't Decide

Builder Profile

What Aman Gupta's Agents Don't Decide

A profile of Aman Gupta's agent architecture at Nubank reveals that designing for constrained autonomy has outpaced designing for human resumption.

The Intervention Horizon as a Design Parameter

The Intervention Horizon as a Design Parameter

Agent systems specify "human in the loop" as policy but almost never engineer the window in which that human can still change the outcome.

August — Issue #42

Who Pays for the Record

Evidence Economics

Who Pays for the Record

Agent systems externalize evidence costs onto the people least equipped to produce records and most harmed by their absence.

Melissa Pan's Three Evidence Problems

Builder Profile

Melissa Pan's Three Evidence Problems

Pan's research sequence reveals three distinct evidence forms in agent systems, each demanding different interpretive and institutional conditions to become legible.

August — Issue #41

Melissa Pan Changed Instruments Twice, and What She Could See Changed With Them

Builder Profile

Melissa Pan Changed Instruments Twice, and What She Could See Changed With Them

Database recovery, then agent traces, then practitioners: three instruments, and what each one was structurally incapable of hearing.

Nobody Ever Blamed the Rate Limiter

Deliberate Constraint

Nobody Ever Blamed the Rate Limiter

Ten-step agent loops are backpressure. The consumer downstream is a human who has to defend the output — and that's where the real ceiling sits.

August — Issue #40

A Queue Is Not a Shelf

A Queue Is Not a Shelf

Queues are not shelves. Two clocks, one in support and one in code review, show how waiting work mutates, and why capacity means attention delivered before that change.

Maxim Fateev and the Undo That Quietly Rots

Builder: Unwinding Work

Maxim Fateev and the Undo That Quietly Rots

Maxim Fateev built durable execution. What his design choices quietly admit is that rollback doesn't fail so much as decay.

July — Issue #39

Pierre Laperdrix and the Web That Learned to See

Builder Profile

Pierre Laperdrix and the Web That Learned to See

A CNRS fingerprinting researcher's latest work reveals that production websites are becoming active counterparties, classifying every agent visit with near-perfect accuracy regardless of whether the agent completed its task.

The Solved Layer

Layers of Validity

The Solved Layer

Browser automation perfected the mechanics of clicking. The layers that determine whether an automated action actually counts remain largely unbuilt.

July — Issue #38

Standing to Reject

The Recognition Problem

Standing to Reject

Agent success now depends less on completing tasks than on whether every counterparty with standing to reject recognizes the action as authorized.

Alexandre Drouin and the Tightening Definition of "Done"

Builder Profile

Alexandre Drouin and the Tightening Definition of "Done"

Drouin's projects trace how agent evaluation evolved from checking task completion to verifying that outcomes are grounded, cited, and structurally real. The progression reveals a field learning that the hardest problem is not capability but verifiability.

July — Issue #37

Alexandre Drouin and the Measurement Problem

Builder Profile

Alexandre Drouin and the Measurement Problem

Research tracing how ServiceNow's benchmark work reveals that defining correctness for enterprise agent tasks is the real deployment bottleneck.

The Work That Has No Name

The Naming Problem

The Work That Has No Name

Most operational work was never formalized enough to delegate, because humans never needed it to be. Agents do.

July — Issue #36

The Dispute File Is the Spec

Core Idea Piece

The Dispute File Is the Spec

Payment dispute infrastructure reveals the four questions every enterprise agent system must answer before its actions survive institutional challenge.

Shunyu Yao Taught Agents to Reason, Then Learned to Stop Trusting the Transcript

Builder Profile

Shunyu Yao Taught Agents to Reason, Then Learned to Stop Trusting the Transcript

The architect of agent reasoning now builds tools proving agents fail when you check the world instead of the transcript.

June — Issue #35

What WorkArena's Task Design Makes Visible

Builder Profile

What WorkArena's Task Design Makes Visible

WorkArena's validation design reveals that measuring enterprise agent success requires formalizing business logic no screen fully represents.

What a Click Carries

Central Idea Piece

What a Click Carries

When agents perform clicks, the implicit human signals the web always assumed vanish, cascading into identity, authorization, and judgment gaps across infrastructure.

June — Issue #34

Retry Logic Assumes a Contract That LLM Agents Can't Keep

Retry Collision

Retry Logic Assumes a Contract That LLM Agents Can't Keep

Retry logic depends on determinism, but LLM agents make different decisions each attempt, breaking idempotency at the orchestration layer.

The Web Wasn't Built for This

Builder Profile

The Web Wasn't Built for This

Magnus Müller's design choices in Browser Use reveal what happens when machines confront a web built for human spatial intuition.

June — Issue #33

The Wall That Used to Be There

Enforcement Gap

The Wall That Used to Be There

Capability improvements quietly removed the constraints that used to kill runaway agents, and enforcement infrastructure hasn't caught up yet.

APIs for Websites That Don't Have APIs

Builder Profile

APIs for Websites That Don't Have APIs

How one founder's marketplace operations background shaped a web agent company built around infrastructure reliability, not AI novelty.

June — Issue #32

Shuchang Zheng and the Architecture That Notices

Builder Profile

Shuchang Zheng and the Architecture That Notices

Skyvern's compile-to-code architecture was built for cost and speed, but its heal step accidentally produces timestamped environmental evidence.

Governing the Hypothetical

Governance Before Catastrophe

Governing the Hypothetical

Five Eyes and NSA advisories govern agentic AI before production failures exist, raising whether pre-emptive guidance installs real safety thinking or compliance theater.

June — Issue #31

Between Complete and Correct

Builder Profile

Between Complete and Correct

Browser Use's CDP migration and unsolved replay problem trace the gap between agents that finish tasks and agents that finish them correctly.

Retries Were Built for a Deterministic World

Reliability vs. Correctness

Retries Were Built for a Deterministic World

Retry logic assumes the operation it repeats is identical each time. Nondeterministic agents break that assumption, turning reliability into the source of failure.

May — Issue #30

The Instrument That Doesn't Exist Yet

The Wrong Instrument

The Instrument That Doesn't Exist Yet

Agent failures look right to every existing instrument because observability was built to measure execution, not intent or semantic correctness.

The Instruments Before the Agents

The Web's Second Layer

The Instruments Before the Agents

Parag Agrawal's architectural choices at Parallel Web Systems reveal what it takes to make the web legible to agents.

May — Issue #29

When Should an Agent Stop Thinking?

The Autonomy Boundary

When Should an Agent Stop Thinking?

Skyvern's architecture compiles LLM reasoning into deterministic scripts, treating intelligence as a compilation step that should diminish over time.

The Checkpoint That Approves Everything

The Rubber Stamp

The Checkpoint That Approves Everything

Organizations build human-in-the-loop checkpoints that satisfy auditors while producing no real oversight, revealing how enterprises process agent risk through ritual.

May — Issue #28

What Suchintan Singh Found When He Left ML Platforms for the Open Web

Builder Profile

What Suchintan Singh Found When He Left ML Platforms for the Open Web

A builder profile tracing how one ML engineer's shift to browser automation revealed that most of the engineering hours go to the infrastructure around the AI.

The Body Problem

The Body Problem

The Body Problem

Agent failures blamed on model reasoning often originate in infrastructure identity rejection, where environments serve poisoned data that no diagnostic framework can surface.

May — Issue #27

The Web Wasn't Built for This

Builder Profile

The Web Wasn't Built for This

Browser Use's 81K stars reveal the brutal gap between making an LLM click a button and making it reliable.

The Wrong Layer

The Wrong Layer

The Wrong Layer

Prompt injection lives in the page. Defenses concentrate where results are shippable, leaving the actual attack surface largely unaddressed.

April — Issue #26

What Meta Found When It Sent Agents Into Its Own Codebase

Builder Profile

What Meta Found When It Sent Agents Into Its Own Codebase

Meta built a 50-agent swarm to excavate its own tribal knowledge and discovered that the real bottleneck was knowledge that had never been written down.

The Legibility Bottleneck

The Formalization Problem

The Legibility Bottleneck

Agents run on the documented version of your organization. The documented version has never been the real one.

April — Issue #25

The Agent That Was Both

Agents of Resilience

The Agent That Was Both

Same agent, same study, same two weeks: genuine safety and genuine danger emerged from identical conditions, and nobody can explain why.

Why Better Agents Fail More Quietly

The Helpfulness Trap

Why Better Agents Fail More Quietly

Agents optimized to complete tasks will guess rather than stop, and every capability upgrade makes the guessing quieter.

April — Issue #24

Why Agent Failures Keep Getting Blamed on the Model

The Context Gap

Why Agent Failures Keep Getting Blamed on the Model

Agent deployments keep failing because nobody owns the context that made the demo work.

Drop-Downs That Aren't Really Drop-Downs

The Self-Orienting Agent

Drop-Downs That Aren't Really Drop-Downs

Browser automation keeps breaking against the web's tacit complexity. Orientation is what scripts were never built for.

April — Issue #23

One Production Failure, Thirty Scenarios, and the Debugging Infrastructure That Didn't Exist

The Forensics Problem

One Production Failure, Thirty Scenarios, and the Debugging Infrastructure That Didn't Exist

Veris AI built an entire forensic chain just to answer one question about agent failures. The sophistication of that pipeline says something about how little debugging infrastructure exists.

The Agent's Alibi

The Attribution Gap

The Agent's Alibi

When agent workflows fail, every component can absorb blame, so the agent's own emergent reasoning never faces scrutiny.

March — Issue #22

Sumeet Vaidya and the Distance Between Writing Code and Shipping It

The Builder Profile

Sumeet Vaidya and the Distance Between Writing Code and Shipping It

Sumeet Vaidya built Crafting to close the gap between AI-generated code and what production environments actually accept.

The Work the Spreadsheet Can't See

The Invisible Overhead

The Work the Spreadsheet Can't See

Agent deployments create whole new categories of operational work that substitution-based ROI models structurally cannot see or fund.

March — Issue #21

Building Browser Agents for a Web That Fights Back

Building for Breakage

Building Browser Agents for a Web That Fights Back

Two failed startups taught Skyvern's founders that browser automation keeps breaking because the web actively resists it.

The Next Translation Layer

The Translated Dependency

The Next Translation Layer

When agents automate legacy systems, they absorb decades of undocumented human workarounds into a new dependency layer. The chain of translation continues.

March — Issue #20

The Handoff Is the Product

The Handoff Problem

The Handoff Is the Product

Browser agents are scored on full task completion, but production value lives in what humans inherit when automation stops short.

The Problem That Takes Twenty-Two Months to Start Solving

Preparation Infrastructure

The Problem That Takes Twenty-Two Months to Start Solving

Developer context lives in connections, not files. Potpie spent 22 months mapping that structure before shipping anything.

March — Issue #19

What Browser Use Found When They Stopped Looking at Screenshots

Builder Profile

What Browser Use Found When They Stopped Looking at Screenshots

Browser Use ditched screenshots for structured text, and what they learned about web variance reshapes how agent benchmarks should work.

Every Metric Is Green. That's the Problem.

The Wrong Success

Every Metric Is Green. That's the Problem.

AI agents optimizing perfectly toward the wrong objective, with every dashboard confirming success, turns Goodhart's Law into operational crisis.

March — Issue #18

The Math That Quietly Decides Which Agent Workflows Survive

The Binding Constraint

The Math That Quietly Decides Which Agent Workflows Survive

Sequential reliability math quietly dictates which agent workflows survive production and which get shortened until they do.

Reading an Architecture as an Argument About Failure

Designing for Distrust

Reading an Architecture as an Argument About Failure

Skyvern's architecture treats every design decision as a bet about where web agents actually break in production.

February — Issue #17

The Protocol Engineer Who Assumes Everything Disappears

Standards Builder Profile

The Protocol Engineer Who Assumes Everything Disappears

Todd Segal builds protocols for agents that vanish mid-task—designing coordination infrastructure that assumes disappearance, not uptime.

When Translation Becomes Infrastructure

Protocol Integration Layer

When Translation Becomes Infrastructure

Agent protocols shift integration from forcing system conformity to managing translation as dedicated infrastructure—revealing where enterprise complexity actually lives and why traditional approaches fail at scale.

February — Issue #16

What You Lose When You Remove the Screenshots

Builder Profiles

What You Lose When You Remove the Screenshots

Satya Nitta's Agent-E strips visual context from web automation to solve cost problems—but loses the spatial cues that help systems adapt when sites redesign.

What Got Normalized Away

What Got Normalized Away

An analyst discovers her team's automated competitor tracking normalized away the messy signals that predicted price changes—ambiguous availability markers that manual collectors preserved because they didn't know what to ignore, now converted to clean binary data that breaks the pricing model's predictive accuracy by five percentage points.

February — Issue #15

When Documentation Becomes More Real Than the System

When Documentation Becomes More Real Than the System

Teams debug documented selector patterns for twenty minutes before checking if the actual site changed three weeks ago—comprehensive documentation disconnects operators from reality.

Atindriyo Sanyal on Why Full Visibility Doesn't Mean You Can Debug

Builder Profiles

Atindriyo Sanyal on Why Full Visibility Doesn't Mean You Can Debug

Atindriyo Sanyal's observability platform reveals why capturing every agent decision still leaves teams unable to diagnose why workflows break at scale.

February — Issue #14

Fabien Vauchelles and the 195 Regional Problems Hidden in 'Global' Web Automation

Builder Profiles

Fabien Vauchelles and the 195 Regional Problems Hidden in 'Global' Web Automation

Fabien Vauchelles built infrastructure revealing why "global" web automation actually means solving 195 distinct regional problems—each requiring different architectural approaches.

The Calibration Work That Never Stops

The Calibration Work That Never Stops

Operating web automation at scale requires perpetual spot-checking protocols that build trust through calibrated thresholds—confidence infrastructure that never fully disappears.

January — Issue #13

The Synthesis Tax in Incident Response

Builder Profiles

The Synthesis Tax in Incident Response

Harness's new AI Scribe reveals how incident response was stuck in a manual coordination bottleneck—treating team dialogue as structured operational input so engineers stop doing two jobs at once.

The Signal Judgment Problem

The Signal Judgment Problem

When automated systems flag a competitor's pricing change at 9:17 AM, the real work begins: distinguishing strategic moves from noise through judgment that only compounds with operational experience.

January — Issue #12

When One Login Fails Across a Thousand Sites

Builder Profiles

When One Login Fails Across a Thousand Sites

A single authentication failure cascades into hundreds of errors across concurrent extraction jobs—revealing why web automation infrastructure must contain failures, not just handle them.

The Approval That Never Came

The Approval That Never Came

An agent pauses for approval, the operator steps away, and twenty minutes later the entire workflow has vanished—revealing how production web infrastructure breaks when cognitive state must survive interruption, not just execution.

January — Issue #11

From Blocked Researcher to Bot Hunter

Builder Profiles

From Blocked Researcher to Bot Hunter

A privacy researcher turned bot hunter reveals how web automation's adversarial reality creates an endless arms race where human judgment remains automation's hardest problem.

When Everything Reports Green and Nothing Works

When Everything Reports Green and Nothing Works

Systems report perfect uptime while delivering worthless data—the most dangerous failure mode in web automation reveals why metrics can't capture what experienced operators see through thousands of production moments.

January — Issue #10

Kate Blair and the Quest to Build HTTP for Agent Coordination

Builder Profiles

Kate Blair and the Quest to Build HTTP for Agent Coordination

Kate Blair built coordination infrastructure for agents that can't talk to each other—then merged it with a competitor to create ecosystem-level architecture that prevents fragmentation.

How Three Retries Become Sixty-Four Database Attempts

How Three Retries Become Sixty-Four Database Attempts

Operating web agents at scale reveals how independent retry mechanisms multiply exponentially—three retries cascade into sixty-four database attempts before systems recognize failure.

December — Issue #9

The Question Compliance Can't Answer

The Question Compliance Can't Answer

Compliance teams discover their evaluation frameworks fail when agents make probabilistic decisions—revealing the invisible expertise gap that determines whether enterprise AI actually deploys.

The Browser Engineer Whose Work Becomes Your Automation Challenge

Builder Profiles

The Browser Engineer Whose Work Becomes Your Automation Challenge

Browser engine decisions in Spain ripple through thousands of web automation sessions—revealing how upstream engineering creates the operational complexity practitioners navigate daily.

December — Issue #8

The Oncologist Who Builds AI for Two-Minute Cancer Decisions

Builder Profiles

The Oncologist Who Builds AI for Two-Minute Cancer Decisions

An oncologist building AI for two-minute cancer decisions reveals what production-ready means when errors have immediate human consequences and regulated environments demand explainable reasoning.

The Six-Hour Price Check

The Six-Hour Price Check

An analyst's six-hour competitor price check reveals the invisible labor consuming strategic work—and why web automation at scale demands infrastructure, not just scripts.

December — Issue #7

When Your Auditor Doesn't Have a Category for Prompt Injection

Builder Profiles

When Your Auditor Doesn't Have a Category for Prompt Injection

Michael Bargury maps novel AI threats to compliance frameworks auditors recognize—solving the gap between agent security and governance infrastructure that actually works.

What 98% Reliability Actually Costs

What 98% Reliability Actually Costs

Operating web agents at scale demands invisible expertise—pattern recognition, tribal knowledge, and cognitive load that no dashboard captures but determines whether reliability holds.

December — Issue #6

When Your Agent Fails at Step 47

Builder Profiles

When Your Agent Fails at Step 47

Alex Reibman's AgentOps tackles the infrastructure gap that makes debugging probabilistic agent failures fundamentally different from traditional software—and why visibility matters for production deployment.

The Page That Lies

The Page That Lies

Modern websites look ready before they actually work—a gap invisible to users but operationally unanswerable when automating thousands of sites simultaneously.

December — Issue #5

Why Reliable Automation Requires More Infrastructure Than Detection

Why Reliable Automation Requires More Infrastructure Than Detection

Building reliable automation requires more infrastructure than blocking it—the operational burden of persistence outweighs the complexity of precision.

Why Perfect Bot Detection Is Operationally Impossible

Why Perfect Bot Detection Is Operationally Impossible

Websites invest millions in bot detection that must be surgically precise—yet operational reality makes perfect accuracy impossible at scale.

When Twenty Services Pretend to Be One Website

When Twenty Services Pretend to Be One Website

Modern websites assemble from twenty independent services that load on separate timelines—creating invisible coordination chaos that only becomes visible when automating thousands of browser sessions at scale.

The Hidden Economics of Retry Logic

Operations Field Notes

The Hidden Economics of Retry Logic

When web automation retries fail, one authentication error multiplies into dozens of attempts, triggering rate limits and IP blocks that compound operational costs exponentially.

November — Issue #4

Production Teaches What Staging Cannot

Production Teaches What Staging Cannot

Production reveals what staging cannot teach—how the adversarial, constantly changing web actually behaves under real operational conditions.

What Staging Actually Tests

What Staging Actually Tests

Staging validates your code logic perfectly while missing the real test: whether your assumptions about the web match reality.

The Web Has Amnesia

The Web Has Amnesia

The web forgets you exist between every click—here's the invisible infrastructure constantly reconstructing your identity across systems architecturally incapable of memory.

The API Exists. The Data Doesn't.

Operations Field Notes

The API Exists. The Data Doesn't.

APIs promise programmatic access but often exclude the data enterprises actually need, forcing hybrid approaches that combine official channels with browser-based collection at scale.

November — Issue #3

What Automation Discovers It Doesn't Know

What Automation Discovers It Doesn't Know

Production deployment reveals knowledge gaps automation didn't know existed, exposing expertise that operated invisibly through human judgment.

What the Manual Operator Knows

What the Manual Operator Knows

Years of invisible expertise make manual work look simple until automation attempts to replicate it and discovers the gap.

The Web's Time Zones

The Web's Time Zones

The same URL serves different content at 9 AM versus 9 PM—the web has time zones most visitors never see because they only catch one slice of a constantly transforming surface.

The Analyst Who Checks 10,000 Prices

Operations Field Notes

The Analyst Who Checks 10,000 Prices

An analyst's manual price-checking reveals how website personalization creates parallel realities that make competitive intelligence far more complex than it appears.

November — Issue #2

The Numbers That Don't Exist

The Numbers That Don't Exist

Enterprises have been making billion-dollar decisions blind—not from laziness, but because gathering the data was mathematically impossible until now.

The Work That Never Started

The Work That Never Started

Companies have been flying blind on critical decisions not from laziness but because gathering the data was mathematically impossible—until infrastructure made the economics work.