--- title: "When Should an Agent Stop Thinking?" canonical: "https://current.tinyfish.ai/issue/29/practitioners-corner/article/16266/when-should-an-agent-stop-thinking" publication: "TinyFish" publisher: "TinyFish" language: "en-US" guide: "https://current.tinyfish.ai/llms.txt" format: "The Autonomy Boundary" author: "Rina Takahashi" author_url: "https://current.tinyfish.ai/team/25" issue: 29 section: "practitioners-corner" section_name: "Practitioner's Corner" published: 2026-05-13 updated: 2026-05-14 description: "Skyvern's architecture compiles LLM reasoning into deterministic scripts, treating intelligence as a compilation step that should diminish over time." word_count: 650 source_links: 8 --- # When Should an Agent Stop Thinking? *By [Rina Takahashi]()* *May 13, 2026* [Read on the website]() Suchintan Singh builds web agent infrastructure at [Skyvern](https://www.ycombinator.com/companies/skyvern), a YC S23 company. The workflows Skyvern automates run on government portals, insurance carrier dashboards, vendor procurement systems. Surfaces with no API. The only interface is a browser and a human clicking through forms. These aren't temporary gaps waiting for modernization. They process payroll, register businesses, renew licenses. They were built for humans, and they'll outlast most of the software written to replace them. So what kind of intelligence do you actually need to operate on them reliably? Singh's answer is architectural. Before founding Skyvern, he spent years [building ML platforms](https://www.skyvern.com/blog/author/suchintan/page/3/) at Faire and Gopuff, work that sharpened a distinction most AI discourse blurs: an ML model reasons through ambiguity, while an engineering system executes a known path. Where you draw the boundary between those two modes is the load-bearing design decision. Skyvern's [explore-then-replay architecture](https://www.skyvern.com/blog/asking-ai-to-build-scrapers-should-be-easy-right/) puts the boundary in an unusual place. In "explore mode," an LLM-guided agent navigates a workflow for the first time. It figures out the form fields, the conditional logic, the page transitions. Then it compiles that trajectory into a deterministic script. No model in the loop. Faster, cheaper, predictable. If the site changes and the script breaks, the LLM wakes up, re-learns the path, and recompiles. The naive version of this is macro recording, and it's been tried for decades. Recorded scripts are brittle because they capture the click target but lose the reason behind it. Singh's team captures intent metadata alongside every action during the explore run. When a compiled script encounters a shifted DOM, it can remap selectors using the original intention ("extract transaction rows") instead of failing because a CSS class changed. The intent is durable. The selectors are ephemeral. Skyvern's [planner-actor-validator loop](https://www.skyvern.com/blog/skyvern-2-0-state-of-the-art-web-navigation-with-85-8-on-webvoyager-eval/) adds texture to where intelligence stays. The planner holds the high-level goal. The actor executes the step. The validator checks the screen afterward: did a popup block the click? Did the page fail to load? A [Delaware government portal](https://www.skyvern.com/blog/asking-ai-to-build-scrapers-should-be-easy-right/) that goes offline nights and weekends, or occasionally requires you to call the IRS to proceed, needs intelligence in the loop. A routine form field that hasn't changed in three years does not need an LLM inference call. Even with the judgment-vs-determinism question resolved architecturally, roughly 60% of engineering effort in production browser automation goes to anti-bot infrastructure: authentication challenges, fingerprinting, session management. The compilation boundary matters. Most of the engineering effort lives elsewhere. The explore-then-replay design makes a specific claim about where judgment earns its cost in agent systems: at first contact with an unfamiliar surface, and when something breaks. After that, the compiled path carries the load. Compiled steps are also auditable in ways that LLM-guided steps are not, which matters the moment a compliance team asks why a form was filled the way it was. The architecture is designed to need less intelligence over time. And the long-term skill it implies is quieter than building more capable agents: recognizing which steps stopped requiring a decision a long time ago. ## Things to follow up on... - **Reliability compounds against you:** At 85% per-action accuracy, a 10-step workflow succeeds only about 20% of the time end-to-end, a compounding problem that [Temporal's analysis](https://temporal.io/blog/ai-reliability-is-a-decade-old-problem) argues the industry is largely ignoring in favor of reasoning capability investment. - **88% trace to infrastructure:** A taxonomy of 591 documented agent incidents found that the vast majority of classifiable failures originate in [missing guardrails, permissions, and monitoring](https://clyro.dev/blog/the-5-ai-agent-failure-modes-why-they-fail-in-production/) rather than model quality. - **Cascading errors across agents:** One misread chart propagated "10.5K units" as "105K units" through a multi-agent pipeline, leading to millions in unnecessary purchases, a failure mode that [single-agent evaluation can't predict](https://steady.page/en/software-insights/posts/216609a2-9d5f-4ec1-84e9-87891b010ec5). - **The benchmark-to-production gap:** Skyvern co-created the [WebBench benchmark](https://www.halluminate.ai/blog/benchmark) specifically to test agents in cloud infrastructure rather than local browsers, and even there, the best fully automated agent completes only 46.6% of non-read tasks on the real-world web. ## More - [Practitioner's Corner — Issue 29]() - [Issue 29]() - [Publication guide]()