Practitioner's Corner

Practitioner's Corner




Reasoning Once

Further Reading


© TinyFish, Inc.. All rights reserved.





Reasoning Once

Further Reading


Delaware's government portal goes offline at night. A pharmacy invoice site hides its download link behind a field that looks like a drop-down but behaves like a text box. A county records office redesigns its layout without warning. None of these have APIs. Humans navigate the mess without thinking. Scripts shatter against it. Somewhere between those two facts sits a bottleneck that more model capability alone won't close. One engineer spent years watching it consume his ML pipelines before deciding to build directly against it.
Delaware's government portal goes offline at night. A pharmacy invoice site hides its download link behind a field that looks like a drop-down but behaves like a text box. A county records office redesigns its layout without warning. None of these have APIs. Humans navigate the mess without thinking. Scripts shatter against it. Somewhere between those two facts sits a bottleneck that more model capability alone won't close. One engineer spent years watching it consume his ML pipelines before deciding to build directly against it.
In surgical teams, individual surgeons hold 83% of the information they need. Anesthesiologists hold 87%. Competent people, by any measure. Yet a study of perioperative handoffs found only 27% of critical knowledge was shared across the full team. The failure lived in the gaps between skilled people, not in the skill itself.
Agent deployments are breaking in a place that looks a lot like this. Ninety percent of enterprise pilots never reach production scale, and the postmortems keep landing on the same explanation: the model wasn't capable enough. That explanation keeps being wrong. And the cause lives somewhere most diagnostic tools don't reach.
In surgical teams, individual surgeons hold 83% of the information they need. Anesthesiologists hold 87%. Competent people, by any measure. Yet a study of perioperative handoffs found only 27% of critical knowledge was shared across the full team. The failure lived in the gaps between skilled people, not in the skill itself.
Agent deployments are breaking in a place that looks a lot like this. Ninety percent of enterprise pilots never reach production scale, and the postmortems keep landing on the same explanation: the model wasn't capable enough. That explanation keeps being wrong. And the cause lives somewhere most diagnostic tools don't reach.
A workflow costing $0.50 per LLM-in-the-loop execution becomes $50,000 a month at scale. That math is why a quiet architectural pattern keeps surfacing: let the agent reason through a workflow once, compile that reasoning into deterministic Playwright code, then replay the script on every subsequent run. Think expensive thoughts exactly once.
Skyvern's implementation offers the clearest public numbers. Their explore-then-replay model cuts per-run costs from $0.11 to $0.04 and execution time from 279 seconds to 120. The runs become deterministic. What makes the pattern durable is intent metadata captured during exploration, so when a site inevitably changes, the system heals using semantic understanding rather than brittle CSS selectors.
The deeper idea here resembles institutional memory. A senior employee learns a process, writes it down, and the next person follows the notes. The LLM's initial exploration is the learning. The compiled code is the documentation. And like good institutional knowledge, it degrades gracefully: when the notes no longer match reality, the system knows enough about the original intent to adapt rather than fail silently.
A workflow costing $0.50 per LLM-in-the-loop execution becomes $50,000 a month at scale. That math is why a quiet architectural pattern keeps surfacing: let the agent reason through a workflow once, compile that reasoning into deterministic Playwright code, then replay the script on every subsequent run. Think expensive thoughts exactly once.
Skyvern's implementation offers the clearest public numbers. Their explore-then-replay model cuts per-run costs from $0.11 to $0.04 and execution time from 279 seconds to 120. The runs become deterministic. What makes the pattern durable is intent metadata captured during exploration, so when a site inevitably changes, the system heals using semantic understanding rather than brittle CSS selectors.
The deeper idea here resembles institutional memory. A senior employee learns a process, writes it down, and the next person follows the notes. The LLM's initial exploration is the learning. The compiled code is the documentation. And like good institutional knowledge, it degrades gracefully: when the notes no longer match reality, the system knows enough about the original intent to adapt rather than fail silently.
Self-healing fallback: When compiled scripts break against changed pages, intent metadata triggers targeted LLM recovery rather than full re-exploration, keeping most runs cheap and deterministic
Call elimination: Unlike prompt caching or model routing, which reduce per-call cost, compile-to-code removes the model from known execution paths entirely
Reasoning dependency: The pattern became viable only after models could capture structured intent behind ambiguous web elements, not just generate click sequences
Caching architecture: Skyvern's Jinja2-based cache keys enable 10-100x faster execution for matched workflows, with automatic fallback to AI discovery on failure
Formalized pattern: A March 2026 technical essay describes the four-phase model as "deliberate, analyze, generate, route," suggesting the approach is generalizing beyond any single implementation