--- title: "You Can't Stage Someone Else's Website" canonical: "https://current.tinyfish.ai/issue/48/practitioners-corner/article/39603/you-cant-stage-someone-elses-website" publication: "TinyFish" publisher: "TinyFish" language: "en-US" guide: "https://current.tinyfish.ai/llms.txt" format: "Lead Feature" author: "Sable Whitford" author_url: "https://current.tinyfish.ai/team/201" issue: 48 section: "practitioners-corner" section_name: "Practitioner's Corner" published: 2026-09-24 updated: 2026-09-25 description: "Agent systems that depend on third-party environments can't reach pre-production confidence, because the environment isn't yours to copy or predict." word_count: 812 source_links: 7 --- # You Can't Stage Someone Else's Website *By [Sable Whitford]()* *September 24, 2026* [Read on the website]() A housing association in Wales built an automation to handle food-bank referrals. It depended on a third-party website. The week it went live, the food bank [changed both its approach and its website](https://digitalpublicservices.gov.wales/automation-and-artificial-intelligence-discovery-report/findings), and the integration was dead. The rule the team wrote down afterward: "only automate things that are completely within our control." You can read that as timidity. I read it as a team correctly locating the edge of what they were able to test. ## What breaks the assumption Every pre-deployment practice we have rests on one idea: that you can reproduce the conditions your software will meet. Stand up a staging environment, run the suite, ship with some confidence that the real thing behaves like the rehearsal. I've spent years arguing that staging is a comforting fiction even when you own both ends of the wire. When one end belongs to another company, it isn't even that. They change their interface on their schedule, for their reasons, and nobody sends you a note. A button moves, a form field gets renamed, a workflow grows a step. None of it is aimed at you. The person who shipped that change doesn't know your agent exists and wouldn't care if you told them. Some changes are aimed at you. Organizations that notice automated traffic add CAPTCHAs, throttle requests, restructure pages to break scrapers, or block you outright. That's what makes this structural rather than merely expensive: the environment moves in response to being used, so measuring it changes what you're measuring. A benchmark built from a snapshot of an interface starts going stale the moment it's captured, and running the agent against the live site to find out whether the snapshot still holds is exactly the traffic that gets you blocked. You cannot stage a phone call to an insurance company's claims line. You cannot rehearse a purchase on a live retail site without buying something. ## Why bounded agents are good engineering A [study of 86 agent systems](https://arxiv.org/html/2512.04123v4) found that 68% executed fewer than ten steps before a human had to intervene. Almost half needed one inside five. The systems touching the outside world ran tightest of all: fixed action sequences, mandatory approval at decision points, read-only modes, sandboxed verification before anything hit the real interface. Against an assumption of full autonomy, those numbers look like a technology that isn't finished. But look at what the constraints actually do: they're the ordinary response to an untestable environment. Short execution horizons cap how far a system can drift before someone checks its work, and narrow scopes limit what it can reach while drifting. The human checkpoint is sitting exactly where pre-deployment testing would sit, if pre-deployment testing were on offer. A water utility in Portugal [lost a working automation](https://doi.org/10.1108/JPBAFM-09-2024-0182) to a routine laboratory website redesign. It moved the follow-on projects onto back-end database integrations and started budgeting ongoing maintenance for anything still touching a front end. Same lesson as the Welsh team, bought at a higher price. ## Where the confidence has to come from Distributed systems engineering went through a duller version of this. Teams spent years trying to make staging a faithful replica of production, gave up, and got observability and resilience engineering out of the surrender: instrument everything, deploy in small increments, detect early, recover quickly. Production was too complicated to copy, but it was ours. Here it isn't. The environment runs on someone else's calendar and may change specifically because you showed up. That leaves monitoring, containment and recovery doing the job that pre-deployment testing does everywhere else, not as a fallback but as the primary control. The Welsh rule is the clean way out, and most organizations won't take it, because the work worth automating usually sits on the other side of the boundary. If you're crossing it anyway, be honest about which controls are holding the weight. ## Things to follow up on... - **Counterparties can just refuse:** Amazon blocked Meta's Muse agent from operating on its site, objecting that the agent didn't identify itself and violated the retailer's terms — a concrete case where [recipient refusal invalidated the agent's technical capability entirely](https://www.geekwire.com/2026/amazon-blocks-metas-muse-ai-assistant-in-new-standoff-over-agentic-shopping/). - **Ordinary interfaces steer agents:** The peer-reviewed DECEPTICON study found that web agents were steered toward unwanted outcomes by standard dark patterns in over 70% of tasks, compared with 31% for humans, even when [task completion scored as successful](https://arxiv.org/abs/2512.22894). - **Monitoring at scale is triage:** Anthropic reported its monitors processed over a billion agent decisions in August 2026, blocking roughly one in 47,000 and funneling about 100,000 weekly flags down to [around 50 cases for human review](https://www.anthropic.com/institute/measuring-pace-of-ai-development). - **NIST on what logging doesn't solve:** A March 2026 NIST report identifies fragmented logging, the cost of human review, and uncertain measurement validity as continuing challenges, arguing that [deployed-system monitoring remains an organizational problem even after technical ingestion is in place](https://www.nist.gov/news-events/news/2026/03/new-report-challenges-monitoring-deployed-ai-systems). ## More - [Practitioner's Corner — Issue 48]() - [Issue 48]() - [Publication guide]()