Three Territories of Web Automation Reliability

At TinyFish, we see a recurring pattern when teams first evaluate their web automation needs: they prototype with scripts that work perfectly in testing, see consistent results across a few dozen runs, then scale to production. Within weeks, they're back. The automation still works—the infrastructure doesn't match the operational demands. The site structure changes that were minor annoyances in testing become operational crises when dashboards go blank and downstream systems stall.

Operating web agents across thousands of sites reveals three distinct operational territories where repeatability behaves differently. Teams hit trouble when they misjudge which territory their use case actually occupies.

Three Operational Territories

Start with what looks like a simple question: how often do you need this data? Production shows that what happens when something breaks determines your infrastructure requirements.

Occasional extraction means you need data once or intermittently. A script that works today and breaks next month is operationally fine because you'll fix it when you need it again. Authentication flows change, class names shift, manual intervention is acceptable. Can someone fix this when it breaks? That's the operational threshold.

Scale that same script to continuous monitoring—daily or hourly runs—and those site changes become operational problems because your dashboards go blank and someone notices within hours. You need retry logic and graceful degradation, though occasional gaps are tolerable. Missing one day's pricing data creates inconvenience without crisis. Can the system recover without human intervention? That becomes the new threshold.

The third territory is where most teams underestimate their requirements. Transactional workflows mean every run must succeed because downstream systems depend on it. A Fortune 500 retailer we studied needed daily pricing across 50,000 products from 200 competitors—their revenue decisions depended on that data arriving reliably. When authentication failures or rate limiting hit, these became business continuity issues. A system that works 95% of the time means 5% of your transactions fail. Can you build SLAs around this? This question reveals infrastructure adequacy.

Where Teams Misjudge

Occasional-use tools work beautifully in testing. You see consistent results, clean data, no obvious brittleness. Production exposes different failure modes. The script that handled a dozen test runs gracefully hits dynamic content loading issues at scale. The authentication that worked reliably in development encounters session timeouts across thousands of concurrent runs. The parsing logic that seemed robust breaks when sites A/B test new layouts.

We see this pattern when evaluating customer architectures: teams building distributed systems for continuous monitoring when they actually need transactional reliability, or trying to scale occasional-use scripts to continuous operation without changing infrastructure. By the time they realize the mismatch, they've built dependencies on brittle systems.

Two questions work before you commit to architecture: How quickly must you know when something breaks? If the answer is "within minutes," you're operating in continuous monitoring territory, even if you're only running once daily. What happens downstream if a run fails? If the answer involves blocked workflows or broken dashboards, you need transactional-grade infrastructure, even if failures seem rare in testing.

This spectrum runs from scripts to scheduled jobs to infrastructure designed for reliability from the ground up. Understanding which territory your use case occupies determines what you build. Misjudge it, and you'll spend your time fighting architecture when you should be using data.