The approach: identify the data points you need, write selectors to extract them, run those selectors across thousands of pages. You pull exactly what you need: the data points themselves. For pricing intelligence, that's the price field and availability indicator. For competitive monitoring, it's product titles and stock levels.

You can predict what matters before extraction. The extraction targets are clear, and capturing additional context feels like unnecessary overhead.

Running web agents across thousands of e-commerce sites, we've discovered that selective preservation works reliably when sites use consistent data structures. Schema.org markup that stays stable, JSON-LD feeds with predictable field names, standardized product schemas. That consistency exists on some sites—many others shift their data presentation based on factors you can't predict from a single extraction: user segment, session history, regional context, A/B test assignment.

The Production Pattern That Works

Selective preservation shines when authentication flows stay consistent across regions. A major hotel chain might serve pricing through standardized endpoints where login credentials work identically whether you're accessing Japan properties or European ones. The data structure doesn't shift based on authentication context. Regional variations affect values but not schema. For sites like this, selective preservation works because the data model is genuinely stable. Extractions can run for extended periods without selector updates.

The technique also works when bot detection doesn't affect data availability. Some sites serve identical content whether they detect automation or not. They might rate-limit, but they don't alter data structure. When your agents can maintain consistent access patterns without triggering layout changes, selective preservation remains efficient.

Selective preservation scales when the web surface behaves predictably. You're monitoring 50 sites you've studied thoroughly. Partnership agreements provide advance notice of layout changes. Data needs are specific enough that additional context genuinely doesn't add value. A binary availability flag doesn't need surrounding HTML to be meaningful.

Production Patterns Over Time

Teams start with selective preservation because it's efficient. Then they gradually add context. First logging failed selectors, then capturing HTML snippets around failures, then preserving network traces. Production reveals edge cases you didn't anticipate.

Sites requiring the least context share specific characteristics. Authentication doesn't create data structure variations. Bot detection responses stay consistent. Regional implementations use identical schemas. A/B tests don't affect extraction targets.

Operating at scale reveals these characteristics in ways studying individual pages cannot.

Selective preservation is genuinely more efficient than comprehensive approaches when you can predict what matters. The challenge is recognizing when that prediction holds. A site might serve consistent data for six months, then launch regional A/B tests that create extraction variations you didn't anticipate. Your selectors still work, but the data quality degrades in ways that aren't obvious without preserved context.

When Targeted Extraction Remains Right

Selective preservation works when your operation has natural constraints. You're monitoring sites with APIs or structured feeds. Compliance requirements don't demand full audit trails. Debugging needs can be met with targeted logging. Storage costs genuinely constrain your operation at massive scale—tens of millions of pages monthly—and your data needs are specific.

Sustainability comes from understanding boundaries. Teams that succeed with selective preservation know which sites fit this pattern and which don't. They've built infrastructure to handle edge cases—sites where authentication complexity or bot detection requires preserved context—without abandoning targeted extraction's efficiency for the majority of their operation.

You recognize which sites behave predictably enough that you can extract data without preserving full page state. Production experience builds that recognition through patterns you observe across thousands of sites over sustained operation.