The Shopping Cart That Made State Management Adversarial

In June 1994, Lou Montulli solved HTTP's statelessness problem with a small text file—a "cookie"—that websites could store in browsers to remember state between page loads. His team at Netscape had rejected simpler alternatives like permanent user IDs because they would enable cross-site tracking. Cookies were designed with privacy protections: they would only exchange information between users and the specific websites they visited.

Within two years, advertising networks learned to do exactly what Montulli had tried to prevent. When a user visits a site with a DoubleClick ad pixel, a cookie gets set by the doubleclick.net domain. Visit another site with DoubleClick ads, and the same cookie ID gets sent. DoubleClick launched in January 1996, and by 2000, the FTC was investigating their tracking practices.

The architecture had a gap: nothing stopped third-party domains embedded in a page from setting their own cookies. In 1997, RFC 2109 specified that third-party cookies should not be enabled by default. Browser vendors ignored the recommendation. The advertising industry had discovered something valuable.

What technical standards couldn't accomplish, regulation eventually attempted. The EU's ePrivacy Directive, amended in 2009, established that websites must obtain user consent before storing cookies—except for "strictly necessary" ones like shopping carts and login sessions. Safari began blocking third-party cookies in 2017, Firefox in 2019. Chrome announced plans to phase them out, then abandoned the effort in 2024 after finding that removal would cause a 34% drop in programmatic ad revenue.

The Infrastructure Challenge Nobody Sees

The cascade shows up as infrastructure complexity when you're building web agents that operate across thousands of sites.

First-party cookies are essential for authentication, but browsers increasingly treat them as suspicious. Session timeouts vary wildly: some expire after 15 minutes of inactivity, others after 24 hours, some never. There's no standard. Reliable automation requires monitoring every session independently and handling refresh logic that differs by domain.

Then there's cookie scoping. A cookie set on example.com might not be accessible to shop.example.com, or vice versa, depending on how the site configured the domain attribute. Authentication flows break unpredictably when cookies don't propagate correctly across subdomains. At TinyFish, we've seen sites where login succeeds but the session cookie only works on specific subdomains. Works fine for human users clicking through pages. Breaks automated workflows that need deterministic behavior.

Consent frameworks complicate this further. When a site shows a consent banner, the page exists in two states: pre-consent (where functionality is limited) and post-consent (where cookies enable full features). For web agents, this means every page load is potentially non-deterministic. The same URL can return different content depending on consent state. Infrastructure needs to detect consent banners, interact with them correctly, and verify that the resulting cookie state matches what the workflow requires.

The third-party cookie restrictions that browsers implemented to address tracking now affect legitimate automation. Sites that rely on third-party authentication providers (OAuth flows through Google, Microsoft, etc.) can break when browsers block third-party cookies. Anti-automation systems use cookie behavior as a detection signal. Agents that don't handle cookies exactly like browsers do get flagged.

The challenge isn't just managing cookies. It's managing them in ways that satisfy both browser restrictions and anti-bot detection, while navigating consent frameworks and unpredictable session timeouts. Cookie architecture specifically created a regime where maintaining state across the web requires navigating active resistance from browsers, regulations, and anti-automation systems. The same mechanism is simultaneously required for basic functionality and restricted because of how it was exploited.

Every consent banner, every third-party cookie restriction, every authentication challenge that treats cookies as essential yet potentially blocked—these are operational consequences of that 1994 decision to solve HTTP's statelessness with a local text file.

Montulli's reflection years later was telling. But the cascade was already in motion. A privacy-conscious solution became surveillance infrastructure, which triggered regulations, which made state management adversarial. The infrastructure challenges only become visible when you're operating web agents at production scale.