When enterprises deploy web agents at scale, authentication becomes the first bottleneck. Not because logging in is technically hard, but because the web's identity layer was never designed for what it's doing now. The complexity traces back to a 1994 decision that seemed elegant at the time.
Netscape engineer Lou Montulli needed to solve a straightforward problem: how do you make a shopping cart remember items when every page load treats the browser like it's meeting the website for the first time? HTTP was stateless by design. Servers didn't track who you were between clicks.
Montulli's solution was deliberately minimal. Instead of building server infrastructure to track millions of shopping sessions, push that responsibility to users' machines. Let browsers store small bits of state—the HTTP cookie specification limited them to 4 kilobytes. MCI, Netscape's client, explicitly wanted this architecture. They didn't want servers managing partial transactions. Montulli's team even rejected permanent browser IDs that could track people across the web.
Infrastructure built for one purpose rarely stays contained. Within two years, the shopping cart mechanism became the identity mechanism. Logging in meant receiving a session cookie. Staying logged in meant that cookie traveling with every request. Browsers weren't just remembering items anymore. They were remembering who you were.
That shift created operational complexity enterprises still grapple with. Because browsers automatically attach cookies to every request, they introduced attack vectors that didn't exist before. Cross-site request forgery isn't just a security vulnerability—it's why enterprises need separate authentication flows for different trust levels, why session tokens need constant rotation, why "staying logged in" and "security" remain in tension. Session fixation means a single compromised cookie can hijack an entire authenticated session. The hybrid architecture—cookies storing identifiers, servers storing actual state—creates constant synchronization overhead.
When a session expires server-side but the cookie persists client-side, users see authentication failures that look like bugs. When cookies expire but server sessions remain active, enterprises waste resources tracking ghost sessions. Cross-domain authentication requires intricate cookie-sharing mechanisms that break differently in every browser.
Then privacy regulations transformed cookies from a technical mechanism into a compliance liability. Enterprises suddenly needed infrastructure to track cookie usage, manage consent preferences, prove compliance across every web property. Now browsers are restricting third-party cookies entirely. Safari, Firefox, and Brave block them by default; Chrome plans full deprecation.
Infrastructure repurposed beyond its original constraints creates operational friction when deployed at different scale and purpose.
People often frame this as a privacy issue. It's actually an architecture forcing function. Enterprises can't simply flip a switch. They're running authentication systems that assume cookies work a certain way, with millions of active sessions that can't break mid-migration. Single sign-on systems that relied on third-party cookies need complete redesign. The operational challenge isn't technical complexity—it's maintaining both old and new authentication flows simultaneously while ensuring nothing breaks.
This pattern explains why enterprises struggle with web agents today. Browsers were built for humans clicking through pages, not for automated systems making thousands of concurrent requests. Authentication was designed for individual sessions, not for fleets of agents that need to maintain state across distributed operations.
Cookies weren't a mistake. But infrastructure built for one scale and purpose creates friction when repurposed for another. When enterprises evaluate agent infrastructure, they should ask whether the system is built for what they're actually trying to do, or whether they're inheriting constraints from a different era. The shopping cart that became an identity system is still shaping what's possible on the web three decades later.
-
The original specification: Lou Montulli outlined his cookie design in a minimalistic four-page proposal that became the de facto standard browsers still follow today, despite multiple attempts at formal RFC specifications.
-
Third-party cookie emergence: Within two years of cookies launching, advertisers learned to hack the mechanism for cross-site tracking, leading to RFC 2109's failed attempt in 1997 to block third-party cookies by default—a recommendation Netscape and Internet Explorer ignored.
-
JWT as alternative: Modern enterprises increasingly use JWT tokens for client-side session state to avoid cookie-based authentication complexity, though this approach introduces its own tradeoffs around token size and revocation.
-
The domain problem: Netscape's original cookie specification struggled with cross-domain complexity that still has no perfect solution—DNS naming conventions don't map cleanly to security boundaries, creating the authentication friction enterprises face today.