Vision
Vision
When the Web's Visitors Start Buying
In 1994, a plain text file solved the problem of non-human visitors on the web. Robots.txt told crawlers where they could look. It went nearly three decades without formal standardization. It didn't need any. The convention held because crawlers only read, and reading is cheap. Nobody gets billed when a crawler misreads a page.
The web is building that same kind of infrastructure again — structured files, machine-readable endpoints, permission signals — for a new generation of automated visitors. The pattern rhymes with the search era so neatly that the analogy holds longer than it should. These visitors transact.
When the Web's Visitors Start Buying
In 1994, a plain text file solved the problem of non-human visitors on the web. Robots.txt told crawlers where they could look. It went nearly three decades without formal standardization. It didn't need any. The convention held because crawlers only read, and reading is cheap. Nobody gets billed when a crawler misreads a page.
The web is building that same kind of infrastructure again — structured files, machine-readable endpoints, permission signals — for a new generation of automated visitors. The pattern rhymes with the search era so neatly that the analogy holds longer than it should. These visitors transact.
Two Lenses
The Web Is Learning to Talk Back
Two years ago, the web had no vocabulary for talking to non-human visitors. A plain text file was the first attempt, a business card left at the door for AI systems that mostly ignored it. Now a Romanian grocery store exposes executable functions to agents, the W3C is drafting formal specifications for how browsers should welcome them, and the infrastructure to govern all of this trails far behind the infrastructure enabling it.
Agents Don't Browse
As the web rebuilds its infrastructure for agent visitors, a familiar industry is forming around it. SEO reshaped the commercial web around search; surely "Agent Experience Optimization" will do the same. The analogy is comforting and already attracting real money. Meanwhile, early e-commerce traffic data contains a small number worth pausing on: 2.31% of agent visits go straight to checkout. Agents buy.
AI Agents Are Already Shopping Your Store — They Just Can't Buy Anything
CONTINUE READINGWho's Moving First
Over 780 websites now serve llms.txt files, the proposed standard that tells AI systems what to pay attention to on a site. The adopter profile is strikingly narrow: developer-facing companies with API documentation. Cloudflare, Vercel, Stripe, Coinbase. Not retailers, not media companies, not banks.
Then there's Google. Its search team publicly compared llms.txt to the abandoned keywords meta tag. In December 2025, someone spotted Google had quietly added the file to its own Search Central docs. Hours later, it vanished. A spokesperson offered only a cryptic emoji.
The pattern worth watching here is the shape of adoption, not the number.
The Agent-Readable Web
