The Instruments Before the Agents

Court rulings in Brazil. Regulatory codes across 60-plus jurisdictions. County-level HOA records buried in state government portals that each work differently. The ordinary texture of the web, once you try to read it with software instead of a browser.

Parag Agrawal founded Parallel Web Systems in 2024 around a specific conviction: agents will consume the web at orders of magnitude beyond human usage. The corollary is where the work lives. Every layer of web infrastructure, from crawling to indexing to ranking to retrieval, was designed for a user who clicks and scans. An agent that needs to reason over Brazilian court filings doesn't scan. It needs structured, dense, verifiable text delivered into a context window. The web doesn't offer that natively. So before agents can do useful work on the web, someone has to build the instruments that make the web legible to them.

The early product is honest about this. Agrawal described the original pitch on the Founders in Arms podcast: APIs deliberately too slow for human patience, designed for agents that would work asynchronously. Traction came only after the index reached sufficient scale to make retrieval fast. That pivot was the first instance of a pattern that defines the whole project: each solved bottleneck surfaces the next unsolved layer.

Harvey runs legal research through Parallel's private index, searching sources across dozens of jurisdictions that no consumer search engine has ever crawled. Opendoor navigates HOA research through wildly inconsistent government portals via Parallel's Task API. Different instruments, same underlying gap: the web's content exists but is effectively invisible to machines.

On their search product page, one detail captures the architectural bet cleanly: their ranker doesn't optimize for which URL sits at position one or two. Consumer search engines spend enormous effort on URL ranking because humans click links. Parallel's ranker spends its compute figuring out which tokens belong in an agent's context window. Same web. Different instrument entirely.

The newest instrument may be the most telling. Index, launched May 19, gives publishers a dashboard showing how agents consume their content and compensates them using a Shapley value model that estimates each source's marginal contribution to a completed agent task. The model measures how much a particular source actually mattered to a specific outcome, so compensation tracks contribution rather than reflecting bargaining power or raw access volume. When agents become primary readers, what content is worth becomes something infrastructure can measure. The economics of content access become a design problem. Publishers like The Atlantic and Fortune are early partners. The platform is designed to extend beyond Parallel's own agents, which means the instrument could eventually make this economic relationship legible across the broader agent ecosystem.

That cadence, as Agrawal told the Wall Street Journal, is what building instruments for an uncooperative surface looks like. The early APIs were too slow, so the index had to scale. The index scaled, so content economics surfaced as the next unsolved layer. The web wasn't designed to be read by machines, and each instrument that makes one dimension legible reveals the next dimension that isn't.