In January 1996, the Basel Committee on Banking Supervision amended its capital accord to let banks use internal models for calculating market-risk capital requirements. The rules were specific: daily Value-at-Risk computation, a 99th-percentile confidence interval, a ten-trading-day holding period, at least one year of historical data. The same document required "rigorous and comprehensive" stress testing for events outside routine VaR output. The regulators knew the model was a simplification. They said so, in writing, in the same amendment that made the model official.
VaR had a quality that stress tests lacked. It produced a number. A single, daily, comparable number that could appear in board reports, annual filings, and regulatory submissions. Stress tests were episodic and qualitative. Over time, the number absorbed the concept it was meant to approximate. Risk, as experienced by the people managing it, increasingly meant the number the model produced.
The warnings were specific and early. In 1999, Artzner, Delbaen, Eber, and Heath published their coherent risk measures framework, demonstrating that VaR failed subadditivity: two portfolios combined could show higher VaR than the sum of their parts, meaning diversification could appear to increase risk under certain conditions. In 2001, Pafka and Kondor evaluated the widely used RiskMetrics methodology and found it assumed normally distributed returns while ignoring the fat tails present in actual financial data. Its apparent success, they argued, was partly an artifact of the chosen significance level.
The objections were published in respected venues, years before the stress arrived. When it did, correlations that historical data treated as stable shifted simultaneously, and losses accumulated in the tails that the models had assigned negligible probability. The map's limitations had been documented. Institutions had navigated by the map anyway, because the map gave them something the caveats couldn't: a daily, actionable, auditable number.
This pattern is worth holding in mind as agent benchmarks become the primary way organizations evaluate AI capability. WebArena reported a 14.41% agent success rate against 78.24% human performance on web tasks. WorkArena tested agents on enterprise-specific workflows and found similar gaps on the kind of multi-step, policy-laden tasks that define actual organizational work. Those numbers are specific and comparable, which makes them useful and gives them the same gravitational pull that VaR had.
More recent work has started testing what happens under stress. τ-bench introduced pass^k, measuring reliability across repeated trials, and found that even state-of-the-art agents succeeding on under half of individual attempts dropped below 25% success over eight consecutive runs in its retail domain. ReliabilityBench showed that semantically equivalent perturbations alone reduced success from 96.9% to 88.1%. These function as stress tests for agent capability, and like their financial predecessors, they exist alongside the single-number benchmarks without displacing them.
No public study yet maps the same agent's benchmark score directly against its live production performance. The controlled measurement exists and the production measurement doesn't, and that gap is where the interesting risk accumulates. Somewhere between the benchmark's calm and the world's weather.