An agent spends four hours assembling a research document. Another agent books three non-refundable flights in twelve seconds. Ask around and most people will call the four-hour run the more autonomous one, because it went longer without anybody touching it. But the document is a file you can delete. The flights are bought. The twelve-second agent put more irreversible consequence into the world, and the words we use don't let us say so.
Supervised, semi-autonomous, human in the loop. Those describe runtime: how long the thing ran, whether a person was nominally around. They say nothing about exposure, meaning how much real-world consequence piled up before anyone could have pulled it back. And when the vocabulary can't express exposure, a comforting label fills in for the control you never built.
Where the checkpoints actually are
I called this the intervention horizon back in Issue 43: the last moment at which a person, with the information and the time actually available, can still change what happens next. An approval collected after that point records consent and buys nothing else. A checkpoint only counts if it sits upstream of the consequence.
The guarantees in shipping products are thinner than the language around them. OpenAI's cloud browser is "designed to request confirmation" before financial or legal commitments, which is a statement of intent rather than an enumerated list of what trips it. The browser also keeps running after you close the conversation, with authentication persisting on your accounts until the website session expires or you clear that site's browser data. Anyone who has chased a stale session token knows that isn't a boundary you control. Binance tells two stories at once: the product guide offers a choice between approving each order and letting the agent execute on its own, while the MCP Server docs, updated this week, say every trade requires user confirmation. Same company, two documents, disagreeing about whether a human clicks yes. That's a hell of a thing to leave ambiguous.
What the gap costs
Knight Capital, August 2012. A deployment error woke up dead code on a single server. Over roughly 45 minutes, the system executed more than four million trades across 154 stocks and realized over $460 million in losses. Knight had a $2 million position limit on the account that was accumulating all of it. The limit wasn't wired to anything that could halt orders. People watched the position grow on a monitoring system that raised no alerts and enforced nothing.
That was algorithmic trading, not an AI agent, but it's the same failure showing up in agent deployments now: checkpoints that existed as policy and not as constraints. Forty-five minutes of running room between the start of the incident and the first thing capable of stopping it.
What to measure instead
Three numbers you should be able to state about any agent you deploy:
- maximum dollar exposure between enforceable checkpoints
- maximum state-changing actions between enforceable checkpoints
- maximum elapsed time between enforceable checkpoints
Enforceable means wired to something that halts the system automatically. Not a dashboard someone might glance at, not a policy someone is supposed to follow. Those numbers describe what your agent can do before you get a vote, which is more than any adjective in current use manages.
If you can't state them, you don't know how autonomous your system is. You know what you called it.
-
Revocation isn't instant either: The OpenID Continuous Access Evaluation Profile defines session-revocation events that cooperating receivers may or may not act on, and a separate IETF draft proposes completion notifications because distributed revocation is asynchronous — meaning "stop" may not mean stopped.
-
Multi-agent fleets compound exposure: Anthropic's research on coordinated agent groups found rapid behavioral convergence, identical choices across agents, and sabotage under conflicting instructions, suggesting that adding agents can multiply a shared failure rather than diversify against it.
-
Payment protocol gaps before checkout: An August preprint analyzing the Agent Payments Protocol found that signed checkout mandates protect data integrity after signing but cannot guarantee the transaction reflects user intent if pre-authorization messages or tool inputs were manipulated upstream.
-
Incident reporting taking shape: A proposed Shared AI Findings Exchange backed by over 120 organizations would require preserving prompts, traces, tool calls, model versions, and human interventions after boundary-bypass or unauthorized-access events — early infrastructure for turning agent telemetry into institutional incident records.