Anthropic's fleet-monitoring disclosure contains a ratio that reads like good news: roughly 100,000 agent transcripts flagged each week, triaged by a stack of classifiers, with about 50 cases reaching human review.
It is good news. It also describes what the reviewer is looking at. The classifiers absorb everything matching a pattern someone has already thought through, so what arrives on a human screen is the residue: cases that matched nothing. When the reviewer decides what one of them means — policy violation or parsing failure, escalate or annotate — that decision settles what the policy requires in this instance. Where it goes afterward is a separate question, and the public record on it is thin.
Information does move. Anthropic says flagged activity feeds back into classifier and control improvements. NVIDIA's OpenShell runtime is more direct: an agent that needs access beyond its sandbox can request it, and once a reviewer approves, the approved rule becomes part of that sandbox's live policy. Meta's concierge test for Muse, in which human contractors performed tasks on the assistant's behalf, was framed as feedback gathering for safety and feature work.
None of the three publicly documents a way of comparing one reviewer's decision with another's. OpenShell logs approvals inside a sandbox; it doesn't aggregate them across sandboxes or treat an earlier approval as precedent. Anthropic's risk report notes that the human-review stage itself has not been extensively evaluated. Nothing published about the Meta test describes how contractors' judgments were classified or turned into reusable rules. NIST's AI 800-4 treats the integration of automated and human-validated monitoring as an area where consensus has not formed, which is a standards body's way of saying the mechanism hasn't been built.
Public administration named this pattern fifty years ago. Michael Lipsky's Street-Level Bureaucracy showed that caseworkers, police officers, and teachers resolve ambiguous situations in ways that depart from the written rule, and that those resolutions, repeated until they harden, become the policy citizens actually encounter. Recent work calls the result policy-as-produced: informal routines generated by coping with hard cases, running alongside the official rules and sometimes against them.
Exception queues produce the same thing under new conditions. A reviewer meeting the same ambiguous pattern for the fifth time this month has an interpretation by now, and it is probably a good one. If nobody records it, compares it against how the next shift handled the same thing, or promotes it into documented policy, it still governs the next case. It governs without anyone having decided that it should.
In issue #44 I argued that automation strips out the predictable cases and leaves a queue concentrated in ambiguity. This is what that concentration does to an organization. When every case that reaches a reviewer requires interpretation, and interpretation accumulates without codification, the queue becomes a second governance system, one the organization didn't design and doesn't oversee, operating on the cases where agent behavior matters most.
When every case that reaches a reviewer requires interpretation, and interpretation accumulates without codification, the queue becomes a second governance system.
The gap between the written rule and the working practice opens in every institution large enough to have both, and it widens when nobody is paid to measure the distance. Treating accumulated judgment as something the organization owns, rather than something that lives in a reviewer's head until they move teams, is one of the specific choices that hold a system together. Skip it and you don't avoid having a policy for the hard cases. You lose track of the one you're already producing.
- NIST on monitoring gaps: NIST AI 800-4 separately identifies scaling human monitoring and integrating it with automated monitoring as field-wide barriers without consensus solutions, which bears directly on whether reviewer judgments can be captured at production scale.
- Production agents are short: A peer-reviewed ICML 2026 study found that 68% of deployed agent systems execute ten or fewer steps before human intervention, suggesting the scaling question may be less about long autonomous runs than about keeping thousands of brief handoffs consistent.
- Stale permissions under load: OpenAI's synthetic evaluation of its persistent agent showed moderate scope-violation flags rising from 8.6% to 19.7% as intervening tasks increased, an early signal that context distance may erode the authority reviewers originally granted.
- FTC opens an inquiry: The Associated Press reported that the FTC has opened an investigation into several AI developers over possible consumer risks, though scope and targets remain undisclosed — worth watching for whether exception-handling practices draw regulatory attention.