Anthropic disclosed in September that its offline agent monitoring flags roughly 100,000 transcripts per week. Automated classifiers screen those flags, resolving most. About 50 cases reach a human.
The classifiers sort against predefined categories — misalignment signals, slowly developing behavioral patterns. Cases that fit a category can become exceptions a person evaluates. Cases outside the taxonomy never enter the queue.
The taxonomy, then, is doing most of the governing. A reviewer exercises judgment only on what the classifier surfaces, and the classifier surfaces only what its designers anticipated. Anthropic acknowledges the constraint: "hard to be certain we have a complete picture of all possible kinds of behaviors." The 2,000-to-1 ratio from flags to human reviews reflects what the organization already knew to look for. Classification systems encode prior assumptions about risk, and prior assumptions cannot catch the failure mode nobody has categorized yet.
The funnel (Anthropic, Sept 2026)
- ~30,000 concurrent agents, one internal platform
- 100% of actions screened by online monitor before execution
- ~0.002% blocked — about 1 in 47,000 decisions
- Offline monitor flags ~100,000 transcripts/week
- Classifiers resolve most flags automatically
- ~50 cases/week reach a human
Block rate vs. detection rate: The 0.002% measures interventions the monitor made, not what it failed to detect. Anthropic notes this distinction.
The content-moderation parallel: Classifiers categorize against a predefined taxonomy. The taxonomy is a policy decision made before the classifier runs. Model quality can't compensate for missing categories.
NIST AI 800-4 identifies human-factors monitoring as the biggest gap in post-deployment AI oversight.