A recent study of enterprise agents allocating costs across accounts and applications found that 92.6% of runs producing correct final numbers contained process-level deviations: data staged with the wrong parameters, verification steps skipped, infrastructure queries scoped too narrowly. The answers were right. The paths that produced them often wouldn't survive a change in inputs.
Human-operated processes rarely offered this kind of visibility. When a clerk processed a transaction correctly, her manager had no practical way to audit the intermediate steps: which records she pulled, in what order, whether she double-checked a figure or skipped the check because she'd seen the same pattern a hundred times. Agent infrastructure logs every tool call, every argument, every intermediate result, and can compare the path taken against the path specified.
That visibility is a real gain, but its costs take a while to surface.
Interpretation is the first. Many of the deviations flagged in that study turned out to be unresolved questions about what the specification required in the first place — whether a particular database check had to happen at a particular point, how to handle a cost that couldn't be split cleanly across applications. The evaluator reported them as failures. They were decisions nobody had made yet. Automated monitoring surfaces ambiguity at scale, but it has no way to settle it.
Maintenance is the second. A production evaluation system for an educational AI product ran 21 automated quality judges across millions of student and teacher messages. When the vendor upgraded one judge model, its measured pass rate for factual accuracy fell from 97% to 11% overnight. The product hadn't degraded; the instrument had changed. A second judge re-examined the 140,000 failures that had accumulated and couldn't corroborate 94% of them.
The team rebuilt around panels of judges, narrower criteria, and a separate channel for ambiguous cases they called a minority report. Their primary alert stream shrank by roughly 99%. The evaluator had become a production system in its own right, with version dependencies, failure modes, and a standing need for calibration and staff.
Anyone who has maintained production software recognizes this pattern: the monitoring layer accretes complexity until it requires its own monitoring.
The enterprise study's authors built dependency modeling to tell reviewers where to start. A single failed run triggered more than six flagged checks on average, which is more threads than one person can hold at once. The modeling compressed that to two or three diagnostic starting points per run, which is genuine progress. But two or three per run, at any serious volume of agent activity, is still a continuous stream of diagnostic work that needs dedicated people and, harder to staff for, the judgment to tell a confirmed defect from a specification gap. I've argued previously that once a structured evaluation carries operational weight, behavior reorganizes around the score. The same risk applies here. The enterprise evaluator surfaced dozens of cases where the specification itself was ambiguous. An organization that optimizes for clearing those flags rather than resolving the underlying policy questions accumulates undecided decisions behind a clean conformance number, and undecided policy is where the next real failure tends to originate.
Constructed visibility is bounded by its own design. Organizations are most exposed in the stretch between losing the old ambient signals and learning to read the new ones, and poor calibration is the expected part of that. The subtler problem is that the new instruments carry an implicit claim to completeness the old signals never made. A queue of frustrated customers was obviously a sample, and everyone treated it as one. A dashboard of execution traces looks like the whole picture. Reading it well means interrogating it the way a good manager once interrogated an anecdote from the floor: asking what it shows, and what it was never designed to catch.
