Designing Review That Resists Its Own Collapse

An FDA-authorized pathology tool called Paige Prostate does something unusual with its workflow. The pathologist classifies a slide — cancer, no cancer, or defer — and records that classification before the system shows its own finding. The AI's recommendation appears only after the human has committed to a position.

Order of operations is doing real work there. A pathologist who has already written "benign" meets the machine's disagreement as a challenge to a position she has staked and will have to defend. Seeing the machine's answer first and being invited to object is a different task, and a much easier one to decline.

What the reviewer sees, and when

The sequencing only holds if the workflow keeps a durable record of the unassisted assessment. Absent that record, the delay is a courtesy, and courtesies are the first thing operational pressure removes, because nobody in the workflow has an incentive to preserve them. In the Paige authorization, the unassisted classification had to be entered in an electronic case-report form before the assisted read. That requirement is what makes the design binding rather than aspirational.

A prior piece in this series noted that detailed agent artifacts can suppress scrutiny on their own: an output that looks complete gives the reviewer a reason to stop looking. So evidence presentation is partly a question of volume. Give the reviewer enough context to reach a judgment, and not so much that the agent's reasoning becomes the default the reviewer would have to work to escape.

When to trigger review

Review invoked on every agent action is review that will stop working. A study of 112 primary-care clinicians found that each additional clinical alert within a single encounter was associated with 30% lower odds of acceptance. The alerts weren't wrong. They were frequent, and frequency taught the clinicians to clear them. That isn't negligence; it is what a reasonable person does when attention is finite and the system treats every item as urgent.

So trigger logic should be tuned to consequence. The variable that matters is how far a consequence can travel before someone can still interrupt it — what I've elsewhere called commit distance. A coding agent that runs hundreds of steps on a branch, where nothing reaches production without an approved merge, has short commit distance no matter how long it works. A purchasing agent that finishes in four steps can cross into money that has already moved. Concentrating review where reversal is hardest means accepting that some agent actions will go unreviewed. That is uncomfortable to write into a policy document. It is still more honest than pretending that a reviewer working through their fortieth approval of the hour is exercising judgment.

Measuring whether review is real

Without measurement, what you have is a step that adds latency and produces approval records. Whether it catches errors is unknown.

The practice that closes the gap is seeded quality sampling: routing items with known correct answers through the ordinary queue, indistinguishable from real work, and tracking whether reviewers handle them correctly. Three details determine whether it tells you anything. Seeded items have to look like production work, because reviewers who can spot the test are demonstrating compliance rather than judgment. Results have to be tracked per reviewer over time, since an aggregate catch rate will hide one person's drift inside a team's average. And the sampling has to run continuously, because a one-off calibration exercise measures how people performed on a day they knew they were being watched.

Evidence presentation and trigger design are decisions you make once and revisit occasionally. Seeded disagreements are the running signal that tells you whether those decisions still hold, or whether the review step has quietly become a timestamp.