Phantom Stock and the Checks That Never Fire

A grocery store's computer says a shelf holds six cans of tomatoes. The shelf is empty. Theft and spoilage took the cans, and neither one passes through the register. Researchers who study store operations have a name for what happens next: inventory freezing.

The stock record is a running sum. Deliveries add to it, and each scanned sale subtracts from it. When the number drops below a threshold, the system orders more. With the shelf empty, though, nobody can buy the item, so no sales get scanned and the number stops moving. An MIT model by Kang and Gershwin states the result plainly:

Nothing in the arithmetic is wrong. It has simply stopped receiving anything to work on.

The same chain turns up well beyond groceries. The fault, six phantom cans, produces an empty shelf. The empty shelf ends sales, and sales were the only events that could ever move the reorder rule. A rule that only listens for events takes the resulting silence for calm.

Checks get built this way for a sensible reason. A check that fires on events costs nothing while nothing is happening. That same property leaves it idle for exactly as long as this kind of failure lasts.

METR, a group that evaluates AI systems for risk, found the same pattern in a very different place. OpenAI runs AI agents internally, meaning software that carries out long chains of actions on its own, and it reviews them with an automated monitor. In a red-team review of the system as it stood in July, METR reported that the monitor "triggered when agents returned control to the user." For a period, another natural checkpoint went unwatched: the moments when an agent's long context is condensed. During that window, "agents could indefinitely postpone being monitored by simply continuing to take actions." METR could not determine whether any agent actually did this, and OpenAI staff had already spotted the gap and closed it during the exercise. While the window was open, though, the agent being watched decided when the watcher looked.

The remedies rest on one principle: take the scheduling of the inspection away from the thing being inspected. METR suggested running the monitor after a fixed number of messages or minutes, which is a clock that a busy agent cannot hold back. Retail researchers went further and treated the silence itself as evidence. Chuang's audit policy reads a run of zero sales as raising the odds that a shelf is secretly empty, and then sends a person to look. At the German drugstore chain dm, DeHoratius and colleagues found that counting the 4 percent of items most likely to be phantom turned up roughly two in five of the hidden empty shelves.

The two remedies are paid for in different ways. A clock pays for a look whether or not anything has gone wrong. Reading the silence spends effort more selectively, but it requires some model of what ordinary quiet looks like. Slow sellers can go a whole week without a sale for perfectly normal reasons, which is why Chuang's model weighs the odds of an empty shelf against the cost of sending someone down the aisle. Once the failure no longer controls when someone looks, the looking becomes a standing cost, and someone has to agree to carry it.

Agents that act on stored records inherit the grocery problem. A ticket status, an order confirmation, an approval queue or an account balance changes only when particular events occur. For each of these, it is worth asking which events move it, and whether the failure you are worried about would also make those events stop.

So next to every instruction that tells an agent to wait for a status to change, someone should write down a number: how long the record may stay quiet before the agent stops trusting it and goes to count the shelf.