Your Agent's Questions Need a Recipient and a Timer

When an AI agent stops partway through a task to ask a human something, people tend to read it as a failure of autonomy. I read it as the agent doing part of its job, and what I care about is whether the ask is built well enough to justify interrupting somebody.

In a recent paper, Ask or Assume?, Nicholas Edwards and Sebastian Schuster took 500 real bug reports from a standard coding benchmark and gutted them, pulling out details like error messages. Two models, Claude Sonnet 4.5 and Kimi K2.6, worked from the stripped reports under three conditions. In one, asking wasn't allowed. In another, the agent was told up front that the report was incomplete and that it should ask. In the last, nobody told it anything, and a second agent watched the work and forced a question whenever something looked missing. With no asking, both models solved about 55% of tasks, against 71% to 73% on the original reports. With the watcher, both landed at exactly 69.4%.

Builders should notice two things. The first is timing. Told up front, Claude asked 97.6% of its questions in the opening stretch of the run; with the watcher, 43% came mid-task. The study doesn't show the later questions were any better (Claude scored about the same either way). But a system that only lets the agent ask at kickoff never learns what it would have asked once it was in the work.

The second is plumbing. Kimi did worse when told up front (47.2%), and much of that came from a failure you'd only catch by watching it run: it sometimes typed its question into the tool that marks a task complete. It asked, and the job closed. Asking needs its own command, separate from "done."

The "user" answering those questions, though, was another model, GPT-5.1, holding the complete original report. Under the watcher, Kimi sent it 3,791 requests across 435 tasks, almost nine per task, and the simulator answered at most three in any task before telling it to press on with what it had. No human wants nine pings per bug either. The study shows that asking recovers most of the lost work when someone has the answer and hands it over fast. It says nothing about what happens when that someone is in a meeting.

Real deployments usually do have a human near the work, though not necessarily the one with the answer. In a survey of teams running agents in production, 41 of 60 said their systems go at most ten steps before a person steps in, planned sign-offs included.

Operations people call a designed interruption a page. Anyone who has carried a pager knows the bad kind: it wakes the whole on-call rotation at 3 AM with "something's wrong" and nothing else, and that's how people burn out. A good page goes to someone who can act on it, says where it hurts and what's been tried, and spells out what happens if nobody responds.

An agent's question needs the same parts:

• A recipient who could know the answer. The person who filed the issue, not whoever happens to be on duty.
• Its location. The step it's on, what it tried, and the one fact it's missing.
• A plan while it waits. Work that doesn't depend on the answer, plus a time limit after which it stops or takes the safest default.
• A path for "I don't know." The simulated user had that reply scripted. Real people say it constantly.

Toyota's andon cord is escalation with the timer built in. In Toyota's account of its Burnaston plant, pulling the cord lights up that workstation's number and brings a team leader to the spot. The line keeps moving as long as the fix fits inside the time the car spends at that station, and it stops only if the fix runs over.

The analogy breaks on distance. Toyota's team leader is seconds away; your issue author may be asleep in another time zone. That's why the timer and the fallback have to be settled before the agent ever asks.