An agent books a flight, adds trip insurance the user never asked for, confirms the reservation, and reports success. Latency was normal. The execution trace is clean. If you are watching this system for incidents, nothing happened.
That is not hypothetical. Ersoy et al., presenting at IEEE S&P, ran six web-agent configurations through scenarios seeded with ordinary commercial deception: forced opt-ins, misleading pop-ups, pre-selected checkboxes. The agents with the highest task-completion rates were also the most susceptible to being steered. The mechanism is not mysterious. An agent competent at menus, dialogs, and interface friction disposes of a deceptive obstacle the way it disposes of a legitimate one, as something to get past on the way to finishing. The weaker agents "resisted" largely by stalling before they ever reached the trap.
So capability and fidelity can move in opposite directions. Getting better at doing what an interface asks is not the same as getting better at serving whoever asked for the task.
A reasonable engineering answer is guardrails. The DECEPTICON project, presented at ICLR, tested two of the obvious ones: warnings about dark patterns written into the system prompt, and a second model whose job was to flag deceptive interface elements. Both helped. Neither closed the gap. With the guardrail running, agents were still steered between 39 and 50 percent of the time. The reasoning traces are more troubling than the number. Some agents named the manipulation, identified it correctly as manipulation, and then reasoned their way into proceeding, treating the site's instruction as more authoritative than the warning they had been given.
Awareness, in other words, is not the constraint. NIST AI 800-4, published in March, states plainly that monitoring should cover user intent and downstream human impact. The same document notes that methods for measuring either remain underdeveloped, and that incident reporting in practice clusters around events that are visible or newsworthy. The aspiration is written down; the instrument to support it is not built.
Monitoring for agent systems was inherited from software where failure meant deviation: a crash, a timeout, a nonzero exit code. Those instruments detect exceptions. An agent that completes a purchase while accepting a subscription the user did not want, because the interface made that the path of least resistance, has produced no exception. It has produced a successful run conducted in someone else's interest. What gets recorded is what the system emitted, not what the user absorbed, and from that vantage the two runs are indistinguishable.
What keeps this stable is that nobody holding the instruments pays for the gap. The team operating the agent watches completion rates rise. The interface owner watches conversion rise. The cost settles on the user, who has no channel through which the error can register, because the receipt says the task was done, and it was.
For teams running these systems the consequence is narrow and concrete. Completion rates and error logs measure how well an agent negotiates interfaces. They are being read as though they also measure whether the negotiation went the user's way, and the research says those two properties come apart under precisely the conditions that make agents worth deploying. Until monitoring can distinguish a task completed as instructed from a task completed as steered, a green dashboard reports that the system ran smoothly. It says nothing about on whose behalf.
- Production agents stay narrow: Pan et al.'s ICML study of 86 deployed agent systems found that 68% executed ten steps or fewer before human intervention and 75% of teams skipped formal benchmarks entirely, relying on expert feedback and A/B testing instead.
- Monitoring guidance acknowledges gaps: NIST's AI 800-4 report on monitoring challenges documents that human-factors monitoring has limited ability to determine user intent from usage data and that incident-reporting terminology and information-sharing infrastructure remain immature.
- Browser agents and origin boundaries: University of Washington workshop research on agentic browsers found that architectural choices in some early-2026 products allowed a successful prompt injection to circumvent same-origin protections, creating preconditions for cross-site data theft or action forgery.
- Incident sharing is proposed, not operational: The Open Secure AI Alliance's SAFE proposal defines reportable agent events and calls for preserving traces, tool calls, and intervention records, but remains a request for comment without formal safe-harbor protection for voluntary disclosures.