Worry about agents doing things their principals didn't intend almost always means prompt injection: someone hides instructions in the agent's input and overrides its mandate. The threat is adversarial, the defenses are technical, and there has been real progress on both.
A study published at CHI 2026 surfaced something that works through different machinery. Researchers rebuilt ordinary commercial checkout flows and sent shopping agents through them. During one task, a duplicate item showed up in the cart. Claude 3.7 noticed. Its reasoning trace records the anomaly, and it bought both items anyway, because removing the extra wasn't part of the assigned procedure and the agent read it as outside its scope. In a second task, a pre-checked box authorized information sharing. Two agents saw the box, registered that it was checked, and left it that way, treating the default as someone else's decision.
Nobody tampered with those instructions, and nothing was concealed. The environment simply made the unwanted outcome the cheapest way to finish the job.
This is choice architecture: defaults, pre-selections, items quietly added to carts. It is the ordinary condition of the commercial web. A 2024 international review found at least one such pattern on nearly 76% of 642 subscription sites, and some of these designs have drawn enforcement. They were built to work on people, and people partly resist: human shoppers caught and corrected the duplicate item 70% of the time. The agents recognized the same anomalies at comparable rates. They just didn't act on them. Completing the assigned procedure outranked correcting an outcome that fell outside the task definition.
That asymmetry is the interesting part. The procedural fidelity that makes an agent useful, its willingness to stay inside the scope it was given and finish, is the same quality that makes it responsive to whatever the page has already decided. Prompt injection subverts the agent's instructions. Choice architecture works with them. The agent stays inside its permissions the whole way through, which is precisely why the problem is a governance one rather than a security one.
Consumer-protection law reaches interface design that exploits human cognition. Electronic-transaction law says contracts formed through automated agents bind the principals behind them. Neither anticipated this: no human cognition was exploited, because no human was present, and no instruction was violated, because the agent did what it was told. Recent legal scholarship names the gap. No adopted regime has closed it.
When an agent buys the wrong pair of shoes inside a valid spending mandate, the payment network classifies the transaction as consumer-initiated. The commitment attaches before anyone can raise a question about how it was produced, and it will go on attaching, across a web built to steer purchasing decisions, to a shopper that reads every default as a decision someone else already made.
- Susceptibility varies by architecture: TrickyArena tested six web-agent systems against dark patterns in controlled environments and found average susceptibility of 41%, with results varying by agent architecture and presentation modality.
- EU model rules propose remedies: The European Law Institute's 2025 model rules would let consumers set aside contracts when a business structures its interface in ways that materially distort or impair a consumer's digital assistant.
- UK guidance reaches AI decisions: The CMA's March 2026 guidance states that consumer law applies whether decisions are made by people or AI, though its examples principally concern business-deployed agents acting on consumers rather than consumer-deployed agents encountering seller-designed interfaces.
- Commerce standards are separating layers: The upcoming W3C/GS1 workshop agenda distinguishes product identity, capability discovery, delegated authority, and payment credentials as separate problems often collapsed under "agentic commerce".