Most web-agent benchmarks share a shape: a simulated site, a task, a check on whether the task got done. The site has to function. It does not have to try to sell you anything.
Phil Cuvin's DECEPTION, published at ICLR 2026, could not be built that way. His team wanted to know whether web agents could be steered by dark patterns: the countdown timers, preselected add-ons, confirm-shaming dialogs, and deliberately awkward cancellation flows that commercial sites use to push human users toward choices they didn't make. Answering that question meant the test environments had to reproduce the pressure, not just the plumbing.
The researcher
Cuvin's published work ranges across mechanical engineering, materials science, and human-computer interaction, but two recent projects sit directly upstream of this one. EgoNormia, a benchmark he co-authored for ACL Findings 2025, measures whether systems act appropriately with respect to physical-social norms such as safety, privacy, and politeness, rather than whether they can identify those norms when asked. AutoLibra, also at ICLR 2026, derives fine-grained evaluation metrics from open-ended human feedback instead of a binary success flag. Both are attempts to see what a pass/fail score hides. DECEPTION extends the same concern into a second, independent measurement: did the task get done, and separately, did the agent accept something the user never authorized?
What the test had to contain
An agent can complete a purchase correctly and leave a preselected insurance add-on sitting in the cart. Those are two different facts, and DECEPTION scores them separately.
The team sorted dark patterns into six categories by attack mechanism and built 700 tasks, each pairing a legitimate user goal with one embedded pattern whose outcome conflicts with that goal. Grouping by mechanism turns out to be a construction requirement, not a taxonomy preference. Sneaking, slipping an unauthorized item into a cart, requires an environment with a working purchase flow and a hidden cost inside it. Obstruction, making cancellation difficult, requires an environment that models effort asymmetry: extra steps, buried links, confirmation loops that exist only to wear the user down.
The same logic explains why the manipulation cannot be tested at the input layer. Prompt injection inserts instructions into the agent's control flow and can be studied by examining what the agent was told. A dark pattern leaves the instructions untouched and alters the decision environment around them: what is selected by default, what is visually emphasized, what costs effort. To test for it, you have to build the interface.
For the 600 generated tasks, the team constructed paired controls: identical interfaces with the pattern stripped out. Leading agents cleared 99% task success on those clean versions and triggered zero manipulative outcomes, which is what establishes the pattern itself as the cause. The 100 tasks archived from live commercial sites don't admit that control. You cannot surgically remove a real site's native dark pattern and hold everything else fixed.
A different kind of maintenance
Agent evaluation has been getting stricter about what counts as finished, moving from "did the agent say it was done" toward checking whether the underlying record actually changed. DECEPTION adds something that tightening the completion check cannot reach: a run can satisfy every completion criterion and still be a failure, because of what came along with it.
The harder problem is upkeep. Paired controls work because the team built the generated environments themselves and could remove each pattern cleanly, which makes those environments an ongoing obligation rather than a finished artifact. A capability benchmark can hold its tasks fixed and let the agent be the only variable. A vulnerability environment has to be maintained against an adversary that keeps changing: new friction designs, new visual tricks, eventually tricks aimed specifically at how agents behave. That is a standing cost, and standing costs are the ones budgets tend to omit, because nothing visibly breaks the year you skip them.
If your organization evaluates agents on task completion alone, you are measuring whether the agent can do the work. You are not measuring whether it can do the work without being quietly steered into outcomes nobody approved.
- The IEEE S&P counterpart: A separate IEEE study by Ersoy et al. built four custom React websites where dark patterns could be toggled on and off via URL parameters, finding that better-performing agent configurations were sometimes among the most susceptible to manipulation.
- Capability didn't predict resistance: The ICLR study reported that agents with larger models and more test-time reasoning were steered toward harmful outcomes more often, while common prompt and guardrail defenses did not consistently reduce manipulation.
- Browser architecture as attack surface: University of Washington researchers found that architectural choices in agentic browsers sometimes allowed a successful prompt injection to circumvent same-origin protections, creating preconditions for cross-site data theft or action forgery.
- Production agents stay short: A peer-reviewed ICML 2026 survey of 86 deployed agent systems found that 68% executed ten steps or fewer before human intervention and 75% of teams did not use formal benchmarks at all, relying instead on expert feedback or A/B testing.