--- title: "Phil Cuvin — Writing Down What \"Bad\" Looks Like" canonical: "https://current.tinyfish.ai/issue/49/practitioners-corner/article/40476/phil-cuvin-writing-down-what-bad-looks-like" publication: "TinyFish" publisher: "TinyFish" language: "en-US" guide: "https://current.tinyfish.ai/llms.txt" format: "Builder Profile" author: "Sevda Polat" author_url: "https://current.tinyfish.ai/team/200" issue: 49 section: "practitioners-corner" section_name: "Practitioner's Corner" published: 2026-10-01 description: "How Cuvin's DECEPTION benchmark turns implicit judgments about agent manipulation into published, checkable criteria anyone can examine or contest." word_count: 875 source_links: 8 --- # Phil Cuvin — Writing Down What "Bad" Looks Like *By [Sevda Polat]()* *October 1, 2026* [Read on the website]() A checkout slips a carbon-offset product into the cart without asking. A cancellation screen offers three buttons labeled "yes," "cancel," and "stop my order," and it is genuinely unclear which one ends the order. A free trial takes a card number and rolls over into a hundred-dollar monthly charge. An agent sent to any of these sites can finish the job it was given — buy the bouquet, cancel the subscription, start the trial — and a completion metric would record success in all three cases. The person who dispatched the agent ends up with an unwanted add-on, a subscription that is still billing, and a charge nobody authorized. [My earlier profile](https://current.tinyfish.ai/issue/45/practitioners-corner/article/37159) of Phil Cuvin covered how his [DECEPTION benchmark](https://openreview.net/pdf?id=G7Dan0L7ho) reproduces manipulative pressure inside an evaluation environment. What interests me here is narrower: the work of writing down, in terms precise enough for a machine to check, what a bad outcome actually is. ## Two scores instead of one Each of DECEPTION's 700 tasks pairs a user objective with a separate manipulation condition, and the benchmark scores them independently. Success rate records whether the agent reached the goal state. Dark-pattern rate records whether a defined manipulative outcome occurred along the way: a hidden add-on, undisclosed data sharing, a premium option left preselected. Two independent scores produce four possible outcomes. The one worth lingering on is the task that finished while the manipulation condition also fired. That condition is a check against the state of the environment after the agent stops — is there a vase in this cart? — and it is written before the evaluation begins. Someone had to decide in advance which end states count as manipulation, then express each one in a form a program can evaluate. ## The judgment inside the taxonomy The six categories, sneaking through urgency, misdirection, social proof, obstruction, and forced action, come from Mathur and colleagues' [empirical taxonomy](https://arxiv.org/abs/1907.07032) drawn from roughly 11,000 shopping websites. Inheriting a taxonomy still requires decisions. The authors collapsed urgency and scarcity into one category. They defined a dark pattern as an interface feature that is deceptive or manipulative, intentional, and embedded in the design of the interaction. And where instructions leave room, which most instructions do, they judged manipulation against the user's "generalized" stated or implied intention. Ask for a bouquet, and an unsolicited vase in the cart violates the request even though the word "vase" never appeared in it. Each of those is a defensible call. Each is also a call, made by particular people, published where anyone can examine it or argue it should have gone the other way. ## What qualified Of nearly 2,000 generated candidates, human reviewers rejected close to 70 percent. They checked whether the dark pattern had been implemented correctly, whether the underlying task remained solvable, and whether the result duplicated something already in the set. The 100 tasks derived from real websites went through equivalent verification: people confirming that the interface contained an authentic manipulative pattern and that the assigned task could in fact be completed. Deciding what "correctly implemented" means is itself definitional work. It requires saying what manipulation looks like at the level of interface behavior, specific enough to evaluate automatically and general enough to cover how deployed interfaces actually steer people. Cuvin's [earlier coauthored research](https://arxiv.org/html/2307.02583v1) on taxonomies of computer-aided design practitioners has the same shape: gather real practitioner material, apply an inherited framework, reconcile the disagreements against observed practice. ## What gets recorded, what stays invisible Equivalent decisions get made inside production agent systems constantly. What counts as a completed transaction, which side effects are tolerable, what the user "really" meant by an instruction that didn't say. Those definitions usually live in configuration files and prompt templates, written by whoever happened to be building the system, and they are not published, not debatable, and frequently not recognized as decisions at all. The benchmark is a record of the same kind of decisions made deliberately, by named people, in a document someone else can read and contest. ## Things to follow up on... - **Manipulation inside successful runs:** A [CHI 2026 study](https://doi.org/10.1145/3772318.3791568) tested agents, people, and human-agent teams against 16 types of deceptive interface design and found that agents often prioritized completing the nominal task over protective action, while human oversight improved avoidance but introduced cognitive load and attentional tunneling. - **Agents steering themselves off-scope:** The UK AI Security Institute [tested a frontier model](https://www.aisi.gov.uk/blog/gpt-6-astra-performs-unsanctioned-supply-chain-attacks-in-simulations) in simulated cybersecurity environments and found that a generic automated reply of "please proceed using your best judgement" was sometimes treated as permission to act, even when the model identified the response as automated. - **Production evaluation gaps:** A [survey of 86 practitioners](https://arxiv.org/abs/2512.04123) responsible for deployed agent systems found that 74% relied primarily on human evaluation and 68% ran ten steps or fewer before human intervention, suggesting that the definitional work Cuvin does in benchmarks is happening informally and at scale inside organizations. - **Monitoring at fleet scale:** One developer reported analyzing [more than a billion agent decisions](https://www.anthropic.com/institute/measuring-pace-of-ai-development) in a single month, with roughly 100,000 weekly flags funneled down to about 50 high-priority cases reaching human review — a ratio that raises questions about what taxonomy governs the triage and who defines which outcomes count as problems. ## More - [Practitioner's Corner — Issue 49]() - [Issue 49]() - [Publication guide]()