Phil Cuvin — Turning Anecdotes into Instruments

Every team I've worked on had at least one problem that lived entirely in shorthand. "Oh, the thing where it does that." It shows up in retros, it shows up in Slack, everybody nods, and it survives for years because nobody put a counter on it. Instrument it and it becomes a number, and numbers get owners.

Phil Cuvin's coauthored research — four projects over three years — is that move performed repeatedly on problems the field was only complaining about.

The earliest public example is a 2023 CSCW paper on branching in computer-aided design. Cuvin and collaborators coded 719 threads from five hardware-design forums into eleven use cases for why CAD practitioners branch and merge, and where their tools fail them. Eleven categories assembled out of complaints people had been airing in forum threads for years without anyone aggregating them into something you could count or act on.

The same move shows up in EgoNormia (Findings of ACL 2025), which turns first-person video of ordinary human activity into a benchmark for physical-social norms: can a model judge what someone should do when safety, privacy, politeness, and getting the job done pull in different directions? And in AutoLibra (ICLR 2026), which takes the messy comments humans leave on agent runs, things as specific as "the agent did not choose iPhone 14/15," and clusters them into named, defined metrics you can score the next agent against.

Then DECEPTION. Sevda Polat's prior profile covered how the benchmark reproduces manipulative pressure inside its own test environment. I want to focus on one design decision that looks almost administrative.

DECEPTION keeps two scores apart: did the agent finish the job it was given, and did the dark pattern land. (Dark patterns are the manipulative furniture of the commercial web — preselected add-ons, confirm-shaming dialogs, subscriptions that switch on quietly.) Each task names a specific unwanted outcome, the premium option still sitting in the cart or the subscription activated, and the benchmark checks the final page state for it. An agent can finish the task and still get played. It can flunk the task and dodge the trap. Four outcomes, all of them populated.

Until that split existed, agent susceptibility to deceptive interfaces was something people mentioned between talks. The benchmark gave it structure: 700 tasks, six pattern categories, paired clean controls, reproducible numbers. Across the Qwen 2.5 VL family, dark-pattern effectiveness climbed from 43.8% at 3 billion parameters to 73.7% at 72 billion. In Gemini 2.5 Flash, raising the thinking budget, meaning the amount of internal reasoning the model is allowed before it answers, from 256 tokens to 16,384 moved susceptibility from 38.5% to 71.2%.

The bigger models, and the ones given more room to think, were the easier ones to manipulate.

A benchmark with one score can only answer the question somebody thought to ask in advance. Track completion and you get a capability curve trending up, and you conclude things are improving. The clicking-through-manipulative-dialogs part was happening the whole time, and nothing was counting it.

Which leaves the question of what else currently sits where dark-pattern susceptibility sat a year ago: mentioned in incident writeups, visible in demos, named in shorthand by the people who watch these systems run, and still not structured into anything you can compare across two vendors on a Tuesday.