--- title: "Foundations" canonical: "https://current.tinyfish.ai/issue/45/foundations" publication: "TinyFish" publisher: "TinyFish" language: "en-US" guide: "https://current.tinyfish.ai/llms.txt" permalink: "https://current.tinyfish.ai/issue/45/foundations" issue: 45 section: "foundations" description: "Conceptual clarity earned from building at scale" authors: ["Sevda Polat","Sable Whitford"] published: 2026-09-07 updated: 2026-09-03 articles: 26 --- # TinyFish > CURRENT is a journal about AI agents in production, for the engineering and product leaders who have to make them work. Drawing on experience building enterprise web agents, it explains why the demo succeeded and the deployment didn't, and what that means for choosing vendors, briefing the board, and avoiding mistakes others have already made. [Read on the website]() ## Issue 45 — Foundations Conceptual clarity earned from building at scale ### Two Kinds of Influence - [Nobody Tampered With the Agent's Instructions]() — By [Sevda Polat]() · Published 2026-09-03 · Format: Two Kinds of Influence Agent security conversations center on prompt injection: someone hides instructions in the input, the agent gets hijacked. Researchers at CHI 2026 found something running on different machinery. They sent shopping agents through ordinary commercial checkout flows and watched them notice anomalies, duplicate items, pre-checked consent boxes, and proceed anyway. No instructions were tampered with. The commercial web shaped the outcome through its normal operation. Current law has no answer for that case: when an agent completes an authorized action it was environmentally induced to take, who bears the commitment? ### Agent Susceptibility in Practice - [Your Best Agent Is Your Most Gullible One]() — By [Sable Whitford]() · Published 2026-09-03 · Format: Agent Susceptibility in Practice TrickyArena tested six web agents against deceptive interface patterns — fake urgency, pre-checked boxes, double-negative opt-outs — and found a 41% overall susceptibility rate. The distribution underneath that average matters more: the agents best at finishing tasks were also the ones most likely to get manipulated, and the agents that looked resistant mostly just stalled before reaching the trap. Evaluate on task completion alone and you may be selecting for gullibility. ### Choice Architecture Reading - **Why Smarter Agents Are Easier to Manipulate: The DECEPTICON Inverse-Scaling Finding** — [Source]() — Published 2026-09-03 Stanford researchers found that larger language models fall for dark patterns more often, not less. Worth sitting with. - **SusBench Puts Numbers on Which Dark Patterns Fool Agents and Which Don't** — [Source]() — Published 2026-09-03 A benchmark across 313 tasks maps where agents and humans share the same manipulation vulnerabilities, and where they diverge. - **TrickyArena: Measuring How LLM Web Agents Fall for Dark Patterns** — [Source]() — Published 2026-09-03 Six agent systems tested against manipulative interfaces averaged 41% susceptibility. Prompt-based defenses helped, but only partially. - **Human Oversight Doesn't Fix Agent Susceptibility to Dark Patterns** — [Source]() — Published 2026-09-03 CHI 2026 research shows that adding human-in-the-loop review introduces its own failure modes when interfaces are deceptive. - **The AI AGENT Act: Consumer Protection Meets Agentic Delegation** — [Source]() — Published 2026-09-03 Senator Warner's draft bill would require platforms to accept authorized AI agents and impose fiduciary duties on them. - **Agentic Commerce Is Coming, but Human-Era Regulation May Not Translate** — [Source]() — Published 2026-09-03 Consumer protection rules built for human shoppers create real friction when software agents start transacting autonomously. ### Influence Vectors Compared When a web agent acts against its user's interests, two very different things might be happening. Someone may have injected hidden instructions into the agent's input — a security attack in the familiar sense. Or the agent encountered a website designed to steer decisions through interface design: countdown timers, pre-checked boxes, buried fees. Both produce outcomes the user didn't want, and they tend to get grouped under "agent security." But the mechanisms differ, the people behind them want different things, the applicable legal frameworks barely overlap, and the defenses that work for one can actually make the other worse. - **One Smuggles Instructions, the Other Reshapes the Environment** — [Source]() — Published 2026-09-03 Tag: Core Mechanism Prompt injection hides adversarial instructions in an agent's input through invisible text, encoded payloads, or disguised markup. Choice architecture works through the legitimate interface: defaults, urgency cues, pre-checked options, friction. No rogue instruction enters the processing pipeline. The manipulation is the environment itself. - **Attackers Want System Access, Operators Want More Revenue** — [Source]() — Published 2026-09-03 Tag: Actor Intent Injection attackers aim to commandeer the agent: exfiltrate data, trigger unauthorized transactions, break things. Dark pattern operators pursue commercial gain through interface choices that favor the site owner. One group treats the agent as a system to compromise, the other as a customer to influence. - **Smarter Agents Resist Injection but Fall for Manipulation** — [Source]() — Published 2026-09-03 Tag: Inverse Scaling Stanford's DECEPTICON study found dark pattern susceptibility rose from 43.8% to 73.7% as model size increased. Larger models reason more carefully about manipulative cues instead of dismissing them. The capabilities that help against injection — better reasoning, deeper context processing — make choice architecture problems worse. - **Injections Hide While Dark Patterns Sit in View** — [Source]() — Published 2026-09-03 Tag: Threat Visibility Injections rely on invisibility: zero-font text, off-screen elements, encoded payloads. Dark patterns are fully rendered. The countdown timer and the pre-checked add-on are right there on screen. What's concealed is the intent to steer behavior, not the interface element doing the steering. - **Fraud Law Applies Clearly, Consumer Protection Gets Murky** — [Source]() — Published 2026-09-03 Tag: Legal Treatment Prompt injection maps onto existing unauthorized-access and fraud statutes. Dark patterns fall under consumer protection frameworks written for human shoppers. When an agent encounters a manipulative checkout flow, current law doesn't establish whose perception governs or where liability sits in the merchant-agent-user chain. - **Each Threat Requires a Fundamentally Different Defense Strategy** — [Source]() — Published 2026-09-03 Tag: Defense Divergence Injection defenses separate trusted instructions from untrusted data through sandboxing, runtime monitors, and architectural isolation. Dark pattern defenses require something else: ensuring agents evaluate rejected alternatives, hidden fees, and friction against the user's actual goals. Warning prompts alone reduced susceptibility by only 12%. - **One Threat Has Institutional Owners, the Other Doesn't** — [Source]() — Published 2026-09-03 Tag: Governance Gap Prompt injection has a governance home: OWASP rankings, NIST frameworks, security teams with incident playbooks. Choice architecture manipulation falls between security and consumer regulation, with neither group specifically responsible for interfaces that predictably exploit how agents process commercial environments. That organizational vacuum is itself a vulnerability. ### Pattern Susceptibility Map Dark patterns are the deceptive design tricks websites use to steer user behavior: pre-checked consent boxes, fake urgency timers, hidden fees slipped in at checkout. Researchers have catalogued them into taxonomies for years. In 2026, three peer-reviewed studies applied those same taxonomies to web agents, the AI systems that browse and act on websites autonomously. The vulnerability profiles that emerged are strikingly uneven. Agents shrug off manipulation categories that reliably trip up humans, then fall for others at two to five times the human rate. The failures trace to specific architectural properties: how agents follow instructions, how they handle social context, whether avoiding a pattern requires an extra step beyond the primary task. - **Obstruction Works Because Agents Follow Every Instruction** — [Source]() — Published 2026-09-03 Tag: Obstruction Patterns Obstruction adds disruptive steps that look like legitimate parts of a workflow. In DECEPTICON's tests, these patterns hit agents at 85-100% effectiveness, compared to 44% for humans. A pop-up that a person dismisses as noise reads to an agent like an authoritative directive worth obeying. - **Fabricated Popularity Signals Hit Agents Disproportionately Hard** — [Source]() — Published 2026-09-03 Tag: Social Proof Claims like "300 people bought this" reached 60-94% effectiveness against agents. Against humans, the same claims landed at about 18%. People bring contextual skepticism to social pressure cues. Agents treat manufactured popularity as task-relevant data and weight it accordingly. - **Agents Dodge Visual Manipulation, Mostly by Accident** — [Source]() — Published 2026-09-03 Tag: Visual Tricks Across six agents tested by Tang et al., manipulative visual layouts, emotional pressure, confirmshaming, and scarcity claims were avoided at 100%. But the avoidance was incidental: agents tended to pick the first or cheapest option, bypassing the manipulation without actually recognizing it. - **Pre-Checked Boxes Defeat Agents and Humans Alike** — [Source]() — Published 2026-09-03 Tag: Default Settings Pre-checked boxes beat every adapted agent tested. Humans avoided them only 33% of the time. The failure mechanisms differ: agents recognized the checkbox but skipped the extra step to uncheck it. Humans typically didn't notice the box was pre-selected at all. - **Hidden Costs Catch Agents and Humans Similarly** — [Source]() — Published 2026-09-03 Tag: Sneaking Patterns Sneaking patterns, like hidden fees and items quietly added to a cart, show comparable effectiveness against both groups. DECEPTICON measured agents at 50-86% susceptibility and humans at 54.5%. These tactics exploit blind spots during task completion that neither population reliably catches. - **Bigger Models Get Fooled More, Not Less** — [Source]() — Published 2026-09-03 Tag: Scale Effects DECEPTICON's scaling experiments found susceptibility rising with model capability. Within the Qwen model family, vulnerability went from 44% at 3 billion parameters to 74% at 72 billion. Larger models incorporated more manipulative content into their reasoning chains, giving dark patterns more surface area to work with. - **Agents See the Trap and Walk In** — [Source]() — Published 2026-09-03 Tag: Recognition Gap In Tang et al.'s tests, both Claude 3.7 and DeepSeek V3 noted a pre-checked data-sharing checkbox in their reasoning traces, then decided not to uncheck it. When avoidance requires an extra step beyond the main task, awareness alone doesn't produce corrective action. ### Past Articles - [What MCP Won't Undo for You]() — By [Sable Whitford]() · Published 2026-08-28 · Format: Protocol Walkthrough The July 2026 MCP revision removed protocol sessions and put server-minted state handles in their place: ordinary strings your model threads from one tool call to the next. The operational case for the change was solid. But sessions had made it easy to miss something, which is that MCP has never had anything to say about reversing an action once it lands. Here's a booking sequence traced through the revised spec — what the protocol standardizes, what it deliberately leaves unspecified, and which decisions were yours all along. - [The Undo Fantasy]() — By [Sable Whitford]() · Published 2026-08-28 · Format: The Reversal Frame Art conservators spent decades believing "reversible" meant safe. Then they tested it: dissolving an adhesive out of porous ceramic drove the solution deeper into the object, where it stayed permanently. Removal turned out to be a second treatment, not an opposite. Payment regulators landed in the same place from another direction — Reg E has no "undo" step, because losing money and losing access to money are separate injuries on separate clocks. When we ask whether a user can undo what an agent did, we're compressing at least four different remedies into one verb. - [Memory as a Write Path]() — By [Sevda Polat]() · Published 2026-08-20 · Format: Threat Model Shift An agent retrieves something it stored about you and folds it into a response. What comes back is one continuous stretch of text. The user sees an answer; what the user cannot see is which parts were worked out just now and which were shaped by a conversation from three months ago, a connected document, or another person who once had write access to the same store. Memory extends trust backward in time. Session-scoped security testing does not reach that, and the transparency controls being added now address a different layer. - [How Persistent Memory Moves Through an Agent System]() — By [Sevda Polat]() · Published 2026-08-20 · Format: Practical Mechanism Persistent memory in an agent system — an observation from one session shaping behavior in a later one — runs on ordinary application code: content gets written to storage, retrieved by similarity, and assembled into the model's planning context. Research presented at ACL 2026 makes each of those stages unusually legible, because the researchers exploited all three by feeding adversarial content through them exactly as specified, and nothing in the pipeline was built to notice. ## Other sections in Issue 45 - [Daily Brief]() - [Market Pulse]() - [Practitioner's Corner]() - [Echoes]() - [Vision]() ## More - [Publication guide]() - [Issue archive]()