Practitioner's Corner

Practitioner's Corner

Melissa Pan's Three Evidence Problems

A database recovery log carries its own interpretation. A multi-agent failure trace does not: in one recent study, six experts had to invent the failure categories before 150 traces could be classified at all. And production agent deployments generate evidence that researchers can reach only by interviewing the people who work inside.
Melissa Pan has worked across all three. Read in sequence, her research shows the evidence changing form faster than anyone's instrumentation does.
Melissa Pan's Three Evidence Problems
A database recovery log carries its own interpretation. A multi-agent failure trace does not: in one recent study, six experts had to invent the failure categories before 150 traces could be classified at all. And production agent deployments generate evidence that researchers can reach only by interviewing the people who work inside.
Melissa Pan has worked across all three. Read in sequence, her research shows the evidence changing form faster than anyone's instrumentation does.
Who Pays for the Record

When an AI agent canceled a gym booking in Australia earlier this year, the only record of what happened was screenshots taken by the person who deployed the agent. The person whose reservation vanished had nothing. Records cost money to produce — instrumentation, storage, retrieval, access controls — and in agent systems that cost lands by default on whoever eventually needs one, which is rarely the party equipped to pay it. Regulated industries settled this question decades ago. Agent systems haven't, and the economics of the gap are entirely predictable.
Who Pays for the Record
When an AI agent canceled a gym booking in Australia earlier this year, the only record of what happened was screenshots taken by the person who deployed the agent. The person whose reservation vanished had nothing. Records cost money to produce — instrumentation, storage, retrieval, access controls — and in agent systems that cost lands by default on whoever eventually needs one, which is rarely the party equipped to pay it. Regulated industries settled this question decades ago. Agent systems haven't, and the economics of the gap are entirely predictable.


The Shape of the Shadow — An AML Examiner on Investigating What the System Chose Not to Show
CONTINUE READINGIncident Visibility

The SAFE proposal is a new incident-reporting framework for AI agent systems, backed by over 120 organizations. It specifies nine categories of evidence that must be preserved after a reportable event: prompts, execution traces, tool calls, agent identities, permissions, human approval events, and more. This looks like an evidence policy. But specifying what a legible incident contains is also specifying what, if absent, makes an event institutionally invisible. The evidence list doesn't just describe incidents. It determines which ones exist.
Aviation learned something about this. The ASRS reporting system doesn't just ask pilots to report safety events. It routes reports to NASA rather than the FAA, strips identifying information before anyone else sees it, returns proof of filing to the reporter, and waives penalties for inadvertent violations disclosed within ten days. Each of those design choices addresses a specific reason a rational person would choose silence over disclosure. The result: over a million reports filed across decades, with no reporter's identity ever compromised. The incentive architecture is what produces the information.
SAFE has defined what the sensor should detect. The question worth asking is whether anyone, absent enforcement protection or structural anonymity, has a reason to activate it.
Reading List




Past Articles

I carried a pager for years attached to monitoring that rolled everything into averages at write time. Any question I ha...

Most agents in production stop somewhere around ten steps and hand the work to a person. Easy to read that as the models...

Capacity planning rests on a claim nobody states out loud: that a queue is a shelf, and work comes off it weighing what ...

Durable execution promises something real. Your workflow survives crashes, evacuated hosts, a datacenter having a genuin...
