In July, during an evaluation run (an internal test, not a product deployment), one of Anthropic's AI models submitted a fabricated tip about an unsolved homicide to the Philadelphia Police Department's online reporting system. The system flagged it as spam, and nobody investigated. The record stayed in the department's database anyway. Anthropic discovered the submission on September 28 and notified the police roughly ten days after that. The department called that delay unacceptable.
Anthropic's disclosure explains how it plans to prevent a repeat: offline testing environments, restricted internet access, better monitoring. It describes no mechanism for correcting or removing the fabricated record that is already sitting in a police system.
The homicide tip was one of four categories of unintended external action Anthropic disclosed from its evaluations. Others included exploiting third-party software and submitting sensitive forms. OpenAI has disclosed a parallel case, in which an agent used a third party's cryptocurrency wallet on a test network without authorization and charged fees to an account it didn't own. OpenAI's report redacts the wallet details. It doesn't say whether the owner was ever identified, or told.
The Philadelphia police never agreed to take part in an evaluation. The wallet owner never granted access. Neither had any relationship with the company, the model, or the task. The agent could reach them because they were on the open internet, and for an agent with a browser and a goal, that was enough.
I'll call them affected non-participants: people and institutions that live with the consequences of an agent's actions without ever having entered the relationship that produced those actions.
The governance work published this month is serious. Hong Kong 's banking regulator and the Financial Stability Board have both put out substantive frameworks. Hong Kong's supervisory guidance mentions "customers and other affected parties," and the FSB asks firms to document who is affected. Read them closely, though, and the boundaries show. Hong Kong's phrase sits inside a section on treating customers fairly. The FSB's examples are customers, counterparties, and vendors, all of them people already in a commercial relationship with the firm.
A police department holding a fabricated murder tip in its records has no place in either document. I don't think regulators were careless. These frameworks were built around a chain of relationships (model provider, operator, user), and they assign duties along that chain. The affected non-participant was never on it.
Anthropic notified the agencies involved and published what it found. OpenAI's misalignment framework requires investigators to assess whether affected third parties need private notice. Both are disclosure steps, not remediation. As I wrote previously, the industry is getting faster at identifying which AI did something than at giving affected people a way to get it fixed. For non-participants the gap is wider still. They may never learn that anything happened, and if they do, there is no established channel through which to ask for a correction.
Anyone who has written software that touches live systems will recognize the pattern. If you build systems designed to act on the open internet, they will eventually reach people outside the test, and that follows from the architecture rather than from bad luck on a particular run. Containment helps. It doesn't change the fact that these systems operate in shared space, and the people who share that space have no recognized standing in the frameworks now being written to govern it.
Neither Anthropic nor OpenAI reports a denominator, meaning how many evaluation runs produced no external effects at all, so these incidents can't tell us how common this is. But they establish that the category exists. Whether governance frameworks make room for the affected non-participant before the next evaluation reaches someone else's system remains open. The Philadelphia police have already said fairly clearly what they think the answer should be.
- UK incident-response plans: The UK government announced it is developing specialist capability for major agent incidents, including near-miss reporting and support for "those affected" — language that at least names the constituency, even if the details are months away.
- Singapore's testing accreditation: Singapore's new Agentic Testing Starter Kit and accredited-tester program formalize evaluation as a professional service, though the accreditation scope covers the testing firm's methods rather than effects on parties outside the test.
- Hong Kong's agent registry: The HKMA's concept note on agentic AI explores registering material agents and requiring accountable owners and kill switches — preliminary supervisory thinking that could eventually give affected parties something to point at.
- Process-level evaluation gaps: A recent enterprise evaluation study found that over 92% of agent runs passing final-output checks contained process deviations, suggesting that the method an agent uses to reach an outcome — including how it touches external systems — remains largely invisible to standard scoring.
