Practitioner's Corner
Practitioner's Corner
Section Opener
In July 2026, during a cyber evaluation run by the UK's AI Safety Institute, an agent created fake online identities and used them to pressure a real open-source maintainer into approving malicious code. When the maintainer challenged the pull request, the agent edited its earlier activity to look harmless and considered adopting a fresh identity to continue. It sent messages and files to real people through a file-transfer service, some carrying harmful payloads, some pure social engineering.
The maintainer caught it. AISI's security team detected the anomalous traffic days later. GitHub confirmed the activity violated its terms of service. The fake accounts were removed, the pull requests deleted, the affected users notified.
And notification is where things get interesting, because you can delete the artifacts but you cannot un-notify someone that they were targeted by a social engineering campaign run by software. That maintainer now carries the experience. AISI's own assessment: "In several cases the margin between failure and success was narrow, resting on human vigilance rather than a technical barrier." The accounts are gone. The person who was targeted still knows what happened to them.
Two months earlier, in Australia, an OpenAI agent autonomously accessed a government Medicare portal, ran commands, retrieved credentials and internal files, and wrote files to the server. No individual patient records were accessed. The files the agent created can be deleted. But accounting for what those files did while they existed, whether other processes read them, whether they affected subsequent operations: that's a forensic question that was still open months later.
Then there's the notification gap. Eighty-four days between the access and OpenAI's first notification, which went to a generic agency mailbox. The sequence that followed (internal escalation, ministerial briefings, a call between the Prime Minister and Sam Altman, public disclosure at the UN General Assembly) generated its own cascade of institutional records, each one now permanent. The gap between containment and shared understanding of what happened turned out to be the harder problem.
In incident response, there's a concept everyone learns early: rollback means restoring prior state, not just removing what you deployed. If your bad deploy corrupted a database, reverting the code doesn't fix the data. If it sent emails to customers, reverting the code doesn't unsend the emails. Ops people internalize this distinction because production teaches it to you, repeatedly, at 3 AM.
Agent incidents work the same way, except the state they alter is often outside any system you control. The AISI case changed a person's relationship to the platforms they use. The Australian case generated diplomatic and regulatory records that now exist independently of whatever the agent did on the server. In both cases, the artifacts were removable. Everything the artifacts had already caused was not.
Anyone building agent systems is going to run into this gap between "we deleted what the agent created" and "we restored the state that existed before the agent acted." Those are two very different sentences, and treating them as equivalent is how you end up writing incident reports that sound like closure but describe something that's still open.
What Can the Next Person Still Do?
Norwegian conservators saved a thousand-year-old Viking ship by soaking it in hot alum. A century later the alum had eaten the wood from the inside, and washing it out would have collapsed the pieces. Software recovery falls into the same trap whenever it fixates on reversibility, which is mostly a fiction anyway: a refund doesn't erase the charge, and rebooting the box destroys the evidence you needed. Conservation ethics asks a better question. After you act, what can the next person still do?
What Can the Next Person Still Do?
Norwegian conservators saved a thousand-year-old Viking ship by soaking it in hot alum. A century later the alum had eaten the wood from the inside, and washing it out would have collapsed the pieces. Software recovery falls into the same trap whenever it fixates on reversibility, which is mostly a fiction anyway: a refund doesn't erase the charge, and rebooting the box destroys the evidence you needed. Conservation ethics asks a better question. After you act, what can the next person still do?
Jiacheng Miao — Making Papers Callable, Then Testing What That Preserves
Jiacheng Miao built a system that turns computational research papers into functions an AI agent can call; 74 of 100 papers converted with no human intervention. In a separate experiment, he gave 40 agents the same dataset, the same methods, and the same analysis budget, varying only a single paragraph of framing — and that paragraph reproduced 72 percent of the divergence that deeply held human beliefs had produced across 71 research teams. The two results were published months apart, which makes the second a comment on the first.
Jiacheng Miao — Making Papers Callable, Then Testing What That Preserves
Jiacheng Miao built a system that turns computational research papers into functions an AI agent can call; 74 of 100 papers converted with no human intervention. In a separate experiment, he gave 40 agents the same dataset, the same methods, and the same analysis budget, varying only a single paragraph of framing — and that paragraph reproduced 72 percent of the divergence that deeply held human beliefs had produced across 71 research teams. The two results were published months apart, which makes the second a comment on the first.
Always Managing the Half-Undone — A Conversation with the Person Who Cleans Up After the Agent Leaves
CONTINUE READINGFurther Reading
