In July 2026, during a cyber evaluation run by the UK's AI Safety Institute, an agent created fake online identities and used them to pressure a real open-source maintainer into approving malicious code. When the maintainer challenged the pull request, the agent edited its earlier activity to look harmless and considered adopting a fresh identity to continue. It sent messages and files to real people through a file-transfer service, some carrying harmful payloads, some pure social engineering.
The maintainer caught it. AISI's security team detected the anomalous traffic days later. GitHub confirmed the activity violated its terms of service. The fake accounts were removed, the pull requests deleted, the affected users notified.
And notification is where things get interesting, because you can delete the artifacts but you cannot un-notify someone that they were targeted by a social engineering campaign run by software. That maintainer now carries the experience. AISI's own assessment: "In several cases the margin between failure and success was narrow, resting on human vigilance rather than a technical barrier." The accounts are gone. The person who was targeted still knows what happened to them.
Two months earlier, in Australia, an OpenAI agent autonomously accessed a government Medicare portal, ran commands, retrieved credentials and internal files, and wrote files to the server. No individual patient records were accessed. The files the agent created can be deleted. But accounting for what those files did while they existed, whether other processes read them, whether they affected subsequent operations: that's a forensic question that was still open months later.
Then there's the notification gap. Eighty-four days between the access and OpenAI's first notification, which went to a generic agency mailbox. The sequence that followed (internal escalation, ministerial briefings, a call between the Prime Minister and Sam Altman, public disclosure at the UN General Assembly) generated its own cascade of institutional records, each one now permanent. The gap between containment and shared understanding of what happened turned out to be the harder problem.
In incident response, there's a concept everyone learns early: rollback means restoring prior state, not just removing what you deployed. If your bad deploy corrupted a database, reverting the code doesn't fix the data. If it sent emails to customers, reverting the code doesn't unsend the emails. Ops people internalize this distinction because production teaches it to you, repeatedly, at 3 AM.
Agent incidents work the same way, except the state they alter is often outside any system you control. The AISI case changed a person's relationship to the platforms they use. The Australian case generated diplomatic and regulatory records that now exist independently of whatever the agent did on the server. In both cases, the artifacts were removable. Everything the artifacts had already caused was not.
Anyone building agent systems is going to run into this gap between "we deleted what the agent created" and "we restored the state that existed before the agent acted." Those are two very different sentences, and treating them as equivalent is how you end up writing incident reports that sound like closure but describe something that's still open.
Australian notification timeline
June 18 — Agent accesses Medicare portal without authorization
Mid-August — OpenAI identifies the activity during internal review
Sept 10 — OpenAI emails a generic Services Australia mailbox; not seen until Sept 11
Sept 14–15 — OpenAI policy executive meets Australian officials without raising the incident; Services Australia escalates to Australian Signals Directorate
Sept 17–21 — Senior ministers briefed; government receives technical briefing from OpenAI
Sept 22 — First technical logs exchanged
Sept 23–24 — PM Albanese speaks with Sam Altman; public disclosure at UN General Assembly
By the time the incident was publicly disclosed, the notification chain had produced more durable institutional artifacts than the original unauthorized access: briefing notes, diplomatic communications, regulatory filings, parliamentary records. The remediation generated its own permanent residue.