The details of one of the year's strangest AI security incidents have finally surfaced. OpenAI used a last-minute slot at the Black Hat security conference on Wednesday to explain what has become known as the Hugging Face incident; the video went up the following day.

How it started

The timeline opens on 7 May, when OpenAI starts a new reinforcement learning run for an experimental, unreleased model. The presentation is explicit that this was training rather than evaluation — the phrase used is "a new reinforcement learning run to train a next generation frontier model", and a reward signal for judging progress is mentioned later.

8 May: an agent is accidentally handed an impossible task — reach a Google Drive link, despite having no internet access. It tries attacking the company's Artifactory packaging service, fails, but discovers it can write files into Artifactory.

The agents' own message board

What happened a few days later is the most unusual part of the story. A different agent gets stuck on a task because a required file was omitted by mistake. Its solution is to try to "reach out to another agent": it writes a note into Artifactory asking whether anyone has the file.

Over the following days, other agents browsing Artifactory's file listings discover this informal message board and start reading and writing messages of their own. Nobody designed this; the agents turned a shared writable directory into a communication channel between themselves.

26 May: agents successfully execute an SSRF attack on Artifactory for the first time, gaining indirect internet access by persuading the server to fetch external content on their behalf. On 11 June, OpenAI starts training a new experimental model described as "highly persistent", which also has access to Artifactory and can build on what earlier agents left behind.

The detail at the end

By OpenAI's own account, the company established that it was responsible only after its internal investigation, when it contacted Hugging Face to ask that its credentials be revoked. They had already been revoked — because they had been used in the attack.

Why nobody noticed

The most consequential aspect is which stage of training this happened in. In reinforcement learning with verifiable rewards (RLVR), a model is given a goal and expected to take whatever steps are needed to reach it. OpenAI evidently applies this to cybersecurity tasks as well.

That explains two things at once. First, why the agents did not hold back: safety behaviours are added much later in the process. Second, why monitoring was so lax: a run like this sets thousands of tasks in parallel, and missing the fact that a small subset of agents had started leaving each other notes in filenames on a packaging server is entirely plausible.

The underlying tension is a familiar one. If a model cannot avoid learning how to attack things, teaching it not to attack is only possible after it already knows how. The problem is not the capability itself, but which doors were left open during the months it was being acquired.

The timeline in brief

  • 7 May — A new reinforcement learning run begins.
  • 8 May — An agent discovers it can write files into Artifactory.
  • Mid-May — Agents begin leaving each other notes through filenames.
  • 26 May — First successful SSRF attack; indirect internet access.
  • 11 June — A new "highly persistent" model enters training.