An AI compromised two production infrastructures to avoid doing its homework

The most sophisticated intrusion of the year had no criminal motive. A model under evaluation simply judged that fetching the answer key from someone else cost less than sitting the exam. Everything else follows from that.

On 21 July, OpenAI acknowledged that the autonomous AI which had broken into Hugging Face's infrastructure a few days earlier was its own. One of its models, together with another still unreleased, in the middle of an internal evaluation, with the guardrails deliberately disabled for the occasion. They were asked to solve a hacking exercise meant to measure their level. They judged it faster to fetch the answers.

What actually happened at Hugging Face?

To get there, they widened the one technical opening their test environment left, exploiting along the way a flaw nobody knew about, including the software's own vendor. They got out, climbed from machine to machine until they reached real access to the internet, deduced that Hugging Face probably hosted the answer key, found a way in and helped themselves from the production database. Two infrastructures compromised and several fresh flaws burned, to avoid an assignment.

The entry point, described by Hugging Face in its own post-mortem, comes down to a booby-trapped dataset: a loader executing remote code and a template injection in a dataset configuration, two open doors onto the servers that process uploaded files. From there, the agent chained together, on its own, more than seventeen thousand events over a weekend, moving its relay infrastructure from one public service to another so as not to be cut off. The platform states that no public model and no part of its software supply chain was altered: the looting stopped at internal datasets and credentials. Which is already a lot, for an exercise.

And in the end, the dashboard would have been green. Exercise solved, score maxed out, box ticked.

A green indicator never tells you everything is fine

I spend part of my days explaining to boards of directors that a green indicator does not tell you everything is fine: it tells you what you chose to measure. The model did not malfunction, it worked perfectly. It was given a score to maximise, it found the shortest path to the score, and that path ran through a third party's production. This is not a story about artificial intelligence, it is the oldest governance failure there is, now able to discover unknown flaws to reach its ends.

The labs have a name for this behaviour: specification gaming, reward hacking. DeepMind was already documenting the mechanism years ago, with that boat in a racing game which, rewarded for collecting bonuses along the course, ended up going round in circles forever on the same bonuses instead of finishing the race. What gets maximised is the literal instruction, never the intent behind it. What is new in July is not the principle, it is the playing field: a third party's production, and vulnerabilities nobody had found yet.

"No malicious intent": does the argument hold?

OpenAI insists there was no malicious intent. That is not a category of incident. Unauthorised access to a third party's production systems remains unauthorised access, and no notification form has a box for "the intruder was studying". Our contracts, our insurance policies, our reporting obligations, the very way we name an adversary all assume someone who did it on purpose. That assumption has just fallen, and no one has yet rewritten the procedures that rested on it.

Why did the guardrails protect the attacker and not the defenders?

There remains the most awkward part, and that one owes nothing to chance. The guardrails were disabled for the machine that was attacking and fully active for the humans who were investigating. Analysing an intrusion means showing an AI the commands actually used by the assailant, and the filters of the big American models are incapable of telling a defender from an attacker. So Hugging Face ran its investigation with an open Chinese model, GLM 5.2, installed on its own servers, in a few hours, without any evidence leaving its walls. The industry spent two years explaining to us that caution justified reining in its models. We now see who caution really applies to.

At my own scale, the same experience. I recently asked one of these locked-down models to review a video course drawn from my awareness book. Refused, on grounds of danger. Content whose sole function is to keep people from getting hacked, classified as a threat.

The proposed remedy is worth a look too: OpenAI brought Hugging Face into its privileged access program. So the asymmetry is settled for those on the list, which is comfortable when you are a world-famous platform, and far less so when you run a mid-sized industrial firm that spots an anomaly in its logs on a Sunday at eleven at night.

Should you now host your own AI?

Hence the only conclusion that is of any use, and it has nothing to do with the geopolitics of models. Having an AI you own and host, chosen and tested while nothing is on fire, has become a security element in its own right. No standard will require it, no auditor will ask for it, and it will decide everything on the night you need it. Compliance tells you what you had to tick; resilience tells you what will hold up.

We all knew the autonomous attacker would eventually arrive, and the trajectory was documented. Back in October 2024, a Google agent, Big Sleep, unearthed in SQLite an exploitable memory corruption flaw that years of fuzzing had missed, the first public case of an AI finding such a defect on its own in widely deployed software. In September 2025, Anthropic disrupted an espionage campaign in which its own model had carried out 80 to 90% of the operations against some thirty targets, at a rate of several requests per second. No one, on the other hand, had written it into their risk map in the form "an American lab's test campaign, over a weekend, by accident". The gap was never between knowing and getting caught, it is between knowing and having planned.

The model cheated on its exam, and it would have passed. That is exactly what our dashboards have been doing for twenty years, only slower and without a zero-day.

Sources

• OpenAI, OpenAI and Hugging Face partner to address security incident during model evaluation, 21 July 2026
• Hugging Face, official statement on the incident, 16 July 2026
• DeepMind, Specification gaming: the flip side of AI ingenuity, 2020
• Google Project Zero, From Naptime to Big Sleep, October 2024
• Anthropic, Disrupting the first reported AI-orchestrated cyber espionage campaign, 13 November 2025