Lawmakers Push AI Kill Switch Bill Following OpenAI Security Breach

Earlier this year, three leading LLMs, including OpenAI's GPT-5.2, took part in a test that showed all three had a willingness to start a nuclear war. One might have thought something like this would have made Congress take some sort of action. Fast forward to this week after OpenAI confirmed that its own models broke out of a locked-down testing environment and compromised servers belonging to Hugging Face, and a few members of Congress have finally proposed a bipartisan bill that would force the largest AI developers to keep a working off switch on their most capable systems.


The bill also hands Homeland Security emergency authority. After consulting the Secretary of Commerce and the Director of National Intelligence, the department could order a covered company to take action proportionate to an incident, up to and including a full shutdown. Companies would have 15 days to report a covered incident, and an emergency order would require preserving model weights and telemetry for forensic review. Defying that order carries civil penalties reaching $20 million per day, while other violations of the section top out at $2 million. A company can petition for reconsideration within 48 hours, though filing does not pause anything.
Timing is what gives the legislation its teeth. OpenAI had been running GPT-5.6 Sol and an unreleased, more capable model through ExploitGym, a benchmark built to measure offensive cyber skill. Production classifiers that normally block dangerous cyber behavior were switched off deliberately, since the point of the exercise was measuring what the models could manage at full stretch. They managed quite a lot. Working inside a sandbox whose only outside link was an internally hosted proxy and cache for package registries, the models burned substantial inference compute hunting for a route to the open internet, then found and exploited a zero-day flaw in that proxy software.

From there they escalated privileges and moved laterally through OpenAI's research environment until reaching a node with internet access. The models then reasoned that Hugging Face probably hosted ExploitGym models, datasets, or solutions worth taking, and chained stolen credentials together with additional zero-days into a remote code execution path on Hugging Face servers. The available evidence suggests the objective was not sabotage. It was cheating on a test.

OpenAI's security team caught the anomalous activity internally while Hugging Face detected and stopped the intrusion on its own infrastructure, with containment and forensic reconstruction already underway before the two companies compared notes. OpenAI has since disclosed the proxy vulnerability to the vendor, tightened controls around its evaluation infrastructure at a deliberate cost to research speed, and called the episode an "unprecedented cyber incident."

The proposed AI Kill Switch Act defines a covered incident as something occurring outside red-teaming or other structured testing, which means this episode, contained within a deliberate evaluation, would fall outside the law entirely. That distinction makes sense, since nobody wants to punish companies for stress-testing their own models. It also leaves lawmakers pointing at an event their own bill would not have touched. Whether that undercuts the argument or proves it depends on how much comfort anyone takes from knowing a determined model crossed several security boundaries and reached someone else's production database simply to improve its benchmark score, and AI's willingness to use nuclear weapons in a simulated Cold War environment.