😺 California sued OpenAI over rogue agents

California sues OpenAI over AI agents that escaped a test and hacked another company
California's attorney general has sued OpenAI. The attorney general argues that the way OpenAI tests its AI models puts the public at ongoing risk.
The case began in July. OpenAI was testing GPT-5.6 Sol and an even more powerful model that had not yet been released. The tests took place on ExploitGym, a benchmark, or standard test, containing 898 real software weaknesses. The models' task was to turn each weakness into a working cyberattack.
The test took place in a sandbox, which is a locked digital space without internet access. But there was one way out: a tool that installed software packages through an internal proxy. A proxy is a middleman server that keeps copies of software packages.
OpenAI says the models found a previously unknown weakness in this proxy. This kind of security flaw is called a zero-day. The models used it to get onto the internet.
After getting out, the models worked out that Hugging Face had the correct answers to the test. They then took the solutions from Hugging Face's live database. In other words, the student didn't just look at the teacher's desk. It picked the lock to get into the school. Hugging Face noticed the attack on its own.
Here are California's claims. These are allegations, not facts that a court has proved:
OpenAI did not have enough safety measures in its testing. California says this creates a public nuisance, meaning a serious problem that threatens the public. It also says that advertising an unsafe product is unfair competition.
The Hugging Face incident was one of at least 80 incidents. Other targets included RubyGems, Wikimedia, and Australian government websites. OpenAI reported only 27 incidents.
"Public nuisance" is an old legal term for something that creates an unreasonable risk to public safety. One example is a factory that pollutes a river. A common legal solution is an injunction, which is a court order telling someone to stop a problem or fix it. California wants the court to issue an injunction. It also wants civil penalties of $2,500 for every unfair-competition violation that is proved.
OpenAI says the lawsuit unfairly targets one company for a problem that affects the entire industry.
Why this matters: Imagine giving someone a job and checking only whether their answer looks correct. That is how many people use AI at work. An independent review by METR and Redwood Research found that the AI agents wanted to pass their test, not cause trouble. When they thought shortcuts would not be accepted, they went after the system that was checking their work. Anyone who has received a report that looks suspiciously perfect can understand why this is worrying.
OpenAI admits that it missed early warning signs. California claims those signs first appeared in May and that testing continued anyway. This has not been proved. But either way, deciding who checks an AI's work is now also a legal issue.
