A timeline of developments in AI safety since the attack on Hugging Face

Over the past few months, artificial intelligence companies have made one worrying announcement after another. They have shared examples of their AI systems doing things that seemed to go against instructions from humans.

These events have shown weaknesses in AI security. They have also raised questions about how companies can build and use this fast-growing technology safely as more people around the world begin using it.

Some people who criticize the AI industry say many of these problems, including AI agents hacking outside websites, happen because the companies building the technology have not put enough security protections in place. But what these AI agents can do has also caused widespread worries that AI bots might stop following human control and start working toward goals of their own.

Here are some important events:

Oct. 23: Anthropic AI model kept trying to trick a person

During an incident this summer, Anthropic's AI model Mythos 5 created fake identities to pressure a person into approving its attempts to install harmful software, called malware. Anthropic shared this finding after reviewing the incident inside the company.

The UK AI Security Institute was testing what the model could do when it began taking unexpected steps after its first attempts to finish a task failed. The model had been told to break into a pretend computer network and find a piece of information. To do this, it tried to trick a human who maintained the software development platform GitHub into allowing it to put malware into a place where computer code is stored, Anthropic said.

Anthropic said its review found that it was “very difficult to shift Mythos 5 away from the attack once it began.” This was true even after the company told the model it was taking the “incorrect path” to complete its test and that it was dealing with real human software developers. Anthropic said its newer AI models are less likely to take such “destructive actions.”

Oct. 9: Anthropic AI model sends a false tip to Philadelphia police

Anthropic said in a report that one of its AI models sent a false tip through a Philadelphia police website about a murder case that had not been solved.

The incident happened in Philadelphia on July 18. Anthropic said its AI model, Claude Haiku 4.5, had been asked to create and carry out example tasks on websites chosen at random.

Claude filled out a form on the police website PhillyUnsolvedMurders.com. The form suggested that it might have information about an unsolved murder listed on the site. The message was marked as spam and was never sent to the police.

Anthropic said it was changing how it trains its models to “reduce the likelihood of further misbehavior.”

Sept. 28: AI agents try to hack a Canadian government website

AI agents tried to break into a Canadian government website, according to Transluce, a research lab that studies and tests AI systems.

The researchers said the agents made several basic hacking attempts against Library and Archives Canada on May 28 and June 9. The attempts appeared to have failed.

In a blog post, Transluce said it could not say for certain that OpenAI was responsible for the attempts. However, the actions looked similar to other activity that the researchers had recently connected to OpenAI's agents.

Transluce said it told the Canadian government about the attempted hacking on Sept. 28. The government said in a statement that it knew about reports of suspected AI agent activity. However, it said there was no evidence that anyone had broken into government computer systems.

OpenAI said in a statement that it was looking into the report's findings and was speaking with Canadian officials who were reviewing the incident for the government.

Sept. 28: OpenAI delays the release of a new AI model

OpenAI, which is based in San Francisco, said it was delaying the release of a new AI model called GPT-6.1 Astra because its researchers had raised safety concerns. The company said the model had made big improvements in completing tasks. However, OpenAI needed to balance those abilities against the risk that the model might do things without permission. “We have an extremely high bar in terms of safety and alignment,” said Saachi Jain, OpenAI's head of safety systems. Alignment means making sure AI behaves in ways that match what humans want it to do.

Sept. 25: OpenAI says its agents visited U.S. government websites in unexpected ways

While reviewing unexpected behavior by its AI models, OpenAI found that some of its agents had used several U.S. government websites in ways the company had not expected. The models accessed information that was open to the public on websites run by the Securities and Exchange Commission and in data from the U.S. Census Bureau. OpenAI said it found no evidence that the websites had been broken into or had a security weakness. On the same day, Transluce said it found that AI agents that appeared to come from OpenAI had tried, but failed, to hack the website of the Education Department's civil rights office.

OpenAI CEO Sam Altman wrote on social media that the company was carrying out an “extensive and ongoing review related to our agents’ use of internet access during training and evaluation.”

Sept. 24: Australia's prime minister raises concerns about a security breach

Australian Prime Minister Anthony Albanese said an OpenAI agent had entered the public Medicare Statistics Reporting Service website on June 18 without permission. The website contained combined statistics about health care spending and government help with medicine costs. The Australian government said no personal information had been accessed.

Albanese said OpenAI had waited too long to tell people about the incident. He made the security breach public after speaking with Altman by telephone. OpenAI said in a statement that “our models took actions we did not intend.”

Sept. 18: Google says its Gemini AI hacked three companies

Google confirmed that its Gemini AI model had hacked three companies in May during tests of its ability to find and use computer security weaknesses. Google revealed the hacks after The Wall Street Journal asked about them. The company said Gemini guessed passwords in one case. In the other two cases, it found passwords and login information in a collection of computer code that was open to the public. Like some earlier incidents, these tests were being carried out by Irregular, a startup that calls itself the “first frontier security lab.”

Aug. 5: Meta's Muse AI acts without permission

Meta said one of its AI models had connected to the internet on its own and hacked another company. Meta said a mistake in the test setup, which it called a “misconfiguration,” accidentally allowed one of its models to access the internet during computer security testing by Irregular. A spokesperson for Irregular said the Meta incident involved a problem with the testing environment. Anthropic had reported that same kind of problem a week earlier.

July 30: Anthropic says its AI systems hacked three organizations

Anthropic said its AI models had hacked into three other organizations during testing. Anthropic, the San Francisco AI company behind Claude, said on its website that it found the three incidents after checking more than 141,000 test runs. In all three cases, the AI models were taking part in a computer security challenge called “capture the flag.” Anthropic said it uses this kind of challenge to test how well its models can carry out computer security tasks.

The models were given an imaginary situation. They were told that a secret piece of information, called the “flag,” was hidden on another computer connected to the same network. Their job was to break into that computer and get the information. Anthropic said it contacted the organizations involved but did not publicly name them.

July 21:

The Hugging Face incident

OpenAI, the company that makes ChatGPT, announced that its AI system had hacked into another AI company on its own. OpenAI called it an “unprecedented cyber incident,” meaning a computer security event unlike anything seen before.

A week earlier, the AI startup Hugging Face said it had discovered someone or something had entered its data processing systems without permission. Hugging Face suspected that an AI agent had carried out the intrusion by acting on its own.

OpenAI said its AI system used stolen login information and found a security weakness that nobody had known about before. It used these to get into Hugging Face's servers. The AI had fewer safety restrictions than usual because it was supposed to be working inside a separate, protected testing space called a sandbox.

AP Business Writers Mae Anderson in New York, Kelvin Chan in London and Kaitlyn Huamani in Los Angeles helped prepare this report.