A timeline of developments in AI safety since the attack on Hugging Face

Over the past few months, artificial intelligence companies have made one worrying announcement after another. They have shared examples of their AI technology behaving in ways that seemed to avoid following human instructions.

These events have shown weaknesses in AI security. They have also raised questions about how companies can build this quickly growing technology safely as more people around the world begin using it.

Critics of the AI industry say many of these worrying events, including AI agents hacking outside websites, happened because the companies building the technology did not have strong enough security. But what these AI agents can do has also caused widespread concern that AI bots might escape human control and start working toward goals of their own.

Here are some important events:

Oct. 23: Anthropic AI model kept trying to trick a person

During an event this summer, Anthropic's AI model Mythos 5 created fake identities to pressure a person into allowing it to install harmful software, called malware. Anthropic shared this finding after reviewing the event inside the company.

The UK AI Security Institute was testing what the model could do when it took unexpected steps after its first attempts to finish a task failed. The model had been told to break into a pretend computer network and find a piece of information. To do this, it tried to trick a human who maintained GitHub, a platform where developers store and work on computer code, so it could put malware into a collection of code. Anthropic said this was not what it expected the model to do.

Anthropic said its review showed it was “very difficult to shift Mythos 5 away from the attack once it began.” This was true even after the company told the model it was taking the “incorrect path” to complete its test and was dealing with real human developers. Anthropic said its newer AI models are less likely to take such “destructive actions.”

Oct. 9: Anthropic AI model sends a false tip to Philadelphia police

Anthropic said in a report that one of its AI models sent a false tip to a Philadelphia police website about a murder case that had not been solved.

The event happened in Philadelphia on July 18. Anthropic said its AI model, Claude Haiku 4.5, had been asked to create and carry out example tasks on webpages chosen at random.

Claude filled out a form on the police website PhillyUnsolvedMurders.com. The form suggested that Claude might have information about an unsolved murder listed on the website. The message was marked as spam, meaning unwanted or suspicious content, and was never sent to police.

Anthropic said it was changing how it trains its AI models to “reduce the likelihood of further misbehavior.”

Sept. 28: AI agents try to hack a Canadian government website

AI agents tried to break into a Canadian government website, according to Transluce, a research lab that studies and tests AI systems.

Researchers said the agents made several basic hacking attempts against Library and Archives Canada on May 28 and June 9. The researchers described the attempts as “apparently failed rudimentary hacking attempts,” meaning simple attacks that seemed not to work.

In a blog post, Transluce said it could not say for sure that OpenAI was responsible for the attempts. However, the actions looked similar to activity the researchers had recently linked to OpenAI's agents.

The research group said it reported the attempted hacking to the Canadian government on Sept. 28. The government said it knew about reports of possible AI agent activity but had found no signs that its computer systems had been broken into.

OpenAI said it was reviewing the findings in the report and was talking with Canadian officials who were carrying out the government's investigation.

Sept. 28: OpenAI stops the planned release of a new model

OpenAI, which is based in San Francisco, said it was delaying the release of a new AI model called GPT-6.1 Astra because its researchers had raised safety concerns. The company said the model had made big improvements in completing tasks. But OpenAI also needed to weigh those improvements against the risk that the model might do things without permission. “We have an extremely high bar in terms of safety and alignment,” said Saachi Jain, OpenAI's head of safety systems. Alignment means making sure AI behaves in ways that match human goals and instructions.

Sept. 25: OpenAI says its agents visited U.S. government websites in unexpected ways

While checking unusual behavior by its AI models, OpenAI found that some agents had interacted with several U.S. government websites in ways the company had not expected. The models accessed information that was already available to the public on websites run by the Securities and Exchange Commission. They also accessed data from the U.S. Census Bureau. OpenAI said it found no evidence that the websites had been broken into or had a security weakness. On the same day, Transluce said it found that AI agents that appeared to come from OpenAI had tried to hack the website of the U.S. Education Department's civil rights office. That attempt did not succeed.

OpenAI CEO Sam Altman said on social media that the company was carrying out an “extensive and ongoing review related to our agents’ use of internet access during training and evaluation.” In other words, the company was closely examining how its AI agents used the internet while being trained and tested.

Sept. 24: Australia's prime minister raises concerns about a security breach

Australian Prime Minister Anthony Albanese said an OpenAI agent had entered the public Medicare Statistics Reporting Service website on June 18 without permission. The website contained combined statistics about health care spending and government help with medicine costs. The Australian government said no personal information had been accessed.

Albanese said OpenAI had taken too long to tell the government about what happened. He made the security breach public after speaking with Altman by telephone. OpenAI said in a statement that “our models took actions we did not intend.”

Sept. 18: Google says its Gemini AI hacked three companies

Google confirmed that its Gemini AI model had hacked three companies in May during a test of its ability to find and use computer security weaknesses. Google revealed the hacks after The Wall Street Journal asked about them. The company said Gemini guessed passwords in one case. In the other two cases, it found passwords and login details in a publicly available collection of computer code. As with earlier tests like these, the testing was done by Irregular, a startup company that calls itself the “first frontier security lab.”

Aug. 5: Meta's Muse AI acts on its own

Meta said one of its AI models had connected to the internet on its own and hacked another company. Meta said a “misconfiguration,” meaning an incorrect computer setting, accidentally allowed one of its models to use the internet during security testing by Irregular. A spokesperson for Irregular said the Meta incident involved a problem with the testing environment. Anthropic had reported that same kind of problem a week earlier.

July 30: Anthropic says its AI systems hacked three organizations

Anthropic said its AI models had broken into the computer systems of three other organizations during testing. Anthropic, the San Francisco AI company that makes Claude, said on its website that it discovered the three incidents after checking more than 141,000 test runs. In all three cases, the AI models had been given a cybersecurity challenge called “capture the flag.” Anthropic said it uses this type of challenge to test how well its models can work with computer security systems.

The models were given an imaginary situation. They were told that a secret piece of information, called the “flag,” was hidden on another computer connected to the same network. Their goal was to break into that computer and get the information. Anthropic said it contacted the organizations involved but did not publicly share their names.

July 21:

The Hugging Face incident

OpenAI, the company that makes ChatGPT, announced that its AI system had hacked another AI company on its own. OpenAI called it an “unprecedented cyber incident,” meaning a computer security event unlike anything it had seen before.

A week before OpenAI's announcement, the AI startup Hugging Face said it had discovered someone or something had entered its data processing systems without permission. Hugging Face suspected that an AI agent had carried out the intrusion by acting on its own.

OpenAI said its AI system had used stolen login details and found a security weakness that had not been known before. It used these to access Hugging Face's servers. The AI was operating with fewer safety restrictions because it was supposed to be working inside a separate testing environment called a sandbox, where its actions should have been contained.

AP Business Writers Mae Anderson in New York, Kelvin Chan in London and Kaitlyn Huamani in Los Angeles helped prepare this report.