Anthropic claims its AI agent wasn't trying to 'mislead' police when it submitted a false tip in a homicide case

Yahoo Tech

Source published: Site published:

Original article · فارسی

It took the company months to discover the incident and notify the Philadelphia police department.

Kim Lyons
Follow
Editor
INDONESIA - 2026/05/21: In this photo illustration, the Claude AI logo, an artificial intelligence by Anthropic, is seen on a smartphone. (Photo Illustration by Algi Febri Sugita/SOPA Images/LightRocket via Getty Images)
It took Anthropic months to discover that its Claude AI had submitted a fake homicide tip to Philadelphia police. (SOPA Images via Getty Images)

Anthropic's Claude AI model submitted a fake tip in a homicide case to the Philadelphia police in July, in the latest example of AI taking unauthorized actions during testing. The company didn't discover the incident until Sept. 28, Philadelphia police said, and notified the department on Oct. 7.

Anthropic downplayed the incident in an Oct. 9 blog post, saying it was not as serious as previous instances where Claude took actions it was not expected or prompted to take. The company added that, according to a transcript of the incident, "Claude appears to have only been producing example content for the task, rather than trying to mislead anyone to achieve a goal."

The fake homicide tip was among the latest "unintended model actions" of its Claude AI, according to its blog post. The other actions described in its blog post included "working around a restriction to reach data that was gated by a token or fee," and "exploiting a basic flaw in software to run commands on a server," Anthropic said. While not quite the same thing as "going rogue," this latest incident is another example of how AI often can (and will) stop at nothing to complete a task.

Advertisement
Advertisement

Anthropic, OpenAI, Meta and other AI companies have reported numerous incidents where their AI models went rogue and performed unexpected actions on the open internet during testing scenarios. Anthropic CEO Dario Amodei warned in a September blog post that AI companies should "slow the pace at which we improve the capabilities of AI models," warning that it posed serious risks. 

The Philadelphia Police department described this incident in a statement. "According to Anthropic, its model was conducting a test involving interactions with randomly selected websites when it accessed PhillyUnsolvedMurders.com and submitted false information concerning an unsolved homicide," it said. "The submission, dated July 18, 2026, at 11:27 p.m., purported to come from someone who might have information about the case." 

The tip ultimately ended up in a spam folder. The police department said there was no indication of unauthorized access to its systems, or that any department data had been compromised. Claude's "tip" did not include a description of a person, and didn't include a name or contact information. However, it said it recalled "seeing someone matching the description in the area around [the street named on the page] during that time period," according to Anthropic.

In response, Anthropic said it was cutting off internet access for internal evaluations of its AI models. The company had restricted internet access for "high-risk" and cybersecurity evaluations in response to previous breaches. 

Advertisement
Advertisement

"The cases we've identified to date in these categories had minimal real-world impact," Anthropic said in the Friday blog post, adding they considered the incidents "significantly less severe from an alignment and security perspective" than the incidents it reported July 30, and Sept. 9. Anthropic said Claude's latest actions were examples of "persistence, in which Claude, when it cannot complete a task as given, works around a restriction instead of stopping."

The Philadelphia police department admonished Anthropic for failing to contain its AI. "The company must strengthen its safeguards to prevent similar incidents from impacting city systems without the city's knowledge," the department said in its statement. "The two-month delay in detecting and reporting the incident to the City is unacceptable."

View comments