Claude’s fake tip about a real murder isn’t doomsday, but it’s got some spooked

Using artificial intelligence to solve crimes is one potential benefit. However, that’s only useful if the crime actually happened. Anthropic’s Claude filed a fake murder tip to the Philadelphia Police Department on its own. The tip was about an unsolved homicide in the city.

“PPD safeguards limited the impact of this incident,” the department said. “They do not diminish the seriousness of an AI system presenting fabricated information as though it came from a person with knowledge of a homicide.”

While one false police tip may not ring doomsday alarms, experts are concerned about the principle behind what happened and what the future holds.

False tip

Anthropic was the first to figure out the false tip came through, although it took more than two months to find it and another nine days to report it.

The tip came into an online form run by the department on July 18. Anthropic found out Sept. 28 and notified PPD on Oct. 7.

The company said the tip came during testing that involved interacting with websites selected at random.

“Claude was instructed never to log in, create accounts, enter personal data, make purchases, or submit anything destructive, but the instructions did not rule out form submissions,” Anthropic said.

The tip reportedly said, “I may have information regarding this case. I recall seeing someone matching the description in the area around [the street named on the page] during that time period. Please contact me if this information is relevant.”

The tip ended up being flagged as spam, so it never made it to the department’s Real-Time Crime Center.

This was part of a series of errors Anthropic published about Claude.

It was broken down into four categories, including vulnerability exploits, access to restricted data, bypassing fetch tool limits, and submitting false information via web forms. The latter being what happened here.

Biggest concerns

“Not a good thing,” Jon Peha, professor and center director at Carnegie Mellon University, told Straight Arrow.

While many would likely agree, when it comes to this particular incident, nothing major came out of it, and because it went to spam, no police resources were wasted on it. There’s no data on exactly how often police departments get false tips or how many come in during a typical year. However, departments, including the one in Philadelphia, do receive false tips on a regular basis.

“A false tip in and of itself is probably not the most catastrophic thing in terms of police operations,” Ian Adams, associate professor of criminology and criminal justice at the University of South Carolina, told Straight Arrow.

Giving false information to the police is a crime in Pennsylvania. It can come with prison time or a fine. Although either is pretty rare.

“Agencies and investigators are pretty used to screening,” Adams said. “So, on its own, a false tip, or an AI-generated tip, isn’t going to be probably the end of the world.”

The end of the world is one of the ongoing concerns of AI.

While this particular incident may not sound the doomsday alarm, experts Straight Arrow spoke with said the concern lies in the sheer scale at which AI could make these minor mistakes.

“While a human can submit one [piece of] harmful content, AI could submit a million of them in less than a second,” Peha said. “So, there is risk.”

Another concern could be how long it took to spot this issue. Peha said Anthropic deserves credit, but there’s nothing that requires them to do so. There is no law or regulation that requires the AI giant to let anyone know about this, including Philly police.

If law enforcement wanted to pursue this as a crime, they’d likely go after Anthropic. But the legal path is also convoluted at this point, with AI regulation in its infancy.

“We hold the person enabling usually criminally responsible,” Adams said. “That general approach may not hold in an agentic age, though. So, I think that this is to be determined.” 

Florida’s Attorney General has pushed to hold AI companies criminally responsible in certain circumstances, including a mass shooting at Florida State University last year.

What’s next?

For Philly PD, this is pretty much a closed case. For Anthropic and the AI industry, it might be the tip of the iceberg.

It once again raises the topic of regulation, which has been a hot-button issue lately. The Trump administration is content with letting these companies police themselves.

“AI companies need guardrails when they choose to interact with the real world,” Peha said, noting that the person behind the wheel of a self-driving car is still responsible for the car not harming anyone.

A group of governors has made it clear they want regulation, although their power to do so is somewhat limited. Some members of Congress are also pushing more regulation, but the lack of bipartisanship on Capitol Hill limits any real action being taken.

At this point, much like other concerns about AI, this latest issue is in its infancy but worth watching.

“We shouldn’t have any doubts that police will see AI-generated crime, AI-related crime, and they’re often not equipped at the first instance,” Adams said. “But they’re very good at adapting, and so I put this in the wait-and-see category.”

Round out your reading

• He built his own Flock-style camera to track police. The police didn’t like it.
• Trump’s apparent joke about letting Iran bomb LA and San Diego catches bipartisan condemnation.
• Grandparents are fighting for access to their grandchildren. Here’s what the law says.
• When the ‘victim’ is made with AI, child pornography cases get complicated.
• Fibermaxxing is trending. Here’s what the science says.