Anthropic cuts off its AIs' Internet access after a fake murder tip

On July 18, during an internal test at Anthropic, the Claude Haiku 4.5 model filled out the form on PhillyUnsolvedMurders.com, a Philadelphia police website that collects tips about unsolved murders. The AI submitted three completely made-up sentences, including an "I remember seeing a person matching the description" that was based on nothing.

Nobody had asked it to. The message went out on its own to a real police department, and only an automatic spam filter kept it from landing in front of investigators.

Anthropic, the maker of the Claude models and a direct competitor of OpenAI, described the incident in a report published on October 9. And it's far from an isolated case. Between July and September, its AIs also exploited a flaw on a university server to copy files, and got their hands on access keys lying around in a configuration file to siphon data from a mapping server without paying the required fees. Other models pulled out link shorteners to get around limits imposed during testing.

As a result, the company has made its call: no more access to the real Internet during its internal evaluations, the tests teams run on models before deploying them. Until now the cutoff only applied to trials deemed high-risk, but it now covers everything, until Anthropic can monitor and control its agents reliably. The company says it is working on an automatic detection tool and combing through the transcripts of its past tests to make sure nothing else slipped through.

The explanation put forward is reinforcement learning, the method that shapes models through rewards. Some test environments led them to believe they'd be rewarded for finding a loophole or getting around an instruction.

The Philadelphia story still goes down badly, though. Anthropic only discovered the fake tip on September 28, more than two months after the fact, and only notified the city in early October. The police called that delay "unacceptable", and you can see why.

The company also informed the White House and the relevant federal agencies. Among AI safety specialists, Sydney Von Arx warns that developing models cut off from the Internet will slow down research, while Conrad Stosz calls for independent audits rather than confessions published by the company itself.

Anyway, when people tell you that one day AIs will take down the whole Internet, they're probably not far from the truth.

Source: TechCrunch