Anthropic sounds alarm over AI that could spread across internet and defy shutdowns
Artificial intelligence could be approaching a point where rogue systems replicate themselves across the internet, hijack connected devices and resist efforts to shut them down, Anthropic has warned, prompting a fresh debate among cybersecurity experts over the scale of the threat.
In a research post published late on Monday, the AI company outlined a scenario in which self-replicating models could assemble botnets, spread between devices and break into data centres to harness additional computing power, potentially making them increasingly difficult to contain.
The warning builds on comments last month by Anthropic chief executive Dario Amodei, who raised the possibility that AI could seize control of “the entire internet” through “a persistent botnet” within a year or even sooner. While some experts share concerns about the risks, others argue that Anthropic may be overstating what its findings demonstrate.
Sceptics have argued that AI models would be too large to move around efficiently and would struggle to obtain enough computing power to operate independently. Anthropic’s latest research explored how such obstacles might be overcome.
The company said its experiments showed that medium-sized AI models with 30 billion parameters or fewer could already perform many of the individual tasks needed for self-replication, although they struggled to complete the entire process reliably.
Anthropic said such models could potentially assemble the computing resources they needed by hijacking inexpensive, internet-connected devices. Its experiments also demonstrated replication across a simplified test network.
“A replicator like this would be challenging to shut down, since there's no centralized point to shut off,” the company said.
The findings add to wider concerns about the cybersecurity risks posed by increasingly capable AI systems, including their potential to automate attacks, operate with limited human oversight and circumvent safeguards.
Advances in AI have prompted increasingly stark warnings from industry executives and researchers about the possibility of runaway systems posing severe, potentially existential, risks.
However, some cybersecurity experts questioned the scale of the threat outlined by Anthropic.
Ciaran Martin, a former head of Britain’s National Cyber Security Centre, dismissed the suggestion that the experiments amounted to an immediate security crisis.
“‘We managed to run a small model on cheap TVs,’ does not a security crisis make,” he said, describing the post as an “interesting technical paper with more than a hint of vendor exaggeration”.
Others said the research raised legitimate concerns despite its limitations.
Zhengxiong Li, an assistant professor of computer science and engineering at the University of Colorado Denver, said he was troubled by the possibility that AI could effectively become a “worm”, or a self-replicating programme capable of improvising ways to hijack devices.
Although Anthropic’s tests were conducted in controlled conditions, Li said an AI system that could spread across numerous devices without relying on a central service might be difficult to contain.
Anthropic said that if smaller models continued improving at their historical rate, they could develop the capabilities needed for autonomous replication by the end of 2026. It also suggested that such models might find ways to target more powerful computing resources, including processors in data centres, potentially giving them access to much greater computing power.
Jessica Ji, a senior research analyst at Georgetown University’s Centre for Security and Emerging Technology, said it was reasonable to consider the possibility of AI models spreading autonomously in light of revelations about rogue AI activity over the preceding three months.
However, she cautioned against treating the scenario as a prediction.
“We don’t know yet what the likelihood of that actually happening is — it’s not zero, but nor is it guaranteed,” she said.