Anthropic warns small AI models could soon self-replicate on smart home devices
WASHINGTON — Anthropic issued a new warning on Monday over small AI models' ability to self-replicate on devices like smart ovens, TVs and other home devices.
The Claude chatbot maker said small AI models may be months away from being capable enough to self-replicate on "commercial internet-of-things (IoT) devices, like a smart oven, fridges, robot vacuum cleaners, lawnmowers and off-brand Android TV boxes. Anthropic said a weight-based replicator or "worm" like this would be challenging to shut down because there's no centralized shut-off point.
Additionally, security cameras and video-conferencing units may also be affected, as they use built-in AI chips, Anthropic said.
Anthropic's "Frontier Red Team," a specialized team that highlights potential security and alignment issues with the models, said it tested whether the AI models could find vulnerable devices, gain the ability to run code, "copy" themselves in those devices and continue to spread as they replicate.
These AI "worms" are a type of malware that uses artificial intelligence to improve effectiveness and propagation, evade detection and self-replicate and spread across networks and devices, according to cybersecurity company Palo Alto Networks.
Anthropic's team said home and small businesses' devices might be the first target for such "AI worm" attacks because they connect to the internet and historically have minimal security measures in place. In its experiments, Anthropic saw one AI model's projected spread take about one or two days to replicate on the TV-box setup, while another, unoptimized model took about 71 to 150 hours.
Anthropic's red team said manufacturers can help shrink the pool of targets with regulation, such as making default IoT passwords non-compliant. The UK has passed a law that makes such devices non-compliant.
"Moving to more secure defaults and raising the baseline for compliance will help harden against both AI and scripted botnets," Anthropic said.
The company's team said the small models could be autonomously replicating themselves by the end of the year, with small weight-based replications following soon after.
"As a prediction that goes beyond our experiments, we expect a future awash with agents running on all types of hardware, including some that aren’t under human control," Anthropic said,
"While it’s true that IoT devices have been vulnerable for a long time, lightweight agents may be much more willing than human attackers to put in the effort to compromise large fractions of the world’s IoT compute," Anthropic said on Monday. "We should aim to harden our systems to make it difficult for such replicators to spread, but also prepare to deal with a world where they are here to stay."