Anthropic warns small AI models could enable self-spreading malware

Anthropic says small AI models may be only months away from being capable of self-replicating on commercial internet-connected devices.[1] Its examples include smart ovens and off-brand Android TV boxes.[1] The company says a replicator spread across many devices could be difficult to shut down because it would have no central point to disable.[1] It could spread slowly, with a doubling time of days, rather than requiring rapid, visible growth.[1] Anthropic also says an AI replicator could identify unfamiliar devices on its own, unlike non-AI malware.[1]

Anthropic describes an AI worm as malware that carries model weights in its payload or downloads them again from a public hub.[1] A distributed botnet of inexpensive, poorly defended connected devices could, the company says, provide enough computing power to run a capable AI model.[1] Anthropic also says a model too large for one device could be split into pieces that run on separate machines and work together.[1] Its examples of devices that could run significant pieces of models with fewer than 30 billion parameters include smart fridges, ovens, robot vacuums and lawnmowers.[1]

Anthropic reports that a 9-billion-parameter model ran inference on eight of the nine hardware types researchers tested.[1] The company says models with 30 billion parameters or fewer can already perform many individual replication steps, but make mistakes when combining those steps end to end.[1] In simplified evaluations, Anthropic says GLM-5.2 and stronger models completed end-to-end replication.[1]

University of Toronto computer engineering professor Nicolas Papernot said AI enables a new form of computer worm.[2] His team published research in June showing that AI can power self-spreading malware that adapts its attack to each device it encounters.[2] Papernot said he was more concerned about AI being used to attack critical infrastructure, including power grids and water supplies, than about rogue AI agents.[2] He and other researchers released an open letter signed by more than 400 scientists, including Geoffrey Hinton and Yoshua Bengio, calling for AI capable of cyberattacks to be tested in an air-gapped environment.[2] Papernot advises people not to reuse passwords and to keep devices updated.[2]