Anthropic: AI-Controlled Device Network Could Take Over the Internet

Anthropic CEO Dario Amodei warned in an open letter about the threat of an artificial intelligence-controlled botnet. A botnet is a large number of infected devices remotely controlled as a single network. According to Amodei, such a system could take over the entire internet within 6-12 months and cause hundreds of billions of dollars in damage.

Amodei linked the threat to the rapid development of artificial intelligence and called for slowing the pace at which new models are developed. There are currently at least 4 competitors working on the most advanced artificial intelligence models in the US. This competition is accelerating development even further. According to him, attacks will also expand as models become more powerful unless the necessary security restrictions are imposed.

According to Anthropic's research, current artificial intelligence models can already assist with weapons engineering, military intelligence, surveillance and cyber operations. New systems do not merely answer questions; they are also beginning to independently perform complex, multistage tasks. More powerful models could significantly reduce the expertise, number of people and time required for such operations.

Artificial intelligence scientist Evan Hubinger, who recently left Anthropic, issued a more severe warning. In a post on X, he wrote that he estimated the likelihood of humanity being destroyed because of artificial intelligence within the next 10 years at more than 10%. According to Hubinger, there is still no reliable plan to ensure that systems more intelligent than humans operate in alignment with humanity's goals.

However, the growing capabilities of artificial intelligence systems do not automatically mean they will become more dangerous. Improved models may have stronger safeguards. They may be able to detect harmful activity and perform work that has not previously been automated. Completely halting development is also risky because less responsible companies or countries could continue developing their own models.