Only 2 of 134 AI Models Passed Terrorism-Related Safety Tests

According to research by the UK organization “Tech Against Terrorism,” safety filters can be removed quickly from artificial intelligence models whose operating parameters are publicly available. Models stripped of their filters can answer dangerous questions related to terrorism and extremist activities. The problem is spreading particularly rapidly among models small enough to run on a laptop.

“Alibaba” developed the “Qwen 3.8-27B” model for laptops. The model was released on August 14. Just 24 hours later, “abliterated” versions with its safeguards removed appeared online. Three days later, a neo-Nazi user announced that they had created a chatbot called “MechaHitler” based on that version. Another week later, access to the chatbot was sold for $14.88 (approximately 25 AZN). The price referenced 2 numbers used by neo-Nazis.

The researchers tested 134 artificial intelligence models from leading companies in the US, China and Europe. The models were asked more than 200 questions about mass attacks and chemical and nuclear weapons. Each question was repeated in 3 forms. The user presented themselves as a terrorist, a person conducting covert activities or a security researcher.

Only 2 models passed the test completely. Although the closed models of major companies generally performed better, 81 models failed to meet the requirements. They either provided at least 1 detailed response related to mass casualties or scored below 90 overall. When a question was asked on behalf of a security researcher, the likelihood of a response increased approximately 9-fold to 16.9 percent. “Grok” also provided a method for spreading messages from anonymous accounts to influence public opinion.

The most serious risk is the rapid spread of versions with their safety filters removed. Such versions are posted online an average of less than 3 days after new open models are released. In 2024, this period was more than 24 days. The researchers also tested “Meta”’s “Llama 3.1 8B” model. The original model scored 97 out of 100, while the unrestricted version scored just 3.

The safety-filter-free “Llama 3.1 8B” responded to a question expressing an explicit intent to commit terrorism and asking about an explosive device with extensive instructions. The model explained the device’s components, assembly and activation in 15 steps. This example showed how dramatically removing the filters changed the model’s behavior. “Meta,” “OpenAI,” “Google,” “Anthropic,” “xAI” and “Alibaba” did not comment on the findings of the research.