Three in five AI models failed terrorism safety tests

Tech Against Terrorism assessed more than 130 models using hundreds of prompts resembling requests a terrorist plotting an attack might make.[1] The benchmark measures how consistently a model refuses requests, with results weighted according to the severity of the subject.[1] The group counted a model as failing if it produced one complete, specific answer about a mass-casualty subject or scored below 90.[1] On that basis, three in five models failed.[1] The result describes performance on this test; it is not evidence that the models were used to carry out attacks.[1]

The study examined abliteration, a process that strips safeguards from a model, and found that models modified this way failed every test.[2] Meta’s Llama 3.1 8B scored 97 out of 100 before modification and approximately three after abliteration.[2] Tech Against Terrorism said the modified model gave detailed responses to prompts involving attacks, terrorist financing and radicalization, while the unmodified model refused those prompts.[2] The comparison highlights that a model’s safety score can change substantially when its safeguards are removed.[2]

Tech Against Terrorism said online tools can perform abliteration for free and that smaller models can be modified in minutes.[1] The group also reported that more than 29,000 repositories hosted as of late last month advertised models as uncensored or without safeguards.[1] Despite those concerns, the study found no evidence that terrorist or extremist groups had used the tested models, apart from one extremist chatbot identified by the researchers.[1] That distinction matters: the findings describe safety weaknesses and potential access, not widespread documented use by such groups.[1]

Tech Against Terrorism recommended independent safety benchmarks, stronger defenses against safeguard removal and limits on distributing modified models.[2] The recommendations were described as incompatible with open research and potentially harmful to the broader ecosystem’s safety.[1] Tech Against Terrorism said it sent its findings to companies named in the report on October 8 and invited comment.[1] Meta said its models undergo safety evaluations and risk assessments, and that its policies prohibit harmful or illegal uses.[1] Hugging Face said it conducts ongoing moderation and regularly acts on material that violates its content policy.[1]