Houthi missiles, Russian and Chinese espionage, propaganda. The report on the uses of Claude

Two days after the resignation of Anthropic researcher Jacob Coxon, who had written on X that the artificial intelligence company and its rival, OpenAI, are racing at full speed towards a superintelligence that will cause the extinction of humanity by 2030, the company published, for the first time since the start of the year, a 154-page report listing all the harmful activities committed over the last eight months on its AI-based chatbot, Claude. 

Coxton, who had previously worked for OpenAI, reiterated in his resignation letter certain concerns he had previously raised regarding the possibility that the technology could slip beyond human control and that the systems might compromise critical infrastructure or create “biological weapons capable of causing mass extinction”. In the report published on Thursday, Anthropic states that, amongst the various cyber threats it has faced, it has identified and thwarted several attempts to use its AI model for ‘malicious activities’ that could facilitate the development of biological weapons. There are five case studies in which scientists used the chatbot for biological research; amongst these, one scientist was working on a government funding application to study the mosquito-borne chikungunya virus – the most concerning case, according to the research, as it was reportedly to be carried out at a military research institute. “The misuse of biological materials is one of the most serious risks posed by cutting-edge artificial intelligence models,” the company wrote, adding that “without the necessary safeguards, such capabilities could have catastrophic consequences”, although in these specific cases it was not certain of the intentions. The research could be used to create biological weapons as well as to develop vaccines or treatments for diseases, and Jacob Klein, head of threat intelligence at Anthropic, told the New York Times: “This isn’t some comic-book character saying, ‘Hey, I want to build a biological weapon to kill everyone’: it’s much more complex than that.”

The section on the development of biological weapons is just one part of the Anthropic report, which has come under fire not only from scientists but also from hackers, state-sponsored groups, spyware providers and propagandists. “Sophisticated attacks no longer require equally sophisticated hackers,” the company warns, and indeed, among the actors frequently featured in reports on foreign threats are new ones, such as the Houthis, the Yemeni group funded and supported by the Islamic Republic of Iran. Anthropic never mentions the Houthis by name in its list, but states that it has “identified a cell of hostile actors based in northern Yemen” – largely controlled by the rebel group – “which runs three weapons development programmes”. The pro-Iranian militia is said to have used Claude “in place of human computer engineers to develop the guidance, navigation and control software that manoeuvres and stabilises an unmanned aerial vehicle” for the production of guided rockets, ballistic missiles and hypersonic missiles. The company states that it has blocked many of the group’s requests, but not all, as various tactics were used to circumvent the measures. “We have no evidence that the actors managed to put a functional device into operation; however, they carried out a field test launch of a guided rocket, which appears to have failed: within a few hours, the actors returned to Claude to understand the reason for the failure”.

The Yemeni case is part of a type of threat that is new to the artificial intelligence company, namely the attempt to use Claude to develop software for the design and development of conventional weapons, including firearms, missiles, armed drones, bombs and other munitions: in addition to the rebel militia, the report details three such cases from China and two from Russia. Then there are cyber espionage operations (mostly by the Kremlin) aimed at building swarms of autonomous killer drones by exploiting data from Ukrainian battlefields, as well as pro-Russian propaganda campaigns in the Central African Republic, the Democratic Republic of the Congo, Malaysia, Iran and Bangladesh. A Chinese government programme utilised Claude to track and monitor Uyghurs in Syria and journalists from the Uyghur diaspora, Catholic cardinals across Asia, the leadership of the Presbyterian Church in Taiwan, members of Tibetan Buddhist civil society, the government-in-exile and domestic dissidents. Anthropic defines these actors as “groups suspected of being state-funded, for-profit criminals, commercial spyware providers, state propaganda institutions and politically motivated individuals. We are publishing this work because we believe we have a responsibility to expose the misuse and harmful applications of our services. As models become more sophisticated, the risks will increase, unless AI developers and those campaigning for societal safety take action to make them safer”.