Call for global AI rules after Claude bioweapon research

The European Commissioner with responsibility for tech regulation has said that global rules on artificial intelligence (AI) are "really needed". Commission Executive Vice-President for Tech Sovereignty, Security and Democracy Henna Virkkunen was responding to questions about AI company Anthropic which said it has disrupted attempts to use its models to develop biological weapons. Anthropic also said it has shut down multiple cases of state-sponsored surveillance operations using its technology in China, Iran and west Africa. Speaking to reporters in Dublin, Ms Virkkunen said in Europe there is the AI Act. "According to our AI Act, if those kinds of very capable models are coming to the market the service provider has to, all the time, assess and mitigate the risks and also, when the models are on the market they have to monitor the situation all the time," Ms Virkkunen said. She said we know that this is not the case globally. "It's very important that we continue international co-operation because I see that global rules are really needed here. "With AI there is, of course, huge potential and so many positive things, but there is also risk, and we have to be aware of that, cyber risks and bio risks, we are aware that these are very serious topics " she added. In a report titled 'Detecting and countering misuse of AI', the San Francisco-based AI company Anthropic listed five case studies of potential biological misuse of its models which it says "is one of the most serious risks of frontier AI models". The scientific research in those case studies involved work around the mosquito-born chikungunya virus, a "highly-pathogenic" strain of the bird flu, a family of viruses that include smallpox and mpox, venoms and other toxins. In the examples identified, Anthropic said the actors circumvented controls imposed to prevent users from unsupported regions accessing their models, while the actors also "engaged in other efforts to obfuscate the purpose of their research to evade our safeguards". "When we detected and investigated these cases, we banned the users' accounts and incorporated our investigative findings into our frontier model safeguards, enforcement, and threat intelligence processes to better prevent, detect, and disrupt these activities in the future," the report said. Anthropic said the intent of the actors was not clear-cut and the AI lab did not identify them. "The individuals implicated in these case studies are working scientists. We do not assert that they intended harm, and identifying them or their labs could expose them to harm," the report said. Anthropic said its aim in sharing the examples is to "spark conversation within the AI industry, and with governments, about emerging biological risks and how best to counter them". Anthropic says China, Iran used its AI to aid spying Also in the report, Anthropic said it shut down multiple cases of state-sponsored surveillance operations that used its AI models, and warned that governments and state-aligned actors are increasingly using AI to spy on ethnic minorities and dissidents. The incidents were found and disrupted between January and July, and they originated in China, Iran and west Africa, Anthropic said in a report about misuse of its models. These surveillance campaigns "targeted the same diaspora and dissident communities these regimes have historically targeted". "These include pro-democracy figures in Hong Kong, Tibetan and Falun Gong communities across Asia, and Iranian minority communities and opponents of the Iranian regime abroad," the report said. In one case, Iranian actors developed a way to identify people through their social media accounts; in another, a contractor working for Malian national security authorities used Claude "to design the underlying software that enabled the intelligence gathering". "AI is now being used in place of an engineering workforce," the report said. Anthropic also disrupted efforts by users to design weapons and create dating scams. The San Francisco-based AI lab also accused Chinese developers of deceptively using Claude to produce responses to user queries while simultaneously "distilling" that data to improve their own models. Distilling is an industry term that refers to a process where one AI model is used to train another one. Chinese developers have previously been accused of using that technique without permission, essentially stealing resources from American labs such as Anthropic and OpenAI. Now Anthropic alleges that China's Moonshot and Deepseek have also secretly used Claude to generate answers given to its users. "In one instance, over a 10-day period, Moonshot relayed almost 300,000 customer requests to Anthropic" through a "network of 5,380 fraudulent accounts, most of which appeared to be located in Singapore and Japan," the report said. Some of that data contained sensitive user information, potentially in violation of privacy agreements. "We do not know if Moonshot notified their customers that their requests were being rerouted to Anthropic and exposed to a third party," the report said. Deepseek used similar techniques. Earlier this week, an artificial intelligence researcher who left OpenAI to join Anthropic has decided to leave the industry, accusing both US companies of "gambling with our lives" in the race to develop AI models capable of self-improvement. By Brian O'Donovan Additional reporting AFP