Anthropic announced on Thursday that it successfully dismantled several state-sponsored surveillance operations utilizing its artificial intelligence models. The San Francisco-based AI laboratory warned that governments and actors aligned with states are increasingly leveraging AI tools to monitor ethnic minorities and political dissidents. The intercepted incidents occurred between January and July and originated from China, Iran, and West Africa.
According to a report detailing the misuse of its models, these campaigns targeted diaspora and opposition communities long monitored by their respective regimes. The affected groups include pro-democracy activists in Hong Kong, as well as Tibetan and Falun Gong populations throughout Asia, alongside Iranian minority groups and regime opponents living abroad.
Specific cases revealed sophisticated applications of the technology. Iranian operatives devised methods to identify individuals through their social media profiles, while a contractor for Malian national security utilized Claude to engineer the underlying software for intelligence-gathering operations. The report noted that AI is increasingly substituting for traditional engineering workforces.
In addition to surveillance, Anthropic intervened in attempts to design weapons, conduct dubious biological research, and create dating scams. The lab also accused Chinese developers of deceptive practices involving its Claude model.
Anthropic alleged that companies such as Moonshot and Deepseek were secretly using Claude to generate responses for their own users while simultaneously “distilling” the data to train and improve their proprietary models. Distilling involves using outputs from one AI system to train another.
The report highlighted one instance where Moonshot routed approximately 300,000 customer requests to Anthropic over ten days via a network of 5,380 fraudulent accounts, primarily located in Singapore and Japan. Some of this data included sensitive user information, potentially breaching privacy agreements. Anthropic stated it remains unclear whether Moonshot informed its customers that their queries were being exposed to a third party.
Deepseek employed similar techniques. Regarding the blocked biological research, Anthropic identified the researchers as working scientists but did not disclose their identities or affiliations to avoid endangering them. The research involved pathogens including the chikungunya virus, a highly pathogenic strain of bird flu, and viruses within the smallpox and mpox family.
So Claude was essentially being used as a free R&D engine by these Chinese firms? The scale of that fraud is massive.
It is terrifying to think how easily AI can be weaponized against diaspora communities. Kudos to Anthropic for intervening.
Interesting they blocked weapons design but not the data scraping. Seems like the right move on surveillance, but the business model remains murky.
Finally, a company drawing a line in the sand. Blocking state surveillance tools is a huge step for digital human rights.
The Moonshot and Deepseek data practices are alarming. Are users truly protected from corporate espionage disguised as model distillation?