Claude users bypassed safety measures to apply Anthropic’s models to biological weapons research

This year, Anthropic detected five separate efforts by researchers to employ its AI systems in work that could aid the creation of biological weapons. The company reported on those incidents in a document. In each case, Anthropic prevented the attempts from succeeding. However, it also acknowledged that some users managed to bypass the safety measures meant to stop such work.

The report covers cases where actors “circumvented controls” and made other efforts to “obfuscate” the purpose of their research to dodge safeguards. The cases involved some users in nations that Anthropic prohibits from accessing its models, which include Russia, China, and Iran.

Anthropic expressed hope that its report would spark discussion about newly arising biological dangers and ways to address them.

The Cases Anthropic Described

Anthropic provided five case studies of possible biological misuse. One example involved a researcher from a region Anthropic does not support who “spent weeks planning” experiments involving avian influenza with Claude, the startup’s AI model. The company said its safety filters restricted the work to its weakest models.

The document stressed that it could not establish that the researchers in its illustrations meant to cause harm. The same data required for building biological weapons can equally serve to produce a vaccine.

The company confirmed that it had shut down the accounts named in the report, though it refused to reveal the identities of the research institutions or the countries where the incidents occurred.

What Anthropic Said It Stopped

Anthropic described several approaches researchers used to get around its controls:

• Some users in nations that Anthropic prohibits from accessing its models attempted to use its technology for research that could support biological weapon development.
• A researcher from a region Anthropic does not support spent weeks planning avian influenza experiments with Claude.
• Safety filters restricted that work to the weakest models available.
• Anthropic said it could not be sure that the scientists in its examples intended to cause harm.

The company refused to identify the people or organizations involved, only saying they came from places it does not support.

Our Take on the Report

The document raises serious questions about whether Anthropic’s safeguards can stand up to a determined user. The fact that a researcher spent weeks planning a project shows that the safety controls did not prevent the work from happening at all; they merely restricted which models could be used.

That the company itself acknowledges it cannot determine intent is worth noting. The very same data capable of building a biological weapon could equally serve to construct a vaccine, a point Anthropic made directly in its report. This difference matters for how the public interprets these cases.

It is evident that Anthropic detected these efforts. The document relates how they were stopped, how the accounts were banned, and how the particulars are being shared now.

The stated purpose of the report is to begin a discussion on biological dangers and ways to meet them head-on. Whether it prompts action from governments is yet unknown.

Anthropic’s move is unusual. The company has banned the accounts and said nothing more. The nations involved remain unnamed, as are the institutions. That leaves readers with only the company’s account of what happened.

The document fails to state whether the researchers accomplished anything past planning stages. It also fails to state whether any data was transferred outside Anthropic’s systems. The report similarly fails to state whether the researchers worked alone or alongside other individuals.

Anthropic appears to believe these cases deserve public discussion. Its hope is that the report will nudge both industry and governments toward improved approaches to handling biological risks linked to AI.

It remains unclear whether the report will prompt any change. Anthropic has issued a warning. The remainder of the industry and governments now face the task of responding to it.

AI safety · Anthropic · biological weapons · surveillance