Anthropic has found five cases where people used its AI models for work that could support biological weapons development. The cases include help preparing a proposal for gain-of-function research on a mosquito-borne virus at a military institute and planning experiments aimed at helping bird flu better infect mammals. Anthropic blocked the work without knowing the intent behind it.
That is the core of a new warning from the company. The warning comes with a set of proposals: governments should verify, screen, and audit DNA synthesis providers, and open-weight AI developers should adopt the same standards as frontier labs.
Five Cases of Suspicious Use
Anthropic’s report describes five cases where its models were used in ways that could support biological weapons development. In each case, the company detected the activity without knowing the intent behind it. The company blocked the work.
One case involved gain-of-function research on a mosquito-borne virus at a military institute. Another involved experiments aimed at helping bird flu better infect mammals. The full details of the five cases are not publicly disclosed.
The warning is notable because it comes from a company whose models are widely available. The report describes the cases and the response without attributing the warning to any individual. That makes the warning different from a statement from a government agency or a research institution. It comes directly from the company that builds the models, which puts it in a position to see how those models are actually being used.
The Rand and Harvard Reports
Two reports this year reached similar conclusions. One came from Harvard, the other from RAND. Both concluded independently that AI could broaden the range of actors able to mount a large-scale biological attack, while making existing state programs more capable.
Neither report claims the most dangerous thresholds have been crossed. Neither rules it out either.
The findings echo a growing concern among security experts. The tools of biotechnology are becoming cheaper, faster, and easier to use. AI accelerates that process by automating the design phase.
“There is no need for someone to ask Claude to ‘build a bioweapon.’ They only need to look like scientists who are working on important biological problems.”
The State Department’s Intelligence Report
The State Department’s annual declassified intelligence report has repeatedly concluded that Russia and North Korea have offensive biological weapons programs. The report also raises concerns about China and Iran.
The report is part of a broader pattern. Sergiev Posad-6, a former Soviet bioweapons site outside Moscow, has been expanding. In 2024, The Washington Post documented a major expansion there, including new high-containment buildings.
The site is a reminder that the threat is not hypothetical. It is operational, and it has been growing.
The 1918 Influenza Experiment
A paper published this year in Nature Communications described an experiment where researchers ordered genetic fragments of the 1918 influenza virus. Thirty-six of the 38 DNA synthesis companies contacted sent the sequences.
A skilled person could reconstruct the virus from them.
The experiment shows how far the industry has come. Companies that manufacture DNA fragments operate globally and often do not know what their customers intend to do with the material they receive.
The Regulatory Gap
The U.S. Select Agent Program prohibits intact genetic material capable of producing dangerous pathogens. Shorter, readily assembled fragments remain unregulated.
That gap is the problem. Genetic blueprints created by AI can become actual pathogens within weeks. The failure rate for turning an idea into an actual pathogen is high, but enough actors are trying that some will succeed.
The industry operates without a global standard for verifying customer intent. Companies that sell DNA fragments often do not know what their customers intend to do with the material they receive. That means a customer with malicious intent can order a fragment, receive it, and use it to build something dangerous without anyone at the company raising an alarm.
What OpenAI Is Doing
OpenAI has also put in guardrails. Other AI companies with open-weight models generally release their products with fewer safeguards and less monitoring for how tools are used.
The distinction matters. The comparison is telling. Frontier labs operate under strict protocols. They screen users, monitor activity, and report suspicious behavior. Open-weight models currently do none of that.
The Case for Government Action
The proposals are specific and pointed. The comparison is telling. Frontier labs operate under strict protocols. They screen users, monitor activity, and report suspicious behavior. Open-weight models currently do none of that.
The proposal is also a recognition of a practical problem. AI is lowering the threshold for people with modest means to build something that could kill on a mass scale. The warning does not say the threshold has been crossed. It says the technology is making it possible for more people to get closer to crossing it.
The Road Ahead
The tools are out. The knowledge is widespread. The question is no longer whether AI will be used for biological weapons. It is how quickly governments respond.
The warning is a call to action. The barriers that kept biological weapons rare are eroding. The question now is whether the world can rebuild them before they break entirely.
The answer depends on whether governments act. Verification, screening, and auditing of DNA synthesis providers is a start. Requiring open-weight AI developers to adopt the same standards as frontier labs would close the gap between closed and open systems.
The stakes are high. A biological weapon built with AI-assisted design could cross borders and spread faster than any conventional weapon.
The paper’s position is clear. Governments should impose verification, screening, and auditing on DNA synthesis providers. Open-weight AI developers should adopt the same standards as frontier labs.
Where the paper stands
The paper backs Anthropic’s narrow, targeted disclosures of the five cases and is against any broad new licensing regime or agency expansion that would raise the cost of entry for smaller AI developers. The company has blocked the work in each case without knowing the intent behind it, and its warning is a genuine contribution to the public record. What the paper opposes is the leap from that specific disclosure to a sweeping new system of licensing and regulation that would favor the largest firms.
Anthropic’s report documents real attempts to misuse its models for work that could support biological weapons development. The company blocked the work, and the warning follows from that. But a licensing regime covering open-weight models would likely lock smaller developers out of the market, just as the paper’s position warns against.
The story itself points to the danger: the Rand and Harvard reports show that AI lowers the threshold for actors to mount large-scale biological attacks, and the Nature Communications experiment shows how easily DNA fragments can be ordered and assembled. These are real risks, and the paper supports narrow, targeted measures against direct harm, such as forcing companies to disclose safety failures they hid. But the proposed regime goes beyond that—it would regulate open-weight models as a class, and the paper opposes that approach.
Source material: “Opinion: AI is eroding the barriers that kept biological weapons rare,” STAT.
Get the Notebook.
The day's best stories and every fresh verdict, in plain English, in your inbox by seven. One email a day, no more.