AI chiefs urge laws to stop bioweapon design by their models
TL;DR:
- The CEOs of Anthropic, OpenAI, Google DeepMind and Microsoft AI have signed a public letter calling for laws requiring DNA and RNA synthesis firms to screen customers and orders.
- Signatories warn AI could erode the “knowledge barriers” that have historically kept biological weapons out of reach.
- A bipartisan US Senate bill would mandate screening, but researchers say screening tools alone are not enough.
A rare show of unity among rival labs: Dario Amodei, Sam Altman, Demis Hassabis and Mustafa Suleyman are among signatories urging the US Congress to require gene-synthesis providers to vet who they sell to and what they print. Organised by the Institute for Progress and the Foundation for American Innovation, the letter concedes there is “a real possibility” that AI will meaningfully lower the barriers to obtaining biological weapons.
Why the alarm now
Synthesising DNA was once painstaking; today dozens of firms “print” custom genetic sequences to order, and not all screen their buyers. The concern is that large language models can both help design dangerous pathogens and tell a user where to source sequences that evade screening. Stanford biosecurity expert David Relman, a signatory, warned that AI tools can advise how to alter an order “so that even those that are screening may be much less able to detect” it. The risks are not hypothetical: in 2017 Canadian researchers reconstituted the extinct horsepox virus using $100,000 of mail-order DNA, and synthesis has only grown cheaper since.
Screening itself is imperfect. Microsoft researchers showed last year that AI protein-design tools could generate dangerous sequences that slipped past existing software. Several signatories, including Safe AI Fund’s Geoff Ralston, argue labs should screen their own users so it is “very difficult, if not impossible” to prompt a model into helping with something imminently dangerous.
Looking forward
For UK readers, the letter is a reminder that frontier-AI risk is increasingly framed as a public-safety question, not just a commercial one — the same theme behind warnings that AI capability is outpacing oversight. With a Senate bill already tabled, the test will be whether voluntary industry screening and statutory rules arrive before the capabilities they are meant to contain. Biosecurity, the signatories suggest, needs multiple points of control — and AI companies will have to own one of them.