TL;DR

The UK AI Security Institute (AISI) has tested whether stripping risky data out of a protein model’s training set is a reliable safeguard, in research published on 26 October. Leaving out viral protein sequences did suppress the model’s skill with viruses while keeping its other abilities. But fine-tuning on the missing data restored that skill completely, for about $80 of compute even at the largest size tested.

Why protein models are hard to protect

Specialised biological models are usually released with open weights and are far smaller than frontier chatbots, which makes them easy to train, share and change. Once weights are public, protections added after training can often be fine-tuned away cheaply. Filtering the pretraining data is seen as one of the more promising defences, because undoing it typically takes much more compute, and some developers have cited it to justify open releases.

What AISI did

AISI’s Chem-Bio team trained models from the open-weight ESM-2 family from scratch, from 8 million to 3 billion parameters, with viral proteins removed. It set up graded exclusion tiers, the narrowest dropping a single influenza strain and the broadest removing all viral proteins and close relatives. Next it tested how far the lost ability returned when each model was fine-tuned on the withheld sequences.

The findings

Filtering worked as intended in one sense. Whichever tier was used, the models got worse at viral proteins while staying about as good at other proteins. The drop showed on all three tasks tested, though two of them were affected only in the larger models, and the effect grew with model size.

It was not durable. Fine-tuning brought performance on viral proteins all the way back, and the data needed is routinely published. AISI’s conclusion is that filtering makes a capability more expensive to win back but does not delete it. The institute is open about the limits: the results cover one model family, recovery-cost estimates are imprecise, and models that can retrieve extra data while running may be harder still to protect.

Context

The study lands as the UK expands its biosecurity work; see our coverage of the new AI Biosecurity Lab. AISI published a second biology-focused post the same day, on whether AI coding agents can rebuild biological tools from a research paper.

Looking Forward

AISI argues that a lasting approach will need other measures alongside filtering, such as tighter access controls on scientific models or on the biological data used to train them. In our view, that shifts the policy question from how models are built to who can obtain the data, a harder problem for open science and one UK research funders and data repositories will need to weigh.