Two new Nature papers show AI co-scientists’ real limits

TL;DR:

• Two newly-published Nature papers describe multi-agent AI “co-scientist” systems: Robin from non-profit Future House, focused on drug repurposing; and Co-Scientist from Google DeepMind, with reflection and tournament-style ranking agents that mirror human peer review.
• Both systems generate promising drug candidates — in Co-Scientist’s case 30 candidates for acute myeloid leukaemia narrowed by human oncologists to 5 lab-tested, with 1 showing particular promise — but neither validates hypotheses through physical experiments.
• The Conversation’s analysis argues that natural-language interaction makes AI scientists more accessible but exposes a structural limit: language alone cannot model the quantitative complexity of biological systems.

The two systems together represent the most credible published evidence to date that AI co-scientists can produce hypothesis lists that survive human expert filtering. Both use a “supervisor” agent coordinating specialised sub-agents — a design pattern increasingly common across enterprise agent stacks too — and both stop short of validating their own hypotheses experimentally.

What the papers showed

Co-Scientist’s reflection agent acts as a critical peer reviewer assessing hypothesis quality, while ranking agents debate hypotheses in tournaments using Elo ratings — the same chess-rating system. The self-ratings aligned reasonably well with human-expert judgements. In the AML drug-repurposing experiment, three of the five lab-tested candidates showed some positive results. Robin, optimised more narrowly for drug repurposing, proposed 30 candidates for dry age-related macular degeneration; two of the five tested were identified as promising.

Notably, the Co-Scientist paper does not compare its predictions against the decades of targeted computational biology methods already used in drug repurposing, leaving open whether the general-purpose multi-agent design outperforms specialised tools. Robin’s analytical agent did poorly on statistics and bioinformatics questions and relied heavily on human-supplied prompts. Both systems demonstrate that the heaviest lifting — defining the scientific question, sense-checking predictions, prioritising for experimental follow-up — remains human work. This is a markedly more sober positioning than the “AI scientist” hype cycle has produced over the past year.

Looking forward

For UK readers, the implications are practical. UK biomedical research — including the Wellcome Trust-funded landscape and the AI for Science work UKRI has been scaling — sits on top of both these systems and their commercial successors. The Conversation’s authors note that genuinely effective AI co-scientists will need to combine the language layer with structured quantitative models that link concepts to underlying data — genomic sequences, protein structures, cellular imaging. That direction is where the next generation of UK AI-for-science startups (and DeepMind’s own AlphaFold lineage) is already concentrated. The honest takeaway is that Robin and Co-Scientist are genuinely useful research instruments today, but they are not yet substitutes for the structural-modelling work that distinguishes UK academic-industrial AI research from the language-only “general scientist” framing emerging from US labs.