TL;DR

The AI evaluation group METR published an assessment on 5 November finding that Anthropic’s models are now gaining capability on clearly defined tasks about twice as fast as in the 19 months to October 2025. METR attributes most of the speed-up to Anthropic’s own heavy use of AI in research. It also says Anthropic redacted the majority of the evidence METR found most informative.

What METR found

METR staff worked inside Anthropic from 12 August to 29 October and were given internal figures on pretraining, reinforcement learning, compute use and benchmarks, plus time with staff. Their central measure was AECI, Anthropic’s in-house take on Epoch’s capabilities index. Since October 2025 those scores have risen at an estimated 1.9 times the earlier rate, with a 95% confidence interval of 1.5 to 2.8 times. In plain terms, roughly two years of past progress was packed into one.

METR says it is confident the jump did not come from faster growth in training compute. What employees told METR fits a picture in which AI help, rather than AI working alone, has doubled the pace, though METR heard there are signs autonomous contributions will soon grow. It saw no evidence of significant bottlenecks in algorithms, compute or data over the next 12 months, though it says its view changed several times and it is not confident in that assessment.

METR treats its central estimate as highly uncertain. It would not be surprised by anything from 1.3 to 3 times. Its estimate could be too high if benchmark gains overstate real-world impact, or if the historical rate it measures against was unusually slow.

The redaction problem

Under the agreement, Anthropic could redact non-public information, and it removed most of what METR considered the strongest evidence, describing this as protecting information that could help rival developers. METR says the public must therefore rely on its judgement far more than is ideal. METR took no payment; Anthropic supplied laptops and token credits.

Why it matters

The risk METR is examining is that AI accelerating AI research could bring sudden jumps in dangerous capability. This follows METR’s embedding of evaluators at Anthropic and AISI’s report on alignment failures in Mythos 5. In our view, the redactions are the lesson for UK policy. An evaluation-led oversight model depends on evidence reaching outsiders. Here a lab with unprecedented openness still withheld the material that mattered most.

Looking Forward

METR warns that without observed bottlenecks, further progress could speed up acceleration itself. Its full report runs to around 100 pages. The next test is whether other labs grant similar access, and whether regulators get to see what the public could not.