METR, the nonprofit evaluator, spent eleven weeks on a high-access review of Anthropic and concluded that the company is probably raising its models’ capabilities on well-specified tasks at about double the rate it managed between March 2024 and October 2025, mostly because its researchers now use AI to do the research. Its public report, published on 5 November, also says Anthropic redacted “a majority of the evidence we found most informative” for those conclusions, so the public “has to rely on METR’s judgment much more than would be ideal”. Our news report covers the findings. This piece looks at the second point, because it is the one that tests how the UK oversees frontier AI.
Strategic Insight: METR’s outside measurement of AI speeding up AI development arrived with its conclusions public and most of its working private. That is the shape of evaluation by voluntary agreement: the evaluator sees the evidence, the company controls what is released, and everyone else gets a verdict.
Why does an R&D speed-up matter to oversight at all?
METR’s reason for measuring this is stated in its executive summary: if AI systems began to accelerate AI development significantly, extreme and risky capabilities could emerge suddenly. Anthropic’s own Responsible Scaling Policy lists faster AI R&D among its key risk indicators. The UK’s AI Security Institute (AISI) names the same concern. In its October priorities for frontier developers, AISI warns that “as AI contributes more to its own development, progress could accelerate, meaning these problems may arise faster and with less direct human oversight involved”.
So the question METR set out to answer is one the UK’s own safety institute wants answered: AISI lists “sharing information on the speed of AI development” among its eight priorities, including when such metrics should be shared “with governments and the public”.
The real story is the access terms, not the multiplier
Read the process sections of the report and a picture of evaluation-led oversight in practice emerges. METR asked for information and Anthropic could approve or refuse each request on its own say. METR says the two sides often compromised on data that was “informative, not extremely sensitive, and easy to share”. It worked on Anthropic devices. It took no payment, though Anthropic supplied free tokens. And Anthropic held unilateral redaction rights over every piece of non-public information in the public report.
By METR’s account, the access was generous: Anthropic shared more than the project had first scoped and surpassed METR’s expectations. The constraint was not what METR saw. It was what METR could say.
| What the report discloses | Detail | Why it matters for oversight |
|---|---|---|
| Central estimate | About 2x on well-specified tasks; METR would not be surprised by 1.3x or 3x | A wide band, with the evidence that narrowed it largely withheld |
| Benchmark acceleration | 1.9x since October 2025 [95% CI 1.5–2.8x] | The only evidence type in METR’s own table both shared and described in full |
| Share attributed to AI | METR’s guess is that AI accounts for most, perhaps about 80%, of the current acceleration | METR’s more detailed estimates by pipeline stage were redacted |
| Evidence METR received, wholly or in part, but could not describe publicly | Key intermediate measures, algorithmic progress estimates, key R&D inputs, bottlenecks and forecasts | The evidence on causes, inputs and forecasts is not public |
| Size of METR team | 2.5 full-time equivalents | A small team for a company-wide question |
Critical Context: METR itself rates the exercise against the AI Evaluator Forum’s AEF-1 checklist. It marks as not fulfilled the requirement to share enough method for independent review (“not possible due to redaction requirements”) and the requirement that the provider lacked the authority to strip out worrying results, noting that Anthropic “had unilateral redaction rights”.
What does the redaction record actually show?
The report is careful not to accuse. METR says it thinks it “extremely unlikely” Anthropic was trying to mislead it, and it accepts that the redactions were confined to confidential material that could help rival developers, adding that “most of the redacted information was very high-level”. What it declines to do is judge whether the redactions were justified, saying it is not well placed to balance a company’s commercial interest in confidentiality against the public interest in understanding AI risk.
Three details in the timeline and scope are worth reading closely.
- The redaction rules tightened at the end. METR delivered its roughly 100-page full report on 22 October. Anthropic issued redaction guidance on 25 October, then tightened it between 27 and 29 October, and the two sides agreed an extension of about five days so METR could rework the public version.
- A scope item was dropped. The original contract included verifying how Anthropic splits compute between capabilities and safety. Anthropic changed the scope midway to remove it, before METR reached any substantive conclusion on compute, and said it would revisit the question later.
- The unredacted findings went to Anthropic. The deliverables list an unredacted full report delivered to Anthropic, a lightly redacted internal version for METR staff that was not produced “for timing reasons”, and the public report. The public report does not mention governments at all.
Reality Check: None of this suggests bad faith. It shows where control sat. For every piece of non-public evidence, the release decision rested with the company being assessed.
How this compares with Anthropic’s own published model
On 15 October, Anthropic set out three tiers of evaluator access. Second-tier evaluators may publish without Anthropic’s sign-off, and “can publish their views on whether our redactions were justified”. The third tier, which Anthropic described for a separate METR team with employee-like access, carries similar publication rights. In that update Anthropic said it “can’t veto publication” but can require redactions in predefined categories, and that it “ultimately decides” what falls into them.
METR is explicit that the R&D assessment is not that embedded project; it began in mid-August and did not involve employee-like access. So the 5 November report is not a test of the third tier. It is a picture of what a well-resourced evaluator could publish under this engagement’s terms, and it gives outsiders a baseline for judging the reports that follow.
Who is affected, and how?
The redaction pattern lands differently on each group that relies on evaluations.
| Group | What changes for them | What is still unknown |
|---|---|---|
| AISI and DSIT | An outside view now exists on a risk AISI has said it wants tracked, but its detail went to Anthropic | Whether the UK government has seen, or asked for, METR’s full report |
| Parliament | MPs, peers and committees calling for AI regulation gain an example of what voluntary access can and cannot surface | Whether ministers will treat it as evidence in that debate |
| Other frontier labs | Anthropic says every frontier company should embed evaluators; this report shows one set of terms in use | Whether others offer comparable access, or comparable redaction rights |
| UK organisations buying frontier models | Assurance about the pace of model change rests on evaluators’ judgement rather than published evidence | How procurement or risk teams should weigh a verdict they cannot inspect |
The UK’s exposure is particular. AISI has no statutory powers and does not regulate, as PoliticsHome reported in October, and it cannot threaten to stop a firm releasing a model in Britain without prior testing. AISI’s own October paper says it is “not seeking to mandate requirements” and will work with developers “to develop voluntary practices”. Our analysis of the Lords vote on a review of AISI’s powers covers the parliamentary side.
That makes the METR report something close to a preview. A voluntary, evaluation-led system produces exactly this artefact: an expert outside verdict, with access the outsider praises and publication terms the company controls.
Strategic Reality: The UK and Anthropic already draw on the same evaluator. In its October account of the Mythos 5 incident, AISI said it is “sharing transcripts and relevant access with METR” for an independent review. Anthropic, for its part, says METR has reviewed several of its Risk Reports over the past year.
The open challenges
The pressure runs one way
METR is frank about informal pressure. It writes that it considered how its report would affect future access, and that being too critical of Anthropic over access limits or redactions “might make other companies or Anthropic itself less willing to work with us in the future”. It also says some Anthropic feedback on its separate Risk Report Review felt “overly forceful”, whilst concluding that the final outputs were not significantly distorted.
That is the same dynamic one campaigner describes for AISI. Andrea Miotti of Control AI told PoliticsHome that AISI is “disincentivised from raising the alarm because, if they did, companies wouldn’t give them access to systems”. Whether or not that is fair to AISI, METR has now described the incentive in its own words, from the evaluator’s side.
Self-reports and AI-assisted analysis
Some of the evidence METR could publish is soft by its own account. It calls staff Slack polls on uplift, which averaged about 12x by October 2026 for Anthropic’s strongest internal model at the time, “only very loose evidence”. It also notes that Claude had a large hand in most of the data and analysis it received and in most of its own analysis, though it found no sign of systematic bias. An evaluation of AI-accelerated research that is itself AI-accelerated adds a further layer to the assurance problem.
Benchmarks are running out
The newest internal Anthropic model in the report, which METR calls Model 4, scored above 50% on every benchmark in its index, and METR says the lack of hard benchmarks makes it “somewhat nervous” about measuring progress now and later. AISI made a similar point in October, noting that benchmarks tend to saturate soon after they are built. If the public numbers depend on benchmarks and the deeper numbers are redacted, the visible evidence could weaken over time.
⚠️ Warning: METR identified no convincing obstacle to fast progress continuing over the next 12 months, and warns there is “a significant risk that further capabilities progress could lead to even faster acceleration”. It also says its view of bottlenecks changed several times and it is not confident in it. A risk that may compound is hard to oversee through periodic, redacted reports.
What to watch
Whether the UK asks for the full report. AISI’s priorities ask what independent evaluators should report “to companies, governments and the public”. A statement from AISI or DSIT that it has received METR’s unredacted findings, or has sought them, would show the UK treating this as evidence for government rather than as a company disclosure.
The next METR report under the embedded terms. If the six-person team with employee-like access publishes, look for a section stating whether METR judges Anthropic’s redactions justified. Anthropic’s own tier descriptions allow for that; this report did not offer one.
The deferred compute check. Anthropic said it would revisit verification of its compute split between capabilities and safety. A completed, published verification would show the scope cut was a delay. Continued silence would leave the question with the company.
The UK’s Centre for AI Measurement. PoliticsHome reports it is due to launch in November with the aim of establishing a trustworthy way to test and evaluate AI. Whether its remit says anything about evaluators’ access and publication rights will indicate whether the UK intends to set those terms or keep accepting the ones labs offer.
The METR report is a credit to both organisations in one sense: a lab opened its books and an evaluator said plainly what it could not show. But it also answers a practical question for UK policy. In this engagement, most of the evidence METR found most informative went no further than the evaluator and the company, and the public got a conclusion given with high uncertainty. If the UK wants to see more than that, it will have to ask for it on its own terms.
Source: Independent Assessment of AI R&D Acceleration at Anthropic in October 2026 (METR, by Tom Cunningham, Nate Rush, Parker Whitfill, Hjalmar Wijk, 2026). Additional sources: AI Security Institute, Towards technical best practice for frontier AI development (2026) and Investigating alignment failures in AISI’s incident (2026); Anthropic, An update on independent evaluators (2026); PoliticsHome, by Ros Taylor (2026).
This strategic analysis was written by Resultsense, a UK-focused AI news and analysis publication. We will be watching whether the UK government asks to see what the public could not. Read more analysis at Insights, or get in touch.