TL;DR
Anthropic said on 15 October that six people from METR, the nonprofit that evaluates frontier models, now work inside the company with laptops, office space and internal tool access roughly equal to its own risk assessment teams. METR will publish its conclusions, and Anthropic says it cannot veto them, only require redactions in narrow categories. SecureBio and Accenture are also evaluating its models, and Hugging Face is among several organisations in early talks.
The announcement follows a commitment Anthropic made last month to stronger third-party assessment, and its signature of the “White House Accord on Super Intelligence”, under which signatories promised to work with outside auditors or evaluators.
How the access works
Staff are encouraged to brief the METR team as they would a colleague working on alignment risk. A few categories stay off limits: customer data, legally privileged material and model architecture details. METR can say publicly that something was redacted and whether it thinks the redaction was fair, provided it does not reveal the content. Other METR researchers, including those on separate Anthropic projects, are walled off from the embedded team.
Anthropic compares the arrangement to resident examiners placed inside the largest banks, while conceding the analogy is imperfect.
Three tiers
The company describes a ladder of access. At the first tier, evaluators run narrowly scoped investigations and need Anthropic’s approval to publish. At the second, they see more and publish without approval, subject to the same redaction rules. METR’s embedded arrangement is the third tier. SecureBio is checking the section of Anthropic’s August Risk Report that deals with chemical and biological weapons, having already published a pilot review of the previous report. Accenture and Faculty are working on a wider programme covering red-teaming, alignment assessments and safeguard testing.
Looking forward
The test is what METR publishes, and whether redaction disputes surface in public. For UK readers, the timing is notable. At the 13 October hearing of the BIST Committee (Business, Innovation, Science and Trade), AI risk researchers told MPs that labs cannot be relied on to regulate their own conduct, and chair Liam Byrne summarised their evidence as describing a “catastrophic” industry safety culture. We covered the inquiry the committee opened afterwards yesterday. In our view, an outside evaluator that sees internal systems and can publish freely is the kind of answer labs will point to in that argument. Anthropic says it wants every frontier developer to embed evaluators voluntarily, but the arrangement remains a company-designed scheme, not a regulatory requirement.