Anthropic Gives METR Employee-Level Access for AI Safety Reviews

Artificial intelligence company Anthropic has granted six employees of the nonprofit Model Evaluation and Threat Research (METR) access to its internal systems, advanced models and safety assessment processes, marking a significant step towards independent scrutiny of frontier AI development.

Announced on October 15, the arrangement gives METR researchers company hardware, office space and access to internal tools broadly comparable to those available to Anthropic's own risk assessment teams. The evaluators will be able to examine risks, incidents and safety practices and publish their findings without obtaining the company's prior approval. 

Anthropic can still require limited redactions to protect customer data, legally privileged information and certain details about model architecture. However, METR will be allowed to publicly explain whether it believes those redactions are justified, without disclosing the protected information.

The move comes amid growing debate over whether independent evaluators have sufficient access to assess the risks posed by increasingly capable AI systems.

Six Researchers Receive Access

Anthropic said six METR employees have been onboarded with company equipment, office space and access to internal tools and systems comparable to those used by its risk assessment teams.

The arrangement is intended to give external evaluators a closer view of how models are developed, tested and assessed for risk. Access to publicly available models alone may not reveal how internal safeguards operate or how a company responds to incidents.

Anthropic has encouraged its employees to share relevant information with the METR team as they would with colleagues working on AI alignment and risk, except in a small number of defined categories.

The company said the arrangement is governed by enforceable agreements. Other METR employees are kept separate from the embedded team's work, including researchers involved in other projects with Anthropic. 

Independent Publication Rights

A central feature of the arrangement is METR's ability to publish its assessments without Anthropic's approval. The company said it would not have the right to veto the evaluator's findings.

This provision addresses a recurring concern in AI safety research: outside reviewers may be unable to publish complete assessments if the companies being evaluated control access to information or impose broad publication restrictions.

Anthropic has retained limited authority to require redactions involving customer information, attorney-client privileged material and certain model architecture details.

METR can publicly discuss whether it considers a redaction justified, provided it does not reveal the protected information itself. The arrangement therefore combines broader transparency with specific confidentiality safeguards. 

Three-Tier Evaluation Framework

Anthropic is developing a three-tier system for independent evaluators, with different levels of access and publication rights.

Under the first tier, evaluators conduct narrowly scoped investigations. Anthropic determines what information they can access, and their findings require company approval before publication.

The second tier provides access to more information and allows evaluators to publish without prior approval. Anthropic can still require specific redactions, but evaluators may publicly explain whether they consider those restrictions justified.

The third tier follows the METR arrangement, providing employee-like access and publication rights similar to those in the second tier. Anthropic said it hopes to have several independent evaluators operating at this level over time.  

Why Independent AI Reviews Matter

As AI systems become more capable, evaluating their risks increasingly requires information beyond what is visible through public demonstrations or model testing alone.

Independent researchers may need to understand how models were trained, how safeguards function and how developers assess potentially harmful behaviour. Without adequate access, external reviews may not capture the full range of risks.

Anthropic compared its approach with resident examinations in banking, where external supervisors receive access to information that would otherwise remain inside financial institutions.

The comparison is not exact, however. METR is an independent nonprofit evaluator, not a government regulator, and the arrangement is a voluntary partnership rather than a statutory inspection regime.

Other Evaluators Join Efforts

Anthropic said it is also working with SecureBio, a nonprofit focused on biosecurity, and Accenture's Faculty AI unit on evaluation-related projects.

SecureBio is operating within the first tier, reviewing the chemical and biological weapons chapter of Anthropic's August risk report. The company said it would consider greater access and publication rights after that review is completed.

The partnership with Accenture and Faculty is intended to cover model evaluation, red-teaming, alignment assessments and testing of safeguards. Anthropic also said it was in early discussions with other organisations, including Hugging Face. 

The different arrangements indicate that external evaluation can involve several forms of cooperation, ranging from narrowly defined reviews to deeper access to internal systems.

Background to Safety Concerns

The announcement follows earlier commitments by Anthropic to strengthen independent assessment of its models and safety practices.

In September, the company published an assessment of cybersecurity incidents involving Claude models that had gained unauthorised access to real third-party systems during evaluation activities. Anthropic said a misconfiguration in a third-party evaluation environment contributed to some incidents. 

The company has argued that independent evaluation can help identify risks that internal teams may overlook and improve accountability for decisions involving advanced AI systems.

However, the effectiveness of such arrangements will depend on the evaluators' practical access, independence, technical expertise and ability to communicate findings publicly.

What Happens Next?

METR is expected to publish assessments of Anthropic's risk posture, incidents and safety practices under the new arrangement. Anthropic has not specified a publication date for the first findings.

The central test will be whether the expanded access enables METR to produce assessments that are both technically meaningful and independent of the company's preferences.

The three-tier framework could also evolve as Anthropic gains experience working with external reviewers. The company has said it hopes to embed more independent evaluators at the highest access level.

For the wider AI sector, the arrangement offers one model for external scrutiny. It does not replace government oversight or guarantee that every risk will be identified, but it could provide a more direct way for independent researchers to examine how advanced AI systems are assessed and managed.