Frontier AI and Independent Evaluations: Some Inconvenient Questions
Public concern about how to address the serious safety risks posed by advanced AI has reached new heights in recent weeks. Without government rules, independent evaluations have been proposed as a possible solution. On September 12, Anthropic CEO Dario Amodei called for, among other things, “embedded third-party evaluators…to verify adherence to safety practices and commitments, report incidents, and help assess the alignment of not just completed AI models but training pipelines and processes.” This means outside experts would work closely with a company to check whether it follows its safety promises, report problems, and assess both finished AI models and the processes used to train them. OpenAI CEO Sam Altman agreed, saying that “independent evaluators with employee-like access is a great idea, and we will do the same.” SpaceX CEO Elon Musk also expressed general agreement.
On September 29, leaders from six companies—Google, Anthropic, OpenAI, Meta, xAI, and Nvidia—signed the White House Accord on Super Intelligence. This agreement does not create legally binding duties. It includes a promise that the companies will “[p]artner with an independent external auditor or evaluator to carry out independent assessments of whether the controls, monitoring, and detection are operating as intended.” In other words, outside reviewers would check whether the companies' safety measures and systems for spotting problems are working as planned.
On October 15, Anthropic announced an update on its commitment to independent evaluation. The company said it had already begun giving METR access similar to that of an employee. Anthropic also said it was working with SecureBio and Accenture as evaluators and was talking with other groups, including Hugging Face. The SpaceX CEO replied to Anthropic's announcement, saying his company “will probably do the same.”
Anthropic's October 15 announcement identifies four key parts of its outside review process: expertise, independence, enough access to information, and the freedom to publish results. As one of the first companies to take this approach, Anthropic may help set the standard for others. It is worth considering whether its approach will make independent reviews of advanced AI more effective and trustworthy or instead weaken meaningful oversight and risk assessment.
Anthropic deserves credit for publicly explaining how it plans to put outside evaluations into practice and for moving quickly. But this approach raises several questions. Anthropic's CEO has said he wants to use the company's leading position in AI to encourage a “race to the top.” The company has also said it is “learning actively about what kinds of partnership work.” The questions below are intended to help improve its approach to outside reviews before it becomes fixed company policy, a standard across the industry, or part of future regulations.
Real Safety Standards? Which Standards? Set by Whom?
Anthropic's October 15 announcement does not explain the actual safety standards that will be used in its audits. Will each company simply agree on safety tests with its chosen outside reviewer? Should experts from government, universities, and civil society help design these standards? Should the public be able to see the standards before the audits begin?
This major gap is especially troubling given Politico's report on Anthropic's lobbying efforts concerning federal AI regulation. The outlet reported that Anthropic's proposal would let companies create their own safety rules and allow the government to block a model's release or use only if it posed a foreseeable and immediate risk of causing more than $1 billion in damage, or of killing or seriously injuring more than 50 people. The risk would also have to come from creating or releasing a biological weapon, the model escaping its developer's controls, or a cyberattack. At the very least, these safety limits are far too weak for the current situation and may shock many people.
Will independent auditors be expected to use safety limits like these when assessing risks? Will the evaluators have any say in deciding what controls, monitoring, and detection systems should be in place to prevent certain dangers? Or, as the Super Intelligence Accord suggests, will they only check “whether the controls, monitoring, and detection are operating as intended” by the company? Simply checking whether a company is doing what it planned to do could set the bar too low. It could even encourage companies to lower their own standards so that they perform better in audits.
Will evaluators consider only safety risks in the United States, or will they also look at risks in other countries, given that these products are used around the world? If they consider risks abroad, will they have the knowledge and skills needed to understand effects beyond the United States?
There should be broad agreement that basic safety standards must be appropriate to the risks and open to public review. Without adequate government rules—or even agreement across the industry on what those standards should be—we can only guess what each contract between an advanced AI company and an evaluator will require. Without regulation, different and unknown safety standards may be used without public input. It will be difficult for people to trust these evaluations when such basic parts of the audits remain unclear from the start.
Questions About Independence
Former SEC Chair Arthur Levitt once said, “The accounting profession must be like Caesar’s wife. To be suspected is almost as bad as to be convicted.” He was referring to the story of Caesar divorcing one of his wives, Pompeia, because she was suspected of misconduct. Given his position, even the appearance of wrongdoing was unacceptable. Because the stakes are so high in advanced AI evaluations, similarly strict standards will likely be needed to protect the independence of outside auditors. This independence is essential if people are to trust their work.
Anthropic has pointed out the difficult questions about who pays for these audits. The company previously proposed pooling funding or having the government help pay. But because neither option is currently available, Anthropic said it would work with some evaluators that have their own funding and pay other evaluators directly.
Anthropic's October 15 update briefly explains that evaluators “shouldn’t hold equity in the company they’re evaluating, or be financially or operationally dependent on it.” People working directly on an audit, other employees of the evaluating organization, and everyone in its management chain up to its leader should not own shares in the company being reviewed. But what about people in those employees' households? What other conflicts of interest should be considered? What if an evaluator recently worked for the company being reviewed? What if senior leaders at the evaluating organization have other strong ties to the company under review? Is it enough to disclose these connections, or are stronger protections needed? It is worth noting that the evaluators will belong to organizations legally separate from Anthropic, and Anthropic does not choose their boards, executives, or staff. Even so, many possible conflicts of interest remain and need to be addressed.
In fact, any organization carrying out these audits could become “financially or operationally dependent” on the company being reviewed if the audits provide a large share of its income or if donated funding depends on how well it performs. This can create pressure to produce reports that please the company so that the money keeps coming. Concerns also arise if the company being reviewed has other business relationships with the evaluating organization. The evaluator may feel pressure to give a favorable review to protect those relationships. Even the appearance of such pressure could put auditors in the same position as Caesar's wife: their honesty might be doubted, even without proof of wrongdoing. Over time, this could undermine public trust in the entire process.
Some people have argued that “randomized selection of qualified auditors combined with finite periods for observation can go a long way toward addressing [such] concerns.” Would these steps help, and would they be enough? Is it practical to rotate auditors, given the specialized knowledge required, the limited number of qualified evaluation firms, and how quickly AI technology and its risks change? More thought is needed about whether and how a system in which the AI industry regulates itself can guarantee truly independent audits.
Access to Information
Anthropic's October 15 announcement says evaluators should have “[e]nough access to meaningfully evaluate risks. Beyond access to the latest models, evaluators also need visibility into how those models were trained, into how our safeguards function, and more.” It later states: “Outside a small number of clearly defined categories (e.g., data about customers, attorney-client privileged information, or information about our model architectures), Anthropic employees are encouraged to share information with them just as they would with a fellow employee working on alignment risk.” A footnote explains that “[i]nformation about architectures presents a high risk of inadvertent leaks, and its relevance to risk assessment is typically low. When it is highly relevant, we will generally seek to share it, though this will require explicit internal approvals.”
The public also needs clear information about the contracts between the company being reviewed and the organization reviewing it. This would help people understand how much information evaluators can actually see, rather than relying on a sample list of information that may be withheld. For example, the term “model architecture” could mean different things. Is it defined more clearly in the contract between the company and the evaluators? Will different evaluators auditing the same company receive different levels of access for different reviews? Will the same evaluator receive different levels of access at different companies across the advanced AI industry? Or will evaluators insist on a minimum level of access before agreeing to take part? Without clearer information about the important terms in what could become many separate contracts, it will be difficult for the public to trust these audits.
Transparency About Results
Anthropic's October 15 announcement says evaluators will have the “freedom to publish observations and assessments of a model or a process.” Public findings and clear explanations in these reports are essential if oversight of these companies is to be trusted and useful. Anthropic explains that, under its specific contract with METR, this means:
“METR will publish its analysis of our risk levels, incidents, or practices. Anthropic can’t veto publication, but we can require them to redact information if it falls into predefined categories like the ones above [e.g., data about customers, attorney-client privileged information, or information about our model architectures]. (Anthropic ultimately decides what information does and does not fall into these categories, although METR reserves the right to make certain statements about the level of access and information it received.)”
We cannot judge how broad Anthropic's power to remove information from reports is because we have not seen the contracts that govern the process and the announcement provides only a short list of examples. It is also unclear whether METR can publicly explain in general terms whether it believes the removals were justified. News reports appear to have noted that an earlier version of Anthropic's October 15 announcement said METR “retains the right to publicly characterize any redactions and state whether it believes they were justified.” According to footnote 2 in the October 15 announcement, that wording appears to have been changed to “the right to make certain statements about the level of access and information it received.” The new wording concerns a different issue from whether information was removed from the report. However, later parts of the announcement suggest that METR has “similar” publication rights to other evaluators, who are allowed to comment on removals. Overall, the announcement does not make clear whether METR has this right. All evaluators should have it.
Beyond Anthropic's ability to remove information for reasons that are not fully clear, are there specific safeguards to ensure that the company's review of an audit report before publication does not otherwise improperly influence what the report says? News reports suggest that companies and evaluators have had heated disagreements about what information should be made public. These disputes are likely to continue.
There are also important unanswered questions about what happens after an evaluation is finished. If evaluators give a company a failing grade, what happens next? Does the contract require the company to make changes? Or will the company simply face criticism from the public? If there are no regulations, will the government start any kind of investigation based on the evaluators' reports?
What if an evaluator recommends future safety improvements? Must the advanced AI company at least respond to those recommendations publicly, or can they simply disappear among the flood of daily AI news? Will evaluators accept information and concerns from outside groups, such as researchers and nongovernmental organizations, or will they rely only on the expertise within their own organizations?
The questions keep coming. Without a system of government rules—or even careful, open review by academic researchers and civil society to support oversight of advanced AI safety—many important issues remain unclear and troubling. These include the independence of evaluators, the safety standards they use, and other basic questions. Anthropic has no doubt put considerable effort into reaching its current stage of independent evaluation, but it clearly needs to make major improvements, and time is short.
These audits should not be about giving participation trophies to some of the richest and most powerful companies in the world. They must meet the serious challenges we face. Much more needs to be done to build an evaluation system for advanced AI that is meaningful, practical, effective, and transparent. Otherwise, the audits may end up like Caesar's wife at best: pushed aside and surrounded by suspicion. At worst, they may fail at the vital task of preventing the grave risks to humanity that the CEOs of these advanced AI companies themselves have warned about.
FEATURED IMAGE: Dario Amodei, co-founder of Anthropic, takes questions alongside Mark Zuckerberg, CEO of Meta Platforms, and U.S. President Donald Trump as Trump holds a press briefing with AI executives following a meeting on artificial intelligence outside the West Wing of the White House on September 29, 2026, in Washington, DC. (Photo by Kevin Dietsch/Getty Images)