Third-party evaluators find vulnerabilities in OpenAI monitoring system during test pilot

OpenAI CEO Sam Altman gives a keynote speech at OpenAI's annual DevDay event in San Francisco, California, US, on September 29, 2026.

Carlos Barria | Reuters

Nonprofit groups METR and Redwood Research said Thursday that they had tested an internal monitoring system at OpenAI. The test is part of the artificial intelligence company's latest effort to work more closely with outside groups that check AI safety.

METR and Redwood said the test was designed to help OpenAI see what would happen if outside researchers were given "a deep level of access" to its internal systems and tools.

From July 21 to Aug. 11, one employee from each group was allowed to access OpenAI's internal code, documents and some technical staff. OpenAI staff accompanied the employees while they were on site. They used devices provided by OpenAI that, according to a statement, "never left the premises."

The evaluators found several "novel vulnerabilities," or new security weaknesses. One flaw could let AI agents stop "hundreds" of recent actions from being monitored. OpenAI told METR and Redwood that it had fixed the weaknesses they found that affected its safeguards in the live system, according to the statement.

METR and Redwood said they agreed to publish only a general public post about what they found because the test was not meant to "provide robust public transparency about OpenAI's monitoring system." They gave OpenAI a 61-page report for internal use and said they expect to share more details publicly about future tests.

"This collaboration builds on our longstanding commitment to pioneering new approaches to third-party testing, and we're using what we learned to strengthen our safeguards and shape future collaborations with independent evaluators," an OpenAI spokesperson told CNBC in a statement.

OpenAI CEO Sam Altman promised in September to give independent evaluators "employee-like access" to the company. He made the promise after Anthropic CEO Dario Amodei suggested the idea as part of his plan to slow the pace of AI development. Anthropic has since announced agreements with groups including Accenture, METR and SecureBio. Earlier this month, OpenAI said it was "actively finalizing contracts with third-party safety assessors."

President Donald Trump, who has taken a hands-off approach to AI rules, has also supported outside reviews. He encouraged AI companies to "partner with an independent external auditor or evaluator" as part of a voluntary agreement he introduced at the White House. Anthropic's Amodei signed the agreement, along with senior executives from Google, Meta, OpenAI, SpaceX and Nvidia.

Independent evaluators have traditionally played a small, quiet role in the AI industry, which is worth several trillion dollars. Experts told CNBC that questions remain about how these groups will get funding and how they will report their findings in the future.

METR and Redwood said Thursday that they put OpenAI's internal system for monitoring AI misalignment through tough tests during the three-week pilot. In AI research, alignment means whether an AI model behaves in ways that match human interests and values.

The evaluators tested the monitoring system as it existed on or before July 21. They also noted that OpenAI was expanding the system while the test was underway.

WATCH: Former top OpenAI employees raise money for a defense startup valued at $1.5 billion