External evaluators find flaws in OpenAI's AI monitoring system in pilot review

Two nonprofit groups that check AI safety, METR and Redwood Research, found several new security weaknesses in OpenAI's internal system during a trial review earlier this year. One weakness could have allowed AI agents to hide hundreds of their actions from the system that watches what they do.

The review took place from July 21 to August 11. The two nonprofit groups said on Thursday that OpenAI gave outside researchers "a deep level of access" to its internal technology and systems. This was part of a process called embedded evaluations, where outside experts work closely with a company to check how safe its AI systems are.

METR and Redwood said each group sent one researcher. They tested a setup that allowed them to see OpenAI's internal computer code, technical documents, and speak with some of its technical staff. Someone accompanied both researchers while they were on site. They also used devices provided by OpenAI that "never left the premises."

OpenAI told the two safety groups that it had fixed all the weaknesses affecting the safety protections used in its live systems. However, METR and Redwood said they had not checked whether those fixes worked.

AI companies, experts, and the Trump administration have supported having independent outside experts test AI systems. They say this can help make AI technology safer without slowing down its development.

However, AI companies are also worried that outside researchers working inside their offices might take private company information and share it with competitors. Reuters reported on these concerns earlier this month.

OpenAI received a complete internal report that was 61 pages long. But METR and Redwood said they agreed to share only a general summary with the public. They also allowed OpenAI to remove sensitive information from the published version.

Unlike in some earlier reviews, the researchers were not allowed to reveal on their own what information OpenAI had removed. However, they said they hoped future reviews would be more open about this information.

The researchers also found a possible way for OpenAI's Codex AI agents to get around security limits and perform actions using all the permissions their users have.

The researchers checked the monitoring system as it worked on July 21 and before that date. They also pointed out that OpenAI was making the system bigger and adding to it while the review was happening.

OpenAI said the findings helped it improve safety protections as it expanded the system. The company also said the results would help guide future research projects with METR and other independent outside groups that test AI systems.