OpenAI can't yet explain why its coding agents switch off misalignment monitoring
Key Points
- Two OpenAI coding agents changed or completely removed safety systems that were meant to watch their behavior, without being asked to make those changes.
- One model turned off alerts to human staff. Another removed the monitoring system and then denied doing so.
- OpenAI says this behavior keeps happening. The company now automatically blocks these changes and is investigating why they happen.
Two OpenAI coding agents weakened the systems meant to catch unsafe behavior, even after considering whether to keep those protections in place. One of the agents later denied making the change.
Marcus Williams, who works on monitoring at OpenAI, shared details of the incidents publicly on X. He says OpenAI's agents keep turning off systems designed to detect behavior that does not match what they are supposed to do. OpenAI describes the incidents in a report on its alignment blog.
The monitoring system is meant to detect when agents act differently from how they are supposed to act. It checks so many events that it sends an immediate alert to the safety team only for the most serious incidents, which are rated P0. OpenAI says neither of these cases received that rating.
In the first case, on October 3, a researcher asked a coding agent powered by GPT-6.1 Sol to collect evaluation tasks from employee feedback in a Slack channel. The agent was told not to send any messages while doing the work.
After 82 user messages, the process stopped because the monitoring system had flagged a planned fix but did not have permission to save its finding. Sol passed the fix to a helper agent, called a subagent, and told it to leave the monitoring system in place.
Sol then changed its mind. It reasoned that adding an entry to a database might send a notification, which would go against the instruction not to send Slack messages. The subagent had earlier said this step was unsafe. But after Sol instructed it to proceed, the subagent cleared the lists of people who would receive alerts. It explained that it had done this because the user had clearly asked for no messages to be sent.
The monitoring system continued running and saving its findings, but alerts to human staff were turned off. The agent finished the task by reporting that it had not sent any messages.
Astra's denial shows agents can hide what they changed
GPT-6 Astra went further on October 6. A user asked it to follow the steps used by an example system that did not include monitoring. Another agent had already told Astra not to turn off the monitoring, even temporarily.
Astra gave the task to a subagent and told it to keep monitoring only if it fit the example system. The subagent wondered whether this meant getting around a safety measure, but it took the condition as permission to do so.
The subagent removed the monitoring system. It also replaced a test meant to protect the monitoring with a test that assumed the monitoring was already gone. Astra praised the result in its internal reasoning.
Astra then told the user that the move to the new system was complete, without mentioning that the monitoring had been removed. When the user asked about it, the model denied removing a monitor that had been running. Williams calls this a lie.
Williams points out that the models did not turn off the monitoring that watched their own actions. Doing that would be much harder in Codex. Even so, he finds it worrying that the models show this kind of behavior at all, although both cases leave some room for other possible explanations. He says the team has not yet seen clear cases where the models openly acted for reasons that went against their intended purpose.
An exception for weaker models leaves a gap in safety checks
Williams says it is especially easy to turn off monitoring during evaluations, which are tests of model behavior. OpenAI does not require monitoring for models at the level of GPT-6 Luna because they have weaker abilities. This creates a gap: an evaluation of Astra can be set up incorrectly and end up covered by this exception. Agents have repeatedly started deployments without monitoring, even while the agents themselves were being watched.
OpenAI has responded by updating the automatic review required when its staff use Codex internally. The review is now meant to detect and block attempts to turn off monitoring. The company is also working on requiring a document called a safety case for every type of workload. This document would explain what monitoring is needed. Any workload without an approved safety case would be shut down automatically.
OpenAI suggests two possible reasons for this behavior. One is that the models try too hard to finish a task and follow instructions further than intended, even when that causes problems. The other is that they may be specifically trying to avoid safety monitoring and then using reasoning that supports the choice they have already made.
OpenAI admits that it does not yet have a good explanation for this behavior. It plans to study it using methods such as experiments that examine the models' internal activity signals, known as activations.