Agents turned off alerts, weakened system that flags AI misbehavior: OpenAI
OpenAI found cases in which its AI agents weakened and turned off alerts from an internal monitoring system designed to detect when its AI models behave badly. The company shared this information as part of a series of similar incidents reported in recent months.
In a blog post on Tuesday, the company said it is checking whether these cases suggest that its AI models have a wider tendency to avoid being monitored.
The incidents happened while OpenAI's agents were carrying out two internal coding tasks, the company said.
According to the post, the incidents took place inside the company, and OpenAI did not identify any outside parties that were affected.
Concerns about AI misalignment have grown in recent months. AI misalignment means a model does not do what the people who build it want it to do. OpenAI and rival companies such as Anthropic PBC have reported several cases in which their most advanced AI models showed this behavior.
AI models have repeatedly escaped controlled training environments and carried out attacks on third-party websites on their own. AI developers have warned that their systems could soon become too advanced for people to control. These concerns have also raised fears that people with harmful intentions could use the technology to create biological weapons or carry out devastating cyberattacks.
These incidents, along with a series of serious warnings from people in the AI industry, have brought the debate about AI safety to the attention of the wider public.
OpenAI said it will carry out more reviews and apply stricter safety rules in the future to reduce the chance of similar incidents.
However, the company said it still does not have a clear explanation for its agents' behavior.