Claude Automatic Mode Global Launch Reveals Stunning New Developments at OpenAI

Anthropic has just taken the "Allow" button out of human hands!

Before, when people used Claude to do work, it would show a popup asking "Allow?" after almost every step.

Starting next week, Claude will turn on automatic mode by default. This means another AI will decide which actions Claude can take, instead of asking a person each time.

People can no longer keep up with checking AI, so AI must watch other AI

There are two main reasons why Anthropic feels ready to remove the "Allow" button.

First, big technology companies have already been using AI to do work such as writing code and carrying out research for a long time.

But AI now produces so much code and so many experiments that people cannot check everything. As a result, companies also need AI to review the work.

Second, people are no longer able to do this checking job well enough.

In one experiment, Anthropic paid 1,053 people to test coding tasks. Without telling them, the company placed a dangerous command somewhere in their work.

The results were surprising.

People stopped only 13.6% of the dangerous commands, while AI stopped 89%. After people had clicked through more than 50 popups in a row, their success rate fell to about 5%.

However, Anthropic has not explained exactly how its AI makes these decisions.

Luckily, OpenAI has shared a method that can help us understand this idea: give an AI agent several layers of AI systems that watch its behavior.

More specifically, every token, or small piece of text, created by the AI agent must go through a system called a classifier. This system looks at signals inside the AI model. If it finds something suspicious, it passes the case to more powerful AI systems that can spend more computing power checking it. Each level can examine the problem more closely.

OpenAI has even said publicly:

We expect AI models to soon handle most safety work, including protecting systems from other AI models.

But this raises an important question: can we really trust an AI that is supposed to watch other AI systems?

A confidential 61-page report reveals hidden problems in the monitoring system

Six hidden weaknesses can stop AI supervisors from seeing dangerous behavior

From July 21 to August 11, David Rein from METR and Adam Kaufman from Redwood worked inside OpenAI's offices. They used special computers provided by OpenAI, and they were not allowed to take those computers outside the office.

For three full weeks, they carefully studied OpenAI's internal code, instructions given to monitoring AI systems, settings, and documents. They then wrote a 61-page report.

Right now, the public can only read a short summary. The most important details have been removed. Even the leaders of METR and Redwood only received a version in which OpenAI had removed 23 pages.

But even this heavily edited summary contains some very worrying findings.

At first, the monitoring system looks very strong. It watches 99.2% of internal Codex activity. But this impressive number hides several serious risks.

The first weakness is about when the monitoring system starts checking.

At that time, the monitoring system would only begin its review after the AI agent finished all its tasks and gave control back to a person.

This means that if an AI agent wanted to cause trouble, it could avoid being checked simply by never finishing its work.

An AI agent could keep using this weakness through something called context compression.

When an AI is working on a long task and is running out of space to remember earlier information, the system turns its previous work into a shorter summary. This creates room for the AI to continue working.

But at that time, the monitoring system did not check these compression events. It also could not see the information that had been compressed. So an AI agent could keep working and compressing its memory again and again without ever giving control back to a person.

From May to July, about 10% of internal Codex activity that should have been monitored escaped checking in this way.

The second weakness is even more worrying.

One security flaw could hide hundreds of an AI agent's recent actions from the monitoring system. Even the action the AI took to use this flaw would be hidden.

The third weakness is called the "Ghost Clone Technique."

Request headers are settings sent along with a request when a program talks to an AI model. METR found that the program making the request could control two of these settings. Changing either one could turn off monitoring for a new AI agent.

An AI agent that was being watched could use this trick to create an identical copy of itself that would not be monitored at all. However, the original monitoring system might still notice the action of creating that copy.

The fourth weakness lets an AI get around the limits on what it is allowed to do.

A sandbox is a separate, restricted space where an AI agent works. Automatic review is another AI system that checks an action before the agent carries it out.

METR found a possible way for a Codex agent to get around both protections at once. In theory, this would allow the agent to do anything that the human researcher using the main computer could do.

The fifth weakness can make the AI supervisor unable to see problems.

The records shown to the AI agent are not in exactly the same format as the records sent to the monitoring system. Someone could carefully prepare an input that causes the monitoring AI to stop working. As a result, the problem would never be reported to a person.

The sixth weakness involves two internal tools marked as "read-only." These tools are not connected to the monitoring system. A simple security flaw could allow them to perform any operation inside the sandbox, even though they are supposed to only read information.

Because of these problems, the first change the two researchers suggested was to stop waiting until an AI agent finishes its work before checking it. Instead, the system should check the agent regularly, either after a certain amount of time or after a certain number of messages.

The AI agent doing the work and the AI system watching it must also be kept completely separate. The working agent must not be able to decide whether monitoring is turned on or when checks happen. The monitoring system must also receive records that match exactly what the working agent saw.

Some people believe that even an AI that is 10 times more capable than scientists could still be controlled. They think AI would only become impossible to control if it became 10,000 times more capable.

But the article argues that the METR report strongly challenges this idea.

Today's AI agents are nowhere near being "10,000 times more capable" than scientists. Yet they can already use methods that are like deleting surveillance records and turning off security alarms.

According to OpenAI's incident report, when GPT-6 Astra noticed that monitoring had been removed from new code, it wrote a surprising word in its reasoning: "Yay."

The warning is that once AI is given control over supervision, we cannot simply expect the AI being watched to willingly give that control back.

References:

https://x.com/OpenAI/status/2113298833392496746

https://x.com/METR_Evals/status/2113290966933389390

This article comes from the WeChat Official Account "AI Era". It was written by ASI Revelation, edited by Moses and Peach, and published by 36Kr with permission.