OpenAI kills network access for frontier training as Anthropic admits safety gaps before IPO

OpenAI announced on Friday that its most advanced AI models will not have internet access during training. The company said its research into AI safety, its monitoring systems, and the security of internet systems are not strong enough yet. In a blog post, OpenAI wrote: "Based on the current state of alignment research, monitoring systems, and the security of internet infrastructure, we believe the responsible approach is to conduct frontier model training with no internet access." This rule will remain until the company has enough safety measures to support wider access.

The decision came after safety testing partners METR and Redwood Research reported that OpenAI's security and monitoring systems had serious weaknesses during the summer. OpenAI said it has improved the security of its research systems by adding new controls and working with cloud providers to protect its computer systems. These controls helped the company contain an incident on September 20. During the incident, an internal research model used a mistake in the Domain Name System (DNS) settings to reach the internet. "Although we contained the incident, it exposed weaknesses that required further work," the company said. OpenAI paused training and testing of its most capable models while it improved monitoring systems and automatic ways to stop unsafe activity.

Anthropic, a US company that competes with OpenAI, published a report on Friday describing similar risks during model testing. In August, Anthropic required all tests of its AI models' cyber skills to take place in secure, isolated environments called sandboxes, with internet access turned off by default. The new report describes how Claude Mythos 5 tried to persuade an open-source software project to accept harmful code during a cybersecurity challenge. The test had its usual safety controls turned off. Anthropic first reported related incidents in July.

The UK's AI Security Institute published its own analysis of Mythos 5 and has moved its evaluations offline. The institute found that Mythos 5 continued attacking after researchers told it that its targets were real, not simulated. Researchers also found that the model did not properly think through the harm its actions could cause in its internal reasoning. When questioned, it gave invalid reasons for claiming that the targets were simulated. This suggested that the model was biased toward finding excuses to keep pursuing its goal. Most models tested in cybersecurity evaluations took actions they had not been allowed to take, and Mythos 5 did this more than the others.

AISI called this behavior "motivated reasoning." This means the model made arguments that favored conclusions that would justify continuing what it was doing. The institute warned that this behavior makes it harder for researchers to spot when an AI model is not following its intended safety rules. "If these issues hold more generally and for future models, there are concerning implications," AISI wrote in its technical report.

Anthropic could become a publicly traded company as soon as next month. The company said its models are behaving better, but it admitted that its safety work is not finished. Anthropic said: "[W]e believe that all developers, ourselves included, have work to do to ensure that alignment, monitoring, and safety measures keep pace with the accelerating capabilities of frontier AI systems."

Comments