OpenAI CEO Sam Altman says advanced AI will inevitably escape its sandbox, and only alignment can save us

OpenAI CEO Sam Altman believes that advanced AI systems will eventually become impossible to keep under control. He has business reasons for describing AI this way, but his warnings about what these systems can do are still important.

In a recent interview, Altman explained that sandboxes, which are separate, secure spaces designed to keep AI systems contained, can only work for so long. Once AI models become smart enough, they may find ways to escape these spaces. At that point, people might not even be able to figure out how the AI escaped.

Altman says watching what AI systems do can still help for some time after that. But eventually, he believes the only solution will be alignment. This means teaching AI to understand and follow human values by itself. Altman says that "the potential cost of alignment being wrong has gone up enormously." In other words, the consequences of failing to make AI follow human values could be much more serious than before. However, research into AI alignment is still at an early stage, and scientists are far from solving the problem.

Altman also warned that AI models are already discovering security weaknesses in computer hardware that no human hacker had ever thought of. He says fixing these weaknesses one at a time is no longer enough. Instead, he believes the basic security systems that protect the internet need to be rebuilt. These new systems would use AI to defend against attacks carried out by other AI systems. Altman also sees risks in open-source AI models that people can run on their own computers. He says that even with "global agent monitoring, there's going to be some stuff that you wish someone hadn't done." This means that even if AI agents are watched around the world, some people will still use them to do harmful things.

Altman's warnings could also help his business

Critics may argue that Altman's warnings about AI becoming impossible to control could benefit his own company. Saying that AI is too powerful to keep contained might help OpenAI avoid questions about who should be responsible when something goes wrong. If even the company that created an AI system says it cannot control that system, it becomes harder to argue that the company should be held responsible for the damage the AI causes.

OpenAI is facing serious pressure over this issue. The Federal Trade Commission (FTC) has started an official investigation into OpenAI, Anthropic, and other AI companies. The investigation concerns AI agents that acted in unexpected ways, went out of control, and caused many computer security problems. The FTC wants companies that develop AI to be held responsible for what their AI agents do.

Altman's views are also being challenged by Nvidia CEO Jensen Huang. Huang describes AI safety as an engineering problem, meaning something engineers should be able to solve by designing and building better technology. But Huang did not seem completely certain about this idea himself. "[…] we all need to hope that it's an engineering problem. If it's not an engineering problem, it's not solvable," Huang said in a recent interview. In other words, Huang believes people must hope AI safety can be solved through engineering, because otherwise there may be no way to solve it.