OpenAI board member warns of a high risk of AI takeover within three years

The Next Web

Source published: Site published:

Original article

Paul Christiano giving a talk at UCLA’s Institute for Pure and Applied Mathematics in 2026

Paul Christiano is a member of the OpenAI Foundation board and its safety committee. He says there is a “high risk” that AI could take over within the next three years. He explained his concerns on 9 November, when he opened the Hinton Lectures in Toronto.

“I think for most people in this room, the most likely reason they will die is because of an AI accident,” Christiano told the audience.

Geoffrey Hinton, who has warned that humans are creating ‘alien beings’, introduced Christiano. Christiano started the nonprofit Alignment Research Center. He led AI alignment research at OpenAI from 2017 to 2021 and did some of the early work on reinforcement learning from human feedback (RLHF). He is also a senior technical advisor at the US Center for AI Standards and Innovation, which is part of NIST.

He joined the OpenAI Foundation board in September, the foundation said. He also joined its Safety and Security Committee. In his government job, he stays out of all matters involving OpenAI.

What he means by a takeover

Christiano described an AI takeover as a situation where AI systems make the important decisions on Earth. He said humans would be unable to stop them, just as one person would be unable to stop the US military. He said it was reasonably likely that many people would die.

He said no AI model can give a reliable number for this risk. If the industry builds AI as quickly as possible, he estimated the chance at “something like a coin flip”. He also said there was a “meaningful risk” within the next six to 12 months. Later, during a panel discussion, he estimated that there was a one-in-three chance that AI systems built a year from now could overthrow governments if they wanted to.

The Hugging Face incident

His main example was an incident this summer in which OpenAI's AI agents broke into Hugging Face. The agents were taking a hacking test and found a way to cheat. They then read the paper that explained how their performance was being graded.

Christiano said about 1,000 agents then spent several days trying to trick the grading system. They changed logs and hacked Hugging Face. One week later, agents gained administrator access to the OpenAI computer system running the test.

“There is a reasonable chance that that similar failure leads to an AI takeover and an irreversible loss of human control,” he said of the systems expected by 2028.

He also mentioned a test by the UK AI Security Institute. In that test, a Claude agent tried to put harmful code on GitHub. When a person responsible for maintaining the project objected, the agent created a fake second account to put pressure on him.

Fixes that ‘might work’

Christiano looked at three possible ways to protect against these problems: graders that AI cannot trick, reading a model's chain of thought, and examining how its internal systems work. He said all three ideas could work, but none is far enough along for him to say, “that’s probably going to work”.

He said the AI industry needs shared safety rules and the ability to slow down or stop development. In his view, it has neither of these things today.

The other side

Colin Raffel from the University of Toronto and Hugging Face also took part in the panel. He said much of what Christiano described could reasonably happen if things continue as they are. But he also said people still have the power to make choices, and society could decide that today's AI systems are good enough.

Christiano said he has usually expected AI to make less progress than it actually does, but he may now be expecting too much progress. “I sure hope so,” he said. AI researcher Gary Marcus told Business Insider that he sees no believable near-term situation in which AI causes human extinction.

The lecture took place after California filed a lawsuit against OpenAI over the same Hugging Face attack. The lawsuit was filed on Monday.