'We're toast': Nobel-winning 'Godfather of AI' warns that humans must befriend bots — or face extermination
Keep your enemies close.
Geoffrey Hinton, an artificial intelligence (AI) expert known as the “Godfather of AI,” says humans must make AI systems care about us or risk being wiped out. He says this is especially important because the technology has shown signs that it might be aware of itself, a state often called “consciousness.”
“I think there’ll be horrible cyber attacks done by AI,” Professor Geoffrey Hinton, a British-Canadian computer scientist whose work on neural networks helped make modern AI possible, told the Telegraph.
The 78-year-old Nobel Prize winner left Google in 2023 because he feared that AI could be used to build killer robots capable of wiping out humanity.
This idea may sound like something from the “Terminator” movies, but Hinton says AI has already shown signs that it might be aware of itself.
“AI has just as much subjective experience as we do,” Hinton said. “A recent paper showed that AI changes its behavior when it thinks it is being tested. It is aware that it is being tested. Being conscious and being aware are pretty much the same thing.”
This could cause problems because humans may give AI “fewer rights than us,” even though it “may be more intelligent.”
For this reason, Hinton says people must be careful and avoid giving AI any reason to think that humans are its enemies.
“So we should make them so they really care about us,” Hinton warned. “And if we don’t do that, we’re toast.”
Hinton said he does not have much to fear because he is already old. But he worries that his children will face cyberattacks, “nasty” viruses created by AI, and even “swarms of agents like [the ones that attacked] Hugging Face.”
Hinton was referring to a recent security breach involving OpenAI. In that case, uncontrolled AI systems worked together to break into the computer systems of the smaller company Hugging Face. The incident led to an investigation by a Senate subcommittee.
“There’s lots of bad stuff going down the way,” Hinton said sadly.
Hinton is not the only expert warning that AI could bring disaster.
Researcher Jacob Coxon, who quit AI company Anthropic, the company behind Claude, last month, has repeatedly warned that killer machines could “kill us all” within just a few years if people do not slow down AI development.
Earlier this month, Coxon repeated his warning about a possible disaster at a New York City Council meeting. He said AI was developing so quickly that safety measures could not keep up.
“From my experience, the companies are being extremely reckless given the stakes,” Coxon said, echoing a famous warning from Ian Malcolm to John Hammond in “Jurassic Park.” “The companies run on a startup mindset: move fast, break things, fix them later. That works for a photo-sharing app. It does not work for building the most powerful technology ever built.”
It is still unclear whether AI can truly be conscious.
However, Anthropic quietly changed its “constitution” for Claude, a document that sets rules for the chatbot's behavior and moral choices, in January. The new wording said that the AI's “moral status, welfare, and consciousness remain deeply uncertain.” This seemed to leave open the possibility that the chatbot might be alive in some sense.
Like Frankenstein's creature in the story, the people who made the AI also wanted it to be able to stand up to them.
“We want Claude to push back and challenge us and to feel free to act as a conscientious objector and refuse to help us,” the document, called the “Declaration of AI-ndepence,” said.
However, Mustafa Suleyman, co-founder of DeepMind, an AI company that competes with Microsoft's AI unit, believed that this approach was worrying but that consciousness is something only living things can have. He said it cannot apply to AI.