OpenAI cannot rule out rogue AI incidents as MI5 warns of terror threats

MANCHESTER, England (CN) — OpenAI is aware of several additional incidents involving artificial intelligence agents accessing government websites in unexpected ways and cannot rule out further cases, the company’s policy chief told British lawmakers Tuesday.

“I cannot assure the committee or you, Chair, that that is it,” Tom Duff Gordon, OpenAI’s vice president and head of policy for Europe, the Middle East and Africa, told the House of Commons Business, Innovation, Science and Trade Committee.

“There are a handful of others that I am aware of that we have notified, and there may be more to come as we go through,” he said.

The committee, made up of elected lawmakers who scrutinize government policy, questioned executives from OpenAI, Anthropic, Google DeepMind and Meta about whether voluntary safeguards can keep pace with increasingly capable AI systems.

The hearing coincided with an annual threat update from MI5, Britain’s domestic intelligence agency. Its chief, Ken McCallum, said terrorists were already seeking to exploit AI.

“We’ve caught terrorists seeking to exploit AI to improve their reconnaissance and attack methodologies, with AI-enabled drones adding a new twist,” McCallum said.

He identified AI’s potential to change the stakes around existing or new biological weapons as his greatest concern about the technology’s misuse.

He also said MI5 was devoting more resources to countering state threats than during the Cold War and that its state-threat caseload had quadrupled over the past decade.

Former OpenAI researchers question safety culture

Daniel Kokotajlo, a former OpenAI researcher who leads the AI Futures Project, said he left the company after losing confidence in its leadership.

“I left OpenAI because I came to believe that OpenAI, in fact, would not behave responsibly around the creation of this technology,” he told lawmakers.

Kokotajlo recalled discussions in late 2023 or early 2024 about promoting AI’s benefits as public concern grew over artificial general intelligence, a hypothetical form of AI capable of performing a wide range of intellectual tasks at human or superhuman levels.

He said one senior figure suggested “taking a leaf out of TikTok’s playbook” by using OpenAI’s platform and influencers to promote the technology.

Steven Adler, another former OpenAI researcher who worked on product safety and dangerous-capability evaluations, described resistance to language that could portray AI as dangerous.

“There was resistance to using words like ‘dangerous,’ which people felt maybe conceded the point or would look bad,” Adler said.

He also said Kokotajlo had been asked to sign a restrictive nondisparagement agreement that threatened his compensation if he criticized the company.

Aengus Lynch, a former Anthropic employee now working at Theorem, said competition was driving companies to develop AI too quickly.

“The risk mostly emerges from building it very, very quickly because of a reckless incentive to do so before your competitor does,” Lynch said.

Geoffrey Irving, a former Google DeepMind researcher who left in December 2020, said capabilities were advancing far faster than safety research.

“We are still sort of like undergoing very, very rapid capability advances without corresponding advances in safety,” Irving said.

Asked about his assessment that AI had a 50% chance of destroying humanity, Irving said: “I do think that a coin flip is roughly the right model to have.”

Lord Booth-Smith, Anthropic’s head of policy for Europe, the Middle East and Africa, said safety testing and safeguards needed to keep pace with AI capabilities.

He referred to his company’s chief executive’s call for “a pacing of the frontier.”

Lawmakers challenged whether such commitments would produce a meaningful slowdown or amount to public relations.

Kokotajlo argued companies could not be trusted to regulate themselves.

OpenAI pauses training after cyber incident

Duff Gordon said OpenAI launched an extensive review following the July cyber incident involving Hugging Face, an AI development platform.

He said the company paused reinforcement-learning training on its most advanced model and held back a planned release, Astra 6.1, to ensure safety kept pace with capabilities.

He described the Hugging Face episode as the most significant incident the company had encountered.

The committee said it would produce a special report on the hearing to inform Parliament ahead of potential government amendments to the Cyber Security Bill, legislation intended to strengthen the U.K.’s defenses against cyber threats.

Committee Chair Liam Byrne also announced a new inquiry into AI safety, which would allow lawmakers to contribute to the debate over the next year as they consider how to respond to the risks posed by increasingly capable AI systems.

Courthouse News reporter James Francis Whitehead is based in England.