'Insane' to see AI in hands of private giants: ex-Anthropic researcher to AFP

KMVU FOX 26 Medford

Source published: Site published:

Original article · فارسی

British AI researcher and former Anthropic employee Jacob Coxon thinks the only way to slow down AI development is political pressure

British AI researcher and former Anthropic employee Jacob Coxon believes political pressure is the only way to slow down AI development.

Karl Mondon

Artificial intelligence (AI) researcher Jacob Coxon left Anthropic in September. He accused the company of taking dangerous risks with people's lives. On Thursday, he told AFP that it was "insane" that huge, money-making companies were developing such powerful technology.

"Today is a very strange day," said the 27-year-old British researcher. His former employer had just made a record-breaking start on the stock market.

Coxon spoke outside Anthropic's headquarters in San Francisco. Protesters there were calling for a ban on developing AI models that can build newer and more advanced versions of themselves. The AI industry calls this process recursive self-improvement (RSI).

AI was once mainly developed by small non-profit groups and university research teams. But now, the industry is controlled by companies worth trillions of dollars, such as Anthropic and its rival OpenAI, which makes ChatGPT.

Anthropic was valued at $1.5 trillion during its initial public offering (IPO) on Wednesday. By the end of its first day of trading on the Nasdaq stock exchange, its value had fallen to $1.3 trillion. According to reports, OpenAI was valued at $1.25 trillion in a private funding round earlier this month.

"I think this symbolizes just how crazy the situation has become, because it's insane. It's really insane that for-profit companies are the ones building this technology," Coxon said.

Coxon graduated from Cambridge University. He spent three years at OpenAI working on model pre-training. This is the stage when an AI model learns by taking in huge amounts of information. He then worked at Anthropic for a few months.

He announced his resignation on X on September 8. His decision made headlines around the world. He accused both companies of "racing straight to self-improving superintelligence."

"The people building AI earnestly believe that it could kill us all by the end of the decade," he wrote.

In theory, this dangerous future could happen because of recursive self-improvement (RSI).

"You end up potentially compressing decades of AI research into months, and if we don't understand the AI, this leads to potentially highly dangerous, highly advanced AI systems by the end of 2027," Coxon told AFP on Thursday.

"Anthropic should refuse to do RSI," he said. He also accused the company of planning to start this process as early as next year.

– "Scared" about the state of the world –

Coxon left Anthropic before all the shares promised to him had fully become his. However, he still owns shares in OpenAI. These could become worth a huge amount of money if the company goes public in 2027, as planned.

During a talk at Stanford University on November 9, he called his OpenAI shares "blood money." Even so, he still owns them.

"I like money, bro," he said. His comment was widely mocked on social media. A few days later, he changed his position and promised to donate the full value of his shares to organizations working to make AI safer. He did not say which organizations would receive the money.

He now plans to work for a non-profit organization, possibly doing research into AI safety.

"Ideally, I'd like to do safety research, but I think there's not enough time for that," he said.

"The only thing that will slow down the AI companies is political action. It's a political problem now, not a technical problem. We need to slow them down," he said.

He supports AI technology overall, but says people should think carefully about how it is used.

"It's a useful technology, but be aware that chatbots are not the final form," he said. He added that AI is moving towards very powerful systems that could do every kind of human job.

More than two months after leaving Anthropic, the huge amount of media attention has affected Coxon. Sometimes he stares into space, but he says he feels "fine."

"I was very surprised by everything," he said. However, he is more "scared about the state of the world than I am feeling like my personal life is turbulent."

Originally published on doc.afp.com, part of the BLOX Digital Content Exchange.