'Insane' to see AI in hands of private giants: ex-Anthropic researcher to AFP

Yahoo Finance UK

Source published: Site published:

Original article · فارسی

British AI researcher and former Anthropic worker Jacob Coxon believes political pressure is the only way to slow down AI development (Karl Mondon)

AI researcher Jacob Coxon, who famously left Anthropic in September and accused the company of putting lives at risk, told AFP on Thursday that it was "insane" that huge companies focused on making money were developing such powerful technology.

"Today is a very strange day," said the 27-year-old British researcher. His former employer had just made a record-breaking start on the stock market.

Coxon spoke outside Anthropic's main office in San Francisco. Protesters there were calling for a ban on developing AI models that can independently build newer versions of themselves. The industry calls this process recursive self-improvement (RSI).

AI was once developed by small non-profit groups and university research organizations. But today, the industry is controlled by companies worth trillions of dollars, such as Anthropic and its rival OpenAI, the company behind ChatGPT.

Anthropic was valued at $1.5 trillion during its initial public offering (IPO) on Wednesday. By the end of its first day of trading on the Nasdaq stock exchange, its value had fallen to $1.3 trillion. Reports say OpenAI was valued at $1.25 trillion in a private funding deal earlier this month.

"I think this shows just how crazy things have become, because it's insane. It's really insane that for-profit companies are the ones building this technology," Coxon said.

Coxon graduated from Cambridge University. He spent three years at OpenAI working on model pre-training, the stage when AI models learn from huge amounts of data. He then worked at Anthropic for a few months.

He announced his resignation on X on September 8. His decision made headlines around the world. He accused both companies of "racing straight to self-improving superintelligence."

"The people building AI earnestly believe that it could kill us all by the end of the decade," he wrote.

In theory, this terrible outcome could happen because of RSI.

"You could end up squeezing decades of AI research into just months. If we don't understand the AI, this could lead to very dangerous and highly advanced AI systems by the end of 2027," Coxon told AFP on Thursday.

"Anthropic should refuse to do RSI," he said. He also accused the company of planning to start this work as early as next year.

Coxon left Anthropic before all his company shares had become fully his. However, he still owns shares in OpenAI. These could become worth a huge amount if the company goes public in 2027, as planned.

During a talk at Stanford University on November 9, he called his OpenAI shares "blood money," but said he still owned them.

"I like money, bro," he said. His comment was mocked on social media. A few days later, he changed his position and promised to donate the full value of his shares to organizations working on AI safety. He did not say which organizations would receive the money.

He now plans to work for a non-profit organization, possibly doing research into AI safety.

"Ideally, I'd like to do safety research, but I think there's not enough time for that," he said.

"The only thing that will slow down the AI companies is political action. It's a political problem now, not a technical problem. We need to slow them down," he said.

He supports AI technology overall, but says people should think carefully about how they use it.

"It's a useful technology, but be aware that chatbots are not the final form," he said. He added that AI is moving towards very powerful systems that will be able to do every kind of human job.

More than two months after leaving Anthropic, the attention from the media has affected him. Sometimes he stares into space, but he says he feels "fine."

"I was very surprised by everything," he said. But he is more "scared about the state of the world than I am feeling like my personal life is turbulent."