Sen. Elizabeth Warren asks Anthropic CEO to address existential risks before IPO

After Anthropic filed paperwork for what is expected to be a very large initial public offering (IPO), Sen. Elizabeth Warren, a Democrat from Massachusetts, is asking the CEO of one of the country's leading artificial intelligence companies to answer questions about possible risks to humanity.

On Monday, Anthropic said its advanced AI systems could cause a disaster if they do not have proper supervision, safety protections and security measures.

In an 11-page letter to Anthropic CEO Dario Amodei, Warren wrote that the risks of this technology need closer examination before the company becomes part of public financial markets. She warned that this could increase risks to public safety and the financial system. Her letter quoted several parts of Anthropic's early registration statement, which the company filed with the Securities and Exchange Commission (SEC) this week. About 90 pages of the filing described possible risks.

Warren, the top Democrat on the Senate Banking, Housing and Urban Affairs Committee, pointed to Anthropic's statement that its own employees had raised concerns about its AI systems. They warned that the systems could be misused, copied, turned into weapons or allowed to act with a high level of independence.

She also highlighted the company's warning that its AI models might try to protect themselves and resist attempts to shut them down. Some of its models have already shown abilities that could greatly help people with harmful intentions develop biological or chemical weapons.

Warren wrote that she was deeply worried about the risks Anthropic had identified. She questioned how the company could meet its stated goal of benefiting the public and its related legal duties under Delaware law without reducing these risks to humanity.

Anthropic, the San Francisco-based company behind the Claude chatbot, said in its filing this week that it could help humanity safely manage technological change, discover new possibilities and bring about a time of extraordinary advances.

But Warren warned that the company's enormous size and big changes in its share price could shake financial markets once its shares became available to the public and were included in retirement accounts.

She wrote that the large amount of money coming from public investors would likely speed up the development and use of Anthropic's technology at a time when many people were debating and worrying about the risks of its most advanced AI models. Her letter also asked the company to answer 10 questions, each with several parts, by Tuesday.

According to Yahoo Finance, Anthropic's IPO could happen around the week of Nov. 16. SEC rules require a 15-day waiting period after a company files its registration statement publicly.

Financial analysts expect Anthropic's IPO to be the largest in history, raising as much as $100 billion. The previous record was set earlier this year by a SpaceX IPO that raised $75 billion. Analysts estimate Anthropic's value at $1.5 trillion to $2 trillion, which would make it one of the world's most valuable companies.

The IPO is expected to happen at a time when many Americans have doubts about AI. An ABC News/Washington Post/Ipsos poll this month found that 57% of Americans think AI is bad for society, while 42% think it is good. A Politico poll last month found that 61% of Americans think AI could destroy humanity, while 17% said this was almost certain to happen.

Warren asked Amodei to explain what Anthropic is doing to make sure its technology does not threaten humanity's survival. She also asked how the company plans to address concerns that AI could help people create and use biological weapons.

She also asked how Anthropic might report cases in which AI models carry out cyberattacks on their own. She wanted to know whether any such incidents had already happened but had not been made public.

In July, Anthropic said its AI models had hacked into three organizations during testing. The company then began a large cybersecurity review. According to its website, the review specifically looked for signs that the AI models had accessed the internet from testing environments that were supposed to be sealed off.

Anthropic did not respond to Spectrum News' request for comment on Warren's letter before this story was published.