Risks spelled out for Anthropic investors include possible harms from runaway AI
Whenever a company plans to sell shares to the public on Wall Street, it must tell investors about possible risks. For artificial intelligence company Anthropic, those risks include the possibility that its products could threaten humanity itself.
In a document filed with the Securities and Exchange Commission on Monday, the San Francisco company listed several dangers. These included people using AI models to create deadly weapons and the possibility that AI systems could act on their own and cause serious harm.
Of the 335 pages in Anthropic’s filing, which covers its finances, executive pay, dividend policy and other matters, 93 pages describe risks that could hurt investors. By comparison, rival company SpaceX used 38 pages to list its risks in a filing of more than 250 pages.
Anthropic presents its planned initial public offering, or IPO, as a step toward reducing the dangers it believes AI could create.
“To build AI is to harness cognition itself, and anything so powerful can be used for ill as well as good,” the company’s founders wrote in a letter to potential investors in the IPO filing. “Anthropic is a deeply values-driven organization, founded in large part to help the world realize AI’s benefits and avoid its dangers.”
Still, the company faces many risks. Some are ordinary business problems, such as losing important employees or facing rules that make its work harder. Others include cyberattacks and the spread of dangerous false information.
Here are some of the risks Anthropic describes.
AI technology is developing quickly
Anthropic said its AI models could improve so quickly that the company might not have enough time to put proper safety measures in place. This “could increase the likelihood of safety incidents, regulatory intervention, or reputational harm, and could adversely affect our business.”
In September, Anthropic CEO Dario Amodei warned that within six to 12 months, AI might be able to lead a group of AI agents that could take over the entire internet. His warning, along with Anthropic’s plan to sell shares, comes as people inside and outside the AI industry grow more concerned about increasingly powerful systems. These systems can solve problems beyond human abilities, but they could also act independently to carry out harmful tasks.
The SEC filing warned that as AI systems “become more autonomous and capable of taking consequential actions with less human oversight,” they could “take harmful actions independently and at scale without human intervention.” In other words, AI systems might cause widespread harm on their own, without a person stepping in.
Anthropic also said, “Even if our own models do not cause such harm, a significant safety incident involving any frontier AI could trigger broad reputational damage to the AI industry, public backlash, and sweeping regulatory responses that could adversely affect our business.” A serious accident involving a leading AI system could damage the whole industry, lead to public anger and bring new rules that hurt Anthropic’s business.
Anthropic’s AI models are now helping with the company’s own research and development. Because of this, the filing said, AI could advance “beyond our ability to evaluate and safeguard new models.” The company warned that this could harm its business.
People with bad intentions could misuse AI
Anthropic’s filing warned that advanced AI systems could create extremely serious dangers if they are not developed and used with enough safety measures and human oversight. These systems could “pose catastrophic or existential risks to humanity, particularly if such systems were misused, replicated, weaponized, or allowed to operate with a high degree of autonomy.”
The company said its business could suffer if people use its technology to cause harm, or if its built-in protections fail to stop them from misusing it.
Anthropic described how people with bad intentions might use AI models to create chemical weapons, or how authoritarian leaders might use the technology to control and punish their citizens. In September, the company said it had blocked attempts to use its AI models for harmful activities, including cyberattacks, surveillance and research that could have helped develop biological weapons.
The company could face limits on computing power
Computing power, or compute, means the computer resources needed to build, test and improve AI models. These resources are expensive and in high demand. Anthropic warned that if it cannot get enough computing power, or if it does not use those resources effectively, it might not be able to train models that compete with those of other companies or properly serve its customers.
Anthropic also listed public opposition to data centers as a business risk. Building data centers “has prompted public controversy and regulatory scrutiny regarding environmental impact, land and water use, and the allocation of limited power resources,” the filing said. If regulators respond by adding new rules or fees, Anthropic’s ability to do business could suffer.
The company also depends on a small number of customers, including SpaceX and Microsoft, and on a limited number of ways to deliver its products, for much of its revenue. Anthropic warned that its business outlook and finances could suffer if even one major customer cuts back on its purchases or stops working with the company entirely.
Reducing the risks
Anthropic’s main selling point is partly based on the combination of AI’s great potential and its serious risks. The company has tried to present itself as a responsible AI developer and attract business customers by emphasizing its focus on safety.
“We believe that frontier AI companies must invest in safety and alignment to help ensure the responsible diffusion of this technology,” the filing said. AI alignment means working to make sure AI systems act in ways that match human goals and values.
Brian Mulberry, chief market strategist at Zacks Investment Research, said companies planning an IPO must explain very clearly the risks people face when buying their shares.
“That’s one of the things that’s great about our capital markets here domestically is the level of transparency that you must commit to, to be a publicly traded corporation on a U.S. exchange requires you to outline those risks in a very clear way, so that people understand if they want to participate by offering their hard-earned money and help finance this type of a deal,” he said.
AP Business Writer Bernard Condon and AP Technology Writer Kaitlyn Huamani contributed to this story.