Risks spelled out for Anthropic investors include possible harms from runaway AI

Whenever a company plans to start selling shares on Wall Street, it must tell investors about risks they might face. For the major artificial intelligence company Anthropic, those risks include the possibility that its products could threaten humanity itself.

The risks listed by the San Francisco company in documents filed Monday with the Securities and Exchange Commission include the possibility that bad actors could use AI models to build deadly weapons or that the technology itself could go out of control, with consequences that cannot be predicted.

Of the 300 pages describing Anthropic's finances, executive pay, dividend policy and other matters, 20 pages are devoted to “risk factors” — things that could hurt the value of shareholders' investments. By comparison, rival SpaceX listed risk factors on more than 100 pages out of more than 300.

By selling shares to investors, Anthropic is taking an initial step toward reducing the risks it believes AI poses.

“Our goal in building AI is to harness intelligence itself, and anything so powerful can be used for harm as well as good,” the company's founders wrote in a letter to potential shareholders in the IPO filing. “Anthropic is a deeply values-driven organization, founded in large part to help the world realize AI's benefits and avoid its dangers.”

Still, the risks are many. They range from everyday business problems — such as the possibility that the company will be unable to keep key employees or overcome regulatory obstacles — to involvement in cyberattacks and the spread of dangerous false information.

Here is a look at some of the risks Anthropic describes:

Anthropic said its models could develop so quickly that the company would be unable to put proper safety measures in place. This “could increase the likelihood of safety incidents, regulatory intervention, or reputational harm, and could adversely affect our business.”

As AI models improve themselves and help Anthropic with its own research and development, the technology could make progress “beyond our ability to evaluate and safeguard new models,” according to the filing. This could harm Anthropic's business.

The filing discussed concerns that advanced AI systems, if developed and used without enough safety measures and oversight, could create catastrophic or existential risks for humanity. These risks could be especially serious if the systems were misused, copied, turned into weapons or allowed to operate with a high degree of independence.

The company said it could be harmed if bad actors used its products to cause damage or if its model protections and safety measures failed to prevent such misuse.

Anthropic described the possibility of bad actors using AI models to build chemical weapons or authoritarian rulers turning to the technology to crack down on their citizens. In September, the company disclosed that it had blocked efforts to use its AI models for harmful activities such as cyberattacks, surveillance and research that could lead to the development of biological weapons.

Computing power — the computer resources needed to build, test and improve AI models — is expensive and in high demand. Anthropic noted that if it cannot obtain enough computing power, or if “we fail to direct our compute effectively across our workloads, we may be unable to train models that remain competitive with those of our competitors or adequately serve our customers.”

Anthropic also mentioned opposition to data centers as a risk to its business. Data center development has prompted public controversy and regulatory scrutiny over environmental effects, land and water use, and how limited power resources are allocated, the filing said. If regulators impose new requirements or fees as a result, the company's ability to do business could suffer.

In addition, the company depends on a small number of customers, including SpaceX and Microsoft, and on distribution channels for a large part of its revenue. Anthropic said that if even one of its major customers decided to reduce its business with the company or leave altogether, its outlook and finances could suffer.

In some ways, AI's enormous risks, combined with its enormous potential, are Anthropic's main selling point. The company has tried to present itself as a responsible AI company and win business customers by emphasizing safety and caution.

“We believe that frontier AI companies must invest in safety and alignment to help ensure the responsible spread of this technology,” the filing said.

Brian Mulberry, chief market strategist at Zacks Investment Research, said AI companies must make the risks facing people very clear when they invest their money in a newly public company.

“That's one of the things that's great about our capital markets here domestically: The level of transparency that you must commit to. To be a publicly traded corporation on a U.S. exchange requires you to outline those risks in a very clear way, so that people understand if they want to participate by offering their hard-earned money and help finance this type of idea,” he said.

AP Business Writer Bernard Condon and AP Technology Writer Kaitlyn Huamani contributed to this story.