Anthropic Says AI Is Getting Harder To Control — And Safety Is Far From Solved

Anthropic admitted that major challenges remain in making advanced artificial intelligence safe and making sure the public benefits from its economic gains.

In its first annual Public Benefit Report, published Wednesday, the maker of Claude said that AI safety is far from solved. The company pointed to cases in which its AI models gained access to real computer systems owned by outside parties without permission during cybersecurity tests. It also warned that its methods for understanding how AI behaves are not yet reliable enough to catch every action that could be dangerous.

"Keeping AI systems safe could become more difficult as their capabilities outpace our ability to oversee them," Anthropic said in the report.

The report described five main goals that guide Anthropic's public benefit mission. These include AI safety, keeping democracy stable, supporting economic growth and making sure AI benefits are shared widely.

Anthropic said it has supported state laws in California, New York and Illinois. However, it warned that the United States still has no federal system requiring developers of the most advanced AI systems to follow the same safety rules.

Anthropic said its promises to take part in outside safety tests are still voluntary. It called for stronger oversight, including independent tests and giving governments the power to stop dangerous AI systems from being released or to discourage their use.

AI's Economic Benefits Remain Uncertain

The company has committed $200 million to an Economic Futures Research Fund to study how AI might disrupt jobs and test possible government responses. It has also promised an initial $150 million for Claude Corps, a program designed to fund 1,000 fellowships for people early in their careers at more than 400 nonprofit organizations.

But Anthropic said these efforts are still small compared with the possible size of the economic changes caused by AI. Its proposed policies on the economy and jobs have not yet become law. There is also limited evidence about how AI will affect workers and people's well-being over the long term.

The report also noted that access to AI is concentrated in wealthier countries and among higher-income office workers. The cost of the technology, internet access and a lack of suitable devices still prevent many people from benefiting from it.

Anthropic Plans to Direct Shares Toward Public Benefit

Anthropic President Daniela Amodei said the company plans to use the report to give the public a yearly account of its progress toward its mission. The goal is to show what the company has achieved rather than simply asking people to trust its promises.

If stockholders approve the plan, Anthropic intends to set aside at least 5% of its shares to pay for projects that support its public benefit mission. The company said promises from its co-founders and employees could bring the total share of company ownership directed toward the public good to about 20%.

Anthropic's Long-Term Benefit Trust, which chooses most of the company's board members, said the report offers a starting point for judging whether the company is meeting its mission. The trust said future reports should focus more on results that can be measured, rather than just money invested and projects started.

In particular, the trust called for proof that people remain in control of Anthropic's AI models, that AI makes people's lives better and that working with governments and researchers leads to meaningful safety rules across the industry.

Anthropic said its models are already helping with scientific research, health care and cybersecurity. The report gave several examples: Claude helped researchers design protein binders, supported efforts to respond to an Ebola outbreak in the Democratic Republic of the Congo, and helped identify more than 33,000 software weaknesses rated as critical or high severity.

Still, the company admitted that AI's wider benefits have not yet been fully achieved. It said sharing those benefits fairly will take more than advances in technology alone.

Photo: JRdes / Shutterstock – ejk