Anthropic to allocate 5% of shares to ‘Public Benefit Fund’ as AI safety concerns grow
Anthropic, the advanced artificial intelligence (AI) company that makes Claude, plans to give 5% of its shares to a new Public Benefit Fund. The company wants to increase donations for the public good as concerns grow about the risks of powerful AI.
The proposed fund will be set up as a limited liability company fully owned by Anthropic. It could receive another 2% of the company's shares each time the company's value rises by 50% above a starting value that has not yet been announced. The company's board would decide whether to make these extra allocations.
Anthropic, led by Dario Amodei, is also preparing for a possible initial public offering (IPO), which would allow people to buy its shares on the stock market. It plans to operate as a public benefit corporation, a type of company that must consider the wider public interest as well as profits for investors.
The planned fund comes as worries about AI safety grow. These concerns follow security breaches involving AI systems and warnings from industry leaders, including Amodei, OpenAI CEO Sam Altman, and Elon Musk, about the possible dangers of developing AI without proper controls.
Anthropic has presented itself as a company focused on AI safety and has warned that AI could pose “catastrophic or existential risks to humanity.” About 80 pages of the 261-page main section of its prospectus explain the risks the company faces.
In a blog post, Anthropic said the Public Benefit Fund could support donations to nonprofit groups. It could also give money directly to specific groups of people and support projects that address how AI affects jobs and the job market.
The company said that, together with its co-founders' promise to donate 80% of their shares and a program that matches employees' donations, about 20% of its shares would be set aside for public benefit. This figure does not include any extra shares that might be allocated in the future.
Anthropic plans to share more details about how the fund will be managed, how it will be organized, and the company-value levels that could trigger extra share allocations in the coming months. It also plans to publish a public-benefit report explaining its progress toward its mission.
The announcement received support from Musk, the CEO of Anthropic's rival SpaceXSI, who suggested giving shares directly to members of the public.
In a reply on X, the platform formerly known as Twitter, Musk said he recommended “giving the shares directly to the public.”
“Next year, I will give one of my SpaceX shares to every kid in America. This will be the first of many,” Musk wrote in a post early on Monday.
Musk owned about 6.42 billion shares in the company as of June 2023.
Anthropic has also made changes to the job market a priority for the proposed fund. This builds on an earlier promise to spend $200 million studying and addressing how AI affects employment.
Companies around the world have been cutting thousands of jobs since generative AI became popular. Many businesses, especially technology companies, say that adopting the technology quickly has helped them work more efficiently.
Last week, International Monetary Fund Managing Director Kristalina Georgieva said the organisation's research suggested that AI could change, reduce the level of, or eliminate about 40% of jobs. In advanced economies, the figure could reach 60%.
Alongside creating the Public Benefit Fund, Anthropic wants the United States government to have more power to oversee advanced AI systems.
According to Politico, company representatives have privately shown lawmakers on Capitol Hill a draft bill designed to regulate the safety and security of the most advanced AI systems.
The proposed law would let the Donald Trump administration and future governments stop an AI model from being released or limit how it is used if it posed a foreseeable and immediate risk of causing more than $1 billion in damage, or of killing or seriously injuring more than 50 people.
Politico reported that these risks would have to result from creating or releasing a biological weapon, an AI model escaping its developer's controls, or a cyberattack.
An Anthropic spokesperson told Reuters that the draft bill reflected ideas set out in the company's Advanced AI Framework, which was released earlier this year.
“The bill would grant the government the ability to block or deter dangerous deployments of the most powerful models and also requires developers to test frontier AI models, be transparent about findings, and get them independently evaluated,” the spokesperson said.
Under the draft bill, independent experts registered with the Commerce Department would regularly assess AI developers and their most advanced models. Companies could pay for these assessments, but their payments could not depend on the results.
If the experts found an immediate risk covered by the proposed law, they would have to report their findings to the Commerce Department within 72 hours of finishing the assessment, Politico reported.
The Commerce Department could then ask a judge to stop an AI model after deciding that it posed one of the risks described in the proposal. The draft clearly states that other federal departments, agencies, and officials cannot stop a model from being developed, released, or used internally.
A federal judge at the District Court for the District of Columbia could order an AI model to be suspended.
These proposals show how Anthropic is trying to combine its stated commitment to public benefit with rules to manage the risks posed by increasingly powerful AI systems.