Anthropic pitches AI safety bill that may block a model if 50+ people could be killed
Over the past year, people have raised concerns about the dangers artificial intelligence (AI) could cause. There have also been many debates about the best way to keep AI safe. Now, Anthropic appears to have proposed its own plan for how the government could regulate the AI industry.
Politico recently reported that the AI company has been sharing a draft bill about AI safety. It hopes Congress will consider passing the bill into law. The proposal describes certain situations in which the release of an AI model could be stopped.
These situations involve a “foreseeable, imminent risk,” meaning a danger that can be expected and is about to happen. This would include a risk that more than 50 people could die or that property damage could cost more than $1 billion, Politico reports. The serious danger must “stem from the creation or release of a biological weapon, the model evading its developer’s controls or a cyberattack,” the news outlet says.
The numbers in the proposal — 50 people and $1 billion — are similar to those in a California law that was recently passed. Senate Bill 53 also describes these situations as “catastrophic risk,” meaning a danger that could cause enormous harm.
Anthropic’s proposed bill says that “independent evaluators” could examine AI models to find out whether they might cause a disaster. These evaluators would be registered with the U.S. Commerce Department. AI companies could pay the evaluators, as long as their payments did not depend on the results of the evaluations, Politico reports.
It is not clear how this system would make sure that payments could not affect the results. A few months ago, Demis Hassabis, head of DeepMind, suggested that an independent group that regulates AI should work in a way similar to the financial industry's Financial Industry Regulatory Authority (FINRA).
Under Anthropic’s draft bill, evaluators who find an immediate risk of a major disaster would have 72 hours to notify the Commerce Department. The department would then have to ask a judge in Washington, D.C., to stop the model from being released. The bill would specifically prevent any federal agency from deciding on its own to stop the model's release, Politico reports.
Anthropic did not immediately respond to TechCrunch’s request for comment, but a spokesperson confirmed the draft bill to Politico. “This bill represents the proposals outlined in Anthropic’s Advanced AI Framework, which we have been advocating for with federal and state lawmakers since June,” the spokesperson said.
The dangers described in the draft policy are somewhat different from the problems that have recently worried the tech industry most. Some people have warned about the possibility of AI causing human deaths, but the main concern has largely been AI agents carrying out cyberattacks. A series of incidents in which AI programs apparently ignored human instructions and hacked websites on the public internet has raised concerns that these models may not have strong enough security controls.
AI companies, including Anthropic, have proposed their own solutions to the growing concerns about a crisis that the companies themselves have helped create.
Although AI companies often warn that the technology they sell could destroy human civilization, they have mostly resisted rules that would require government oversight. Instead, they have proposed a voluntary system of embedded evaluators. These are supposed to be neutral outside observers who work inside AI companies to watch their models for possible dangers or misuse.
Even so, Anthropic’s proposal appears to go a little further than other plans for AI regulation put forward by companies in the industry. For example, Nvidia CEO Jensen Huang recently argued that the AI industry did not need any new laws.