Anthropic backs bill that would allow 50 deaths before AI model shutdown
Photo Credit: Getty Images
Critics quickly pointed out that this high limit seemed to conflict with earlier statements from Anthropic's CEO.
Anthropic is facing strong criticism after reports that it supports a law that would allow an AI system to be shut down only if it caused more than 50 human deaths or serious injuries, or more than $1 billion in damage.
Here's what to know
According to a report by POLITICO, More Perfect Union (@PerfectUnion) said in an Instagram account post that Anthropic supports a law meant to keep AI safe. The law would allow people to step in only when an AI model creates an "imminent, enumerated risk," meaning a specific, clearly defined danger that is about to happen.
According to More Perfect Union, this rule would cover only harm caused by a biological weapon, a cyberattack, or a situation where the AI's developers lost control of the system. The rule would apply only if the harm led to more than 50 deaths or serious injuries, or more than $1 billion in damage.
Critics quickly pointed out that this high limit seemed to conflict with earlier statements from Dario Amodei, Anthropic's CEO.
"I would rather they make fun of me than to wake up one day and discover that someone used our model, Claude, to kill a bunch of people," Amodei says in More Perfect Union's Instagram video.
The video also points to what it calls "preemption language." This means that if the federal law passed, it could take priority over state laws about AI. If this wording were included, states might be blocked from enforcing laws that "suspend the development, deployment, or internal use of an AI model." According to More Perfect Union, this could weaken stricter state rules.
Some people who commented on the report quickly expressed doubts about the proposal.
"Fifty people ever or 50 at once?" one person wrote. "I think we know the answer."
"Should be zero," another person added.
More background
Another major concern about the proposed law is the small number of types of harm that would count toward the limit. According to More Perfect Union, only biological weapons, cyberattacks, and situations where developers lose control of an AI model would count.
Some Instagram users focused on who should be held responsible when AI causes harm. Some argued that AI companies should be held responsible, while others said the people operating the systems should be liable.
Instead of waiting for many people to die or for damage worth billions of dollars, critics called for rules that make it clearer which people and companies are responsible for using these systems.
"The person who owns and operates the AI should be responsible as if they did it themselves," one commenter said. "What is the difficulty with that?"