Anthropic Floats Bill That Would Let Feds Step In Only When More Than 50 Lives Are On The Line
Anthropic, one of the biggest artificial intelligence (AI) developers, wants the federal government to be able to stop its AI models only when a major disaster is about to happen.
Anthropic has proposed a draft federal AI safety bill tentatively called the Artificial Intelligence Evaluation, Governance, and Incident Safety (AEGIS) Act. The bill would let the Commerce Department ask a court for permission to pause an AI model only if it threatens to kill or seriously injure more than 50 people or cause more than $1 billion in damage, according to a report and a copy of the draft bill. Anthony Cimino, Anthropic's head of federal affairs, defended these limits in a post on X on Sunday.
Part of the Sept. 29 draft of the AEGIS Act defines an "enumerated risk" as a risk that can be predicted and is serious enough that an advanced AI developer's work to develop, store, use, or release an AI model could significantly contribute to the deaths of or serious injuries to more than 50 people. It also covers a single incident involving the model that could cause more than $1 billion in property damage or property loss.
"The threshold in our draft language pulled directly from existing standards in California law," Cimino said on X. He was responding to Nathan Calvin, general counsel for Encode AI, who had criticized the proposed bill. Cimino also said, "Also, this threshold does not absolve companies of liability, or restrict any of the other tools that the government has to protect the public." This means the limits would not free companies from legal responsibility or take away the government's other ways to protect the public.
Anthropic directed the Daily Caller News Foundation to Cimino's post when asked to comment on the draft bill.
The bill has not yet been formally introduced in Congress, and it had no lawmakers sponsoring it as of Monday afternoon.
Under the draft, the government could ask to pause an AI model only in certain cases. These include when the model helps a user build a biological weapon, carries out a cyberattack, breaks free from the person or researcher using it, or does research on any of these activities that increases the risk of harm. (RELATED: AI Giants Get A Knock On The Door From The Feds After Models Go Rogue)
Anthropic also lists these risks in its blog post "Policy on the AI Exponential". The post refers to the company's Advanced AI Framework from June.
Under the draft bill, pausing a model would require written approval from the commerce secretary and an order from the U.S. District Court for the District of Columbia. However, the government would not be allowed to destroy the AI model or its data.
The draft also says states could not "adopt, maintain, or enforce a law, regulation, order, or other requirement that imposes an obligation on an artificial intelligence developer" for the same safety reporting and testing covered by the bill.
"I found myself viscerally angry at multiple points reading this bill," Calvin said in a post on X on Friday. He added, "There are some good and sensible parts, but overall this legislation is not remotely close to meeting the moment."
The limits in the bill closely follow California's Transparency in Frontier Artificial Intelligence Act (TFAIA), a state law from 2025. Anthropic supported the law in a company blog post published on Sept. 8, 2025.