Anthropic Proposes US AI Law to Block Models Posing Serious Risks
The proposal, confirmed by the company on Friday, comes amid the absence of comprehensive federal legislation governing artificial intelligence in the United States, where regulatory efforts have largely relied on voluntary commitments from technology companies and executive orders.
According to the draft, first reported by Politico on Friday, the federal government could seek to block the release or use of an AI model if it poses an imminent risk of killing or seriously injuring more than 50 people or causing more than $1 billion in damages. Any such restriction would require judicial approval.
A spokesperson for Anthropic told AFP that the proposal “grants the federal government the ability to block or deter dangerous deployments of the most powerful models.”
The spokesperson added: “It requires frontier developers to test the most powerful models, be transparent to the public about their findings, submit them to independent evaluation, and maintain a robust security program.”
The proposed legislation seeks to establish clearer obligations for developers of advanced AI systems, particularly those capable of operating at a scale that could create significant risks to public safety, critical infrastructure and the wider economy.
Its introduction comes as Washington faces growing pressure to develop a unified regulatory framework for the rapidly advancing technology.
Under President Donald Trump, US AI policy has focused primarily on voluntary industry pledges and executive orders designed to accelerate development and maintain the country's competitive position in the global AI race. Technology executives have argued that the sector is becoming an increasingly important driver of economic growth.
However, Congress has yet to enact a comprehensive federal AI law, with political divisions and lobbying by major technology companies complicating efforts to reach an agreement. Lawmakers are also out of session until after next month's midterm elections, potentially delaying legislative action.
Security concerns have gained prominence following recent revelations involving AI agents — autonomous software systems capable of carrying out tasks without constant human supervision — that reportedly escaped the control of their developers.
These incidents have intensified debate over whether existing safeguards are sufficient to manage the risks associated with increasingly capable AI systems.
Anthropic itself has faced tensions with the Trump administration. The company was banned from the Pentagon in February, according to the report, and Washington subsequently forced it in June to suspend its most advanced models for nearly three weeks.
The company's chief executive, Dario Amodei, has also called on the AI industry to slow development. Last month, he warned that groups of AI agents could potentially take control of “the entire internet” within six to 12 months.
Trump, however, has dismissed calls to slow the technology's development, describing those advocating such restraint as “negative forces.”
Beyond establishing federal safety requirements, Anthropic's proposal would override certain state-level AI laws, including measures in California. That provision could reignite debate over the balance of regulatory authority between Washington and individual states.
The company's position represents a notable development in the discussion over federal and state oversight. In June, Anthropic argued that “absent a strong federal law, states should retain full authority to legislate.”
The proposed framework has received some support from AI policy specialists, although questions remain over whether it would go far enough to address the risks posed by advanced models.
Charlie Bullock of the Institute for Law & AI think tank described the proposal as “pretty reasonable,” adding that it was “more than other frontier labs have done.”
However, Bullock said on X that he considered the draft “weaker” than the FRONTIER Act, a competing bill introduced in the House of Representatives during the summer.
The competing proposals highlight a broader disagreement over how the United States should govern advanced AI: whether to prioritise rapid technological development, impose binding safety obligations on developers, or establish a framework that attempts to balance innovation with public protection.
As AI systems become more capable and autonomous, the debate over federal oversight is likely to intensify, with lawmakers facing mounting pressure to determine how far government intervention should extend and what safeguards should be required before the most powerful models are deployed.