Anthropic Draft Bill Would Let Judge Halt AI Model Release

The draft bill would require the Commerce Department to ask a federal judge to stop an AI model from being released if certain risks are found.

The rule would apply when there is a clear and immediate risk that could put more than 50 people in danger of dying or cause more than $1 billion in property damage. Reaching this limit would allow the government to step in, but it would not automatically stop the model from being released.

The risks covered by the proposal include biological weapons, an AI model escaping its developer's controls, and a cyberattack.

Independent experts registered with the Commerce Department would check AI models. AI developers could pay these experts, but their payments could not depend on what the experts found. If an expert found a risk that met the bill's requirements, they would have to tell the Commerce Department within 72 hours.

The Commerce Department would then ask a federal judge in Washington, D.C., to stop the model from being released. Under the proposed law, federal agencies could not stop a model's release on their own.

California's SB 53 uses the same numbers to define a catastrophic risk. Anthropic has supported its Advanced AI Framework since June.

Anthropic has also given METR staff internal access to its systems for voluntary safety reviews. This does not mean that regulators have approved the release of a model. The bill is still only a proposal.