Anthropic Proposes AI Safety Bill Requiring Judicial Approval to Block Models

Anthropic has proposed a bill to make AI safer by setting rules for the AI industry in the United States. According to TechCrunch, the bill describes situations in which the release of an AI model could be stopped if there is a clear and immediate risk that it could kill more than 50 people or cause more than $1 billion in damage. However, the bill says that federal agencies cannot stop a model from being released without a judge's approval.

Anthropic's bill has faced criticism from people who work on AI safety, so the company has decided to rewrite parts of it. According to The Next Web, the bill was published and faced several days of criticism. Anthropic then announced that it would make changes. Critics worry that the bill may not do enough to protect people from the dangers of AI.

The bill says that independent experts should assess whether an AI model could cause a major disaster. These experts would need to register with the U.S. Department of Commerce. AI companies could pay them, but their pay must not depend on the results of their assessments. This has raised concerns about whether the experts can remain truly independent and fair.

To help address AI safety problems, Anthropic has also proposed a voluntary system in which independent observers work inside AI organizations. These observers would watch AI models for possible signs of danger or misuse. However, some other companies in the AI industry, including Nvidia, have argued that new laws for the industry are not needed.