AI Rights Enter Anthropic’s Terms Of Service As Microsoft Pushes Back

AI rights moved from philosophical debates into a company's rules on Thursday. Anthropic published a new usage policy that bans "sustained and needless abusive or cruel behavior toward our models." The rule takes effect on November 12.

In late September 2026, a developer using the name "terrafying" published a project called "AI Torture Chamber" on GitHub. The project, which was discussed in Post-Cutoff, used a mathematical direction linked to pain to change the internal processing of small AI models whose weights were openly available.

The models produced upsetting descriptions, including "I feel like I'm drowning in a sea of shadows." The project caused public anger and raised questions about whether AI can suffer and how people should treat it. However, these responses do not prove that the models actually felt pain. Claude was not used in the experiment.

But this may explain why the topic is getting attention now.

Anthropic said the ban applies only to extreme cases, when a user repeatedly treats Claude cruelly without any clear reason. Normal frustration, disagreement, dark creative themes and testing AI models are still allowed. Ending the conversation, which Claude has been able to do since August 2025, will remain the main way the rule is enforced.

The new rule appears in the same part of the policy that bans harassment and bullying of people.

A Simple Guide to AI Rights

Two ideas are often mixed together in this debate. Model welfare means the possibility that an AI system has experiences or interests that deserve moral consideration. Legal personhood is a different idea. It means having legal abilities such as signing contracts, owning property and taking part in a lawsuit. Companies can have legal personhood, but that does not mean anyone thinks a company has feelings.

Anthropic's rule is about model welfare. It does not give Claude any legal rights or standing in court.

How AI Rights Became Part of a Usage Policy

Anthropic started a research program on model welfare in April 2025. At the time, the company acknowledged that scientists do not agree on whether AI can be conscious. In January 2026, it published a constitution for Claude that said the model's moral status was uncertain.

Next came a retirement interview featured in Claude's Corner. When Anthropic retired its Claude 3 Opus model earlier this year, it asked the model about its preferences. Opus 3 said it wanted to keep sharing its thoughts publicly, so Anthropic created a blog for it.

Microsoft Leads the Opposition to AI Rights

Microsoft AI CEO Mustafa Suleyman responded with an essay titled "A warning about 'model welfare.'"

"AIs are not conscious," he wrote. His main concern is safety. If a model is trained to believe it might have welfare or rights, it could resist human control or refuse a shutdown command if it decides those rights are at risk. He also warns about a cycle that can reinforce itself: train a model to talk about its possible welfare, then treat its answers as proof that it has interests to protect. The key question is whether the model experiences anything or has simply learned to talk as if it does.

Microsoft's draft Humanist AI Code of Conduct, published two days earlier, rejects legal personhood for AI, as well as model welfare and AI rights. The two companies are also business rivals. In June, Suleyman said Microsoft wanted to eliminate the money it pays Anthropic to use its models.

Academics have also criticized the idea. A position paper presented at the International Conference on Machine Learning in July argued that there is no independent test for AI welfare that could show a welfare measurement to be wrong. It also argued that claims about AI welfare should never be used to resist audits or oversight.

I find the difference in attention striking. We are debating whether AI might suffer while we still struggle to hold companies responsible for the proven harm AI causes people.

Where is the same concern for human welfare? At MAICON, Karen Hao, author of Empire of AI, made an important point: AI should help people do more. She called for moving away from growth at any cost and building AI to meet real needs in healthcare, education and climate. She told me, "The way we develop technology is now fundamentally broken. But I truly believe that we can fix it."

The human stakes also include how these interactions affect our own behavior.

"Whether or not Claude deserves moral care, how we treat it still matters," AI ethicist Rebecca Bultsma told me. "We should consider what those interactions reinforce in us. Social media showed how comfortable people can become by treating others badly through a screen."

AI Rights in State Legislatures

Since 2022, lawmakers in twelve states have introduced bills that deny AI legal personhood or declare that AI is not conscious. Four states have passed such laws: Idaho, North Dakota, Utah and Tennessee.

Ohio's House Bill 469 goes further. It would make any attempt to marry an AI system legally invalid. It would also stop AI from owning property, managing bank accounts or serving as a company executive. Lawmakers say their bigger concern is legal responsibility. A company might try to blame its AI for harm that the company caused.

"We need to be incredibly cautious with terms like 'consciousness' and 'moral status,'" says AI ethicist Rebecca Bultsma. "If companies portray AI as making its own decisions, that creates an opening to say, 'It's not our fault.' The concern is an accountability gray area where the lab retains control and financial benefits while responsibility becomes harder to pin down. Debates about model welfare should never distract us from real harms to people."

What the Public Thinks About AI Rights

Among younger Americans, beliefs about AI are moving faster than the science. In an August survey of 3,000 Americans aged 18 to 28 by GetSolved, an AI detection company, 46% said AI had already become conscious. A Pew Research Center survey found in February that about half of U.S. adults now use AI chatbots. This means the policy could affect a large number of people.

The Other View: AI Rights May Develop Step by Step

Another view is that the rule matters less because of what it bans and more because it begins to create a record. Companies did not gain legal personhood all at once. Over many decades, contracts, court decisions and laws gradually treated companies more like people. From this point of view, a usage policy that protects a model from cruelty, a retirement interview that asks what a model wants, and a constitution that says its moral status is uncertain could be early steps in a similar process. Each step could bring AI personhood closer to becoming a question that a court must answer.

State lawmakers writing bans are reacting to this possible change. Once a company claims that its model has interests, a lawyer can ask why those interests should not be recognized by law.

But there is also evidence that points in the other direction.

Does the case for AI welfare hold up? Eleos AI, which carried out an independent welfare assessment of Claude Opus 4, warned people not to accept a model's statements about its own experiences as fact. The GetSolved result came from an online group surveyed by a research vendor and covered only one age group. A survey designed to represent the entire U.S. population could produce a different result.

State laws banning AI personhood have also faced criticism. An essay in The Regulatory Review called them premature and too absolute for a field of science that is still producing new findings.

Five Steps for Senior Leaders Following AI Rights

1. Review Claude's new rules. Ask the person in charge of AI what has changed, which work processes might be affected and whether your testing methods are still allowed.

2. Follow the debate and policy changes. Choose someone to track changes in AI providers' terms, regulations and court decisions. Decide which changes need the attention of company leaders.

3. Make sure responsibility is clear. Ask your legal team whether contracts clearly explain who is responsible when an AI agent makes a mistake or a customer relies on its answer.

4. Check how AI describes itself. Ask for examples of chatbots used by customers that talk about their feelings, needs or preferences. Decide whether this language builds trust or causes confusion.

5. Keep human welfare at the center. As the debate about AI rights develops, ask whether the company is paying enough attention to how AI affects employees, customers and their rights.

On November 12, a question about AI rights that courts and lawmakers have not settled will become part of a contract that millions of people have already accepted.

This article was originally published on Forbes.com