Google, OpenAI, Anthropic and Microsoft back one AI opt-out
Website owners may soon be able to tell Google, OpenAI, Anthropic and Microsoft to stop using their web pages to train AI with just one line of text. All four companies have agreed to follow the same opt-out rule in robots.txt, a file that tells web crawlers which parts of a website they can visit. This is important for anyone who publishes content online. Until now, each company needed its own rule to block access for AI training.
The token
The new signal is called `__usage_ai-training`. A website can use it with a Disallow rule to stop AI training. It can also leave some parts of the site open, such as a folder containing press releases. Normal search indexing will continue because this rule only applies to AI training.
Kevin Kelley from Anthropic wrote the proposal, reports Search Engine Roundtable. It is an individual Internet-Draft at the IETF, the group that maintains internet standards. This means it is still a draft and has not been accepted as an official standard.
Google's John Mueller said that if a website uses this token without mentioning Google-Extended, Google plans to treat it as if the website owner had blocked Google-Extended. Older rules for individual crawlers will continue to work too. This means website owners will still have to manage more than one setting. Websites that allow Google-Extended can still appear in Gemini answers and receive clicks.
The limits
The token cannot remove information that an AI model has already collected. According to AI Weekly, material already collected will stay in existing systems. The protection only applies to future collection.
The token is also a polite request, not legal protection. Nobody has independently tested whether crawlers will follow it. Legal arguments about scraping continue separately. For example, Wikimedia's argument in court says that using content for AI training can be fair, while copying articles is not.
There is one more detail: daily.dev notes that Microsoft's control over whether content is used to provide information in answers is separate from the opt-out for AI training.
The rollout
The companies will start supporting the shared rule on different dates:
- Anthropic: Support has already started. - OpenAI: Support is expected "in the coming months," but there is no exact date. - Microsoft: 1 February 2027. - Google: 2027.
Until then, website owners who want to opt out should keep their existing rules for individual crawlers. The shared token should be treated as an extra layer once each company starts supporting it. The announcement was made on 30 October 2026, and the draft could still change before it is finalized.