OpenAI adds ChatGPT abuse warnings and free child-safety detection tool
OpenAI's new ChatGPT intervention removes flagged conversations, suspends access for 24 hours and directs users to specialist support.

OpenAI's new ChatGPT intervention removes conversations that are flagged, blocks access for 24 hours and points users towards specialist support.
OpenAI is adding warnings and temporary account suspensions to ChatGPT. It is also offering a free image-checking tool to help safety teams find possible child sexual abuse material (CSAM). This includes images that have not been identified before and images made using AI.
The announcement also includes advice and tools for developers and specialist law enforcement investigators. Only approved organizations that pass checks can use the image-checking tool through the OpenAI API. Investigative tools will also require approval from the relevant agency and must meet the program's rules.
ChatGPT warnings interrupt harmful behaviour
The new ChatGPT warning interrupts attempts to find abusive material. It explains the harm involved and directs users to specialist support. OpenAI says it will remove the flagged conversation and block the user's access for 24 hours. Support information will remain available during this time.
The company says that people who break the rules repeatedly may face longer restrictions, account closure or reports to the authorities. This new measure builds on existing rules that ban CSAM and safety systems that refuse requests for such material.
Groups and academic experts who helped develop the approach include Protect Children, the Lucy Faithfull Foundation, Dr. Caoilte Ó Ciardha, Researcher and Honorary Reader in Forensic Psychology at the University of Kent, and Joel Scanlan at the University of Tasmania. Ó Ciardha and Scanlan separately studied a trial of warning messages designed to discourage harmful behaviour.
Scanlan, Associate Professor at the CSAM Deterrence Centre, explains how this knowledge helped shape the new intervention: "OpenAI has worked with several outside experts who have already designed and tested warning messages. It developed the intervention with them using research-based methods. It also aims to direct users to local support services in as many places as possible."
Image-checking tool aims to find images not identified before
OpenAI developed the image-checking tool with help from the National Center for Missing & Exploited Children (NCMEC). It looks for visual signs of possible abuse and gives reviewers scores to help them decide which images to check first. It also gives a separate signal showing whether suspected material may have been made using AI.
OpenAI's own tests suggest that the tool could reduce the number of images that need to be checked. In a test using images uploaded by users, the company estimates that its tool would flag 48% fewer images while finding about the same number of abusive images as its current tool. Images flagged by the new tool were about twice as likely to contain abuse.
These results come from OpenAI's own testing, not an independent study. Early partners have carried out further tests. In fal's test of real images checked by people, reviewers confirmed that 72% of images given high-confidence flags were previously unidentified CSAM.
John Shehan, Senior Vice President at NCMEC, says: "This new image-checking tool gives more organizations access to technology that can help find CSAM that has not been identified before, improve reporting and provide important information to people working to identify and protect victims."
OpenAI has worked with the Tech Coalition and ROOST to help make the tool available. It has also started taking part in the Tech Coalition's Lantern program. This program allows platforms to share information about online child sexual exploitation and abuse.
Guidance for developers and training for investigators
New guidance for developers explains how to prevent abuse, let users report it, find harmful material, review reports and report suspected abuse. The guidance was developed with help from the Tech Coalition, Thorn, NCMEC and the Internet Watch Foundation. It also explains how to support people who review disturbing content.
Separately, OpenAI, Anthropic and Google jointly asked the Tech Coalition to prepare guidance on how AI systems should respond when users reveal child sexual exploitation or abuse.
For specialist law enforcement teams, the first investigative tool is designed to examine long chat records and information received through legal requests. It can look for possible links between cases and create timelines. Its findings will be connected to the original source material.
The first training sessions are planned to include an overview and demonstration of the tool, along with a practical guide for investigators. Program administrators will control who can use it. More training is also planned for regional, national and international conferences.
