OpenAI has introduced a new set of child-safety measures designed to help organizations detect child sexual abuse material (CSAM), prevent the misuse of artificial intelligence to create abusive content, and support investigations into online child exploitation. Announced on November 16, 2026, the measures include a new image classifier, warning messages and temporary account suspensions in ChatGPT, developer guidance, and additional resources for law enforcement.

The announcement comes as generative AI creates new challenges for online safety organizations. AI tools can produce previously unseen abusive imagery, making it harder for traditional detection systems that rely on matching known images to identify harmful material. OpenAI says its latest initiative is intended to help safety teams identify potential abuse more efficiently while strengthening protections across its products and the wider technology industry.

OpenAI Introduces a New CSAM Image Classifier

The central element of the announcement is the OpenAI CSAM Image Classifier, an image-analysis tool designed to identify potential child sexual abuse material, including images that may have been generated or modified using artificial intelligence.

Unlike systems that primarily identify previously documented images through matching techniques, the new classifier analyzes visual characteristics to identify potentially abusive material that may not have appeared in existing databases. It produces risk scores that help trained reviewers prioritize images requiring further investigation.

OpenAI describes the tool as the first broadly available CSAM classifier with a dedicated signal indicating whether suspected abusive material may be AI-generated.

The company developed the classifier with assistance from the National Center for Missing and Exploited Children (NCMEC), a US nonprofit that operates the CyberTipline, a central reporting system for suspected online child exploitation.

OpenAI plans to make the classifier available at no cost through its API to vetted organizations. Access will require approval, allowing child-safety organizations and other eligible services to incorporate the technology into their existing review processes.

Early Tests Show a Potential Reduction in Review Work

One of the main challenges facing child-safety teams is the volume of images that must be examined. Reviewing large numbers of flagged images requires trained personnel, and false positives can consume time that investigators could otherwise spend examining genuine cases.

OpenAI says early evaluations suggest its classifier could reduce this workload.

In an internal evaluation involving user-uploaded images, the company estimated that the new classifier could flag 48% fewer images while identifying roughly the same number of abusive images as its existing protection system. OpenAI also reported that images flagged by the new classifier were approximately twice as likely to contain abuse.

Early partners have reported additional results:

OrganizationReported evaluation result
OpenAI internal evaluation48% fewer images flagged while finding roughly the same number of abusive images
fal.aiReviewers confirmed 72% of high-confidence flags as novel CSAM
InVideo AIOne in four flagged cases led to a ban
SafetyKitDetected 90% of tested cases involving drawings or animations that sexualize children

These findings are early evaluation results, not a guarantee of performance across every platform. Different services handle different types of content, so organizations will need to evaluate the classifier against their own material and review processes.

How the Classifier Works

OpenAI’s model documentation describes an architecture combining an image encoder based on CLIP with a three-layer machine-learning classifier.

The image encoder converts visual information into a numerical representation. The classifier then uses that representation to produce separate risk scores for suspected CSAM and suspected AI-generated CSAM.

The API exposes two signals:

  • Likely CSAM: Indicates that an image may contain child sexual abuse material.
  • Likely AI-generated CSAM: Indicates that suspected abusive material may have been generated using AI.

These scores are intended to help organizations prioritize content for further examination. They do not establish whether an image is illegal, confirm its origin, or independently determine whether it must be reported.

OpenAI also emphasizes that the classifier is not a replacement for existing detection systems. It is designed to complement known-image matching, contextual and behavioral detection, and trained human review.

The company has identified limitations that organizations must consider. The classifier can produce false positives and false negatives, and its performance may vary depending on the images processed by a particular service. OpenAI has also stated that it has not tested the model on video frames, meaning its published image results should not be treated as evidence of video-detection performance.

ChatGPT Will Show Warnings and Temporarily Suspend Accounts

Alongside the image classifier, OpenAI is introducing new interventions within ChatGPT to discourage attempts to find or access material that sexualizes children.

When ChatGPT identifies a request seeking abusive material, it can interrupt the conversation and display a warning explaining the harm associated with child sexual exploitation. The warning also makes clear that AI-generated material depicting the sexualization of children is harmful.

The intervention includes several measures:

  • ChatGPT displays a warning explaining the seriousness of the behavior.
  • The flagged conversation is removed and cannot be continued.
  • The account may be suspended from ChatGPT for 24 hours.
  • Specialist support resources are provided to help people stop harmful behavior.

Repeated violations can result in longer restrictions, permanent account closure, or reports to relevant authorities.

OpenAI says it developed the warning approach with input from organizations and researchers working to prevent child sexual abuse. These include Protect Children, the Lucy Faithfull Foundation, and academic advisers.

The company says the objective is not simply to block prohibited requests. It also wants to interrupt harmful behavior and direct people toward specialist support.

New Guidance for Developers Building AI Products

OpenAI is also publishing guidance for developers using its technology to build applications and services.

The guidance explains how developers can prevent, identify, and respond to child sexual abuse material and exploitation-related activity. It covers the use of detection tools, account-level enforcement, reporting procedures, and support for teams responsible for reviewing sensitive content.

Developers are encouraged to combine several layers of protection rather than depend on a single classifier. These can include known-image matching systems, AI-based detection, behavioral monitoring, and trained human reviewers.

The new image classifier can be used alongside OpenAI’s Moderation API and other available child-safety tools. OpenAI is also working with organizations including the Tech Coalition and ROOST to help eligible services access detection technology.

The company has additionally participated in collaborative work with Anthropic and Google to develop guidance for responding to disclosures of child sexual exploitation and abuse involving AI systems.

Additional Resources for Law Enforcement

OpenAI’s announcement also includes support for law enforcement agencies investigating child exploitation.

The company is providing authorized investigators with access to AI-powered capabilities, investigative resources, and training intended to help them analyze complex information and work with digital evidence.

Access to these resources will depend on agency authorization and program requirements. OpenAI also plans to provide practical training and a field guide to help investigators understand how AI tools can support child-safety investigations.

These efforts reflect a broader industry challenge: technology companies, child-protection organizations, and law enforcement agencies need ways to share information and respond to emerging forms of online exploitation.

The Bigger Picture

The development highlights how generative AI is changing the child-safety landscape. Traditional image-matching systems remain important for identifying known abusive material, but AI-generated and altered images create new detection challenges. Tools that can identify previously unseen material may help organizations respond to those challenges, although they cannot eliminate the need for human investigation.

The scale of the problem is also putting pressure on reporting and investigation systems. The National Center for Missing and Exploited Children received approximately 400,000 reports involving AI-generated child sexual abuse material in 2025, compared with around 67,000 in 2024, according to figures cited by The Verge. A growing volume of reports makes efficient prioritization increasingly important.

OpenAI’s approach combines technical detection with product-level restrictions, developer guidance, and cooperation with outside organizations. Its effectiveness will depend on how well these measures perform in real-world conditions and how widely eligible organizations adopt them.

Looking Ahead

OpenAI’s new classifier could give child-safety organizations another way to identify previously unseen abusive images, including material created with generative AI. Offering the technology free to vetted organizations may also lower the cost of adopting advanced detection systems. However, independent evaluation, careful human review, and cooperation between platforms will remain essential to establishing its reliability.

The new ChatGPT warnings and account restrictions represent a separate but complementary effort to prevent misuse before it escalates. As AI-generated content becomes more sophisticated, technology companies will face growing pressure to demonstrate that their products include effective safeguards. OpenAI’s latest measures are an important addition to that effort, but their long-term impact will depend on implementation, transparency, and measurable improvements in child protection.

Get the day’s top stories in your inbox

One concise email. No spam, unsubscribe anytime.