Memorandum
- From
- Caroline Mercer via FOX News
- Date
- Filed
- Society·4 min to read
- Re
OpenAI launches classifier to flag AI-generated child sexual abuse material
ReOpenAI launches classifier to flag AI-generated child sexual abuse material
OpenAI has introduced a classifier designed to detect suspected child sexual abuse material, including AI-generated content that existing hash-based systems may miss. The tool was developed with the National Center for Missing & Exploited Children and is offered free to vetted organizations.
OpenAI has launched a classifier aimed at detecting suspected child sexual abuse material, with a focus on AI-generated content rather than only previously catalogued files. The company is offering the tool free through its API to vetted organizations, and it was developed with the National Center for Missing & Exploited Children, known as NCMEC.
Most traditional detection systems rely on hashes, which match images against databases of known abusive files. That approach struggles with newly created material, which is why the new classifier scores images so human reviewers can prioritize the most urgent cases. OpenAI says the tool flags 48% fewer images while maintaining similar abuse detection, and that flagged images are about twice as likely to contain abuse. The practical goal is to direct limited reviewer time toward the material most likely to involve a child who needs immediate protection, rather than spreading attention across a larger pool of uncertain flags.
Results cited by the company come from OpenAI and its partners. Fal reports that 72% of high-confidence flags were confirmed as new child sexual abuse material, while SafetyKit reports detecting 90% of sexualized drawings or animations of children. Those figures cover different kinds of content, so they should be read as separate measurements rather than a single combined rate. Independent testing of these figures has not yet been completed, and the company's own numbers have not been verified by outside researchers.
OpenAI has also changed how ChatGPT responds to attempts to find abusive material. Users who try it now see warnings, and violations can lead to removal of the conversation and a 24-hour pause on account access. Repeated violations may result in longer bans, account closure and, where warranted, reports to authorities. The warnings were developed with child protection groups and researchers, but their effectiveness has not been proven, and the company has not published data showing how often they deter users from continuing such attempts.
Additional measures extend beyond its own platform. OpenAI has joined the Tech Coalition's Lantern program, a cross-platform information-sharing initiative that lets companies share signals about abuse so that activity pushed off one service does not simply reappear on another. The company has also released an investigative tool that helps authorized agencies review lengthy chat logs. Access to that tool depends on agency authorization, which limits who can use it and keeps review tied to legal process. OpenAI is publishing developer guidance on detecting and reviewing suspected abuse as well, so that other builders can apply similar safeguards.
The effort comes as reporting to NCMEC involving generative AI has grown rapidly. The organization received more than 400,000 reports involving generative AI in 2025. Those reports are not a direct count of abuse, because not all of them involved the creation or sharing of AI-generated abuse material, and the figure therefore reflects a wider range of concerns flagged to the clearinghouse.
42
