Google Flags 50k Likely AI-Generated Child Abuse Files in 30 Days

Google has developed a new system that flags likely AI-generated child abuse files on its platforms. During a 30-day period, the system identified around 50,000 such files across Google and YouTube, representing about 6% of all files analyzed.

The labels are sent to the National Center for Missing & Exploited Children (NCMEC), giving analysts and law enforcement an early indication of whether material may have been created using AI. However, the system does not make a final determination of whether a file depicts real-life or AI-generated harm.

John Shehan, Senior Vice President at NCMEC, said that being able to distinguish likely AI-generated content gives analysts and law enforcement valuable context, helping them prioritize cases involving real children who may be in immediate danger.

The new system is part of a wider update to Google's work with law enforcement agencies. A pilot with Costa Rica's Organismo de Investigación Judicial aims to reduce the time required for forensic analysis by 80-90%, while also limiting investigators' exposure to child sexual abuse material.