🔗 Share this article

Tech firms and child protection organizations will be granted authority to evaluate whether artificial intelligence tools can produce child abuse material under new UK laws.

Substantial Increase in AI-Generated Harmful Material

The declaration came as revelations from a protection monitoring body showing that cases of AI-generated CSAM have more than doubled in the past year, rising from 199 in 2024 to 426 in 2025.

New Regulatory Structure

Under the amendments, the government will permit approved AI companies and child safety groups to inspect AI models – the underlying technology for conversational AI and image generators – and verify they have sufficient safeguards to stop them from creating depictions of child exploitation.

"Fundamentally about stopping abuse before it happens," stated the minister for AI and online safety, noting: "Specialists, under strict protocols, can now identify the danger in AI models early."

Tackling Regulatory Obstacles

The changes have been introduced because it is against the law to produce and own CSAM, meaning that AI developers and others cannot create such images as part of a testing regime. Previously, authorities had to delay action until AI-generated CSAM was published online before addressing it.

This law is aimed at averting that problem by enabling to stop the production of those images at source.

Legal Framework

The changes are being added by the government as modifications to the criminal justice legislation, which is also establishing a ban on possessing, creating or sharing AI systems developed to generate child sexual abuse material.

Practical Consequences

This week, the official visited the London headquarters of Childline and heard a simulated conversation to advisors involving a report of AI-based exploitation. The call portrayed a teenager requesting help after being blackmailed using a explicit deepfake of themselves, constructed using AI.

"When I hear about children experiencing extortion online, it is a cause of intense anger in me and rightful concern amongst parents," he said.

Alarming Statistics

A leading internet monitoring organization stated that instances of AI-generated exploitation material – such as online pages that may include multiple images – had more than doubled so far this year.

Cases of category A content – the most serious form of exploitation – increased from 2,621 visual files to 3,086.

• Girls were predominantly victimized, accounting for 94% of illegal AI images in 2025
• Depictions of newborns to toddlers rose from five in 2024 to 92 in 2025

Industry Reaction

The law change could "constitute a crucial step to guarantee AI products are secure before they are launched," stated the head of the online safety organization.

"Artificial intelligence systems have enabled so survivors can be targeted repeatedly with just a few clicks, providing offenders the capability to create potentially endless amounts of sophisticated, photorealistic child sexual abuse material," she continued. "Content which additionally commodifies survivors' trauma, and renders children, particularly girls, more vulnerable both online and offline."

Counseling Interaction Data

The children's helpline also published information of counselling interactions where AI has been mentioned. AI-related risks mentioned in the conversations include:

• Employing AI to evaluate body size, physique and looks
• AI assistants discouraging young people from consulting trusted adults about harm
• Facing harassment online with AI-generated material
• Digital blackmail using AI-faked images

Between April and September this year, the helpline delivered 367 counselling sessions where AI, chatbots and related terms were discussed, four times as many as in the same period last year.

Half of the mentions of AI in the 2025 sessions were connected with mental health and wellbeing, including utilizing chatbots for assistance and AI therapy apps.