‘Disturbingly realistic’ AI child sex abuse material seized in NZ
The rise of artificial intelligence (AI) is bringing fresh challenges for law enforcement around the world, with computer-generated child sex abuse becoming an increasing problem.
And now New Zealand police, Customs and the Department of Internal Affairs (DIA) have also started seizing “disturbingly realistic” material.
Last year, the Internet Watch Foundation (IWF) produced a report into how AI is increasingly being used to create child sex abuse material.
In the report, the IWF found 20,254 AI-generated images posted to one dark web forum in a month. More than half were found to be criminal.
While AI child sex abuse material represents a small portion of the vast numbers of “real” child sex abuse material, the AI created abuse is realistic enough to be treated as “real”.
The most convincing AI generated material is indistinguishable from real child sex abuse material, even for trained IWF analysts, the report said.
The report found there was reasonable evidence that AI created abuse also has the potential for re-victimisation of known child sex abuse victims.
Famous children and children of known perpetrators are also at risk.
“The same holds for celebrity children – just as the IWF has for a long time seen many examples of ‘shallowfake’ and deepfake images featuring these well-known individuals, now the IWF is seeing entirely AI-generated images produced using finetuned models for these individuals.”
The IWF found many users of the dark web on these forums are requesting certain images.
“Time will tell whether this trickle becomes a flood,” the IWF report said.
The IWF is concerned about the potential for rapid growth
Chief customs officer for the child exploitation operations team, Simon Petersen, said Customs has seized AI-generated child sexual exploitation material in the last year as part of collections for offenders who have been arrested for import or export-related offending.
“As with any technological development which enables the victimisation of our most vulnerable, this concerns us greatly,” Petersen said.
The DIA’s digital child exploitation team manager, Tim Houston, said the department has also investigated offenders in New Zealand who have been found with computer-generated child sexual abuse material which appears “disturbingly realistic”.
There’s also offenders in New Zealand who are creating the abuse.
“AI generated content is becoming easier to create, which normalises and encourages the physical abuse of children,” Houston said.
“It takes significant resource and time to determine whether a child is AI-generated or not and takes our investigators away from identifying real-world children who are at risk.”
Detective senior sergeant Kepal Richards, officer in charge of the Online Child Exploitation Across New Zealand team (OCEANZ), said offenders are also able to commit sextortion without needing to acquire real explicit material from victims.
The evolution of technology means AI child sex abuse material is becoming increasingly realistic and convincing, Richards said.
How to report sextortion:
■ If you are in immediate danger ring 111.
■ Report the content to the platform, such as Facebook, Instagram or Snapchat and request it to be removed.
■ If you are under 18, and if you have copies of the material the blackmailer is threatening you with, you can use a free tool called Take It Down.
■ If you are 18 or older contact StopNCII.
■ Contact Netsafe on 0508 NETSAFE and the police via 105.