Software to weed out violence and porn

WASHINGTON: “She got caught up in the likes” was how a prosecutor described the 18-year-old Ohio teen accused of live- streaming the alleged rape of her 17-year-old friend. There is no question that live-stream apps, such as Periscope and Facebook Live Video, carry the risk of exposing audiences to heinous images. In rare but harrowing incidents, users have watched as suicide, rape or domestic violence unfold in real time before their eyes.

The job of watching and removing violent or pornographic content from these live apps, as well as video sites like YouTube, has primarily been a human undertaking. The sheer amount of content makes it a challenge for manpower alone. Now, artificial intelligence is poised to help with this task.

Dextro, a New York- based start-up, uses video recognition AI to easily search through content on live-stream apps. The technology or similar software could easily be used to weed out porn or violence.

Co-founder David Luan said the challenge lies in creating software that can interpret not just still images but moving images, audio and other “signifiers”.

“What is challenging about these videos is that they are much more complex than just a single picture. Even though it is a series of images all one after the other, there’s the motion element, the audio, so much of that gets thrown away if you just analyse image after image,” he said. “So you really need to treat it like a whole piece and analyse that.”

Dextro’s software can recognise objects and signifiers in a frame without human intervention, such as a gun in a potentially violent video. And AI’s recognition software can analyse a video within 300 milliseconds of posting.

Facebook confirmed the company does not use AI to filter out pornographic videos and declined to comment on whether software was being developed.

But in the aftermath of videos and live-streams of the deaths of Philando Castile and Alton Sterling at the hands of the police, the question of when censorship is ethical or appropriate poses a challenge for tech developers in the role of content moderator. How would a machine handle those videos, if it eventually takes over as the prime moderator? – Washington Post