AI synthetic media tech enters perilous phase
"Do you want to see yourself acting in a movie or on TV?" said the description for one app on online stores, offering users the chance to create AI-generated synthetic media, also known as deepfakes.
"Do you want to see your best friend, colleague, or boss dancing?" it added. "Have you ever wondered how would you look if your face swapped with your friend's or a celebrity's?"
The same app was advertised differently on dozens of adult sites: "Make deepfake porn in a sec," the ads said. "Deepfake anyone."
How increasingly sophisticated technology is applied is one of the complexities facing synthetic media software, where machine learning is used to digitally model faces from images and then swap them into films as seamlessly as possible.
The technology, barely four years old, may be at a pivotal point, according to Reuters interviews with companies, researchers, policymakers and campaigners.
It's now advanced enough that general viewers would struggle to distinguish many fake videos from reality, the experts said, and has proliferated to the extent that it's available to almost anyone who has a smartphone, with no specialism needed.
"Once the entry point is so low that it requires no effort at all, and an unsophisticated person can create a very sophisticated non-consensual deepfake pornographic video-that's the inflection point," said Adam Dodge, an attorney and the founder of online safety company EndTab.
"That's where we start to get into trouble."
With the tech genie out of the bottle, many online safety campaigners, researchers and software developers say the key is ensuring consent from those being simulated, though this is easier said than done. Some advocate taking a tougher approach when it comes to synthetic pornography, given the risk of abuse.
Non-consensual deepfake pornography accounted for 96 per cent of a sample study of more than 14,000 deepfake videos posted online, according to a 2019 report by Sensity, a firm that monitors synthetic media. It added that the number of deepfake videos online was roughly doubling every six months.
ExoClick, the online advertising network that was used by the "Make deepfake porn in a sec" app, told Reuters it was not familiar with this kind of AI face-swapping software. It said it had suspended the app from taking out adverts and would not promote face-swap technology in an irresponsible way.
"This is a product type that is new to us," said Bryan McDonald, ad compliance chief at ExoClick, which like other large ad networks offer clients a dashboard of sites they can customise themselves to decide where to place adverts. "After a review of the marketing material, we ruled the wording used on the marketing material is not acceptable.”