No./3 Deep-diving into deepfakes
How the AI face-swapping technology could make the move from viral videos to the big screen
IF YOU’VE BEEN on social media recently, you may have witnessed Tom Cruise taking time out of his busy schedule to play golf, perform magic, mimic a snapping turtle, and — in one instance — fall flat on his face. Or, at least, you thought you did.
“I’m still getting comments from people who think those videos are real,” laughs Chris Umé, a Belgian artificial-intelligence artist, and creator of the eerily convincing ‘Deeptomcruise’ Tiktok account. “I made them with a [Cruise] impersonator — they’re meant to entertain, but also show how rapidly the technology is evolving.”
Until recently, news coverage of ‘deepfakes’ (synthetic face replacements powered by AI) has tended towards either the quirkily trivial (face-swapping apps) or the irredeemably grim (fake news, hoax pornography). However, one area where the tech is making more positive waves is filmmaking. Last year, Disney unveiled a megapixel-quality deepfake demo, and Umé himself is one of several AI artists recruited by South Park’s Matt Stone and Trey Parker for the first ever deepfake movie studio: Deep Voodoo. The intention was to produce a feature-length deepfake comedy — a plan that Covid scuppered. Instead, a trial-run Youtube sketch was created, starring Peter Serafinowicz as ‘Fred Sassy’ — a preposterously bewigged reporter who bears a striking resemblance to Donald Trump.
“It’s all about data collection,” explains Jeffrey White, another Deep Voodoo staffer. “We feed the computer thousands of images [of Trump], so it can learn every aspect of his face and mimic Peter’s performance. It’s perfect for comedy — like a very realistic mask.”
Not just realistic: the process is also quicker and cheaper than other VFX techniques. “Think about Gemini Man,” says Umé, of the 2019 Will Smith actioner, in which the actor battles his younger self. “They used 3D modelling, a huge mo-cap rig — you probably had 20 people working on [ageing Smith down]. With deepfake, you could just collect data of a young Will Smith and train the computer to use it.”
“The possibilities are huge,” White agrees. “You could place an actor’s face over a stuntman’s, or shoot guerrilla-style without [legal] clearance, as you could face-swap people in the background.” One media company has even partnered with James Dean’s estate, with the (mildly unsettling) goal of having a deepfake Dean appear in new movies. “It’s scary but also cool,” Umé says. “Imagine an Edith Piaf biopic —
‘starring’ Edith Piaf!”
The options are plentiful, then, and Umé claims his inbox is “booming” with production companies keen to get in on the deepfake cinematic revolution. But would any major studio currently choose deepfake over traditional CGI? Framestore CCO Tim Webber, who won an Oscar for his VFX work on Gravity, tells Empire: “I’ve spoken about using it on a few lower-budget movies recently — for things like swapping an actress’ face onto a stand-in’s. But there are limitations to it. If you look at the [Fred Sassy] video, the actors are mostly talking to camera, as deepfake doesn’t work well in profile. With any machine learning [AI], what you gain in speed and cost, you lose in control. Building a CG face is complex and expensive, but you can tweak and finesse it in any way you like — which is what we do constantly on big Hollywood movies.”
Matt Johnson, VFX supervisor on Doug Liman’s Chaos Walking, is similarly cautious: “I can see it being used for de-ageing at some point. But so far, the only conversations I’ve had about deepfake are with actresses concerned about the less salubrious ways it could be used to manipulate their image without consent. So, it might need a rebrand if it’s to be seriously considered. The other worry is resolution: this stuff looks great on your phone, but would it hold up at the Empire Leicester Square?”
That is a question without a definitive answer yet — though Webber is confident it will get there eventually: “I can’t see this technology ever replacing traditional CG, but in a few years I’d say it will be a significant part of movie visual effects.” Whether deepfake actually graduates beyond Tom Cruise turtle impressions remains to be seen. But the next face you see on a screen may not be the one you think it is.