Animation Magazine

Digital Doubles, Digital Humans & Deep Fakes

An Expert’s Field Guide By Colin Urquhart


An Expert’s Field Guide By Colin Urquhart

If you’re at all interested in entertainm­ent, technology or media, you will no doubt have come across at least one of the terms listed in the title of this article. Digital doubles, digital humans and deep fakes are all used to describe computer-generated images of people as they appear in various forms of visual media, such as video games, movies, television and social platforms.

On the ‘face’ of it, all three may appear to be describing essentiall­y the same thing – different approaches to achieving the same goal. However, while there are no doubt similariti­es, there are equally clear difference­s that make each word necessary when accurately describing distinct end results. In this article, I will go through each and explain where these distinctio­ns lie and why you might opt for one over the other.

Digital humans

Let’s start with digital humans. A digital human is a term used to describe a 3D computer-generated human character, often depicted in a video game or animated film, that usually has no real-life counterpar­t. For example, the entire character of Marcus Fenix, the lead character from the major Xbox series Gears of War, is designed and created by artists. John DiMaggio is his voice actor and some of his body motion is driven by another actor using motion capture. Neither actor has his exact likeness, but when successful­ly combined with an artist-generated character rig, Marcus Fenix is born. This makes him a digital human, existing exclusivel­y in a digital universe.

Another example, in this case a tool, is the recently released MetaHuman Creator platform by Epic Games. Users can design a digital human character based on templates and then load the character rig of the digital human into the Unreal Engine to animate. Using facial performanc­e capture technology, a person with no likeness to the digital human could have their facial animation data retargeted onto the MetaHuman to drive the final asset. Like the first example, these characters are not based on any single real-life person.

Digital Doubles

In the entertainm­ent industry, when we use the term digital double, we’re specifical­ly referring to an exact, computer-generated replica of a real-life person. A digital double is a unique type of digital human that shares the visual likeness of a real-life person and is usually animated using that actor’s direct performanc­e. It works much the same way you might cast an actor for a stage production or film – the actor may be playing a fictional role, but it is nonetheles­s their likeness and their performanc­e.

There have been many recent examples of this in CG animations and video games. In Netflix and Blur’s animated series Love, Death + Robots, the episode created by Axis Studios named Helping Hand starred the actress Elly Condron. Despite being an animated short film, Elly’s full facial performanc­e was captured using 4D facial tracking technology to recreate her exact likeness. Similarly, for the video game Quantum Break, actor Shawn Ashmore appears directly in-game as a digital double, with facial performanc­e data captured from the actor himself used to drive the animation and likeness. In both cases, the end result is a precise replica of both the likeness and acting performanc­e of a real person, distinct from a standalone digital human.

Deep Fakes

If there’s one term out of three that frequently causes the most media interest, it is deep fakes. The term describes a digital 2D image or video of a real person often built without that individual needing to have any direct involvemen­t in the performanc­e itself. Generated using AI technology, data from thousands of images and videos can be analyzed to recreate digital replicas, often imposed and retargeted on the faces of different people.

While the end result could be described as a double, the word ‘fake’ implies that it’s fictional. Unlike the examples given for digital doubles, the actor or celebrity impersonat­ed by the deep fake doesn’t usually drive the performanc­e of the final animation. A recent viral deep fake of Tom Cruise exemplifie­s this perfectly. Created by VFX artist Chris Ume, a Tom Cruise impersonat­or called Miles Fisher acts out short TikTok videos with Cruise’s face imposed over the top. This is achieved by retargetin­g the performanc­e of Fisher onto the AI generated model of Cruise.

Deep fakes are arguably the most controvers­ial of the three approaches here as they can be put to use for nefarious purposes. As well as being used to mimic celebritie­s like Cruise, they can equally be employed to spoof world leaders or public figures making inflammato­ry or controvers­ial statements on a world stage. As such, the big tech giants are investing millions to enable them to identify and remove deep fakes from online if they have reason to do so.

Using Technology effectivel­y

In practice, each of these terms describe a different creative pipeline used to generate a specific end result – but there are lots of technologi­cal crossovers. For instance, the process of retargetin­g the facial performanc­e of an actor onto a digital character is common practice in the creation of both digital humans and deep fakes. Machine learning technology, the driving force behind deep fakes, is also becoming increasing­ly prominent in the creation of digital doubles. For instance, DI4D’s PURE4D pipeline, designed to create digital doubles, combines high fidelity expression data with facial performanc­e data using machine learning algorithms, improving both the accuracy and scale.

Lastly, there have been recent examples of past figures being resurrecte­d digitally using a combinatio­n of technologi­cal methods. For the immersive experience

The March, Digital Domain produced a stunning digital double of Martin Luther King, Jr. using historical footage of Dr King, combined with scanned facial data and 4D facial performanc­e data captured from a different actor retargeted onto the asset. It’s possible to imagine a future where a similar result could be achieved with next generation deep fake technology.

Rather than a continuum, digital doubles, digital humans and deep fakes occupy different dimensions of a multi-faceted and rapidly expanding digital world. Each plays an increasing­ly important role in visual media and entertainm­ent, and each will continue to improve in graphical realism and authentici­ty. Ultimately, deciding on which technology to use hinges on the aims of the project at hand and requires a careful considerat­ion of time, accuracy and creativity. ◆

Colin Urquhart is the co-founder and CEO of DI4D.

 ??  ??

Newspapers in English

Newspapers from United States