PURE4D: Game-changing facial animation
PURE4D delivers world-leading facial animation by combining DI4D’S proprietary 4D facial capture technology with cutting-edge machine learning
Discover this cutting-edge facial capture technology
LA and Glasgow-based DI4D provides the highest fidelity facial performance capture for leading visual effects and pre-rendered animation projects. Facial performance capture is the process of filming the facial movement of actors, analysing that data and using it to drive the facial animation of digital characters in video games and movie visual effects. By driving the facial animation of a digital character with an actor’s performance, especially when the character is a digital double of the actor, it is possible to obtain a very high level of realism while remaining faithful to the actor’s performance.
Since its formation in 2003, DI4D has contributed extensively to the innovation of facial performance capture technology by pioneering the use of stereo-photogrammetry and ‘4D capture’. Stereo-photogrammetry is a technique that allows a ‘3D scan’ to be derived simply from two digital images taken from different positions, similar to the way that humans’ stereo vision allows us to sense depth. One of the advantages of stereophotogrammetry is that it can be used with synchronised video cameras to capture ‘3D scans’ at video frame rates.
DI4D has also developed a method to track a dense 3D base mesh through such per-frame ‘3D scan’ data, resulting in ‘4D capture’, with time being the fourth dimension. Tracking a mesh with thousands of vertices in 3D allows 4D capture to produce much higher fidelity facial animation than is possible with traditional marker-based or sparse facial feature-based approaches to facial performance capture.
DI4D has developed and introduced two different types of system that utilise stereo-photogrammetry for 4D facial performance capture. The first is the DI4D PRO system, which uses multiple stereo pairs of high-resolution video cameras to capture extremely high-fidelity data of a seated performer. The second is the DI4D HMC, which is a wireless, lightweight, head-mounted stereo camera system that is designed to allow facial capture from several actors simultaneously while they move and act naturally together.
INTRODUCING PURE4D
DI4D’S new PURE4D solution now allows it to extend its services to much larger projects, such as in-game facial animation,
“THE LEVEL OF REALISM SEEN IN RECENT PRE-RENDERED CINEMATICS IS NOW BECOMING THE BENCHMARK FOR IN-GAME ANIMATION” Colin Urquhart, CEO of DI4D
while maintaining the highest possible level of realism. PURE4D delivers animation that is completely faithful to the actors’ original facial performances by capturing them using DI4D HMCS and processing them with DI4D’S proprietary 4D capture solution. It further enhances the detail and fidelity of the 4D performance data by combining it with even higher fidelity 4D expression data captured using a DI4D PRO system. Finally, PURE4D utilises advanced machine learning technology to increase dramatically the efficiency with which large volumes of data can be processed.
By combining DI4D HMC performance capture, DI4D PRO expression capture, and highly efficient machine learning, PURE4D takes facial performance capture to a new level. As a result, DI4D can provide high-fidelity facial performance capture and post-processing services to all sizes of projects. PURE4D is particularly effective for digital doubles, allowing the facial performance of the actor to be replicated with absolute precision, and ensuring that every subtlety and nuance of the actor’s performance is represented in the subsequent animation.
HOW IT WORKS
The PURE4D pipeline begins with the DI4D PRO system. Comprising nine synchronised 12-megapixel machine vision cameras, the DI4D PRO system captures sequences of the high-fidelity, colour 4D facial performance data, using standard video lighting without markers, make-up or structured light projection. PURE4D uses the DI4D PRO system to acquire highfidelity 4D facial expression data from
each actor. This often includes the actor performing phonetic pangrams, which are special phrases designed to cover all spoken sounds and the full range of mouth shapes. All of this expression data helps provide the building blocks for an accurate digital recreation, tracking and storing the full range of possible facial expressions and mouth movements.
Next up is a session with the headmounted DI4D HMC system – a lightweight, easily adjustable head rig fitted with two small, synchronised highresolution digital video cameras, onboard LED lights, a compact video recording system and a battery pack. The HMCS are used to capture 4D facial performance data, often from multiple actors, simultaneously with body motion capture and sound recording. This is designed to ensure that body, face and voice performances are all synchronised, essential for ensuring active scenes achieve highly realistic performance results. This also allows directors and developers to capture emotional, facial nuance alongside the physical performance of the body, which will provide a greater level of authenticity.
Leading-edge, machine learning algorithms efficiently combine both sets of data to significantly increase the scalability of the solution. By combining the data in this way, studios can acquire 4D data with the same high level of fidelity as the DI4D PRO system, all while the actors move freely within a motion-capture volume – something that no other portable headmounted system can achieve. The machine learning system refines and improves continuously as more performance data is added which, in turn, significantly increases scalability – there’s no project too big.
The DI4D team provides a complete service from start to finish, including the capture and then subsequent processing of 4D data with its 4D reconstruction pipeline. The service is designed this way to fit into any major studio’s pipeline without any extensive training or infrastructure expansion required.
SAY GOODBYE TO COMPLEX ANIMATION RIGS
One of the game-changing new features now available with the launch of PURE4D is its unique approach to driving facial animation. It doesn’t require a traditional facial animation rig as part of the facial capture process – significantly reducing both complexity and cost from the delivery of any project. The pipeline is designed to faithfully capture an actor’s facial performance in its purest and exact form, without any artist corrections or enhancements required.
For example, during the production of many movies and video games, studios will often capture an actor’s performance on set with the expectation that artist corrections (or ‘polish’) will be needed to reach the desired level of realism in the final animation. While entirely non-human, fictional characters usually still require this extra layer of animation, the vast majority of video games and movies that require digital human characters can remove
“WE BELIEVE THAT PURE4D IS A REAL GAME CHANGER FOR FACIAL ANIMATION IN VIDEO GAMES” Colin Urquhart, CEO of DI4D
this step altogether. Instead, PURE4D is designed to make it far more efficient to use digital doubles, driving their animation directly with the facial performance of their real-life counterparts.
This has the potential to dramatically change traditional production pipelines, shifting focus away from post and back towards the performance itself. It’s ideal for any studio opting to create exact digital doubles for any large-scale media production project, saving time while improving fidelity.
IN-GAME ANIMATION? NO PROBLEM
The PURE4D pipeline can be used for both in-game animation and cinematics without any drop in the quality of the facial animation displayed. The advent of the next generation of gaming consoles has raised the bar for in-game animation standards, with the same level of fidelity seen in cinematic sequences now expected for ingame graphics.
Every nuance and subtle expression from the actors’ facial performance is captured and transferred faithfully to the in-game animation of their digital double characters. It’s the perfect solution for narrative-driven experiences, attaining the same level of realism found in Hollywood movies.
SCALE-UP
PURE4D is designed to meet the requirements of large-scale next-gen game projects that require thousands of seconds of the highest-quality facial animation. The machine learning algorithm serves to facilitate scaling beyond the capabilities of previous DI4D services and cater to higher production demands of triple-a projects.
PURE4D is a huge step forward for DI4D, combining almost 20 years of technological advancements and experience into an efficient, high-quality pipeline for game developers and filmmakers to achieve photoreal digital doubles.
Find out more about PURE4D at di4d.com/pure4d