The secrets behind 3d scanning
Professional 3D scanning company, Pixel Light Effects, reveals its photogrammetry process for getting actors into the digital realm
Professional 3D scanning company, Pixel Light Effects, reveals its photogrammetry process for getting actors into the digital realm
Whenever you see a comic-book hero flying through space or crashing through a wall, chances are they’re a digital double of a real actor expertly crafted by a visual effects studio.
But how is an actor’s likeness captured with such absolute photorealism? These days that’s often the domain of 3D scanning providers, which rely on portable – and often custom – photogrammetry rigs that they can bring right onto the set. That way they can whisk actors in and out of the scanning process and generate a CG model of their face and body as fast as possible, helping save the film studio a lot of expensive production time.
One company in the 3D scanning services fold is Vancouver-based Pixel Light Effects. Using its mobile photogrammetry setup, the studio recently scanned principal actors and extras on location in British Columbia for Matt Reeves’ War
for the Planet of the Apes, creating detailed CG models to give to Weta Digital for the subsequent digital double work.
3D World asked Pixel Light Effects, which also has a presence in Beijing, how it tackles a typical actor photogrammetry scan – from the capture process right through to producing a useable highresolution CG model.
The basis of a 3D actor scan
Over the years, several methods have been used to accurately capture the essence of an actor in CG. These include laser scans, specialised ‘light stage’ contraptions, and now most commonly, photogrammetry. This process essentially involves taking hundreds of photographs of the actor from multiple angles. The photos are fed into computer software, which then sets about comparing the images and using them to build a 3D model.
Photogrammetry is considered an efficient method of capture because it is quick, especially in a camera rig that has many cameras at many angles taking all the
pictures at the same time. Indeed, that’s what Pixel Light Effects uses, and further mobilises the scan by having it take place in the back of a truck that has the photogrammetry rig permanently installed inside. The truck can be driven directly to the set where the actors are, or just about any other location.
“Mobility is the biggest issue with a rig consisting of hundreds of cameras,” says Pixel Light Effects’ CEO, Jingyi Zhang. “Taking them apart and putting them back together is very time-consuming and labour intensive. It just doesn’t feel right. However, due to the tight schedule of the production, sometimes it’s just impossible to have the talent come to us. Being able to perform the service on set is very essential.”
What’s in the truck?
Pixel Light Effects’ photogrammetry rig, contained in a modified Mercedes-benz Sprinter van, is made up of 144 Canon DSLR cameras. These are all synchronised using a proprietary ‘Camera Hub’ device, which supplies power and triggers 16 cameras at once. “It’s designed to be daisy-chained together, so we use multiple devices to trigger the larger array,” explains Zhang.
Lighting is an essential part of the photogrammetry rig. For this, multiple flashes and bounce light ensure that there is enough depth of field in the resulting images, and that the actor is lit evenly. The van interior is also decked out
“Everyone who does photogrammetry knows you just can’t have Enough cameras” Jingyi Zhang, CEO, Pixel Light Effects
in white and the environment is calibrated so that it is as diffuse as is possible.
The camera positions inside the rig were R&D’D by Pixel Light Effects for several months before an optimal layout was reached. “Everyone who does photogrammetry knows you just can’t have enough cameras,” notes Zhang. “You always want more. Having a constrained budget and space means we must have just enough coverage at every angle.”
Inside a scanning session
When an actor comes into the truck for a scan, Pixel Light Effects typically has just two technicians running the scanning session. First they will ask the actor and on-set supervisors if it is okay for the talent to wear a hair net. “This gives a more accurate skull shape, which will be helpful down in the pipeline,” explains Zhang. However, it may be crucial for an actor to retain their exact wardrobe, such as a helmet or a faux hairpiece.
For a full-body scan (Pixel Light Effects also carries out detailed facial scans – see Facetime boxout), the actor will be instructed to make an A-pose, with their elbows and knees slightly bent for rigging purposes. The actor’s face is typically posed in a neutral way and then aligned with extra witness cameras.
The actual capture is like taking a photo, again highlighting the benefits of a photogrammetry rig. “The capture itself is as fast as taking a photo at 1/1000th of a second,” says Zhang. “All the cameras are synchronised, which means we can scan animals – who of course don’t stay still – as long as they fit in the capture volume.”
Scan and deliver
The resulting photographs are simply raw images, but they are
“the capture is as fast as taking a photo” Jingyi Zhang, Pixel Light Effects