HOW DNEG DOES… ENVIRONMENTS
Our six-part look behind the scenes at DNEG now ventures into how the visual effects studio handles complex 3D environments
Part 4 of our behind-the-scenes series at DNEG, this month we discover how the studio handles complex CG environments
Films transport us to different spaces, different locations and different worlds. And it’s often a VFX studio’s job to craft that world as a digital environment.
At DNEG, the environments department – known as the Envgen department – brings to life various places for visual effects scenes, sometimes as completely synthetic locations, or as augmentations of existing photography. “The department is very wide, and goes from very creative digital matte painters, to very technical technical directors, with everyone in between capable of doing more generalist work,” outlines DNEG global head of digital matte-painting Ludovic Iochem. “It’s because people are so different, but together are really complete, that the Envgen department is a toolbox capable of achieving pretty much every possible challenge.”
The tools of the Envgen team also vary greatly, but generally include a Photoshop and Nuke workflow. More 3D tasks are typically carried out in Maya, Clarisse, Houdini and Substance (Iochem also notes that Unreal Engine is finding a place more and more often in the early stages of Envgen production). In this breakdown of how DNEG does environments, we explore work undertaken by the Envgen team for Fantastic Beasts: The Crimes Of Grindelwald, Mission: Impossible – Fallout and Venom.
A CEMETERY CONSTRUCTION
In the climax of The Crimes Of Grindelwald, both sides from the wizarding world gather at Père Lachaise cemetery to hear from dark wizard Gellert Grindelwald (Johnny Depp), before a showdown breaks out. The location was actually a combination of three cemeteries in real life – Père Lachaise, Brompton and Highgate – with DNEG merging the most appealing areas from each.
“We’re talking about hundreds of tombs, over 150 foliage species in strong wind conditions, and to make it a bit more challenging, everything needed to be destroyed by several fire creatures,” says DNEG environment supervisor Nacho Thomas. “We had some plate elements, but had to discard a few as the lighting emitted by the creatures needed to fit with the CG renders. So, from the beginning we planned to do a full-cg, destruction-friendly environment.”
Building the cemetery began with LIDAR and photographic reference acquired by DNEG’S ‘shoot and capture’ team, providing Envgen with HDRIS, drone footage, reference images and photogrammetries. “The main challenge was, of course,
plants,” says Thomas. “Foliage, trees, ivy, dead leaves – it is not possible to capture them with survey techniques, and they are everywhere, so capturing and cleaning the data was much more difficult for this film.”
The 3D build process relied heavily on Houdini, with the assets needing to be destructioncompatible and ‘light-animation friendly’. Thomas notes that the team spent significant time preparing several OTLS to destroy assets, burn foliage and scatter rubble. “But all the time spent in building these tools really paid back when we had to do shots,” he says. “After that point, artists could concentrate on creatively dressing each shot with easy tools.”
“We also developed a burned texture tool in post to keep textures consistent with the amount of destruction of the cemetery,” adds Thomas. “This directly linked the textures to the destruction and fire coverage, so we didn’t have to paint specifically each shot. Render times were so huge that we didn’t have the option of re-doing them that much, so getting things right before doing a full 4K render was vital.”
Wind and vegetation in the sequence were some of the biggest challenges, as each shot had a specific wind strength and direction. DNEG’S setup contained hundreds of variations, resulting in a complex foliage system. “Mixing this with the burning process of trees was definitely one of the things that gave us more headaches,” says Thomas. JUMPING INTO PARIS Mission: Impossible – Fallout features a daring halo jump sequence in which Ethan Hunt (Tom Cruise) and August Walker (Henry Cavill) launch themselves out of a military plane over Paris and land on the Grand Palais des Champs-élysées. While actual Halo jumps were filmed over Abu Dhabi, Parisian environments and the landing site were synthetic creations by the Envgen group.
DNEG began by building from the Grand Palais outwards. Helicopter footage of Paris was the reference point, and from it artists used photogrammetry to make a 3D representation of the city. “As the shot rotated through so many different angles at ever-changing heights, it was important to have a setup that was flexible enough to work wherever our camera may be pointing,” details DNEG environment generalist William Foulser. “This became even more crucial when trying to stitch together the matchmove cameras, and position them in a way that gave us a satisfactory blend when Tom Cruise lands on the Grand Palais. For these reasons, it wasn’t possible to have a purely 2.5D approach, so a hybrid approach was used – filling gaps with projections and matte painting when needed.”
The night-time shots also required street lights and hero lighting on key landmarks to be realised. Street lights were created using scatterers in Clarisse and scatter maps were generated using maps of Paris to ‘tell’ Clarisse where the roads were. “Our 3D representation of Paris was then illuminated by these scattered
“Several layers of Street lights were created AND GRADED to recreate Paris” william Foulser, environment generalist, DNEG
lights,” says Foulser. “Several layers of street lights were created and graded to recreate Paris. Hero landmarks such as the Notre Dame, the Eiffel Tower and the Louvre were called out and hero lit individually. Distant rivers and water bodies were done with projected top-down specular maps, which comp could play around with and grade accordingly.”
Trees were crafted in Speedtree, then scattered in Clarisse. Proxy vehicles were laid out and animated with head and rear lights constrained to them to provide a sense of movement. The CG Grand Palais building itself was built using reference of the practical set, which was, according to CG build supervisor James Benson, “like a cake slice of the central glass dome and a bit of one of the aisle roofs. To augment this around 360 degrees we modelled, textured and look-dev’d the exterior of the Grand Palais roof, adapted a little to match the slice of set that was built. This was lit with flashing lights inside to match both lighting on the actors and also wide helicopter shots of the location.”
“Anything off the roof – the ground, trees, more distant roof areas and the Paris backdrop – was created with 2.5D projection and DMP from the environment department,” adds Benson. “The interior was also built in CG and textured to match photography of the location. Some shots started with a greenscreen studio set of the gantry, while others were shot on location with additional crowd elements and gantry extensions as required for certain plot points.”
For the shots where Ethan and Walker land on the Grand Palais after the jump, the environment generalist team took the build, put it in all the shots, and modelled and textured in a more shotcentric way, also using some 2.5D projection work where necessary. “We then worked up the surrounding roads in CG and dressed them with street furniture, trees from Speedtree and moving cars, all laid out and rendered in Clarisse,” explains DNEG CG environment generalist supervisor Robin Konieczny. “Beyond this we had multiple plates of Paris
stitched together by 2D sequence supervisor Luke Letkey and the comp team. Then we added skies to match the rest of the sequence.”
CREATING KASHMIR
Later in Fallout, Hunt and his team take on Walker amid the mountains of Kashmir in a thrilling helicopter chase and one-on-one fist fight. The chopper scenes were filmed around the glaciers of New Zealand, with DNEG using a variety of means to transform the locations.
“To capture as much as possible, we bolted an array of six RED cameras to the front of a helicopter, and flew down canyons and over glaciers to get the most beautiful tiled photographic plates,” says DNEG visual effects supervisor Huw Evans. “The six-tiled plates were digitally stitched together to give us a wide field of view on hours of breathtaking moving footage. We also used a combination of these stitched photographic plates to create 360 photographic bubbles, topped up and balanced with some digital matte painting and projection fixes.”
This approach allowed the filmmakers to look in any direction, which was necessary when, later, the action went back to Leavesden Studios where a helicopter buck on bluescreen would be filmed, and the background needed to be an environment spinning full-circle as the helicopter spirals out of control.
“There were a few environment sections that required a digital set build in order for our helicopters to interact with, and crash into,” states Evans. “For these we shot photogrammetry plates from the helicopter, making large arcs around the features we required, mixed with tighter, more detailed arcs so we could get as much information as possible into Reality Capture where the solve was put together. Combined with projected photography and detailed matte paintings, we could create accurate and detailed geometry of areas where we couldn’t capture the data using LIDAR scanners.”
A key role of the visual effects crew was also to take helicopter plates, that had been chosen based on performance and action beats, and deal with inconsistent lighting and backgrounds. Says DNEG 2D sequence supervisor Travis Porter: “Considering the sequence was quite lengthy, we started by splitting it into smaller, more manageable mini-cuts based on each environment. By taking the real shots of Ethan flying the helicopter, we had specific marker points throughout to match landscapes and grading of our projected environments.”
A CLIFF FACE-OFF
Finally, Hunt and Walker fight on a precarious cliff top. DNEG brought together several elements filmed at a pulpit rock (Preikestolen) in Norway and New Zealand, with additional set pieces shot in the UK. “The brief was to create an environment that transitioned from the higher snow-covered peaks at the start of the sequence, to a rocky crevasse, and then blended into the pulpit rock location,” says Benson.
“The cliff edge above and the crevasse were a mix of augmented set pieces and some full-cg shots,” continues Benson. “A mixture of rock materials created in Substance Designer and some DMP overpainting were used here, especially to extend beyond the sets and to create our environment mashup. We created photogrammetry for the pulpit rock location from helicopter and drone shoots, which was then retopologised. Some of the close-up shots of the rock were full CG and were created using a Zbrush and Substance Designer workflow with scattered pebbles in Clarisse.”
The work to marry different locations was intended to be seamless and largely invisible to the audience. “Fallout was the most Dmp-heavy show I have worked on,” notes Konieczny. “We had projection setups from north, south, east, west, and a top setup in Nuke by our lead Emilis Baltrusaitis, which gave us great coverage for most of the shots. Where there were gaps we would work them up on a shot-by-shot basis, feeding that back into the main setup – it worked very well.” Mission: Impossible – Fallout is now available on Blu-ray and Digital