VISIT THE LAND OF THE DEAD WITH COCO VR
You may have seen Pixar’s latest hit, but how about immersing yourself into the film’s world with VR?
Available now for free on Oculus Rift and Gear is Pixar’s stunning debut into virtual reality – Coco VR. Dive into the film’s incredibly rich and colourful setting, the Land of the Dead, filled with a cast of quirky skeletons.
“Magnopus handled the production of the VR experience and have done a lot of work with traditional animated films, so they have a good understanding of the workflow for film and how that would work in VR,” states Unity Technologies XR producer Sarah Stumbo. “Whereas Pixar had never done anything in VR before and they have this high standard for their animation.” Costume tools needed to be built by Magnopus to get the animation files from Pixar’s proprietary software Presto into the Unity 2017 game engine. “Usually, for VR experiences the heaviest asset that you’re going to deal with is going to be your texture files. However, for this one because the story is told by the way the characters’ eyes move and faces light up, the character animation needs to be 1:1 to the original version from Pixar. Everything else, from the environment assets to textures, were all reduced and optimised for VR.”
Most of the assets came from final Renderman RIS versions from the original film. “Part of the virtual reality experience is that you go to an art gallery and in the movie, there’s an art gallery as well,” remarks Stumbo. “All of the art assets in there are concept art pieces from the film. Those all came directly from Pixar. Then all of the models and environment of the plaza came directly from Pixar. They ended up rearranging a lot of things because certain things don’t work well in VR. There were quite a few assets created on top of that for the VR experience.” Timing played a major role in determining what appeared in
Coco VR. “If the environment was mostly completed by Pixar and could be converted into the VR experience then it would go. Magnopus had a lot of brainstorming with Pixar upfront about what pieces could be put into virtual reality.
“They had a few tricks with the mirrors,” notes Stumbo. “The environment was baked into the mirror so that the only thing left for real-time reflection was the character.” Sound was done as spatialised audio. “I would be talking to someone and move away to go look at something else then they couldn’t hear me anymore.” The sound was kept accurate to real world. “That did make for a nice effect because when you’re in the
plaza and there is a lot going on, having the spatialised audio made it easier to navigate.”
Assisting with the user kinematics was final IK. “Because the characters are skeletons if the Inverse Kinematics didn’t work exactly right or hung for a second it doesn’t look that odd,” remarks Stumbo. “It almost makes sense. In the film there is a one liner where they joke about how silly the skeletons walk. With this experience we were able to get away with more wild movements. At one point when I was talking to some of the creators in the experience I had my arms crossed and looked down to see that my skeleton arms were also crossed. I thought that was good. for every object that you picked up in the experience there was an original hand pose which was fun.”
Users are also able to toss around their skeleton head. “That doesn’t affect what you see and do. You can watch your skeleton head fly away.” One of the biggest achievements was the use of multiplayer in the experience, and it was even “easy to have design reviews inside the experience.” As for the length of time that someone can spend inside Coco VR, it varies. “I spent about 30 minutes inside but I had also seen the original film the night before so was in the mood to explore all the nooks and crannies. When I went through it as a group I was in there for about 45 minutes.”
Meet You on The holodeck
Designing the Coco VR experience began with first incorporating the multiplayer experience, which led to the creation of a new workflow involving a holodeck room. “They wanted it to be a multiplayer experience which was prototyped first,” remarks Sarah Stumbo. “Magnopus is based in Los Angeles and Pixar is situated in Emeryville, so they were able to get people from both teams inside a VR space together to talk about the design and development as it was occurring.”
Through nine months of production, meetings were held within the VR experience allowing for experimentation and demonstrations to take place. “It created a whole new workflow,” notes Stumbo. “Coco was still being worked on while the VR experience was coming together so a lot of the times it was Pixar handing the files over to Magnopus as they finished them.”
lighting Tricks
Most of the lighting was baked into the imagery so as to limit the amount of realtime lighting required. “One trick that they did come up with was for the backdrop of the plaza,” explains Stumbo. “You look up and see these buildings stacked on top of each other as if they are infinite. They created a 3D skybox where they were able to layer different 360-degree images on top of each other, so depending on where you’re looking you’ll see a different layer.”
“Magnopus were able to get people from both teams inside a Vr space together to talk about the development” Sarah Stumbo, Unity Technologies XR producer