ANIMATING IN VR
Opportunities abound as an undiscovered world of immersive, interactive storytelling awaits
While gaming is the predictably flash poster boy helping to grab headlines and revenue for VR right now, it’s arguable that the real creative fireworks are currently found in the animated story/experience arena. Games developers have worked with real-time engines and first-person viewpoints for so long that the leap to immersive stereo 3D is, if not painless, then at least a logical. For storytellers, on the other hand, virtual reality is a foreign land that is ripe for exploration.
360 degree video arguably offered a kind of soft introduction via the 2014 launch of Google Cardboard and subsequent arrival of Youtube and Facebook support for the format. But there’s a gulf between creating a story to render out and play back in stereo as pre-canned spherical animations and creating a true VR animated experience. VR is, by definition an interactive medium, and that requires real-time rendering and animation that reacts on the fly to the actions of the user. That necessitates a different approach to conception and execution.
“The real-time nature of the engine is a second order artefact of a primary order challenge – user choice,” explains Keith Lango, a veteran of CG studios including Blur and now an animator at games giant Valve. “The real-time nature is to accommodate user input, not the animation. Additionally the animation itself needs to make the same accommodations for user input and viewing choices.”
This has several practical implications for artists and designers working with VR, not least the need to create characters and other assets that react to player movements and interactions. In general terms, the more interactive the story and environment, the more immersive and powerful the experience will be. And of course, the greater the number of possible interactions, the greater the range of story permutations, all of which need to be carefully plotted out for story purposes and accounted for when animating.
INTERACTIVE APPROACH
“In one sense it’s not very different at all, and in another we’re still figuring it out,” says Keith. “From a pure animation asset perspective, it’s similar to game animation, with a slightly more careful eye towards engagement accuracy. You build a motion tree of actions and motions, work in blending rules to adjust to changing environmental or player choice inputs, and so on. One could limit how the user views or interacts with a VR world, but then
you run up against the question as to whether the content is even a best fit for VR execution in the first place,” he explains.
Giving the viewer a degree of agency also fundamentally changes the way the three dimensional environment is experienced. Gone is control over how scenes are viewed and framed, while the experience of moving within the story space rather than viewing it remotely fundamentally changes the way the world needs to be designed and built.
“It’s about learning how to release control to the user and building motion solutions that are robust enough to deal with those user inputs,” says Keith. “The biggest hurdles are emotional and mental in the artist, not in the actual tech. Some artists love the fine control they get in building singular solutions to the highest degree of singular quality possible, and for those it may be a tougher transition.”
“On our VR project We Wait, we started the production talking in cinematic terms and ended up discussing it in terms of theatre,” reveals Darren Dubicki, director/ designer at Aardman Animations. “That was kind of an epiphany for us. You begin to appreciate the physicality of the space and also realise you don’t have the usual cinematic things like fast edits. Instead you employ stage direction and smoke and mirrors, leaning on theatrical techniques to push the story along. At Aardman we’re even considering using theatre directors for future VR projects, to help blockout scenes with actors.”
In terms of tools and pipeline, at this stage of VR’S evolution, there simply isn’t a one-size-fitsall solution. “What makes a good pipeline?” muses Keith. “‘Good’ is really open to interpretation,” he says. “If you can get your stuff into the VR engine and users can experience it and if you can iterate in reaction, all in a way that doesn’t drive the artists or the technicians crazy, then that seems like that’s enough to be called ‘good’. The biggest factor in pipelines is really team size and project scope, which is as much true in VR as it is in game or film or TV.”
Products like Tilt Brush, Oculus Medium and the Unreal Engine VR Editor are introducing the concept of intuitive WYSIWYG asset and environment creation inside the VR environment, but for now at least, the packages used for modelling, texturing and sculpting typically remain the same as those used for 3D animation or games development. The key difference (at least when compared to the 3D animation field) is that rather than animating and passing data over
IT’S ABOUT LEARNING HOW TO RELEASE CONTROL TO THE USER AND BUILDING MOTION SOLUTIONS THAT ARE ROBUST Keith Lango, animator and VR trainer, Valve
to an offline renderer, a real-time engine such as Unity or Unreal Engine (currently the two most popular options for animation projects) is required.
Inevitably, for artists without games development experience, adapting to this involves something of a learning curve. “The kind of artists we look for to work on VR are those with experience in gaming,” admits Dan Efergan, creative director of digital at Aardman Animations. “A great animator is a great animator, but with VR, you also need a good understanding of blend trees.”
VR CHOICES
“One area of difference is that in VR there is no image frame, so the animator needs to really observe how users are viewing their animation,” adds Keith. “There are other considerations from an asset creation perspective, but the stuff that interests me are the social-psychological or behavioural problems for the NPCS the user interacts with, as well as what kinds of social-psychological effects that VR choices can have on the users of VR itself.”
While this may not sound like something an animator needs to worry about, Dan says they wrestled with these very problems when deciding how to immerse viewers in the hard-hitting, factbased VR experience We Wait: “If we’d gone for a more photoreal approach it would have been too emotionally jarring. And even with a more ‘poetic’ approach we found ourselves toning things down and neutering some of the body language animations, as we were worried about pushing the emotion too far. The fear of creating an experience that’s too emotionally powerful is an interesting one. Doctors right now are discussing the potential for PTSD, for people to use VR to play horror games.”
Given the immersive and emotive potential of interactive animation it’s little wonder so many artists are so keen to don headsets. “Every time a new platform comes along, storytellers jump on it as there’s a desperate human desire to tell stories. It’s just that with VR the line between story and game is going to get blurrier,” says Dan.
While VR cinema venues are still somewhat limited and more geared towards passive 360 animation, the trick now is for publishers to find a way to keep this momentum going by monetising animated or interactive content as options for online distribution. One tactic may involve following the lead of the broadcast market. “The space exists now to construct the VR version of serialised content, and people are already looking for a way to create the VR equivalent of Breaking Bad,” says Dan. “Games obviously offer a returnable experience, now we just need to find a way to do the same with VR storytelling. It’s certainly something Aardman is keen to have a punt at.”