An Animation and Real-Life Crossover
CG pioneers push the VR envelope at Tribeca, Annecy and other major global festivals.
CG pioneers push the VR envelope at Tribeca, Annecy and other major global festivals.
Not long ago, VR, AR and other immersive experiences were sideshow attractions at tech fests like SIGGRAPH.
Or they were shoot-’em-up stars at E3 gaming confabs. Not anymore. Now, interactive animated stories are being displayed at premier film festivals in Venice, Annecy and Tribeca, and their creators include pioneering filmmakers in computer animation.
Festival attention represents a major advance for virtual reality, believes Ken Perlin, who worked on the breakthrough “Light Cycle” animation done at MAGI for 1982’s TRON. At this year’s Tribeca Film Festival, Perlin and his collaborators at the VR studio Parallux screened a six-minute experience called Cave, unveiling a technology that allowed 16 people to experience the piece all at once. “The difference between showing Cave at SIGGRAPH in 2018 and at Tribeca is that the New York Times culture reporter never showed up at SIGGRAPH,” he laughs.
Already an Academy Sci-Tech Award winner for developing landmark CG software, Perlin has now spearheaded development of a Mass Synchronization Solution. This enables untethered, headset-wearing audiences to experience the animation of Cave as well as see their fellow viewers as avatars in the experience. Cave was first demonstrated for 30-person audiences at SIGGRAPH, although Tribeca could only accommodate 16 people at once.
This tool grew from Perlin’s Future Reality Lab at NYU, which receives support from Facebook, Unity, Bose and Nvidia. Parallux is a licensee, and Perlin hopes other VR studios will also license the technology for their own productions: “If this medium is to survive, we
need to think in terms of startup companies, not research labs.”
From Antz to Bonfire
Perlin isn’t alone in remembering the early days of CGI when studios had to invent their tools. Eric Darnell, co-founder of the VR studio Baobab, was a key member of the seminal CG house Pacific Data Images, and directed its first feature Antz after DreamWorks acquired PDI. “Those times had a ‘flying by the seat of our pants’ feel to them,” recalls Darnell. “PDI’s toolset made it possible to do things we couldn’t do with off-the-shelf software. That’s
“VR can take empathy to the next level. People can make choices based on how they feel. If we can motivate people to act because they care about a character and they’re in the river of a story, then we’ve succeeded.” — Bonfire director/producer Eric Darnell
where we are now in VR. We have to build it ourselves.” Of course, the secret sauce for VR studios now involves artificial intelligence systems — not simply making CG surfaces look believable. “At PDI, we’d cheer if we could put fur on something!”
Darnell, who later directed DreamWorks’ megahit Madagascar trilogy and three acclaimed VR projects at Baobab (Asteroids, Invasion and Crow: The Legend), chose VR experimentation over what he calls the “groove” of making large institutionalized films. He remembers being blown away by VR’s potential when Baobab co-founder Maureen Fan showed him an Oculus headset around four years ago. “I first thought VR would be like making movies in super 3D. But everything we do is an experiment. I quickly learned I needed to get humble!”
The latest project from Baobab’s 19-person team is Bonfire, an alien tale with a comedic tone. Tribeca attendees experienced it as an adventure that ran between 15 to 20 minutes — its running time was determined by the
choices a viewer made. For Darnell, the goal is to foster a sense of relationship between the audience and a VR character: “How can we come up with an interactive paradigm that’s effortless, and doesn’t require you to learn?”
That’s a key distinction for Darnell and Perlin — who approach VR from a filmmaking
rather than a videogame background. Darnell asserts, “Interactivity in games is motivated by killing X number of zombies and getting to the next level. VR can take empathy to the next level. People can make choices based on how they feel. If we can motivate people to act because they care about a character and
they’re in the river of a story, then we’ve succeeded.”
A shared interest in character and story is what led Ken Perlin to collaborate with Kris Layng as the director of Cave. Layng’s background is not in videogames but in production design and concept art, including the TV series Maniac. “Cave is a theatrical experience in which you see other people’s responses while you are together with them,” says Layng. “Not everyone wants to play a game, but everybody goes to movies. There’s a way for VR technology to reach millions of people — not just gamers.”
The View from the Holodeck
Layng’s approach was to direct the actors in Cave like theater, and then use motion capture and Maya to animate the characters. The production also utilized Google headsets with six degrees of freedom and Nvidia graphics cards for real-time rendering. Because the Mass Synchronization technology enables audiences to experience Cave collectively, Layng likens this approach to the invention of the projector that made movie theaters possible. As Perlin remarks, “Right now, VR is focused on the individual viewer, which is a mistake. The decisive moment for cinema was not the kinetoscope in 1892, but when the Lumière Brothers pushed the projector in 1895. My mantra is that the ‘holodeck’ is other people.”
Both Parallux and Baobab are betting on a future when VR technology becomes scalable enough that large studios owning branded property will join the field. “We need a distribution system where individuals don’t have to make hardware purchases to experience VR — they just have to buy a ticket,” Perlin maintains.
Darnell says that Baobab’s work can already been seen for the price of a ticket in location-based experience facilities. “There’s a fair number of them in Asia, though it’s not a big money maker yet. But they’re great opportunities for people who aren’t sure if they want to invest in a headset.” Darnell adds, “We’re really conservative about this. Even if the hardware is affordable and comfortable enough, what happens if you go through the available content in a day or two? It’s a chicken-and-egg situation.”
Many VR aficionados have pointed to Facebook’s closure of the Oculus Story Studio, and the demise of Google’s Spotlight Stories as daunting omens. But Darnell observes, “I think that both Google and Oculus thought they would use those story groups to seed and inspire the industry. My understanding is that Oculus thinks they accomplished that. Now there are lots of little companies that have formed, and rather than compete with them, they’ll take the money they’ve spent on Oculus Story Studio and start distributing it to companies that are trying to make great content. That makes sense from a business perspective.” (One sign that Darnell may be right is that Oculus Story Studio alumni founded the VR shop Fable, and have had their Wolves in the Walls VR piece accepted at both Tribeca and Annecy.)
At Baobab, Darnell explains,“We’ve received lots of support from HP and Oculus and Samsung as well as Hollywood investors. We’ve received around 30 million, which will allow us to do this stuff for at least a few years. We’re happy for that, since we don’t know whether it will take three years or 10 to have a viable market.”
Perlin isn’t making any predictions either, but he’s hoping that NYU’s Future Reality Lab will continue to serve as a technology incubator while studios like Parallux develop libraries of content. Since he has been involved in computer animation technology from the earliest days of the field, he finds it amusing that his IMDB page lists only two credits: TRON and Cave. “Maybe I’ll get another one in the next 32 years!” ◆
Ellen Wolf is an award-winning journalist who covers animation, vfx and new technologies.
“We need a distribution system where individuals don’t have to make hardware purchases to experience VR — they just have to buy a ticket.” — Cave director Ken Perlin