Vr: Changing the story
We hear from Jan Pinkava, Google ATAP director, about the innovative Google Spotlight Stories and how they offer a new, unique way of telling an engaging narrative
Trevor Hogg speaks to Jan Pinkava about how Google Spotlight Stories embraces a new, innovative way of crafting a narrative
hen using virtual reality as a tool for crafting entertainment experiences, the tried and tested process of storytelling is disrupted somewhat by the fact that the audience is now firmly in control of what they see and what direction they take. It’s changed the game, and that’s something that has to be at the forefront of any VR creator’s mind.
Collaborating with various filmmakers to produce diverse content that pushes the boundaries of storytelling and technology within the realm of virtual reality is the primary objective of Oscarwinner and Google ATAP director Jan Pinkava, who created the first of the Google Spotlight Stories four and a half years ago with
Windy Day. “VR is this immersive and potentially interactive space that is pushing at the edges of the categories of what constitutes a story. Our focus has been on trying to help storytellers tell sequential stories in this format that have a beginning, middle and an end. This question of what kind of story works and fits in VR is fascinating. We’ve been learning that through
the business of making shows. Now your audience is inside the story world and what that means is that they suddenly own the camera. If you’re interested in telling a structured narrative you have to work harder to make sure that the audience gets the story you’re trying to tell in the right sequence in order for it to make sense.”
Technology can assist with the audience distraction factor so that important story points are not missed. “You can have a ‘look at trigger’ at a point where the action pauses until you’re actually looking in that direction, and carries on so that we can guarantee you have witnessed that important story event,” explains Pinkava. “Then you can hope to structure more stories that make sense to the audience [whether] they are paying attention or distracted… You try to cope with everybody’s different ways of experiencing the space that they’re in. But of course, the same old rules apply. If you have a compelling idea that is well-presented your audience is going to pay attention. It’s like if you’re a street magician or doing street theatre you’re gathering the attention of the audience and then presenting something that they want to watch. That’s a challenge for any kind of filmmaker, and also in VR.”
Storytelling in VR is more like a stage play than filmmaking. “It turns out that people from live theatre take quickly to VR because
“VR Offers A Sense Of Presence, the idea that YOU’RE TRANSPORTED to SOMEWHERE else” Jan Pinkava, Google ATAP director
they understand the business of presenting a show in a space to an audience,” notes Pinkava. “One thing that VR seems to offer is this sense of presence, the idea that you’re transported to somewhere else and it’s tempting to say, ‘What if you are part of a story? What if you are not just an observer but an actor in this story?’ That also comes from gaming, from having a sense of autonomy in a scene and actively doing something. If you are given a first-person perspective of being there then the experience has to be delivered in such a way that it feels authentic. You can do everything that the experience suggests that you can do. You expect to be able to touch things, pick things up and look around; that means a whole other level of interactivity. It is then a question of, ‘Is that part of the story or a game? When do those things crossover?’ Because it is a new kind of thing it challenges all of these categories. People start having to push on the edge of that. When is a game a story and when is story a game? It makes it an interesting exploration.”
Google Spotlight Stories tend to be told from the third-person perspective. “What you get is a different sense of when you are in a VR situation, even a stylised one,” observes Pinkava. “For our show Pearl, we put you in the passenger’s seat of this car and take you on a journey with the father and daughter for 20 years of their life. In the mobile version of that story we have you very much as a third-person camera in a space and you’re hovering above this stick shift in this old 1980s sedan. But when you’re in virtual reality and 360 degrees of freedom and you’re able to look around and move around a little bit, we found that was too constraining. Your sense of presence meant much more and it felt uncomfortable to be hovering above the gearshift. We had to
put you in a seat of the car so that you could feel that your being there had a reason, a rational.”
Virtual reality experiences can vary when it comes to runtimes. “If you’re experiencing a show entirely with a VR headset on then you can be in that space and look around, and depending on the quality of the show and how compelling it is maybe 20 minutes is great,” remarks Pinkava. “If you’re holding up a mobile phone board to your face or just looking through the window of the phone, you can’t hold something to your face that long so it makes sense to think of a shorter format. We’re dealing with a new film-like format; the early days of cinema short films were the dominant form back in the day, and this is true again now. As people are exploring how to present stories in this form it makes sense to keep things short, sweet and easy to get from beginning to end without a huge investment of time. It fits better with our busy mobile lifestyle these days. Everyone is distracted. Everyone’s time is cut into shorter and shorter chunks. We want to present shows that are bite-size and can be experienced almost on the fly wherever you have time to be. A story that is two to five minutes long makes sense in so many different circumstances. It’s a good idea to try to go in that direction.”
Sound has a major role to play in telling virtual reality stories. “We say in filmmaking that sound is half the show but it’s more than half in VR,” remarks Pinkava. “From the perspective of putting you in a scene emotionally and guiding your attention, sound is indispensable. Ambisonic binaural rendering provides a true surround-sound environment.” Sounds are heard in correlation to where the audio source is relative to the viewer. “The sonic landscape that you’re in is richly realistic and natural. We can use it not only to inform you as to where to look and what’s going on, but also put you in the scene with all of the other traditional ways of music. You have a whole layered set of possibilities with sound that come into their own in surround sound with VR. The next challenge for full-on VR where you are able not just to look around but also move around the scene is to create sound designs that retain that authentic sense of space.
“Every show we have published has been new in some way,” states Pinkava. “We can’t rest on our laurels and experience and don’t want to say, ‘We’ve done one of these before.’ The idea is to do something we don’t know how to do each time. That means it is correspondingly difficult and [there are] all kinds of problems we hadn’t anticipated, which is a point to push the technology forward. What we are doing is developing the tools and techniques and ideas of how to tell stories in VR. Each show that we choose is slightly different and has some new demand, either in the way it looks or sounds, the way the story is structured or how interactive it is. We’re constantly expanding the territory bit by bit of what kind of stories our
“SON Of JAGUAR WAS Designed AS A ROOMSCALE VR experience” rachid el Guerrab, technical project lead, GSS
technology and VR storytelling as a conceptual form can support.”
Along with working with Aardman Animation (Special
Delivery), Mark Oftedal (Buggy Night), Shannon Tindle (On Ice), Patrick Osborne (Pearl), Justin Lin (Help) and Felix Massie (Rain
or Shine), Google Spotlight Stories has an upcoming new title called
Son of Jaguar by Jorge Gutierrez who previously directed The Book
of Life. “Jorge has a very Mexican story set in the world of lucha libre Mexican wrestling,” states Pinkava. “We also have some more internal experiments which includes a beautiful show called Sonaria which is designed by Scot Stafford, our creative director of sound, and Kevin Dart’s Chromosphere in Los Angeles. It’s a fantasy journey of various creatures in a stylised world that almost looks like stained glass. You’re going from scene to scene with a sound design that is taking full advantage of the space around you in terms of the music, the sound effects and the sense of atmosphere. It’s a rich experience and exploration of how much we can do with virtual reality sound. There’s another one which I am working on with my old friend Mark Oftedal from Pixar days, which is an exploration of how far we can go with interactivity before something turns from being a story into a game.”
Each new show is a big challenge. “Somehow if you do something for a long period of time you feel like you get better and better at it,” observes Pinkava. “Because we keep pushing on the edges of what we can do it doesn’t feel that way in this process. The thing to take away from all of this is that as this impressive and amazing technology is used and develops it’s important for us all to hold on to what telling a story really is. Whatever the technology or medium, we as audiences still want story. We want people who put their time, energy and talent into a show to present us with something delightful and engaging that usually has a beginning, middle, end and a point to it. I don’t think story is going away any time soon. It’s a fixed point in human experience.” Stories need to be driving the technology, not the other way around. “No amount of amazing technology is going to save a bad story or undo a poorly structured story. The story itself is the core of everything and if you hold onto that you’ll be going in the right direction.” •
The beautiful world of Sonaria features a mix of creatures, and the story is made more immersive by being largely sound-driven.
The Sonaria project was co-directed by Scot Stafford and animation studio Chromosphere.
Justin Lin, best known for highoctane action franchise The Fast and the Furious, directs Help, the first live-action title produced by Google Spotlight Stories where a young woman encounters an alien in the middle of downtown Los Angeles.
Google Spotlight Stories cover a range of media, from the Cardboard to mobile devices
The delivery of a special set of sunglasses results in Ella dampening everyone’s spirits whenever she puts them on in Rain or Shine directed by Felix Massie.