Spotlight on technology
How Google innovates with story always firmly in mind
Filmmakers producing Google Spotlight Stories need tools to support their creative visions and Google provides them with the technical expertise.
Along with producing plugins for Maya and Pro Tools, Google has developed a proprietary software program that helps with the end-toend production of VR stories. “Story Editor is a custom visual tool to create and edit non-linear stories starting from animation clips or videos,” explains Google Spotlight Stories technical project lead Rachid El Guerrab. “This also includes tools to edit and control object-based or Ambisonic audio and its interaction with the viewer.”
A rendering and animation engine was created because none of the offthe-shelf solutions provided the right amount of flexibility or performance. “This enabled us to shrink it to almost the size of a video codec, which allowed us to integrate into the Youtube app,” explains El Guerrab. “But even then, we had to make it as close to playing a video as possible, even though it’s a full real-time game. This meant sub-second startup, continuous nonlinear streaming based on how you’re watching, instant shutdown if you switch to something else, and many other similar challenges.”
The Sonaria and Son of Jaguar Stories carry on the tradition of each new title for Google Spotlight Stories, resulting in technical innovation.
“Sonaria started as a technical showcase experiment for our audio technology,” states El Guerrab. “So naturally we used various aspects of the audio engine, including but not limited to: occlusion and real-time filtering of audio, reverb and similar effects, and mixing 360 soundscapes in a way that allows six degrees of freedom movement. Son of Jaguar, on the other hand, was our first show designed from the ground up as a room-scale VR experience. We evolved many of our interactive tools to deal with the viewer moving around the scene and the story reacting properly to it. The show also pushed our rendering tech to handle the high-detailed characters and animation, and still downscale properly for mobile.
“As GSS transitioned from interactive 360 content for mobile, to stereo VR and then room-scale VR,” notes El Guerrab, “we’ve been evolving the same storytelling concepts throughout, and will be moving the same way into the new realities of AR and MR. That said, with each format, we have to work hard on how to expose the possibilities to artists and directors as creative elements they can take advantage of. We also have to constantly think about how the same story plays out in all these different outlets, which is the crux of our R&D effort at Google.”