Tech talk: USD + Renderman
USD+RENDERMAN
What is USD, and how has it taken the animation and
VFX industries by storm? Renderman’s Mark Manca tells all
Renderman product manager, Mark Manca, delivers an introduction to USD and its benefits for the future of filmmaking
Over the past several years, Pixar’s USD has stormed into the collective consciousness of the animation and visual effects industry. But what is USD, exactly? And what does it mean to artists? And beyond that, how will my applications of choice use USD, and how does it work with renderers such as Renderman?
WHY USD AND WHAT IT DELIVERS:
The first thing to know is that USD is not just a file format. While USD stands for ‘Universal Scene Description’ and does ship with two main file formats, it is the additional features within the technology stack that have caused USD to be so compelling to studios. USD’S secret sauce is in its architecture and implementation. USD enables:
• Collaboration – teams can use USD within modern, overlapping data pipelines
• Non-destructive workflows – opinions, variations and layers allow artists from different areas of the pipeline to work on the same asset at the same time
• Interchange – pass many kinds of data between applications
• Schemas, geometry, materials, lights, volumes, crowd rigs (more coming!)
• Performance – fast reads and writes will keep you working and creative
• Renderers such as Renderman to plug into it – Usd-enabled applications can render to the renderer of their choice via a sister technology named Hydra
USD is the culmination of all of Pixar’s years of experience in describing geometry and scenes, in a technology that fosters collaboration between teams within non-destructive workflows. An important cornerstone of the effort is data interchange between applications. And while listed towards the end of the list, performance is not least. Performance is critical to enable artists to interact with large amounts of scene data and to work within the larger context of their worlds. Finally, by providing the ability for multiple rendering backends to plug into Hydra, USD + Hydra offer a highly capable abstraction layer for renderers to
render USD no matter where Hydra is used.
PERFORMANCE, COMPLEXITY, COLLABORATION AND CREATIVITY
The complexity of today’s animated and VFX films is several orders of magnitude higher than it was in the days of the original Toy Story in 1995. USD and Hydra have been written to scale to the demands of modern productions.
An example of this is what we call the ‘fractal complexity’ of Toy Story 4. The full environment was modelled and instanced. You can start from a satellite's eye view of the environment, and then travel down to the level of detail of an individual button contained within the breadth of knickknacks within the antiques mall, one of the larger set pieces within the film.
By allowing artists to work within the full complexity of the main Mountain County set and its individual locations, they had maximum context as they worked to build the world of Toy Story 4 during production.
Robert Moyer, the supervising technical director of Toy Story 4, reports, “The main advantage besides the ‘universal’ part of USD is the speed. So, say with the antiques mall in Toy Story 4, it was modelled and dressed with tens of thousands of props. Our camera and our animation departments could load the whole thing up and get pretty responsible responsive feedback. Those two features, both the universality and the speed, are really what have been the big wins for Pixar.”
COLLABORATION – BACK TO THE ANTIQUES MALL
Let’s dive into the antiques mall as an example of how USD is used within Pixar to facilitate artists from different departments to work concurrently. In particular, we will zoom into a section of the antiques mall and examine how USD enables collaboration via layers.
In this scene, the modeller has created the base assets. The layout artist collects the models together and builds out the set. The animator brings motion to the scene, and the lighter illuminates the scene. Together the group of artists (plus many more!) bring life to the shot.
In the context of the USD technology stack, each artist is working with independent layers within the USD stage. Everyone can edit their individual layers without affecting anyone else.
The road to collaboration that USD offers helped elevate the artistic process during the creation of Toy Story 4. Bob Moyer continues, “The more we see USD not just being about geometry and animation, and actually take on things like shading, the more you discover how different departments or different disciplines can work together and base their work off of each other. When it’s in that same format, it gives you those exciting opportunities that you hadn’t even thought of before.”
UNDERSTANDING LAYERS AND COMPOSITION
The underpinnings of USD’S ability to allow for collaboration is its composition framework, which
in turn is based on the notion of layers. Let’s dive into these concepts using the text version of the USD file format (though remember, USD is more than a file format).
The first file is used to describe the base-level geometry. This example contains points, colours, and other bits to describe a box. While this example contains the material description in the same file as the geometry, it could be split into a separate file or ‘layer’ so that a lookdev artist can work on the asset at their own pace.
The second file shows two instances of the box being composed together with what USD calls a ‘composition arc’.
At any stage of the pipeline, overrides can be applied to any part of the USD scene. They are layered into the USD stage via composition. In this example, we set the position of objects as well as override the colour of one of the boxes.
There are multiple ways to override geometry and materials. In the example above, we set the data literally. This kind of workflow is performed at Pixar when overlaying deforming geometry that comes from the animation or simulation departments, to override the default T-pose geometry of the base model.
Another way to override geometry and materials is to select a ‘variant’. USD offers you the ability to provide multiple variants of your data. You can provide multiple materials to choose from, or you can provide multiple sets of geometry. For example, in Toy
Story 4 one dresser model had over 45 production blessed materials, mixing age, dirt and paint colour.
All of these materials are easily managed using shader variants in USD. This also applies to geometry. For example, if you have a clean vehicle and a damaged vehicle, USD variants can be used to select the damaged geometry and the damage shading when you need it.
WHO IS ADOPTING USD?
While USD originated at Pixar for media and entertainment workflows, its usage has blossomed outwards to the DCC (Digital Content Creation) applications themselves, and further beyond the media and entertainment industry. The collaboration and data exchange features of USD, together with the rendering features of its sibling, Hydra, have made it the technology of choice for next-gen pipelines, tools and applications.
Outside of Pixar, studios like Animal Logic, Blue Sky, Walt Disney Animation Studios, Dreamworks, Luma Pictures, Rodeo FX, MPC and Mill Film have either adopted USD in some portion of their pipeline or have embarked on significant R&D projects centered on USD.
DCC applications such as Katana, Houdini and Maya have all adopted pieces of the USD technology stack already, or are in the process of doing so. For example, Katana uses Hydra as the native 3D viewer backend. Houdini uses both USD and Hydra as the complete technology stack for Solaris, Sidefx’s new lighting and scene composition toolset. Maya is adopting the USD importer and exporter and is bringing together various studio efforts into a unified technology for all artists.
Beyond our industry, Apple has adopted USD as a significant part of its Augmented Reality toolkit (ARKIT), and NVIDIA is using both USD and Hydra as a significant part of its toolsets, including within Omniverse.
GREAT! BUT WHAT IS USD, EXACTLY?
So we’ve seen the ‘What’ of USD in terms of what it brings artists, studios, and other organisations working with 3D. But what is USD, exactly?
USD comes with both a text format (with extension usda) and a binary format (extension usdc). The .usd file format is preferred and can be used to denote either one. There is also a .usdz file format that allows you to wrap all geometry and textures needed for an asset, in a comfortable zip-like archive,
THE COLLABORATION AND DATA EXCHANGE FEATURES OF USD HAVE MADE IT THE TECHNOLOGY OF CHOICE FOR NEXT-GEN PIPELINES Mark Manca, Renderman product manager
and is the format that Apple uses for ARKIT.
Going beyond the file format, USD is a suite of technologies that allow for scene description and composition together with technology for visualising the composed stage. USD provides a set of APIS that can be used at any layer that allow tool and application creators to dive in and build tools and features on top of what Pixar provides.
The visualisation layer is called Hydra and ships with an Opengl backend called Storm. Hydra is extensible and allows for other renderers, such as Renderman, to present themselves as alternatives for the artist to use within their tools.
THE ARCHITECTURE OF USD
For those of you that are more technologically minded, to understand USD it can help to dive into its architecture. Instead of being just a black box, USD offers several pieces of technology that are separable and extensible.
The USD layer can be extended with additional file format plugins (for example, USD supports the reading of Alembic directly via a file format plugin). USD can be extended with asset resolvers that allow pipeline developers to enable USD to read from any asset management system. Furthermore, it provides the ability for any renderer to have renderer-specific patterns and materials. And it gives studios the ability to extend USD with additional schemas that let it define anything they would like.
Hydra, the scene visualisation component, can be separated from USD and wired into a scene graph. For example, Katana has done this – it uses Hydra as the rendering engine for its 3D viewport.
Different rendering backends can be plugged into Hydra. This is what we’ve done with Renderman. For applications that choose to expose the functionality, Renderman can be used to render from within USD and Hydraenabled applications.
Houdini Solaris is the best example of an application that leverages the complete USD technology stack. The USD stage is at the heart of the application, which enables artists to leverage the power of USD for scene construction and layout. Hydra is the rendering engine, which allows any renderer that provides a Hydra Renderer Delegate to work within Solaris.