Rokoko: Motion capture made easy
Rokoko CEO and founder Jakob Balslev gives 3D World six reasons to be excited for their game-changing Smartgloves
Find out more about the revolutionary Smartgloves
“USERS CAN STREAM THEIR LIVE DATA DIRECTLY INTO OTHER 3D TOOLS” Jakob Balslev, CEO and founder, Rokoko
Rokoko, the creators of accessible character animation and motion capture tools, is putting the future in its users’ hands with the revolutionary Smartgloves.
The notoriously difficult art of animating 3D finger and hand motion has historically been out of reach for the vast majority of creators. With pairs available for $995 and a commitment to ease of use, Rokoko aims to make professional-quality finger tracking a reality for all creators. The gloves can even be paired with Rokoko’s intuitive Smartsuit Pro to create a full-body mocap solution. Rokoko’s founder and CEO Jakob Balslev spoke to 3D World about why the community should be excited about this pioneering solution.
YOU CAN USE ROKOKO HARDWARE ANYWHERE
My graduation project at film school was this super ambitious animation where we used motion capture to live-render the data from actors and show them as a live animation in front of an audience. We pulled it off but the only mocap option we had was to buy an expensive optical system. You needed a dedicated studio, a whole staff of people that knew all kinds of stuff, and it was incredibly expensive. That’s what spurred the idea of inventing our own systems. It had to be affordable, something I could operate myself and could use anywhere. We took sensors like those in your cellphone that each has a gyro, an accelerometer, and a compass. We built them into this textile suit with a little computer that sends through wi-fi to your computer or smart device. All the tech is inside a textile suit, so if you know how to put on your clothes, you know how to set up the system. With optical systems, you need distance from walls and you can’t have direct sunlight because of infrared. All of that is gone, now you can use it anywhere.
THEY TOOK THEIR TIME
Initially, the Smartgloves were planned as part of the Smartsuit Pro. We naively thought they were two sides of the same coin and they just weren’t. Finger movements are so complex and unpredictable in completely different ways to the body. When you have an inertial sensor-based system it’s all math, there’s no camera locking it in place. It’s all based on our algorithms and understanding of kinematics. That’s easier with the body than the fingers. We decided to ship the suit, and good thing we did because it’s been three years. Now we have many thousands of them out at smaller studios that never had access to motion capture before. If you’re an animator sitting at your desk and you get an idea you can just stand up, record it, and look at it. If it wasn’t what you wanted, then you just saved yourself two weeks. We ended up using everything we learned from the suit and then worked really hard on some rules for what the Smartgloves could do.
NO MORE DRIFT
With the Smartgloves, the biggest issue we had was drift. Drift is the big thing with inertial sensor-based systems. The beauty is you can use them anywhere and you never lose any data. The downside is that they can drift over time because there’s no optical system holding them in place. With a suit, you can do stuff to correct it but for gloves, it’s much more tricky. It’s very clear if fingers are overlapping or the hand suddenly looks odd. So that’s what we’ve been really focusing on and trying to perfect. There are different kinds of drift. There’s gyroscope drift and there’s magnetic interference.
The magnetometer on the suit can drift if you’re close to metal or a magnet of some sort. That’s a big problem, especially with fingers as you could be touching your computer or a metal beam. What we did was remove the magnetometer from the gloves so they’re not sensitive to interference from metal at all. With some of the other products in the field if your hands get magnetised you’ll have to ship it back. That is something that cannot happen with the Smartgloves.
UNPARALLELED HAND TRACKING
Another thing we did for the Smartgloves was to add a wrist sensor. You have seven sensors – one for each finger, one on the back of your hand, and then you have one on your wrist. This was important for us. Other systems don’t have hand tracking, they have finger tracking but don’t know what the hand is doing. The great thing about the Smartgloves is if you’re using them without a suit you can pretty accurately calculate the entire arm based on what the wrist is doing.
THEY WORK WITH YOUR FAVOURITE TOOLS
Users being able to stream their data into other 3D tools is something we’re really focused on. In Rokoko Studio Live users can stream their recorded or live data directly into our native plugins for Unity, Unreal, Maya, Blender, Houdini, Cinema 4D, and more. You set up the gloves with Studio but then you stream it into the plugin. You then have a panel inside the plugin where you can control Studio. You need to open the Studio but you can do everything inside of the plugin. You can do your calibration pose, start your recording, start streaming, anything. We know that this is something users are very concerned with. They don’t want more clicks or more apps, they just want to work where they are and in the environment that they are comfortable with.
THE BEST IS YET TO COME
A major upgrade will be coming to our Smartglove technology in 2021. For the past five years, we’ve been researching the optimal way to combine the freedom you get with sensor-based solutions with the fixed position of optical systems. This technology is already in the Smartgloves, we just haven’t activated it yet. When the software is ready, we’ll turn it on and there will no longer be any drift or limit to how and when you can use your Smartgloves. This is what the interactive world has been waiting for. An affordable, precise way to interact with virtual worlds. It should be pretty transformative.