3D World

Rokoko: Mo­tion cap­ture made easy

Rokoko CEO and founder Jakob Bal­slev gives 3D World six rea­sons to be ex­cited for their game-chang­ing Smart­gloves

- Productivity · Tech · Lifehacks · Finland · Blender · Harry Houdini · Cinema 4D

Find out more about the rev­o­lu­tion­ary Smart­gloves


Rokoko, the creators of ac­ces­si­ble char­ac­ter an­i­ma­tion and mo­tion cap­ture tools, is putting the fu­ture in its users’ hands with the rev­o­lu­tion­ary Smart­gloves.

The no­to­ri­ously dif­fi­cult art of an­i­mat­ing 3D fin­ger and hand mo­tion has his­tor­i­cally been out of reach for the vast ma­jor­ity of creators. With pairs avail­able for $995 and a com­mit­ment to ease of use, Rokoko aims to make pro­fes­sional-qual­ity fin­ger track­ing a re­al­ity for all creators. The gloves can even be paired with Rokoko’s in­tu­itive Smart­suit Pro to cre­ate a full-body mo­cap so­lu­tion. Rokoko’s founder and CEO Jakob Bal­slev spoke to 3D World about why the com­mu­nity should be ex­cited about this pi­o­neer­ing so­lu­tion.


My grad­u­a­tion project at film school was this su­per am­bi­tious an­i­ma­tion where we used mo­tion cap­ture to live-ren­der the data from ac­tors and show them as a live an­i­ma­tion in front of an au­di­ence. We pulled it off but the only mo­cap op­tion we had was to buy an ex­pen­sive op­ti­cal sys­tem. You needed a ded­i­cated stu­dio, a whole staff of peo­ple that knew all kinds of stuff, and it was in­cred­i­bly ex­pen­sive. That’s what spurred the idea of in­vent­ing our own sys­tems. It had to be af­ford­able, some­thing I could op­er­ate my­self and could use any­where. We took sen­sors like those in your cell­phone that each has a gyro, an ac­celerom­e­ter, and a com­pass. We built them into this tex­tile suit with a lit­tle com­puter that sends through wi-fi to your com­puter or smart de­vice. All the tech is in­side a tex­tile suit, so if you know how to put on your clothes, you know how to set up the sys­tem. With op­ti­cal sys­tems, you need dis­tance from walls and you can’t have di­rect sun­light be­cause of in­frared. All of that is gone, now you can use it any­where.


Ini­tially, the Smart­gloves were planned as part of the Smart­suit Pro. We naively thought they were two sides of the same coin and they just weren’t. Fin­ger move­ments are so com­plex and un­pre­dictable in com­pletely dif­fer­ent ways to the body. When you have an in­er­tial sen­sor-based sys­tem it’s all math, there’s no cam­era lock­ing it in place. It’s all based on our al­go­rithms and un­der­stand­ing of kine­mat­ics. That’s eas­ier with the body than the fin­gers. We de­cided to ship the suit, and good thing we did be­cause it’s been three years. Now we have many thou­sands of them out at smaller stu­dios that never had ac­cess to mo­tion cap­ture be­fore. If you’re an an­i­ma­tor sit­ting at your desk and you get an idea you can just stand up, record it, and look at it. If it wasn’t what you wanted, then you just saved your­self two weeks. We ended up us­ing ev­ery­thing we learned from the suit and then worked re­ally hard on some rules for what the Smart­gloves could do.


With the Smart­gloves, the big­gest is­sue we had was drift. Drift is the big thing with in­er­tial sen­sor-based sys­tems. The beauty is you can use them any­where and you never lose any data. The down­side is that they can drift over time be­cause there’s no op­ti­cal sys­tem hold­ing them in place. With a suit, you can do stuff to cor­rect it but for gloves, it’s much more tricky. It’s very clear if fin­gers are over­lap­ping or the hand sud­denly looks odd. So that’s what we’ve been re­ally fo­cus­ing on and try­ing to per­fect. There are dif­fer­ent kinds of drift. There’s gy­ro­scope drift and there’s mag­netic in­ter­fer­ence.

The mag­ne­tome­ter on the suit can drift if you’re close to metal or a mag­net of some sort. That’s a big prob­lem, es­pe­cially with fin­gers as you could be touch­ing your com­puter or a metal beam. What we did was re­move the mag­ne­tome­ter from the gloves so they’re not sen­si­tive to in­ter­fer­ence from metal at all. With some of the other prod­ucts in the field if your hands get mag­ne­tised you’ll have to ship it back. That is some­thing that can­not hap­pen with the Smart­gloves.


Another thing we did for the Smart­gloves was to add a wrist sen­sor. You have seven sen­sors – one for each fin­ger, one on the back of your hand, and then you have one on your wrist. This was im­por­tant for us. Other sys­tems don’t have hand track­ing, they have fin­ger track­ing but don’t know what the hand is do­ing. The great thing about the Smart­gloves is if you’re us­ing them with­out a suit you can pretty ac­cu­rately cal­cu­late the en­tire arm based on what the wrist is do­ing.


Users be­ing able to stream their data into other 3D tools is some­thing we’re re­ally fo­cused on. In Rokoko Stu­dio Live users can stream their recorded or live data di­rectly into our na­tive plug­ins for Unity, Unreal, Maya, Blender, Hou­dini, Cinema 4D, and more. You set up the gloves with Stu­dio but then you stream it into the plugin. You then have a panel in­side the plugin where you can con­trol Stu­dio. You need to open the Stu­dio but you can do ev­ery­thing in­side of the plugin. You can do your cal­i­bra­tion pose, start your record­ing, start stream­ing, any­thing. We know that this is some­thing users are very con­cerned with. They don’t want more clicks or more apps, they just want to work where they are and in the en­vi­ron­ment that they are com­fort­able with.


A ma­jor up­grade will be com­ing to our Smart­glove tech­nol­ogy in 2021. For the past five years, we’ve been re­search­ing the op­ti­mal way to com­bine the free­dom you get with sen­sor-based so­lu­tions with the fixed po­si­tion of op­ti­cal sys­tems. This tech­nol­ogy is al­ready in the Smart­gloves, we just haven’t ac­ti­vated it yet. When the soft­ware is ready, we’ll turn it on and there will no longer be any drift or limit to how and when you can use your Smart­gloves. This is what the in­ter­ac­tive world has been wait­ing for. An af­ford­able, pre­cise way to in­ter­act with vir­tual worlds. It should be pretty trans­for­ma­tive.

 ??  ?? Seven sen­sors, in­clud­ing one on the wrist, help to en­sure ac­cu­rate track­ing of the en­tire arm as well as the hand/fin­gers
Seven sen­sors, in­clud­ing one on the wrist, help to en­sure ac­cu­rate track­ing of the en­tire arm as well as the hand/fin­gers
 ??  ??
 ??  ??
 ??  ?? The Smart­gloves can be used as a stand­alone de­vice, but they can also be synced to the Rokoko Smart­suit Pro
The Smart­gloves can be used as a stand­alone de­vice, but they can also be synced to the Rokoko Smart­suit Pro
 ??  ??

Newspapers in English

Newspapers from Australia