vi­con: Mas­ters of Mo­tion Cap­ture

Tim Dou­ble­day tells us how Vi­con are con­tin­u­ing to push the bound­aries of mo­tion cap­ture, in­clud­ing their work on the real-time dig­i­tal char­ac­ter Siren

3D World - - CONTENTS - Find out more about Vi­con’s work at­

We chat to mo­cap prod­uct de­vel­op­ers Vi­con about their work bring­ing dig­i­tal hu­man Siren to life, as well as the po­ten­tial fu­ture of mo­tion cap­ture tech­nol­ogy

“Mo­tion cap­ture or the idea of record­ing some­one’s mo­tion has been around for a very long time,” says Vi­con’s VFX prod­uct man­ager Tim Dou­ble­day. He refers to ro­to­scop­ing, a tech­nique that in­volves a hu­man trac­ing over film footage to pro­duce re­al­is­tic move­ment. “It was in­vented by Max Fleis­cher all the way back in 1915,” he adds.

Dou­ble­day de­fines mo­cap as the abil­ity to record a per­son’s move­ment, in­clud­ing ev­ery­thing from fa­cial to fin­ger move­ment. “The kind of op­ti­cal mo­tion cap­ture that we know today came to promi­nence in 2001 thanks to Andy Serkis’s per­for­mance as Gol­lum in The Lord of the Rings,” he ex­plains. “Nowa­days, mo­tion cap­ture is used not only in the ma­jor­ity of video games and an­i­mated movies but also in tech­nolo­gies like ro­bot­ics, VR and even drone track­ing.”

Vi­con has been lead­ing the charge in mo­tion cap­ture for the past 30 years, having come out of re­search un­der­taken by two PHD stu­dents at the Univer­sity of Strath­clyde in the late Sev­en­ties. The com­pany it­self was then es­tab­lished in 1984 as a man­age­ment buy-out from the Ox­ford In­stru­ments Group.

“It was the in­tro­duc­tion of re­al­is­tic 3D graph­ics that pro­pelled the use of mo­tion cap­ture in the en­ter­tain­ment in­dus­try,” ex­plains Dou­ble­day. “The pop­u­lar­ity of gam­ing sys­tems like the Playsta­tion and Xbox meant that game de­vel­op­ers needed re­al­is­tic hu­man mo­tion in their games. Whether it’s FIFA, Call of Duty or Fort­nite, the ma­jor­ity of today’s video games use mo­tion cap­ture.”

As they con­tinue to pi­o­neer the tech­nol­ogy, Vi­con have been in­volved in bring­ing the real-time dig­i­tal hu­man Siren to life, along with epic Games and Ten­cent. “Orig­i­nally de­signed as a demo for a chi­nese au­di­ence, Siren was re-imag­ined as a dig­i­tal hu­man show­case at GDC 2018,” con­tin­ues Dou­ble­day. “This meant find­ing a new english-speak­ing ac­tress, Alexa Lee, who per­formed as Siren dur­ing the show. cu­bic Mo­tion de­liv­ered stun­ningly re­al­is­tic fa­cial an­i­ma­tion all in real-time and Vi­con supplied the body and fin­ger an­i­ma­tion. All this tech­nol­ogy brought Siren to life and helped de­liver a be­liev­able and en­gag­ing dig­i­tal hu­man live on stage.”

As Dou­ble­day ex­plains, Siren has grand im­pli­ca­tions for the fu­ture of mo­cap: “Tra­di­tion­ally, high-end per­for­mance cap­ture has been used to cre­ate dig­i­tal char­ac­ters like Supreme Leader Snoke in the new Star Wars films or the more stylised char­ac­ters seen in Ready Player One. Th­ese movies all re­quire a ton of work in post and take hours to ren­der each frame. Video games now fea­ture hy­per-re­al­is­tic char­ac­ters, like those seen in Detroit:

Be­come Hu­man. While th­ese are ren­dered in real-time they still re­quire a lot of an­i­ma­tion work to make them look so be­liev­able.”

He con­tin­ues: “Siren was de­signed to not only run in real-time, but also be driven in real-time thanks to 3Lat­eral’s stun­ning fa­cial rig that runs, in-en­gine, at 60fps as well as cu­bic Mo­tion’s low-la­tency fa­cial solver. This al­lowed Siren to de­liver a high level of per­for­mance all in real-time. While the qual­ity won’t be as high as in some film and game char­ac­ters, it’s a big leap for­wards and al­lows an­i­ma­tors and artists to fo­cus on adding their fi­nal touches to the per­for­mance.”

Vi­con were re­spon­si­ble for an­i­mat­ing Siren’s hands and body through­out the project, util­is­ing 18 of their own lowla­tency cam­eras and mo­tion-cap­ture plat­form, Shogun. “La­tency was key for this project. If the an­i­ma­tion was out of sync with Lexi’s per­for­mance the whole piece would fall apart,” Dou­ble­day ad­mits. “To re­duce la­tency fur­ther, we solved the mo­tion-cap­ture mark­ers di­rectly onto the Siren cus­tom skele­ton. This was then streamed di­rectly into

“Cu­bic Mo­tion de­liv­ered stun­ningly re­al­is­tic fa­cial an­i­ma­tion all in real-time and vi­con supplied the body and fin­ger an­i­ma­tion. all this tech­nol­ogy brought siren to life” Tim Dou­ble­day, VFX prod­uct man­ager, Vi­con

Un­real en­gine us­ing epic’s new Live Link sys­tem.”

Pro­duc­ing re­al­is­tic fin­ger an­i­ma­tion re­quired Vi­con to em­ploy an en­tirely new so­lu­tion: “We placed mark­ers on each of Lexi’s fin­gers in a zig-zag pat­tern. This al­lowed for each fin­ger to be an­i­mated in­di­vid­u­ally with­out let­ting the mark­ers get so close to­gether that they would be hid­den. This helped de­liver high-fidelity fin­ger an­i­ma­tion running in real-time.”

Dou­ble­day con­tin­ues: “each morn­ing at GDC we would ad­just the Siren skele­ton based on the new marker po­si­tions used on Lexi’s suit. To do this, we would get Lexi to stand in an ‘A pose’ with her hands fac­ing for­wards. Since we were stream­ing in real-time, we could see the re­sults in both Shogun and Un­real. We could then pause the real-time stream and make ad­just­ments to the skele­ton in real-time and see the re­sults up­date live in Un­real.”

Siren rep­re­sents a mile­stone in mo­tion­cap­ture tech­nol­ogy, but the tech­nol­ogy con­tin­ues to de­velop at an as­ton­ish­ing rate. Dou­ble­day has ideas about where it might be headed: “While mo­tion-cap­ture sys­tems are still con­sid­ered high-end so­lu­tions, the cost has dropped dra­mat­i­cally over the last five years. It’s not quite at the point of having a Vi­con sys­tem in your liv­ing room, but the in­tro­duc­tion of lo­ca­tion-based VR means you might see one in your lo­cal cin­ema com­plex.

“One re­cent ex­am­ple is that Dream­scape Im­mer­sive re­cently launched the Alien Zoo vir­tual re­al­ity experience in Los An­ge­les, al­low­ing six peo­ple to go on a vir­tual sa­fari and in­ter­act with each other as well as the an­i­mals they en­counter on the way. each par­tic­i­pant wears a clus­ter of mark­ers on their hands and feet, with ad­di­tional clus­ters on the back­pack Pc and VR head­set. Th­ese six clus­ters are used to cre­ate a real­is­ti­cally pro­por­tioned 3D char­ac­ter within the vir­tual world. This al­lows peo­ple to shake hands and in­ter­act with each other, mak­ing for an in­cred­i­bly im­mer­sive experience. Lo­ca­tion-based vir­tual re­al­ity is a way for you and a load of your friends to all go and experience vir­tual re­al­ity in a so­cial en­vi­ron­ment.”

“it’s Not quite at the point of having a vi­con sys­tem in your liv­ing room, but the in­tro­duc­tion of lo­ca­tion-based vr Means you Might see one in your lo­cal Cin­ema Com­plex” Tim Dou­ble­day, VFX prod­uct man­ager, Vi­con

Far left: cu­bic Mo­tion pro­vided the fa­cial per­for­mance cap­ture, track­ing, solv­ing and an­i­ma­tion for siren

left: siren’s fa­cial rig and un­der­ly­ing con­trols were both pro­vided by 3lat­eral, which also han­dled all of the 3D and 4D scans of the per­for­mance

Be­low: since the 1980s much of vi­con’s work has been in mo­tion cap­ture for clin­i­cal gait anal­y­sis

the char­ac­ter of siren was cre­ated us­ing the like­ness of chi­nese ac­tress Bingjie Jiang

Newspapers in English

Newspapers from Australia

© PressReader. All rights reserved.