3D World

Oped: Aug­ment­ing AR

James Bur­rows, tech­ni­cal di­rec­tor at Im­mer­sive Stu­dios, on how we’re sidestep­ping the lim­i­ta­tions of hand­held de­vices

-

James Bur­rows of Im­mer­sive Stu­dios talks AR ex­pe­ri­ences

“QUITE SIM­PLY, THIS PROCESS RAISES THE BAR FOR THE VIS­UAL QUAL­ITY”

By now, we’re all well-versed in what aug­mented re­al­ity can do. It lets us place vir­tual ob­jects in real en­vi­ron­ments and in­ter­act with them through a hand­held de­vice. Ever since Poké­mon Go’s launch, the ca­pa­bil­i­ties of aug­mented re­al­ity have be­come more and more so­phis­ti­cated. But there are still lim­i­ta­tions – and vis­ual qual­ity is one of them. That’s why, at Im­mer­sive Stu­dios, we’ve been ex­per­i­ment­ing with ways to aug­ment the vis­ual as­sets in AR ex­pe­ri­ences with­out over­load­ing the de­vices used to view them. Tech­ni­cal di­rec­tor James Bur­rows ex­plains.

STRIV­ING FOR MORE

A lit­tle while back, Nvidia shared a demo for its 5G-based Cloud XR where it streamed highly de­tailed CAD mod­els to VR and AR de­vices. What was strik­ing was see­ing AR being able to sup­port the level of de­tail in a CAD model. At Im­mer­sive, we cre­ate AR ex­pe­ri­ences for a range of uses and in­vari­ably come up against the same bal­anc­ing act: how to get the best vis­ual qual­ity for the as­sets while keep­ing the app’s size down. It’s al­ways a bit of a com­pro­mise, although AR plat­forms are evolv­ing all the time. How­ever, we wanted to push the bound­aries a bit fur­ther to find a way of build­ing AR ex­pe­ri­ences with high-fidelity, Pc-grade graph­ics our­selves. So we did.

PUSH­ING THE TECH BOUND­ARIES

We de­vised a way of us­ing video stream­ing to en­able a cen­tral PC to stream high-end as­sets to a hand­held de­vice, where they can be viewed in real-time in AR – es­sen­tially tak­ing the job of ren­der­ing 3D as­sets away from the de­vice and giv­ing it to a more pow­er­ful com­puter in­stead. In a nut­shell, the phone sends po­si­tional in­for­ma­tion to the PC, which ren­ders the frame in a game en­gine, that frame data is then en­coded into a video stream, which is trans­mit­ted back to the phone and com­pos­ited against the phone’s cam­era video feed.

WHAT DOES THIS MEAN?

This means that for any sit­u­a­tion where you need a desk­top-class GPU to get the best re­sults, you can now do it in AR on a cheaper de­vice. This opens up AR as a plat­form for com­plex 3D mod­els that mo­bile hard­ware wouldn’t usu­ally be able to ren­der.

SO WHAT ARE THE BEN­E­FITS?

• It looks bet­ter – Quite sim­ply, this process raises the bar for the vis­ual qual­ity, and even the in­ter­ac­tion avail­able, for AR – open­ing it up as a more use­ful tool ca­pa­ble of show­ing highly re­al­is­tic mod­els, in turn mak­ing the ex­pe­ri­ence more im­mer­sive.

• Stream to mul­ti­ple de­vices – With a cen­tral PC stream­ing the graph­ics over Wifi, it’s pos­si­ble to stream the same feed to mul­ti­ple de­vices with­out the need for multi­net­work­ing, al­low­ing mul­ti­ple users to see and in­ter­act with the same ses­sion.

• Prac­ti­cal – This process can make it prac­ti­cal for mod­ellers to pre­view their as­sets with­out hav­ing to build a whole app or push it to the App Store. In­stead, us­ing PC ren­der­ing, they can stream their mod­els to a de­vice and drop it into the real world eas­ily.

• Smaller app sizes – With the onus of ren­der­ing as­sets re­moved from the de­vice, the app size will come down, mak­ing it both more ap­peal­ing to down­load and en­abling you to have more fun with your ex­pe­ri­ence.

• Eas­ier to update – If your app’s con­tent is streamed from a cen­tral PC and the mo­bile app ef­fec­tively works as just the viewer, then you don’t have to update the app it­self if you need to make con­tent changes. In­stead, you can update via the cen­tral source.

• Level play­ing field – De­vices of dif­fer­ing lev­els of power and ca­pa­bil­ity can all ac­cess the same graph­i­cal ex­pe­ri­ence so long as they’re able to ren­der a sim­ple video feed.

WHAT NEXT?

At the mo­ment, this process re­quires a good Wifi con­nec­tion so cur­rently it will be more fea­si­ble in set­ups where you can con­trol the lo­cal Wifi net­work and how peo­ple con­nect to it. How­ever, with the up­com­ing ar­rival of 5G and Edge stream­ing, it’s fea­si­ble this way of run­ning com­plex AR ex­pe­ri­ences could be­come much more com­mon. In the mean­time, we’re en­joy­ing ex­pe­ri­enc­ing AR that looks re­ally, re­ally good.

Find out more at weareim­mer­sive.co.uk

?? ??
?? ??

Newspapers in English

Newspapers from Australia