VR: A TechLife primer


TechLife Australia - - WELCOME - [ DAR­REN YATES ]

IT’S BEEN DE­SCRIBED as ev­ery­thing from “the ul­ti­mate story-telling tool” to “gam­ing nir­vana”, but what­ever you call it, vir­tual re­al­ity (VR) is all about bend­ing your mind’s view on the world you’re look­ing at. And while it’s just come off a break­out year in 2016, VR looks set to start do­ing even big­ger busi­ness, with mar­ket re­search pre­dict­ing that as many as 52 mil­lion VR head­sets will be sold by the end of this decade ( tinyurl.com/jsa3pzs). Al­ready, many of the ma­jor tech brands from Face­book to Google, Mi­crosoft to Sony and HTC are in­vest­ing heav­ily in its de­vel­op­ment, cov­er­ing gam­ing PCs, consoles and smart­phones. But what is VR, how does it work and what ap­pli­ca­tions does it have be­yond gam­ing?


By smart­phone and con­sole stan­dards, the tech in most VR head­sets is com­par­a­tively sim­ple. At a ba­sic level, a VR head­set is not much more than a dis­play de­vice — it takes in a video in­put (nor­mally HDMI) and con­verts it into a stereo out­put to feed one or two small OLED dis­play pan­els, de­pend­ing on the head­set de­sign (the Ocu­lus Rift CV1 has two sep­a­rate OLED pan­els, one for each eye, while Sony’s PlayS­ta­tion VR has just the one). Ei­ther way, you view the panel/s through two spe­cial lenses that al­low your eyes to fo­cus on some­thing just two fin­ger-widths from your eye­balls.

The video-to-OLED con­ver­sion is usu­ally per­formed by a ded­i­cated chip known as an ‘in­ter­face bridge’, which takes in the HDMI sig­nal and out­puts a ‘Dis­play Se­rial In­ter­face’ (DSI) data stream that ul­ti­mately ap­pears on the OLED pan­els. In the case of the PlayS­ta­tion VR and Ocu­lus Rift CV1, that chip is Toshiba’s TC358870XBG, the first de­signed to work with high-res­o­lu­tion ‘4K’ video.


Un­like a com­puter pro­ces­sor, this Toshiba chip is a sin­gle-func­tion de­vice, mean­ing it has one job to do, but it needs to be con­trolled by a spe­cial type of com­puter pro­ces­sor called a ‘mi­cro­con­troller’. Your smart­phone has a main gen­eral-pur­pose pro­ces­sor or ‘CPU’ to run your apps, but it also uses mi­cro­con­trollers to han­dle spe­cific tasks such as telecom­mu­ni­ca­tion con­nec­tions and Wi-Fi. Mi­cro­con­trollers are used in many ap­pli­ca­tions where smaller amounts of com­put­ing power are re­quired, rather than a full-blown com­puter. Your fit­ness band will have one, for ex­am­ple.

Un­like com­puter CPUs, mi­cro­con­trollers come with their own stock of mem­ory and stor­age to store data and code. Yet like their CPU cousins, mi­cro­con­trollers come in var­i­ous per­for­mance grades. What might sur­prise you is that most VR head­sets only need the very en­try-level of mi­cro­con­trollers to work, mostly be­cause all of the hard work is done by spe­cific chips like that Toshiba in­ter­face bridge. The Ocu­lus Rift has an ST Mi­cro­elec­tron­ics STM32F070 mi­cro­con­troller, the PSVR uses a Nu­vo­ton NUC123SD4 — both are from the ARM Cor­tex M0 series, which is the ‘baby brother’ de­sign to the Cor­tex A-series pow­er­ing your smart­phone and just about ev­ery other phone, right up to the lat­est Galaxy S7.

Check out the ex­cel­lent tear-downs at iFixit of the Rift CV1 ( tinyurl.com/jrofglj) and PSVR ( tinyurl.com/h4­tyqbf) for all the geeky de­tails.


But the real trick of VR is its abil­ity to track your head move­ment and up­date the dis­play ac­cord­ingly as if you’re ac­tu­ally mov­ing through a real world. That’s done with the help of what’s called an ‘in­er­tia mea­sure­ment unit’ (IMU) sen­sor. Ev­ery smart­phone has one — it’s the de­vice that tells your phone when you’re hold­ing it in ver­ti­cal (por­trait) or hor­i­zon­tal (land­scape) mode. How­ever, IMU sen­sors can do much more than that — for ex­am­ple, they’re also used in quad­copters/drones to sta­bilise flight and check gen­eral axis-of-di­rec­tion. The CV1 and PSVR both use a high-res­o­lu­tion BMI055 six-axis IMU sen­sor from Ger­man

in­dus­trial gi­ant Bosch. These sen­sors are of­ten de­scribed as be­ing ‘6DOF’ or ‘9DOF’ (de­grees of free­dom), but you’ll also see them listed as ‘six-axis’ or ‘nine-axis’ de­vices. In 3D space, you have three stan­dard axes — X and Z at right-an­gles hor­i­zon­tally and the ver­ti­cal Y-axis. When a de­vice has a six- or nine-axes rat­ing, it means it has mul­ti­ple sen­sors, able to mea­sure dif­fer­ent phys­i­cal as­pects across the three axes. For ex­am­ple, the BMI055 has a tri-axis ac­celerom­e­ter, for de­tect­ing for­ward and back­ward (called ‘lin­ear’) move­ment and grav­ity, plus a tri-axis gy­ro­scope, which mea­sures the rate of ro­ta­tion of an ob­ject about an axis. As you move your head, the ac­celerom­e­ter and gy­ro­scope mea­sure and record that move­ment down to minute de­tail. That data is sent back to your PC or con­sole via the mi­cro­con­troller.

Get­ting that im­por­tant data back to your PC or con­sole is the role of the sec­ondary (usu­ally USB) con­nec­tion. The re­turn data pro­vides feed­back to the soft­ware or game, which then ad­justs the dis­play to match your move­ment, trick­ing your brain into think­ing the vi­sion you see mov­ing around you is gen­uine.


But one ques­tion you might have is if the PSVR and other sim­i­lar head­sets all need spe­cialised chips, dis­plays and mi­cro­con­trollers to cre­ate VR, how does Google’s low-cost Card­board VR sys­tem work if all it has is just a card­board frame and a cou­ple of lenses? It all re­lies on your smart­phone, which is re­ally a pow­er­house com­puter that con­tains all the hard­ware needed to make ba­sic VR work — it has the dis­play, mi­cro­con­trollers and IMU sen­sors (these sen­sors aren’t typ­i­cally as pre­cise as the ones in­side VR head­sets, but they’ll do). It doesn’t need HDMI or USB con­nec­tions since the smart­phone is fully self-con­tained. Af­ter that, it’s just a mat­ter of hav­ing the lenses and the right soft­ware to take ad­van­tage of the combo.


So far, the most com­mon and recog­nis­able ap­pli­ca­tion for VR is gam­ing, which has been the fo­cus for most com­mer­cial de­vel­op­ment. How­ever, there is con­tin­u­ing re­search into the use of VR in other ap­pli­ca­tions, par­tic­u­larly med­i­cal ther­a­pies, cov­er­ing ev­ery­thing from men­tal health and psy­chol­ogy to neu­ro­science and mo­tor skills re­ha­bil­i­ta­tion. The Univer­sity of South­ern Cal­i­for­nia’s In­sti­tute for Cre­ative Tech­nolo­gies, for ex­am­ple, has set up the MedVR Lab ( medvr.ict.usc.edu) to fur­ther re­search how VR can be utilised. Its pri­mary goal is men­tal and be­havioural health, de­vel­op­ing tech­niques for as­sess­ment and treat­ment of stress dis­or­ders such as anx­i­ety and PTSD. At the other ex­treme, VR is even bring­ing med­i­cal stu­dents (and oth­ers) right into the ac­tion — the Royal Lon­don Hospi­tal streamed the first live surgery via VR on­line in April 2016 ( tinyurl.com/z64e9uc).


As the pop­u­la­tion ages, one of the most de­bil­i­tat­ing dis­or­ders af­fect­ing greater num­bers of older peo­ple is de­men­tia, which in­creas­ingly robs suf­fer­ers of their lu­cid­ity and mem­o­ries. Early tests con­ducted by Alzheimer’s Aus­tralia found that a new VR game de­vel­oped by the or­gan­i­sa­tion led to a sig­nif­i­cant re­duc­tion in the amount of med­i­ca­tion re­quired by some de­men­tia pa­tients ( tinyurl.com/hfj4r7e). Called ‘the Vir­tual For­est’ ( tinyurl.com/

zh­foszf), the sys­tem uses Mi­crosoft’s Kinect tech­nol­ogy to en­able users to move ob­jects about in a vir­tual for­est by sim­ple hand move­ments. It re­quires a quad-core Win­dows PC with an Nvidia GTX 970 graph­ics card, 8GB of RAM and Kinect adapter for Win­dows.

It fol­lows on from Ed­u­ca­tion De­men­tia Im­mer­sive Ex­pe­ri­ence (EDIE), an ini­tia­tive launched by Alzheimer’s Aus­tralia in Septem­ber 2016, to give ev­ery­one a taste through a smart­phone VR app and Google’s Card­board VR head­set of what it’s like for peo­ple liv­ing with de­men­tia ( vic.fight­de­men­tia.org.au/edie).


VR has also long helped pi­o­neer new treat­ments in stroke re­ha­bil­i­ta­tion. In 2000, re­searchers at Rut­gers Univer­sity, New Jer­sey, re­ported in the IEEE Trans­ac­tions on Neu­ral Sys­tems and Re­ha­bil­i­ta­tion En­gi­neer­ing of us­ing force-feed­back gloves to al­low stroke pa­tients who lost hand func­tion to move through a vir­tual en­vi­ron­ment and pro­vide re­ha­bil­i­ta­tion ( tinyurl.com/hmv557g). In Aus­tralia, a team from the Mur­doch Univer­sity School of En­gi­neer­ing and IT, in con­junc­tion with the Western Aus­tralian Neu­ro­science Re­search In­sti­tute, cre­ated a VR app called Neu­romender. In­side, pa­tients fly a vir­tual wing-man suit through a vir­tual cloud­scape, while a com­puter mon­i­tors the pa­tient’s up­per body strength through a series of au­to­mat­i­cally-ad­justed chal­lenges ( tinyurl.com/hq9vno5).


Just about ev­ery new tech­nol­ogy at­tract­ing eye­balls also at­tracts ad­ver­tis­ers, and VR is no ex­cep­tion. Right now, you al­most have your choice of mar­ket re­searchers of­fer­ing re­ports on how VR will change the face of mar­ket­ing. We’ve read oth­ers say VR will so change the world, no-one will any longer do busi­ness in the ‘real world’. Puh-lease!


While the more ex­pen­sive VR head­set op­tions prom­ise a more im­mer­sive ex­pe­ri­ence, you can still get a feel for what VR is for as lit­tle as $20, thanks to Google Card­board and Card­board clones ( tinyurl.com/zvlzpw2). They turn your smart­phone into a VR de­vice, us­ing its built-in screen and IMU sen­sors. The trick is in the lenses that make it pos­si­ble for your eyes to fo­cus on your phone screen at such close dis­tance. Don’t for­get the Card­board demo app from Google Play ( tinyurl.com/oyed­b3g) — it should work on any Android de­vice with at least Android 4.1/Jelly Bean. There are al­ter­na­tives, but it’s a good start.


Mak­ing your own con­tent is a lit­tle more dif­fi­cult, but if you love your cre­ative side, it’s in­fin­itely more fun and there are a num­ber of dif­fer­ent ways you can go. De­pend­ing on your skill and bud­get, the most ba­sic is to cre­ate ‘360 x 180-de­gree’ still images. With a fish-eye lens at­tach­ment, you can cap­ture these images us­ing a smart­phone and stitch them to­gether us­ing apps such as PT­Gui ( pt­gui.com), Hugin ( hugin.source­forge.net) and GoPro’s Kolor ( kolor.com).

At the other end of the scale is the new wave of 360° cam­eras specif­i­cally de­signed for cap­tur­ing VR video. While the more pro­fes­sional rigs in­clude mul­ti­ple cam­eras point­ing ev­ery which way, new bud­get sin­gle­lens cam­eras, in­clud­ing the pop­u­lar Ri­coh Theta S, are a good place to start. The up­com­ing dual-lens Kodak PixPro 4KVR360 also looks promis­ing.

And if you want to share your con­tent, Face­book and YouTube al­ready of­fer 360° video up­loads and play­back, whether it’s through a stan­dard web browser on your PC and head­set or, in the case of Google, through your smart­phone and Google Card­board.

VR OR 360°?

How­ever, the real de­bate amongst purists is what to call this con­tent — is it VR con­tent or is it just 360° video? The ar­gu­ment re­volves around the fact that mak­ing your own con­tent with 360° cam­eras doesn’t au­to­mat­i­cally gain you en­try into the VR club. In most cases, 360° video is just that — video. You en­joy the ride, but you can’t con­trol the ac­tion as you would in gen­uine VR. It might seem an ar­gu­ment bet­ter left to the purists, but like the mess ‘HD’ TVs dis­solved into, it may af­fect your pur­chases one day, so it’s im­por­tant to be across the ideas now.


So now that VR has ar­rived in gam­ing consoles and smart­phones, where does the tech­nol­ogy go from here? The first ma­jor steps on the dis­play side are con­tin­ued im­prove­ment in screen res­o­lu­tion, as well as get­ting rid of the wires and mak­ing the whole sys­tem com­pletely wire­less. Right now, teams from Rivvr ( tinyurl.

com/hj9d­h9j) and oth­ers are work­ing on dif­fer­ent al­ter­na­tives to cut­ting the cables — Rivvr’s so­lu­tion will work with Ocu­lus Rift and HTC’s Vive, the up­com­ing Su­lon Q ( su­lon.

com) is a stand­alone wire­less head­set. There are also ef­forts to im­prove the ‘touch’ side of things, with more re­spon­sive hand-con­trollers that let you nav­i­gate your way through a vir­tual re­al­ity with greater pre­ci­sion and dex­ter­ity.

On the con­tent-gen­er­a­tion side of things, 2017 should be a break­out year for low-cost com­pact con­sumer-grade 360° video cam­eras that make cap­tur­ing spher­i­cal video a snap. While we saw a few in 2016, the flood gates should open this year.

But while there’s been plenty of progress in VR be­com­ing main­stream over the last 12 months or so, it’s the growth in VR re­search, par­tic­u­larly in med­i­cal ther­a­pies cov­er­ing ev­ery­thing from men­tal health to den­tistry and be­yond, that could po­ten­tially lead to a far more en­rich­ing fu­ture for VR in the real world.

For help with de­men­tia, call the Na­tional De­men­tia Helpline on 1800 100 500.

The PlayS­ta­tion VR com­bines mo­tion sen­sor with video dis­play tech­nol­ogy.

Rivvr’s new wire­less VR mod­ule for Ocu­lus Rift and HTC Vive cuts the wires.

Plenty of chips drive the Ocu­lus Rift CV1. (Source: ifixit.com)

Sam­sung’s new Gear 360 cam­era makes VR video cap­ture af­ford­able.

The Fit­bit Alta uses a sim­i­lar Cor­tex-M chip to that found in most VR head­sets.

PlayS­ta­tion VR uses sim­i­lar tech to the Ocu­lus Rift CV1. (Source: ifixit.com)

EDIE helps you ex­pe­ri­ence what it’s like to live with de­men­tia. (Source: Alzheimer’s Aus­tralia)

Newspapers in English

Newspapers from Australia

© PressReader. All rights reserved.