Aug­mented re­al­ity: A TechLife primer


TechLife Australia - - WELCOME - [ DAR­REN YATES ]

IF YOU WERE look­ing for a can­di­date for ‘Mo­bile Game of 2016’, it’d be dif­fi­cult to go passed Poké­mon GO. The game it­self man­aged to gen­er­ate a stag­ger­ing US$950 mil­lion over the sec­ond half of the year and in­stantly pop­u­larised this seem­ingly new tech­nol­ogy col­lo­qui­ally re­ferred to as ‘AR’. But what is ‘aug­mented re­al­ity’, how does it dif­fer from vir­tual re­al­ity and, im­por­tantly, where are we likely to see it next?


Last month, we found that what con­sti­tutes ‘vir­tual re­al­ity’ is up for de­bate, and there’s prob­a­bly even more ques­tions sur­round­ing how to best de­fine ‘aug­mented re­al­ity’. For many of us, our first con­tact with what usu­ally passes for AR was a holo­gram of a feisty space princess in Star Wars: A New Hope.

But the tech in­dus­try wants to broaden hori­zons and avoid pi­geon-hol­ing AR as just fancy holo­graphic dis­plays. An­other def­i­ni­tion gain­ing trac­tion is that AR is any ‘real world’ view over­layed with dig­i­tal data, whether vi­sion, sound or graph­ics. Yep, that’s pretty broad, but it does en­sure AR doesn’t have to nec­es­sar­ily in­volve strap­ping on a head­set. In fact, you’ve prob­a­bly al­ready en­joyed plenty of AR in the com­fort of your own liv­ing room, such as the ‘Hawk-Eye’ ball-track­ing over­lays used by the Nine TV Net­work dur­ing cricket cov­er­age and the Seven Net­work dur­ing the Aus­tralian Open.

When it comes to de­ci­pher­ing the dif­fer­ence be­tween aug­mented and vir­tual re­al­ity, it’s much eas­ier — aug­mented re­al­ity still deals in the real world as its base, whereas even re­al­ity it­self is dig­i­tally loaded in VR. De­spite plenty of re­cent VR ex­am­ples, Poké­mon GO clearly holds reign as the pop-cul­ture icon for AR over the last 12 months.


But while Poké­mon GO may have in­tro­duced the main­stream to AR, in re­al­ity, AR is hardly new — in fact, you could ar­gue its ori­gins date back to World War II. The scourge of night bomb­ing over Lon­don dur­ing the early pe­riod of the war drove the Royal Air Force (RAF)

to de­velop tech­nol­ogy to deal with the men­ace. The en­gi­neers and sci­en­tists at the Telecom­mu­ni­ca­tions Re­search Es­tab­lish­ment in­vented Air In­ter­cept radar to help fighter pi­lots lo­cate en­emy air­craft at night. Ac­cord­ing to the book Bri­tish Air­craft Ar­ma­ment Vol. 2: RAF Guns and Gun­sights from 1914 to the Present Day by R Wal­lace Clarke, one of the later mod­i­fi­ca­tions de­vel­oped dur­ing 1942 in­volved us­ing a cath­ode-ray tube (CRT) to project an im­age of the radar dis­play, a gun­sight ring and an ar­ti­fi­cial ‘hori­zon’ onto the air­craft wind­screen — the world’s first head-up dis­play. De­spite some suc­cess dur­ing tri­als, test pi­lots re­port­edly of­fered plenty of sug­gested im­prove­ments, and with de­vel­op­ment ef­forts more fo­cussed on radar it­self, the Pi­lot’s In­di­ca­tor Dis­play never made it into ac­tive ser­vice dur­ing the war. A prac­ti­cal sys­tem for air­craft de­tec­tion us­ing ra­dio waves had only been proven to work in 1935, so for sci­en­tists and en­gi­neers to have a work­ing pre­de­ces­sor of to­day’s HUD by 1942 us­ing tech­nol­ogy many would now con­sider pre­his­toric was im­pres­sive.

Since those early days, the com­put­ing horse­power re­quired to power most AR ap­pli­ca­tions has kept the con­cept out of the lounge­room. How­ever, thanks to the rapid per­for­mance growth in mo­bile pro­ces­sors and the over­all fea­ture ca­pa­bil­ity of smart­phones, AR is now break­ing into the main­stream — and not just with Poké­mon GO.


If you’re not a Harry Pot­ter fan, skip this next bit, but if you’ve ever dreamt of in­cant­ing “Win­gardium leviosa!” and watch­ing an ob­ject lev­i­tate at the end of your wand, the op­por­tu­nity might be closer than a year at Hog­warts. Dig­i­tal artist Asad Ma­lik (aka ‘1RIC’) saw Poké­mon GO as in­spi­ra­tion to try a lit­tle magic of his own and de­vel­oped an MR world where spo­ken spells can be cast at the point of a wand (­cyo). His method is built around Mi­crosoft’s HoloLens AR holo­graphic computer to in­ter­pret hand ges­tures and wand move­ments as trig­gers for voice-con­trolled com­mands. At time of writ­ing, his sys­tem had three work­ing spells, in­clud­ing ‘Ex­pecto Pa­tronum’ for ward­ing off De­men­tors and ‘In­cen­dio’ for set­ting fire to ob­jects.

We’re not sure how this sits in terms of copy­right, but we’d be sur­prised if Harry’s cor­po­rate min­ders weren’t ka-ching­ing their cash reg­is­ters at the thought of fu­ture mer­chan­dis­ing pos­si­bil­i­ties. If noth­ing else, it’s a clever pop-cul­ture ex­am­ple of MR.


HoloLens has been avail­able in the US for most of the last year, but only be­gan ship­ping in Aus­tralia last Novem­ber. At the mo­ment, it’s only re­ally de­signed for soft­ware devel­op­ers (hence the ‘De­vel­op­ment Edi­tion’ tag) — and those with deep pock­ets. With a price tag start­ing at $4,369, it might still be a while yet be­fore the rest of us get our hands on one.

But if that sounds a bit too rich for your back pocket, how does $30 grab you? UK group Zap­par re­cently com­pleted a Kick­starter cam­paign to bring to life Zap­box, an af­ford­able al­ter­na­tive to HoloLens, but with greater con­trol and cam­era func­tion­al­ity (

In­stead of a built-in computer, Zap­box re­lies on your smart­phone to do the heavy-lift­ing of mix­ing re­al­i­ties, mak­ing it a kind of AR ver­sion of Google Card­board and its up­mar­ket cousin Day­dream View (see is­sue 59, page 14). How­ever, while Day­dream View re­quires (and cur­rently only works with) Google’s own Pixel and Pixel XL smart­phones and is a VR-only head­set plat­form, Zap­box of­fers a full AR ex­pe­ri­ence and is said to work with any An­droid de­vice with a 4.5-inch or larger dis­play panel and at least An­droid 4.1. Ba­si­cally, if you can run the Google Card­board demos ad­e­quately, you should be Zap­box-ready.

Mean­while, Google has re­port­edly stated that most of the ma­jor phone mak­ers will pro­duce new phone mod­els that are Day­dream­ready, in­clud­ing Sam­sung, LG, Al­ca­tel, Huawei, Xiaomi and ZTE.


GPS is the com­mon method for de­ter­min­ing a smart­phone’s cur­rent global lo­ca­tion and it’s the method Poké­mon GO uses to place the user in close prox­im­ity with a ‘pocket mon­ster’ to col­lect. But it’s also not par­tic­u­larly ef­fi­cient — the rate of data trans­mis­sion, for ex­am­ple, is as low as 50 bits per sec­ond (50bps). Com­pare that with even old dial-up modems that down­loaded data a thou­sand times faster and it takes a lot more time to get the in­for­ma­tion you need. Throw in the fact that your phone has to com­mu­ni­cate with satel­lites in space to get that data and it shouldn’t sur­prise that GPS is pretty thirsty when it comes to drain­ing your de­vice’s bat­tery life.

A cou­ple of years ago, search gi­ant Google set up Project Tango (, an AR/MR plat­form that, among other things, doesn’t need GPS to pre­cisely iden­tify lo­ca­tion. By the end of 2016, Len­ovo had pro­duced the first Tango-ready de­vice, the Phab 2 Pro. Mean­while, Tai­wanese maker ASUS is about to go a step fur­ther, com­bin­ing Tango with Day­dream in the up-com­ing ZenFone AR (­hbf4), a 5.7-inch smart­phone first shown at CES 2017 in Las Ve­gas and ex­pected for re­lease later this year.


Many of us won’t dare to walk out of the house with­out a smart­phone, even just to go to the cor­ner shop for milk. But it’s no se­cret that the bricks-and-mortar re­tail in­dus­try is fac­ing up to the on­slaught of on­line shop­ping, par­tic­u­larly with Ama­zon’s im­mi­nent ar­rival into the Aus­tralian mar­ket. Sports gi­ant Nike is hop­ing to rein­vig­o­rate the phys­i­cal shop­ping ex­pe­ri­ence the next time you walk into its Paris store. Here, shop­pers are able to cus­tom-de­tail their own shoe colours and styles through holo­graphic MR tech packed into a touch­screen kiosk (

How­ever, it’s not just bricks-and-mortar re­ceiv­ing an MR makeover. To­wards the end of last year, Chi­nese on­line re­tail­ing gi­ant Alibaba un­veiled Buy+, an MR/VR head­set app that prom­ises to rev­o­lu­tionise the on­line shop­ping ex­pe­ri­ence. Rather than just a bunch of web­page lists, buy­ers get a 360-de­gree view and are able to walk through a vir­tual depart­ment store, where they can pick up prod­ucts, spin them around and get a bet­ter idea of size, colour and form (

zh8ovxl). You can get a glimpse of the idea through Alibaba’s YouTube video (


For some of us, pri­mary school only got as in­ter­est­ing as play­ing around with Cuise­naire Rods. But to­day, there’s a grow­ing body of re­search ap­ply­ing AR to nu­mer­ous ar­eas of teach­ing from gen­eral teach­ing of Year 2 stu­dents to more spe­cific ap­pli­ca­tions such as teach­ing math­e­mat­ics and ge­om­e­try (

More prac­ti­cally, there are dozens of AR apps de­signed to help teach stu­dents di­rect from their smart­phone or tablet. For ex­am­ple, El­e­ments 4D is a cool app us­ing AR to aid in teach­ing the prin­ci­ples of chem­istry and how el­e­ments com­bine. It uses wooden or paper blocks as tar­gets to launch AR video. You’ll find the app and de­tails for les­son plans via Google Play (

The suc­cess of Poké­mon GO will no doubt spawn an in­creased up­take of AR in the class­room, al­though that up­take will likely be de­pen­dent on costs ­— one rea­son why the rapid fall in AR tech prices will be watched with in­ter­est in ed­u­ca­tion cir­cles.


Not sur­pris­ingly, AR has some se­ri­ously use­ful ap­pli­ca­tions within medicine — and not nec­es­sar­ily with ex­ces­sive price tags. Re­searchers at the Mas­sachusetts In­sti­tute of Tech­nol­ogy (MIT), for ex­am­ple, have re­cently de­vel­oped a sys­tem that uses AR to automate dig­i­tal mea­sure­ment of med­i­cal in­stru­ments via smart­phones with­out the need for plug-in or wire­less de­vices. Ba­si­cally, any elec­tronic med­i­cal de­vice that plugs into some­thing — or some­one — to mea­sure any kind of vi­tal sign rightly re­quires reg­u­la­tory ap­proval.

But MIT re­searchers found that us­ing a sim­ple AR marker or ‘tar­get’ can pro­vide a ref­er­ence point for cap­tur­ing ac­cu­rate data from mea­sure­ment de­vices at ex­tremely low cost us­ing a smart­phone cam­era. The ex­am­ple the au­thors use is a peak flow me­ter, the kind used to mea­sure lung ca­pac­ity where you blow as hard and as fast as you can for a short pe­riod. Peak flow me­ters are sim­ple de­vices with no real tech to speak of, but us­ing a small AR tar­get and a spe­cific app, the smart­phone can ac­cu­rately cap­ture the peak flow read­ing via the phone’s on-board cam­era. The idea is be­ing pro­moted as a low-cost way of au­tomat­ing data col­lec­tion in de­vel­op­ing coun­tries ( Else­where, there is con­sid­er­able in­ter­est for AR/MR in emer­gency re­sponse train­ing. A re­cent re­search paper from the Univer­sity of Birm­ing­ham de­tails a low-cost con­cept that uses MR to aid train­ing emer­gency per­son­nel in air­craft pro­ce­dure. Since emer­gency re­sponse air­craft are not al­ways on hand dur­ing train­ing, the re­search con­cept brings in MR to repli­cate that air­craft ex­pe­ri­ence (

But ar­guably one of the most im­pres­sive ap­pli­ca­tions of MR is use by sur­geons dur­ing op­er­a­tions on pa­tients. At Stan­ford Univer­sity, breast ra­di­ol­ogy and surgery re­searchers are de­vel­op­ing a pro­to­type sys­tem us­ing HoloLens to over­lay pre­vi­ous MRI scans onto the pa­tient’s breast, en­abling sur­geons to more ac­cu­rately re­move a can­cer tu­mour and re­duce the need for fol­low-up surgery. How­ever, the sys­tem cur­rently can only be used pre-surgery as it can’t ac­count for move­ment once surgery com­mences. To fix the is­sue, Stan­ford mechanical en­gi­neers have been brought in to model how hu­man skin tis­sue moves, which will al­low sur­geons to use the sys­tem dur­ing surgery as well (


As mo­bile pro­ces­sor per­for­mance con­tin­ues to rocket along at a rate of knots, the fu­ture for AR looks ex­ceed­ingly bright. Ac­cord­ing to a re­cent Citi Re­search re­port, the VR/AR mar­ket is fore­cast to top US$692 bil­lion by 2025 (­veqz5). There’s even talk it may pass US$1 tril­lion by 2035.

Poké­mon GO made a truck­load of money and brought AR to main­stream con­scious­ness, but if the fore­casts are right, it looks like we’re in for one heck of a ride.

HoloLens is pow­ered by a 1GHz 32-bit In­tel pro­ces­sor. Chi­nese on­line re­tail gi­ant Alibaba launches Buy+ — an AR shop­ping ex­pe­ri­ence.

Google’s Project Tango will bring AR to a range of new phones and tablets.

The PlayS­ta­tion VR com­bines mo­tion sen­sor with video dis­play tech­nol­ogy.

The suc­cess­fully crowd­funded Zap­box de­liv­ers AR via your smart­phone for just US$30. AR can cast spells and lev­i­tate ob­jects, thanks to HoloLens and the world of Harry Pot­ter. Nike’s new AR kiosk in its Paris store al­lows shop­pers to de­sign and style...

Mi­crosoft’s HoloLens is on sale in Aus­tralia for devel­op­ers.

ASUS’s up­com­ing ZenFone AR com­bines Google Day­dream and Tango tech. Len­ovo’s new Phab 2 Pro fea­tures Google’s Tango AR tech­nol­ogy.

Newspapers in English

Newspapers from Australia

© PressReader. All rights reserved.