HOW AUGMENTED REALITY WILL CHANGE EVERYTHING
IT COULD REVOLUTIONISE EVERYTHING FROM MEDICINE TO ONLINE SHOPPING AND JUST ONE IMPLEMENTATION OF THIS TECH GENERATED OVER US$950 MILLION IN 2016. DARREN YATES EXPLAINS AUGMENTED REALITY AND WHAT IT MEANS FOR OUR FUTURE.
WHAT IS IT, HOW DOES IT DIFFER FROM VIRTUAL REALITY AND, MOST IMPORTANTLY, WHERE ARE WE LIKELY TO SEE IT NEXT?
If you were looking for a candidate for ‘Mobile Game of 2016’, it’d be difficult to go past Pokémon GO. The game itself managed to generate a staggering US$950 million over the second half of the year and instantly popularised this seemingly new technology colloquially referred to as ‘AR’. But what is ‘augmented reality’, how does it differ from virtual reality and, importantly, where are we likely to see it next?
WHAT IS ‘AR’?
What constitutes ‘virtual reality’ is up for debate, and there’s probably even more questions surrounding how to best define ‘augmented reality’. For many of us, our first contact with what usually passes for AR was a hologram of a feisty space princess summoning an ageing warrior in Star Wars: Episode IV — A New Hope. However, the tech industry wants to broaden horizons and avoid pigeon- holing AR into just fancy holographic displays. A broader definition gaining traction is that AR is any real-world view overlayed with digital data, whether vision, sound or graphics. Yep, that’s pretty broad, but it does ensure AR doesn’t have to necessarily involve strapping on a headset. In fact, you’ve probably already enjoyed plenty of AR in the comfort of your own living room without realising it — the ‘Hawk-Eye’ ball-tracking overlays used by the Nine TV Network during cricket coverage and the Seven Network during the Australian Open tennis are perfect examples of the growing use of AR in television ( tinyurl.com/zbc7t64).
When it comes to deciphering the difference between augmented and virtual reality, it’s much easier — augmented reality still deals in the real world as its base, whereas even reality itself is digitally loaded in VR. Despite plenty of recent VR examples,
Pokémon GO clearly holds reign as the pop-culture icon for AR over the last 12 months.
STRATOSPHERIC BEGINNINGS
But while Pokémon GO may have introduced the mainstream to AR, in reality, AR is hardly new — in fact, you could argue its origins date back to World War II. The scourge of night bombing over London during the early period of the war drove the Royal Air Force (RAF) to develop technology to deal with the menace. The engineers and scientists at the Telecommunications Research Establishment (TRE) invented Air Intercept (AI) radar to help fighter pilots locate enemy aircraft at night. According to the book British Aircraft Armament Vol. 2: RAF Guns and Gunsights from 1914 to the Present Day by R. Wallace Clarke, one of the later modifications developed during 1942 involved using a cathode-ray tube (CRT) to project an image of the radar display,
a gunsight ring and an artificial ‘horizon’ onto the aircraft windscreen — the world’s first head-up display (HUD). However, despite some success during trials, test pilots reportedly offered plenty of suggested improvements and with development efforts more focussed on radar itself, the ‘Pilot’s Indicator Display’ (PID) never made it into active service during the war. A practical system for aircraft detection using radio waves had only been proven to work in 1935, so for scientists and engineers to have a working predecessor of today’s HUD by 1942 using technology many would now consider ‘prehistoric’ was seriously impressive.
Since those early days, the computing horsepower required to power most AR applications has kept the concept out of the loungeroom. However, thanks to the rapid performance growth in mobile processors and the overall feature capability of smartphones, AR is now breaking into the mainstream — and not just with Pokémon GO.
MIXED REALITY
But just to confuse things a little more, there is a third reality form that has the potential to exceed both AR and VR and it’s called ‘mixed reality’ (MR) — this is where real and virtual realities not only collide, but interact with each other, the virtual world affecting the real world. In practice, the terms ‘AR’ and ‘MR’ are being used interchangeably and while there’s still debate over the benefit of having the two terms, the interaction between the real and virtual worlds makes MR the most complex reality form to produce. That said, we’re already seeing some impressive examples reaching fruition.
HARRY POTTER AND THE AUGMENTERS
If you’re not a Harry Potter fan, skip this next bit, but if you’ve ever dreamt of incanting ‘Wingardium leviosa!’ and watching an object levitate at the end of your wand, the opportunity might be closer than a year at Hogwarts. Digital artist Asad Malik (also going by the
“UK group Zappar recently completed a Kickstarter campaign to bring to life Zapbox, a kind-of ‘Google Cardboard’ alternative to HoloLens, but with greater control and camera functionality.”
handle ‘1RIC’) saw Pokémon GO as inspiration to try a little magic of his own and developed an MR world where spoken spells can be cast at the point of a wand ( tinyurl.com/hvp9cyo). His method is built around Microsoft’s HoloLens AR holographic computer to interpret hand gestures and wand movements as triggers for voice-controlled commands. At time of writing, his system had three working spells, including ‘Expecto patronum!’ for warding off Dementors and ‘Incendio!’ for setting fire to objects.
We’re not sure how this sits in terms of copyright, but we’d be surprised if Harry’s corporate minders weren’t ka-chinging their cash registers at the thought of future merchandising possibilities. If nothing else, it’s a clever pop-culture example of MR.
HOLOGRAPHIC COMPUTERS ARRIVE
HoloLens has been available in the US for most of the last year, but only began shipping in Australia last November. At the moment, it’s only really designed for software developers (hence the ‘Development Edition’ tag) — and those with deep pockets. With a price tag starting at $4,369, it might still be a while yet before the rest of us get our hands on one ( tinyurl.com/zaye4sc).
But if that sounds a bit too rich for your back pocket, how does US$30 grab you? UK group Zappar completed a Kickstarter campaign back in December to bring to life Zapbox, a kind-of ‘Google Cardboard’ alternative to HoloLens, but with greater control and camera functionality ( tinyurl.com/ jclwksj). Instead of a built-in computer, Zapbox relies on your smartphone to do the heavy-lifting of mixing realities, making it a kind-of AR version of Google Cardboard and its upmarket cousin Daydream View. However, while the Daydream only works with a small set of Android phones (including Google’s own Pixel and Pixel XL) and is a VR-only headset platform, Zapbox offers a full AR experience and is said to work with any Android device with a 4.5-inch or larger display panel and at least Android 4.1 operating system. Basically, if you can run the Google Cardboard demos adequately, you should be Zapbox-ready.
Meanwhile, Google has reportedly been working to get more Android phone-manufacturers to make their handsets Daydream-ready, including Samsung, LG, Alcatel, Huawei, Xiaomi and ZTE ( tinyurl.com/hp234co).
DANCING THE TANGO
GPS (global position system) is the common method for determining a smartphone’s current global location and it’s the method Pokémon GO uses to place the user in close proximity with a ‘pocket monster’ to collect. But it’s also not particularly efficient — the rate of data transmission, for example, is as low as 50 bits per second (50bps). Compare that with even old dial-up modems that downloaded data a thousand times faster and it takes a lot more time to get the information you need. Throw in the fact that your phone has to communicate with satellites in space to get that data and it shouldn’t surprise that GPS is pretty thirsty when it comes to draining your device’s battery life.
“The success of Pokémon GO will no doubt spawn an increased uptake of AR in the classroom, although that uptake will likely be dependent on costs.”
A couple of years ago, search giant Google set up Project Tango ( tinyurl.com/
jk3hgss), an AR/MR platform that, among other things, doesn’t need GPS to precisely identify location. By the end of 2016, Lenovo had produced the first Tango-ready device, the Phab 2 Pro. Meanwhile, Taiwanese maker ASUS is about to go a step further, combining Tango with Daydream in the up-coming ZenFone AR ( tinyurl.com/jabhbf4), a 5.7-inch smartphone first shown at the 2017 Consumer Electronics Show (CES) in Las Vegas and expected for release later around July.
SHOPPING WITH AR
Many of us won’t dare to walk out of the house without a smartphone, even just to go to the corner shop for milk. But it’s no secret that the bricks-and-mortar retail industry is facing up to the onslaught of online shopping, particularly with Amazon’s imminent arrival into the Australian market. Sports giant Nike is hoping to reinvigorate the physical shopping experience the next time you walk into its Paris store. Here, shoppers are able to custom-detail their own shoe colours and styles through holographic MR tech packed into a touchscreen kiosk ( tinyurl. com/hcxmlpm).
However, it’s not just bricks-andmortar receiving an MR makeover. Towards the end of last year, Chinese online retailing giant Alibaba unveiled ‘Buy+’, an MR/VR headset app that promises to revolutionise the online shopping experience. Rather than just a bunch of webpage lists, buyers get a 360° view and are able to walk through a virtual department store, including fashion and clothing, where they can pick up products, spin them around and get a better idea of size, colour and form ( tinyurl.com/zh8ovxl). You can get a glimpse of the idea through Alibaba’s YouTube video ( youtu.be/-HcKRBKlilg).
AR IN SCHOOLS
There’s a growing body of research applying AR to areas of teaching from general teaching of Year 2 students to more specific applications such as maths and geometry ( tinyurl.com/ zeeodr9). More practically, there are dozens of AR apps designed to help teach students from their smartphone or tablet. Elements 4D ( tinyurl.com/ kgpmbdf) is a cool app using AR to aid in teaching the principles of chemistry and how elements combine. It uses wooden or paper blocks as targets to launch AR video.
The success of Pokémon GO will no doubt spawn an increased uptake of AR in the classroom, although that uptake will likely be dependent on costs — one reason why the rapid fall in AR tech prices will be watched with interest in education circles.
MEDICAL AR RESEARCH
Not surprisingly, AR has some seriously useful applications within medicine. Researchers at the Massachusetts Institute of Technology (MIT) have recently developed a system that uses AR to automate digital measurement of medical instruments via smartphones without the need for plug-in or wireless devices. Any electronic medical device that plugs into something — or someone — to measure vital signs rightly requires regulatory approval, but MIT researchers found that using a simple AR marker or ‘target’ can provide a reference point for capturing accurate data from measurement devices at low costs using a smartphone camera. An example is a peak flow meter (that can measure lung capacity where you blow as hard and as fast as you can for a short period). These are simple devices with no real tech to speak of, but using a small AR target and an app, the phone can accurately capture the peak flow reading via the phone’s on-board camera. The idea is being promoted as a low-cost way of automating data collection in developing countries ( tinyurl.com/zkg65rr).
Elsewhere, there is considerable interest for AR/MR in emergency response training. A recent research paper from the University of Birmingham details a low-cost concept that uses MR to aid training emergency personnel in aircraft procedures. Since emergency response aircraft are not always on hand during training, the concept brings in MR to replicate that experience ( tinyurl.com/hue4y6f).
But one of the most impressive MR applications is by surgeons. Breast radiology and surgery researchers at Stanford University are developing a prototype system using HoloLens to overlay previous MRI scans onto the patient’s breast, enabling surgeons to more accurately remove a cancer tumour and reduce the need for follow-up surgery. However, the system currently can only be used pre-surgery as it can’t account for movement once surgery commences. To fix the issue, Stanford mechanical engineers have been brought in to model how human skin tissue moves, which will allow surgeons to use the system during surgery as well ( tinyurl.com/hbgnpj8).
THE FUTURE
As mobile processor performance continues to improve, the future for AR looks exceedingly bright. According to a recent Citi Research report, the VR/AR market is forecast to top US$692 billion by 2025 ( tinyurl.com/zbveqz5). There’s even talk it’ll pass US$1 trillion by 2035.
Pokémon GO brought AR to the mainstream, but if the forecasts are right, it looks like we’re in for one heck of a ride to come.