Augmented reality: A TechLife primer
IT COULD REVOLUTIONISE EVERYTHING FROM MEDICINE TO ONLINE SHOPPING AND JUST ONE IMPLEMENTATION OF THIS TECH GENERATED OVER US$950MILLION IN 2016. WE EXPLAIN AUGMENTED REALITY AND WHAT IT MEANS FOR OUR FUTURE.
IF YOU WERE looking for a candidate for ‘Mobile Game of 2016’, it’d be difficult to go passed Pokémon GO. The game itself managed to generate a staggering US$950 million over the second half of the year and instantly popularised this seemingly new technology colloquially referred to as ‘AR’. But what is ‘augmented reality’, how does it differ from virtual reality and, importantly, where are we likely to see it next?
WHAT IS ‘AR’?
Last month, we found that what constitutes ‘virtual reality’ is up for debate, and there’s probably even more questions surrounding how to best define ‘augmented reality’. For many of us, our first contact with what usually passes for AR was a hologram of a feisty space princess in Star Wars: A New Hope.
But the tech industry wants to broaden horizons and avoid pigeon-holing AR as just fancy holographic displays. Another definition gaining traction is that AR is any ‘real world’ view overlayed with digital data, whether vision, sound or graphics. Yep, that’s pretty broad, but it does ensure AR doesn’t have to necessarily involve strapping on a headset. In fact, you’ve probably already enjoyed plenty of AR in the comfort of your own living room, such as the ‘Hawk-Eye’ ball-tracking overlays used by the Nine TV Network during cricket coverage and the Seven Network during the Australian Open.
When it comes to deciphering the difference between augmented and virtual reality, it’s much easier — augmented reality still deals in the real world as its base, whereas even reality itself is digitally loaded in VR. Despite plenty of recent VR examples, Pokémon GO clearly holds reign as the pop-culture icon for AR over the last 12 months.
But while Pokémon GO may have introduced the mainstream to AR, in reality, AR is hardly new — in fact, you could argue its origins date back to World War II. The scourge of night bombing over London during the early period of the war drove the Royal Air Force (RAF)
to develop technology to deal with the menace. The engineers and scientists at the Telecommunications Research Establishment invented Air Intercept radar to help fighter pilots locate enemy aircraft at night. According to the book British Aircraft Armament Vol. 2: RAF Guns and Gunsights from 1914 to the Present Day by R Wallace Clarke, one of the later modifications developed during 1942 involved using a cathode-ray tube (CRT) to project an image of the radar display, a gunsight ring and an artificial ‘horizon’ onto the aircraft windscreen — the world’s first head-up display. Despite some success during trials, test pilots reportedly offered plenty of suggested improvements, and with development efforts more focussed on radar itself, the Pilot’s Indicator Display never made it into active service during the war. A practical system for aircraft detection using radio waves had only been proven to work in 1935, so for scientists and engineers to have a working predecessor of today’s HUD by 1942 using technology many would now consider prehistoric was impressive.
Since those early days, the computing horsepower required to power most AR applications has kept the concept out of the loungeroom. However, thanks to the rapid performance growth in mobile processors and the overall feature capability of smartphones, AR is now breaking into the mainstream — and not just with Pokémon GO.
HARRY POTTER AND THE AUGMENTERS
If you’re not a Harry Potter fan, skip this next bit, but if you’ve ever dreamt of incanting “Wingardium leviosa!” and watching an object levitate at the end of your wand, the opportunity might be closer than a year at Hogwarts. Digital artist Asad Malik (aka ‘1RIC’) saw Pokémon GO as inspiration to try a little magic of his own and developed an MR world where spoken spells can be cast at the point of a wand ( tinyurl.com/hvp9cyo). His method is built around Microsoft’s HoloLens AR holographic computer to interpret hand gestures and wand movements as triggers for voice-controlled commands. At time of writing, his system had three working spells, including ‘Expecto Patronum’ for warding off Dementors and ‘Incendio’ for setting fire to objects.
We’re not sure how this sits in terms of copyright, but we’d be surprised if Harry’s corporate minders weren’t ka-chinging their cash registers at the thought of future merchandising possibilities. If nothing else, it’s a clever pop-culture example of MR.
HOLOGRAPHIC COMPUTERS ARRIVE
HoloLens has been available in the US for most of the last year, but only began shipping in Australia last November. At the moment, it’s only really designed for software developers (hence the ‘Development Edition’ tag) — and those with deep pockets. With a price tag starting at $4,369, it might still be a while yet before the rest of us get our hands on one.
But if that sounds a bit too rich for your back pocket, how does $30 grab you? UK group Zappar recently completed a Kickstarter campaign to bring to life Zapbox, an affordable alternative to HoloLens, but with greater control and camera functionality ( tinyurl.com/jclwksj).
Instead of a built-in computer, Zapbox relies on your smartphone to do the heavy-lifting of mixing realities, making it a kind of AR version of Google Cardboard and its upmarket cousin Daydream View (see issue 59, page 14). However, while Daydream View requires (and currently only works with) Google’s own Pixel and Pixel XL smartphones and is a VR-only headset platform, Zapbox offers a full AR experience and is said to work with any Android device with a 4.5-inch or larger display panel and at least Android 4.1. Basically, if you can run the Google Cardboard demos adequately, you should be Zapbox-ready.
Meanwhile, Google has reportedly stated that most of the major phone makers will produce new phone models that are Daydreamready, including Samsung, LG, Alcatel, Huawei, Xiaomi and ZTE.
DANCING THE TANGO
GPS is the common method for determining a smartphone’s current global location and it’s the method Pokémon GO uses to place the user in close proximity with a ‘pocket monster’ to collect. But it’s also not particularly efficient — the rate of data transmission, for example, is as low as 50 bits per second (50bps). Compare that with even old dial-up modems that downloaded data a thousand times faster and it takes a lot more time to get the information you need. Throw in the fact that your phone has to communicate with satellites in space to get that data and it shouldn’t surprise that GPS is pretty thirsty when it comes to draining your device’s battery life.
A couple of years ago, search giant Google set up Project Tango ( tinyurl.com/jk3hgss), an AR/MR platform that, among other things, doesn’t need GPS to precisely identify location. By the end of 2016, Lenovo had produced the first Tango-ready device, the Phab 2 Pro. Meanwhile, Taiwanese maker ASUS is about to go a step further, combining Tango with Daydream in the up-coming ZenFone AR ( tinyurl.com/jabhbf4), a 5.7-inch smartphone first shown at CES 2017 in Las Vegas and expected for release later this year.
SHOPPING WITH AR
Many of us won’t dare to walk out of the house without a smartphone, even just to go to the corner shop for milk. But it’s no secret that the bricks-and-mortar retail industry is facing up to the onslaught of online shopping, particularly with Amazon’s imminent arrival into the Australian market. Sports giant Nike is hoping to reinvigorate the physical shopping experience the next time you walk into its Paris store. Here, shoppers are able to custom-detail their own shoe colours and styles through holographic MR tech packed into a touchscreen kiosk ( tinyurl.com/hcxmlpm).
However, it’s not just bricks-and-mortar receiving an MR makeover. Towards the end of last year, Chinese online retailing giant Alibaba unveiled Buy+, an MR/VR headset app that promises to revolutionise the online shopping experience. Rather than just a bunch of webpage lists, buyers get a 360-degree view and are able to walk through a virtual department store, where they can pick up products, spin them around and get a better idea of size, colour and form ( tinyurl.com/
zh8ovxl). You can get a glimpse of the idea through Alibaba’s YouTube video ( youtu.be/-HcKRBKlilg).
AR IN SCHOOLS
For some of us, primary school only got as interesting as playing around with Cuisenaire Rods. But today, there’s a growing body of research applying AR to numerous areas of teaching from general teaching of Year 2 students to more specific applications such as teaching mathematics and geometry ( tinyurl.com/zeeodr9).
More practically, there are dozens of AR apps designed to help teach students direct from their smartphone or tablet. For example, Elements 4D is a cool app using AR to aid in teaching the principles of chemistry and how elements combine. It uses wooden or paper blocks as targets to launch AR video. You’ll find the app and details for lesson plans via Google Play ( tinyurl.com/kgpmbdf).
The success of Pokémon GO will no doubt spawn an increased uptake of AR in the classroom, although that uptake will likely be dependent on costs — one reason why the rapid fall in AR tech prices will be watched with interest in education circles.
MEDICAL AR RESEARCH
Not surprisingly, AR has some seriously useful applications within medicine — and not necessarily with excessive price tags. Researchers at the Massachusetts Institute of Technology (MIT), for example, have recently developed a system that uses AR to automate digital measurement of medical instruments via smartphones without the need for plug-in or wireless devices. Basically, any electronic medical device that plugs into something — or someone — to measure any kind of vital sign rightly requires regulatory approval.
But MIT researchers found that using a simple AR marker or ‘target’ can provide a reference point for capturing accurate data from measurement devices at extremely low cost using a smartphone camera. The example the authors use is a peak flow meter, the kind used to measure lung capacity where you blow as hard and as fast as you can for a short period. Peak flow meters are simple devices with no real tech to speak of, but using a small AR target and a specific app, the smartphone can accurately capture the peak flow reading via the phone’s on-board camera. The idea is being promoted as a low-cost way of automating data collection in developing countries ( tinyurl.com/zkg65rr). Elsewhere, there is considerable interest for AR/MR in emergency response training. A recent research paper from the University of Birmingham details a low-cost concept that uses MR to aid training emergency personnel in aircraft procedure. Since emergency response aircraft are not always on hand during training, the research concept brings in MR to replicate that aircraft experience ( tinyurl.com/hue4y6f).
But arguably one of the most impressive applications of MR is use by surgeons during operations on patients. At Stanford University, breast radiology and surgery researchers are developing a prototype system using HoloLens to overlay previous MRI scans onto the patient’s breast, enabling surgeons to more accurately remove a cancer tumour and reduce the need for follow-up surgery. However, the system currently can only be used pre-surgery as it can’t account for movement once surgery commences. To fix the issue, Stanford mechanical engineers have been brought in to model how human skin tissue moves, which will allow surgeons to use the system during surgery as well ( tinyurl.com/hbgnpj8).
As mobile processor performance continues to rocket along at a rate of knots, the future for AR looks exceedingly bright. According to a recent Citi Research report, the VR/AR market is forecast to top US$692 billion by 2025 ( tinyurl.com/zbveqz5). There’s even talk it may pass US$1 trillion by 2035.
Pokémon GO made a truckload of money and brought AR to mainstream consciousness, but if the forecasts are right, it looks like we’re in for one heck of a ride.
Microsoft’s HoloLens is on sale in Australia for developers.
The PlayStation VR combines motion sensor with video display technology.
The successfully crowdfunded Zapbox delivers AR via your smartphone for just US$30. AR can cast spells and levitate objects, thanks to HoloLens and the world of Harry Potter. Nike’s new AR kiosk in its Paris store allows shoppers to design and style their own shoes.
HoloLens is powered by a 1GHz 32-bit Intel processor. Chinese online retail giant Alibaba launches Buy+ — an AR shopping experience.
Google’s Project Tango will bring AR to a range of new phones and tablets.
ASUS’s upcoming ZenFone AR combines Google Daydream and Tango tech. Lenovo’s new Phab 2 Pro features Google’s Tango AR technology.