VR: A TechLife primer
IT TAKES THE VIEWER INTO ANOTHER WORLD, BUT VIRTUAL REALITY IS CREATING A REALITY THAT GOES WELL BEYOND GAMING. TECHLIFE INVESTIGATES.
IT’S BEEN DESCRIBED as everything from “the ultimate story-telling tool” to “gaming nirvana”, but whatever you call it, virtual reality (VR) is all about bending your mind’s view on the world you’re looking at. And while it’s just come off a breakout year in 2016, VR looks set to start doing even bigger business, with market research predicting that as many as 52 million VR headsets will be sold by the end of this decade ( tinyurl.com/jsa3pzs). Already, many of the major tech brands from Facebook to Google, Microsoft to Sony and HTC are investing heavily in its development, covering gaming PCs, consoles and smartphones. But what is VR, how does it work and what applications does it have beyond gaming?
HOW VR WORKS
By smartphone and console standards, the tech in most VR headsets is comparatively simple. At a basic level, a VR headset is not much more than a display device — it takes in a video input (normally HDMI) and converts it into a stereo output to feed one or two small OLED display panels, depending on the headset design (the Oculus Rift CV1 has two separate OLED panels, one for each eye, while Sony’s PlayStation VR has just the one). Either way, you view the panel/s through two special lenses that allow your eyes to focus on something just two finger-widths from your eyeballs.
The video-to-OLED conversion is usually performed by a dedicated chip known as an ‘interface bridge’, which takes in the HDMI signal and outputs a ‘Display Serial Interface’ (DSI) data stream that ultimately appears on the OLED panels. In the case of the PlayStation VR and Oculus Rift CV1, that chip is Toshiba’s TC358870XBG, the first designed to work with high-resolution ‘4K’ video.
VR PROCESSING
Unlike a computer processor, this Toshiba chip is a single-function device, meaning it has one job to do, but it needs to be controlled by a special type of computer processor called a ‘microcontroller’. Your smartphone has a main general-purpose processor or ‘CPU’ to run your apps, but it also uses microcontrollers to handle specific tasks such as telecommunication connections and Wi-Fi. Microcontrollers are used in many applications where smaller amounts of computing power are required, rather than a full-blown computer. Your fitness band will have one, for example.
Unlike computer CPUs, microcontrollers come with their own stock of memory and storage to store data and code. Yet like their CPU cousins, microcontrollers come in various performance grades. What might surprise you is that most VR headsets only need the very entry-level of microcontrollers to work, mostly because all of the hard work is done by specific chips like that Toshiba interface bridge. The Oculus Rift has an ST Microelectronics STM32F070 microcontroller, the PSVR uses a Nuvoton NUC123SD4 — both are from the ARM Cortex M0 series, which is the ‘baby brother’ design to the Cortex A-series powering your smartphone and just about every other phone, right up to the latest Galaxy S7.
Check out the excellent tear-downs at iFixit of the Rift CV1 ( tinyurl.com/jrofglj) and PSVR ( tinyurl.com/h4tyqbf) for all the geeky details.
MOTION SENSORS
But the real trick of VR is its ability to track your head movement and update the display accordingly as if you’re actually moving through a real world. That’s done with the help of what’s called an ‘inertia measurement unit’ (IMU) sensor. Every smartphone has one — it’s the device that tells your phone when you’re holding it in vertical (portrait) or horizontal (landscape) mode. However, IMU sensors can do much more than that — for example, they’re also used in quadcopters/drones to stabilise flight and check general axis-of-direction. The CV1 and PSVR both use a high-resolution BMI055 six-axis IMU sensor from German
industrial giant Bosch. These sensors are often described as being ‘6DOF’ or ‘9DOF’ (degrees of freedom), but you’ll also see them listed as ‘six-axis’ or ‘nine-axis’ devices. In 3D space, you have three standard axes — X and Z at right-angles horizontally and the vertical Y-axis. When a device has a six- or nine-axes rating, it means it has multiple sensors, able to measure different physical aspects across the three axes. For example, the BMI055 has a tri-axis accelerometer, for detecting forward and backward (called ‘linear’) movement and gravity, plus a tri-axis gyroscope, which measures the rate of rotation of an object about an axis. As you move your head, the accelerometer and gyroscope measure and record that movement down to minute detail. That data is sent back to your PC or console via the microcontroller.
Getting that important data back to your PC or console is the role of the secondary (usually USB) connection. The return data provides feedback to the software or game, which then adjusts the display to match your movement, tricking your brain into thinking the vision you see moving around you is genuine.
SMARTPHONE VR
But one question you might have is if the PSVR and other similar headsets all need specialised chips, displays and microcontrollers to create VR, how does Google’s low-cost Cardboard VR system work if all it has is just a cardboard frame and a couple of lenses? It all relies on your smartphone, which is really a powerhouse computer that contains all the hardware needed to make basic VR work — it has the display, microcontrollers and IMU sensors (these sensors aren’t typically as precise as the ones inside VR headsets, but they’ll do). It doesn’t need HDMI or USB connections since the smartphone is fully self-contained. After that, it’s just a matter of having the lenses and the right software to take advantage of the combo.
MEDICAL RESEARCH
So far, the most common and recognisable application for VR is gaming, which has been the focus for most commercial development. However, there is continuing research into the use of VR in other applications, particularly medical therapies, covering everything from mental health and psychology to neuroscience and motor skills rehabilitation. The University of Southern California’s Institute for Creative Technologies, for example, has set up the MedVR Lab ( medvr.ict.usc.edu) to further research how VR can be utilised. Its primary goal is mental and behavioural health, developing techniques for assessment and treatment of stress disorders such as anxiety and PTSD. At the other extreme, VR is even bringing medical students (and others) right into the action — the Royal London Hospital streamed the first live surgery via VR online in April 2016 ( tinyurl.com/z64e9uc).
TREATING DEMENTIA
As the population ages, one of the most debilitating disorders affecting greater numbers of older people is dementia, which increasingly robs sufferers of their lucidity and memories. Early tests conducted by Alzheimer’s Australia found that a new VR game developed by the organisation led to a significant reduction in the amount of medication required by some dementia patients ( tinyurl.com/hfj4r7e). Called ‘the Virtual Forest’ ( tinyurl.com/
zhfoszf), the system uses Microsoft’s Kinect technology to enable users to move objects about in a virtual forest by simple hand movements. It requires a quad-core Windows PC with an Nvidia GTX 970 graphics card, 8GB of RAM and Kinect adapter for Windows.
It follows on from Education Dementia Immersive Experience (EDIE), an initiative launched by Alzheimer’s Australia in September 2016, to give everyone a taste through a smartphone VR app and Google’s Cardboard VR headset of what it’s like for people living with dementia ( vic.fightdementia.org.au/edie).
TREATING STROKE
VR has also long helped pioneer new treatments in stroke rehabilitation. In 2000, researchers at Rutgers University, New Jersey, reported in the IEEE Transactions on Neural Systems and Rehabilitation Engineering of using force-feedback gloves to allow stroke patients who lost hand function to move through a virtual environment and provide rehabilitation ( tinyurl.com/hmv557g). In Australia, a team from the Murdoch University School of Engineering and IT, in conjunction with the Western Australian Neuroscience Research Institute, created a VR app called Neuromender. Inside, patients fly a virtual wing-man suit through a virtual cloudscape, while a computer monitors the patient’s upper body strength through a series of automatically-adjusted challenges ( tinyurl.com/hq9vno5).
MARKETING/ADVERTISING
Just about every new technology attracting eyeballs also attracts advertisers, and VR is no exception. Right now, you almost have your choice of market researchers offering reports on how VR will change the face of marketing. We’ve read others say VR will so change the world, no-one will any longer do business in the ‘real world’. Puh-lease!
MAKE YOUR OWN VR HEADSET
While the more expensive VR headset options promise a more immersive experience, you can still get a feel for what VR is for as little as $20, thanks to Google Cardboard and Cardboard clones ( tinyurl.com/zvlzpw2). They turn your smartphone into a VR device, using its built-in screen and IMU sensors. The trick is in the lenses that make it possible for your eyes to focus on your phone screen at such close distance. Don’t forget the Cardboard demo app from Google Play ( tinyurl.com/oyedb3g) — it should work on any Android device with at least Android 4.1/Jelly Bean. There are alternatives, but it’s a good start.
MAKE YOUR OWN CONTENT
Making your own content is a little more difficult, but if you love your creative side, it’s infinitely more fun and there are a number of different ways you can go. Depending on your skill and budget, the most basic is to create ‘360 x 180-degree’ still images. With a fish-eye lens attachment, you can capture these images using a smartphone and stitch them together using apps such as PTGui ( ptgui.com), Hugin ( hugin.sourceforge.net) and GoPro’s Kolor ( kolor.com).
At the other end of the scale is the new wave of 360° cameras specifically designed for capturing VR video. While the more professional rigs include multiple cameras pointing every which way, new budget singlelens cameras, including the popular Ricoh Theta S, are a good place to start. The upcoming dual-lens Kodak PixPro 4KVR360 also looks promising.
And if you want to share your content, Facebook and YouTube already offer 360° video uploads and playback, whether it’s through a standard web browser on your PC and headset or, in the case of Google, through your smartphone and Google Cardboard.
VR OR 360°?
However, the real debate amongst purists is what to call this content — is it VR content or is it just 360° video? The argument revolves around the fact that making your own content with 360° cameras doesn’t automatically gain you entry into the VR club. In most cases, 360° video is just that — video. You enjoy the ride, but you can’t control the action as you would in genuine VR. It might seem an argument better left to the purists, but like the mess ‘HD’ TVs dissolved into, it may affect your purchases one day, so it’s important to be across the ideas now.
THE FUTURE
So now that VR has arrived in gaming consoles and smartphones, where does the technology go from here? The first major steps on the display side are continued improvement in screen resolution, as well as getting rid of the wires and making the whole system completely wireless. Right now, teams from Rivvr ( tinyurl.
com/hj9dh9j) and others are working on different alternatives to cutting the cables — Rivvr’s solution will work with Oculus Rift and HTC’s Vive, the upcoming Sulon Q ( sulon.
com) is a standalone wireless headset. There are also efforts to improve the ‘touch’ side of things, with more responsive hand-controllers that let you navigate your way through a virtual reality with greater precision and dexterity.
On the content-generation side of things, 2017 should be a breakout year for low-cost compact consumer-grade 360° video cameras that make capturing spherical video a snap. While we saw a few in 2016, the flood gates should open this year.
But while there’s been plenty of progress in VR becoming mainstream over the last 12 months or so, it’s the growth in VR research, particularly in medical therapies covering everything from mental health to dentistry and beyond, that could potentially lead to a far more enriching future for VR in the real world.
For help with dementia, call the National Dementia Helpline on 1800 100 500.