Augmented reality
Cooler than the virtual kind?
TIM COOK has been talking up the potential of augmented reality (AR) for months, so it wasn’t entirely surprising that Apple had something to say about it during its developer conference. But we weren’t expecting to be so thrilled by what was shown.
The idea is that when you hold up your iPhone or iPad, information from the device’s camera is analyzed by an app, which is able to detect surfaces in your surroundings and then composite objects onto the live view on your device’s screen.
Why would you want to do this? One embryonic AR example from the early days of the App Store used your location and the coordinates of nearby train stations to render virtual signage that would show you the general direction and distance to those stations. As long as you weren’t in a rush, it was a fun way to explore a new neighborhood while making your way somewhere, occasionally checking in with the app.
AR for everyone
Apple is banking on AR becoming a big thing, so it has given third-party developers a new framework, ARKit, to enable them to more easily build apps and games around the concept.
The software uses a device’s camera and motion sensors to provide fast, stable motion tracking of the world around you. It’s able to detect planes – surfaces such as tables, walls, and floors – and it’s designed to work in your real-world surroundings.
ARKit can estimate ambient light, which Apple says helps with the rendering side of things, and also with estimating the scale at which virtual objects should be rendered into a scene. ARKit works with Apple’s own SceneKit rendering engine, and Apple also announced support for Unity and Unreal Engine, popular third-party alternatives often used in game development.
The company boasted that the results of this effort will work on many iOS devices already in
use, and that this means ARKit is instantly the largest AR platform in the world.
Scandinavian furniture retailer Ikea has used AR for a while now, so it’s little surprise that it threw its weight behind Apple’s announcement soon after WWDC, with plans to use it to show how its products will look in your home.
On the lighter side, ARKit enables games like Pokémon Go to superimpose characters upon the real world, making the popular outdoor collectathon a lot more like what we had in mind when it was first announced – and then some, because swiping a Poké Ball to capture a critter makes use of ARKit’s ability to detect surfaces, and causes the ball to bounce against the ground for added realism.
Perhaps the most impressive ARKit demo shown off at WWDC was from Wingnut AR, filmmaker Peter Jackson’s studio dedicated to creating augmented reality content. Wingnut’s Creative Director, Alasdair Coull, showed an environment rendered in Unreal Engine 4 and “projected” on an iOS device’s screen so that the action took place on a large, flat tabletop (see the upper-left picture). Coull showed how you can physically walk around and move in and out of a scene to see things from any position you choose. He promised an AR experience from Wingnut will be released on the App Store later this year.
AR’s impact on battery life is unclear; rendering complex 3D scenes can be draining, on top of which ARKit keeps the camera active. But the demos we’ve seen so far have our minds racing over how AR could change scripted storytelling, never mind interactive entertainment – and let’s not forget its more serious applications.