What is Tango Augmented Reality?
G oogle’s version of Augmented Reality is a huge leap forward from Pokémon Go or any other AR app you may already be familiar with. Announced early in 2014, Tango is Google’s attempt to get mobile phones and tablets to see the way the human eye sees. This is no simple task. It involves an extensive camera array that uses computer vision, depth sensing and motion tracking sensors to grant the device full spatial awareness; in other words: the ability to understand your environment and your relation to it. Tango doesn’t require GPS or any other external signal, which means it can do indoor navigation - something that’s never been done on a mobile device before. Current VR devices like the HTC Vive require carefully calibrated external sensors to know where you are, but with Tango’s “inside out tracking”, everything you need is inside your smartphone. And if that’s not impressive enough, Tango doesn’t just know where you are, it also maps and tracks every single 3D object in the same room as you. Google’s first Tango device was a large, bulky prototype tablet made available only to developers back in 2014. Since then Google has formed a partnership with Lenovo to bring a Tango-enabled device to consumers, packing all of the sensors and processing required to run Tango into a portable, consumerfriendly form factor. That device is finally here: the 6.4-inch Lenovo Phab 2 Pro phablet.
How does it work?
The Phab 2 Pro utilizes three types of technology that all work together to make Tango work:
1. Motion Tracking
Motion Tracking lets the Phab 2 Pro track its own movement and orientation through 3D space. Walk around with the Phab 2 Pro and move it forward, backward, up, or down, or tilt it in any direction, and it can tell you where it is and which way it’s facing. This is accomplished on the Phab 2 Pro with a wide angle fisheye camera, an accelerometer, and a gyroscope. The image from the fisheye camera is used to identify key visual features such as edges and corners. The device then tracks how much these features move between frames to determine the distance traveled. The data from the accelerometer and gyroscope determine how fast the device is moving and in which direction it is turning. All this information is fused together to track where the device is in 3D space.
2. Area Learning
Area Learning means the device remembers what it sees and can also recall that information later on. With Motion Tracking alone, the device “sees” the visual features of the area it is moving through but doesn’t “remember” them. The device needs to use Area Learning to improve the accuracy of Motion Tracking by aligning the real-time data with the saved data. Area Learning is accomplished through Google’s Tango core software as it processes all the spatial information gathered by the sensors on the Phab 2 Pro.
3. Depth Perception
Depth perception gives the device the ability to measure distances to objects in the real world. This lets you augment virtual objects that not only appear to be a part of your actual environment, but can actually interact with your environment. The Phab 2 Pro accomplishes depth perception by combining the information from a “time of flight” IR (infrared) emitter and standard RGB camera. The IR emitter sends out infrared light which bounces back and is measured within a few nanoseconds, while the RGB camera creates a stereo image that is used to generate depth information.
What can Tango do?
The applications for Tango are endless, but to start with, Lenovo and Google have targeted three key areas of interest. The first is indoor navigation. Imagine having a Google Maps style layout in every mall, museum and supermarket, telling you exactly where you are, and where you want to go. Extra information could be overlaid on top of navigation elements, for example, if you’re in a supermarket, the AR display could point your attention to shelves with special deals as you walk around.
The second area is gaming. Imagine if Pokémon Go had Tango augmented reality. Instead of merely seeing a Pokémon appear on screen, it could interact with objects and your surrounding environment. Pokémon could literally hide in the tall grass near your house. For other AR games, levels could be created using the layout of your house, school, or office, making the game far more personal and familiar.
Finally, Google is looking at AR enhanced utilities. These are apps that you could use, for example, to map your living room and then re-decorate by moving around, adding or removing furniture. We’ve already seen AR apps like this, but with Tango, you can move through that space to get a real-time 3D idea of what your actual room will look like from every angle.