Choose Your Own Reality
Like it or not, we all live in the same reality: all of us exist under the effects of Earth’s gravitational field, atmospheric chemistry, diurnal cycle and our own innate biology. The world rolls on, and will until the heat death of our sun in about five billion years, in full compliance with the laws of physics. None of us can change that — however, we can change the way it looks. Here are some options:
AUGMENTED REALITY (AR)
By superimposing visual information between us and the real world, AR allows us to simultaneously perceive the world around us, augmented with relevant content displayed by a computer system. The first practical, widespread use of AR took the form of Heads-Up Displays (HUDs) in military aircraft, which display the aircraft’s weapons status, attitude, altitude, heading, remaining fuel, radar target lock and other crucial flight information on a transparent screen located directly in the pilot’s line of sight.
The technology was actually pioneered during the Royal Air Force during World War II and has since become a universal fixture on all military and even some commercial aircraft.
Epson Moverios employed by Embry-Riddle researchers in this study provide a HUD-type capability for drone pilots, by projecting their aircraft’s video link and telemetry into their field of view, while simultaneously keeping the aircraft in sight.
VIRTUAL REALITY (VR)
A person employing a VR system blocks out the real world, in favor of a computer-generated simulation. VR is a burgeoning sector of the computer gaming industry, allowing players to immerse themselves in fantasy worlds and use the movement of their entire body as a game controller. This technology has also found applications in fields as diverse as architecture and urban design, healthcare, occupational health and safety, education and many others.
VR has been used to create tours of inaccessible locations, such as the International Space Station or ancient cities that have long since fallen into ruin, providing a lifelike experience for virtual visitors who could otherwise never see them.
The first VR experiences were created by artists in the 1970s, using powerful computers made available by the Jet Propulsion Laboratory and the California Institute of Technology in Pasadena. One challenge that the industry has yet to address is how to prevent VR users from looking like world-class dweebs.
Sharing elements of both AR and VR, mixed reality allows its users to perceive their actual surroundings through a transparent screen. However, the mixed-reality system uses this screen to display a virtual object anchored at a specific location in the real world. Combined with simultaneous localization and mapping (SLAM) technology, mixed reality allows multiple individuals in the same physical space to see the same virtual object, each from their own perspective.
One use case might involve a group of architects working together on a new building design. The design exists only as a virtual 3D object, perceived to be displayed on a real conference table that they have all gathered around. The participants are able to walk around the model, examining it from different sides and exchange comments and ideas with their peers.
The best-known mixed reality system currently available is the Microsoft HoloLens, first released in 2016. It borrowed its tracking technology from the Kinect module produced for the Xbox gaming system.
ADVANTAGE: AUGMENTED REALITY
When the pilots flew a comparable mission using AR technology, the results were dramatic: very nearly the reverse of the previous test. Wearing the Moverios, the pilots spent more than half of the time looking up at the aircraft. For Thirtyacre, this was an important insight—one that will require further research to confirm, but also one that hinted AR might have an important role to play in the future of UAS operations.
“In accordance with Part 107, we need to maintain VLOS with the aircraft. Does hearing the aircraft behind me while looking at the GCS constitute VLOS? I don’t think so,” he said. “I think it is very important that we understand where the aircraft is and the environment around it. If you’re not looking at your aircraft, how do you know you’re not flying over people? How do you know where or not there are power lines nearby?”
One question Thirtyacre would like to see addressed by a future study is the question of “dwell time.” That is, how long are the uninterrupted stretches pilot spend looking at the GCS display, before visually checking in with the aircraft.
“Manned pilots are constantly scanning the environment while they are flying. They periodically glance down at their instruments, but that interval is measured in seconds,” he said. “My guess is that we’ll find people stare at the display for two, three or four minutes at a time. We need to move toward an approach that more closely resembles what happens in manned aviation.”