The new Apple Vision Pro blends virtual, physical worlds
Apple’s biggest hardware drop in years is a nearly $4,000 virtual reality headset that allows people to work, watch movies and play games that appear on screens that merge seamlessly with the real world.
In 15-minute demonstrations at the Apple store in Shadyside, college students, retirees and business professionals are seeing how Apple Vision Pro could teleport them to loved ones and faraway landscapes across the world.
“Is it going to change everything?” a reporter asked. “Totally,” one customer replied. Although inaccessible to most consumers at the current price point, the technological advancement could change how people think about screen-based content and allow researchers and businesses to develop new systems — similar to how the IOS app store and Google Play stores gave rise to app developers.
An extended reality center is already part of Carnegie Mellon University’s plans for a new technology center in Hazelwood. And Joe Zeff, a Pittsburgh digital artist, recently used his headset to manipulate and examine a 3D-rendering of the Peregrine lander, which was built by North Side-based space startup Astrobotic, and which burned up in the Earth’s atmosphere last month following a failed attempt to land on the moon.
“There’s fun and also professional opportunity,” said Mr. Zeff, who developed some of the first iPad apps when that system hit the market in 2010.
Apple’s new goggles join a variety of virtual and augmented reality tools being put to use in Pittsburgh.
Bally Design, based in Lawrenceville, has used virtual reality to demonstrate products such as engines, allowing people to don a headset and walk through the machines — which feels both eerily realistic and is totally groundbreaking as it allows a perspective that wouldn’t be possible in the real world. From Larimer, KEF Robotics is using VR to control attack drones that could be flown in Ukraine. And in the Strip District, Sandbox VR is bringing immersive gaming to the masses.
Exactly how Apple Vision Pro fits into the existing headset
ecosystem remains to be seen.
Justin Kaufman spent his first weekend with the headset playing Fruit Ninja, a video game where you slash through floating fruit with a saber. But the managing partner of PwC’s Pittsburgh office sees huge potential.
“If it’s used the right way, I think it could be hugely beneficial for all of us,” he said, adding that clients in retail, banking and health care already are exploring potential uses.
“They’re all starting to look at these technologies, and if they’re not, they’re behind,” he said.
Researchers at CMU believe headsets could soon replace smartphones, laptops and tablets for a variety of tasks. Paired with haptic gloves and other accessories, the immersive goggles could help connect doctors and patients thousands of miles apart, and better engage students in both K-12 and higher education.
“It’s not just about replacing things we can do now, just better — it’s about expanding our human capabilities,” David Lindlbauer, director of CMU’s Augmented Perception Lab, told the Post-Gazette when the school launched its extended reality center in November.
“Right now, there are many companies — Google, Meta, Apple — that are heavily invested in this, and we have a critical mass of people at CMU working in different areas, we just needed a way to bring all these people to work together,” the center’s co-director Fernando De La Torre said.
Perhaps the most notable advancement of Apple Vision Pro over other headsets is passthrough, the ability to merge digital content into the physical world.
When users put the goggles on, cameras mounted on the outside replicate the room around them on screens inside, making it feel almost like there’s nothing in the way.
“It’s like a pane of glass,” said Mr. Zeff, the graphic designer.
The latency — or delay between the camera and video feed — is so small that people can play ping pong with the headset on.
“I’ve seen videos of people driving with it,” Mr. Zeff said. “But that’s obviously totally irresponsible.”
Early adopters also are experiencing an advanced form of FaceTime, which renders faces in strange, animated 3D, rather than capturing real time video. Mr. Zeff said it’s “weird” but also certain to improve, like many features embedded in the first generation device.
The fake FaceTime is also an example of how Apple Vision Pro distorts reality. The initial setup process immortalizes people in whatever look they initially capture, meaning if an executive did their scan in a hoodie, that casual wear would follow them into a business meeting.
Last weekend, in the real world, Mr. Zeff went over the handlebars of his bike, scuffing his chin. But in FaceTime, his face was flawless.