ARKit 2 extends Apple’s lead in mobile AR
Jason Cross looks at what’s new
Apple is serious about augmented reality. Tim Cook has been clear about how important it is, saying it will “change the way we use technology forever”. With ARKit in iOS 11, Apple brought AR capabilities to hundreds of millions of phones. With iOS 12, it brings developers ARKit 2 with enhanced capabilities like 3D object detection, persistent experiences, and shared AR spaces.
It’s a robust improvement on what is arguably the best set of AR tools for mobile devices anywhere.
What’s new in ARKit 2
When ARKit debuted with iOS 11 last September it immediately made waves, but the limitations were obvious. With iOS 11.3, the ARKit developer tools got more powerful, adding the ability to track vertical surfaces and non‑flat surfaces. Now, just one year later, Apple is updating to ARKit 2. Here’s what Apple says is new.
Improved face tracking: Apple didn’t dive into this, but the name is self-explanatory.
Realistic rendering: The rendering of augmented reality objects is already quite good. Apple didn’t say exactly how rendering has improved, but we expect more accurate lighting is key.
3D object detection: ARKit detects flat rectangle objects like posters or book covers, but ARKit 2 allows developers to detect fully 3D objects. That’s going to be huge.
Persistent experiences: You can save AR spaces and objects that are linked to physical objects (such as toys) or physical spaces (like classrooms), so you can pick up where you left off later.
Shared experiences: Multiple users can use their iOS devices to view the same virtual environment, each from their own view. Apple is releasing a block‑breaking multiplayer game as a code demo, and Lego showed off a virtual plays pace where up
to four players can interact with a combined virtual and real play space all at once.
Measure app
When ARKit launched, it was quickly followed by a flood of measurement apps. Apps to take virtual measurements of lengths, volumes, room sizes, you name it. Apple is getting in on the game with its own measurement app, aptly named Measure.
Measure looks like a simple, reliable version of the many AR measurement apps out there. You can make linear measurements, even of 3D objects to quickly
compute volume. It even recognizes rectangular objects automatically, so you just point at a poster, photo, or rectangular table, tap, and get a full set of measurements. This is one of Apple’s simpler apps, such as the Calculator, Compass, or the built-in Flashlight. And like those apps, it’s an appropriate thing to have built into your phone, but not the sort of groundbreaking experience that will make people switch to iPhone.
USDZ format
Apple worked together with Pixar to develop a new file format for augmented reality objects called USDZ. It’s going to be supported by third‑party apps, including ones by Autodesk, Adobe, Sketchfab, and more. Adobe CTO Abhay Parasnis took to the stage to say that Creative Cloud will support the format, and that their tools will even allow what-you-see-is-whatyou-get editing of VR objects with iOS devices. Does this mean that USDZ will become a widely supported format, even beyond Apple devices? Apple wasn’t clear, but it looks like the odds are good that USDZ could becomes the next HEIC.
Supported devices
According to Apple, “ARKit 2 and USDZ support will be available this fall as part of a free software update for iPhone 6s and later, all iPad Pro models, iPad fifth generation and iPad sixth generation.” That’s pretty much every device that supported the original ARKit release.