Fun with Face ID and TrueDepth

Face recog­ni­tion is stan­dard on all the lat­est iPhones — and get­ting smarter

Mac|Life - - CONTENTS - Adam Banks

When Ap­ple de­signed the iPhone X to be ‘all screen’, it cre­ated a cou­ple of co­nun­drums. There had to be some­where to put the front–fac­ing cam­era, so the notch was born. (Ap­ple is one of sev­eral com­pa­nies that have ap­plied for patents on ways for cam­eras to shoot through or be­tween pix­els, so a truly all–screen phone should be pos­si­ble one day.)

Mean­while, the Home but­ton had to be sac­ri­ficed, and with it Touch ID. Ap­ple could sim­ply have put the fin­ger­print sen­sor some­where else, but in­stead it switched to a new method of bio­met­ric au­then­ti­ca­tion us­ing a sen­sor sys­tem, built into the notch, that it calls TrueDepth. It’s this hard­ware in­no­va­tion that en­ables the iPhone X, XS, XS Max, and XR to sup­port Face ID.

Fa­cial recog­ni­tion is a sta­ple of sci­ence fic­tion and heist movies, but mak­ing it work in real life is hard. Sim­ply pick­ing out a face in the im­age cap­tured by an or­di­nary cam­era would mean any­one who waved a photo of you at your iPhone would be as likely to get into it as you are. Ear­lier phones, in­clud­ing Sam­sung’s Galaxy Note 8, could be fooled in this way.

In­stead of tak­ing a pic­ture, TrueDepth de­tects the three–di­men­sional shape of your face. The hard­ware, in­cor­po­rated into the notch, is made for Ap­ple by STMi­cro­elec­tron­ics. On the right–hand side is a ver­ti­cal–cav­ity sur­face–emit­ting laser (VCSEL). Just as an LED is a diode that emits light, a VCSEL is a diode that emits a laser beam.

The tech­nol­ogy has been around for decades, but it’s new to the mass mar­ket, and the iPhone’s de­mand has vastly ex­ceeded pre­vi­ous man­u­fac­tur­ing ca­pac­ity. In De­cem­ber 2017, Ap­ple awarded $390m from its Ad­vanced Man­u­fac­tur­ing Fund to help op­ti­cal com­po­nent maker Fin­isar build a VCSEL fac­tory in Texas, cre­at­ing 500 jobs.

3D mod­el­ling

In front of the VCSEL is an op­ti­cal fil­ter that rapidly redi­rects its laser beam us­ing tiny glass mir­rors to project a grid of over 30,000 dots in a frac­tion of a sec­ond. The dots are in­vis­i­ble to the hu­man eye, but a night–vi­sion cam­era would see them

spring into ac­tion when the iPhone’s prox­im­ity sen­sor — third from the left in the notch — tells it a face is loom­ing.

To the left again is an­other in­frared source, a flood il­lu­mi­na­tor: ba­si­cally in­frared flash. With both this and the laser shin­ing on your face, the TrueDepth in­frared (IR) cam­era, the notch’s left­most com­po­nent, can cap­ture both the dot pat­tern and a de­tailed im­age. As it’s IR, it works as well in day­light or dark­ness.

If the dots landed on a flat sur­face, they’d form a reg­u­lar grid. On your face, the di­ver­gence of their po­si­tions can be used to cal­cu­late a 3D model.

It’s this 3D model that en­ables Face ID to rec­og­nize you. The op­ti­cal im­age that’s cap­tured si­mul­ta­ne­ously adds de­tail that’s used by iOS’s At­ten­tion Aware fea­tures, which rec­og­nize that your eyes are point­ing at the de­vice and avoid auto– dim­ming the screen while you’re look­ing at it, for ex­am­ple. By de­fault, Face ID re­quires at­ten­tion, so you have to make eye con­tact when pre­sent­ing your face. This is to help avoid ac­ci­den­tal un­locks and pre­vent some­one grab­bing your phone and un­lock­ing it with your face. (The FBI, how­ever, has al­ready used a search war­rant to de­mand that a sus­pect look at his iPhone X to un­lock it.)

TrueDepth has much wider ap­pli­ca­tions than Face ID. It’s also the ba­sis of An­i­moji and Me­moji. One day, in­cor­po­rat­ing a sys­tem sim­i­lar to TrueDepth into a de­vice’s rear cam­era ar­ray could en­hance iOS’s aug­mented re­al­ity ca­pa­bil­i­ties, which cur­rently rely on the reg­u­lar cam­era im­age and ori­en­ta­tion sens­ing.

Ap­ple has al­ready added TrueDepth ca­pa­bil­i­ties to the new iPad Pro. As for Macs, re­cent Ap­ple patents de­scribe both fa­cial recog­ni­tion (with au­to­matic lo­gin) and us­ing a depth sen­sor to al­low macOS to be con­trolled by ges­tures in the air.

Five years ago, Ap­ple ac­quired PrimeSense, which was be­hind the tech in Mi­crosoft Kinect, but we have yet to see Ap­ple pur­sue this kind of in­ter­ac­tion on the desk­top.

Visu­al­iza­tions of Face ID’s in­frared flood im­age and dot mesh pat­tern.

Newspapers in English

Newspapers from Australia

© PressReader. All rights reserved.