Face masks give facial recognition software an identity crisis
It is an increasingly common modern annoyance: arriving at the front of the queue to pay in a shop, pulling out a smartphone for a hygienic contact-free payment, and staring down at an error message because your phone fails to recognise your masked face.
As more and more nations mandate masks to prevent the spread of coronavirus, technology companies are scrambling to keep up with the changing world. But some experts are warning that the change may have to start with users themselves.
Apple’s Face ID is the most wellknown example of a consumer facial verification system. The technology, which uses a grid of infrared dots to measure the physical shape of a user’s face, secures access to the company’s iPhones and iPads, as well as other features such as Apple Pay.
But although the service can work through many barriers, including heavy makeup, thick beards and even sunglasses, it fails with masks. Users can still avoid some hassle with a littleknown feature called Express Transit, which allows them to use Apple Pay on public transport without authentication. But other than that, there has only been one tweak to the system to account for the new reality. In mid-May, Apple launched an update that removed the delay between failing to recognise a face and showing a passcode for an alternative authentication.
That speeds up the fallback option, but is less than many had hoped for by now. Unfortunately, says Andrew Bud, chief executive of facial verification company iProov, that might be the best we see in the near term.
Face masks do not actually make it significantly harder to recognise a person, Bud says. “Modern face recognition relies quite heavily on the area around the eyes. The old paradigm – measuring the geometry of the shape of the face in general – that was obsolete five years ago.”
Instead, the problem masks have is that they make it harder for a system to distinguish between a real face and a spoof. That task, called “genuine presence assurance”, is at the core of facial verification – and the reason why Bud thinks we may not see alternatives arising any time soon. “Most genuine presence assurance solutions will struggle to deal with a large mask covering the majority of someone’s face.”
That problem has also plagued alternatives. In 2017, for instance, Samsung brought out the Galaxy S8, a new smartphone with iris recognition. Within a month, researchers had posted a video of them unlocking the same phone with just a photo of someone’s eye hidden behind a contact lens. “By far [the] most expensive part of the iris biometry hack was the purchase of the Galaxy S8,” the group wrote.
Other technologies might show more promise. Bud’s company, which provides verification services to the UK government, has developed a palm verification system that can successfully recognise people from a smartphone picture of their hand – with the same protection against artificial spoofs. But ultimately, he warns, the most likely outcome is a change in attitudes.
“In the future, when you want to authenticate, you just briefly take your mask off. We think that’s OK. We as a company are focused on usability, and that strikes me as being an imposition you can live with, even in the world of
Covid.”
There is one silver lining. The problems that facial verification systems have with masks have also been plaguing manufacturers of a similar technology, facial recognition – used for surveillance purposes, to track people using CCTV footage. A July study by the US government revealed that widespread mask wearing reduced the accuracy of facial recognition algorithms by between 5% and 50%.