Masks thwarting facial recognition technology
Companies are creating software to make identifications using features still visible despite face coverings
Masks are confusing many commercial facialrecognition systems, a new study finds, leading to error rates as high as 50 percent.
A preliminary study published last week by the National Institute of Standards and Technology found that facial-recognition algorithms could be tripped up by such variables as mask color and shape. But industry players already are working on software that adjusts for masks — a requirement in many public spaces to contain the spread of COVID-19 — which the NIST plans to study this summer.
“With respect to accuracy with face masks, we expect the technology to continue to improve,” said Mei Ngan, a NIST computer scientist and co-author of the report, produced alongside U.S. Customs and Border Protection and the Department of Homeland Security.
Ngan and other researchers tested how 89 top facial-recognition algorithms performed “one to one matching,” which compares two photos of the same person — a common verification method for such tasks as unlocking a smartphone or checking a passport. They used more than 6 million pictures of a million individuals and added masks digitally, accounting for variations by using a range of colors, shapes and nose coverage.
Without masks, the top-performing algorithms usually have error rates of about 0.3 percent. But when the most accurate algorithms were confronted with the highest-coverage masks, error rates jumped to about 5 percent.
“This is noteworthy given that around 70 percent of the face area is occluded by the mask,” the report reads. “Some algorithms that are quite competitive with unmasked faces fail to authenticate between 20 percent and 50 percent of images.”
Companies are rushing to develop software that can make identifications based only on facial features that are still visible with a mask — a challenge, given that such algorithms depend on getting as many data points as possible. Researchers have been combing social media for masked selfies to create data sets to train facialrecognition algorithms, CNET reported in May.
Though controversial, the use of facial-recognition software by federal and local investigators has become routine, turning the technology into a ubiquitous presence in people’s lives, whether they are aware of it or not. Authorities harness it to scan hundreds of millions of Americans’ photos, often drawing on state driver’s license databases or booking photos. It’s deployed to unlock cellphones, monitor public areas and guard entrances to schools, workplaces and housing complexes.
Even retailers have made tentative steps in the arena. A recent Reuters investigation found that Rite Aid had been quietly adding facialrecognition systems to its stores for eight years. The technology was installed in 200 locations, mostly in lower-income urban areas, in what the report called one of the largest rollouts for an American retailer. The drugstore chain, after being presented with the findings, told the news organization the cameras had been turned off.
“This decision was in part based on a larger industry conversation,” the company told Reuters in a statement, adding that “other large technology companies seem to be scaling back or rethinking their efforts around facial recognition given increasing uncertainty around the technology’s utility.”
A growing chorus of lawmakers and privacy advocates say the technology threatens to erode American protections against government surveillance and unlawful searches, and that inaccuracies in the systems could undermine criminal prosecutions, unfairly target people of color and lead to false arrests. In a landmark 2019 study, NIST found that facial-recognition systems misidentified people of color more often than white people: Asian and African American people were up to 100 times more likely to be misidentified than white men, depending on the particular algorithm and type of search.
In January, a Michigan man was wrongfully arrested based on a faulty facialrecognition match in the first known case of its kind, the New York Times reported. The case was later dismissed, and the county prosecutor’s office said the man’s case and fingerprint data could be expunged.
Some facial-recognition software makers are rethinking their relationship to the technology. IBM discontinued its facialrecognition software in
June on the grounds that it promoted racism. The following day, Microsoft said it would stop selling its software to law enforcement until the technology is federally regulated. Soon after, Amazon, the largest provider of facial-recognition systems to law enforcement, said it would place a one-year moratorium on police use of the technology.
Last month, Democratic lawmakers introduced legislation that would ban federal agencies from using facial recognition and encourage state and local law enforcement to follow suit by making bans a requirement for certain grants.