Amazon’s facial recognition falsely matches lawmakers with arrest photos
The ACLU of Northern California ran a test on Amazon’s facial-recognition system and found something disturbing: It falsely matched the faces of 28 members of Congress with the mug shots of people who have been arrested.
After releasing the results of its report Thursday, the American Civil Liberties Union reiterated its call for amoratorium on government use of facial recognition.
“Congress should press for a federal moratorium on the use of face surveillance until its harms, particularly to vulnerable communities, are fully considered,” said Neema Singh Guliani, ACLU legislative counsel, in a statement. “The public deserves a full debate about how and if face surveillance should be used.”
The ACLU noted that its test also showed that “Congressional members of color were disproportionately identified incorrectly, including six members of the Congressional Black Caucus,” among them civil rights leader Rep. John Lewis, D- Georgia.
Another of the lawmakers whose face was mismatched by Amazon’s system was one from the Bay Area: Rep. Mark DeSaulnier, D- Concord. Thursday, he and Sen. Ed Markey, D- Massachusetts, and Rep. Luis Gutierrez, D-Illinois, who were also misidentified, sent a letter to Amazon CEO Jeff Bezos, asking him to provide details about law enforcement’s use of Rekognition.
“Amazon must slow down and consider not just its profit margin, but the impact on society that this technology will have,” DeSaulnier said in an email statement to this news organization Thursday.
Amazon has been facing other resistance against Rekognition, its facial- recognition software, including from its own employees. In June, Gizmodo reported that Amazon workers sent a letter to Bezos, urging him to stop selling facial recognition to law enforcement. Amazon shareholders also asked Bezos the same thing, saying, “Such government surveillance infrastructure technology may not only pose a privacy threat to customers and other stakeholders across the country, but may also raise substantial risks for our company, negatively impacting our company’s stock valuation and increasing financial risk for shareholders.”
Despite the controversy, “We remain excited about how image and video analysis can be a driver for good in the world, including in the public sector and law enforcement,” an Amazon Web Services spokeswoman said Thursday.
She added that ACLU’s test did not set the confidence threshold — the likelihood that the system found a match— high enough.
“While 80 percent confidence is an acceptable threshold for photos of hot dogs, chairs, animals, or other social media use cases, it wouldn’t be appropriate for identifying individuals with a reasonable level of certainty,” she said. “When using facial recognition for law enforcement activities, we guide customers to set a threshold of at least 95 percent or higher.”
In response, the ACLU pointed out Thursday that Amazon sets 80 percent confidence as Rekognition’s default threshold, and that the company’s own website shows the company “is recommending” an 80 percent confidence score in “FaceBased User Verification.”
As for the ACLU’s point about bias in facial recognition, especially when it comes to people with darker skin, it’s a common knock against the technology — and one that is being addressed by Microsoft and IBM, they announced recently.
“Face surveillance will be used to power discriminatory surveillance and policing that targets communities of color, immigrants, and activists,” said Jacob Snow, technology and civil liberties attorney at the ACLU of Northern California, in a statement. “Once unleashed, that damage can’t be undone.”
In addition, Microsoft President Brad Smith last month called for regulation of facial recognition, saying tech companies can’t regulate themselves on this issue, especially if government is using the technology for various purposes.