Facial recognition tools falsely ID 28 lawmakers
ACLU says findings put minorities at risk
WASHINGTON — Amazon’s facial recognition tools incorrectly identified Rep. John Lewis, a civil rights leader, and 27 other members of Congress as people arrested for a crime during a test commissioned by the American Civil Liberties Union of Northern California, the watchdog group said.
The ACLU said its findings show that Amazon’s so-called Rekognition technology — already in use at some law enforcement agencies in Florida and Oregon — is hampered by inaccuracies that disproportionately put people of color at risk and should prompt regulators to halt “law enforcement use of face surveillance.”
For its test, the ACLU of Northern California created a database of 25,000 publicly available arrest photos, though the civil liberties watchdog did not give details about where it obtained the images or the kinds of individuals in the photos. It then used Amazon’s Rekognition software to compare that database against photos of every member in Congress.
Amazon’s technology flagged photos of 28 members of Congress as likely matches with the ACLU’s collection of mug shots. Among the misidentified lawmakers were Sen. Edward Markey, D-Mass., who has called for federal privacy legislation, and six members of the Congressional Black Caucus, including Lewis, D-Ga.
Two months earlier, the CBC wrote a letter to Amazon stressing that the lawmakers are “troubled by the profound negative unintended consequences this form of artificial intelligence could have for African-Americans, undocumented immigrants, and protesters.”
The CBC said the software was particularly risky because “communities of color are more heavily and aggressively policed than white communities,” meaning mistakes caused by faulty facial-recognition software could prove especially harmful.
On Thursday, Amazon questioned the ACLU’s methodology for its test, stressing that the threshold the watchdog set for what qualifies as a match — a “confidence,” or similarity rating, of 80 percent — had been too low.
“While 80 percent confidence is an acceptable threshold for photos of hot dogs, chairs, animals, or other social media use cases, it wouldn’t be appropriate for identifying individuals with a reasonable level of certainty,” an Amazon spokeswoman said.
But the ACLU of Northern California countered that 80 percent is the default setting on Amazon’s facial recognition tool. “Amazon should not be encouraging customers to use that confidence level for recognizing human faces,” said Jacob Snow, a technology lawyer at the organization.
Snow said the findings affirm their worst fears: that facial-recognition technologies are too unsophisticated to be deployed by law enforcement agents, where misidentification isn’t just a privacy concern — it “could cost people their freedom or even their lives.”
The privacy watchdog called on Congress to halt the federal government’s use of the technology, though lawmakers long have struggled to write any federal privacy rules around facial recognition or other high-tech tools adopted by police, including location tracking technologies.
Earlier this month, one of Amazon’s competitors, Microsoft, urged Congress to regulate facial recognition technology. Microsoft had faced intense criticism for providing a suite of cloud-computing tools to the country’s immigration enforcement agency.
Amazon’s facial recognition technology has worried civil liberties activists since May, after the ACLU of Northern California obtained and released an open records request showing Rekognition in use by law enforcement agencies around the country.