New tech problem must be faced
Concerns grow over likely bias in facial recognition
An unusual consensus emerged recently between artificial intelligence researchers, activists, lawmakers and many of the largest technology companies: Facial recognition software breeds bias, risks fueling mass surveillance and should be regulated.
Deciding on effective controls and acting on them will be a lot harder.
This month, the Algorithmic Justice League and the Center of Privacy & Technology at Georgetown University Law Center unveiled the Safe Face Pledge, which asks companies not to provide facial AI for autonomous weapons or sell to law enforcement unless explicit laws are debated and passed to allow it.
Microsoft said the software carries significant risks and proposed rules to combat the threat. Research group AI Now, which includes AI researchers from Google and other companies, issued a similar call.
“Principles are great — they are starting points. Beyond the principles we need to be able to see actions,” said Joy Buolamwini, founder of the Algorithmic Justice League.
None of the biggest makers of the software — companies such as Microsoft, Google, Amazon.com, Facebook and IBM — has signed the Safe Face Pledge.
Large tech companies may be reluctant to commit to a pledge like this, even if they’re concerned about negative consequences of the software.
That’s because it could mean walking away from lucrative contracts for the emerging technology. The market for video surveillance gear is worth $18.5 billion a year, and AI-powered equipment for new forms of video analysis is an important emerging category, according to researcher IHS Markit. Microsoft and Facebook said they’re reviewing the pledge. Google declined to comment.
“There are going to be some large vendors who refuse to sign or are reluctant to sign because they want these government contracts,” said Laura Moy, executive director of the Center on Privacy & Technology.
Microsoft is still selling facial recognition software to governments, a fact that the American Civil Liberties Union took the company to task for last week. It asked Microsoft to halt the sales and join the organization’s call for a federal moratorium on government use of the technology.
The use of facial recognition for surveillance, policing and immigration is being questioned because researchers, including Buolamwini, have shown the technology isn’t accurate enough for crucial decisions and performs worse on darker-skinned people.
Providers have responded differently to the scrutiny. Microsoft is defending government contracts generally, while asking for laws to regulate the space.
Amazon took issue with research by the ACLU into the Rekognition program it sells to police departments, but the company has also said it’s working to better educate police on how to use the software.
Companies including Microsoft, Facebook and Axon, a maker of police body cameras, have formed AI ethics boards, and Google published a set of moregeneral AI principles in June.
The Safe Face Pledge asks companies to “show value for human life, dignity and rights, address harmful bias, facilitate transparency” and make these commitments part of their business practices. This includes not selling facial recognition software to identify targets where lethal force may be used.
The pledge also commits companies to halt sales of face AI products that are not “subject to public scrutiny, inspection, and oversight.”
There are also commitments to internal bias reviews as well as checks by outside experts, along with a requirement to publish easyto-understand information on how these technologies are used and by which customers.
Startups Simprints Technology, Robbie AI Inc. and Yoti Ltd. were the inaugural signers of the pledge.
“It’s kind of the wild west when it comes to use of automated facial analysis technology, and it’s also an area that’s shrouded in secrecy,” Moy said. The Safe Face pledge tries to address both areas, but Moy also believes new laws are needed.
That’s where Microsoft is focusing its attention. The company recently detailed the laws it would like to see passed. Microsoft president and chief legal officer Brad Smith put the chances of federal legislation in 2019 at 50-50, most likely as part of a broader privacy bill.
But he said there’s a far better shot at getting something passed in a state or even a city next year. If it’s an important enough region, say California, that would probably be enough to make software sellers change their products and practices overall, he said.
In the meantime, Microsoft said it will turn down some AI contracts where it has concerns, and already has. Smith wouldn’t specify which deals it has rejected, and he has also said Microsoft will continue to be a key vendor to the U.S. government.
“We’ve turned down business when we thought there was too much risk of discrimination, when we thought there was a risk to the human rights of individuals,” Smith said.
Amazon thinks it’s too soon to regulate.
“There are many positive and important uses of this technology that are being implemented today, to include preventing human trafficking, reuniting missing children with their parents, and improving security,” the company said in a statement. “It is too early to come out with blanket statements supporting broad regulation, given this technology is in the early stages of deployment, and we have received no indications of misuse.”
Still, the company said it will work with governments on standards and guidelines for the technology to maintain privacy and civil liberties.
Facebook said it’s committed to using the technology responsibly and supports thoughtful proposals. The social-networking company, which uses face recognition to identify people in photos users post, said it’s eager to work with Microsoft and others.