Does facial recognition have a future?
Regulation against the future tech is looming amid concerns about its accuracy for policing and other public uses. Nicole Kobie reveals the future of facial recognition
Regulation against the future tech is looming amid concerns about its accuracy for policing and other public uses, discovers Nicole Kobie.
In January, Robert Julian-Borchak Williams was handcuffed and arrested in front of his family for shoplifting after being identified by facial recognition used by the Detroit Police Department. The system was wrong and he wasn’t a criminal but, because a machine said so, Williams spent 30 hours in jail.
Williams has the distinction of being the first person arrested and jailed after being falsely identified by facial recognition – or, at least, the first person that we the public have been told about. The Detroit police chief said at a meeting following the reports of Williams’ arrest that the system misidentified suspects 96% of the time. Given the wider discussion around reforming policing in the US following the killing of George Floyd by Minneapolis officers, it’s no wonder calls for bans of the tech are starting to be heard.
Amazon, Microsoft and IBM soon paused sales of facial-recognition systems to police, although it’s worth noting that there are plenty of specialist companies that still sell to authorities. Politicians are calling for a blanket ban until the technology is better understood and proven safe. “There should probably be some kind of restrictions,” Jim Jordan, a
Republican representative, said in a committee hearing. “It seems to be it’s time for a timeout.”
That’s in the US. In the UK, police continue to use the controversial technology. The Met Police used it at the Notting Hill Carnival and outside Stratford station in London, but the tech is also used by police in South Wales. “Facial recognition has been creeping across the UK in public spaces,” Garfield Benjamin, researcher at Solent University, told
PC Pro. “It is used by the police, shopping centres, and events such as concerts and festivals. It appears in most major cities and increasingly other places, but is particularly prevalent across London where the Met Police and private developers have been actively widening its use.”
That’s despite a growing body of evidence that suggests the systems aren’t accurate, with research from activist group Big Brother Watch claiming that 93% of people stopped by the Met Police using the tech were incorrectly identified. A further study by the University of Essex showed the Met Police’s system was accurate 19% of the time.
Can facial recognition ever work? Is a temporary ban enough? Or is this a technology that should forever be relegated to photo apps rather than serious use? The answers to these questions will decide the future of facial recognition – but the road forward isn’t clear.
The problems with facial recognition tech
The problems with facial recognition aren’t limited to a few instances or uses – it’s across the entire industry. A study by the US National Institute of Standards and Technology tested 189 systems from 99 companies, finding that black and Asian faces were between ten to 100 times more likely to be falsely identified than people from white backgrounds.
What causes such problems? Sometimes the results are due to poorquality training data, which could be too limited or biased – some datasets don’t have as many pictures of black people as other racial groups, for example, meaning the system has less to go on. In other instances, the algorithms are flawed, again perhaps because of human bias, meaning good data is misinterpreted.
That could be solved by having a “human in the loop”, when a person uses data from an AI but still makes the final decision – what you would expect to happen with policing, with a facial-recognition system flagging a suspect for officers to investigate, not blindly arrest. But we humans too easily put our faith in machines, says Birgit Schippers, a senior lecturer at St Mary’s University College Belfast. “There’s also concern over automation bias, where human operators trust, perhaps blindly, decisions proposed by a facial-recognition technology,” she said. “Trained human operators should in fact take decisions that are based in law.”
Even a sound system trained well on a solid dataset can have downsides. “It has a profound impact on our fundamental human rights, beginning with the potential for blanket surveillance that creates a chill factor, which impacts negatively on our freedom of expression, and perhaps our willingness to display nonconformist behaviour in public places,” Schippers explained. “Another fundamental concern is lack of informed consent… we do not know what is going to happen with our data.”
Then there’s the other side of human intervention: misuse. “Another key concern is the way that facial-recognition technology can be used to target marginalised, vulnerable, perhaps already overpoliced communities,” she said.
Whether we allow the tech in policing or elsewhere should depend on whether the benefits outweigh the downsides, argues Kentaro Toyama, a computer scientist at the University of Michigan. “The technology provides some kinds of convenience – you no longer have to hand-label all your friends in Facebook photos, and law enforcement can sometimes find criminal suspects quicker,” Toyama said. “But, all technology is a doubleedged sword. You now also have less privacy, and law enforcement sometimes goes after the wrong people.” And it’s worth remembering, added Toyama, that facial recognition isn’t a necessity. “There was no such technology – at least, none that was very accurate – until five to ten years ago, and there were… no major disasters and no geopolitical crises
because of the lack.”
Fixing facial recognition
“Another fundamental concern is lack of informed consent… we do not know what is going to happen with our data”
New technologies don’t arrive fully formed and perfect. They need to be trialled and tested to spot flaws and bugs and knock-on effects before being rolled out more widely. Arguably, facial recognition has been rolled out too quickly because we’re still clearly in the phase of finding the problems with and caused by this technology.
So, now that we know about bias, inaccuracy and misuse, can we fix those problems to make this technology viable? “If people are seeking to make them fairer, then on a technical level we need to address bias in training data which leads to misidentification of women and ethnic minorities,” said Solent University’s Benjamin. But, he added, you must be vigilant:
“The audit data that is used to test these systems as biased audits often conceal deeper flaws. If your training data is mostly white males, and your audit data is mostly white males, then the tests won’t see any problem with only being able to correctly identify white males.”
There have been efforts to build more diverse facial-recognition training sets, but that only addresses one part of the problem. The systems themselves can have flaws with their decision-making, and once a machine-learning algorithm is fully
trained, we don’t necessarily know what it’s looking for when it examines an image of a person.
There are ways around this. A system could share its workings, telling users why it thinks two images are of the same person. But this comes back to the automation bias: humans learn quickly to lean on machinemade decisions. The police in the Williams case should have taken the facial-recognition system as a single witness, and further investigated – had they asked, they would have learned Williams had an alibi for the time of the theft. In short, even with a perfect system, we humans can still be a problem.
Regulation, regulation, regulation
Given those challenges and the serious consequences of inaccuracy and misuse, it’s clear that facial recognition should be carefully monitored. That means regulators need to step in.
However, regulators aren’t always up to the job. “Facial recognition crosses at least three major regulators in the UK: the CCTV Commissioner, Biometrics Commissioner and Information Commissioner,” said Benjamin. “All three have logged major concerns about the use of these technologies but have so far been unable to come together to properly regulate their use. The Biometrics Commissioner even had to complain to the Met Police when they misrepresented his views, making it seem like he was in favour of its use. We need more inter-regulator mechanisms, resources and empowerment to tackle these bigger and more systemic issues.”
Beyond that, there is no specific regulation that addresses these concerns with facial recognition in the UK, noted St Mary’s University College Belfast’s Schippers, but there is currently a private members bill working its way through parliament, seeking to ban the use of facial recognition in public places. In Scotland, MSPs have already made such a recommendation, but plans by Police Scotland to use the technology had already been put on hold.
Such a pause could let regulators assess how and when to use the technology. “As the pros and cons become clearer, we should gradually allow certain applications, at progressively larger scales, taking each step with accompanying research and oversight, so we can understand the impacts,” said the University of Michigan’s Toyama.
That’s worked for other potentially dangerous, but useful, advanced technologies. “The most effective form of this is the development of nuclear energy and weapons – not just anyone can experiment with it, sell it, or use it,” Toyama added. “It’s tightly regulated everywhere, as it should be.”
Time for a ban?
Facial recognition is flawed, has the potential for serious negative repercussions, and regulators are struggling to control its use. Until those challenges can be overcome, many experts believe the technology should be banned from any serious use. “There is little to no evidence that the technologies provide any real benefit, particularly compared to their cost, and the rights violations are too great to continue their deployment,” Benjamin said.
Toyama agrees that a moratorium is necessary until the potential impacts are better understood.
“Personally, I think that many uses can be allowed as long as they are narrowly circumscribed and have careful oversight… though, I would only say that in the context of institutions I trust on the whole,” Toyama explained.
Schippers would also like to see a ban – but not only on facial recognition technology’s use by police forces, but by private companies too. “Retailers, bars, airports, building sites, leisure centres all use facialrejection technology,” Schippers said. “It’s becoming impossible to ignore.”
“It’s just another arrow in the quiver of technologies that corporations and governments use to erode our privacy”
What’s next?
Facial recognition is quickly becoming a case study in how not to test and roll out a new idea – but other future technologies could also see the same mistakes.
Look at driverless cars or drones: both are being pushed hard by companies and governments as necessary solutions to societal problems, despite the technologies remaining unproven, regulation not yet being in place, and the potential downsides not being fully considered.
That said, facial recognition seems more alarming than its fellow startup technologies. “There’s something about facial recognition that many people feel to be particularly creepy – but facial recognition is just another arrow in the quiver of technologies that corporations and governments use to erode our privacy, and therefore, our ability to be effective, democratic citizens,” said Toyama.
Due to that, and the inaccuracies, missteps and misuse, facial recognition faces a reckoning – and it’s coming fast.
“I think the next five years will see a strong tipping point either way for facial recognition,” said Benjamin. “With some companies ceasing to sell the technologies to the police, and some regulatory success, we could see them fall out of favour. But the government and many police forces are very keen on expanding their use, so it will be a matter of whether rights and equality or security and oppression win out.”
The pace of technology development continues to accelerate, but we should control its pace. We need to either slow it down via regulatory approval and testing, or speed up our own understanding of how it works and what could go wrong – or we risk more people like Williams being made victims of so-called progress.