Supermarket AI mistakes M¯aori shopper for thief
ROTORUA: It is not surprising that a supermarket trialling facial recognition technology mistakenly identified a Ma¯ori woman as a thief, Ma¯ori AI and data experts say.
The woman was shopping at New World in Rotorua — one of 25 Foodstuffs’ North Island supermarkets trialling the technology — when she was approached by two staff members and asked to leave.
The technology scans faces and compares images with those on the store’s databases of known offenders or suspects.
Ma¯ori AI and data ethicist Karaitiana Taiuru said he was not surprised to see Ma¯ori singled out in the trial and believed other Ma¯ori had been affected but had not yet come forward.
The systems Foodstuffs was using were trained on an international dataset of people, not on people in a New Zealand context, he said.
‘‘We know from international data and international research that the systems are based on Europeanlooking men, so it’s only in the last few years that the system has been modified to consider people of colour, women, men with beards etc.
‘‘Anyone who is Ma¯ori, Pasifika, any person of colour is not going to be recognised by the system. That’s why it’s important that we have humans doublechecking the system, which in this case failed.’’
The woman was trespassed from the Rotorua supermarket despite offering three forms of photo identification to staff.
Mr Taiuru said people manually checking positive matches from facial recognition technologies often still relied on AI and not on their own common sense.
‘‘It’s a phenomenon that’s identified in America where the staff rely on the AI more than their own personal judgement. So regardless of how obvious a mismatch is, humans still rely on the AI to be correct,’’ Mr Taiuru said.
This bias could be reduced with appropriate staff training — which would have to be customised to the New Zealand population, he said.
As far as he was aware, there was no outofthe box training available in New Zealand yet.
If you were misidentified by facial recognition technology, it was best to complain to the organisation using it or the Privacy Commissioner, he said.
The chief executive of data publishing business Figure NZ, Ngapera Riley, from Rotorua, said it was outrageous to see Ma¯ori being profiled.
It was a clearcut example of why the technology might not yet be reliable.
‘‘I’m not surprised this happened in Rotorua with the supermarket trials. But these supermarkets, they’ve got a lot of money and access to resources so they really should be doing a better job at preventing these situations.’’
It was a bad look when something like this happened in a high Ma¯ori population area like Rotorua, she said.
There were beneficial and concerning elements to biometrics and Ms Riley said she understood why supermarkets and retailers were interested in the technology.
‘‘The reason that supermarkets and retailers are implementing this technology is because they’ve got a huge increase in crime and their staff getting assaulted while doing their jobs . . . so they’re wondering, can these new technologies help them with that?’’
The office of the Privacy Commissioner was asking the public to have its say on a draft biometric code of practice.
New Zealand did not have special rules for biometrics at present.
Ms Riley said biometric data included fingerprints, facial, voice, irises, palms and hand technology.