Computer says no, not ok
The law lets online services turn customers away without an immediate explanation, reports Madison Reidy.
There may be a need to have an obligation on companies to explain how an artificial intelligence-made decision was made. John Edwards.
Asplit payment service can decline you from using it within seconds, without telling you why, but the privacy commissioner says that could soon change.
If you want to pay a purchase off over a few weeks with no interest, some retailers will let you do it through third parties like PartPay, Laybuy and Genoapay- but only if you pass their algorithm tests.
When applying to use the services at the checkout, you enter your personal details and within seconds, it checks your credit score and identification.
It does this to find out how likely it is that you will pay back the money on time. If you are not eligible, (deemed too financially risky), you will be declined with no explanation.
Privacy commissioner John Edwards, said a refresh of the Privacy Act could make those businesses explain their automatic decision-making formula to customers.
‘‘There may be a need to have an obligation on companies to explain how an artificial intelligence-made decision was made. We don’t really have anything like that at the moment,’’ Edwards said.
There is no principle in the Act that mandates businesses to explain the technology they adopt behind the scenes. Legally, they just need a decent reason to take and use your data.
However, Edwards said under the law customers were entitled to an explanation if they asked for it.
But technology consulting company Infosys’ New Zealand country manager Patrick Kouwenhoven, said it was up to customers to read the terms and conditions and decide if they had a problem with any technology.
If you had an issue with artificial intelligence (AI), do not sign up to the service, Kouwenhoven said.
Edwards said that was irresponsible.
Customers could not be expected to analyse masses of terms and conditions and make a fully informed decision about the use of their data, he said.
PartPay chief executive John O’Sullivan, said his business’ algorithm was not artificially intelligent. It gathered and crunched online information, but could not think for itself.
Kouwenhoven said telecommunication companies and banks were using algorithms in the same way.
A computer code algorithm is confusing, so an explanation could tell you what data was used and what data it compared yours to.
Wellington Institute of Technology IT professor Steve Mckinlay, said the computer programming industry was shrouded in secrecy under patents and it was unlikely it would share the algorithms it built for businesses with the public.
O’Sullivan said PartPay would communicate its algorithm to customers the best way it could, if a law change required it to.
He said new legislation would spark an ‘‘industry wide change’’ and affect peer-to-peer lenders and other credit providers.
Edwards agreed the complexity of algorithms made transparency difficult.
But it was important to be open, because if machines were making decisions for customers, they had a right to know how they worked, especially if they were declined, he said.
‘‘Automated decision-making based on large data sets … [has] the potential to affect people’s lives without people really understanding how those decisions are made, and that can be a concern.’’
Edwards said New Zealanders expected companies to use their data for the purpose they said they would.
‘‘You have got to be very confident in it and justified, otherwise you could be leaning to unjust outcomes for people.’’