How algorithms (secretly) run the world
WHEN you browse online for a new pair of shoes, pick a movie to stream on Netflix or apply for a car loan, an algorithm likely has its word to say on the outcome.
The complex mathematical formulas are playing a growing role in all walks of life: from detecting skin cancers to suggesting new Facebook friends, deciding who gets a job, how police resources are deployed, who gets insurance at what cost, or who is on a “no fly” list.
But while such automated tools can inject a measure of objectivity into erstwhile subjective decisions, fears are rising over the lack of transparency algorithms can entail, with pressure growing to apply standards of ethics or “accountability.”
Data scientist Cathy O’Neil cautions about “blindly trusting” formulas to determine a fair outcome.
“Algorithms are not inherently fair, because the person who builds the model defines success,” she said.
O’Neil argues that while some algorithms may be helpful, others can be nefarious. They too often amplify prejudice against the disadvantaged, says O’Neil.
A White House report last year also warned that algorithmic systems “are not infallible – they rely on the imperfect inputs, logic, probability, and people who design them.”
The report noted that data systems can ideally help weed out human bias but warned against algorithms “systematically disadvantaging certain groups.”
Zeynep Tufekci, a University of North Carolina professor who studies technology and society, said automated decisions are often based on data collected about people, sometimes without their knowledge.
“These computational systems can infer all sorts of things about you from your digital crumbs,” Tufekci said in a recent TED lecture.
“They can infer your sexual orientation, your personality traits, your political leanings. They have predictive power with high levels of accuracy.”
Such insights may be useful in certain
contexts – such as helping medical professionals diagnose postpartum depression – but unfair in others, she said.
Part of the problem, she said, stems from asking computers to answer questions that have no single right answer. “They are subjective, open-ended and value-laden questions, asking who should the company hire, which update from which friend should you be shown, which convict is more likely to reoffend.”
Frank Pasquale, a University of Maryland law professor and author of The Black Box Society: The Secret Algorithms That Control Money and Information, shares the same concerns.
He suggests one way to remedy unfair effects may be to enforce existing laws on consumer protection or deceptive practices.
Pasquale points at the European Union’s data protection law, set from next year to create a “right of explanation” when consumers are impacted by an algorithmic decision, as a model that could be expanded.
This would “either force transparency or it will stop algorithms from being used in certain contexts,” he said.
Alethea Lange, a policy analyst at the Center for Democracy and Technology, said the EU plan “sounds good” but “is really burdensome” and risked proving unworkable in practice.
She believes education and discussion may be more important than enforcement in developing fairer algorithms.
Lange said her organisation worked with Facebook, for example, to modify a much-criticized formula that allowed advertisers to use “ethnic affinity” in their targeting.
Others meanwhile caution that algorithms should not be made a scapegoat for societal ills.
“People get angry and they are looking for something to blame,” said Daniel Castro, vice president at the Information Technology and Innovation Foundation.
“We are concerned about bias, accountability and ethical decisions but those exist whether you are using algorithms or not.” – AFP