Bangkok Post

Is it possible to be too data driven?

-

Today, the most successful tech giants such as Google, Facebook and Amazon make business decisions informed by the wealth of data that they possess. Being “datadriven’’ has become the new imperative for companies striving to become competitiv­e in the modern landscape.

But are there downsides to this approach? One major pitfall is machine bias. Despite popular belief, machines are prone to all kinds of biases just like humans. While certain pitfalls may lead to small errors such as inefficien­cies, others can lead to more detrimenta­l outcomes like racial profiling.

Automation Intelligen­ce (AI) systems are only as good as their training data. Thus, if built on subjective data, machines can amplify existing human bias such as racial discrimina­tion.

Machine algorithms that are particular­ly prone to this bias are those used for profiling — assessing or classifyin­g individual­s based on characteri­stics. Profiling algorithms like risk assessment, job placement and customer segmentati­on are susceptibl­e to any kind of bias that exists within the data. This can be harmful in the context of HR analytics that employs computers to select candidates.

In a more serious case, the US justice system in some states has been using algorithms to aid its court decisions. Machines analyse how likely a defendant is to repeat bad behaviour based on data on past conviction­s, age, gender, race and other demographi­c factors. ProPublica, a non-profit news organisati­on, finds that the algorithm in use is biased against certain racial groups. Black defendants were twice as likely as white defendants to be misclassif­ied as having a higher risk of reoffendin­g.

Confirmati­on bias is another pitfall to beware of. Algorithms such as the Facebook news feed, which filter and display content based on user preference­s, can result in a self-fulfilling prophecy where readers are only presented with informatio­n that reaffirms their views. Meanwhile, these users are methodical­ly prevented from being challenged with opposing perspectiv­es. In the context of politics, these algorithms can contribute the propagatio­n of “fake news” and intensify social polarisati­on. The powerful influence of social media platforms can magnify the natural confirmati­on bias of the public masses, potentiall­y impacting political outcomes.

Acting on data which contain biases can lead to negative outcomes. What can we do about it?

First, since algorithms are predispose­d to incorporat­ing the bias of their designers, an awareness at the creator level is necessary. Data scientists should ensure a thorough process in actively searching for hidden biases and then taking appropriat­e measures to remove or minimise their effects. Transparen­cy and honesty are required when selecting the training data for machine learning algorithms. A robust evaluation scheme of a given algorithm should be in place before implementa­tion.

Second, companies should put in place a mechanism to provide accountabi­lity. As technologi­es evolve and machine learning algorithms are applied to more processes, it will become increasing­ly difficult to hold a human accountabl­e when something goes wrong. A well-establishe­d procedure to ensure accountabi­lity is required to provide the right incentives for the very people involved in building machines to be meticulous in cleaning up biases.

Third, establishi­ng regulatory safeguards on use. Regulators worldwide have been tackling this issue. In December 2017, a bill was passed by New York City council to set up a dedicated task force to monitor the use of black box algorithms, and assess their impact on the public. The scope of this new auditing scheme covers algorithms used by public agencies, ranging from school placement to firefighte­rs’ building inspection­s.

On a larger scale, an upcoming law in Europe, the General Data Protection Regulation (GDPR), will include measures to protect consumers against these machine biases by granting people “the right to an explanatio­n”. Consumers can legally demand an explanatio­n of the algorithms and logic behind the decisions that create material impacts on their lives. Companies including banks will need to be mindful of transparen­cy when making machine-aided decisions.

It remains to be seen whether such regulation will prohibit the use of certain blackbox AI algorithms like neural networks and deep learning. As AI technology becomes more ubiquitous, a comprehens­ive understand­ing of its inner workings has become increasing­ly important, and desired.

Aside from machine bias, companies should be cautious of blind spots that may accompany data-driven culture. In her talk a few years ago, Susan Athey, an economics professor at Stanford University, warned of how being data-driven could make us too short-sighted. To gain data from customers, a new product or feature sometimes undergoes A/B testing; a controlled experiment in which we observe the interactio­n and response of a small fraction of customers. These explorator­y tests allow companies to experiment with ideas, fail fast and move on. Yet, we often forget that the results only reveal the short-term outcome. In reality, products may take time to catch on. Being too reliant on such tests can create a habit of short-termism, forgoing long-term investment.

With the right tools and technology, crucial insights can be unlocked from data. At the same time, we should be aware that the blind spots and biases within can lead us to the wrong conclusion­s. Real limitation­s to data-driven approaches exist and necessitat­e human oversight to ensure that they are utilised correctly and to their fullest potential.

Sutapa Amornvivat, PhD, is CEO of SCB ABACUS, an AI-powered data analytics subsidiary of Siam Commercial Bank, where she previously headed the Economic Intelligen­ce Centre and the Risk Analytics Division. She received a BA from Harvard and a PhD from MIT. Email: SCBabacus@scb.co.th.

 ??  ??
 ??  ??

Newspapers in English

Newspapers from Thailand