Weapons of math destruction or a force for social good?

Data has the power to change lives, and not always for good. It can help us to spot inequality, drive change and allow greater understanding of the lives of New Zealanders.

This country has been quick to embrace the lessons we can learn from data by using advanced analytical tools, such as problemsolving processes called computer algorithms. Computer algorithms, a procedure or formula for solving a problem or carrying out a task, are now a fundamental element of data analytics.

These technologies help us to identify patterns in relevant data and predict potential outcomes. Every day around New Zealand algorithms are helping us to do everything from forecasting the weather, to identifying native species at risk of extinction. Every time you stop at a traffic light, or step into an elevator, there’s an algorithm at work.

For all the benefits of this technology, there are challenges to manage as well. In her bestselling book, Weapons of Math Destruction, American data scientist Cathy O’Neil highlights many ways in which we need to watch out for unintended consequences as we use advanced data analytics such as algorithms.

In one such example, she describes how some business owners in the United States are now using algorithms to examine their data to more accurately predict how busy a store or restaurant is expected to be at any given moment. There are obvious benefits to the bottom-line profit, from matching more staffing to when most customers are in stores.

There are also some less obvious consequences from doing this. Increasingly, these businesses have moved away from a predictable 40-hour week for their staff, with some asked to work more and shorter shifts, at strange hours, and without much notice.

O’Neil highlights the impact of this change on workers who must juggle childcare, and other commitments, around such changeable working hours. She notes that this sort of work practice means that higher-level study to train for a new vocation is increasingly out of reach for some.

This example highlights the need to be aware of both the obvious and less-than-obvious consequences of using data to change the way things are done. In embracing new technologies to harness the power of data, there should be adequate safeguards to avoid unexpected results.

For the New Zealand Government, data is a treasure or taonga. Officials who collect and look after this precious resource must be responsible stewards in the same way we would for any other national treasure. We must always earn and maintain public trust by showing an ethical and thoughtful approach to data use. If we lose this trust, we lose the chance to make significant improvements to the lives of people in New Zealand.

Earlier this year I worked with the privacy commissioner to develop six principles for the safe and ethical use of data and analytics by government agencies. These principles are designed to support transparency and promote a best-practice use of data and analytics for decision-making. They’re a first step in helping the government lift its game in the use of data, and the way we engage with the public.

This includes getting the basics right; making sure that we collect, store and manage data properly and ensure that decision-makers are aware of its limitations. It also means taking into account the views of stakeholders and considering their perspectives where In the past, a simple series of operations for defining a process may have been considered an algorithm. Precise definitions meant a computer could calculate a result very quickly, leading to great increases in productivity. Such processes are still in widespread use today – we might call them automated business rules. One example is a system that automatically calculates weekly pay when given hours worked and pay rates. With more data and as more advanced statistical uses of data have evolved, techniques now allow computers to use previously collected data and learn statistical rules that can predict the likelihood of future outcomes. These techniques are known by terms such as machine learning, deep learning, and Artificial Intelligence. They can be differentiated from earlier types of algorithms because they make predictions of likely outcomes, and don’t merely give welldefined, precise results. For example, a bank might use an algorithm to calculate a credit risk score, based on a range of data about a person’s past financial management and earnings.

appropriate. Ensuring that a te ao Ma¯ ori perspective is embedded, through a Treaty-based partnership approach, in the same way we would with any policy or initiative.

I’ve also been working with the government chief digital officer on a review of algorithm use by government agencies. The first phase has assessed the operational algorithm use of 14 mainly social sector agencies with powerful data and analytics capability.

We’ve focused on reviewing algorithms in those organisations that are used to help make significant decisions that affect the lives of New Zealanders and people who come to this country. The full report will be published in September and will make recommendations to improve transparency and accountability of government algorithm use.

Technology, and the way we use data will keep changing, but the need to ensure we maintain public trust will not. I look forward to continuing the conversation on behalf of all New Zealanders.

Cathy O’Neil will be the keynote speaker at the Data Summit ’18 (September 27-28) in Wellington, which has the theme of informed decision-making through the ethical use of data.