We could turn into ‘algocracy’ with lives ruled by computers

‘Unaccountable data scientists’

IT SOUNDS like the plot of a bleak science fiction film – where machines take over the world and govern every aspect of our lives.

But a top City official has warned we really are in danger of becoming a so-called ‘algocracy’ where most life-altering decisions are taken by algorithms.

In this dystopian future, intelligent computer programs will decide whether customers are given mortgages and insurance, the chairman of the Financial Conduct Authority said.

‘Some have said that in the future we will live not in a democracy, where citizens decide how we are governed, nor in a bureaucracy, where officials like me decide, but in an algocracy, where algorithms decide,’ Charles Randall said. ‘How do we make sure that our financial future is not governed by rules set by unaccountable data scientists, and implemented by firms who may not even understand how these algorithms reach their decisions?

‘Companies have created enormous datasets containing information about the different aspects of our lives.’

As our homes, appliances and mobile phones produce more data, these datasets will grow even bigger.

Advancements in technology, meanwhile, mean companies are able to mine this data for patterns, he said.

So whereas before firms could draw conclusions about only broad groups of people – such as those of a certain age – now they can make predictions about our personal behaviour.

Mr Randall, who was speaking at a Reuters event in London yesterday, also raised concerns over the type of personal data some firms were using. He highlighted an example of credit card companies in the US which had cut people’s credit limits if they paid for marriage counselling as there was a strong correlation between a breakup and missed payments.

He said: ‘It may be acceptable to use detailed data about people’s eating and drinking habits to price their health insurance, but is it fair to use that data to price their mortgage?

‘ People, not machines, need to understand and control the outcomes that the technology they are designing is producing; people, not machines have to make the judgment as to whether these outcomes are ethical.’

Mr Randall added that firms must clearly explain to customers what data they are using or face new rules.