Juliet SAMUEL
From elections to the economy, clever people can’t predict the future – but that’s no bad thing
No one likes making the same mistake twice, and America’s pollsters didn’t. They made a whole new set of mistakes instead. In the days before the presidential election, an average of all the main polls in America suggested that Joe Biden was leading by 8 per cent. But as results came in, it became clear that his lead was far narrower, probably closer to 4 per cent or less. As the Republican pollster Frank Lutz put it: “That’s not within the margin of error. That’s simply being wrong.”
We are now in the age of “big data”, in which data scientists, “superforecasters” and epidemiologists are the new high priests reading the runes of the future. Yet on the big judgment calls, we seem to keep following these maths whizzes down blind alleys.
American pollsters were badly burned by their failure to spot the Trump wave in 2016, so they were determined not to do the same thing again this time. They carefully adjusted their techniques to take account of the so-called “shy Trump voter”, but while they fixated on that problem, they missed another one.
Currently, US polling models treat Hispanic and African American voters as fairly homogenous blocks. In fact, it seems that the country’s ethnic minorities are not the monolithic voting fodder the models predict. They fragmented into different groups, with Hispanic voters in Florida and North Carolina moving towards Trump and African-american voters in parts of the south simply not turning out as they were “meant” to. That means pollsters need to start sampling these groups in much more detail.
But it isn’t just polls. Government scientists have justified England’s new lockdown on the basis of models that predicted we would currently be seeing three to four times more Covid deaths than we actually are. Before that, Sage had incorrectly predicted that demand for Covid tests would not surge until October, catching out the Government testing system in early September, when the surge actually appeared. Likewise, the City is littered with the lost credibility of economists and fund managers, who keep trying to second-guess markets and failing.
Decades of expertise and thought have gone into the models that generate these predictions. The phenomena they are trying to measure are also generating more usable information than ever before, such as location data showing how much we move around or purchasing records correlating our shopping habits with our news consumption.
All of this has lulled clever people into thinking they can predict the future. It’s an echo of the marvellous Enlightenment ideal proposed in the Tom Stoppard play Arcadia: “If you could stop every atom in its position and direction, and if your mind could comprehend all the actions thus suspended, then if you were really, really good at algebra you could write the formula for all the future; and although nobody can be so clever as to do it, the formula must exist just as if one could.”
If that formula does exist, we are still very far from finding it. It turns out that huge, dynamic systems like national votes, disease contagion and economic growth are not mechanical schemes easily captured by models. They are messy and organic. It is very hard to spot when an anomaly becomes a trend or when a trend reaches a tipping point.
This does not mean we should give up on “the new Enlightenment”, as one data maven calls it. There are niches in which data is starting to generate incredibly powerful predictions, such as the impending need to restock shop inventories or how likely a wind turbine is to fail in the next month. We are at the start of an era in which our use of data will become more refined. But when it comes to complicated systems, we are still better off relegating models to their proper place: stimulating thought experiments.
This might be suboptimal for the polling business, but it’s wonderful news for our sense of humanity and the possibility that we will defy artificial intelligence systems designed to control or predict everything we do. If the data guys keep getting it wrong, it means our potential for growth and creativity is still boundless. So long live modelling upsets. They might make for hours of defunct punditry and some terrible policy mistakes, but so long as they happen, it means human beings are still free.