The big idea: should governments run more experiments?
If you’re looking for recent reasons to be proud of Britain, it would be hard to find a better example than the Recovery series of clinical trials. Conceived in haste in the early days of the pandemic, Recovery (which stands for Randomised Evaluation of Covid-19 Therapy) sought to find drugs to help treat people seriously ill with the novel disease. It brought together epidemiologists, statisticians and health workers to test a range of promising existing drugs at massive scale across the NHS.
The secret of Recovery’s success is that it was a series of large, fast, randomised experiments, designed to be as easy as possible for doctors and nurses to administer in the midst of a medical emergency. And it worked wonders: within three months, it had demonstrated that dexamethasone, a cheap and widely available steroid, reduced Covid deaths by a fifth to a third. In the months that followed, Recovery identified four more effective drugs, and along the way showed that various popular treatments, including hydroxychloroquine, President Trump’s tonic of choice, were useless. All in all, it is thought that Recovery saved a million lives around the world, and it’s still going.
But Recovery’s incredible success should prompt us to ask a more challenging question: why don’t we do this more often? The question of which drugs to use was far from the only unknown we had to navigate in the early days of the pandemic. Consider the decision to delay second doses of the vaccine, when to close schools, or the right regime for Covid testing. In each case, the UK took a calculated risk and hoped for the best. But as the Royal Statistical Society pointed out at the time, it would have been cheap and quick to undertake trials so we could know for sure what the right choice was, and then double down on it.
There is a growing movement to apply randomised trials not just in healthcare but in other things government does. Overseas development is perhaps the most advanced example. The results of trials run by the Abdul Latif Jameel Poverty Action Lab (JPAL), for example, have deeply influenced how the UK government spends aid money. Applying scientific rigour to government programmes seems unobjectionable, especially in a world of fake news and backfiring populist policies. So why hasn’t it taken off faster?
One obvious objection is that experimental methods simply don’t apply to the questions that government deals with. You can’t run an experimental budget to test out whether the bond markets like it, much though that might be desirable. And many decisions that governments take are not straightforwardly about “what works”, but more about values. A randomised trial cannot tell you the correct ethical stance on immigration policy or the redistribution of wealth.
But these concerns shouldn’t be overblown. The fact is that the vast majority of what governments do on a day-to-day basis is deeply practical, concerned with how to get things done, rather than with ideology. If the scientific method can make the small things work well, that is incredibly useful.
Another objection to the use of trials is that they undermine the knowledge and expertise of people who deliver public services, from teachers to police officers to aid workers. There is a long and ignoble tradition of inappropriate targets and overcentralised control being used to dampen the initiative of staff in the public sector. Aren’t experiments, with their presumption of scientific authority, just another bureaucratic imposition?
As it happens, similar concerns were raised about randomised trials in medicine when they were first developed in the 1940s. Doctors feared that they would undermine the authority of the medical profession, by giving research findings priority over professional judgment. Over time, though, doctors came to accept that controlled trials were an important complement to their practice, rather than a threat to it.
Then there is the question of whether it’s fair to randomly allocate which citizen gets access to grants or publicly funded programmes, even in the interests of learning how to make them work better. Medical trials are sometimes stopped when the superiority of the treatment being tested becomes so clear that to give some subjects a placebo instead would be unethical; in the early days of the Aids crisis, some doctors prescribed antiretrovirals before they had been proven effective, because they felt they had an ethical duty to help patients who otherwise were expected to die.
Of course it would be wrong to deprive people of vital services even in the interests of research. But there is already plenty of variation when it comes to how many government policies are delivered. Introducing experiments is unlikely to make anyone worse off than random chance or “postcode lotteries”.
So if ideology, professional autonomy and ethics are not deal-breakers, what’s the problem? Politics is the obvious place to look. After all, the risk of running a trial is that it might show that a policy the government has advocated is actually no good, and politicians do not like being forced to admit they are wrong.
But recent evidence suggests that it might not just be politicians’ fault. Researchers found that ordinary members of the public disapproved of experiments, even when they approved individually of all the different policies or interventions that were being tested. If “experiment aversion”, as the study’s authors call it, is something we all feel, it is hard to blame politicians for giving voters what they want.
This may seem depressing. But in fact it points to a way forward. Those who want to see better government need to beat the drum for the experimental mindset, making the case not only to officials and politicians, but to citizens directly. We need to show people how experiments free us from bad policies, allow us to take calculated risks to change things for the better, and ultimately improve people’s
lives. This kind of campaigning may be uncomfortable ground for scientists and technocrats, but it is a battle worth fighting.
Stian Westlake is CEO of the Royal Statistical Society. Further reading: Spike by Jeremy Farrar and Anjana Ahuja (Profile, £9.99)
Doing Good Better by William MacAskill (Avery, £20.94)
A Field Guide to Lies and Statistics by Daniel Levitin (Penguin, £10.99)