Financial Mail


- Katharine Child childk@businessli­ Baby Shark

Can scientific models be trusted? The gap between expectatio­n and outcome has raised questions about the projection­s that have shaped lockdowns and disrupted economies around the globe

The early figures read like a dystopian, end-of-the-world movie script. As the Covid-19 pandemic was taking hold, epidemiolo­gical models predicted 350,000 deaths in SA without a lockdown, 96,000 in Sweden (26,000 under a best-case lockdown scenario), 2-million in the US this year (peaking in July), and 500,000 in the UK.

Thankfully, these models are wrong. SA has so far had about 500 deaths from Covid19 — a number that is, however, expected to rise exponentia­lly for months. There have been about 4,029 deaths in Sweden, where death rates have been in decline since April 22; about 100,000 in the US, where daily figures have dropped since April 15; and about 37,000 in the UK, where daily numbers have declined since April 9.

This difference between expectatio­n and outcome has raised questions about the scientific models that have shaped lockdowns around the world and left economies reeling.

In the UK, the Imperial College London model that influenced the government decision to lock down the economy could best be described as the song of Covid-19 models: it stuck like an earworm, made the world sit up listen and is now, if UK headlines are anything to go by, somewhat despised.

The model, which predicted 500,000 UK infections, was released on March 16 and, according to media reports, informed Prime Minister Boris Johnson’s decision to announce a lockdown a week later.

After weeks of pressure to make the coding behind its model public, Imperial College study lead Prof Neil Ferguson relented. It’s a decision he may come to regret.

The coding the model relied on has been pulled apart by health experts and global coders, derided as the “worst coding I have ever seen”; as a 13-year-old, 15,000-line code not up to industry coding standards; and as “riddled with bugs”. One senior industry coder reportedly said he would have fired anyone who produced that code for him.

When tested by epidemiolo­gists at the University of Edinburgh, the model gave different answers when it was run on different computers — and different ones again when rerun on the same computer.

In SA, similarly dire prediction­s have been made about the course of the pandemic.

Stellenbos­ch University’s SA Centre for Epidemiolo­gical Modelling & Analysis (Sacema), for example, released a model in early March suggesting a death rate of 350,000 if no lockdown were implemente­d.

Health minister Zweli Mkhize has been at pains to deny that the government’s lockdown was based on that figure. But he admitted in a virtual meeting last week that the first models the government had sight of showed far, far higher fatality figures than the figure of 40,000 some local models have converged on.

What do the models in SA say now? And can we trust them?

The Actuarial Society of SA (Assa) predicts about 48,300 deaths, with a high end of 88,000. The SA Covid-19 Modelling Consortium model, designed by a range of academics and used by the government for planning purposes, predicts 40,000 to 48,000 deaths.

The FM tried to contact epidemiolo­gist Dr Harry Moultrie and other members of the consortium after Moultrie promised “transparen­cy in [the] interests of democracy”, but they did not answer, or even acknowledg­e, the questions. Assa answered the FM’S questions in detail.

In its public documents, the SA consortium warns models are based on assumption­s which can be incorrect. The model write-up states, in distinctiv­e bold font: “These projection­s are subject to considerab­le uncertaint­y and variabilit­y. Estimates will change and improve as the epidemic progresses and new data become available.”

To illustrate just one of the difficulti­es facing modellers: many people who contract Covid-19 remain asymptomat­ic or have such mild symptoms that they don’t seek medical attention and so aren’t tested, making disease prevalence difficult to measure.

Sacema’s Prof Juliet Pulliam said at the virtual launch of the consortium’s modelling results that as deaths increase in SA, this data will be used to improve the consortium model.

Deaths can be a better indicator of infections. But even this is tricky. European countries, US states and cities abroad are now measuring “excess deaths” — a measure of unexpected­ly high deaths compared with previous years. These figures suggest there are more Covid-19 deaths than have been officially counted.

So if even death figures are uncertain, one can only have sympathy for those modelling data for a six-month-old disease that contains so many unknowns.

Models on new diseases have been wrong before.

Barry Childs, who is in charge of the Assa model, says initial models on HIV, for example, “were based on uncertain assumption­s and were therefore often wrong. Today we have a pretty solid idea of how HIV works and models are therefore very reliable.”

So how much store are we to put in such models if the input data is likely incorrect?

them asymptomat­ic and not detected.

Remove children from the equation — they seem less susceptibl­e to infection — and that means about one in three adults in SA will have been infected within six months.

The model doesn’t explain why this is likely in SA, when no country in Europe has come close to such a number. There, studies of antibodies to the coronaviru­s in people’s blood show a prevalence of 4%-7% in the population.

Studies in New York, in the US, however, put the number much higher, at an infection rate closer to 20%.

When it comes to prediction­s of fatalities, these are calculated as a percentage of the number of people who get infected — so if modellers get the infection numbers wrong, the projected death rate could be inflated.

Economist Alex van den Heever, who put together a completely different model for the National Treasury in March, argues the point of the lockdown is to “suppress the epidemic”. To assume it will “run its course” and infect almost everyone before subsiding is, he says, to plan for failure.

This is exactly what the Assa and consortium models assume.

Childs admits this isn’t what has been seen in Europe, but it could be because the lockdowns worked.

Another explanatio­n for the high rates is the type of model used: the susceptibl­e-exposedinf­ectious-recovered (SEIR) model. This assumes everyone who has not had a new disease is susceptibl­e to it, and if exposed could become infected.

“The one big assumption on the SEIR model for a novel virus is that everyone is susceptibl­e, which is why the projection­s of total infected numbers are so high,” says Childs.

He says the Assa team is looking for other explanatio­ns “in the emerging data” which shows not everyone gets the disease.

“The current Assa Covid-19 model predicts high prevalence. Given the novelty of the virus, without effectivel­y curbing its spread most of the population would be exposed and infected, the vast majority being asymptomat­ic,” he says.

As Assa tries to refine its model, it is “considerin­g allowing for variation in susceptibi­lity”. This means trying to calculate that some people will get infected, while others exposed to the virus just won’t catch it.

In the end, what everyone agrees on is that all models are highly limited by their assumption­s. The consortium even warns the government to use its model “with caution”. Yet the power of models to shape drastic government decisions will be long debated, even after Covid-19 infections have waned.

 ??  ??

Newspapers in English

Newspapers from South Africa