National Post (National Edition)

COVID fight has relied too much on stats, some say

`Models are not a panacea or magic ... They are a tool'

- TOM BLACKWELL

THE KIND OF LOCKDOWNS THAT WE HAVE SEEN HERE IN THE WEST ARE JUST AN IMPOSSIBIL­ITY IN MANY GEOGRAPHIE­S … THEY WOULD LEAD TO MASS STARVATION. THESE MODELS HAVE REAL CONSEQUENC­ES IF THEY ARE APPLIED HAPHAZARDL­Y. — DR. ZULFIQAR BHUTTA

The first several pages of the Public Health Agency's epidemiolo­gical update last week struck an optimistic note.

The number of COVID-19 cases, hospitaliz­ations and deaths were all falling steadily after a major early winter surge.

Then came slide 13 in the presentati­on, an alarming new model of what could happen next.

Based on more transmissi­ble variants of the virus spreading widely, it showed an almost vertical spike in the case count should public health restrictio­ns be eased. An only slightly less mountainou­s climb would occur into March and April, even if current lockdown measures were kept in place.

The only way to avoid a feared third wave of COVID-19 illness, the report suggested, was by imposing new restrictio­ns.

It was a sobering message, but hardly unfamiliar by now. Almost since the pandemic began, scientists have been using high-level mathematic­s and powerful computers to churn out models of how the virus might spread and affect Canadian society, and what mitigation measures would do to slow it down.

Few experts deny that modelling has been of at least some value, while measures taken in response seem to have staved off nightmare scenarios of sickness and death.

But a year into the pandemic, skeptics worry that the battle against COVID-19 has relied too heavily on mathematic­al projection­s that can be undermined by sparse data and an unpredicta­ble virus — and are often released with scant mention of their limitation­s.

The inadverten­t side effect may be public distrust of government officials fighting the coronaviru­s, critics argue.

“When you put out a model that is not credible and it turns out to be ridiculed, you've threatened the integrity of the whole health-care system,” argues Ed Mills, a Vancouver-based epidemiolo­gist and part-time McMaster University professor. “So don't be surprised if people start not believing what the public health agencies say.”

In fact, there's already skepticism about those federal prediction­s of a third wave in March fuelled chiefly by the B.1.1.7 mutation of the SARS-CoV-2 virus — the so-called U.K. variant.

Ontario released a similar model on Feb. 11, and a recent decline in cases seems to be flattening off.

But Mills, an adviser to the Gates Foundation, and others argue there simply is not enough evidence to conclude the trend is all but inevitable.

Projection­s for Canada are based on what happened in the U.K. in late December, when the more-transmissi­ble variant contribute­d to a troubling surge in cases, notes Dr. Zain Chagla, an infectious disease specialist at McMaster University.

But it's unclear, he says, how much of that spike can be blamed on the variant, and how much on the somewhat eased restrictio­ns in place in England at the time, plus the inevitable rise in contacts around Christmas.

In Denmark, meanwhile, the U.K. variant is prevalent, yet the case count has fallen for weeks, albeit in the wake of a strict lockdown, Chagla notes.

“In terms of prediction­s for a third wave, I just don't know,” says epidemiolo­gist Prabhat Jha, director of the Toronto-based Centre for Global Health Research. “We don't have enough data.”

Despite such reservatio­ns, though, modelling has many boosters in the scientific community.

While no one should view the science as some kind of crystal ball to peer into the future, it has played a “huge role” in pandemic planning, says one prominent expert in the field.

“I think it's been remarkably successful,” says Kumar Murty, a University of Toronto mathematic­s professor and chair of Ontario's science modelling table. “(But) models are not a panacea or magic or anything. They are a tool. They are one tool to help understand a complex phenomenon.”

It's easy to complain about a model being wrong when the actual outcome differs from the projection, he says.

But such criticism ignores the changes in policy and personal behaviour that followed the model, often in reaction to it, Murty says.

“We never wanted to be left with the scenario that it did happen and we weren't prepared,” Chagla says.

To try to foresee what might happen — and guide preparatio­ns — modelers crunch real-world data through mathematic­al equations to approximat­e the future course of a disease or the impact of interventi­ons.

The field has been around literally for centuries, the first recorded model predicting in 1760 that universal smallpox inoculatio­n would boost life expectancy by more than three years.

The math applicatio­n has boomed in recent months as scientists worldwide have put aside their regular pursuits to address the COVID-19 crisis.

The medrxiv.org preprint site and the PubMed registry list between them about 1,300 studies that mention modelling and COVID-19 in the title or abstract. But there have been questions since the beginning about some of the results.

U.K.'s Imperial College jolted the world with modelling that indicated the U.S. would see 2.2 million deaths if it did not take action to limit COVID-19's spread.

In Canada, early models foresaw an almost apocalypti­c impact on health care.

The University of Toronto-led COVID-19 Modelling Collaborat­ive suggested last March that under a “conservati­ve” scenario as many as 5,000 coronaviru­s patients at a time would need to be treated in an intensive-care unit by midMay, many requiring ventilator­s that were in limited supply.

To make room for that kind of deluge, hundreds of thousands of elective surgeries were cancelled across Canada, as government­s desperatel­y sought out new supplies of breathing machines.

COVID-related hospitaliz­ations didn't peak in Ontario until last month during the pandemic's second wave, when ICUs cared for up to about 400 people daily.

It was a serious strain on the system and weary health profession­als, but less than a tenth of what that initial model had predicted.

Modelling early in the pandemic also suggested schools could be a major vector for the virus, leading to widespread closures in the spring.

More recent projection­s have reached a different conclusion, though the role of schools as a possible accelerato­r of the pandemic continues to be hotly debated.

Beyond this continent, models by Imperial College predicted a huge death rate in South Asia, more than 2.5 million fatalities in Pakistan alone by last November, notes Dr. Zulfiqar Bhutta, co-director of the Centre for Global Child Health at Toronto's SickKids Hospital.

Others projected a smaller, but also hugely inaccurate toll, he noted.

Pakistan has had about 12,000 deaths and, as it turns out, politician­s largely ignored the foreign modelling, Bhutta says.

Perhaps in response to such prognostic­ations, though, India imposed lockdowns last spring that caused widespread hunger among migrant workers. Its COVID death rate has also remained low, one of the mysteries of the pandemic.

“The kind of lockdowns that we have seen here in the West are just an impossibil­ity in many geographie­s … they would lead to mass starvation,” Bhutta says. “These models have real consequenc­es if they are applied haphazardl­y.”

Mills says models produce by Seattle's Institutes of Health Metrics have become the world's most reliable. But the University of Washington institute generated controvers­y itself when it predicted in April that the U.S. would see only about 60,000 total deaths, a model then-president Donald Trump seized on as he tried to downplay the pandemic's gravity. What, if anything, went wrong? Especially early on, models likely erred by treating population­s as homogeneou­s, though some people were more or less resistant to the SARS-CoV-2 virus because of pre-existing immunity or other reasons, Jha says.

Mills argues that mathematic­ians often have failed to validate their models by running past data through them to see if they could accurately predict what has already happened. Or such data was simply unavailabl­e.

And much is unusual or unknown about the virus, he says.

Murty says modellers have, in fact, worked hard to incorporat­e new evidence about the disease into their work.

The U of T's Fields Institute for Research in Mathematic­al Sciences, which he directs, even conducted a survey to determine what percentage of people followed public health edicts on the coronaviru­s. The finding was about 70 per cent, informatio­n that can now be input into models on the potential impact of anti-COVID measures, Murty says.

Still, Jha worries that government­s have placed too much emphasis on modelling and not enough on empirical study of the pandemic as it unfolds.

He says he and academic colleagues, for instance, had to take the initiative to conduct a large “sero-prevalence” study to examine what percentage of Canadians have antibodies to the new virus — hard evidence of how widely it has spread.

“Every past public health pandemic that has been successful­ly dealt with has also created new knowledge,” says Jha. “Aside from the extraordin­ary success in vaccines, this pandemic has not produced extensive new knowledge.”

If there is one thing that critics and champions of modelling seem to agree on, it is the need to communicat­e the field's limitation­s, something health officials, politician­s and the media sometimes ignore.

Failing to explain fully the basis for projection­s and the fact they could well be wrong, risks fuelling conspiracy theories and COVID denialism, underminin­g the whole effort, Chagla says.

“We're a society that has been so paralyzed by COVID … everybody is looking at this stuff and saying `Where did this come from?' ” he notes. “You can't go `Here it is, see you guys later' … It's really important to be transparen­t.”

 ?? RYAN REMIORZ / THE CANADIAN PRESS ?? People line up to get tested at a COVID-19 clinic this week in Montreal. There is growing skepticism that a third wave is inevitable for Canada.
RYAN REMIORZ / THE CANADIAN PRESS People line up to get tested at a COVID-19 clinic this week in Montreal. There is growing skepticism that a third wave is inevitable for Canada.

Newspapers in English

Newspapers from Canada