CDC coding error led to overcount of 72,000 Covid deaths
A quiet change to how the US Centers for Disease Control and Prevention (CDC) publicly reports Covid death details underscores the need for the agency to communicate clearly and transparently about rapidly evolving science, experts say.
The past two years have created numerous communication challenges for the agency, which works with massive amounts of data from scores of different sources, including states and territories.
“Mistakes are inevitable because humans are fallible, but there should always be an effort promptly to explain what happened and what’s being done to prevent it from happening again,” said Tom Frieden, a former CDC director and the president and CEO of Results Save Lives.
“You have to over-communicate, basically,” he said. “Any time there is something that needs to be corrected, be upfront about it: here’s what happened, here’s what we know, here’s what we don’t know.”
Last week, after reporting from the Guardian on mortality rates among children, the CDC corrected a “coding logic error” that had inadvertently added more than 72,000 Covid deaths of all ages to the data tracker, one of the most publicly accessible sources for Covid data.
The agency briefly noted the change in a footnote, although the note did not explain how the error occurred or how long it was in effect.
A total of 72,277 deaths in all age groups reported across 26 states were removed from the tracker “because CDC’s algorithm was accidentally counting deaths that were not Covid-19-related”,
Jasmine Reed, a spokesperson for the agency, told the Guardian.
The problem stemmed from two questions the CDC asks of states and jurisdictions when they report fatalities, according to a source familiar with the issue.
One data field asks if a person died “from illness/complications of illness,” and the field next to this asks for the date of death. When the answer is yes, then the date of death should be provided.
But a problem apparently arose if a respondent included the date of death in this field even when the answer was “no” or “unknown”. The CDC’s system assumed that if a date was provided, then the “no” or “unknown” answer was an error, and the system switched the answer to “yes”.
This resulted in an overcount of deaths due to Covid in the demographic breakdown, and the error, once discovered, was corrected last week. The CDC did not answer a question on how long the coding error was in effect.
“Working with near real-time data in an emergency is critical to guide decision-making, but may also mean we often have incomplete information when data are first reported,” said Reed.
The death counts in the data tracker are “real-time and subject to change”, Reed noted, while numbers from the National Center for Health Statistics, a center within the CDC, are “the most complete source of death data”, despite lags in reporting, because the process includes a review of death certificates.
Reporting on causes of death is difficult even in non-pandemic times, experts said.
“It’s really hard to get accurate numbers,” said Glen Nowak, a former director of media relations at the CDC and co-director of the Center for Health & Risk Communication at the University of Georgia’s Grady College. “It’s not just with Covid – it happens with pretty much any infectious disease or even foodborne illnesses and waterborne illnesses, where there’s a large outbreak.”
There are a few reasons for that. Healthcare providers usually report the initial data, but treating patients is a more pressing priority. Death certificates take time to complete, and the cause of death may be subject to political pressure or difficulty determining the underlying reason or reasons. States and other jurisdictions may have lags in sending the data to the CDC; currently, one-third of deaths are reported after 10 days. Then the CDC processes the data, which can also be an involved process.
Most CDC data comes from state and local governments, and the quality can vary widely “because there has been a decades-long under-investment in public health at the national, state, city and local levels,” Frieden said.
“The public health and healthcare system we have in this country makes it extraordinarily difficult to collect data well.”
The CDC estimates that more than 968,000 Americans have died of Covid, and this change does not seem to have affected that estimate. The tracker shows demographic data on about 785,000 deaths, which means there may be more than 180,000 deaths not yet tallied in these breakdowns.
The recent change to demographic data shows the difficulty of offering up-to-date assessments while data reporting and analysis have lagged in the face of a massive outbreak. Data on the same topic across the CDC can also vary depending on the source and how numbers were calculated.
“The level of precision that you see in these numbers makes you think that they must be really super accurate,” Nowak said. Instead, they are informed estimates that help contextualize the scope of Covid compared with other illnesses.
“I don’t think public health and others do a good-enough job of reminding people that these numbers have significant margins of errors,”
Nowak said. “The caveats need to be clear that these are our best estimates based on the data that has been reported to CDC.”
As the scientific evidence accumulates, adjustments and changes are inevitable and frequent. But significant changes in calculations and records need to be explained clearly, particularly in an emergency where the public is frequently attuned to data – and to unexplained changes – like this.
“The best practice, really, is to have virtually daily briefings, so that you’re updating daily about what you’re seeing and you’re answering questions daily,” Frieden said. That’s how the agency addressed past outbreaks of Ebola, Zika and H1N1, also known as swine flu.
These briefings should be held by the scientists with expertise in many areas, not just the director of the CDC, he said. The current CDC administration has “gradually been getting back in the habit of doing that, and I hope that trend will continue”.
The CDC is a government agency that provides data to inform national policy, and public health policies often have some political component, Frieden said. It can never be apolitical. “But you should never have any concerns about the accuracy of the data.”