Mishap with state’s data obscures virus trends
ATLANTA — Where does Sunday take place twice a week? And May 2 come before April 26?
The state of Georgia, as it provides up-to-date data on the COVID-19 pandemic.
In the latest bungling of tracking data for the novel coronavirus, a recently posted bar chart on the Georgia Department of Public Health’s website appeared to show good news: New confirmed cases in the counties with the most infections had dropped every single day for the past two weeks.
In fact, there was no clear downward trend. The data is still preliminary, and cases have held steady or dropped slightly in the past two weeks. Experts agree that cases in those five counties were flat when Georgia began to reopen late last month.
DPH changed the graph Monday after more than a day of online mockery, public concern and a letter from a state representative. Gov. Brian Kemp’s office issued an apology and its spokespeople said they’d never make this kind of mistake again.
“Our mission failed. We apologize. It is fixed,” tweeted Candice Broce, a spokeswoman for the governor.
This unforced error — at least the third in as many weeks — is confounding observers who have noted sloppiness in case counts, death counts and other measures that are fundamental to tracking a disease outbreak. Georgians check the data daily to decide whether it’s safe to reopen their businesses or send their children to day care. Policymakers use it for decisions affecting the health of more than 10 million Georgians.
In recent weeks, DPH data issues caused confusion over whether novel coronavirus deaths had topped 1,000 — they are now more than 1,490. The agency erroneously posted at least twice that children died.
Some of these errors could be forgiven as mistakes made during a chaotic time. But putting days in the wrong order, as the recently withdrawn chart did, makes no sense.
“It’s just cuckoo,” said state Rep. Scott Holcomb, D-Atlanta, who sent the letter outlining his concerns to the governor’s office on Monday. The bar chart that stirred the latest controversy was revised shortly afterwards. “I don’t know how anyone can defend this graph as not being misleading. I really don’t.”
A spokeswoman for DPH told The Atlanta Journal-Constitution the chart was incorrect because of an error in how it sorted dates. An aide to the governor told Holcomb that a software vendor caused the problem, Holcomb said. A tweet from a Kemp spokesman said the data team behind the chart published it because they thought it would be “helpful.”
Kemp spokeswoman Broce said the office does not dictate what data DPH publishes.
“We are not selecting data and telling them how to portray it, although we do provide information about constituent complaints, check it for accuracy, and push them to provide more information if it is possible to do so,” said Broce.
Others worry the data is being portrayed in a way that favors Kemp’s early easing of restrictions. A separate graph on DPH’s page has led readers to think that cases were dropping dramatically, even though lower case numbers were the result of a lag in data collection.
“I have a hard time understanding how this happens without it being deliberate,” said state Rep. Jasmine Clark, D-Lilburn, who received her doctorate in microbiology and molecular genetics at Emory University. “Literally nowhere ever in any type of statistics would that be acceptable.”
Wrong information about Georgia’s battle against COVID19 is already shaping the way the public sees the state. A Friday column in The Wall Street Journal dubbed Kemp’s controversial decision to begin reopening, “The Georgia Model.”
It said the state is experiencing “a welcome trend of declining new cases and deaths.”
In fact, seven-day rolling averages of cases show only a slight decline over two weeks. Deaths appear to have plateaued, according to an Atlanta Journal-Constitution analysis of daily DPH reports. The impact of reopening Georgia’s economy is still too early to measure for both new infections and deaths because of the lag time between an infection, testing, diagnosis and, potentially, death.
RIPPLE EFFECTS
The latest flubbed chart lists case counts from the most recent 14 days, but data collection lags and a quirk in the state’s method of recording cases mean that counts for recent dates are often a fraction of what they turn out to be when the data is more complete.
“You really don’t want to be using very recent data to make decisions, given those delays,” said Benjamin Lopman, an infectious disease epidemiologist and an expert on using statistical analysis and other tools to address public health issues.
Interpreting data needs to be done with caution, especially now that DPH assigns the date of a new case in two different ways on its site, experts said.
When the pandemic began, the agency assigned a date to a case based on the day results came into its office. Starting in late April, DPH added charts that date a new coronavirus case back to the day a patient said symptoms started. If that data isn’t reported, DPH substitutes the date the test sample was collected or when it was received results.
But because it can take weeks for case information to come in, the new method always appears to show that cases are declining, even if they are not. The charts that used it stirred suspicion and confusion, and ran afoul of principles for communicating during a public health crisis, experts said.
Leaders must craft their messages carefully at a time like this, said professor Joseph Cappella, an expert on public health communication at the University of Pennsylvania’s Annenberg School for Communication.
“They need to be clear, they need to be consistent, they need to be credible and they need to be apolitical,” Cappella said.
MISTAKES HARM CREDIBILITY
DPH has made some improvements in recent days by apologizing and updating its online status report.
But among certain observers, the damage is done. Dr. Harry J. Heiman, a clinical associate professor at the Georgia State University School of Public Health, called the most recent mix-up “criminal” and said DPH has shown a pattern of reporting misleading data.
One example is a map of Georgia cases and infection rates that colors counties in shades of blue or red based on local rates of infection. In recent weeks, DPH raised the bar on how high an infection rate needs to be before a county is colored red.
“Based on the [key] they were using a couple weeks ago, a good third to a half of our state would show up as red right now,” Heiman said. “Because they keep moving the goalposts, if you will, it doesn’t look that way.”
The data reporting problems continue.
On Monday morning, DPH reported about 2,400 more confirmed cases than actual tests performed, said J.C. Bradbury, an economics professor at Kennesaw State University. Bradbury isn’t an epidemiologist, but he regularly tracks the DPH dashboard and is accustomed to processing large data sets.
The error was quickly fixed, but it should have never happened.
“It looks like you are doing something funny when you are just catching a mistake,” Bradbury said.
On Tuesday, DPH updated its charts again with clearer labeling that some of its most recent data is preliminary. But one chart for cases and deaths uses such a large numeric key, the number of deaths appear to be almost zero.
DPH may need to present its data more clearly, but it is improving at the urging of lawmakers, said state Sen. Kay Kirkpatrick, R-East Cobb, a physician who recovered from COVID-19 and is part of Kemp’s coronavirus task force.
She said she does not believe the state is acting in bad faith.
“I don’t feel there’s any intention … to mislead anyone or present [the data] in a way other than how they are,” Kirkpatrick said.
DPH’s epidemiology division is not political. It is chronically understaffed, said Dr. Melanie Thompson, principal investigator of the AIDS Research Consortium of Atlanta who has worked with the state epidemiology team for 30 years on the HIV/AIDS epidemic.
“I think they’re trying to do something that they think represents the epidemic,” Thompson added. “I think it ends up being extremely confusing to the average person.”
DPH can recover from any loss of credibility by providing more information about its current data reporting procedures and showing why its leaders think the new approach is a more accurate and responsible way of reporting, Cappella said.
The agency can also reassure the public by reporting case and death counts consistently across its public site, and conveying information thorough a person who is not a politician.
“I don’t think you can repair credibility overnight,” Cappella said.