Very flawed studies used to justify bans on evictions
California Gov. Gavin Newsom recently signed a threemonth extension of the state’s COVID-19 eviction moratorium, which now runs through Sept. 30. But are these eviction bans necessary to protect public health during the pandemic?
Two studies that got widespread media attention and have been cited by politicians and government officials to support eviction moratoriums claim to show that the moratoriums saved thousands of lives.
“Researchers estimated that the lifting of moratoriums could have resulted in between 365,200 and 502,200 excess coronavirus cases and between 8,900 and 12,500 excess deaths,” said National Public Radio in an interview with postdoctoral researcher Kathryn Leifheit of UCLA’s Fielding School of Public Health.
Leifheit was the lead author of “Expiring Eviction Moratoriums and COVID-19 Incidence and Mortality,” a study cited by the Centers for Disease Control and Prevention in its order extending the federal eviction moratorium.
The second study, authored by a team of researchers at Duke University, claims nationwide eviction moratoriums would have reduced COVID-19 deaths by 40%. It got widespread media attention and was cited twice by the Consumer Financial Protection Bureau in the federal register as justification for its rulemaking.
These studies are deeply flawed.
Their underlying data is incomplete and inconsistent. Their results are implausible. The size of the effect is wildly disproportionate to other public health interventions. The researchers also claim an absurd amount of certainty in their results despite the large uncertainties in the data they use, and they assert a causal effect based solely on correlation.
If the authors were correct, they would have one of the greatest public health discoveries in history.
Consider that it took over a year and perhaps $100 billion to develop and administer COVID-19 vaccines, which researchers hope will reduce COVID-19 infections and deaths by similar amounts to what the two studies claim can be accomplished immediately with simple legal changes on eviction moratoriums.
Yet these eviction moratorium researchers find high confidence for a gigantic immediate effect from a legal change affecting a tiny subset of the population, using only population-level observational data.
The authors of the Duke study declined to even share their dataset or answer questions about apparent inconsistencies. That is a violation of basic research ethics in the case of a study that the authors and Duke University have promoted heavily in the media and to government agencies as justification for federal policies.
Doing our best to replicate what we think the Duke authors did, we find no significant relation at all showing moratoriums would have reduced COVID-19, much less anything of the size and certainty the authors claim. Although they say all their data comes from public sources, they refuse to say which ones; many of their data inputs are not available at the county level (which they need), nor for the time period covered by the study.
The data that is available has large uncertainty attached, far greater than the uncertainty the authors claim for their conclusions. The authors refused to explain how they can use input data with errors of 30% to 100% to form conclusions to three decimal places and claimed uncertainty under 10%.
The UCLA study, by contrast, is a model of transparent scientific reporting, with all data and methodologies disclosed, although the lead author also refused to answer our questions about inconsistencies.
These two flawed studies strongly influenced federal and state public policy on eviction moratoriums. They provided justification for a series of temporary measures that have had an impact on the livelihoods of landlords all over the country by denying them the legal recourse to enforce their contracts and reduced the supply of rental housing to financially stressed households.
To the extent we are going to involve scientific researchers in policy discussions, we should concentrate on issues that have been studied by many people openly sharing data and arguing back until there is a consensus among experts.
These shoddy one-off studies are just ammunition for people who want to put a link saying “studies prove” in their otherwise completely speculative articles.