Was polling really as bad as they say?
The polling surprise of this year’s presidential election — the Democratic “blue wave” that was a no-show — has inspired vivid and unflattering descriptions about the surveys.
The Atlantic declared, for example, that 2020 was “a disaster for the polling industry.” Nate Cohn, who writes about polls and elections for the New York Times asserted: “The national polls were even worse than they were four years ago” when misfires in key Great Lakes states tipped the presidency quite unexpectedly to Donald Trump.
Easily the most colorful characterization came in a Twitter posting by Sean Trende, senior elections analyst for RealClearPolitics.com. “The polls,” Mr. Trende wrote, “were a stinking pile of hot garbage and there’s really no two ways about it.”
But will the polls-failed-us-again storyline remain intact? It’s a useful question, given the narrative shift that took place after the 2016 election. On the day after Donald Trump’s election, the American Association for Public Opinion Research said in a statement, “The polls clearly got it wrong this time.”
The organization’s assessment shifted with time. In a detailed report released in May 2017, AAPOR described the 2016 polls, nationally, as “generally correct and accurate by historical standards.” When all the returns were tabulated, Hillary Clinton won the popular vote by 2.1 percentage points, which was close to the RealClearPolitics aggregate that showed her ahead at campaign’s end by 3 percentage points. (The AAPOR report also acknowledged: “The state polls had a historically bad year in terms of forecasting” outcomes in 2016.)
The “generally correct and accurate” interpretation gained circulation and found expression in some pre-election analyses in 2020. Even so, the revised narrative masked the spectacular miscalls of poll-based forecasts in 2016, notably those of HuffPost, the Princeton Election Consortium, and the New York Times’ “Upshot” model.
These confident-seeming, widely publicized forecasts were essential to setting popular expectations about the outcome. Their failure contributed to the profound shock that accompanied Ms. Clinton’s defeat.
A few hints have emerged of an embryonic narrative shift about election polling of 2020.
The polls, the would-be revisionist interpretation goes, were “directionally accurate” in signaling Joe Biden’s victory in popular and electoral votes. Democrats could still win control of the U.S. Senate, depending on two runoff elections in
Georgia in January.
In addition, AAPOR has cautioned that “hasty conclusions based on incomplete returns may be misleading,” adding: “It will take weeks for election officials to carefully count all early, absentee, inperson and provisional ballots.”
Courtney Kennedy, lead author of AAPOR’s 2016 election polling report, suggested, “Let’s pump the brakes a bit” before passing judgment on polling performance this year.
When vote-counting is complete, Mr. Biden’s advantage in the national popular vote (which on Wednesday stood at 3.8 percentage points) may approach 5 points. That would not be terribly far from the average lead of 7.2 percentage points that RealClearPolitics reported on Election Day.
Yet, the average obscures several substantial misses by nationally prominent polls. For example, Quinnipiac University’s final pre-election survey pegged Mr. Biden’s lead over Mr. Trump at 11 percentage points. Polls conducted for NBC News/Wall Street Journal, CNBC/ Change Research, and the YouGov/Economist magazine all estimated Mr. Biden’s advantage at 10 percentage points.
Such margins, pollster Mark Penn noted the other day, “would have produced about a 40-state landslide — a result that on its face should have been dismissed as impossible.”
No revisionist interpretation of the 2020 polls will likely cloak such dramatic misses.
Misfires were perhaps even more striking in some battleground states this year, such as Wisconsin, home to a major polling surprise in 2016.
Mr. Biden’s average lead there on
Election Day was 6.7 percentage points; he carried Wisconsin by less than a point. Especially striking among the state’s off-target polls was the survey conducted for the Washington Post and ABC News that gave Mr. Biden a lead of 17 percentage points.
Polls also were off in aggregate in Florida, Iowa, Ohio and Texas, either in wrongly estimating Mr. Biden was ahead or in projecting a narrow lead for Mr. Trump that he easily exceeded.
Particularly troubling was that polls nationally and in key states tended to underestimate votes for Mr. Trump and other Republicans.
The Pew Research Center noted: “it’s clear that national and many state estimates were not just off, but off in the same direction: They favored the Democratic candidate.”
Polling, metaphorically, now finds itself “on the emergency room table,” Charles Franklin, director for the Marquette University Law School poll, has been quoted as saying. “You notice I did not say ‘we are on the autopsy table.’ I think the patient can still be helped.”
In time, perhaps, that sympathetic view will become widely embraced. But after another sharp and unanticipated blow to its reputation, election polling likely faces a prolonged convalescence, one disrupted by reminders of a blowout that wasn’t.