The Washington Post

It’s impossible to trust polls — or ignore them. Here’s how to stay sane.


It feels impossible to trust polls — and equally impossible to ignore them. The polling industry faces well-known problems: Only 6 to 7 percent of people will pick up a phone call from a pollster; polls completely failed to find segments of Donald Trump’s base in 2020; and predicting exactly who will vote is mathematic­ally thorny.

Yet polls are indispensa­ble. They’re the only data source that directly asks people how they’re going to vote and credibly attempts to represent the whole electorate.

This situation makes political junkies miserable. They constantly check Fivethirty­eight, Realclearp­olitics and other poll aggregator­s for new data, yet they can’t shake nagging doubts about every number on the screen.

As a profession­al political junkie, I feel this pain, too. So, I collected thousands of Senate polls from past elections, crunched the numbers and came up with some rules of thumb that can help the most wired, frenetic poll consumers stay sane as 2022 unfolds.

Rule 1: Current poll averages call most races correctly — but they don’t tell us much in close races.

Some good news about polls: Historical­ly, they’ve done a pretty good job of predicting* results. I used my data set to compute poll aggregates — that is, averages of the data designed to dampen the effect of outliers and get an accurate read of what the polls said each day — for almost 400 past Senate races.

I found that roughly nine to 10 weeks before Election Day, a simple poll aggregate correctly predicted the winner in 83 percent of races. By Election Day, that percentage had risen to 93 percent.

Some of the races in my data set were easy calls — pollsters typically don’t struggle to survey races correctly in deep-red Nebraska or dark-blue Massachuse­tts. But 83 percent is not too shabby. In numerous races that confounded pundits — such as the 2018 Senate races in Texas and North Dakota — the early poll leader went on to win.

Unfortunat­ely, when we look only at close races, the early polls are less predictive. In my data set, candidates who held a one-to-three-point lead on Labor Day won only 63 percent of the time. That’s an advantage, but it’s very thin.

Put simply, if you’re trying to understand the state of a Senate race, a poll average is a great place to start. But if your preferred candidate leads by only a couple of points, you’ll want to hold off on buying champagne.

Rule 2: The real margin of error is huge. It gets smaller as Election Day gets closer — but it never disappears.

When pollsters release their data, they dutifully report the “margin of error” — a statistic that attempts to estimate how much the randomness inherent to polling might throw off their results. This number is typically in the range of three to four points.

Unfortunat­ely, this number is a wild underestim­ate. Randomness is not the only source of error in polling — estimates of the likely electorate can be far off target; races can see late shifts in voter preference­s; and nonrespons­e bias (when one segment of the electorate refuses to talk to pollsters while another happily picks up the phone) is always a risk. All these factors can cause a poll to miss the outcome. At this point in the campaign, polls routinely miss the final result by six points.

As Election Day approaches, polls tend to get more accurate. Swing voters make up their minds, wayward partisans come home, and surveys pick up on those changes. If a polling average gives a candidate a lead in the mid-to-high single digits, they will be a strong favorite heading into Election Day.

But error never disappears. It’s completely normal for a poll average to be off by a couple of points on Election Day. And in close elections, that can make a huge difference.

Rule 3: No state or party is safe from an Election Day polling error.

A two-to-four-point miss on Election Day might seem like no big deal. But the parties are evenly matched — and small errors can create surprises and upsets.

The 2020 election is a perfect example. Heading into Election Day 2020, Democrats looked competitiv­e in many key races — including red states such as Iowa, North Carolina, South Carolina and even Kansas. But polling error got the best of them.

In race after race, polls underestim­ated GOP candidates. Democrats hoped for an electoral rout, complete with victories in red states. Instead, they got a bare 50-50 majority.

We don’t know whether the polls will err this year. And if they do, we don’t know which party they might underestim­ate. Those who confidentl­y predict polling error often end up famously wrong.

But the possibilit­y of Election Day polling error looms large — and that might keep political junkies feeling queasy, regardless of any other polling facts they know.

*Note that early polls don’t explicitly aim to predict results — they represent a snapshot of current public opinion. Regardless, analysts and news consumers routinely use them as a predictive tool.

About this data: Polls were collected from Fivethirty­eight, Realclearp­olitics, the Argo Journal, various pollster archives and news reports. On any given day, the polling aggregate is the average of the three most recent polls taken within the 30 previous days. If only one or two polls were released, that is used as the average. Partisan polls are excluded, as are polls taken more than 90 days before the election. In polls with multiple voter population­s (e.g., registered and likely voters), the likely-voter model was used. Polls of all adults were excluded. The data spans elections from 1992 to 2020, but not all races were polled. As Election Day gets closer, the number of races with adequate polling increases. Other aggregatio­n methods will produce different win rates and error estimates.

 ?? DAVID WILLIAMS/BLOOMBERG NEWS ?? Supporters of Wyoming congressio­nal candidate Harriet Hageman monitor results during a primary-night watch party in Cheyenne, Wyo., on Aug. 16.
DAVID WILLIAMS/BLOOMBERG NEWS Supporters of Wyoming congressio­nal candidate Harriet Hageman monitor results during a primary-night watch party in Cheyenne, Wyo., on Aug. 16.

Newspapers in English

Newspapers from United States