How to read political polls and stay sane
One good rule is to trust findings that are replicated by different organizations.
WASHINGTON — Midterm elections are nearing, the outcome remains close, and many readers are eagerly — perhaps obsessively — watching the latest polls, trying to ascertain whether their side will prevail.
Can Democrat John Fetterman capture a Senate seat in Pennsylvania by continually ridiculing his Republican opponent, Dr. Mehmet Oz? (So far, seems like it.) Can Republican Blake Masters overcome an extremely slow start and unseat Democratic Sen. Mark Kelly in Arizona? (No sign of it so far.) Can incumbent Democratic Sens. Catherine Cortez Masto and Raphael Warnock hold their seats in, respectively, Nevada and Georgia? (Control of the Senate probably turns on the answer, and anyone who pretends to know is lying.)
With just a little more than seven weeks left in the fall campaigns, flocks of surveys on those and other close races have started to fly into view, touted in fundraising emails, bannered in headlines and clamoring for the attention of voters — many of whom say they mistrust polls even as they can’t stop looking at them.
Polls provide a vital tool for understanding America, but a limited one. Here’s a guide to following them while keeping your equilibrium between now and election day, Nov. 8.
Polls can tell us a lot. Because of polls, for example, we know that the Supreme Court’s June decision that overturned Roe vs. Wade and ended the nationwide guarantee of abortion rights caused a significant shift in public opinion, said Natalie Jackson, director of research at the Public Religion Research Institute, which surveys Americans extensively on their cultural values.
Polls by numerous organizations have shown that a large majority of Americans disapproved of the court’s decision; that respect for the court as an institution has dropped, especially among Democrats; that abortion has risen as an issue of importance to voters; and that a significant number of voters have turned against the Republican Party in the aftermath.
Without polls, in a country where neighborhoods, occupations, churches and other activities are increasingly separated along lines of ideology and partisanship, Americans would know how they and their friends reacted to the decision, but would have nothing but occasional anecdotes to gauge what people outside their social circles think.
“In the absence of polls, we’d just have public officials claiming whatever they want,” said Scott Keeter, director of survey research at the Pew Research Center.
But polls aren’t as comprehensive or clear as people might like.
“Polls are fuzzy estimates,” Jackson said. “They’re not meant to be super precise — they can’t be because they’re samples.”
One good rule of thumb is to trust findings that are replicated by different organizations, rather than overreacting to whatever the latest poll says.
Take President Biden’s job approval, for example. Lots of polls track it. At any given time, one or another poll might have Biden up or down. Collectively, however, the polls show a clear pattern: Approval of Biden hit a low point in July and has improved significantly since.
The polling average maintained by the Five-Thirty-Eight site had Biden’s approval at 42% on Thursday, compared with 53% disapproving — not great, but about the level of approval that Presidents Reagan, Clinton and Obama had at this point in their first terms.
Averages have pluses and minuses — anyone putting together an average has to make decisions about which polls to include, for example. But their great strength is that they pool information from lots of different surveys conducted lots of different ways.
“What I pick up, someone else might miss. What I miss, someone else might pick up because we’re all using different methodologies,” said pollster David Paleologos, director of the Suffolk University Political Research Center in Boston, which conducts surveys around the country for a large number of media clients, including USA Today and the Boston Globe.
“Understanding the limits” of what any one poll can provide “is a good place to start,” Paleologos said.
Take, for example, Ohio’s closely fought Senate race. Suffolk’s most recent survey, released Tuesday, showed Democratic Rep. Tim Ryan with 47% support and Republican J.D. Vance with 46%.
That’s not a lead for Ryan; it’s a toss-up, given the poll’s margin of error of about 4.5 percentage points in either direction. But it’s consistent with other polls, which, on average, show Ryan just slightly ahead of Vance.
So it’s safe to say that the Ohio race is very close and that Ryan has a shot at winning a seat that Republicans have held since 1998. If his edge holds through election day, it would be a huge loss for Republicans, almost guaranteeing a Democratic majority in the Senate.
But will the Democratic edge hold up?
It could. Obama carried Ohio twice, and Democratic Sen. Sherrod Brown has won three statewide elections. But there are reasons to be skeptical.
Beyond Obama and Brown, Ohio has mostly gone for Republicans over the last generation. It’s important to think about polls in the context of everything else known about a race. Ohio’s record of GOP victories suggests a decent likelihood that people who are uncertain about voting could join ranks behind Vance by election day.
Sure enough, in Suffolk’s poll, Ryan has a bigger lead among people most certain to vote. Vance is ahead among people who are less certain to cast ballots, Paleologos said. So Vance has room to improve if he can motivate voters, and thanks to a huge infusion of money from Senate Majority Leader Mitch McConnell’s super PAC, he’ll have the resources to try.
Who will turn out to vote is one of the biggest question marks pollsters deal with: They not only have to come up with a sample that accurately reflects the population, but they also have to correctly forecast which part of that sample will cast a ballot. That’s why election polls have more uncertainty built into them than issue surveys.
The Ohio polls highlight another point: Democrats often think their side will always do better when turnout rises, but that’s not true. Increasingly, Democrats do best among college-educated voters, a group that votes pretty reliably.
Republicans in the Donald Trump era rely heavily on blue-collar white voters and have improved their position somewhat among blue-collar Latino voters, both groups that tend to have spottier turnout records.
College-educated voters are also the most likely to tell pollsters that they’re motivated by concerns about abortion. In special elections this summer, turnout among those voters was unusually high. That might continue into November — big upturns in voter registration by women, especially younger women, in several states could be an indicator of greater enthusiasm to vote — but that remains uncertain.
A final source of uncertainty for surveys: Polls in 2016 and 2020 tended to underestimate the Republican vote, especially in Midwestern states with lots of blue-collar white voters — places such as Ohio and Wisconsin, where, in another tight race, Democrats hope to unseat Sen. Ron Johnson but Republicans are strenuously attacking his opponent, Lt. Gov. Mandela Barnes, as weak on crime.
Pollsters have spent a lot of time studying the 2016 and 2020 results to try to figure out who they missed and why. The most likely problem is a significant number of Trump-supporting Republicans who refuse to respond to surveys, said Keeter, who has worked with Pew’s researchers to improve techniques for getting fully representative samples.
No one knows if that “nonresponse bias,” as pollsters refer to it, is something that happens only when Trump is on the ballot — polls in the 2018 midterm elections were more accurate — or whether it’s affecting polls this year, as well. And while pollsters have tried a variety of ways to improve their samples, there’s no way to know in advance if they made the right adjustments.
So, what’s the bottom line? Pay more attention to averages; don’t over-focus on any individual survey. Use some common sense. And don’t expect polls to deliver more precision than they can accomplish.
Polls have a good track record of being able to “predict the outcome within a few percentage points,” Keeter said. That’s a good thing, but “elections are often decided by less than a few percentage points.”