Pollsters try to coax out ‘Shy Trump’ voters
These reluctant, educated Republicans could impair tallies in battleground states — again
Meetings of the American Association of Public Opinion Research tend to be pretty staid affairs. But when members of the group gathered for a conference call at this time in 2016, the polling industry was experiencing a crisis of confidence.
Donald Trump had swept most of the Midwest to win a majority in the Electoral College, a shocking upset that defied most state-bystate polls and prognoses. An association task force, which was already working on a routine report about pre-election poll methodologies, was suddenly tasked with figuring out what had gone wrong.
“We moved from doing this sort of niche industry report to almost like more of an autopsy,” said Courtney Kennedy, the director of survey research at Pew Research Center, who headed the task force. “Something major just happened, and we have to really understand from A to Z why it happened.”
The group released its report the following spring. Today, with the next presidential election less than a year away, pollsters are closely studying the findings of that document and others like it, looking for adjustments they can make in 2020 to avoid the misfires of 2016.
“Polling is one of those things like military battles: You always refight the last war,” said Joshua D. Clinton, who co-directs the University of Vanderbilt’s poll and served on the AAPOR committee. The 2020 election “might have a different set of considerations,” he said, but pollsters have an obligation to learn from the last cycle’s mistakes.
By and large, nationwide surveys were relatively accurate in predicting the popular vote, which Hillary Clinton won by 2 percentage points. But in crucial parts of the country, especially in the Midwest, individual state polls persistently underestimated Trump’s support. And election forecasters used those polls in Electoral College projections that gave the impression Clinton was a heavy favorite.
AAPOR’s analysis found several reasons the state polls missed the mark. Certain groups were underrepresented in poll after poll, particularly less educated white voters and those in counties that had voted decisively against President Barack Obama in 2012. Respondents’ unwillingness to speak honestly about their support for Trump may have also been a factor.
These and other issues could reappear in 2020, pollsters warn, if they’re not addressed directly.
To make sure their results reflect the true makeup of the population, pollsters typically “weight” their data, adding emphasis to certain respondents so that a group that was underrepresented in the random sample still has enough influence over the poll’s final result. Polls typically weight by age, race and other demographic categories.
But some state-level polls in 2016 did not weight by education levels, therefore giving short shrift to less educated voters, who tend to be harder to reach.
This often understated Trump’s support, since he was markedly more popular than past Republican nominees among less educated voters — and noticeably less popular among those with higher degrees, who research suggests are more likely to participate in polls.
The AAPOR analysts found that many polls in swing states would have achieved significantly different results had they been weighted for education. This, in turn, would have noticeably decreased Clinton’s lead in much-watched polling averages and forecasts of these states.
A pre-election study by Morning Consult warned that wealthier, more educated Republicans appeared slightly more reluctant to tell phone interviewers that they supported Trump, compared with similar voters who responded to online polls.
Pollsters refer to this phenomenon as the “shy Trump” effect. Studies have affirmed that in races where a candidate or cause is perceived as controversial or otherwise undesirable, voters can be wary of voicing their support, especially to a live interviewer.
One polling firm that showed Trump narrowly leading in some of the most inaccurately polled states — Michigan, Pennsylvania and Florida, all of which he won — was Trafalgar Group, a Republican polling and consulting firm that uses a variety of nontraditional polling methodologies.
It sought to combat the shy Trump effect by asking respondents not only how they planned to vote but also how they thought their neighbors would vote — possibly offering Trump supporters a way to project their feelings onto someone else.
The AAPOR report posited that the neighbor question could help overcome shyness among Trump supporters, particularly in phone interviews. It “warrants experimentation in a broad array of contests,” the report said.
That was not the only way Trafalgar innovated. Polls typically use a formula based on past elections to determine which voters are likely to show up on Election Day. They then discard or devalue responses from those who seem less predisposed — typically those without much history of voting, or who don’t express much enthusiasm about politics.
Trafalgar used a generously inclusive model, with a particular eye toward less frequent voters whom Trump’s anti-establishment campaign had drawn in.
“With Trump, we saw in the primary how new people were being brought into the process, and so we widened the net of who we reached out to,” Robert C. Cahaly, a pollster at Trafalgar, said in an interview.
When the Census Bureau in 2017 released detailed voting information from the 2016 election, it revealed that turnout had surged in many counties that Obama had lost by 10 points or more in 2012 — particularly in Michigan, Pennsylvania and Wisconsin. It is a reminder that who voted in the previous election is not always a good indicator of who will vote the next time.
Compounding all the other factors in 2016 was the simple fact that — in a race with two historically unpopular candidates — many voters didn’t reach a decision until just before Election Day.
In Michigan, Pennsylvania and Wisconsin, between 13 percent and 15 percent of respondents in exit polls said they had decided in the last week of the campaign. Those voters broke for Trump by a wide margin; in Wisconsin, it was about 30 points.
Pew researchers also called back respondents of their preelection polls and found that many had changed their minds and voted differently than they’d said they would, which is not uncommon. But these voters broke for Trump by a 16-point margin — a heavier tilt than in any other year on record.
So, in a volatile election, even a perfectly effective poll might not be able to gauge the outcome.
“I’m not sure people understand how these probabilistic projections are produced or what they mean,” Gary Langer, a pollster who works with ABC News, said in an email. “I’d suggest that predicting election outcomes is the least important contribution of pre-election polls. Bringing us to a better understanding of how and why the nation comes to these choices is the higher value that good-quality polls provide.”