DIGESTING A POTTAGE OF POLLING RESULTS
Like grief (OK, not entirely), there appear to be five stages to the way journalists, political junkies and various interested folks consume political polls.
Stage 1 is Keen Top-line Craving, in which survey readers gluttonously gobble data tables, mumbling, mouths full, “gimme the numbers, the sweet, sweet horse race numbers!” In this stage, the primary concern involves devouring knowledge of who’s up, who’s down, and who might win an election. It’s time to regurgitate in Stage 2: Amplification without Context. This is where poll readers share on social media the single polling data point with which they profoundly agree (ooooh!) — or vehemently disagree (oh no!) — with corresponding glee or outrage. Indigestion is the primary sensation that accompanies Stage 3: Comparison Confusion, wherein the realization that (gasp!) different pollsters may have different results contributes to feelings of bewilderment, dizziness, hostility and a general sense of unease.
No one wishes to linger in the gastric pain that is Stage 3, so like a thick, soothing, pink spoonful of bismuth subsalicylate, it’s on to Stage 4: Pseudo Expertise. The pollsters don’t all say the same thing because of malfeasance! Grand conspiracy! General mischief! String them up! Hang them high!
Finally, Stage 5: Ennui. Drained from the excitement, outrage and heartburn, poll consumers ask, “What do political polls matter in the middle of summer anyway?”
Matter or not, summer or not, there will always be an impulse to measure and understand what society is thinking. There may be squabbling over what scientific polls say. But without them, we are left with more ham-handed attempts, such as those truly dyspeptic hamburger polls at one’s local diner. This is an election year. More polls are coming. Surely there’s a better way to enjoy the smorgasbord of data that will be steadily streamed in front of you over the next 12 weeks? Well, there is. Try some of these hacks.
Start with running a diagnostic check on the poll you’re interpreting. Consider the “sample size” (the number of people who were asked the questions). More isn’t necessarily better for certain findings, but for others, the greater the number of respondents, the more deeply and finely one can dice the data, and understand what different demographics are trying to tell us. What questions were asked? How were they worded? In what order were they asked? These factors make a difference to the answers respondents give. Did the pollster publish the whole questionnaire? On what dates were the questions fielded? Who paid for the survey?
Then there’s methodology.
I’ve watched non-practising, so-called polling experts insist that only one form of data collection meets a gold standard. Maybe that was true in the 1980s, but polling methodology has been evolving from the advent of the survey: from face-to-face interviews in the living rooms of housewives to landline phone calls, to online panels, to surveying via responsive text chat and instant messenger.
Each method has its strengths. Each has its limitations. Buy me dinner; I’ll spend three hours walking you through it all in excruciating detail. But in the absence of this, don’t discount, or accept, any one methodology as inherently better or worse than another. A polling organization’s track record will probably tell you more.
I also invite you to dig deeper, beyond the top-line findings. In a country increasingly fractured along regional, income and generational lines, policy issues represent different things to different segments of the electorate.
For example, we spend a lot of time talking about “the economy” in this country. It’s not “the” economy, however, it’s “whose” economy. For younger people living in cities, economic conversations centre on the cost of living, transit, affordability. For baby boomers transitioning towards estate planning, those same conversations focus on taxation levels and government spending.
Mood, past behaviour, values and world views of voters go a long way to helping us understand the dynamics of a campaign. If I am biased about one thing, it’s the value that these elements bring to survey data. If you focus on more than the horse race results, you, too, can taste the learnings from polls with true gourmandize.