‘Perfect storm’ behind inaccurate Calgary civic election polling, firm says

The polling firm that badly misjudged the outcome of Calgary’s October civic election is blaming a “perfect storm” of factors for its insistence that Mayor Naheed Nenshi was destined for defeat.

Toronto-based Mainstreet Research was commissioned by Postmedia, which owns the Calgary Herald and Calgary Sun, to conduct three horse-race polls ahead of the Oct. 16 vote. In each poll, the pollster found Nenshi trailing his chief rival, businessman Bill Smith, by wide margins, at one point by as much as 17 per cent.

However, on election night Nenshi was re-elected for a third term, earning 51.4 per cent of the vote, compared with Smith’s 43.7 per cent.

After its failure to properly forecast the election results, Mainstreet launched a review of its practices led by the pollster’s VP of analytics, Joseph Angolano, which was further vetted by a panel of independent polling experts.

Mainstreet president Quito Maggi said the review will ultimately improve the company’s future public opinion surveys.

“This has been a humbling experience,” Maggi said in a statement.

“But the results of this review will most definitely make us better. Dr. Angolano’s analysis along with the advice of other experts, has been eye-opening, and a valuable lesson for Mainstreet.”

While the review found no evidence the polling errors were intentional or malicious, it determined the use of IVR (interactive voice response) to conduct the surveys led to a series of miscalculations that resulted in a near 25-per-cent deviation between the company’s last pre-election poll and the actual outcome.

Angolano’s review determined several factors were central to the pollster’s Calgary stumble, including a failure to poll in non-official languages in a city where immigrants make up nearly a third of its population, a tendency among Nenshi voters not to respond to Mainstreet polls, and Calgary’s primarily youthful population.

“The ratio of young people in Calgary, and a few other western urban centres is unlike anything we see in other parts of the country,” Maggi said, noting the socalled “Calgary effect” was also seen in the 2010 election in which Nenshi was first elected.

“We missed a lot of these young voters because they are harder to reach. Even those we did reach had significant response bias as revealed by the report. Like the election in Calgary in 2010, the increase in youth turnout produced a result not remotely close to preelection polling.”

The review suggested a number of recommendations, including the use of random digit dialing, longer questionnaires, polls conducted over multiple days and times and the release of raw data among others.

The use of IVR and its shortcomings is also being looked at.

Those recommendations will be implemented in a new poll ahead of Thursday’s provincial byelection in Calgary-Lougheed, to be released Tuesday, the firm said.

Maggi added Mainstreet also learned lessons on how it publicly dealt with criticism of its polls during the Calgary civic campaign. While local experts and pundits raised red flags about some questionable underlying numbers in Mainstreet’s polling, the pollster staunchly, and sometimes fiercely, defended its findings.

Postmedia put its relationship with Mainstreet on hold in light of its polling failure and plans to reassess any future co-operation with the pollster in the new year.

“We’ll be meeting with Mainstreet in January to review the results of its internal investigation,” said Gerry Nott, the newspaper chain’s senior vice-president of content. “We have no further comment at this time.”