Don’t lament end of longform census.

StatsCan’s National Household Survey is just the long-form census made voluntary

The 2011 National Household Survey (NHS) is the voluntary survey that replaced the longform census. More accurately, it’s the long-from census transformed into a voluntary survey. It’s also the only government-mandated change in a statistical program to precipitate the resignation of the Chief Statistician. A ferocious controversy is still raging more than four years later. To many, the replacement of the long-form by the NHS has become a symbol of the Conservative government’s rejection of evidence-based policy making.

A bill recently introduced by Liberal MP Ted Hsu to reinstate the mandatory long-form questionnaire was hotly debated in the House before being voted down by government members. Much ink has been spilled writing opeds and editorials denouncing the NHS. One from the Martin Prosperity Institute alarmingly claimed that “in scrapping the longform census, the Harper administration has threatened the country’s long-term economic prosperity.”

Is all this sound and fury justified? Like most economists and commentators, I was against to the replacement of the long-form census. And I had fairly low expectations when the NHS was released.

However, upon examining the results of the survey and supporting documentation, I was pleasantly surprised to find basically all the same data as collected in the long-form.

But is the data usable? Statistics Canada reassuringly characterizes its quality at the national, provincial, territorial and census metropolitan area levels as high.

This means that Statistics Canada is putting its prestige as one of the best statistical agencies in the world behind the data — a statistical Good Housekeeping Seal of approval so to speak.

The main concern of most analysts is that the integrity of the NHS data would be compromised by a low response rate (predicted to fall to as low as 50 per cent by at least one statistician at the time the NHS was announced).

Yet perhaps out of a sense of civic duty Canadians have continued to respond at acceptable levels without the threat of penalties. And a very sophisticated statistical methodology was utilized for sampling and weighting and reliability checking to ensure that the sample was representative of the population. This is an approach that Statistics Canada pioneered and uses with great success in all its voluntary surveys to produce reliable information.

Granted that the NHS data is not suitable for all the purposes for which longform data was used. Statistics Canada cautions that in some smaller areas and for some smaller populations, the re- sponse rate may be insufficient to provide a valid statistical picture. In these cases, users have to use the data at higher geographic levels or for larger subgroups of the population.

Most economists and statisticians are preoccupied with data quality and not so worried about response burden. Mandatory surveys backed by legal sanctions and penalties certainly have higher response rates than voluntary surveys. But politicians are understandably reluctant to imprison people who refuse to respond to surveys, which they consider overly intrusive. That is probably why there are so few mandatory surveys of households in Canada.

Other than the Census of Population (eight pages of questions), only the Labour Force Survey (21 pages) is still mandatory. And these two surveys are much shorter than the 40-page NHS.

Around the world concern about compliance and collection costs is resulting in the use of administrative data for statistical purposes instead of more costly surveys. These efforts are most advanced in Europe. There is also concern about the timeliness of census data which is only collected every five or 10 years and is only available after a one or two-year processing lag. This is why the United States has implemented an annual mandatory American Community Survey (ACS) to replace its census long-form. With a sample of currently around 3.5 million housing units, it is significantly smaller than the 4.5 million in the NHS, both in absolute terms and especially as a percentage of the population.

The NHS is far from the unmitigated statistical disaster portrayed by many. Plans are already well advanced for the 2016 NHS. The experience gained with the 2011 survey and the subsequent data analysis should enable Statistics Canada to improve the NHS even more. So does it really make sense to reverse course and reinstate the long-form census as advocated by some? This would just transform the existing voluntary NHS back into a mandatory survey, not replace it with something totally different and possibly more appropriate. Moreover, it would be difficult, if not impossible, to get as high a response rate for the so-called reinstated long-form as for the old version now that the long-running controversy has publicized the fact that the penalties under the Statistics Act are never imposed. Reinstating the long-form is no panacea.

The NHS is far from the unmitigated statistical disaster portrayed by many