Fear over ‘stale’ data in census

The continued threat of incomplete data has Statistics New Zealand again pushing the deadline for Census 2018 results.

Documents released under the Official Information Act show the department is planning for an August, 2019, deadline, and continues to stare down a high risk it may not produce viable results.

A low turnout during Census 2018 has caused considerable pain for Stats NZ, which last week announced it would not make the third deadline it set – July 2019 – and promised to announce a release date in April.

Experts are concerned the repeatedly slipping deadlines will have a detrimental impact on planning and funding decisions to come in 2019, and that the data might already be ‘‘stale’’.

Stats NZ failed to count an estimated one in 10 New Zealanders in Census 2018, previously recording an interim response rate of at least 90 per cent of individuals providing full or partial information.

It was announced in June that other sources of government data would be used to fill gaps in the results.

Both the actual number of responses received and the number of additional data for 2018 were redacted from documents, and Stats NZ declined to release them.

The documents show the ‘‘high risks’’ Stats NZ continues to face include: the failure of methods to patch census data; census data failing to be fit for purpose, leading to ‘‘less than ideal’’ decisions being made; and a failure to provide information for the re-drawing of electorate boundaries.

Also described are a ‘‘severe incident’’, where some data fields on census forms failed to cross over to another system and contributed to a month-long setback.

There was also a loss of about 200 records, requiring some households to be reinterviewed, and the data collated was not as clean as expected.

More than six months after switching off the online census, the department continues to test two methods of data imputation using additional government figures.

‘‘Stats NZ will use imputation methods to add to the census counts where we have sufficient evidence that a person exists but we have no administrative record or census response for them,’’ a spokesman said yesterday.

To manage the unexpected work, Stats NZ has hired an additional 11 fixed-term staff.

Documents also show that

contractors have stayed on longer than anticipated, and some are working overtime.

BERL chief economist Dr Ganesh Nana said that Stats NZ needed to ‘‘draw a line’’ under Census 2018.

‘‘We all might have to put our hands together and say, ‘OK, the data won’t be as rigorous and robust as previous census’. We might just have to admit that.’’

A standard census release would have BERL using the data intensively over a six-month period from about now, he said.

‘‘That’s clearly been put out of kilter this time around. The biggest concern for us is the timing of the data . . . the data is already starting to become stale.’’

The data was vital to decisionmaking in local governments, particularly in the regions where little other population data was available.

‘‘One of the lowest reported response rates was from Ma¯ ori, which does send a lot of warning signals in terms of the robustness and usefulness of the data which does come out.’’

National Party statistics spokesman Dr Jian Yang said Census 2018 had become a ‘‘shambles’’, and had likely missed the deadline for being used in planning Budget 2019.

‘‘The census is a fundamental basis for funding, DHBs, schools, transport. All the funding for these departments and agencies is based on census.

‘‘It means that next year the Budget will be based on previous data. If it’s further delayed, Budget 2020 could be affected.’’

Yang said the five-year budget of $120 million for Census 2018, granted by the previous Government, was adequate. Stats NZ had poorly chosen how to apply it.