Bad Score Keeping

ARE SURVEY RESULTS BEING WITHHELD OR DISCREDITED BECAUSE THEY DON’T MATCH THE GOVERNMENT LINE OR IS THERE AN INHERENT PROBLEM IN THE WAY INDIA CONDUCTS ITS STATISTICS?

Access to data, even with limitations, is of immense importance to policy makers; unavailability makes them handicapped

On November 15, the Ministry of Statistics and Programme Implementation (MoSPI) announced that it will not publish the results of a national household consumer expenditure survey carried out by its wing, the National Statistics Office (NSO), during the period July 2017 to June 2018 on account of “data quality issues”. The MoSPI decision will impact the government’s GDP base year revision plans as the scrapped data set is crucial in revising the base year for calculating India’s economic (GDP) growth. The absence of this data will also deny India an opportunity to understand the changes that have happened in areas of poverty, food consumption, nutrition, malnutrition and a host of other issues during the last six years, as that was when a similar exercise was carried out last to generate comparable data points. This is also the second instance, in recent times, of the results of a survey-based data analysis, prepared by an official government agency, being junked by the government itself. Results of the periodic labour force survey (PLFS) of NSO, again for the 2017/18 period, had met a similar fate. While the PLFS survey did get released after a long delay, it was discredited by the government, which said its findings related to job losses were not correct. These may seem like isolated events but a possible dent

to the credibility of India’s official data can have wider implications for the way other countries and potential investment, technology and trade partners see India. BT reached out to Pravin Srivastava, Chief Statistician of India and Secretary, MoSPI, for an official response, but could not elicit any. At the time of going to print, a response from Bimal Kumar Roy, Chairman, National Statistical Commission, was awaited.

NSO routinely processes two types of data sets – generated through its own surveys and compiled from other sources. Agriculture data, for instance, comes from the agriculture ministry, while the MCA 21 database of the Ministry of Corporate Affairs is the source of company data. Then there are high-frequency data sets like the Index of Industrial Production

(IIP), which is based on fixed items from a fixed panel of factories. IIP reflects short-term changes in the manufacturing sector and is released monthly with a time lag of six weeks. The Consumer Price Index (CPI), which NSO compiles separately for rural, urban and combined sectors on a monthly basis, is another high-frequency data set. All of these have problems in terms of weightage given to certain items (thereby skewing the outcome) or the relevance of the product basket. Efforts are on to revise the base and methodology of IIP, for example. So, while there are questions regarding the data, they are shared in a timely manner. This is not the case with survey-based reports.

The government’s decision to withhold the consumer ex

penditure survey has raised two key questions. One, is there a problem with India’s official statistical system and is it time for an overhaul to make it more credible and reliable? And two, if India’s statistical system and its practices are by and large credible, then is the government playing politics by not releasing the data which perhaps runs contrary to its development claims? If so, is there a threat of unfavourable data getting suppressed in future thereby putting the credibility of India’s official statistical system at risk? Both the questions are valid, to some extent.

Need For Improvement The theme of the 26th annual Conference of the Central and State Statistical Organisations (COCSSO) held in November 2018 was “Quality Assurance in Official Statistics” as the issue of quality assurance in statistics produced and disseminated by various official agencies at the Centre and in the states has had MoSPI’ attention for quite some time The conference made an attempt to understand the extent to which India followed the United Nation’s Statistical Commission’s template. The result was that of a total of 250 global quality benchmarks, India’s statistical system was fully compliant (where respondents said ‘yes’) in

108 parameters. It was work in progress (‘partially true’) in

123 parameters, while a non-starter in 18. Assuring quality commitment, adequacy of resources and relevance, and managing metadata were the areas where the Indian statistical system needed to improve a lot.

The COCSSO said the average implementation percentage comes out to be 68.1 per cent and that government agencies need to work towards bringing this to 100 per cent.

Even in the case of the now discredited consumption expenditure survey, there were issues with data quality, and the ministry’s worries are not completely out of sync. MoSPI defended its decision to scrap the survey: “the results of the survey were examined and it was noted that there was a significant increase in the divergence in not only the levels in the consumption pattern but also the direction of the change when compared to the other administrative data sources like the actual production of goods and services”. It also said that concerns were raised about the ability or sensitivity of the survey instrument to capture consumption of social services by households, especially in health and education, thereby prompting the ministry to jettison the survey.

“Consumer expenditure is a very difficult survey to be done because you are supposed to collect the entire expenditure of the household on consuming goods or services during the month or year. There is always a problem of recall lapse. You may recall yesterday’s expenditure, but you may not recall the monthly expenditure,” says P.C. Mohanan, former Chairman of National Statistical Commission (NSC). “The second issue is that a lot of consumption is difficult for households to record. You get a lot of free services and products like subsidised ration, so the consumption expenditure might be lower because they are not spending money on those products. So even if you are consuming the same thing, your expenditure will be reduced.” These issues, however, are not new, and are present not only in India, but everywhere, he adds.

Inconvenient Data

Over 200 academicians and prominent civil society members, including Nobel prize winner Angus Deaton and French economist Thomas Piketty, have prepared an open letter to the Narendra Modi government, demanding that the govern

ment release the report and data of all NSO surveys that have been completed and approved by the NSO’s internal systems, including the results of the 75th round of Survey of Consumer Expenditure, 2017/18. The open letter states that consumption surveys are crucial for monitoring trends in poverty and inequality, for national income accounting, and for updating macro-economic data such as price indices. “They can provide a check on administrative and macroeconomic data, which is important both for policy makers and the general public. The fact that data on supply of goods and household consumption are diverging, points to the need to question supply side data (which are being widely questioned within and outside India) as much as it points to the continuing need for improving survey methods. It is of fundamental importance for the nation that statistical institutions are kept independent of political interference, and are allowed to release all data independently. The record of the present government on this score has been very poor,” the letter states.

Academicians allege in the letter that the government has chosen to attack the credibility of the statistical institution simply because their results do not match with its own narrative on the economy, “without providing adequate reasons, and by misrepresenting essential features of the surveys”.

“It (government) has repeatedly shown its disinclination to make public any information that may show its own performance in poor light. Last year, before the parliamentary elections, the results of the Periodic Labour Force Survey were not allowed to be released until the Parliamentary Elections were over, despite the resignation of two members of the National Statistical Commission, and a leak in the media. Subsequently, results of other surveys including the 75th round (Consumer Expenditure), 76th round (Drinking Water, Sanitation, Hygiene, and Housing Conditions) and more recent data of the PLFS surveys, have not been released”, the letter states.

Pronab Sen, former Chief Statistician of India, agrees that the concerns regarding data suppression are not unfounded. “The consumer expenditure data has just shown that it was an unusual year (year of demonetisation, a major economic disruption).

Open For Discussion Controversies associated with NSO data and methodology are not new. When the MCA 21 database was used as an input to measure corporate performance for the first time, the move was questioned. But the results were in the public domain for discussion.

It must also be noted that the two surveys that the government is being accused of suppressing (on consumers and labour) are for the period when demonetisation happened, and the data is likely to show the effect. If that was indeed the reason for discrediting the surveys, it sets a bad precedent. The drinking water and hygiene survey could well be a report card for the Swachh Bharat mission as some of the questions asked to respondents were about the use of toilets, etc. With the government claiming near complete success in bringing about a behavioural shift in the minds of people regarding toilet use, any conclusion to the contrary could be embarrassing.

NSO surveys are cyclical, and get repeated in five or six years. Access to data, even with limitations, is of immense importance to policy makers; unavailability makes them handicapped. Apart from that, when academicians say that “undermining the objectivity and credibility of an independent statistical system is fundamentally against the national interest”, they have a point. Any move to improve the accuracy of data collection and analysis should not come at the cost of transparency and accountability. One can only hope that the survey results in the future are released without delay, thereby proving the controversy a non-issue. That does not mean the data is wrong,” he says. “The PLFS survey (2017/18) data was not released until it got leaked. The consumer expenditure data also got leaked, but instead of releasing it, they have suppressed it. The PLFS data takes very little time to process. It is collected on a hand held machine and is uploaded real time. It (PLFS data for 2018/19) should have been ready by July 2019, but we haven't heard about it. There is data suppression going on,” he says.

“( Periodic labour force survey data for 2018/19) should have been ready by July 2019, but we haven't heard about it. There is data suppression going on” Pronab Sen

Former Chief Statistician of India “Consumer expenditure is a very difficult survey to be done... . There is always a problem of recall lapse. A lot of consumption is difficult for households to record” P.C. Mohanan

Former Chairman, National Statistical Commission