Let the market for data collection bloom; tap private sector capabilities
In recent weeks, some scholars, including respected voices such as Jean Dreze, have criticised the Centre for Monitoring Indian Economy (CMIE)’s Consumer Pyramids Household Survey (CPHS) for its alleged inadequate inclusion of poor households. In their view, the CPHS survey execution inadequately covers households who inhabit the outskirts of villages and since the poor reside there, the sample under-represents the poor. We have explained elsewhere that this is not possible and so there is no case for under-representation of the poor.
Nevertheless, to ensure that we do not err, and to dispel all doubts, we have decided to investigate the matter and make corrections, if warranted, by April 2022.
The issue has also led to a larger question, raised in an editorial in this newspaper — on whether the private sector has sufficient incentives to collect adequate data about the poor. We believe, and it is our experience, that there is a market for honest data on the poor like there is a market for the rest.
All our current revenues from CPHS work come from the academic world where interest in the poor is particularly high. Given that CPHS is funded entirely out of subscriptions to the database by researchers who want to study it, CMIE will be seriously hurting its business if it ignores the poor. This business interest, at the very least, will drive CMIE to take academic critics seriously. And CMIE cannot ignore non-academic markets either.
As poverty reduces, the investment required to find the poor in adequate numbers in a sample survey increases. The sample needs to get larger and the execution needs to go deeper. In this respect, it helps that the CPHS sample is at least 60% higher than the official sample. Economic shocks don’t announce their arrival. Only a continuous survey can measure their impact when they happen. It helps that CPHS is a continuous survey. The government has been unable to expand its sample size for decades. And by design, it conducts its surveys only in “normal” years.
It is not necessary anymore to assume that only the government can provide data on unemployment, consumption expenditure or even poverty and inequality. CMIE demonstrates that at least capabilities and investments are not a problem. The markets are solving the problems that the government seems unable to.
But we need to retain the public goods nature of some data. An estimate of the unemployment rate of India, for example, cannot be the private property of CMIE. It just does not seem right. CMIE, therefore, makes this available freely as a public good with no restrictions, no fees or strings attached, except the courtesy of an acknowledgement of the source. CMIE continues to remain responsible for the underlying data and the computations.
If the private sector produces public goods, what is the role of the government? There is lots of data to be collected yet, and the government cannot do it all. It is best for the government to acknowledge available capabilities in the private sector and divide the job of data collection with it. If the privately produced data is of acceptable quality, then the government can use it.
It would be an interesting world with competing estimates of unemployment, inflation or even the Gross Domestic Product, for example. That would be better than a monopoly that is unable to produce useful, timely data except to somehow satisfy the elementary needs of the International Monetary Fund’s Special Data Dissemination Standard.
A public-private partnership, where the government issues tenders to invite private sector to collect data on its behalf, is a bad idea for all. It limits the potential of the Indian Statistical Service cadre. And, the private sector would not be able to innovate under such arrangements except in optimising operations for profit maximisation.
But, society needs to tap into private sector innovations far beyond such profit maximisation. Let’s unshackle the statistical system from being a government monopoly and still retain the character of some data being public goods. Let that market for data collection bloom and the government, like others, can go shopping to buy some of the data it needs. There is no reason to assume market failure here.