Evaluating the Aadhaar-PDS link
Both its supporters and critics have valid points: Leakage did go down, but exclusion also went up
Whether you believe its proponents or its critics, Aadhaar may be one of the most transformative investments in State capacity ever made. It now touches the life of nearly every Indian and is increasingly necessary to obtain social welfare benefits, something the Supreme Court recently permitted. The main policy rationale for requiring Aadhaar to access benefits is to reduce leakage and corruption, and advocates have claimed large fiscal savings from doing so. Critics worry that this requirement will lead to an increase in exclusion errors, denying genuine beneficiaries their entitlements. Yet, there is very little systematic evidence on the impact of introducing Aadhaar into welfare programmes.
A leading use case for Aadhaar has been to reduce leakage in India’s largest welfare programme, the public distribution system (PDS). The PDS has historically suffered from high rates of leakage (often exceeding 50%) as grains are diverted to the open market. Starting in 2015, the government began rolling out electronic Point-of-Sale (ePOS) devices at fairprice ration shops (FPS), and requiring that beneficiaries use the Aadhaar-based biometric authentication (ABBA) to collect their benefits. Subsequently, states began using the historical authenticated transaction data from ePOS devices to determine (or “reconcile”) monthly grain disbursals.
We evaluated the impact of these reforms in a recently released study, using a large-scale randomised-controlled trial or RCT (the research methodology recognised by the most recent Nobel Prize in Economics) conducted across 132 blocks in 10 districts in Jharkhand during 2016-17. We studied impacts using nearly 16,000 original household surveys conducted across these districts and matched with administrative data on disbursals.
We find that ABBA by itself (without reconciliation) did not significantly change either leakage or the value of PDS goods received by households on average. However, for the 23% of beneficiaries who had not linked an Aadhaar to their ration cards at baseline, ABBA reduced the value of benefits received by 10% and increased the fraction of beneficiaries receiving no benefits at all by 2.8%. ABBA also increased transaction costs for the average beneficiary by 17% (~7 on a base of ~41), driven in part by more unsuccessful trips to ration shops.
Turning to reconciliation, this reform coincided with a considerable reduction in the value of grain disbursed, received by households, and in the difference (i.e. leakage). In the control group, the value of grain disbursed fell by 18%; of this drop, 4% represented reduced value received by beneficiaries while the remaining 14% represented reduced leakage. In the treatment group, the declines were larger, reflecting the fact that dealers in this group had ABBA-based electronic records for a longer period and were expected to have larger stocks of undisbursed grains. Here, value disbursed fell 36% of which 12% represented a drop in value received by beneficiaries and the remaining 22% represented reduced leakage. PDS dealers in the treatment group also reported a 72% lower bribe price that they would expect to pay to obtain PDS licences, suggesting that they realised the package of reforms would sharply reduce their ability to siphon off grains.
Overall, these results highlight that attempts to reduce corruption can also generate exclusion errors and pain to genuine beneficiaries. Our calculations also suggest that the government can make progress on leakage without excluding beneficiaries by introducing reconciliation but not holding dealers accountable for past diversion as was done in Jharkhand. This is now being done in most states.
We take three broader policy implications from these findings. First, while technology can help reduce leakage, the extent to which this helps the poor depends on programme design. In prior work on the impact of biometric Smartcards on the Mahatma Gandhi National Rural Employment Guarantee
Scheme and pension payments in (unified) Andhra Pradesh (AP), we also found a sharp reduction in leakage, but this saving was passed on to beneficiaries who received more benefits. This reflected the choice made by AP to prioritise beneficiary experience over fiscal savings. In contrast, the ABBA reform to the PDS in Jharkhand (and more generally) prioritised fiscal savings over the beneficiary experience. Of course, these fiscal savings could be returned to beneficiaries in other ways, but there is no reason to expect that this would effectively compensate those excluded from PDS benefits.
Second, major policy reforms call for systematic, independent evaluation. Our results show that both supporters and critics of ABBA in the PDS had valid points: Leakage did go down, but exclusion also increased. The combination of an experimental study design and administrative data ground-truthed against original, systematic beneficiary survey data are what enable us to quantify both components — and also identify ways in leakage can be reduced while minimising pain to beneficiaries.
Third, directly and regularly measuring beneficiaries’ outcomes is important to understand how they are impacted by well-intentioned reforms and to motivate beneficiary centric design of those reforms. One practical and promising way to do so at scale may be to use outbound call centres to regularly call a sample of beneficiaries and generate real-time reports. Our recent work in Telangana, for example, suggests that this can be a highly effective way of improving last-mile service delivery.