Eco­nomic data is not al­ways about num­bers

The Pak Banker - - OPINION - Tim Har­ford

It was one of the most in­flu­en­tial eco­nom­ics stud­ies to have been pub­lished in the past 20 years, with a sim­ple ti­tle: "Worms". Now, its find­ings are be­ing ques­tioned in an ex­change that some­how man­ages to be en­cour­ag­ing and frus­trat­ing all at once. De­vel­op­ment eco­nom­ics is grow­ing up, and get­ting acne.

The au­thors of "Worms", econ­o­mists Ed­ward Miguel and Michael Kre­mer, stud­ied a de­worm­ing pro­ject in an area of western Kenya where par­a­sitic in­testi­nal worms were a se­ri­ous prob­lem in 1998. Miguel and Kre­mer con­cluded three things from the ran­domised trial. First, de­worm­ing treat­ments pro­duced not just health ben­e­fits but ed­u­ca­tional ones, be­cause health­ier chil­dren were able to at­tend school. Sec­ond, the treat­ments were crack­ing value for money. Third, there were use­ful spillovers: when a school was treated for worms, in­fec­tion rates in nearby schools also fell.

The "Worms" study was in­flu­en­tial in two very dif­fer­ent ways. Ac­tivists be­gan to cam­paign for wider use of de­worm­ing treat­ments, with some suc­cess. De­vel­op­ment econ­o­mists drew a sep­a­rate les­son: that run­ning ran­domised tri­als was an ex­cel­lent way to fig­ure out what worked. In this, they were fol­low­ing in the foot­steps of epi­demi­ol­o­gists. Yet it is the epi­demi­ol­o­gists who are now ask­ing the awk­ward ques­tions. Alexan­der Aiken and three col­leagues from the Lon­don School of Hy­giene and Trop­i­cal Medicine have just pub­lished a pair of ar­ti­cles in the In­ter­na­tional Jour­nal of Epi­demi­ol­ogy that ex­am­ine the "Worms" experiment, and find it want­ing. Their first ar­ti­cle fol­lows the orig­i­nal method­ol­ogy and un­cov­ers some er­rors, one of which calls into ques­tion the claim that de­worm­ing pro­duces spillover ben­e­fits. Their sec­ond ar­ti­cle uses epi­demi­o­log­i­cal meth­ods rather than the sta­tis­ti­cal tech­niques pre­ferred by econ­o­mists. It raises the con­cern that the cen­tral "Worms" find­ings may be a fluke.

Ev­ery­one agrees that there were some er­rors in the orig­i­nal pa­per; such er­rors aren't un­com­mon. There's agree­ment, too, that it's very use­ful to go back and check clas­sic study re­sults. But on the key ques­tions, there is lit­tle com­mon ground. Miguel and Kre­mer stoutly de­fend their find­ings, ar­gu­ing that the epi­demi­ol­o­gists have gone through sta­tis­ti­cal con­tor­tions to make the re­sults dis­ap­pear.

Other de­vel­op­ment econ­o­mists sup­port them. Af­ter re­view­ing the con­tro­versy, Berk Ozler of the World Bank says: "I find the find­ings of the orig­i­nal study more ro­bust than I did be­fore." Yet epi­demi­ol­o­gists are un­easy. The re­spected Cochrane Col­lab­o­ra­tion, an in­de­pen­dent net­work of health re­searchers, has pub­lished a re­view of de­worm­ing ev­i­dence, which con­cludes that many de­worm­ing stud­ies pro­duce rather weak ev­i­dence of ben­e­fits. What ex­plains this dif­fer­ence of views? Partly this is a clash of aca­demic best prac­tices. Con­sider the treat­ment of spillover ef­fects.

To Miguel and Kre­mer, these were the whole point of the clus­ter study. Aiken, how- ever, says that an epi­demi­ol­o­gist thinks of such ef­fects as "con­tam­i­na­tion" - an un­de­sir­able source of sta­tis­ti­cal noise. Miguel be­lieves this may ex­plain the dis­agree­ment. The epi­demi­ol­o­gists fret about the sta­tis­ti­cal headaches the spillovers cause, while the econ­o­mists are en­thused by the prospect that these spillovers will help im­prove child­hood health. Another cul­tural dif­fer­ence is this: epi­demi­ol­o­gists have long been able to run rig­or­ous tri­als but, with big money some­times at stake, they have had to de­fend the in­tegrity of those tri­als against bias.

Econ­o­mists, by con­trast, are used to hav­ing to make the best of nois­ier data. Con­sider a cen­tury-old in­ter­ven­tion, when John D Rock­e­feller funded a pro­gramme of hook­worm erad­i­ca­tion across the Amer­i­can south. A few years ago, the economist Hoyt Bleak­ley teased apart cen­sus data from the early 20th cen­tury to show that this pro­gramme had led to big gains in school­ing and in in­come. To an economist, that is clever work. To an epi­demi­ol­o­gist, it's a cu­rios­ity and of lim­ited sci­en­tific value. As you might ex­pect, my sym­pa­thies lie with the econ­o­mists. I sus­pect that the ef­fects that Miguel and Kre­mer found are quite real, even if their meth­ods do not quite match the cus­toms of epi­demi­ol­o­gists.

Newspapers in English

Newspapers from Pakistan

© PressReader. All rights reserved.