Mas­sag­ing data to fit a the­ory is not the worst re­search sin, ar­gues Mar­tin Co­hen

The furore over Brian Wansink’s data han­dling over­looks the fact that science is above all about ex­pla­na­tion, ar­gues Mar­tin Co­hen

THE (Times Higher Education) - - CONTENTS -

The re­cent fall from grace of the Cor­nell Univer­sity food mar­ket­ing re­searcher Brian Wansink is very re­veal­ing of the state of play in mod­ern re­search.

Wansink had for years em­bod­ied the ideal to which all aca­demics aspire: in­no­va­tive, highly cited and me­dia-friendly. But, in Septem­ber, Cor­nell found against him on a range of aca­demic mis­con­duct charges, in­clud­ing “mis­re­port­ing of re­search data, prob­lem­atic sta­tis­ti­cal tech­niques, fail­ure to prop­erly doc­u­ment and pre­serve re­search re­sults, and in­ap­pro­pri­ate au­thor­ship”. Wansink ten­dered his res­ig­na­tion.

His re­search, now crit­i­cised as fa­tally flawed, in­cluded stud­ies sug­gest­ing that peo­ple who go gro­cery shop­ping while hun­gry buy more calo­ries, that pre-or­der­ing lunch can help you choose health­ier food, and that serv­ing peo­ple out of large bowls leads them to eat larger por­tions.

Such stud­ies have been cited more than 20,000 times and even led to an ap­pear­ance on The Oprah Win­frey Show. But Wansink was ac­cused of ma­nip­u­lat­ing his data to achieve more strik­ing re­sults. Un­der­ly­ing it all is a sus­pi­cion that he was in the habit of form­ing hy­pothe­ses and then search­ing for data to sup­port them. Yet, from a more gen­er­ous per­spec­tive, this is, after all, only sci­en­tific method.

Be­hind the crit­i­cism of Wansink is a much broader cri­tique not only of his work but of a cer­tain kind of study: one that, while it might have quan­ti­ta­tive el­e­ments, is in essence ethno­graphic and qual­i­ta­tive, its chief value be­ing in sto­ry­telling and in­ter­pre­ta­tion. The quan­ti­ta­tive “er­rors” spot­ted in such mul­ti­faceted re­search stud­ies are am­mu­ni­tion in what is re­ally a much broader meta­phys­i­cal bat­tle.

We for­get too eas­ily that the his­tory of science is rich with er­rors. In a dash to claim glory be­fore Wat­son and Crick, Li­nus Paul­ing pub­lished a fun­da­men­tally in­co­her­ent hy­poth­e­sis that the struc­ture of DNA was a triple he­lix. Lord Kelvin mis­es­ti­mated the age of the Earth by more than an or­der of mag­ni­tude. In the early days of ge­net­ics, Fran­cis Gal­ton in­tro­duced an er­ro­neous math­e­mat­i­cal ex­pres­sion for the con­tri­bu­tions of dif­fer­ent an­ces­tors to in­di­vid­u­als’ in­her­ited traits. We for­get be­cause these er­rors were part of broader nar­ra­tives that came with bril­liant in­sights.

I ac­cept that Wansink may have been guilty of shoe­horn­ing data into pre­con­ceived pat­terns – and in the process may have mixed up some of the fig­ures too. But if the lat­ter is un­for­giv­able, the former is surely re­search as nor­mal. The crit­ics are in­dulging them­selves in a myth of neu­tral ob­servers un­cov­er­ing “facts”, which rests on a view of knowl­edge as pris­tine and eter­nal as any­thing Plato might have dreamed of.

It is thanks to Western phi­los­o­phy that, for thou­sands of years, we have be­lieved that our think­ing should strive to elim­i­nate ideas that are vague, con­tra­dic­tory or am­bigu­ous. To­day’s or­tho­doxy is that the world is gov­erned by iron laws, the most im­por­tant of which is if P then Q. Part and par­cel of this is a belief that the main goal of science is to pro­vide de­ter­min­is­tic – cause and ef­fect – rules for all phe­nom­ena. All ex­per­i­ments rely on cor­re­la­tions, pre­dictabil­ity and repli­ca­bil­ity.

And then there’s the law of non-con­tra­dic­tion. This was first for­mu­lated by Par­menides but Plato also refers to it in the Sophist, phras­ing it as: “Never shall this be proved: that things that are not, are.”

How­ever, what­ever to­day’s no­tions of re­search va­lid­ity, the fact is that Par­menides’ law of non-con­tra­dic­tion in it­self is far from un­con­tro­ver­sial. In­deed, it rep­re­sented a rad­i­cal break from the Io­nian phi­los­o­phy of na­ture that pre­ceded it. This older phi­los­o­phy was based on ob­ser­va­tion and ex­pe­ri­ence in the or­di­nary sense.

The dif­fer­ence in ap­proach re­sulted from a dis­agree­ment about what the an­cients be­lieved were the proper ob­jects of thought: the com­plex­i­ties of the messy real world or ab­strac­tions from a pris­tine but purely the­o­ret­i­cal one. And to­day this re­mains an is­sue at the heart of broad swathes of re­search, ex­pressed and fought over in terms of logic and sta­tis­ti­cal va­lid­ity.

Plato at­tempted to avoid con­tra­dic­tions by iso­lat­ing the ob­ject of in­quiry from all other re­la­tion­ships. But, in do­ing so, he ab­stracted and di­vorced those ob­jects from a re­al­ity that is mul­tire­la­tional and mul­ti­tem­po­ral. This same ar­ti­fi­cial­ity dogs much re­search.

Math­e­ma­ti­cians and me­te­o­rol­o­gists have been grudg­ingly work­ing to­wards ac­cep­tance of this at least since the work of chaos the­ory pi­o­neer Ed­ward Lorenz in the 1960s. Econ­o­mists and bi­ol­o­gists, too, now work with chaotic sys­tems, in which pre­dic­tion of ef­fects is phys­i­cally and log­i­cally im­pos­si­ble.

But what is less ap­pre­ci­ated is that science has never re­ally been about pre­dict­ing. As the com­puter sci­en­tist No­son Yanof­sky puts it in his 2004 book The Outer Lim­its of Science: “What is im­por­tant in science and what makes science sig­nif­i­cant is ex­pla­na­tion and un­der­stand­ing.”

Even if the quan­ti­ta­tive el­e­ments do not con­vince and need re­vis­ing, stud­ies like Wansink’s can be of value if they of­fer new clar­ity in look­ing at phe­nom­ena, and stim­u­late ideas for fu­ture in­ves­ti­ga­tions. Such un­der­stand­ing should be the re­searcher’s Holy Grail.

After all, ac­cord­ing to the tenets of our cur­rent ap­proach to facts and fig­ures, much sci­en­tific en­deav­our of the past amounted to wasted ef­fort, in fields with ab­so­lutely no yield of true sci­en­tific in­for­ma­tion. And yet science has pro­gressed. Mar­tin Co­hen is vis­it­ing re­search fel­low in phi­los­o­phy at the Univer­sity of Hert­ford­shire. His lat­est book, I Think, There­fore I Eat, on food science, is pub­lished by Turner.

Newspapers in English

Newspapers from UK

© PressReader. All rights reserved.