CAN WE TRUST SCI­EN­TISTS IN A POST-TRUTH WORLD?

IN THESE TEST­ING TIMES, WE NEED OUR SCI­ENCE TO BE SOLID

BBC Earth (Asia) - - The Last Word -

It’s the big ques­tion be­ing asked around the world. In these post-truth, fake news, al­ter­na­tive-fact times, who can we trust? Most peo­ple are pretty sure of one thing: it’s not politi­cians or the me­dia. For years, they’ve been at the bot­tom of sur­veys of trust­wor­thi­ness. Amaz­ingly, a re­cent global poll re­vealed that what lit­tle trust they once en­joyed has now plunged to the low­est level ever recorded.

For­tu­nately, those same polls also high­light the ex­is­tence of the ul­ti­mate source of re­li­able in­sight: sci­ence. Not sur­pris­ingly, the cur­rent cri­sis of trust has prompted high-minded aca­demics to pen pieces in­sist­ing it’s time we all put our trust in the meth­ods of sci­ence.

What’s strik­ing about these calls to arms is their naivety. While sci­ence has an im­pres­sive track record of de­bunk­ing mis­con­cep­tions, blun­ders and plain lies, it doesn’t fol­low that we should there­fore put our com­plete trust in sci­en­tists. For that as­sumes sci­en­tists can be trusted to know what they’re do­ing. And sadly, that’s just not the case. Too many re­searchers seem to think that hard data alone is the hall­mark of re­li­able sci­ence. Yet hard data from badly de­signed stud­ies is quite ca­pa­ble of giv­ing com­pelling sup­port for claims that are just plain wrong.

For ex­am­ple, imag­ine there’s a new idea for re­duc­ing ju­ve­nile crime: take the worst of­fend­ers to a tough jail to see what awaits them if they don’t mend their ways. To test the idea, we can sim­ply check to see if the vis­its trig­ger a fall in re-ar­rest rates among those tak­ing part.

Chances are the data will show the idea works – but that doesn’t mean it ac­tu­ally does. That’s be­cause of an ef­fect that’s called ‘re­gres­sion to the mean’, which rears its head when deal­ing with ex­treme cases.

Those young of­fend­ers were cho­sen to take part pre­cisely be­cause they were ar­rested an ex­treme num­ber of times. But that’s partly the re­sult of chance: they just ran out of luck too of­ten. Once they’ve had their pri­son visit, their spate of bad luck is likely to ‘regress’ back to a more av­er­age rate. As a re­sult, they’ll evade re-ar­rest – and thus ap­pear to have mended their ways, when in re­al­ity they haven’t.

This isn’t some es­o­teric pos­si­bil­ity ei­ther. For decades a scheme called Scared Straight was used in the US fol­low­ing claims it dra­mat­i­cally cut re-of­fend­ing rates. It’s now clear that the ap­par­ently rock-solid ev­i­dence was any­thing but. When the idea was tested us­ing stud­ies de­signed to cope with re­gres­sion to the mean, the ben­e­fit van­ished. In­deed, a ma­jor re­view of the ev­i­dence pub­lished in 2013 showed it was ac­tu­ally worse than use­less, and in­creased of­fend­ing rates.

Over the years, re­gres­sion to the mean has fooled re­searchers in fields from medicine and busi­ness to psy­chol­ogy and fi­nance. Which wouldn’t be so bad, ex­cept the phe­nom­e­non has been known about since Vic­to­rian times.

And that’s one of the strik­ing things about these traps. Warn­ings about them have been cir­cu­lat­ing for years, seem­ingly with lit­tle ef­fect. That’s be­cause many – per­haps even most – work­ing sci­en­tists have a sur­pris­ingly poor un­der­stand­ing of how to avoid the many pit­falls in turning data into re­li­able in­sights.

To be fair, a lot of sci­en­tists recog­nise this. A re­cent poll in the jour­nal Na­ture ranked ‘bet­ter un­der­stand­ing of statis­tics’ top among fac­tors that would lead to more re­li­able sci­ence.

There has never been a greater need for trust­wor­thy ev­i­dence on is­sues that af­fect us. The sci­en­tific process is with­out ques­tion the best way to gather such ev­i­dence. But those claim­ing to use its tech­niques need to up their game if they are to jus­tify our trust in them.

“TOO MANY RE­SEARCHERS SEEM TO THINK

THAT HARD DATA ALONE IS THE HALL­MARK OF RE­LI­ABLE

SCI­ENCE”

Robert Matthews is a vis­it­ing pro­fes­sor in sci­ence at As­ton Uni­ver­sity, Birm­ing­ham

Newspapers in English

Newspapers from Singapore

© PressReader. All rights reserved.