Psy­chol­ogy re­search of­ten ques­tion­able, study finds

The China Post - - LIFE - BY KERRY SHERI­DAN

Sci­en­tific stud­ies about how peo­ple act or think can rarely be repli­cated by out­side ex­perts, said a study Thurs­day that raised new ques­tions about the se­ri­ous­ness of psy­chol­ogy re­search.

A team of 270 sci­en­tists tried re­pro­duc­ing 100 psy­chol­ogy and so­cial science stud­ies that had been pub­lished in three top peer-re­viewed U.S. jour­nals in 2008.

Just 39 per­cent came out with same re­sults as the ini­tial re­ports, said the find­ings in the jour­nal Science.

The study top­ics ranged from peo­ple’s so­cial lives and in­ter­ac­tions with oth­ers to re­search in­volv­ing per­cep­tion, at­ten­tion and mem­ory.

No med­i­cal ther­a­pies were called into ques­tion as a re­sult of the study, although a sep­a­rate ef­fort is un­der­way to eval­u­ate can­cer bi­ol­ogy stud­ies.

“It’s im­por­tant to note that this some­what dis­ap­point­ing out­come does not speak di­rectly to the va­lid­ity or the fal­sity of the the­o­ries,” said Gil­bert Chin, a psy­chol­o­gist and se­nior editor at the jour­nal Science.

“What it does say is that we should be less con­fi­dent about many of the orig­i­nal ex­per­i­men­tal re­sults.”

Study co-au­thor Brian Nosek from the Univer­sity of Vir­ginia said the re­search shows the need for sci­en­tists to con­tin­u­ally ques­tion them­selves.

“A sci­en­tific claim doesn’t be­come be­liev­able be­cause of the sta­tus or au­thor­ity of the per­son that gen­er­ated it,” Nosek told re­porters.

“Cred­i­bil­ity of the claim de­pends in part on the re­peata­bil­ity of its sup­port­ing ev­i­dence,” he told re­porters.

Prob­lems can arise when sci­en­tists cherry-pick their data to in­clude only what is deemed “sig­nif­i­cant,” or when study sizes are so small that false neg­a­tives or false pos­i­tives arise.

Nosek said sci­en­tists are also un­der pres­sure to pub­lish their re­search regularly and in top jour­nals, and the process can lead to a skewed pic­ture.

“Not ev­ery­thing we do gets pub­lished. Novel, pos­i­tive and tidy re­sults are more likely to sur­vive peer re­view and this can lead to pub­li­ca­tion bi­ases that leave out neg­a­tive re­sults and stud­ies that do not fit the story that we have,” he said.

“If this oc­curs on a broad scale, then the pub­lished literature may be­come more beau­ti­ful than the re­al­ity.”

Some ex­perts said the prob­lem may be even worse that the cur­rent study sug­gested.

John Ioan­ni­dis, a bi­ol­o­gist at Stan­ford Univer­sity in Palo Alto, Cal­i­for­nia, told Science mag­a­zine that he sus­pects about 25 per­cent of psy­chol­ogy pa­pers would hold up un­der scru­tiny, about the same “as what we see in many bio­med­i­cal dis­ci­plines,” he was quoted as say­ing.

Key Cau­tion

One study au­thor who par­tic­i­pated in the pro­ject as both a re­viewer and re­vie­wee was E.J. Ma­si­campo, as­sis­tant pro­fes­sor at Wake For­est Col­lege in North Carolina.

He was part of a team that was able to repli­cate a study that found peo­ple who are faced with a con­fronta­tional task, like hav­ing to play a vi­o­lent video game, pre­fer to lis­ten to an­gry mu­sic and think about neg­a­tive ex­pe­ri­ences be­fore­hand.

But when out­side re­searchers tried to repli­cate Ma­si­campo’s own study — which hy­poth­e­sized that a sug­ary drink can help col­lege stu­dents do bet­ter at mak­ing a com­pli­cated de­ci­sion — they were not suc­cess­ful.

Ma­si­campo ex­pressed no bit­ter­ness, chalk­ing up the dif­fer­ences to ge­o­graph­i­cal fac­tors, and stress­ing that the experiment showed how com­pli­cated it can be to do a high-qual­ity repli­ca­tion of a study.

“As an orig­i­nal au­thor whose work was be­ing repli­cated, I felt like my re­search was be­ing treated in the best way pos­si­ble,” he said.

There are ways to fix the process so that find­ings are more likely to hold up un­der scru­tiny, ac­cord­ing to Dorothy Bishop, pro­fes­sor of de­vel­op­men­tal neu­ropsy­chol­ogy at the Univer­sity of Ox­ford.

Newspapers in English

Newspapers from Taiwan

© PressReader. All rights reserved.