Killing us softly

Anony­mous peer re­view is death of in­no­va­tion

THE (Times Higher Education) - - FRONT PAGE - Sui Huang is a pro­fes­sor at the In­sti­tute for Sys­tems Bi­ol­ogy in Seat­tle.

Re­search fund­ing agen­cies face a daunt­ing task when de­cid­ing which pro­posed re­search project to fund. It takes a great deal of ex­per­tise to dis­tin­guish be­tween what in­vestor War­ren Buf­fett once called the “three I’s”: in­no­va­tors, im­i­ta­tors and id­iots. The anony­mous peer re­view sys­tem that has emerged as the univer­sal and un­ques­tioned tool for as­sess­ing re­search grant ap­pli­ca­tions re­li­ably elim­i­nates the id­iots. But, alas, it in­ad­ver­tently sup­presses the in­no­va­tors, too.

Left with the im­i­ta­tors, who per­form solid, some­times use­ful in­cre­men­tal re­search, we are mov­ing in a cir­cle in­stead of for­ward, like cir­cus ele­phants fol­low­ing each other’s tail.

I am not the first per­son to note that sci­en­tific break­throughs can­not be pre­dicted. Most peo­ple will agree that a key in­gre­di­ent of ground­break­ing dis­cov­er­ies is pure cu­rios­ity: the pur­pose-free, pas­sion-driven re­search of cre­ative “out­side-the-box” thinkers, whose pre­pared minds are likely – as Pas­teur put it – to be favoured by chance. A sec­ond in­gre­di­ent, less ro­man­tic but not less im­por­tant, is tech­ni­cal rigour: the ad­her­ence to log­ics of rea­son­ing and the sci­en­tific method. In­no­va­tors pos­sess these two in­gre­di­ents; im­i­ta­tors only the sec­ond; and id­iots nei­ther.

While they ap­pear to be po­lar op­po­sites, pas­sion­ate orig­i­nal­ity and method­olog­i­cal rigour are not mu­tu­ally ex­clu­sive. To spot the in­no­va­tor, one must iden­tify the grant ap­pli­ca­tions that are the pro­duc­tive mix of the two. But while

stan­dard grant re­view pan­els sieve out pro­pos­als that fail to meet the stan­dard of tech­ni­cal rigour, they are so blunt a tool that, in the process, they kill the most in­no­va­tive, vi­sion­ary projects.

How could it be oth­er­wise, when re­view­ers are charged with re­duc­ing com­plex, novel re­search ideas to nu­mer­i­cal scores – the av­er­ages of which are used by pro­gramme man­agers to de­ter­mine their fund­ing de­ci­sions, as if the re­view­ers were as in­fal­li­ble as the Pope? This con­sen­sus eval­u­a­tion avoids risks but neu­tralises the ex­cep­tional eval­u­a­tor with an eye for the ground­break­ing pro­posal that evades the av­er­age mind. Only a medi­ocre pro­posal will thrive in such an en­vi­ron­ment.

The poet Wil­liam Cy­ples warned in 1864 of the dan­ger to hu­man­ity should the “av­er­a­gar­ian” bu­reau­crats pre­vail, and this lim­i­ta­tion of peer re­view is well recog­nised by sea­soned re­search pro­gramme of­fi­cials. Yet more and more grant agen­cies have adopted what the US Na­tional In­sti­tutes of Health calls “scoredriven fund­ing de­ci­sions”, sur­ren­der­ing their abil­ity to make de­ci­sions based on the holis­tic judge­ment of ex­pe­ri­enced pro­gramme man­agers who can take into ac­count un­quan­tifi­ables, such as orig­i­nal­ity and pas­sion. Per­haps the cause is noth­ing more sin­is­ter than a mis­guided but well­mean­ing sense of fair­ness. But the ef­fects are no less dis­as­trous for that.

Nor does re­viewer anonymity help. It is ob­served, of course, be­cause it is thought to sup­press bias and main­tain ob­jec­tiv­ity by pro­tect­ing re­view­ers from pos­si­ble vendet­tas in case of a neg­a­tive cri­tique and by pre­vent­ing exchange of favours through un­duly pos­i­tive eval­u­a­tion. But no so­ci­ety be­stows anonymity on any gov­ern­ing body that makes im­por­tant, di­rec­tion­set­ting de­ci­sions. Anonymity di­lutes mag­na­nim­ity and ac­count­abil­ity. With­out mag­na­nim­ity, the will to re­sist per­sonal bi­ases di­min­ishes. And with­out ac­count­abil­ity, dili­gence drops – and, with it, the drive to go the ex­tra mile to iden­tify non­main­stream, true in­no­va­tion.

The peer re­view sys­tem be­comes an echo cham­ber that nur­tures group­think. The im­pres­sion­able minds of ju­nior re­view­ers, still sus­cep­ti­ble to the fads of science, will favour pro­pos­als close to their main­stream think­ing, and the vi­cious cy­cle goes on whereby the rich labs get richer. More than 40 per cent of re­search dol­lars dis­pensed by the NIH go to the top­funded 10 per cent of re­searchers. Di­ver­sity is sup­pressed as re­search is chan­nelled in one di­rec­tion.

To add in­sult to in­jury, peer re­view­ers are rarely true peers in the sense of hav­ing equal ex­per­tise. There is barely any vet­ting of their in­tel­lec­tual fac­ulty to com­pre­hend the highly tech­ni­cal con­tent of ap­pli­ca­tions. Hyper­spe­cial­i­sa­tion in mod­ern re­search al­most guar­an­tees that re­view­ers will have less ex­per­tise in the do­main of a pro­posal than its often more se­nior au­thors. What one does not un­der­stand, one does not judge favourably; only the most no­ble, wise and vi­sion­ary will pos­sess in­tel­lec­tual em­pa­thy for a for­eign idea that does not spring from their own mind and may even con­tra­dict their own views.

A new course is needed. Fun­ders must re­claim the dis­cre­tion that they once had for fos­ter­ing game­chang­ing dis­cov­er­ies to give in­no­va­tion a chance. If anonymity of peer re­view and its low qual­ity is the ma­jor cul­prit, then log­i­cally there are two so­lu­tions: ei­ther anonymity must be lifted or the pro­gramme man­agers should treat the re­view panel as what it is: a group of un­vet­ted, anony­mous sci­en­tists whose eval­u­a­tion must be taken with a pinch of salt, con­sid­ered only as op­tional “sec­ond opin­ions”. The wis­est and most hon­ourable sci­en­tists and pro­gramme man­agers, nat­u­rally above the fray, must be charged with the task of de­tect­ing orig­i­nal­ity and de­vo­tion among ap­pli­cants, and they should seek to pro­mote open­ended ex­plo­ration ir­re­spec­tive of ob­vi­ous util­ity.

Oth­er­wise, bil­lions of dol­lars will con­tinue to be wasted as the fu­ture is im­pov­er­ished of sci­en­tific break­throughs.

Only the most no­ble, wise and vi­sion­ary will pos­sess in­tel­lec­tual em­pa­thy for a for­eign idea that does not spring from their own mind

Newspapers in English

Newspapers from UK

© PressReader. All rights reserved.