Why we shouldn’t always follow the science
Scientists’ funding and prestige depends on them publishing more and more studies — but a new book reveals the hidden cost...
William Summerlin, a dermatologist working at a prestigious New York cancer institute in 1974, claimed he had made a major breakthrough in skin grafting.
He had, he said, successfully grafted a section of skin from a black mouse on to a white one. The only problem was, he hadn’t. What he’d actually done was take a black felt-tip pen and colour in a patch of the white mouse’s fur.
His trickery was soon discovered. (How could he have imagined it wouldn’t be?). He was sacked.
Not all the frauds discussed in the first, most entertaining section of Ritchie’s revelatory book are as blatant as Summerlin’s. Nor are they all comfortably in the past. in 2011, a Dutch social psychologist, Diederik Stapel, hit the headlines with studies that seemed to prove people showed more racial prejudice in dirtier environments.
‘Where there’s rubbish, there’s racism,’ one newspaper reported.
Stapel failed to mention that none of the data in his studies was real. He’d made it all up. He would print off worksheets to give to imaginary participants in his research. He would show these to his colleagues and students, announcing loudly that he was off to start his investigations. Then, when no one was looking, he’d dump them all in the recycling bin. it was easier just to type the results he wanted into a spreadsheet.
more dangerous than Stapel was italian surgeon Paolo macchiarini. He invented a synthetic trachea which he used in transplant operations on cancer patients. in scientific papers, he described his successes in glowing terms. it was only some years later, in 2015, that an official report revealed the truth: macchiarini had falsely claimed that patients had improved when he hadn’t even bothered to examine them.
MaNY experienced severe complications. Some even died after receiving his artificial trachea. macchiarini was a major fantasist both in and out of the operating theatre. He told a girlfriend that he was the personal doctor to the Pope. The Vatican later denied any knowledge of him.
The surgeon also informed his girlfriend that celebrities such as the Obamas and elton John were personal friends and would be turning up to their wedding. Unfortunately, he didn’t let her know that he was already married and had two children.
as Ritchie writes: ‘if this kind of fraud occurs at the very highest levels of science, it suggests that there’s much more of it that flies under the radar.’
and fraud isn’t the only, or even the most important, problem faced by science.
in 2005, an analyst named John ioannidis wrote a paper entitled: ‘Why most Published Research Findings are False’.
He reached this conclusion through mathematical modelling, writing: ‘Once you consider the many ways that scientific studies can go wrong, any given claim in a scientific paper is more likely to be false than true.’
This seems an extraordinary claim but Ritchie catalogues those ‘many ways’ in telling detail. all kinds of unconscious bias can affect research — ‘biases towards getting clear or exciting results, supporting a pet theory, or defeating a rival’s arguments’.
error can creep in to the most carefully devised studies.
in 2016, Dutch scientists created what they called ‘statcheck’ — a kind of spellchecker for statistics. They used it on 30,000 scientific papers. Nearly half included some numerical inconsistency and 13 per cent contained a serious mistake which could change the interpretation of the results.
Scientists are too often subject to what Ritchie calls ‘perverse incentives’. They must publish or perish. if they don’t crank out papers, they risk losing funding, and missing out on prestigious jobs. The competitive arena of modern science also demands that researchers hype their work beyond its merits. a recent analysis found that use of such words as ‘unprecedented’ and ‘ground-breaking’ had shot up in the past 20 years.
One Twitter account retweets impressive-sounding headlines from studies, such as: ‘Compounds in carrots reverse alzheimer’s-like symptoms.’ it then adds two important words which were originally omitted — ‘in mice’.
Perhaps there is simply too much ‘science’ being undertaken. in 2013, 2.4 million new papers were published. There are thousands of journals — not all of them are very discriminating. So-called ‘predatory’ journals pester researchers by email with requests for papers.
One exasperated scientist, weary of their attentions, sent a predatory journal a ‘paper’ which consisted solely of the sentence, ‘Get me off your f***ing mailing list’, repeated 800 times.
it was judged ‘excellent’ and accepted for publication.
Stuart Ritchie, a lecturer at the institute of Psychiatry, Psychology and Neuroscience at King’s College, london, is very definitely not ‘anti-science’.
it has, he writes, ‘cured diseases, mapped the brain, forecasted the climate and split the atom; it’s the best method we have of figuring out how the universe works and of bending it to our will’.
What he wants is to save science from what he sees as its present crisis. His book brilliantly highlights the problems in current practices and sets out a path towards new ones.