‘Whether journals in a world of preprints will continue to carry out pre-publication review is open to question’
As scientists, many of us have a love/hate relationship with peer review. Most of us believe it is necessary to maximise the integrity of the scientific literature, yet we chafe when our own manuscripts are criticised. And we complain when reviewers take too long to review our manuscripts even as we procrastinate about completing that review request sitting in our own inboxes.
There are good reasons to say yes to those requests, such as building a reputation with editors of a journal in which you might want to publish and to preview cool new science. However, the increasing volume of manuscripts coupled with ever greater pressure on scientists’ time means that it is necessary to turn most of them down if you want to get any of your own work done. Finding the right balance isn’t easy.
For their part, journals continue to struggle to secure appropriate reviewers (especially during the dreaded summer and holiday periods of high submission and low availability) even as initiatives such as Publons have been established in the hope of giving more recognition for a task that is usually performed anonymously.
Further challenges to peer review are presented by the accelerating use of preprint servers in biomedical sciences, replicating the well-established practice in physics. Preprints are posted on bioRxiv after minimal oversight by “affiliates” (of which I’m one of many), who check only that they are scientific.
Journals’ lucrative existing role as science’s middlemen will not be readily surrendered, and it is plausible that the top journals at least will be able to continue leveraging their brands to select and curate the best preprints, conferring on them a badge of quality that, notwithstanding pressure to stop judging scientists on the basis of where they have published, is likely to remain of importance to recruitment and promotion committees for years to come.
But whether journals in such a world will continue to carry out pre-publication peer review is open to question, not least because while fewer journals would mean less demand for reviewers, there will be less incentive for reviewers to take part. After all, if a manuscript can be accessed online immediately after submission, reviewers will no longer enjoy the opportunity to get a sneak preview of the latest research. Payment of reviewers could compensate but it would bring with it other issues, such as conflicts of interest.
Journals may decide, instead, to rely on post-publication peer review by archive users. Most preprint servers allow for comments (and subsequent corrections of the preprint) to be added. There may well be further developments in this regard, including inclusion of new data in a review as a means to extend, clarify or rebut findings, and it is possible to imagine a journal approaching an author of a preprint to offer to publish it conditional on such points being addressed.
One issue would be reviewer identification. The fact that post-publication reviewers are typically identified may attract some reviewers in search of greater exposure. But it probably puts off far more. Given the option, most reviewers choose not to make their identity known for several reasons, including potential backlash from disgruntled authors and fellow reviewers – even though most recognise that science would benefit if this largely hidden but substantive and valuable literature were exposed to daylight. Early career academics in particular are probably wise to be wary in this regard.
This problem could be solved by providing anonymity to online reviews. However, it is unclear how much time busy scientists would be prepared to put into post-publication review when the benefits of previewing or building a relationship with a particular journal are no longer in place. While it seems unlikely that many people would be motivated to submit the kind of comprehensive review typically submitted during today’s pre-publication process, the hope might perhaps be that enough people would be willing to comment briefly, on aspects of the paper that particularly strike them – so that the whole approximates to something substantive. Previous experiments with post-publication peer review are not encouraging, but were competing with established pre-publication review.
While the demise of expert pre-publication review by peers might be greatly lamented by the dreaded reviewer #3, the damage it would inflict on true scientific progress is open to question.
Indeed, perhaps it is time to ask whether peer review itself warrants reviewer #3’s unique brand of attention.