Our research funding system is shortchanging the humanities
The government’s research excellence framework (Ref) is perhaps the ultimate in bureaucratic exercises. It aims every seven years to assess, department by department, every “research active” academic in the UK. The aim is laudable: to ensure that a stream of research funding (known as QR) is distributed to universities fairly and transparently. But for the humanities, the Ref does nothing but harm.
Few would quarrel with the principle of a system of assessment for the humanities based on reading and judging work submitted, rather than one using citation indexes and other bibliometric data. But the scale of the task makes meaningful or honest assessment impossible. There are too few assessors to provide competent, specialised judgement on the range of work submitted. The workload imposed on them requires superhuman capacities: along with their normal teaching and research, panel members must read the equivalent of a full-length book every day for nine months.
They can hardly be blamed if they skim or sample (“10 minutes for a book, two minutes for an article” is what one source told me). Compare the care taken to judge an article for a journal: two specialist referees must justify their verdict in detail, before editors make further checks. The Ref assessment turns out to be a sham, an exercise in bad faith, giving only the appearance of judging academics’ work fairly, not the reality.
To make matters worse, there is a very crude system of scoring, which in effect considers each output as either good (“world-leading” = 4*), alright (“internationally excellent”=3*) or useless (“recognised internationally”=2*; “recognised nationally” =1* – both of which attract zero funding). Monographs – which still represent much of the best work in most humanities disciplines, providing new, thorough analyses in depth – can be counted as only two outputs at most, although they might be 20 times the length of an article and require 50 times as much work. As a result, ambitious researchers focus on articles, which are often superficial, at the expense of monographs of lasting value.
Attempting to measure impact on the world outside the universities is also failing the humanities. In the next Ref, in 2021, impact will count for a quarter of the whole score. The criteria are not designed for humanities subjects, but rather for scientific discoveries or technological advances. They exclude academic books that reach a wide audience of general readers – the real way scholars in history, literature, art and philosophy make an impact. Since the Ref tends to give low ratings to general and popularising publications, the best academics are discouraged from sharing their knowledge and ideas with the public.
Bad though the Ref is, many would say that it, or something similar, is needed to distribute QR. But that disguises the truth. The fees paid by humanities students more than cover the full costs of their teaching, including their teachers’ sabbatical research leave, and another 100% for overheads. A high-performing humanities department might increase the amount its university receives in QR, or even be rewarded financially by the university for doing so (though probably not) – but if the humanities departments in the UK are regarded as a whole, no QR money reaches them. They are being obliged to compete in the Ref for money they never receive.
The Ref, then, causes real harm, distorting the working patterns of academics in the humanities, discouraging them from popularising their ideas, and deceiving them with assessments made in such a way that they cannot be reliable. It wastes vast amounts of time and energy that could be spent on teaching and research. Yet there is no realistic way of making this bad system work. Good assessment would cost far more working hours, more assessors, more assessors to check the assessors. Far better to recognise that, since the
humanities do not gain financially from the Ref, and work in them is hindered by it, they should be excluded from the system. Colleagues in other disciplines would be grateful if such a change led to a lighter system of assessment. In the humanities, most would heave a sigh of relief.
John Marenbon is a senior research fellow of Trinity College, Cambridge. His Intangible Assets: Funding Research in the Arts and Humanities is published by Politeia