Rachel Brazil in Nature: The term ‘REF-able’ is now in common usage in UK universities. “Everyone’s constantly thinking of research in terms of ‘REF-able’ outputs, in terms of ‘REF-able’ impact,” says Richard Watermeyer, a sociologist at the University of Bristol, UK. He is referring to the UK Research Excellence Framework (REF), which is meant to happen every seven years and is one of the most intensive systems of academic evaluation in any country. “Its influence is ubiquitous — you can’t escape it,” says Watermeyer. But he and other scholars around the world are concerned about the effects of an extreme audit culture in higher education, one in which researchers’ productivity is continually measured and, in the case of the REF, directly tied to research funding for institutions. Critics say that such systems are having a detrimental effect on staff and, in some cases, are damaging researchers’ mental health and departmental collegiality.
Unlike other research benchmarking systems, the REF results directly affect the distribution of around £2 billion (US$2.6 billion) annually, creating high stakes for institutions. UK universities receive a significant proportion of their government funding in this way (in addition to the research grants awarded to individual academics).
Since its inception, the REF methodology has been through several iterations. The rules about which individuals’ work must be highlighted have changed, but there has always been a focus on peer-review panels to assess outputs. Since 2014, a team in each university department has been tasked with selecting a dossier of research outputs and case studies that must demonstrate societal impact. These submissions can receive anything from a four-star rating (for the most important, world-leading research) to just one star (the least significant work, of only national interest). Most departments aim to include three- or four-star submissions, often described as ‘REF-able’.
But the process is time-consuming and does not come cheap. The most recent REF, in 2021, was estimated to have cost £471 million. Tanita Casci, director of the Research Strategy & Policy Unit at the University of Oxford, UK, acknowledges that it’s resource-intensive, but says that it’s still a very efficient way of distributing funds, compared with the cost of allocating money through individual grant proposals. “I don’t think the alternative is better,” she concludes. The next exercise has been pushed back a year, until 2029, with planned changes to include a larger emphasis on assessment of institutional research culture.
Many UK academics see the REF as adding to an already highly competitive and stressful environment. A 2021 survey of more than 3,000 researchers (see go.nature.com/47umnjd) found that they generally felt that the burdens of the REF outweighed the benefits. They also thought that it had decreased academics’ ability to follow their own intellectual interests and disincentivized the pursuit of riskier, more-speculative work with unpredictable outcomes.
Some other countries have joined the assessment train — with the notable exception of the United States, where the federal government does not typically award universities general-purpose research funding. But no nation has chosen to copy the REF exactly. Some, such as the Netherlands, have instead developed a model that challenges departments to set their own strategic goals and provide evidence that they have achieved them.
Whatever the system, few assessments loom as large in the academic consciousness as the REF. “You will encounter some institutions where, if you mention the REF, there’s a sort of groan and people talk about how stressed it’s making them,” says Petra Boynton, a research consultant and former health-care researcher at University College London.
More here.
