Study finds that while the Research Excellence Framework boosts productivity of UK universities, it doesn’t enhance their efficiency.

A new study led by City, University of London suggests that the introduction of a Research Excellence Framework (REF) to measure UK universities’ research performance has resulted in significant increases in the amount of their high-quality research published, but not in the output of active academics, and that they must do more to support their research strategies. undergraduate.

Many countries around the world use Performance Based Research Systems (PRFS) as a means of determining the distribution of funding for research across institutions of higher education. In 2008 REF replaced Research Assessment Exercises (RAE) as a measure of this across UK universities, with the first results in 2014 covering the period between 2009 and 2014.

Research by Julia Iori and Albert Banal-Estanyol, professors of economics at City, together with academics from the University of Barcelona, ​​the London School of Economics and Political Science and the University of Palermo, looked at the performance of institutions and the increasing impacts of REF2014 on the production and quality of research delivered across UK universities in contrast to Similar institutions in the United States, which are not subject to such action.

The sample included 103 UK universities that submitted research to the REF2014 Economics and Econometrics or Business and Management panel. As a control group, 135 US universities have an economics department or business school from the top quartile according to the Research Papers in Economics (RePEc) database as of December 2018.

Key findings from the research include:

• REF2014 significantly increased the research output of UK universities compared to US institutions, but the number of publications per author did not grow with it. This indicates that REF has generally encouraged universities to hire more academics to increase activity rather than supporting existing researchers to produce a greater volume of work.
• Thereafter, the number of UK university publications increased on average per department by 41.37 per cent across the time period, with the most noticeable change occurring towards the end of the course, between 2012 and 2014.
• Research excellence—as measured by publications in top journals (3*, 4*, and 4** journals in the Association of Business Schools’ ranking of scholarly journals)—also increased compared to other countries after implementation of the REF.
• The increases in research output for REF2014 were generally more pronounced for Russell Group universities than for non-Russell Group universities, with a stronger positive effect than REF2014 in terms of the number of major publications per author.
• Applications from universities that chose the RAE2008 Economics and Econometrics panel but switched to the Business and Management panel for REF2014 were significantly reduced compared to those that remained in the same panel. Thus, the percentage of their trade publications in top journals did not increase.

The researchers used Scopus, the world’s largest database of peer-reviewed literature, to obtain all articles that included at least one of the 103 UK universities or 135 US universities as an affiliate, and had either “economics, econometrics, and Finance” or “Business Administration” and “Accounting” as subject areas, and were published between 2001 and 2014. Institutions that submitted fewer than ten papers per year on average between 2001 and 2008 were excluded from the total sample size, which was 145,536 publications. Unique.

Professor Iwry said the study suggested REF needed to do more to encourage universities to invest in supportive processes and strategies.

“The Research Excellence Framework is an important tool for universities to be able to attract funding, recruit students, and increase their national and global standing,” she said.

When compared to US universities, our data shows that referencing incites more publications and research articles published in top journals. However, since the number of publications per author remained constant throughout the research period, this appears to be driven by the institutions hiring more academics rather than It is an internal incentive for academics themselves to publish in leading journals. This recruitment is simply a redeployment of academics rather than a strengthening of research capacity.”

“Ultimately, this benefits the elite universities who can attract the best academics, and hurts the minority who lose out to competitors, but it does little to improve overall standards.”

Professor Banal-Istanyol agreed that the study highlighted areas for improvement needed in the evaluation framework.

“REF should encourage greater cooperation between local competitors, rather than pitting them against each other,” he said.

“Comparing universities nationally is not very meaningful because the best will always outpace the worst, especially as academic redistribution polarizes. Where there is a very large gap, this can discourage weaker institutions from producing and delivering research that generally has a detrimental effect.” ”

“Instead, the REF needs to reward universities that outperform others in a similar trajectory, rather than pitting the strong against the weak, which is why international comparisons are so much more useful for driving aggregate standards.”

“This study provides the basis for alternative measures of success of a performance-based research system, to ensure equitable means of distributing funds while maintaining competition and increasing overall standards.”