srhe

The Society for Research into Higher Education

Ian Mc Nay


Leave a comment

The REF: Digging beyond the headlines

By Ian McNay

Recent headlines on the Research Excellence Framework (REF) were  a bit over the top when the scores are scrutinised closely, particularly in Education, as detailed in my earlier blog post back in February. So, I won’t take long, but want to add to what I said then to emphasise the impact of the three elements of scoring and the discontinuities between exercises. Andrew Pollard, chair of the Education panel, concentrates on ‘research activity’ in his comment on the BERA website, where 30 percent was at 4* level. However, output had only 21.7 per cent at that level; the scores were boosted by scores of over 40 per cent for impact and environment.

If you look hard you can find a breakdown of scores by element for 2008, and these show how things have changed. For environment, in 2008, 5 units scored 50 per cent or higher at 4*, with a top score of 75 per cent; 19 scored 50 per cent or more when 3* and 4* are combined. This time, at 4* 18 units scored 50 per cent or more and 8 scored 100 per cent. Combining 3* and 4* shows that 52 units scored more than 50 per cent with 23 scoring 100 per cent across those top two levels.  That grade inflation suggests either considerable investment for development or less demanding criteria.

On impact there is no precedent. In 2008 the third element was esteem indicators, where the top score for 4* was 40 per cent, by only two units, with a further 10 getting 30 per cent. For impact in 2014, 13 units scored more than 50 per cent – well above the highest score for esteem. Perhaps we judged our peer academics more harshly than users of research did. Or, perhaps, they were less obsessed with long term, large scale, statistical studies using big data sets which successive panels have set as the acme of quality work, and more concerned with ’did it make a difference?’ Continue reading

Charlotte Mathieson


Leave a comment

A Culture of Publish or Perish? The Impact of the REF on Early Career Researchers

By Charlotte Mathieson

This article aims to highlight some of the ways in which the REF has impacted upon early career researchers, using this as a spring-broad to think about how the next REF might better accommodate this career group.

In my role at the Institute of Advanced Study at the University of Warwick I work closely with a community of early career researchers and have experienced first-hand the many impacts that this REF has had on my peer group; but I wanted to ensure that this talk reflected a broader range of experiences across UK HE, and therefore in preparation I distributed an online survey asking ECRs about their experiences and opinions on the REF 2014.

Survey overview

– 193 responses collected between December 2014 and March 2015
– responses gathered via social media and email from across the UK
– 81.3 % had completed PhDs within the last 8 years
– 41.5 % were REF returned
– 18.7% were currently PhD students
– 10.9% had left academia since completing a PhD

5 main points emerged as most significant from among the responses: Continue reading

Image of Rob Cuthbert


Leave a comment

Was that a foul REF?

By Rob Cuthbert

The Research Excellence Framework, the UK’s latest version of research quality assessment, reached its conclusion just after the SRHE Research Conference. Publication of the results in mid-December led to exhaustive coverage in all the HE media. 

In the Research Season 2008-2014 the controversy was not so much about who ended up top of the league, but whether the English premier league can still claim to be the best in the world.

Big clubs were even more dominant, with the golden triangle pulling away from the rest and filling the top league positions. But controversy raged about the standard of refereeing, with many more players being labelled world class than ever before. Referees supremo David Sweeney was quick to claim outstanding success, but sponsors and commentators were more sceptical, as the number of goals per game went up by more than 50%.

During the season transfer fees had reached record heights as galactico research stars were poached by the big clubs before the end of the transfer window. To secure their World University League places the leading clubs were leaving nothing to chance. It was a league of two halves. After positions based on research outcomes had been calculated there was a series of adjustments, based on how many people watched the game (impact), and how big your stadium was (environment). This was enough to ensure no surprises in the final league table, with big clubs exploiting their ground advantage to the full. And of course after the end of the season there is usually a further adjustment to ensure that the big clubs get an even bigger share of the funding available. This process, decreed by the game’s governing body, is known as ‘financial fair play’.

Some players had an outstanding season – astronomers were reported to be ‘over the moon’ at the final results, but not everyone was happy: one zoologist confided that he was ‘sick as a parrot’. The small clubs lacked nothing in effort, especially at Northampton, where they responded superbly to their manager’s call to put in 107%. But not everyone can be a winner, research is a results business and as always when a team underperforms, some clubs will be quick to sack the manager, and many more will sack the players.

Scepticism about the quality of the league lingers among the game’s governing body, suspicious about high scoring, and there is a risk that the money from the Treasury will finally dry up. The game may not have finished yet, but some … some people are running onto the pitch, they think it’s all over. It is for now.

Rob Cuthbert is Emeritus Professor of Higher Education Management, University of the West of England, Joint Managing Partner, Practical Academics rob.cuthbert@btinternet.com, Editor, Higher Education Review www.highereducationreview.com, and Chair, Improving Dispute Resolution Advisory Service www.idras.ac.uk