srhe

The Society for Research into Higher Education

Ian Mc Nay


Leave a comment

Ian McNay writes…

An interesting follow-up to the item last time on research into not doing something. The German government put out a TV message featuring two couch potatoes…doing nothing, and advocating staying on the couch as a contribution to not spreading the Covid-19 infection. Somebody has a sense of humour.

On the pandemic, one group that has emerged with credit is the research community, the speed of decision making and the extent of international co-operation in sequencing the genetic code of the virus, using the code to design a vaccine and then developing it in record time. I suggest that by the end of 2021 the number of lives saved by the actions of researchers will be greater than the number lost through the actions and inactions of politicians. Experts have gained in respect. On the other hand, in this country…

On a (perhaps) less contentious issue, closer to members’ interests, I recommend the book edited by Stephen Gorard and published by Routledge: Getting Evidence into Education. Evaluating the routes to policy and practice. He has a salutary listing in the final chapter of barriers to the widespread use of high quality evidence. First is the regrettable lack of quality in research, with the growth of work he identifies as ‘small-scale, uninventive, journalistic or [only] purportedly theoretical work’ lacking scientific replicability. Second is the low ability or willingness to communicate findings to users, which is now improving, possibly because of the impact factor in REF funding. On the other side, he questions whether users really appreciate and want to use good evidence, particularly when it runs counter to values that underpin ideology. Finally, ‘teachers are still largely unaware of the availability of good evidence’ or lack the authority or resources to make changes in practice, and ‘school leaders often appear content to plan school improvement without referring to robust evidence. In my experience, much of that is also true in higher education, as well as in government policy making for the sector.

The latest data on membership of REF panels, issued in December, show that, despite government commitment to diversity and levelling up, the academic capitalists among the elite universities still control the commanding heights of the research economy. On the main panels, pre-92 universities have 46 full members, post-92 institutions have one – Kingston on Panel D. International universities have 15, which shows where competition in Lisa Lucas’s research game is focussed. On the sub-panels the figures are 636 to 87, with assessor members at 112 and 24. This affects grading. I make no accusation of crony capitalism, but there may be an unconscious bias of common cultural identity, as in the Eurovision Song Contest, where votes go to ‘people like us’, so the same old same old may be rewarded ahead of new approaches and findings challenging the established corpus of work done by members. That in turn affects funding. A parliamentary reply on 17 November listed overall government research funding (much of it QR funding from REF) to the 13 universities in the West Midlands. Between 2015 and 2019, Birmingham and Warwick (33 members) got an increase of 21% to £256m, mainly attributable to Warwick gaining an immediate £16 after the 2014 REF and a similar amount over the period; Aston and Keele (8 members) had no increase on £30m – Aston gained £1m, Keele had a matching reduction; Coventry gained £3m to £9m after a good REF. The other 8 institutions had £12 m among them. So two universities, dominating regional representation, got 83% of the funds distributed in 2018/9.

Amanda Solloway, Minister for HE in England, at a recent HEPI webinar, committed to reviewing the nature of excellence in research, acknowledged the need for diversity on interpretations and a need to link to ‘levelling up’. There may be a lesson from the Covid pandemic, where approaches by elite western countries failed; under-regarded countries did better. In 2019 the Johns Hopkins Global Health Security Index ranking capacity to deal with outbreaks of infectious disease ranked the USA first and the UK second; New Zealand came in at 35th and South Korea at 9th. The article in the Guardian from which I took those figures (Laura Spinney on 30 December) quotes Sarah Dalglies in the Lancet – ‘The pandemic has given the lie to the notion that expertise is concentrated in, or at least best channelled by, legacy powers and historically rich states’. Maybe that applies to research, too. REF panels, and Amanda Solloway, please note.

SRHE Fellow Ian McNay is emeritus professor at the University of Greenwich.

For answers to Ian’s SRHE News Quiz 2020, they are now online here.

Ian Mc Nay


Leave a comment

The REF: Digging beyond the headlines

By Ian McNay

Recent headlines on the Research Excellence Framework (REF) were  a bit over the top when the scores are scrutinised closely, particularly in Education, as detailed in my earlier blog post back in February. So, I won’t take long, but want to add to what I said then to emphasise the impact of the three elements of scoring and the discontinuities between exercises. Andrew Pollard, chair of the Education panel, concentrates on ‘research activity’ in his comment on the BERA website, where 30 percent was at 4* level. However, output had only 21.7 per cent at that level; the scores were boosted by scores of over 40 per cent for impact and environment.

If you look hard you can find a breakdown of scores by element for 2008, and these show how things have changed. For environment, in 2008, 5 units scored 50 per cent or higher at 4*, with a top score of 75 per cent; 19 scored 50 per cent or more when 3* and 4* are combined. This time, at 4* 18 units scored 50 per cent or more and 8 scored 100 per cent. Combining 3* and 4* shows that 52 units scored more than 50 per cent with 23 scoring 100 per cent across those top two levels.  That grade inflation suggests either considerable investment for development or less demanding criteria.

On impact there is no precedent. In 2008 the third element was esteem indicators, where the top score for 4* was 40 per cent, by only two units, with a further 10 getting 30 per cent. For impact in 2014, 13 units scored more than 50 per cent – well above the highest score for esteem. Perhaps we judged our peer academics more harshly than users of research did. Or, perhaps, they were less obsessed with long term, large scale, statistical studies using big data sets which successive panels have set as the acme of quality work, and more concerned with ’did it make a difference?’ Continue reading

Charlotte Mathieson


Leave a comment

A Culture of Publish or Perish? The Impact of the REF on Early Career Researchers

By Charlotte Mathieson

This article aims to highlight some of the ways in which the REF has impacted upon early career researchers, using this as a spring-broad to think about how the next REF might better accommodate this career group.

In my role at the Institute of Advanced Study at the University of Warwick I work closely with a community of early career researchers and have experienced first-hand the many impacts that this REF has had on my peer group; but I wanted to ensure that this talk reflected a broader range of experiences across UK HE, and therefore in preparation I distributed an online survey asking ECRs about their experiences and opinions on the REF 2014.

Survey overview

– 193 responses collected between December 2014 and March 2015
– responses gathered via social media and email from across the UK
– 81.3 % had completed PhDs within the last 8 years
– 41.5 % were REF returned
– 18.7% were currently PhD students
– 10.9% had left academia since completing a PhD

5 main points emerged as most significant from among the responses: Continue reading

Image of Rob Cuthbert


Leave a comment

Was that a foul REF?

By Rob Cuthbert

The Research Excellence Framework, the UK’s latest version of research quality assessment, reached its conclusion just after the SRHE Research Conference. Publication of the results in mid-December led to exhaustive coverage in all the HE media. 

In the Research Season 2008-2014 the controversy was not so much about who ended up top of the league, but whether the English premier league can still claim to be the best in the world.

Big clubs were even more dominant, with the golden triangle pulling away from the rest and filling the top league positions. But controversy raged about the standard of refereeing, with many more players being labelled world class than ever before. Referees supremo David Sweeney was quick to claim outstanding success, but sponsors and commentators were more sceptical, as the number of goals per game went up by more than 50%.

During the season transfer fees had reached record heights as galactico research stars were poached by the big clubs before the end of the transfer window. To secure their World University League places the leading clubs were leaving nothing to chance. It was a league of two halves. After positions based on research outcomes had been calculated there was a series of adjustments, based on how many people watched the game (impact), and how big your stadium was (environment). This was enough to ensure no surprises in the final league table, with big clubs exploiting their ground advantage to the full. And of course after the end of the season there is usually a further adjustment to ensure that the big clubs get an even bigger share of the funding available. This process, decreed by the game’s governing body, is known as ‘financial fair play’.

Some players had an outstanding season – astronomers were reported to be ‘over the moon’ at the final results, but not everyone was happy: one zoologist confided that he was ‘sick as a parrot’. The small clubs lacked nothing in effort, especially at Northampton, where they responded superbly to their manager’s call to put in 107%. But not everyone can be a winner, research is a results business and as always when a team underperforms, some clubs will be quick to sack the manager, and many more will sack the players.

Scepticism about the quality of the league lingers among the game’s governing body, suspicious about high scoring, and there is a risk that the money from the Treasury will finally dry up. The game may not have finished yet, but some … some people are running onto the pitch, they think it’s all over. It is for now.

Rob Cuthbert is Emeritus Professor of Higher Education Management, University of the West of England, Joint Managing Partner, Practical Academics rob.cuthbert@btinternet.com, Editor, Higher Education Review www.highereducationreview.com, and Chair, Improving Dispute Resolution Advisory Service www.idras.ac.uk