srhe

The Society for Research into Higher Education

Ian Mc Nay


Leave a comment

The REF: Digging beyond the headlines

By Ian McNay

Recent headlines on the Research Excellence Framework (REF) were  a bit over the top when the scores are scrutinised closely, particularly in Education, as detailed in my earlier blog post back in February. So, I won’t take long, but want to add to what I said then to emphasise the impact of the three elements of scoring and the discontinuities between exercises. Andrew Pollard, chair of the Education panel, concentrates on ‘research activity’ in his comment on the BERA website, where 30 percent was at 4* level. However, output had only 21.7 per cent at that level; the scores were boosted by scores of over 40 per cent for impact and environment.

If you look hard you can find a breakdown of scores by element for 2008, and these show how things have changed. For environment, in 2008, 5 units scored 50 per cent or higher at 4*, with a top score of 75 per cent; 19 scored 50 per cent or more when 3* and 4* are combined. This time, at 4* 18 units scored 50 per cent or more and 8 scored 100 per cent. Combining 3* and 4* shows that 52 units scored more than 50 per cent with 23 scoring 100 per cent across those top two levels.  That grade inflation suggests either considerable investment for development or less demanding criteria.

On impact there is no precedent. In 2008 the third element was esteem indicators, where the top score for 4* was 40 per cent, by only two units, with a further 10 getting 30 per cent. For impact in 2014, 13 units scored more than 50 per cent – well above the highest score for esteem. Perhaps we judged our peer academics more harshly than users of research did. Or, perhaps, they were less obsessed with long term, large scale, statistical studies using big data sets which successive panels have set as the acme of quality work, and more concerned with ’did it make a difference?’ Continue reading

Ian Mc Nay


Leave a comment

Applications to UK universities: digging beyond the headlines…

By Ian McNay

‘Record numbers apply to enter higher education’ was the good news reported at the end of the first application period in January by most papers – I have not seen one to query this, but I have not checked them all. The March figures are now out, almost unreported, and confirm what was true in January: applications are up on last year, by just over 2 per cent. The increase for the UK is lower than that, at about 1.6 per cent.

That is good, but the claim of a record depends what you count and compare with. The secondary headline was that this record showed that high fee levels had had no impact, implying that the record applied where fees had been raised. Not true. Continue reading