srhe

The Society for Research into Higher Education

Ian Mc Nay


Leave a comment

Some reflections on learning during lockdown…

by Ian McNay

This is a listing of thoughts that came over 3 months of isolation when learning was in a different context.

  •  Policy based on science becomes policy blamed on science in the hands of politicians, who rarely, if ever, admit being at fault, which they see as weakness.
  • Researchers therefore need to be very sure of what they publish or advice they give, because the nuances of conditionality of research findings do not transfer easily to a political mindset. Do not rush to publish when data are still emerging in a fluid situation. Rigorous peer review becomes even more important, but seems to have been neglected by The Lancet and The New England Journal of Medicine over hydroxychloroquine. The reputation of the Imperial group and their influential model was also called in to question when examined by the excellent Radio 4 programme, More or Less, and later in Private Eye (19 June) which discovered that their forecast about the rate at which infections doubled – 5 days – neglected data from Italy and the early days of the UK experience – which gives nearer to 3 days – and therefore led to a delay in lockdown. Maybe not a 4* rating for impact, after all. The cherry picking by ministers and the pressures to edit findings from those in a draft (I have experienced both) may have been a learning experience for some.
  • The definition of ‘world leading’ adopted by government in reviewing its own policy in operation over Covid-19 must use criteria even lower than those in REF derided by Johnston (Ron, that is – ex-VC of Essex, then professor emeritus at Bristol)
  • Presentism was shown to be less essential than Jacob Rees-Mogg thinks. I won’t comment much on the teaching experience since most of mine has been at a distance for some years. Some research approaches – interviews, focus groups – needed to be adapted by my students, whose field work was disrupted. Anthropological immersion in a community for study purposes was challenging – but there is a lot of material being gathered virtually and more to follow when retrospective work is done. Anthropologists and ethnologists may have a field day examining how societies and communities changed – norms, habits, rituals, relationships, communications – and how quickly they responded to crisis: better locally than when driven from the centre. Outside the academic, meetings were shorter with more respect for others in terms of interruptions. Some international conferences had higher attendances than at times in the past when the time and cost of travel was a deterrent. This has been true of SRHE, where some events offered by Networks have had over 100 participants, when the room they were held in BC (Before Covid) could hold a maximum of 50. A higher proportion were from overseas. Currently, therefore, some people, less advantaged because of geography or funding, may get access they could not previously afford. Fees for non-members have also been suspended, though this has led to a drop in the number of members joining or renewing. Please do pay membership fees – they give value for money.
  • The opposite is true for some students where they do not have home technology, and so inequality of opportunity has increased. Universities need to reflect on this and recall that in the first years of the Open University students were provided with not only home experiment kits (including a rotor arm which one postal worker left outside the door of a 7th floor flat in Toxteth: I had to argue hard with the administration to stop the student being charged), but with home computers, so all had access.
  • Working from home had its challenges. As someone who has always gone to work to work, with the journey allowing a role transition from place to place, entering the dining room to work at the table does not have the same liminal impact. The morning walk to the newsagent, which allowed thoughts to organise themselves and next paragraphs and passages to be planned, has been suspended. Papers are now delivered. Lockdown has had a differential impact by gender. Submission rates of journal articles have gone down for women, up for men, with a knock on consequence for REF submissions.
  • The ‘unknown knowns’ of inequality, prejudice and discrimination are now out in the open and, if continued are deliberate and systemic, done knowingly, not some deterministically ‘systemic’ feature about which we can do nothing. The claim that Covid-19 hit high and low alike was based around two people – the Prince of Wales and the Prime Minister – neither of whom died. The figures I quoted on deaths of clinical staff last time became even more alarming, with 94% of doctors killed by Covid-19 reported as being from BAME backgrounds. The interconnection of class and race in the distribution of infection and deaths shows that responsibility rests with policies on disadvantage of the poor increasing exposure. We should also have absorbed another ‘known’, that value is not reflected in wage/salary levels, but should be. Humanity and decency should mean that policy seeks to redress inequality, and universities have roles to play in this and need to commit to performing them, beyond looking at their own patterns of discrimination. Especially, perhaps, those running police training courses, who need to review them as economists needed to review their courses after the 2008 crash (some even did so) and those leading MBA programmes after the report from CIPD that only 8 per cent of managers thought about the relevance of values. The history curriculum at all levels needs examining for balance. One of my newspapers surveyed decolonisation and found only 20% of universities had done anything and even fewer on a whole university basis. The ubiquitous media academic de nos jours, David Olusonge, who as I write, as well as appearing on the news, has just started a BBC4 programme on Black British History to sit alongside one on BBC2 about a house in Guinea Street , Bristol, built by a likely slave trader, could be the person to lead on it.
  • Of course universities are not racist; senior managers have issued statements saying so, not after the glaring picture of the statistics but by joining the Black Lives Matter bandwagon of corporate guilt, denial and claimed commitment following recent police killings and decades of discrimination. The heads of Oxford colleges did it most publicly through a high profile letter in The Guardian; others were less limelight-seeking. I did not see any comment from David Lammy, but the next day the Guardian had a report of racist language and harassment in election hustings at an Oxford college … for a cake representative (I kid you not; this is Oxford, remember). More seriously, the day after that came a report that BAME student societies had withdrawn involvement in Oxford’s outreach programme because of the perceived lack of support following student entry. Lessons for us all there. Only three percent of Rhodes scholarships go to those from Africa. The governors of Oriel College have now decided to remove the statue of Rhodes: sometimes people power can achieve things mainstream processes of deliberation do not.
  • Lessons, too, on leadership, where what has emerged during the crisis echoes work by myself and others. What people want is: clarity of policy, so there is certainty about expectations; consistency and continuity rather than constant change, which makes us feel like experimental guinea pigs where different things are tested on us (REF 2021 has 12 major changes); and confidence in leaders, which the first two will help promote, but which also needs a sense of common identity, where we are visibly and evidentially ‘all in it together’. There is a Leadership Foundation in HE report saying exactly that https://uwe-repository.worktribe.com/output/828871 . (Thanks to Rob C for digging that out) The involvement of those at devolved level with expertise is also essential; driving from the centre, with autocratic control, is neither efficient nor effective, and, in some cases not economic. True at university institutional level, too.
  • We have learned that the crisis, like others, offers opportunities. HEPI had a report outlining possibilities for enhancing access; for many, the exposure to distance learning has stimulated broader thinking about curriculum process, and there will be strategic thinking about the portfolio of provision, though English government decisions on extra undergraduate numbers give them to elite universities whatever their TEF grade and ignores many judged excellent by TEF judgements. Yet the elite universities are dropping down international league tables because of, say the compilers (according to a report in…The Guardian 10 June) ‘poor teaching and declining research impact’. Of the 84 in the rankings (very few were modern universities, though Greenwich made it) 66 had a drop in SSRs, 59 a drop in research citations, and 51 a drop in international student numbers, who, for them, will now be replaced by extra domestic students.
  • Finally, we have learned that some academics are market sensitive and see a promotional opportunity when it comes. For those on television, bookshelves became advertising hoardings, with their latest output showing, cover to the front, not spine, just over their shoulder and very legible. In one case, a blown up photograph had been framed and hung on the wall.

SRHE Fellow Ian McNay is emeritus professor at the University of Greenwich

Ian Mc Nay


2 Comments

A period of reflection

By Ian McNay

At the beginning of what some people mistakenly think of as the beginning of a new decade – who counts to ten by starting at zero and finishing at nine? – the pressure is to reflect on the past and project for the future. I am going to mainly eschew the former, but do have concerns for the next five or ten years. In other countries where a populist government has been elected, and moved to authoritarianism, such as Hungary, Turkey, or even the USA, the auguries are not good for higher education. I am not claiming that the new UK administration is as extreme as those examples, but the indications are there about its attitude to dissenting voices – the BBC and Channel 4 coverage of the election, elected parliamentarians defying the party whip, and even the supreme court, to whose rulings the government has twice had to conform, reluctantly, in the interests of constitutional democracy. The manifesto commitment to reviewing the organs of government and the judiciary has been seen by some as ominous.

Whatever the politics, there are other reasons to be concerned for HE. The eight years since fees were last tripled, to £9,000, have been fairly comfortable, financially, for most universities, if not their staff at the sharp end of operations. Marginal costs per student will often be low, especially in non-STEM subjects, so surpluses expand with every expansion of numbers. The Augar Report recommendations, if accepted, may lower fees with little guarantee that government will cover the loss of income. The cost of student loans, some of which now comes within current public spending, will increase dramatically with the demographic bulge in 18-year-olds, starting now, unless the cap on numbers in England is re-imposed, as seems likely, given views on ‘useless’ degrees, unnecessary experts, and pressure to prefer apprenticeships and FE recovery over investing in people who, on graduation, are less likely to vote Conservative than those without a university education. Graduates move to cities where there are jobs, leaving their home communities to an ageing population with different political predilections, made evident in December, and considerable resentment against what they see as graduate elitists in Westminster disregarding their needs and views. That may then convert to resentment against the universities that produce them and whose students affect the availability of property to rent and ‘studentify’ sections of a community. If the low rate of HE access of white working class males, and ‘over-representation’ of British BAME students is added to the mix, there is a base for Powellite stirring in a search for somebody to blame.

HE will not, then, be a high priority among competing, vote-winning, initiatives. Savings from not having to give EU students access to UK loans may not be re-invested. Even for research, where specific protective commitments have been made, the loss of EU funding and the greater difficulty in recruiting and partnering internationally because of visa restrictions, the prospects are not good. UK universities have already begun to drop down international league tables, and there is little reason to believe that that trend will stop. If income becomes tight, consider where funds might come from and the political risks of dependence on Chinese students and partnerships, or grants from oil rich regimes in the Middle East, or big pharma to a greater extent than now. Governors and senior managers will be faced with moral issues, testing the robustness of asserted values.

If universities are to overcome being seen as part of the problem, what has to change? Over the end of year break, I have been reading a collection of essays arising from an event 50 years after Chomsky published ‘The responsibility of intellectuals’. . That is the book’s title; it is edited by Nicholas Allott, Chris Knight, and Neil Smith, published by UCL Press. For us, as individuals who might be regarded as intellectuals, the three responsibilities set out by Chomsky remain: ‘to speak truth and expose lies; to provide historical context, and to lift the veil of ideology’ (Allott et al, 2019:7). The context has changed in 50 years: we ‘speak’, as do others, on social media, where regulation is lax; truth must be told to the powerless as well as the powerful, needing a different level of discourse; there is recognition that ‘the elite need to have an accurate idea of what is going on’ (p10) which means listening to others’ legitimate and valid truths derived from an experience, a background and axioms that differ from those of the people in power; and there is need for active engagement with that alternative reality, not just commentary from a distance, however sympathetic. This may lead to a better informed and value-oriented set of intellectuals.

At institutional level, that applies within universities, too. The gap between the governors, including the senior managers, and the governed is dysfunctional – can you name, say, three lay governors? When did you last speak with one? Some years ago, I reviewed the work of the Greenwich governing body, as recommended in the Dearing Report. It was clear that there was no communication with the governed, either up or down, no communication with ‘constituencies’, since governors could not identify their constituency. There was only an oral report on Academic Board meetings, by the VC, with all other information for the governors coming from the SMT, sometimes incomplete, at times misleading. SMT/staff communication has improved, but is still poor and unsystematic, avoiding anything that might highlight negatives.

As with many modern universities, there are two seats on Academic Board for professors elected by and from the professoriate; this year, as too often in the past, there were no nominations, nobody willing to stand, for a body that has no power beyond ‘advising’ the CEO and where the 1988/92 laws require there to be a majority of people with management responsibilities … on an academic board. My work with staff in many universities suggests that disengagement is widespread: academics have reverted to being what Hoyle labelled ‘restricted professionals’ – classroom based and classroom bound, by choice, since there is a fear of repercussions/reprisals if there is any expression of dissent. So compliance produces conformity, not the creative diversity essential to a healthy academic community. That may also develop at corporate level with the increasingly intrusive regulation by the Office for Students. Interviewing vice chancellors some years ago, even then there was a fear of speaking against ministerial policy, which might result in financial discrimination against their university. There might also be targeted supplementary ‘regulation’ (=control) from the Office for Students. Only in England, of course, which already has more surveillance from government and its agencies than other parts of the UK, as shown by Michael Shattock and Aniko Horvath in their 2019 book The Governance of British Higher Education. Possibly as a factor of size, but only partly, I suspect, transferring Chomsky’s concern over ideology to this context, there is also – Shattock and Horvath, again – less solidarity among the different mission groups, who act like ideological factions in a political party. Perhaps some reflections on common values (echoing urgings in one such party) might bring them together. I recommend reading chapter 5 of the Dearing Report as a basis for a period of reflection on values in an academic (and political) community.

I wish you a good new year, with hope that my concerns prove to be unfounded.

SRHE Fellow Ian McNay is emeritus professor at the University of Greenwich

Brenda Leibowitz


3 Comments

Brenda Leibowitz 1957 – 2018

It is with much sadness that the SRHE community notes the passing of Brenda Leibowitz, a South African scholar in academic development and higher education. Her recent work on academic staff development features twice in the SRHE/Routledge book series; first, a chapter in the edited 2016 book “Researching Higher Education: International perspectives on theory, policy and practice”, and then, with Vivienne Bozalek and Peter Kahn, a 2017 book “Theorising learning to teach in higher education”. She also presented her work at the SRHE annual conference and will be known to many in the community for her engagement across a wide range of higher education conferences in South Africa and abroad.

Brenda’s engaged scholarship over nearly 30 years was strongly rooted in her activist commitment to recognizing a democratic and transformed South Africa through education and higher education. She began teaching in secondary schools designated for ‘coloured’ pupils and this sharpened sense both of the inequities of apartheid and the possibilities in education led to a most formative stint in the Academic Development Centre at the University of the Western Cape (UWC), where her practice and emerging scholarship focused on language issues in the university. She followed this with a period of curriculum work as a Director in the national Department of Education, completing a PhD from the University of Sheffield, and moved from here to nearly a decade directing the Centre for Teaching and Learning at Stellenbosch University. Here her work moved from a focus on student development to staff development, bringing with it a critical edge and an exceptionally strong commitment to collaboration and empowerment. In 2014 her scholarship was noted with the appointment to a chair in Teaching and Learning at the University of Johannesburg (and more recently, with the award of an National Research Foundation (NRF) funded South African Research Chair Initiative (SARChI) position on Post School Education and Training).

Brenda was one of the Principal Investigators on an ESRC Newton/NRF funded project entitled “Southern African Rurality in Higher Education” (SARiHE), which began in 2016 and will complete work in 2019. Brenda’s long-term interest in social justice in higher education especially for students from rural backgrounds in South Africa helped to secure funding for this project. The Southern African University Learning and Teaching (SAULT) forum, which she helped to build, has also been important in this project and has facilitated the involvement of academics and academic developers from across nine Southern African countries.

Brenda was absolutely prolific in her deep scholarship, and pulled many others along in her wake. She published across national and international journals, book chapters and books. A flavour of the evolution of her distinctive scholarship can be seen in the perusal of some of her article titles that drew on direct quotes from her research participants:

* “Why now after all these years you want to listen to me?” Using journals in teaching history at a South African university. The History Teacher, 1996 

* “Communities isn’t just about trees and shops”: Students from two South African universities engage in dialogue about ‘community’ and ‘community work’. Journal of Community and Applied Social Psychology, 2008 

* What’s Inside the Suitcases? An investigation into the powerful resources students and lecturers bring to teaching and learning. Higher Education Research and Development, 2009 

* “Ah, but the whiteys love to talk about themselves”: Discomfort as a pedagogy for change. Race, Ethnicity and Education, 2010 

* “It’s been a wonderful life”: Accounts of the interplay between structure and agency by “good” university teachers. Higher Education, 2012 

The title of her most recent paper, with colleague Vivienne Bozalek, ‘Toward a Slow scholarship of teaching and learning in the South’ is also revealing. ‘Slow scholarship’ foregrounds qualities such as thoughtfulness, attentiveness, the valuing of relationships, creativity, and depth of engagement – qualities that embody so well Brenda’s own scholarship and her way of being in the world.

In her research, Brenda leaves an extraordinary written record of scholarship; however, more importantly, there are the many, many lives that this extraordinary educator and scholar touched and influenced deeply. Brenda had an openness and generosity of spirit that allowed her to traverse boundaries and bring together collaborative teams across all the usual divisions of discipline, social background and institutional type. She had a solid compass that never deviated from its pointing towards the long arc of social justice, but she accomplished all she did with notable humility and serious interest in others and their educational and research journeys.

The period of late apartheid bred a distinctive sort of higher education researcher, many of these working in academic development at UWC in the 1990s. In this group of hugely influential higher education scholars, including Chrissie Boughey and Melanie Walker, Brenda made a distinctive and important contribution, cut much too short by her cancer diagnosis. We will remember her with love and admiration.

Jenni Case, Lisa Lucas and Delia Marshall

Ian Mc Nay


1 Comment

Professional and Professionalism

By Ian McNay

My views on ‘professional’ and ‘professionalism’ in HE have been tested in several ways recently. One of my doctoral students has just got his award for a study on the topic. A major part of his work was a case study in a modern university, with a survey of teaching professionals with fellowship status in HEA either by a PGCE or a reflective portfolio of experience route. The survey group presented a homogeneous monochrome picture of what Hoyle, many years ago, labelled ‘restricted’ professionals – classroom bound with little engagement in the wider professional context, focused on subject and students, with punctuality and smart dress as professional characteristics. That reflected the response I got from some academics when I was appointed as a head of school: I met each one of my staff and, as part of the conversation, asked their view on development issues and future possibilities for the school. The response of several can be summarised by two: ‘I don’t have a view; not my role and above my pay grade’, and ‘You’re the boss. Tell me what to do and I’ll do it’. Continue reading

Paul Temple


Leave a comment

Steering Column

By Paul Temple

The SRHE Blog hasn’t featured a motoring column before – and actually it’s a bit late to start: if you’ve recently bought a new-ish car, it may well be your last one. That’s because the car makers and the big tech companies are betting the farm on driverless (“autonomous”) cars being the future of road travel – not in some “weekend breaks on Mars”-type sci-fi scenario, but in the next couple of years. At the end of February, an autonomous Nissan Leaf drove six miles around East London, including negotiating a roundabout on the A13 that scares me. It’s generally assumed that these cars mostly won’t be owned by individuals, but will be driverless taxis, summoned to your door (at least, in towns). Most new cars are already at or near what the industry calls “Level 3”, with sensors for parking, automatic braking, lane guidance and so on; “Level 4” cars will add all this to artificial intelligence and so do away with the human driver. The computer won’t make the stupid mistakes that all human drivers do – so one effect that’s already been noted will be the “nice to have” problem of a reduction in the number of transplant organs available.

It’s the combination of the scale and the imminence of this revolution that makes it so interesting social scientifically: this won’t be a gradual evolution, but a big bang – one year, cars like we’ve always known them; a year or two later, a transformation. Like an avalanche, unnoticed high up on the mountain, it is about to sweep down. (Look at one of the many blogs on this, such as “Connected Cars”, to get a sense of how fast things are moving.)

Why should this be of interest to higher education researchers? Continue reading

Ian Mc Nay


Leave a comment

Ian McNay writes…

By Ian McNay

Good to see action being taken on gender-based pay inequality after all the talk and all the evidence. Congratulations to Essex and, recently, LSE where women will get rises to close the gap of over 10% between their salaries and those of men with equal rank, length of service, research outputs and age. Essex earlier gave a one-off boost to female professors.

But … way to go fellas! I saw little coverage of the Women in Science Fellowships (we need a different title, surely). There were five awards of £15,000 to be spent in any way the winners choose in order to help them continue research in their chosen fields. What, then, did they choose?

  1. Help with childcare and some conference travel
  2. Flexible childcare arrangements to allow the further development of key collaborations and partnerships
  3. Childcare to allow an early return to work
  4. Practical help with childcare fees, with some on equipment and conferences
  5. Resolving the tough decision between my research and looking after my son

I cannot imagine a group of five male winners having such unanimity about such a ‘choice’. Thank you to the Evening Standard for its coverage and editorial stance.

Holiday reading

I spent some time over the summer copying Ulysses and cruising, less excitingly, the northern Mediterranean, Aegean and Adriatic seas. The ship’s library, for 1800 passengers, had about 120 books, including a history of Test Match Special. More relevant to readers here was one by Jonathan R Cole (2016) Toward a More Perfect University, NY, Public Affairs. One can criticise the title for thinking that ‘perfect’ can be susceptible to gradations, and for implying that the top 300 US research universities – the author’s basic starting point, are already perfect. Indeed, he criticises Johns Hopkins for distorting their mission to overstate research. Among his recommendations are a reduction in doctoral students; more emphasis on undergraduates – the quality of teaching and academic standards; and a compulsory one-year training programme for new members of governing bodies, Boards of Regents or their counterparts, to include a module on the nature of a university and its place in USA society. Good luck with that one.

The second one was a surprise. It was a John Grisham imitation, a courtroom drama. The following comes when the defence lawyer is about to cross-examine and discredit an academic expert. He sends his assistant to dig out his publications from 2008, 2004 and 2000.

‘If you’re cross-examining an academic witness, you have to look at their publications. Those years were the ARAE: the American Research Assessment Exercise. The more articles published by academic staff in the ARAE, the more money comes to their university and the more money those nerds take home. During those years everybody writes like crazy, and probably reasonable academics write crap things they would not dream of writing ordinarily. Writing for volume does not promote good theories, and pretty soon they’re writing papers on fairies and UFOs. Back then, articles meant cash. So, if there’s anything out there that will give us something on [him] that’s where we’ll find it.’

Steve Cavanagh (2016) The defense, NY, Flatiron Books, p51

And there was, and they did, and attacked the credibility of the witness to distract from the credibility of his evidence. Successfully.

There is, by the way, no such thing as the ARAE. It was, after all, a work of fiction, but bearing an uncanny resemblance to reality. The author’s details give no obvious connection to a background in higher education.

SRHE Fellow Ian McNay is emeritus professor at the University of Greenwich.

Ian Mc Nay


Leave a comment

Ian McNay writes…

By Ian McNay

How many Eleanors can you name? Roosevelt, Marx, Bron, Aquitaine, Rigby…add your own. Why am I asking this? Because it is a new metric for widening access. The recent issue of People Management, the journal of CIPD, reports that in 2014 the University of Oxford admitted more girls named Eleanor than students who had received free school meals. Those who were taught at private schools were 55% more likely to go to Oxbridge than student who received free school meals. Those two universities have even reduced the proportion of students they admitted who came from lower socio-economic groups in the decade from 2004=5, from 13.3% to 10% at Oxford and from 12.4% to 10.2% at Cambridge. Other Russell Group universities also recorded a fall, according to HESA data. So, second question: how many people do you know who have had free school meals or whose children have had? Not a visible/audible characteristic: they do not wear wristlet identifiers. But your university planning office will have the stats if you want to check its record. Continue reading

Ian Mc Nay


Leave a comment

Post election, Post budget: The shifting landscape of Higher Education in the UK

By Ian McNay

It says something about the Guardian and its reader profile when it builds a crossword round knowing the names of the chancellors of Russell Group universities, as it did on 27 June. I also liked its headline the previous day: ‘New dinosaur found in university store cupboard’. It has now been re-united with older colleagues in the department of economics.

My serious considerations here concern the post-election agenda – what I called Jo-Jo’s in-tray issues in a recent workshop at Coventry (to where/whom, congratulations on their Guardian league table ranking on student views on teaching quality: second only to Cambridge, and, more importantly, above Warwick). That system level policy focus will be balanced by treatment of emergent concerns at institutional level in a later piece.

The most immediate issue is a cut of £450m in the DBIS budget, which may be followed by further longer-term cuts as the failed austerity project continues. Nick Hillman at Coventry suggested an easy step was to convert grants to loans, which reduces the deficit but still increases the debt. I am writing before the budget, but I expect a loosening of fee limits, not ruled out during the election and possibly linked to teaching excellence, with high scorers being allowed to increase fees, as UUK want. Then there will be the sale of further tranches of the loan book, possibly to universities for their own alumni. Research Fortnight expects science to be protected Continue reading

Ian Mc Nay


Leave a comment

The REF: Digging beyond the headlines

By Ian McNay

Recent headlines on the Research Excellence Framework (REF) were  a bit over the top when the scores are scrutinised closely, particularly in Education, as detailed in my earlier blog post back in February. So, I won’t take long, but want to add to what I said then to emphasise the impact of the three elements of scoring and the discontinuities between exercises. Andrew Pollard, chair of the Education panel, concentrates on ‘research activity’ in his comment on the BERA website, where 30 percent was at 4* level. However, output had only 21.7 per cent at that level; the scores were boosted by scores of over 40 per cent for impact and environment.

If you look hard you can find a breakdown of scores by element for 2008, and these show how things have changed. For environment, in 2008, 5 units scored 50 per cent or higher at 4*, with a top score of 75 per cent; 19 scored 50 per cent or more when 3* and 4* are combined. This time, at 4* 18 units scored 50 per cent or more and 8 scored 100 per cent. Combining 3* and 4* shows that 52 units scored more than 50 per cent with 23 scoring 100 per cent across those top two levels.  That grade inflation suggests either considerable investment for development or less demanding criteria.

On impact there is no precedent. In 2008 the third element was esteem indicators, where the top score for 4* was 40 per cent, by only two units, with a further 10 getting 30 per cent. For impact in 2014, 13 units scored more than 50 per cent – well above the highest score for esteem. Perhaps we judged our peer academics more harshly than users of research did. Or, perhaps, they were less obsessed with long term, large scale, statistical studies using big data sets which successive panels have set as the acme of quality work, and more concerned with ’did it make a difference?’ Continue reading

Ian Mc Nay


1 Comment

REF outcomes: Some comments and analysis on the Education Unit of Assessment

By Ian McNay

A recent report to the Leadership Foundation for HE (Franco-Santos et al, 2014) showed that ‘agency’ approaches to performance management favoured by managers, which ‘focus on short-term results or outputs through greater monitoring and control’ are ‘associated with …lower levels of institutional research excellence’. A small project I conducted last summer on the REF experience showed such approaches as pervasive for many institutions in the sample. Nevertheless, the main feature of the 2014 REF was the grade inflation over 2008.

Perhaps such outstanding global successes, setting new paradigms, are under-reported, or are in fields beyond the social sciences where I do most of my monitoring. More likely ‘strategic entry policies’ and ‘selectivity in many submissions’, noted by the panel, have shifted the profile from 2008. A second successive reduction of 15 percent in staff numbers submitted, with 15 institutions opting out and only nine coming in since 2008 will also have had an effect. Welcome back to Cardiff, hidden in a broad social sciences submission last time. The gain from Cardiff’s decision is that it will scoop all of the QR funding for education research in Wales. None of the six other institutions making submissions in 2008, with some 40 staff entered, appeared in 2014 results. That may be a result of the disruptions from mergers, or a set of strategic decisions, given the poor profiles last time

What has emerged, almost incidentally, is an extended statement of the objectives/ends of the exercise, which now does not just ‘inform funding’. It provides accountability for public investment in research and evidence of the benefit of this investment. The outcomes provide benchmarking information and establish reputational yardsticks for use within the HE sector and for public information. Still nothing about improving the quality of research, so any such consequence is not built in to the design of the exercise, but comes as a collateral benefit.

My analysis here is only of the Education Unit of Assessment [UoA 25]. More can be found at www.ref.ac.uk .

Something that stands out immediately is the effect of the contribution of scores on environment and impact on the overall UoA profile. Across all submissions, the percentage score for 4* ratings was 21.7 for outputs, but 42.9 for impact and 48.4 for environment.  At the extreme, both the Open University and Edinburgh more than doubled their 4* score between the output profile and the overall profile. Durham, Glasgow and Ulster came close to doing so, and Cardiff and the Institute of Education  added 16 and 18 points respectively to their percentage 4* overall profile.

Seven of the traditional universities, plus the OU, had 100 per cent ratings for environment. All submitted more than 20 FTE staff for assessment – the link between size and a ‘well found’ department is obvious. Several ‘research intensive’ institutions that might be expected to be in that group are not there: Cambridge scored 87.5 per cent. Seven were judged to have no 4* elements in their environment. Durham, Nottingham and Sheffield had a 100 per cent score for impact in their profile, making Nottingham the only place with a ‘double top’.

In England, 21 traditional universities raised their overall score; 6 did not. Among the less research intensive universities, 29 dropped between the two, but there were also seven gains in 4* ratings: Edge Hill went from 2.1 per cent [output] to 9 percent [overall] because of a strong impact profile; Sunderland proved King Lear wrong in his claim that ‘nothing will come of nothing’: they went from zero [output rating] to 5 per cent overall, also because of their impact score. Among the higher scoring modern universities, Roehampton went down from 31 to 20 at 4*, but still led the modern universities with 71 percent overall on the 3*/4* categories.

I assume there was monitoring of inter-assessor comparability, but there appears to have been those who used a 1-10 scale and added a zero, and those who used simple fractions, for both impact and environment. Many do not get beyond 50/50. For output, it is different; even with numbers submitted in low single figures, most scores go to the single decimal point allowed.

For me, one of the saddest contrasts was thrown up by alphabetic order. Staffordshire had an output rating of 25 percent at 4*; Southampton had 18 percent; when it came to overall profile, the rankings were reversed: Staffordshire went down to 16 because it had no scores at 3* or 4* in either impact or environment; Southampton went up to 31 percent. There is a personal element here: in 2001 I was on the continuing education subpanel. In those days there was a single grade; Staffordshire had good outputs – its excellent work on access was burgeoning, but was held down a grade because of concerns about context issues – it was new and not in a traditional department. Some other traditional departments were treated more gently because they had a strong history.

The contribution of such elements was not then quantified, nor openly reported, so today’s openness at least allows us to see the effect.  I believed, and still do,that, like the approach to A level grades, contextual factors should be taken account of, but in the opposite direction. Doing well despite a less advantaged context should be rewarded more. My concern about the current structure of REF grading is that, as in 2001, it does the exact opposite. The message for units in modern universities, whose main focus is on teaching, is to look to the impact factor. A research agenda built round impact from the project inception stage may be the best investment for those who wish to continue to participate in future REFs, if any. There is a challenge because much of their research may be about teaching, but the rules of the game bar impact on teaching from inclusion in any claim for impact. That dismisses any link between the two, the Humboldtian concept of harmony, and any claims to research- led teaching. As a contrast, on the two criticisms outlined here, the New Zealand Performance –Based Research Fund [PBRF] has, among its aims:

  • To increase the quality of basic and applied research
  • To support world-leading research-led teaching and learning at degree and postgraduate level.

Might the impending review of REF take this more integrated and developmental approach?

Reference: Franco-Santos, M., Rivera, P. and Bourne, M.(2014) Performance Management in UK Higher Education Institutions, London, Leadership Foundation for Higher Education

SRHE Fellow Ian McNay is emeritus professor at the University of Greenwich.