SRHE Blog

The Society for Research into Higher Education

Tag Archives: grade inflation


by Leave a comment

Walk on by: the dilemma of the blind eye

by Dennis Sherwood

Forty years on…

I don’t remember much about my experiences at work some forty-odd years ago, but one event I recall vividly is the discussion provoked by a case study at a training event. The case was simple, just a few lines:

Sam was working late one evening, and happened to walk past Pat’s office. The door was closed, but Sam could hear Pat being very abusive to Alex. Some ten minutes later, Sam saw Alex sobbing.

What might Sam do?

What should Sam do?

Quite a few in the group said “nothing”, on the grounds that whatever was going on was none of Sam’s business. Maybe Pat had good grounds to be angry with Alex and if the local culture was, let’s say, harsh, what’s the problem? Nor was there any evidence that Alex’s sobbing was connected with Pat – perhaps something else had happened in the intervening time.

Others thought that the least could Sam do was to ask if Alex was OK, and offer some comfort – a suggestion countered by the “it’s a tough world” brigade.

The central theme of the conversation was then all about culture. Suppose the culture was supportive and caring. Pat’s behaviour would be out of order, even if Pat was angry, and even if Alex had done something Pat had regarded as wrong.

So what might – and indeed should – Sam do?

Should Sam should confront Pat? Or inform Pat’s boss?

What if Sam is Pat’s boss? In that case then, yes, Sam should confront Pat: failure to do so would condone bad behaviour, which in this culture, would be a ‘bad thing’.

But if Sam is not Pat’s boss, things are much more tricky. If Sam is subordinate to Pat, confrontation is hardly possible. And informing Pat’s boss could be interpreted as snitching or trouble-making. Another possibility is that Sam and Pat are peers, giving Sam ‘the right’ to confront Pat – but only if peer-to-peer honesty and mutual pressure is ‘allowed’. Which it might not be, for many, even benign, cultures are in reality networks of mutual ‘non-aggression treaties’, in which ‘peers’ are monarchs in their own realms – so Sam might deliberately choose to turn a blind eye to whatever Pat might be doing, for fear of setting a precedent that would allow Pat, or indeed Ali or Chris, to poke their noses into Sam’s own domain.

And if Sam is in a different part of the organisation – or indeed from another organisation altogether – then maybe Sam’s safest action is back where we started. To do nothing. To walk on by.

Sam is a witness to Pat’s bad behaviour. Does the choice to ‘walk on by’ make Sam complicit too, albeit at arm’s length?

I’ve always thought that this case study, and its implications, are powerful – which is probably why I’ve remembered it over so long a time.

The truth about GCSE, AS and A level grades in England

I mention it here because it is relevant to the main theme of this blog – a theme that, if you read it, makes you a witness too. Not, of course, to ‘Pat’s’ bad behaviour, but to another circumstance which, in my opinion, is a great injustice doing harm to many people – an injustice that ‘Pat’ has got away with for many years now, not only because ‘Pat’s peers’ have turned a blind eye – and a deaf ear too – but also because all others who have known about it have chosen to ‘walk on by’.

The injustice of which I speak is the fact that about one GCSE, AS and A level grade in every four, as awarded in England, is wrong, and has been wrong for years. Not only that: in addition, the rules for appeals do not allow these wrong grades to be discovered and corrected. So the wrong grades last for ever, as does the damage they do.

To make that real, in August 2025, some 6.5 million grades were awarded, of which around 1.6 million were wrong, with no appeal. That’s an average of about one wrong grade ‘awarded’ to every candidate in the land.

Perhaps you already knew all that. But if you didn’t, you do now. As a consequence, like Sam in that case study, you are a witness to wrong-doing.

It’s important, of course, that you trust the evidence. The prime source is Ofqual’s November 2018 report, Marking Consistency Metrics – An update, which presents the results of an extensive research project in which very large numbers of GCSE, AS and A level scripts were in essence marked twice – once by an ‘assistant’ examiner (as happens in ‘ordinary’ marking each year), and again by a subject senior examiner, whose academic judgement is the ultimate authority, and whose mark, and hence grade, is deemed ‘definitive’, the arbiter of ‘right’.

Each script therefore had two marks and two grades, enabling those grades to be compared. If they were the same, then the ‘assistant’ examiner’s grade – the grade that is on the candidate’s certificate – corresponds to the senior examiner’s ‘definitive’ grade, and is therefore ‘right’; if the two grades are different, then the assistant examiner’s grade is necessarily ‘non-definitive’, or, in plain English, wrong.

You might have thought that the number of ‘non-definitive’/wrong grades would be small and randomly distributed across subjects. In fact, the key results are shown on page 21 of Ofqual’s report as Figure 12, reproduced here:

Figure 1: Reproduction of Ofqual’s evidence concerning the reliability of school exam grades

To interpret this chart, I refer to this extract from the report’s Executive Summary:

The probability of receiving the ‘definitive’ qualification grade varies by qualification and subject, from 0.96 (a mathematics qualification) to 0.52 (an English language and literature qualification).

This states that 96% of Maths grades (all varieties, at all levels), as awarded, are ‘definitive’/right, as are 52% of those for Combined English Language and Literature (a subject available only at A level). Accordingly, by implication, 4% of Maths grades, and 48% of English Language and Literature grades, are ‘non-definitive’/wrong. Maths grades, as awarded, can therefore be regarded as 96% reliable; English Language and Literature grades as 52% reliable.

Scrutiny of the chart will show that the heavy black line in the upper blue box for Maths maps onto about 0.96 on the horizontal axis; the equivalent line for English Language and Literature maps onto 0.56. The measures of the reliability of the grades for each of the other subjects are designated similarly. Ofqual’s report does not give any further numbers, but Table 1 shows my estimates from Ofqual’s Figure 12:

 Probability of
 ‘Definitive’ grade‘Non-definitive’ grade
Maths (all varieties)96%4%
Chemistry92%8%
Physics88%12%
Biology85%15%
Psychology78%22%
Economics74%26%
Religious Studies66%34%
Business Studies66%34%
Geography65%35%
Sociology63%37%
English Language61%39%
English Literature58%42%
History56%44%
Combined English Language and Literature (A level only)52%48%

Table 1: My estimates of the reliability of school exam grades, as inferred from measurements of Ofqual’s Figure 12.

Ofqual’s report does not present any corresponding information for each of GCSE, AS or A level separately, nor any analysis by exam board. Also absent is a measure of the all-subject overall average. Given, however, the maximum value of 96%, and the minimum of 52%, the average is likely to be somewhere in the middle, say, in the seventies; in fact, if each subject is weighted by its cohort, the resulting average over the 14 subjects shown is about 74%. Furthermore, if other subjects – such as French, Spanish, Computing, Art… – are taken into consideration, the overall average is most unlikely to be greater than 82% or less than 66%, suggesting that an overall average reliability of 75% for all subjects is a reasonable estimate.

That’s the evidence that, across all subjects and levels, about 75% of grades, as awarded, are ‘definitive’/right and 25% – one in four – are ‘non-definitive’/wrong – evidence that has been in the public domain since 2018. But evidence that has been much disputed by those with vested interests.

Ofqual’s results are readily explained. We all know that different examiners can, legitimately, give the same answer (slightly) different marks. As a result, the script’s total mark might lie on different sides of a grade boundary, depending on who did the marking. Only one grade, however, is ‘definitive’.

Importantly, there are no errors in the marking studied by Ofqual – in fact, Ofqual’s report mentions ‘marking error’ just once, and then in a rather different context. All the grading discrepancies measured in Ofqual’s research are therefore attributable solely to legitimate differences in academic opinion. And since the range of legitimate marks is far narrower in subjects such as Maths and Physics, as compared to English Literature and History, then the probability that an ‘assistant’ examiner’s legitimate mark might result in a ‘non-definitive’ grade will be much higher for, say, History as compared to Physics. Hence the sequence of subjects in Ofqual’s Figure 12.

As regards appeals, in 2016, Ofqual – in full knowledge of the results of this research (see paragraph 28 of this Ofqual Board Paper, dated 18 November 2015) – changed the rules, requiring that a grade can be changed only if a ‘review of marking’ discovers a ‘marking error’. To quote an Ofqual ‘news item’ of 26 May 2016:

Exam boards must tell examiners who review results that they should not change marks unless there is a clear marking error. …It is not fair to allow some students to have a second bite of the cherry by giving them a higher mark on review, when the first mark was perfectly appropriate. This undermines the hard work and professionalism of markers, most of whom are teachers themselves. These changes will mean a level-playing field for all students and help to improve public confidence in the marking system.

This assumes that the legitimate marks given by different examiners are all equally “appropriate”, and identical in every way.

This assumption. however, is false: if one of those marks corresponds to the ‘definitive’ grade, and another to a ‘non-definitive’ grade, they are not identical at all. Furthermore, as already mentioned, there is hardly any mention of marking errors in Ofqual’s November 2018 report. All the grade discrepancies they identified can therefore only be attributable to legitimate differences in academic opinion, and so cannot be discovered and corrected by the rules that have been in place since 2016.

Over to you…

So, back to that case study.

Having read this far, like Sam, you have knowledge of wrong-doing – not Pat tearing a strip off Alex, but Ofqual awarding some 1.5 million wrong grades every year. All with no right of appeal.

What are you going to do?

You’re probably thinking something like, “Nothing”, “It’s not my job”, “It’s not my problem”, “I’m in no position to do anything, even if I wanted to”.

All of which I understand. No, it’s certainly not your job. And it’s not your problem directly, in that it’s not you being awarded the wrong grade. But it might be your problem indirectly – if you are involved with admissions, and if grades play a material role, you may be accepting a student who is not fully qualified (in that the grade on the certificate might be too high), or – perhaps worse – rejecting a student who is (in that the grade on the certificate is too low). Just to make that last point real, about one candidate in every six with a certificate showing AAA for A level Physics, Chemistry and Biology in fact truly merited at least one B. If such a candidate took a place at Med School, for example, not only is that candidate under-qualified, but a place has also been denied to a candidate with a certificate showing AAB but who merited AAA.

And although you, as an individual, are indeed not is a position to do anything about it, you, collectively, surely are.

HE is, by far, the largest and most important user of A levels. And relying on a ‘product’ that is only about 75% reliable. HE, collectively, could put significant pressure on Ofqual to fix this, if only by printing “OFQUAL WARNING: THE GRADES ON THIS CERTIFICATE ARE ONLY RELIABLE, AT BEST, TO ONE GRADE EITHER WAY” on every certificate – not my statement, but one made by Ofqual’s then Chief Regulator, Dame Glenys Stacey, in evidence to the 2 September 2020 hearing of the Education Select Committee, and in essence equivalent to the fact that about one grade in four is wrong. That would ensure that everyone is aware of the fact that any decision, based on a grade as shown on a certificate, is intrinsically unsafe.

But this – or some other solution – can happen only if your institution, along with others, were to act accordingly. And that can happen only if you, and your colleagues, band together to influence your department, your faculty, your institution.

Yes, that is a bother. Yes, you do have other urgent things to do.

If you do nothing, nothing will happen.

But if you take action, you can make a difference.

Don’t just walk on by.

Dennis Sherwood is a management consultant with a particular interest in organisational cultures, creativity and systems thinking. Over the last several years, Dennis has also been an active campaigner for the delivery of reliable GCSE, AS and A level grades. If you enjoyed this, you might also like https://srheblog.com/tag/sherwood/.

Image of Rob Cuthbert


by Leave a comment

SRHE News on teaching and learning

By Rob Cuthbert

One of the benefits of SRHE membership is exclusive access to the quarterly newsletter, SRHE News, archived at https://www.srhe.ac.uk/publications/. SRHE News typically contains a round-up of recent academic events and conferences, policy developments and new publications, written by editor Rob Cuthbert. To illustrate the contents, here is part of the January 2021 issue which covers Teaching and Learning.

Academic development and Pro VC roles can go together

Fiona Denney (Brunel) reported her research in International Journal of Academic Development (online 13 December 2020) based on interviews with four Pro VCs with academic development backgrounds: “Over the past two years, four research-intensive universities in the UK have appointed senior academic leaders from academic development backgrounds, a new phenomenon in this sector of UK higher education that may suggest a changing pattern. This study interviewed these four leaders to explore what the appointment means for their academic identity. The interviewees identified internal and external drivers for change and noted their backgrounds as academic developers made their routes into these senior roles different from their peers. For this reason, their ‘academic credibility’ was critical in order to implement culture change effectively.”

How metrics are changing academic development

Roni Bamber (Queen Margaret University) blogged for Wonkhe on 18 December 2020 about her monograph for SEDA, Our days are numbered. Great title, good read.

SoTL in action

The 2018 book edited by Nancy Chick was reviewed by Maik Arnold (University of Applied Sciences, Germany) for Arts and Humanities in Education (online 12 October 2020).

Innovations in Active Learning in Higher Education

The new book by SRHE members Simon Pratt-Adams, Uwe Richter and Mark Warnes (all Anglia Ruskin) grew out of an Active Learning conference at Anglia Ruskin University, leading to a book which, in the words of the foreword by Mike Sharples (Open University) “shows how to put active learning into practice with large cohorts of students and how to grow that practice over many years. The authors come from a variety of institutions and discipline areas … What they have in common is a desire to improve student engagement, experience and outcomes, through active learning approaches that work in practice and are scalable and sustainable.” Free to download from the publishers, Fulcrum.

Now that’s what I call a publishing event

The new book by Keith Trigwell (Sydney) and Mike Prosser (Melbourne) Exploring University Teaching and Learning: Experience and Context, was launched on 10 December 2020, more than 20 years since Understanding Learning and Teaching appeared in 1999. The book focuses on university teachers’ experience of teaching and learning, discussing the qualitative variation in approaches to university teaching, the factors associated with that variation, and how different ways of teaching are related to differences in student experiences of teaching and learning. The authors extend the discussions of teaching into new areas, including emotions in teaching, leadership of teaching, growth as a university teacher and the contentious field of relations between teaching and research.

Psychological contract profiling for managing the learning experience of higher education students

László Horváth (ELTE Eötvös Loránd University, Budapest) used a service marketing approach for his article in the European Journal of Higher Education (online 27 January 2020): “Combining … six factors for expectations (personalization, development of soft skills, competent teachers, labour market preparedness, support, flexibility) and three factors of obligations (performance and activity, preciseness and punctuality, obedience and respect), we created Psychological Contract Profile Clusters (outcome-centred, teacher-centred, learner-centred, learning-centred, content-centred and self-centred students).”

“Grade inflation remains ‘a significant and pressing issue’”

That was how the OfS chose to present its analysis of degree outcomes published on 19 November 2020, quoting OfS chief executive Nicola Dandridge. The report itself said the rate of increase in ‘grade inflation’ had slowed in 2018-2019, and buried in the text was this: “It is not possible to deduce from this analysis what factors not included in the modelling (such as improvements in teaching quality, more diligent students or changes to assessment approaches) are driving the observed changes in degree attainment.” No recognition by OfS of the research by Calvin Jephcote (Leicester), SRHE members Emma Medland and Robin Lygo-Baker (both Surrey) published in Assessment and Evaluation in Higher Education, which concluded: “The results suggest a much more positive and proactive picture of a higher education system that is engaged in a process of continuous enhancement. The unexplained variables, rather than automatically being labelled as grade inflation, should instead point to a need to investigate further the local institutional contextual factors that inform grade distribution. The deficit lens through which ‘grade inflation’ is often perceived is a damaging and unhelpful distraction.” Perhaps Nicola Dandridge was auditioning for Queen of Hearts in the OfS Christmas panto: “Sentence first, verdict afterwards”.

Jephcote, Medland and Lygo-Baker had also blogged for Wonkhe on 14 October 2020 about their research: “Evidence for why grades are trending upwards, or the less loaded phrase of grade improvement, reveal a complex landscape. According to our recent research, the most influential determinants of grade improvement were shown to be the geographic location of an institution, research output quality and the increasing quality of student cohorts – although even this variable was determined on grade entry points, which the recent A Level debacle in the UK has pulled into question. … What this evidence reveals is that a combination of student aptitude, and changes to the structure and quality of UK higher education, appear to be largely accountable for graduates attaining higher grades. It also, importantly, points to the problems associated with our criterion-referenced approaches to assessment being critiqued using a norm-referenced rationale.”

Peer review of teaching in Australian HE: a systematic review

The article by Alexandra L Johnston, Chi Baik and Andrea Chester (all Melbourne) was in Higher Education Research and Development (online 18 November 2020) “A thematic synthesis revealed teaching development outcomes gained through peer review of teaching span factors at organisational … program … and individual … levels. Organisational factors included disciplinary context, program sustainability, collegiality and leadership. Program factors included framework, program design, basis of participation, observation, feedback and reflective practice. Factors at the individual level included prior experience and participants’ perceived development requirements.”

What do undergraduate students understand by excellent teaching?

SRHE member Mike Mimirinis (West London) published the results of his SRHE-funded research in Higher Education Research and Development (online 21 November 2020): “This article explores undergraduate students’ conceptions of what constitutes excellent teaching. … semi-structured interviews with students at two English universities yields five qualitatively different conceptions of excellent teaching. In contrast to the current intense policy focus on outcome factors (eg graduate employability), students predominantly discern process factors as conducive to excellent teaching: how the subject matter is presented, what the lecturer brings to the teaching process, how students’ personal understanding is supported, and to what extent the questioning and transformation of disciplinary knowledge is facilitated. More importantly, this study demonstrates that an expansion of students’ awareness of the nature of teaching is internally related to the expansion of their awareness of the nature of disciplinary knowledge.”

The German sense of humour

The article in Studies in Higher Education (online 3 June 2019, issue 2020:12) was based on two large surveys of how teachers used humour in their teaching, and how students responded. It seems to come down to what the teachers meant by using humour. The research was by Martin Daumiller and three other colleagues at Augsburg.

Teaching in lifelong learning: A guide to theory and practice

The third edition was published in 2019, edited by James Avis, Roy Fisher and Ron Thompson (all Huddersfield).

A conceptual framework to enhance student learning and engagement

Alice Brown, Jill Lawrence, Marita Basson and Petrea Redmond (all Southern Queensland) had an article in Higher Education Research and Development (online 28 December 2020) about using course learning analytics (CLA) and nudging strategies, based on “a 12-month research project, as well as by the theoretical perspectives presented by communication and critical literacies. These perspectives were applied to develop a conceptual framework which the authors designed to prioritise expectation management and engagement principles for both students and academics. The article explains the development of the framework as well as the elements and key communication strategies it embodies. The framework contributes to practice by explaining and justifying the accessible, time-efficient, student-focused approaches that can be integrated simply into each course’s online learning pedagogy to support both academics’ and students’ engagement.”

Rob Cuthbert is the editor of SRHE News and Blog, emeritus professor of higher education management, Fellow of the Academy of Social Sciences and Fellow of SRHE. He is an independent academic consultant whose previous roles include deputy vice-chancellor at the University of the West of England, editor of Higher Education Review, Chair of the Society for Research into Higher Education, and government policy adviser and consultant in the UK/Europe, North America, Africa, and China. He is current chair of the SRHE Publications Committee.

Ian Mc Nay


by Leave a comment

Ian McNay writes …

By Ian McNay

The news from Ukraine is that, at least in Odesa (one ‘s’ in Ukrainian) market, my country is known as ‘Bye, Bye, Britain’. I was there as part of a project on developing leadership training. At the rectors’ round table, we were thanked by the British Council rep. for being honest. We were discussing HE governance, and lessons from the UK, without doing the usual thing of pretending our approach is wonderful and everybody should imitate it. We learn from mistakes more than from things that went well, perhaps because they imply that there is a need to learn.

One challenge in Ukraine is the nostalgia for the old days. When I first went there 20 years ago, I asked an undergraduate class for their models of good leaders. My first three answers were Hitler, Stalin and Thatcher, which led to a discussion of the difference between ‘strong’ and ‘good’. That preference for strength over everything else is still there. In a survey of the ex-Soviet republics, the question was asked: ‘would you rather have democracy or a dictator who solves problems?’ Ukraine topped the table of those opting for the second, with over 50% choosing efficient despotism. The Czech Republic scored only 13%.

This is relevant to us because Theresa May has been claiming to be strong and has resisted the operations of democracy. At organisational level, since power tends to corrupt, the signs are not good: a recent survey of UK managers for the Chartered Institute of Personnel and Development revealed that only 8 per cent claimed to have a strong personal moral compass, and so are susceptible to corruption. Even UK university managers would score better than that, despite the disappearance of collegial democracy.

Wouldn’t they?

Did you notice…? The Universities UK blog reported a survey of the teams who prepared the institutional submissions to the Teaching Excellence Framework, and found that even they were dismissive of its validity and reliability – basic requirements for us as researchers. 72 per cent of those most closely involved in the exercise did not believe that it ‘accurately assesses teaching and learning excellence’. Only 2 per cent, 2 per cent, thought it did. Even they might change their view, since the views of students – those ‘at the heart of the system’ and the alleged beneficiaries of the exercise – are to be given a lower weighting, since their voice, through NSS, gave the ‘wrong’ message. More weight will now be given to post-graduation data on jobs and earnings, which are more heavily conditioned by accidents of birth, and employer prejudice, than the quality of teaching and learning. So much for promoting social mobility, another claimed objective. Russell Group universities will benefit, since they scored poorly on NSS, and recruit more of those privileged by birth. That couldn’t be a reason for the change, surely? That would suggest that corruptive pressure had been applied to the reward process, as in the awarding of Olympic Games to cities or the football world cup to countries. Or in awarding Olympic medals – gold, silver, bronze – in boxing. Or bonuses to bankers. Still, footballers and bankers are now our benchmarks, according to the head of the world’s leading university, so we still have some way to fall.

Don’t we?

‘That way madness lies’ (I have just played Lear in a local ‘Best of the Bard’ concoction).

Recent reports from some universities suggest grade inflation is just as much an issue as the cost of living index. UK wide figures are not yet available for the latest batch of graduates, but in 2016, 73 per cent of first degree graduates got a first (24%) or upper second (49%), with the gender split favouring women by 75/71. Four years previously, the figure had been ‘only’ 66 per cent. So, despite expansion lowering entry tariffs, more ‘value’ is added to compensate. If 50 per cent of an age cohort now study for a degree, that means that 12 percent of an age group got a first class degree. A few years ago, when I passed the 11+, only 11 percent of the age group in my home town did so.

Did you notice the figures for ‘alternative providers’ from HESA, interesting in the light of the recent report from the HE Commission? Of the 6,200 graduates they produced (2,000 more than the previous year), 58 per cent got ‘good’ degrees. No Inflation – it was 61 per cent in 2015. 14 per cent got firsts, and women again outperformed men, by nine percentage points – 63/54.

The Commission’s report goes well beyond simply comparing the provision of full-time first degrees, emphasising the potential role of apprenticeships in adding to diversity of routes; urging flexibility of funding to allow flexibility of study patterns across the sector and outlining the greater part employers should play in developing work-related and work-relevant provision. I was interested that, of over 120 names on the attendance list, only 6 were from mainstream universities, and three of those had given evidence to the enquiry. Does the sector not think there is a challenge from the alternatives? Will they just wait for the demographic upturn early in the next decade, and then supply the same-old to a similar sub-set of the market? Are they aware that some of that demographic upturn is of children of EU immigrants who may well choose to return to their parents’ home country to study where fees are much lower, if they exist at all? And that nearly all recent growth in demand has been from BAME applicants, who suffer from admissions decisions which imply unconscious (I hope) decisions, particularly in elitist universities, as work by Vicki Boliver and Tariq Modood and statistics from UCAS show?

Finally, and still on my campaign for equity…I have a plea. At a recent symposium, participants commented on the inequity, at a global level, of the monopoly role of the English language, which has an exclusionary impact on those outside the Anglo-Saxon countries. Some national governments are bothered about its impact on knowledge transfer within the country that sponsored the work that produces journal articles. My suggestion is that any journal with ‘international’ in its title or its statement of aims should publish abstracts in, preferably, three languages, but at least two: the second being the author’s first language or that of the host institution of the research reported; the third another global language, probably Spanish. So, if you are on the editorial board of journals, or review articles submitted, can I urge you to make representation about this. It would enhance awareness across a broader landscape of HE, and allow those beyond the current privileged language enclave initial access to relevant work and to follow up with some contact with authors, since email addresses are now commonly given. It would also support the Society’s role in encouraging newer researchers. Simples!

SRHE Fellow Ian McNay is emeritus professor at the University of Greenwich