SRHE Blog

The Society for Research into Higher Education

Tag Archives: University Admissions


by Leave a comment

Pragmatic problem-solving for inclusive doctoral admission

by Bing Lu, Rebekah Smith McGloin and Scott Foster

This blog post reflects on ongoing collaborative efforts to advance more equitable doctoral admissions between a group of UK institutions. It argues that transforming graduate admissions is not simply driven by competitive logic, nor by a search for a single, universal framework that can be applied across the sector. Instead, sector-level change emerges through collective, interactional, and often emotional work.

Inclusive postgraduate research (PGR) admission and recruitment have become an increasing global concern (Posselt, 2016; Bastedo, 2026; Boghdady, 2025). Drawing on ongoing collaborative work between a group of UK institutions, this blog post reflects on collective efforts to advance more equitable doctoral admissions. We argue that inclusive doctoral admission is not a competition to produce an exhaustive, finished framework, but an ongoing process of collective problem solving, one that requires humility, openness, and sustained commitment across institutional boundaries.

PGR students are strategically vital to the UK’s research capacity, innovation and future academic workforce. PhD programmes increasingly function as the primary entry route into academic careers and shape who is able to imagine themselves, and be recognised, as future researchers. Within the doctoral lifecycle, admission is a particularly critical intervention point. Yet, compared with undergraduate or taught postgraduate recruitment, the mechanisms shaping PGR admissions have historically received less sustained scrutiny.

A report commissioned by the Higher Education Funding Council for England (HEFCE) in 2014 highlighted that UK institutions primarily value academic attainment, the quality of research proposals, and evidence of prior research skills when selecting candidates (Mellors-Bourne et al, 2014). Since 2020, a growing body of UK-based scholarship has begun to highlight equity issues in doctoral selection (McGloin & Wynne, 2022; Oyinloye & Wakeling 2023; Mateos‑González & Wakeling, 2022; Britton et al, 2020), and has sought to explore the ascriptive nature of systems and processes that underpin doctoral recruitment and admission.  Together, these studies identify a range of barriers. These include the persistence of ‘elite pipelines’, whereby attending a Russell Group university at undergraduate level strongly predicts access to elite postgraduate education, as well as the significant under-representation of British candidates from minoritised backgrounds at doctoral level, particularly within funded studentships. These patterns underscore the need to interrogate how merit, potential, and excellence are operationalised in practice.

The initiatives and the community of practice

Initiatives funded by Research England and Office for Students, including the Equity in Doctoral Education through Partnership and Innovation (EDEPI) programme, represent important attempts to push forward the agenda of inclusive PGR admissions in English Higher Education Institutions (HEIs). In 2022, EDEPI conducted a national survey on PGR admissions practices in UK HEIs. The study identified ten key barriers to inclusive admission in its final report EDEPI Postgraduate Researcher Admission Framework and led to the development of the Postgraduate Researcher Competency-Based Admission Framework. This framework deliberately shifts focus away from previous institutional prestige and historical academic attainment towards the specific skills, experiences and competencies which demonstrate future potential for doctoral research.

From 2024, EDEPI has fostered an inter-institutional Community of Practice involving a group of international and UK institutions to explore approaches for enhancing inclusive PGR admissions collectively. Within this community, three institutions engaged as case studies to trial new approaches to evaluating applicants beyond conventional academic metrics, building on the Competency Framework. Through regular facilitated discussions, shared reflective practices, collaborative webinars and a jointly organised symposium on Fostering inclusive doctoral admission, participating institutions work alongside the EDEPI team to explore challenges and embed equity-driven principles into their PGR admissions processes.

Key learning from collective work

One of the most important lessons drawn from this collective institutional effort is that, while institutions hold different conceptions of fairness and merit shaped by their unique contexts, they nonetheless share a commitment to addressing persistent equity issues. This aligns with the findings of the sector survey (Smith McGloin et al, 2024) which found an overwhelming commitment to inclusive practice, an awareness of the need for change and huge complexity in existing processes with multiple stakeholders and drivers. This work is neither straightforward nor purely normative; it is complex, negotiated, and deeply pragmatic.

For example, in staff training workshops, academic colleagues described their deliberate efforts to apply equity principles when making departmental admissions decisions. Professional services staff, meanwhile, highlighted their role in carefully matching applicants’ proposals and disciplinary backgrounds to appropriate departments, ensuring that applications reach the review stage rather than being filtered out prematurely. Where resistance or hesitation arose around the introduction of yet another ‘framework’, this was less about rejecting equity goals and more about uncertainty regarding feasible, appropriate, and sustainable implementation.

Debates around distributive fairness versus procedural fairness illustrate this tension clearly (Boliver et al, 2022). Graduate admissions are not objective measurements of worth but sites of intense organisational boundary work, where judgements about potential, fit, and excellence are continuously negotiated. These discussions echo longstanding sociological insights into academic evaluation. Lamont (2009), for instance, argues that in real-world academic review, excellence and diversity are not alternative principles but additive ones. Staff involved in PGR admissions are often guided by pragmatic, problem-solving considerations, caught between institutional principles, personal commitments, and procedural constraints. Panels are typically required to reach consensus on a limited number of candidates within tight timeframes, and these practical pressures shape how fairness is understood and enacted.

Within this ‘black box’ of academic decision-making, Bourdieu’s Homo Academicus is frequently cited to explain how scholars’ legitimate visions of high-quality research and defend disciplinary boundaries, with conflicts often most pronounced among those occupying similar positions. Our collective work over the past 12 months, however, suggests a more nuanced picture. Admissions staff, both academic and professional, are motivated not only by positional interests but also by a shared, pragmatic curiosity about how to solve persistent problems together. The Community of Practice created space for dialogue, uncertainty, and learning, enabling participants to reflect on their own assumptions while engaging with others’ institutional constraints.  Transforming graduate admissions, then, is not simply driven by competitive logic, nor by a search for a single, universal framework that can be applied across the sector. Instead, sector-level change emerges through collective, interactional, and often emotional work. A recent WonkHE article, How to level the PhD playing field, posed a critical question: does the sector have the collective will to move beyond well-intentioned initiatives towards the structural changes required to address inequities among PGRs?

The experiences emerging from EDEPI offer cautious but promising evidence. They demonstrate how institutions with differing histories, resources, and institutional affordances can nonetheless work together pragmatically to enhance admissions practices. Inclusive doctoral admission, in this sense, is not a finished model to be adopted but an ongoing process of collective problem solving, one that requires humility, openness, and sustained commitment across institutional boundaries. Through the established Community of Practice, the EDEPI framework has also begun to attract interest from institutions in international contexts, despite differing governance structures, as a means of collectively developing equity-oriented approaches to PGR admissions through shared learning.

Closing summary

Inclusive PGR admissions require ongoing, collaborative work, as shown through EDEPI’s efforts to help institutions rethink how fairness, potential, and merit are assessed. Colleagues across academic and professional roles demonstrate that excellence and diversity can be mutually reinforcing when supported by reflective practice and shared experimentation. Future progress depends on refining competency-based approaches, tracking applicant journeys, expanding training and co-creation, and translating these insights into clearer sector guidance and policy.

Dr Bing Lu is a higher education scholar based at Nottingham Trent University and University of Warwick. Bing’s research critically engages with access, equity, and sustainability in postgraduate education, focusing particularly on underrepresented groups and the global flows of academic labour. Bing is currently guest editing a Special Issue on Taboos in Doctoral Education Across Cultures hosted by Higher Education Quarterly.

Dr Rebekah Smith McGloin is Director of Research Culture and Environment at Nottingham Trent University and Chair of the UK Council for Graduate Education. Her focus is on innovations in practice and national policy work related to new and emerging forms of doctorate that align with the changing research, innovation and skills policy landscape; including research culture reform, civic-engaged and inclusive doctoral education and equity-focused admissions.

Scott Foster is a professor specialising in postgraduate research culture and academic leadership. He has published extensively on equity, well-being, and innovation in doctoral education. Through influential articles and forthcoming book projects, he advances global research culture while supporting institutions to strengthen policy, supervision, and the doctoral experience.

Paul Temple


by 2 Comments

Nostalgia isn’t what it used to be

by Paul Temple

The thing about golden ages is that people mostly don’t realise that they’re living through one, expecting the nice stuff that’s happening to go on for ever. In retrospect, the Institute of Education, when it was a more-or-less autonomous college of the University of London, was an almost-nearly-perfect university institution. It was big enough – about 400 academic staff when I was there – but not too big; although of course you didn’t know everyone personally, you were on a nodding acquaintance with a lot of them. It was academically-focused, naturally, on education, so there was a common intellectual thread, but, given the nature of education as an area of study, this meant that a range of disciplines – philosophy, economics, sociology, psychology, history, and more – were in the mix, producing a stimulating creative setting. And that was before you counted the teacher-training specialists: I was in awe of my colleagues who were able to give fresh-faced graduates the confidence to stand in front of London school or college classes. So there was a lot going on, all packed into a few Bloomsbury buildings.

This function of scale, intellectual connectivity, and physical proximity produced a high level of social capital, in particular the kind that the political scientist Robert Putnam (2000) has called “generalised reciprocity”. This means that you are in a setting where people you don’t know personally will (probably) help you when required, and you in turn will do your best to help someone else who may be unknown to you. This implies the existence of large reserves of trust, which in turn (as many studies have shown – see for example Fukuyama (1995)) promotes organisational efficiency. Things get done quickly without lengthy discussion: I might ask the Academic Registrar to bend the rules on student admissions, for example, and she would probably agree because she thought I was a basically sensible bloke. Well, I think that’s what she thought: the example I have in mind involved accepting someone on to a master’s course whose highest academic achievement was five O-levels. As the student concerned went on, with a little help from us, to become a PVC in a major university, I think you might say that we called it right.

When the discussions began about the Institute merging with UCL – it finally took place at the end of 2014 – colleagues sometimes asked those of working on higher education policy and management what we thought of the idea: some of us had actually studied previous university mergers (for example, Shattock, 2010; Temple, 2002; Temple and Whitchurch, 1994). I usually replied that there would be good news and bad news. The good news was that we’d be part of a world-leading university in a global city, so that student recruitment problems would be a thing of the past, and it wouldn’t do research earnings any harm either. (While the Institute’s stand-alone reputation attracted plenty of interest from international students, the Curse of the Rankings – as a single-faculty institution, the IoE wasn’t included – meant that many found it difficult to obtain financial support from their home countries.) The bad news was that, as a small cog in a big machine, the IoE would have to fit in with UCL ways of working, which would be unlike those we had all been used to. Things would be done in more bureaucratic ways (I use the word in its sociological sense, not as a lazy insult), in a low-trust environment without the social capital-rich networks that we’d known previously. Also, the IoE top brass who had negotiated the merger deal (“Don’t worry, it’ll work out just fine”) would move on, and their replacements would be appointed by UCL to manage what had become just another UCL faculty. Sympathy with quirky IoE methods wouldn’t be in their job descriptions.

I think I’ve been proved right.

References

Fukuyama, F (1995) Trust: The Social Virtues and the Creation of Prosperity London: Hamish Hamilton

Putnam, R (2000) Bowling Alone: The Collapse and Revival of American Community New York, NY: Simon and Schuster

Shattock, M (2010) Managing Successful Universities (2nd ed.) Maidenhead: SRHE & Open University Press

Temple, P (2002) ‘Reform in a Fragmented System: Higher Education in Bosnia-Herzegovina’ Higher Education Management and Policy, 14 (2), 87-98

Temple, P and Whitchurch, C (1994) ‘An International Perspective: Recent Growth Mergers in British Higher Education’ in Martin, J and Samels, J (eds) Merging Colleges for Mutual Growth Baltimore: The Johns Hopkins University Press

Dr Paul Temple is Honorary Associate Professor in the Centre for Higher Education Studies, UCL Institute of Education.

Image of Rob Cuthbert


by 6 Comments

Mr Sherwood v The Office of Qualifications and Examinations Regulation[1]

[1] The ITV programme ‘Mr Bates v The Post Office’ was shown on British TV during the first week of January 2024 and has generated in the UK a media firestorm and a swift government response. Those, probably mostly outside the UK, who are unfamiliar with the story might like to read this explainer from Private Eyebefore reading this editorial. Or just Google it.

by Rob Cuthbert

Mr Sherwood, you’re the only one who’s been reporting these problems …

We have complete confidence that our system is robust.

This is a story of injustice on a massive scale, over a long period. The story of someone affronted by the unfairness who refused to give up, even though the authorities lined up to oppose him and try to make him go away. A story which has not yet attracted the attention it seems to deserve, given the way it affects the lives of tens of thousands of people who put their faith in a flawed system.

Every year a new group of tens of thousands of people are subject to the same repeated injustice. Most of them have no idea that they might have been unfairly treated. If they try to use official procedures for complaint and recompense most of them will fail. The authorities’ repeated mantra is that the system is ‘the best and fairest way’.

It could be, but it isn’t. And one person’s attempts to make things better have been met with denial, opposition, obfuscation, and the use of official processes to discourage media attention, by a public agency which is “independent of government”.

The Office of Qualifications and Examinations Regulation (Ofqual) is charged with regulating and maintaining standards and confidence in GCSEs, A levels, AS levels, and vocational and technical qualifications. Ten years ago Ofqual were aware of some potential problems in grading. To determine the extent of the problem, they took entire cohorts of GCSE, AS and A Level scripts and re-marked them, comparing the marks given by an ordinary examiner to comparable re-marks given by a senior examiner. Eventually this led to two careful and scholarly reports: Marking Consistency Metrics in 2016 and Marking Consistency Metrics – An Update  in 2018.

The reports showed varying reliability in the grades awarded by examiners, compared with the ‘true’ or ‘definitive’ grade awarded by a senior examiner. Dennis Sherwood, an independent analyst and consultant, interpreted Ofqual’s measurements of grade reliability as a consequence of what he termed ‘fuzziness’. Fuzziness is the range around a senior examiner’s ‘definitive’ mark that contains the ‘legitimate’ marks given by an ordinary examiner. The 2018 report found that grades for, say, English and History are much less reliable than those for Maths and Physics. In Sherwood’s terms, the ‘fuzziness’ of the marks associated with English and History is greater than for Maths and Physics.

Problems arise when a marking range straddles a grade boundary. For example, if a script is legitimately marked in a range from 38-42, but a grade boundary is set at 40, then more than one grade could result from that one script, depending on who marks it and how. Ofqual have admitted that this is the case:

“…more than one grade could well be a legitimate reflection of a student’s performance and they would both be a sound estimate of that student’s ability at that point in time based on the available evidence from the assessment they have undertaken.” (Ofqual, 2019).

The 2016 report says: “… the wider the grade boundary locations, the greater the probability of candidates receiving the definitive grade.” GCSEs have nine grades plus unclassified, and A-levels have six plus unclassified, meaning grade widths are inevitably narrower than, for example, university degree classifications with just four plus fail. With comparatively narrow grade widths more candidates will be close to a boundary. In other words, and however good the marking is, grading for many candidates will not always give a ’true’ or ‘definitive’ grade.

This situation is admitted by Ofqual and has been known for more than five years, since the 2018 Report. Dr Michelle Meadows, formerly Ofqual’s Executive Director for Strategy, Risk and Research, said in evidence to the House of Lords Education for 11-16 year olds Committee (2023) on 30 March 2023:

“It’s really important that people don’t put too much weight on any individual grade. … I know, unfortunately, that a lot of weight is placed on particular GCSEs for progression, maths and English being the obvious ones. In maths that is less problematic because the assessment in maths is generally highly reliable. In English that is problematic. This is not a failure of our GCSE system. This is the reality of assessment. It is the same around the world. There is no easy fix, I am afraid. It is how we use the grades that needs to change rather than creating a system of lengthy assessments.” (emphasis added).

Dame Glenys Stacey, Ofqual’s Chief Regulator until 2016, was reappointed as Acting Chief Regulator after the departure of Sally Collier in the aftermath of the 2020 results, and she said in 2020 (House of Commons Education Committee, 2020a: Q1059):

“It is interesting how much faith we put in examination and the grade that comes out of that. We know from research, as I think Michelle mentioned, that we have faith in them, but they are reliable to one grade either way.”  (emphasis added)

According to Ofqual’s own research, we have a national system of grading that is only 95% reliable – and then only if you accept that grades are reliable within plus or minus a grade. The problem is that most people use grades more precisely than that. If you don’t get a grade 4 or above in GCSE English or Mathematics, you may be allowed to progress to educational routes post-16, but you must take a resit alongside your next phase of study, and will not be allowed to continue if your resit grade is still 3 or below. If you miss out by just one grade at A-level, your chosen  university may reject you. Although marking meets the best international standards, grading still contains much individual unfairness. That means many students may miss out on their preferred university, be forced to wait a year to try again, or decide not to enter higher education at all.

We know this mainly because of the efforts of Dennis Sherwood, who started writing about problems with grading five years ago. Sherwood’s analyses attracted media attention but often his findings were rejected by Ofqual, for example in Camilla Turner’s Daily Telegraph report of 25 August 2018, when an Ofqual spokesman was quoted as saying: ‘Mr Sherwood’s research is “entirely without merit” and has drawn “incorrect conclusions”’ (Turner, 2018).

Ofqual tried to shut down Sherwood’s commentaries, and complained to the Independent Press Standards Organisation (IPSO) about a Sunday Times article headlined ‘Revealed – A-level results are 48% wrong’ published on 11 August 2019. IPSO’s finding upheld the complaint, but only on the narrow grounds that the newspaper had not made it sufficiently clear that the use of the word ‘wrong’ was the newspaper’s, and not Ofqual’s, characterisation of the research. However the IPSO ruling said:

“It was not significantly misleading to report that 48% of grades could be “wrong”, in circumstances where the research indicated that, in 48% of cases, a senior examiner could have awarded a different grade to that awarded by the examiner who had marked the paper. The complainant had accepted that different grades could be awarded as a result of inconsistencies in marking, but disagreed with the characterisation of the research which had been adopted by the publication.”

Sherwood’s argument has never been refuted. Ofqual, with its statutory responsibility to maintain public confidence in qualifications, was trying to ignore or attack stories that ‘one grade in four is wrong’. That tactic might have succeeded, were it not for Covid. The story of the infamous examinations algorithm, ultimately abandoned, need not be repeated here. However it showed, first, that few parents and indeed teachers understood how the grading system worked. Secondly, Ofqual’s defence of the flawed 2020 algorithm was so focused on the collective unfairness of grade inflation between one year and the next that they failed to recognise that their ‘solution’ moved grading from a national competition to an intensely local one. That made individual unfairnesses very visible, there was a public outcry and the algorithm was abandoned. Individual unfairness in grading persists – but has reverted to its former obscurity.

Dennis Sherwood accordingly wrote a book, Missing the Mark, which I reviewed for HEPI, setting out his arguments in detail. It seemed to be persuading more in the educational media to give his arguments the space they deserved. He was no longer entirely alone, with a small group (including me) finding his arguments convincing. Support from various media, notably the HEPI blog, gave him space to make his argument. However, as in the case of Mr Bates and the Post Office, there were still just a few individuals ranged against the forces of Ofqual and (some of) the educational establishment.

On 8 June 2023 I wrote ‘If A-level grades are unreliable, what should admission officers do? for HEPI, arguing that universities should recognise the limited reliability of A-level grades by giving candidates the benefit of the doubt, uplifting all achieved results by one grade. That blog was perhaps provocative but it did at least recognise the problem and suggest a short-term fix. My 2020 explanation about the algorithm had become the most-read HEPI blog ever, and I was invited, as I had been every year since 2020, to contribute a further blog to HEPI, to be published near to A-level results day. My follow-up to the June blog advised students and parents how to respond if they had fallen short of an offer they had accepted. I submitted it to HEPI but it was not accepted. HEPI did however publish a blog by one of its trustees, Mary Curnock Cook, on 14 August, the Monday before results day on Thursday.

Curnock Cook is the widely-respected former head of UCAS. She began:

“In this blog, I want to provide some context and challenge to two erroneous statements that are made about exam grades:

  • That ‘one in four exam grades is wrong’
  • That grades are only reliable to ‘within one grade either way’”

She asserted that the statement ‘one in four exam grades is wrong’ was a ‘gross misunderstanding’, but then said:

“In many subjects there will be several marks either side of the definitive mark that are equally legitimate. They reflect the reality that even the most expert and experienced examiners in a subject will not always agree on the precise number of marks that an essay or longer answer is worth. But those different marks are not ‘wrong’.”

In other words, as admitted by Ofqual, more than one grade could be a ‘legitimate’ assessment of the outcome for an individual. Huy Duong, another critic of the 2020 algorithm, had been widely quoted in the media in 2020 after he predicted the exact outcomes of the algorithm a week before the publication of results. He commented on Curnock Cook’s blog:

”… a lot of this is simply playing with words … whichever definitions of ‘wrong’ and ‘rights’ the establishment chooses to use, it is irrefutable that students are subjected to a grade lottery … If, as the author and the establishment contend, for a given script, both “Pass” and “Fail” are equally legitimate, then for the student’s certificate to state only either “Pass” or “Fail”, that certificate is stating a half truth.”

Curnock Cook then addressed the supposedly ‘erroneous’ statement that “grades are only reliable to ‘within one grade either way” – the statement made by Glenys Stacey as Chief Regulator – saying:

“Some commentators have chosen to weaponise this statement in a way that shows poor understanding of the concepts underpinning reliable and valid assessment and risks doing immense damage to students and to public confidence in our exam system.” 

How it is that Sherwood’s analysis shows ‘poor understanding’ is not explained. On the contrary, he seems to have a clear understanding of what Ofqual themselves have admitted. Curnock Cook said the claim about reliability had been taken out of context, but the context is not international tests of collective grading reliability, but the way universities and individual students actually use the grades.

Curnock Cook’s blog was welcomed by influential commentators like Jonathan Simons of Public First, a government favourite for research and PR, and some educationists such as Geoff Barton of the Association of School and College Leaders. She said that talking about unreliable grades “risks doing immense damage to students and to public confidence in our exam system”. Indeed it does, but the risk lies not in pointing out that the emperor has no clothes. The real risk is in not changing the system which remains unfair to so many individuals. The emperor still has no clothes, and it is time to redress things.

Most people who suffer injustice in grading do not even know it has happened. For individuals who do know, most will find that using official procedures to complain or appeal is expensive, and unlikely to change the outcome. In his campaign to illuminate the problem Mr Sherwood, like Mr Bates, met denial, opposition and the use of official processes to discourage the media from continuing to cover the story. People in the organisations concerned know how the system actually works, but they don’t want it to be widely known, for the sake of public confidence in the system. Groupthink puts collective inter-cohort ‘fairness’ ahead of fairness to every individual in every cohort. There was even, in 2020, blind faith in a computer system which was later proved to be faulty.

Public confidence in the qualifications and examinations system is of course absolutely vital. But the need for public confidence does not mean that individual unfairness on a large scale should be tolerated and ignored. There are several possible solutions to the problems of grading unreliability, and many would have little direct cost. HE institutions would have to take even greater care in using grades, as part of their wider assessment of the potential and abilities of candidates for their courses. That is a small price to pay for maintaining public confidence in a national system which everyone could be proud of for its fairness as well as its international standing.

This editorial draws on my article first published in The Oxford Magazine No 458, ‘Maintaining public confidence in an unfair system – the case of school examination grades’, and uses some parts of the text with permission.

Rob Cuthbert is Emeritus Professor of Higher Education Management, University of the West of England and Joint Managing Partner, Practical Academics rob.cuthbert@btinternet.com. Twitter @RobCuthbert


by Leave a comment

Make the tacit explicit: how to improve information on university webpages for potential doctoral applicants

by Dangeni, James Burford and Sophia Kier-Byfield

Working out how to apply for a doctoral programme can be a challenging process for many potential applicants. As countless Youtube videos, blogposts and twitter threads attest, there is much confusion and plenty of (sometimes contrasting) advice on the internet about what to do, whom to contact, and how to contact them. Some applicants find this process so challenging that they turn to a range of paid services that help them to learn how to contact a potential doctoral supervisor or develop a research proposal. There is clearly much demand for guidance on how to make a successful application to doctoral study, but for many academics and professional services staff doctoral admissions is a familiar and routine process where quick assessments can be made about the enquiries of a given applicant.

We began our recent exploratory research project, ‘Opening up the Black Box of Pre-application Doctoral Communications’, with an interest in the somewhat opaque processes that occur prior to formal doctoral admissions, but which often form a crucial part of the pathway to applying. We were concerned that the mysteries of the pre-application stage may have Equity Diversity and Inclusion (EDI) implications, making it easier for some to navigate toward doctoral study than others. We conducted a study to examine the pre-application stage of doctoral admissions in a single university context, the University of Warwick.

At the start of our study, we conducted a search for public-facing, institutional webpages relevant to doctoral admissions. Webpages are one of the key spaces where potential doctoral applicants can gather information about the application process, including institutional pages (eg produced by a Doctoral College) and departmental pages (eg departmental guidance or a potential supervisor’s webpage).

We aimed to identify and characterise information aimed at doctoral applicants prior to their making formal applications to study. Our primary goal in conducting this review was to understand: (a) the nature of pre-admissions information on university webpages; (b) whether this pre-admissions information was consistent across the institution; and, (c) whether the detail was a sufficient and adequate explanation of key pre-application steps to potential applicants.

This blog post gives our top six tips for stakeholders involved in doctoral admissions to consider for potential applicants, so that they have all the information they need from public-facing web pages.  

  1. Avoid complicated web designs, texts and duplicated material

All the webpages we reviewed provided ‘opening pages’ which covered the basic details and specifications of programme, but there was a wide variety of detail in terms of the introductions to departments. Some departments included short paragraphs, others offered more elaborate introductions which included orienting students to the research areas of the department, the ranking of the department in UK league tables, and student testimonials with multiple tabs and long paragraphs, sometimes with invalid links. These layouts can be confusing on a computer screen, but institutions and departments could also consider that potential applicants may use phones or tablets to access the information, and thus the webpage design should be tablet/phone friendly. It is also necessary to check whether the page is accessible, eg for visually impaired visitors or those with learning difficulties.

  • Display a checklist and flowchart for the pre-admissions process

We found two categories of admission information across department webpages. The first category was a link to signpost applicants to the central university portal for application advice and guidance, which provides an overview of the pre-application procedure for potential applicants to follow. The second category of admission information is commonly more tailored to a department’s specific procedures and is often accessed via a ‘how to apply’ section. However we noted that several departments did not undertake much departmental level ‘translation’ of general admissions information, perhaps simply linking applicants to the central university portal. We do not believe this would give potential applicants sufficient information to know how to get started and what to do in local contexts. In particular, decision-making related information and explanations were rare: very few departments explained details such as evaluation criteria, who is involved, the maximum cohort size each year, and the timeframe for decision-making. Therefore, we suggest that departmental webpages should consider displaying a checklist of key steps and a flowchart explaining the timeframe, decision-making process and who is involved.

  • Outline what is expected from applicants in terms of locating a supervisor before applying

Most departmental webpages advise applicants to contact prospective supervisors in advance of the application to discuss research interests and compatibility, although some do not require a nominated supervisor for application. Our web review identified that most departments do consider this process to be a key pre-application step, and some provide relevant information and guidance regarding how to identify a supervisor. Therefore, a clear indication of what is expected from applicants to contact prospective supervisors is needed on departmental webpages. Additionally, institutions/departments should encourage academic staff to update their staff profile web pages with consistent information eg current projects supervised, information on interests (topic, methodology/approach, country contexts, and capacity to take on new students) would be helpful for applicants in the preparation and communications stages.

  • Explain what counts as a ‘good’ research proposal

Another key category of pre-application information concerns how to draft a research proposal. Most departments require a research proposal for an application to be considered; research proposal-related guidance can be found in two categories. Most departments link to the central university portal for application advice and guidance, which contains the general structure of a research proposal (eg an overview of research question(s), main objective of research, potential contribution to existing research field/literature, research techniques, suggested data collection procedures and an outline timeline) and a list of department requirements. In contrast, in several departments, a webpage or a link to department-specific guidance can be found, providing an outline/structure with word count and what to include in detail. It is suggested that clear guidance on a ‘good’ research proposal (disciplinary equivalent) is necessary for applicants, including information on expected sections and length, as well as the evaluation criteria for the proposal.

  • Include clear contact information for the department for potential applicants

As emphasised on the central university portal for application advice and guidance, one of the most important points to consider is whether the academic department shares the academic interests of the applicant. While all departments suggest applicants make contact before proceeding with their application any further, different formats and categories of making the initial contact and sending inquiries can be found across departments. Though all departments provide email addresses for applicants to make general inquiries, some provide the Academic Director of PGR (who manages the department’s PGR programme) and relevant professional staff contact details. It is suggested that institutions/departments should include clear contact information for potential applicants, including which queries should be directed at which named members of staff and how long the wait time may be for responses.

  • Welcome applicants from underrepresented groups implicitly and explicitly

Only two departments across all faculties featured EDI-related information in their pre-admission information webpages. The first was in the form of a statement explaining why plenty of information was provided (‘in order to demystify the admissions process, as part of our commitment to enhancing inclusivity in doctoral education’). The second featured a video clip which sought to detail principles of an inclusive working and learning environment and welcome applications from individuals who identify with any of the protected characteristics defined by the Equality Act 2010. Websites which clearly communicate all required information to applicants serve an EDI function in that they do not require applicants to draw upon tacit information to make sense of pre-application steps that have not been carefully explained. In addition to clear and accessible information, welcome statements that determine a departmental position on inclusion can be helpful in that they directly acknowledge those under-represented in higher education. This could be written in collaboration with existing minoritised students.

Our aim is to share the findings from our institutional case study, but also to encourage reflection, review and conversation amongst colleagues about pre-application practices. We highly recommend involving staff and students in review processes as much as possible to ensure that webpages are readable, relevant and useful.

Further information

Two linked Pre-Application Doctoral Communications Research Projects have been carried by a research team based at the University of Warwick including Dr James Burford (PI), Dr Emily Henderson (Co-I), Dr Sophia Kier-Byfield, Dr Dangeni and Ahmad Akkad. The projects were funded by Warwick’s Enhancing Research Culture Fund. The team have produced a suite of open access resources including project briefings. For more information on the project see the website (www.warwick.ac.uk/padc) or #PADC_project on Twitter.

Dr Dangeni is a Professional Development Advisor at Newcastle University, where her teaching and research focus broadly on teaching and learning provision in the wider context of the internationalisation of higher education. She is particularly interested in research and practices around international students’ access, engagement and success in postgraduate taught (PGT) and postgraduate research (PGR) settings.

Dr James Burford is an Associate Professor at the University of Warwick. James’ research interests include doctoral education and the academic profession, higher education internationalisation and academic mobilities. Dr Sophia Kier-Byfield is a Postdoctoral Research Fellow at the University of Warwick, where she works on the ‘Opening Up the Black Box of Pre-Application Doctoral Communications’ projects. Her research interests broadly concern equity in higher education, feminisms in academia and inclusive pedagogies.

Image of Rob Cuthbert


by 1 Comment

Some different lessons to learn from the 2020 exams fiasco

by Rob Cuthbert

The problems with the algorithm used for school examinations in 2020 have been exhaustively analysed, before, during and after the event. The Royal Statistical Society (RSS) called for a review, after its warnings and offers of help in 2020 had been ignored or dismissed. Now the Office for Statistics Regulation (OSR) has produced a detailed review of the problems, Learning lessons from the approach to developing models for awarding grades in the UK in 2020. But the OSR report only tells part of the story; there are larger lessons to learn.

The OSR report properly addresses its limited terms of reference in a diplomatic and restrained way. It is far from an absolution – even in its own terms it is at times politely damning – but in any case it is not a comprehensive review of the lessons which should be learned, it is a review of the lessons for statisticians to learn about how other people use statistics. Statistical models are tools, not substitutes for competent management, administration and governance. The report makes many valid points about how the statistical tools were used, and how their use could have been improved, but the key issue is the meta-perspective in which no-one was addressing the big picture sufficiently. An obsession with consistency of ‘standards’ obscured the need to consider the wider human and political implications of the approach. In particular, it is bewildering that no-one in the hierarchy of control was paying sufficient attention to two key differences. First, national ‘standardisation’ or moderation had been replaced by a system which pitted individual students against their classmates, subject by subject and school by school. Second, 2020 students were condemned to live within the bounds not of the nation’s, but their school’s, historical achievements. The problem was not statistical nor anything to do with the algorithm, the problem was with the way the problem itself had been framed – as many commentators pointed out from an early stage. The OSR report (at 3.4.1.1) said:

“In our view there was strong collaboration between the qualification regulators and ministers at the start of the process. It is less clear to us whether there was sufficient engagement with the policy officials to ensure that they fully understood the limitations, impacts, risks and potential unintended consequences of the use of the models prior to results being published. In addition, we believe that, the qualification regulators could have made greater use of  opportunities for independent challenge to the overall approach to ensure it met the need and this may have helped secure public confidence.”

To put it another way: the initial announcement by the Secretary of State was reasonable and welcome. When Ofqual proposed that ranking students and tying each school’s results to its past record was the only way to do what the SoS wanted, no-one in authority was willing either to change the approach, or to make the implications sufficiently transparent for the public to lose confidence at the start, in time for government and Ofqual to change their approach.

The OSR report repeatedly emphasises that the key problem was a lack of public confidence, concluding that:

“… the fact that the differing approaches led to the same overall outcome in the four countries implies to us that there were inherent challenges in the task; and these 5 challenges meant that it would have been very difficult to deliver exam grades in a way that commanded complete public confidence in the summer of 2020 …”

“Very difficult”, but, as Select Committee chair Robert Halfon said in November 2020, things could have been much better:

“the “fallout and unfairness” from the cancellation of exams will “have an ongoing impact on the lives of thousands of families”. … But such harm could have been avoided had Ofqual not buried its head in the sand and ignored repeated warnings, including from our Committee, about the flaws in the system for awarding grades.”

As the 2021 assessment cycle comes closer, attention has shifted to this year’s approach to grading, when once again exams will not feature except as a partial and optional extra. When the interim Head of Ofqual, Dame Glynis Stacey, appeared before the Education Select Committee, Schools Week drew some lessons which remain pertinent, but there is more to say. An analysis of 2021 by George Constantinides, a professor of digital computation at Imperial College whose 2020 observations were forensically accurate, has been widely circulated and equally widely endorsed. He concluded in his 26 February 2021 blog that:

“the initial proposals were complex and ill-defined … The announcements this week from the Secretary of State and Ofqual have not helped allay my fears. … Overall, I am concerned that the proposed process is complex and ill-defined. There is scope to produce considerable workload for the education sector while still delivering a lack of comparability between centres/schools.”

The DfE statement on 25 February kicks most of the trickiest problems down the road, and into the hands of examination boards, schools and teachers:

“Exam boards will publish requirements for schools’ and colleges’ quality assurance processes. … The head teacher or principal will submit a declaration to the exam board confirming they have met the requirements for quality assurance. … exam boards will decide whether the grades determined by the centre following quality assurance are a reasonable exercise of academic judgement of the students’ demonstrated performance. …”

Remember in this context that Ofqual acknowledges “it is possible for two examiners to give different but appropriate marks to the same answer”. Independent analyst Dennis Sherwood and others have argued for alternative approaches which would be more reliable, but there is no sign of change.

Two scenarios suggest themselves. In one, where this year’s results are indeed pegged to the history of previous years, school by school, we face the prospect of overwhelming numbers of student appeals, almost all of which will fail, leading no doubt to another failure of public confidence in the system. The OSR report (3.4.2.3) notes that:

“Ofqual told us that allowing appeals on the basis of the standardisation model would have been inconsistent with government policy which directed them to “develop such an appeal process, focused on whether the process used the right data and was correctly applied”.

Government policy for 2021 seems not to be significantly different:

“Exam boards will not re-mark the student’s evidence or give an alternative grade. Grades would only be changed by the board if they are not satisfied with the outcome of an investigation or malpractice is found. … If the exam board finds the grade is not reasonable, they will determine the alternative grade and inform the centre. … Appeals are not likely to lead to adjustments in grades where the original grade is a reasonable exercise of academic judgement supported by the evidence. Grades can go up or down as the result of an appeal.” (emphasis added)

There is one crucial exception: in 2021 every individual student can appeal. Government no doubt hopes that this year the blame will all be heaped on teachers, schools and exam boards.

The second scenario seems more likely and is already widely expected, with grade inflation outstripping the 2020 outcome. There will be a check, says DfE, “if a school or college’s results are out of line with expectations based on past performance”, but it seems doubtful whether that will be enough to hold the line. The 2021 approach was only published long after schools had supplied predicted A-level grades to UCAS for university admission. Until now there has been a stable relationship between predicted grades and examination outcomes, as Mark Corver and others have shown. Predictions exceed actual grades awarded by consistent margins; this year it will be tempting for schools simply to replicate their predictions in the grades they award. Indeed, it might be difficult for schools not to do so, without leaving their assessments subject to appeal. In the circumstances, the comments of interim Ofqual chief Simon Lebus that he does not expect “huge amounts” of grade inflation seem optimistic. But it might be prejudicial to call this ‘grade inflation’, with its pejorative overtones. Perhaps it would be better to regard predicted grades as indicators of what each student could be expected to achieve at something close to their best – which is in effect what UCAS asks for – rather than when participating in a flawed exam process. Universities are taking a pragmatic view of possible intake numbers for 2021 entry, with Cambridge having already introduced a clause seeking to deny some qualified applicants entry in 2021 if demand exceeds the number of places available.

The OSR report says that Ofqual and the DfE:

“… should have placed greater weight on explaining the limitations of the approach. … In our view, the qualification regulators had due regard for the level of quality that would be required. However, the public acceptability of large changes from centre assessed grades was not tested, and there were no quality criteria around the scale of these changes being different in different groups.” (3.3.3.1)

The lesson needs to be applied this year, but there is more to say. It is surprising that there was apparently such widespread lack of knowledge among teachers about the grading method in 2020 when there is a strong professional obligation to pay attention to assessment methods and how they work in practice. Warnings were sounded, but these rarely broke through to dominate teachers’ understanding, despite the best efforts of education journalists such as Laura McInerney, and teachers were deliberately excluded from discussions about the development of the algorithm-based method. The OSR report (3.4.2.2) said:

“… there were clear constraints in the grade awarding scenario around involvement of service delivery staff in quality assurance, or making the decisions based on results from a model. … However, we consider that involvement of staff from centres may have improved public confidence in the outputs.”

There were of course dire warnings in 2020 to parents, teachers and schools about the perils of even discussing the method, which undoubtedly inhibited debate, but even before then exam processes were not well understood:

“… notwithstanding the very extensive work to raise awareness, there is general limited understanding amongst students and parents about the sources of variability in examination grades in a normal year and the processes used to reduce them.” (3.2.2.2)

My HEPI blog just before A-level results day was aimed at students and parents, but it was read by many thousands of teachers, and anecdotal evidence from the many comments I received suggests it was seen by many teachers as a significant reinterpretation of the process they had been working on. One teacher said to Huy Duong, who had become a prominent commentator on the 2020 process: “I didn’t believe the stuff you were sending us, I thought it [the algorithm] was going to work”.

Nevertheless the mechanics of the algorithm were well understood by many school leaders. FFT Education Datalab was analysing likely outcomes as early as June 2020, and reported that many hundreds of schools had engaged them to assess their provisional grade submissions, some returning with a revised set of proposed grades for further analysis. Schools were seduced, or reduced, to trying to game the system, feeling they could not change the terrifying and ultimately ridiculous prospect of putting all their many large cohorts of students in strict rank order, subject by subject. Ofqual were victims of groupthink; too many people who should have known better simply let the fiasco unfold. Politicians and Ofqual were obsessed with preventing grade inflation, but – as was widely argued, long in advance –  public confidence depended on broader concerns about the integrity and fairness of the outcomes.

In 2021 we run the same risk of loss of public confidence. If that transpires, the government is positioned to blame teacher assessments and probably reinforce a return to examinations in their previous form, despite their known shortcomings. The consequences of two anomalous years of grading in 2020 and 2021 are still to unfold, but there is an opportunity, if not an obligation, for teachers and schools to develop an alternative narrative.

At GCSE level, schools and colleges might learn from emergency adjustments to their post-16 decisions that there could be better ways to decide on progression beyond GCSE. For A-level/BTEC/IB decisions, schools should no longer be forced to apologise for ‘overpredicting’ A-level grades, which might even become a fairer and more reliable guide to true potential for all students. Research evidence suggests that “Bright students from poorer backgrounds are more likely than their wealthier peers to be given predicted A-level grades lower than they actually achieve”. Such disadvantage might diminish or disappear if teacher assessments became the dominant public element of grading; at present too many students suffer the sometimes capricious outcomes of final examinations.

Teachers’ A-level predictions are already themselves moderated and signed off by school and college heads, in ways which must to some extent resemble the 2021 grading arrangements. There will be at least a two-year discontinuity in qualification levels, so universities might also learn new ways of dealing with what might become a permanently enhanced set of differently qualified applicants. In the longer term HE entrants might come to have different abilities and needs, because of their different formation at school. Less emphasis on preparation for examinations might even allow more scope for broader learning.

A different narrative could start with an alternative account of this year’s grades – not ‘standards are slipping’ or ‘this is a lost generation’, but ‘grades can now truly reflect the potential of our students, without the vagaries of flawed public examinations’. That might amount to a permanent reset of our expectations, and the expectations of our students. Not all countries rely on final examinations to assess eligibility to progress to the next stage of education or employment. By not wasting the current crisis we might even be able to develop a more socially just alternative which overcomes some of our besetting problems of socioeconomic and racial disadvantage.

Rob Cuthbert is an independent academic consultant, editor of SRHE News and Blog and emeritus professor of higher education management. He is a Fellow of the Academy of Social Sciences and of SRHE. His previous roles include deputy vice-chancellor at the University of the West of England, editor of Higher Education Review, Chair of the Society for Research into Higher Education, and government policy adviser and consultant in the UK/Europe, North America, Africa, and China.

MarciaDevlin


by Leave a comment

Making admissions better in Australia

By Marcia Devlin

In Australia, the federal government has been focused on improving the transparency of higher education admissions. I’ve been concerned and written about this matter for some years, particularly the confusion in prospective students and their families around exclusive admissions criteria being used as a proxy for quality.

The government-appointed Higher Education Standards Panel (HESP) were asked to consider and report on how the admissions policies and processes of higher education providers could be made clearer, easier to access and more useful, to inform the choices and decisions of prospective students and their families.

In the context of an increased variety of pathways through which a prospective student can apply or be accepted into higher education in Australia, the HESP found that prospective students, their families and others, including schools, are finding it increasingly difficult to understand the full range of study options and opportunities available, and to understand how they can best take advantage of these options to meet their education and career objectives.

The HESP made 14 recommendations Continue reading →

Vikki Boliver


by Leave a comment

Universities must act collectively to remedy lower offer rates for ethnic minority applicants

By Vikki Boliver

The Runnymede Trust has just launched its publication Aiming Higher: Race, Inequality and Diversity in the Academy which shines a spotlight on ethnic inequalities in UK universities. The report brings together 15 short essays written by academics and policy makers which make clear that radical change is needed to address ethnic inequalities in university admissions, student experiences, degree attainments, graduate labour market outcomes, and access to academic positions especially at senior levels.

In my contribution to the Runnymede publication (see chapter 5) I focus on the issue of ethnic inequalities in university admissions chances. Although British ethnic minorities are more likely to go to university than their White British peers, some ethnic minority groups – notably the Black Caribbean, Black African, Pakistani and Bangladeshi groups – remain strikingly underrepresented in the UK’s most academically selective institutions including Russell Group universities. Of course this is partly due to ethnic inequalities in secondary school attainment which means that members of these groups are less likely to have the high grades required for entry to highly selective universities. But we also know, from analysing university admissions data that British ethnic minority applicants are less likely to be offered places at highly selective universities even when they have the same grades and ‘facilitating subjects’ at A-level as White British applicants. Continue reading →