by Paul Temple
“Fancy coming along to a session with our creative team?” said my friend who works in advertising.
“Sure”, I replied. “Sounds fun. Who’s the client?” Continue reading
Category Archives: Quality
The ‘Holy Grail’ of pedagogical research: the quest to measure learning gain
by Camille Kandiko Howson, Corony Edwards, Alex Forsythe and Carol Evans
Just over a year ago, and learning gain was ‘trending’. Following a presentation at SRHE Annual Research Conference in December 2017, the Times Higher Education Supplement trumpeted that ‘Cambridge looks to crack measurement of ‘learning gain’; however, research-informed policy making is a long and winding road.
Learning gain is caught between a rock and a hard place — on the one hand there is a high bar for quality standards in social science research; on the other, there is the reality that policy-makers are using the currently available data to inform decision-making. Should the quest be to develop measures that meet the threshold for the Research Excellence Framework (REF), or simply improve on what we have now?
The latest version of the Teaching Excellence and Student Outcomes Framework (TEF) remains wedded to the possibility of better measures of learning gain, and has been fully adopted by the OfS. And we do undoubtedly need a better measure than those currently used. An interim evaluation of the learning gain pilot projects concludes: ‘data on satisfaction from the NSS, data from DHLE on employment, and LEO on earnings [are] all … awful proxies for learning gain’. The reduction in value of the NSS to 50% in the most recent TEF process make it no better a predictor of how students learn. Fifty percent of a poor measure is still poor measurement. The evaluation report argues that:
“The development of measures of learning gain involves theoretical questions of what to measure, and turning these into practical measures that can be empirically developed and tested. This is in a broader political context of asking ‘why’ measure learning gain and, ‘for what purpose’” (p7).
Given the current political climate, this has been answered by the insidious phrase ‘value for money’. This positioning of learning gain will inevitably result in the measurement of primarily employment data and career-readiness attributes. The sector’s response to this narrow view of HE has given renewed vigour to the debate on the purpose of higher education. Although many experts engage with the philosophical debate, fewer are addressing questions of the robustness of pedagogical research, methodological rigour and ethics.
The article Making Sense of Learning Gain in Higher Education, in a special issue of Higher Education Pedagogies (HEP) highlights these tricky questions. Continue reading
Beware of slogans
by Alex Buckley
Slogans, over time, become part of the furniture. They start life as radical attempts to change how we think, and can end up victims of their own success. Higher education is littered with ex-slogans: ‘student engagement’, ‘graduate attributes’, ‘technology enhanced learning’, ‘student voice’, ‘quality enhancement’, to name just a few. Hiding in particularly plain sight is ‘teaching and learning’ (and ‘learning and teaching’). We may use the phrase on a daily basis without thinking much about it, but what is the point of constantly talking about teaching and learning in the same breath? Continue reading
Examining the Examiner: Investigating the assessment literacy of external examiners
By Dr Emma Medland
Quality assurance in higher education has become increasingly dominant worldwide, but has recently been subject to mounting criticism. Research has highlighted challenges to comparability of academic standards and regulatory frameworks. The external examining system is a form of professional self-regulation involving an independent peer reviewer from another HE institution, whose role is to provide quality assurance in relation to identified modules/programmes/qualifications etc. This system has been a distinctive feature of UK higher education for nearly 200 years and is considered best practice internationally, being evident in various forms across the world.
External examiners are perceived as a vital means of maintaining comparable standards across higher education and yet this comparability is being questioned. Despite high esteem for the external examiner system, growing criticisms have resulted in a cautious downgrading of the role. One critique focuses on developing standardised procedures that emphasise consistency and equivalency in an attempt to uphold standards, arguably to the neglect of an examination of the quality of the underlying practice. Bloxham and Price (2015) identify unchallenged assumptions underpinning the external examiner system and ask: ‘What confidence can we have that the average external examiner has the “assessment literacy” to be aware of the complex influences on their standards and judgement processes?’ (Bloxham and Price 2015: 206). This echoes an earlier point raised by Cuthbert (2003), who identifies the importance of both subject and assessment expertise in relation to the role.
The concept of assessment literacy is in its infancy in higher education, but is becoming accepted into the vernacular of the sector as more research emerges. In compulsory education the concept has been investigated since the 1990s; it is often dichotomised into assessment literacy or illiteracy and described as a concept frequently used but less well understood. Both sectors describe assessment literacy as a necessity or duty for educators and examiners alike, yet both sectors present evidence of, or assume, low levels of assessment literacy. As a result, it is argued that developing greater levels of assessment literacy across the HE sector could help reverse the deterioration of confidence in academic standards.
Numerous attempts have been made to delineate the concept of assessment literacy within HE, focusing for example on the rules, language, standards, and knowledge, skills and attributes surrounding assessment. However, assessment literacy has also been described as Continue reading
The Thirty Years Quality War
By Rob Cuthbert
Ten years ago David Watson[1] (2006 p2) said that in England since the 1980s: “the audit society and the accountability culture have collided (apparently) with academic freedom and institutional autonomy”. He called this clash between accountability and autonomy the ‘Quality Wars’ and identified five major casualties: the shrinking of higher education’s sectoral responsibilities; truth – managers mistaking criticism for resistance, staff mistaking resistance for criticism; solidarity – because of the rise of the ‘gangs’ – the Russell Group and others; students, as quality assurance became ever less effective at delivering enhancement; and the reputation of UK HE abroad, as our determination to label things unsatisfactory advertised the few deficiencies of our sector and obscured our strengths.
Ten years on, the hostilities continue and the casualties mount. Continue reading