SRHE Blog

The Society for Research into Higher Education

Category Archives: Performance


by Leave a comment

Collegiality and competition in German Centres of Excellence

by Lautaro Vilches

Collegiality, although threatened by increasing competitive pressures and described as a slippery and elastic concept, remains a powerful ideal underpinning academic and intellectual practices. Drawing on two empirical studies, this blog examines the relationships between collegiality and competition in Centres of Excellence (CoEs) in the Social Sciences and Humanities (SSH) in Germany. These CoEs are conceptualised as a quasi-departmental new university model that contrasts with the ‘university of chairs’, which characterises the old Humboldtian university model, organised around chairs led by professors. Hence my research question: How do academics experience collegiality, and how does it relate to competition, within CoEs in the SSH?

In 2006, the government launched the Excellence Strategy (then known as the Excellence Initiative), which includes a scheme providing long-term funding for Centres of Excellence. Notably, this scheme extends beyond the traditionally more collaborative Natural Sciences, to encompass the Social Sciences and Humanities. Germany, therefore, offers a unique case to explore transformations of collegiality amidst co-existing and overlapping university models. What, then, are the key features of these models?

In the old model of the ‘university of chairs’ the chair constitutes the central organisational unit of the university, with each one led by a single professor. Central to this model is the idea of collegial leadership according to which professors govern the university autonomously, a practice that can be traced back to the old scholastic guild of the Middle Ages. During the eighteenth century, German universities underwent a process of modernisation influenced by Renaissance ideals, culminating in the establishment of University of Berlin in Prussia in 1810 by Wilhelm von Humboldt. By the late nineteenth century, the Humboldtian model of the university had become highly influential, as it offered an organisational template in which the ideals of academic autonomy, academic freedom and the  integration of research and teaching were institutionalised.

Within the university of chairs, collegiality is effectively ‘contained’ and enacted within individual chairs. In this structure, professors have no formal superiors and academic staff are directly subordinate to a single professor (as chair holder) – not an institute or faculty. As a result, the university of chairs is characterised by several small and steep hierarchies.

In recent decades – alongside the rise of the United States as the hegemonic power – the Anglo-American departmental model spread across the world, a shift that is associated with the entrepreneurial transformation of universities as they respond to growing competitive pressures.

Remarkably, CoEs in the SSH in Germany are organised as ‘quasi-departments’ resembling a multidisciplinary Anglo-American department. They are very large in comparison with other collaborative grants, often comprising more than 100 affiliated researchers. They are structured around several ‘Research Areas’ and led by 25 Principal Investigators (mostly professors) who must agree on the implementation of the multidisciplinary and integrated research programme on which the CoE is based.

The historical implications of this new model cannot be overstated. CoEs appear to operate as Trojan horses: cloaked in the prestige of excellence, they have introduced a fundamentally different organisational model into the German university of chairs, an institution that has endured over centuries.

Against the backdrop of these two models, what are the implications for collegiality and its relation to competition? A few clarifications are necessary. First, much of the research on collegiality has focused on governance, ignoring that collegiality is also practised ‘on the ground’. Here, I will define collegiality (a) as form of ‘leadership and governance’, involving relations among leaders as well as interactions between leaders and those they govern; (b) as an ‘intellectual practice’ that can be best observed in the enactments of collaborative research; and (c) as a form of ‘citizenship’, involving practices that signify belonging to the CoE and its academic community.

Second, adopting this broader understanding requires acknowledging that collegiality is not only experienced by professors (in governing collegialy the university) but also by the ‘invisible’ academic demos, namely Early Career Researchers (ECRs). Although often employed in precarious positions, ECRS are nonetheless significant members of the academic community, in particular in CoEs, which explicitly prioritise the training of ECRs as a core objective. Whilst ECRs are committed full time to the CoE and sustain much of its collaborative research activity, professors remain simultaneously bound to the duties of their respective positions as chairs.

A third clarification concerns our normative assumptions underpinning collegiality and its relationship to competition. Collegiality is sometimes idealised as an unambiguously positive value and practice in academia, whilst competition – in contrast – is seen as a threat to collegiality. However, this idealised depiction tends to underplay, for example, the role of hierarchies in academia and often invokes an indeterminate past – perhaps somewhere in the 1960s – when universities were governed autonomously by male professors and generously funded through block grants – largely protected from competition pressures or external scrutiny.

These contextual conditions have evidently changed over recent decades: competition, both at institutional and individual terms, has intensified in academia, and CoE schemes exemplify this shift. CoE members, especially ECRs, are therefore embedded in multiple and overlapping competitions: at the institutional level through the CoE’s race for excellence; and at the individual level, through the competition for getting a position in the CoE, as well as for grants, publications, and networks necessary for career advancement.

How are collegiality and competition intertwined in the CoE? I identify three complex dynamics:

  • ‘The temporal flourishing of intellectual collegiality’ refers to the blooming of collegiality as part of the collaborative research work in the CoE. ECRs describe extensive engagement in organising, leading or co-leading research seminars (alongside PIs or other postdoctoral researchers), co-editing books, developing digital collaborative platforms, inviting researchers from abroad to join the CoE or organising and participating in informal meetings. Within this dynamic, competition is presented as being located ‘outside’ the CoE, temporarily deactivated. However, at the same time, ECRs remain aware of the omnipresence of competition, which ultimately threatens collegial collaboration when career paths, research topics or publications begin to converge. For this reason, intellectual collegiality and competition stand in an exclusionary relationship.
  • ‘The rise of CoE citizenship for the institutional race of excellence’ captures the strong sense of engagement and commitment shown by ECRs (but also professors) towards the CoE. It is expressed through initiatives aimed at enhancing the CoE’s collective research performance, particularly in anticipation of competition for renewed excellence funding. This dynamic reveals that, for the CoE, citizenship and institutional competition are not oppositional but complementary, as collective engagement is mobilised in the service of competitive success.
  • ‘Collegial leadership adapting to multiple competitions’ highlights the plurality of leadership modes, each one responding to different levels and forms of competition. At the level of professors and decision-making processes at the top, traditional collegial governance is ‘overstretched’. Although professors retain full authority, they struggle to reach consensus and to lead these large multidisciplinary centres effectively. This suggests a growing demand for new skills more closely associated with the figure of an academic manager than a professor. The institutional race for excellence thus places considerable strain on collegial governance rooted in the chair-based system. Accordingly, ECRs describe different and, apparently, contradictory modes of collegial leadership. For example, the ‘laissez faire’ mode aligns with the ideals of freedom and autonomy underpinning intellectual collegiality, but also with competition among individuals. They also describe leadership as ‘impositions’, which, on the one hand, erodes trust in professors and decision-making, but, on the other hand, intersects with notions of citizenship that compel ECRs to accept decisions, even when imposed. Yet many ECRs value and expect a more ‘inclusive leadership’ that support the development of intellectual collegiality. Overall, the relationship between collegial leadership and competition is heterogeneous and adaptive, closely intertwined with the preceding dynamics.

How, then, can these dynamics be interpreted together? Overall, the findings suggest that differences between university models matter profoundly for collegiality. Expectations regarding how academics collaborate, participate in governance and decision-making processes and form intellectual communities are embedded in specific institutional contexts.

Regarding the relation between collegiality and competition, I suggest two contrasting interpretations. The first emphasises the flourishing of intellectual collegiality and the emergence of CoE citizenship, understood as a collective, multidisciplinary sense of belonging that is driven by – and complementary to – the institutional race for excellence. The second interpretation, however, views this flourishing as a temporal illusion. From this perspective, competition is omnipresent and stands in a fundamentally exclusionary relationship to collegiality: it threatens intellectual collaboration even when temporarily deactivated; it compels academics to engage in CoE-related work they may not intrinsically value; and it overstretches traditional forms of collegial leadership, promoting managerial modes that erode trust in both academic judgement and decision-making processes. Viewed in this light, competition ultimately poses a threat to collegiality. These rival interpretations may uneasily coexist, and the second one possibly predominates. More research is needed on how organisational contexts affect the relationship between collegiality and competition.

Lautaro Vilches is a researcher at Humboldt University of Berlin and a consultant in higher education. His current research examines the implications of excellence schemes for transforming universities’ organisational arrangements and their effects on academic practices such as collegiality, academic mobility and research collaboration, particularly in the Social Sciences and Humanities. As a consultant he advises universities on advancing strategic change.


by Leave a comment

Unmasking the complexities of academic work

by Inger Mewburn

Hang out in any tearoom and you will hear complaints about work – that’s if there even is a tea room at the end of your open plan cubicle farm. Yet surprisingly little is known about the mundane, daily realities of academic work itself – despite the best efforts of many SRHE members.

Understanding the source of academic work unhappiness is important: unhappy academics lead to unhappy students and stressed-out administrators. If we know more about academics’ working lives, we are better placed to care for our colleagues and produce the kind of research and teaching our broader communities expect of us.

To understand more about academics’ working lives, we are embarking on an ambitious research project to survey 5000 working academics and would love you to take part.

Who is doing the ‘academic housework’?

Higher education institutions are major employers and substantial contributors to national economies. Yet there is a notable lack of comprehensive research on the practicalities of academic work, particularly with respect to how we bring our ‘whole self’ to work.

Just about everyone in academia is dealing with some aspect of their lives which affects how they do their work. Some are neurodiverse, with neurodiverse teenagers at home. Others may have a disability and are part of an under-represented group. More of us than you would think face financial precariousness and just being a woman can result in being given more of the ‘academic housework’. The impact of these various circumstances can be negative or positive from the employer point of view. For example, we know that neurodivergent academics spend a lot of energy ‘masking’ to make other people’s work lives easier, often at the expense of their own wellbeing (Jones, 2023). But we also know that including neurodiverse people in research groups can increase scientific productivity. At the same time, many neurodivergent people avoid disclosing for fear of stigma (even the word ‘disclose’ suggests that individuals should feel shame for merely being who they are).

Benefits for our employers can come at a great cost for us as individuals. While a body of literature exists on factors that affect student academic performance in university settings, there is no equivalent focus on university staff. The literature on students helps us design appropriate processes and services to try to even out the playing field and help everyone reach their potential. But we do not show this same compassion towards ourselves. The existing discourse on academics as workers tends to revolve around output metrics and shallow performance measures. This narrow focus fails to capture the full spectrum of academic labour and our lived experiences.

Our research aims to fill this gap by exploring how academics experience their work from their own perspectives. We seek to understand how the production of knowledge occurs, how academic work is constructed and experienced through daily practices, with a specific focus on academic productivity and distraction. We want to see how various bio-demographic factors interrelate and impact feelings like overwhelm and exhaustion.

Why this research matters

The importance of this study is multifaceted:

1. Informing Policy and Practice: By gaining a deeper understanding of academic work patterns, institutions can develop more effective policies to support their staff and enhance productivity and wellbeing.

2. Addressing Inequalities: The COVID-19 pandemic has highlighted and exacerbated existing inequalities in academia. Our research will explore how factors such as gender, caring responsibilities, and neurodiversity impact academic work experiences.

3. Adapting to Change: As the higher education sector continues to evolve, particularly in the wake of the pandemic and the rise of digital technologies like AI, it’s crucial to understand how these changes affect academic work practices.

4. Supporting Well-being: By examining the interplay between productivity, distraction, and work intensity, we can identify strategies to better support academics’ well-being and job satisfaction.

5. Enhancing Knowledge Production: Ultimately, by understanding and improving the conditions of academic work, we can enhance the quality and quantity of knowledge production in higher education and make better classrooms for everyone.

A comprehensive approach

Our study employs a mixed-methods approach, combining a large-scale survey with follow-up interviews. This methodology allows us to capture both broad trends and individual experiences, providing a nuanced picture of academic work life.

The survey covers a wide range of topics, including:

– Perceptions of academic productivity

– Experiences of distraction and focus

– Work distribution across research, teaching, and administration

– Impact of factors such as neurodiversity, caring responsibilities, and chronic conditions

– Use of technology and AI in academic work

– Feelings of belonging and value within the academic community

We are particularly interested in exploring how these factors intersect and influence each other. For instance, how does neurodiversity impact experiences of productivity and distraction? How do caring responsibilities interact with gender in relation to the number of hours worked and where the work takes place? And who thinks AI is helpful to their work and how are people ‘cognitively offloading’ to machines?

Call for participation

The success of this research hinges on wide participation from across the academic community. We are seeking respondents from all career stages, disciplines, and geographical locations. Whether you’re a seasoned professor or a new PhD student, whether you identify as neurodivergent or not, whether you love academic life or find it challenging – your experiences are valuable and needed.

Moreover, this research provides an opportunity for self-reflection. By engaging with the survey questions, you may gain new insights into your own work practices and experiences, potentially leading to personal growth and improved work strategies.

Looking ahead

The findings from this study will be disseminated through various channels, including academic publications, teaching materials, and potentially, policy recommendations. We are committed to making our results accessible and applicable to the wider academic community.

We stand at a critical juncture in higher education. As the sector faces unprecedented challenges and changes, understanding the nature of academic work has never been more important. By participating in this research, you can play a crucial role in shaping the future of academia.

To participate in the survey or learn more about the study, please visit the survey here: https://anu.au1.qualtrics.com/jfe/form/SV_eEeXg1L3RZJJWce.

Professor Inger Mewburn is the Director of Researcher Development at The Australian National University where she oversees professional development workshops and programs for all ANU researchers. Aside from creating new posts on the Thesis Whisperer blog (www.thesiswhisperer.com), she writes scholarly papers and books about research education, with a special interest in post PhD employability, research communications and neurodivergence.

Reference

Jones, S (2023) ‘Advice for autistic people considering a career in academia’ Autism 27(7) pp 2187–2192


by 4 Comments

Interdisciplinarity

by GR Evans

Historian GR Evans takes the long view of developments in interdisciplinary studies, with particular reference to experience at Cambridge, where progress may at times be slow but is also measured. Many institutions have in recent years developed new academic structures or other initiatives intended to promote interdisciplinary collaboration. We invite further blogs on the topic from other institutional, disciplinary, multidisciplinary or interdisciplinary perspectives.

A recent Times Higher Education article explored ‘academic impostor syndrome’ from the point of view of an academic whose teaching and research crossed conventional subject boundaries. That seemed to have made the author feel herself a misfit. She has a point, but perhaps one with broader ramifications.  

There is still a requirement of specialist expertise in the qualification of academics. In its Registration Conditions for the grant of degree-awarding powers the Office for Students adopts a requirement which has been in used since the early 1990s. An institution which is an established applicant seeking full degree-awarding powers must still show that it has “A self-critical, cohesive academic community with a proven commitment to the assurance of standards supported by effective quality systems.”

A new applicant institution must show that it has “an emerging self-critical, cohesive academic community with a clear commitment to the assurance of standards supported by effective (in prospect) quality systems.” The evidence to be provided is firmly discipline-based: “A significant proportion (normally around a half as a minimum) of its academic staff are active and recognised contributors to at least one organisation such as a subject association, learned society or relevant professional body.” The contributions of these academic staff are: “expected to involve some form of public output or outcome, broadly defined, demonstrating the  research-related impact of academic staff on their discipline or sphere of research activity at a regional, national or international level.”

The establishment of a range of subjects identified as ‘disciplines’ suitable for study in higher education is not much more than a century old in Britain, arriving with the broadening of the university curriculum during the nineteenth century and the creation of new universities to add to Oxford and Cambridge and the existing Scottish universities. Until then the medieval curriculum adapted in the sixteenth century persisted, although Cambridge especially honoured a bent for Mathematics. ‘Research’, first in the natural sciences, then in all subjects, only slowly became an expectation. The higher doctorates did not become research degrees until late in the nineteenth century and the research PhD was not awarded in Britain until the beginning of the twentieth century, when US universities were beginning to offer doctorates and they were established as a competitive attraction in the UK .

The notion of ‘interdisciplinarity’ is even more recent. The new ‘disciplines’ gained ‘territories’ with the emergence of departments and faculties to specialise in them and supervise the teaching and examining of students choosing a particular subject. In this developing system in universities the academic who did not fully belong, or who made active connections between disciplines still in process of defining themselves, could indeed seem a misfit. The interdisciplinary was often disparaged as neither one discipline nor another and often regarded by mainstream specialists as inherently imperfect. Taking an interest in more than one field of research or teaching might perhaps be better described as ‘multi-disciplinary’ and requires a degree of cooperativeness among those in charge of the separate disciplines. But it is still not easy for an interdisciplinary combination to become a recognised intellectual whole in its own right, though ‘Biochemistry’ shows it can be done.

Research selectivity and interdisciplinarity

The ‘research selectivity’ exercises which began in the late 1980s evolved into the Research Assessment Exercises (1986, 1989, 1992, 1996, 2001, 2008), now the Research Excellence Framework. The RAE Panels were made up of established academics in the relevant discipline and by the late 1990s there were complaints that this disadvantaged interdisciplinary researchers. The Higher Education Funding Council for England and the other statutory funding bodies prompted a review, and in November 1997 the University of Cambridge received the consultation paper sent round by HEFCE. A letter in response from Cambridge’s Vice-Chancellor was published, giving answers to questions posed in the consultation paper. Essential, it was urged, were ‘clarity and uniformity of  application of criteria’. It suggested that: “… there should be greater interaction, consistency, and comparability between the panels than in 1996, especially in cognate subject areas. This would, inter alia, improve the assessment of interdisciplinary work.”

The letter also suggested “the creation of multidisciplinary sub-panels, drawn from the main panels” or at least that the membership of those panels should include those “capable of appreciating interdisciplinary research and ensuring appropriate consultation with other panels or outside experts as necessary”. Universities should also have some say, Cambridge suggested, about the choice of panel to consider an interdisciplinary submission. On the other hand Cambridge expressed “limited support for, and doubts about the practicality of, generic interdisciplinary criteria or a single interdisciplinary monitoring group”, although the problem was acknowledged.[1]

Interdisciplinary research centres

In 2000 Cambridge set up an interdisciplinary Centre for Research in the Arts, Humanities, and Social Sciences. In a Report proposing CRASSH the University’s General Board pointed to “a striking increase in the number and importance of research projects that cut across the boundaries of academic disciplines both within and outside the natural sciences”. It described these as wide-ranging topics on which work could “only be done at the high level they demand” in an institution which could “bring together leading workers from different disciplines and from around the world … thereby raising its reputation and making it more attractive to prospective staff, research students, funding agencies , and benefactors.”[2]

There have followed various Cambridge courses, papers and examinations using the term ‘interdisciplinary’, for example an Interdisciplinary Examination Paper in Natural Sciences. Acceptance of a Leverhulme Professorship of Neuroeconomics in the Faculty of Economics in 2022 was proposed on the grounds that “this appointment serves the Faculty’s strategy to expand its interdisciplinary profile in terms of research as well as teaching”.  It would also comply with “the strategic aims of the University and the Faculty … [and] create a bridge between Economics and Neuroscience and introduce a new interdisciplinary field of Neuroeconomics within the University”. However the relationship between interdisciplinarity in teaching and in research has still not been systematically addressed by Cambridge.

‘Interdisciplinary’ and ‘multidisciplinary’

A Government Report of 2006 moved uneasily between ‘multidisciplinary’ and ‘interdisciplinary’ in its use of vocabulary, with a number of institutional case studies. The University of Strathclyde and King’s College London (Case Study 2) described a “multidisciplinary research environment”. The then Research Councils UK (Case Study 5b) said its Academic Fellowship scheme provided “an important mechanism for building interdisciplinary bridges” and at least 2 HEIs had “created their own schemes analogous to the Academic Fellowship concept”.

In sum it said that all projects had been successful “in mobilising diverse groups of specialists to work in a multidisciplinary framework and have demonstrated the scope for collaboration across disciplinary boundaries”. Foresight projects, it concluded, had “succeeded in being regarded as a neutral interdisciplinary space in which forward thinking on science-based issues can take place”. But it also “criticised the RAE for … the extent to which it disincentivised interdisciplinary research”.  And it believed that Doctoral Training Projects still had a focus on discipline-specific funding, which was “out of step with the growth in interdisciplinary research environments and persistent calls for more connectivity and collaboration across the system to improve problem-solving and optimise existing capacity”.

Crossing paths: interdisciplinary institutions, careers, education and applications was published by the British Academy in 2016. It recognised that British higher education remained strongly ‘discipline-based’, and recognized the risks to a young researcher choosing to cross boundaries. Nevertheless, it quoted a number of assurances it had received from universities, saying that they were actively seeking to support or introduce the ‘interdisciplinary’. It provided a set of Institutional Case Studies. including Cambridge’s statement about CRASSH, as hosting a range of externally funded interdisciplinary projects. Crossing paths saw the ‘interdisciplinary’ as essentially bringing together existing disciplines in a cluster. It suggested “weaving, translating, convening and collaborating” as important skills needed by those venturing into work involving more than one discipline.  It did not attempt to explore the definition of interdisciplinarity or how it might differ from the multi-disciplinary.

Interdisciplinary teaching has been easier to experiment with, particularly at school level where subject-based boundaries may be less rigid. There seems to be room for further hard thought not only on the need for definitions but also on the notion of the interdisciplinary from the point of view of the division of provision for posts in – and custody of – individual disciplines in the financial and administrative arrangements of universities. This work-to-be-done is also made topical by Government and Office for Students pressure to subordinate or remove established disciplines which do not offer the student a well-paid professional job on graduation.

SRHE member GR Evans is Emeritus Professor of Medieval Theology and Intellectual History in the University of Cambridge.

[1] Cambridge University Reporter, 22 April (1998).  

[2] Cambridge University Reporter, 25 October (2000).  


by 1 Comment

Towards a better, granular, understanding of differences in awarding gaps

by Kathryna Kwok and Siân Alsop

As a sector, we have long known that a student’s ethnicity has a relationship to the average class of degree they are awarded. Students from Black, Asian and minority ethnic (or ‘BAME’) backgrounds can expect to leave UK universities with lower classifications than their White counterparts – up to 20% lower (UUK & NUS, 2019). Universities have been paying attention to how institutional factors like curricula, assessment and staffing decisions might contribute to this awarding gap in a welcome shift away from focusing on the role student background might play. But not much is really changing. The OfS (2022) reports that although the gap has somewhat narrowed in the last five years, Black students in particular continue to receive 17.4% lower degree outcomes than White students. At this point, it is clear that sweeping, quick fix, or isolated interventions are not fit for purpose. The mechanisms underpinning awarding disparities are complex and entrenched. It seems more likely that a sharpened focus and multi-pronged approach to unpicking the multiple ways in which we disadvantage certain students would be useful, if we are serious about change. 

In this vein, we took as our starting point the need to get granular – to dig into ‘who’, as binary ‘BAME’/White distinctions are uninformative at best, and ‘how’, as a degree award is the culmination of many parts. We focused on understanding differences in awarded marks at the module level. But what’s the best way to operationalise module mark differences between two groups? One obvious option would be, simply, to calculate the difference in means (ie mean of group A – mean of group B). However, this has the potential to be misleading. What if, for instance, there were a lot of variation in students’ marks?

Our solution, as we describe in our 2019 SRHE Research Award final report, was to use the formula for the t-statistic, which is a measure reflecting the difference between two groups’ means scaled to group variation and size. In the context of calculating module mark gaps, we refer to this as the difference index (DI) to avoid confusion with the t-values calculated as part of other statistical testing. The formula for DI is below – here, n is the number of students in a group, s is the standard deviation and x̄ is the group mean.

A larger absolute value indicates a larger difference between the two group means (with group size and variation held constant). A positive value means that group A outperformed group B, while a negative value means that group B outperformed group A. If multiple comparisons are being performed with one common baseline group, it is recommended that the baseline group is consistently positioned as group A.

The DI offers a straightforward yet nuanced way to operationalise module mark gaps using data that universities already routinely collect. As a measure of module-level differences, it also offers a way to characterise, monitor and investigate the awarding gap at a granular level – something which percentage point differences in final degree outcomes are much less able to do.

This said, the DI does have its disadvantages. It is not as intuitively interpretable as a simple difference in means. This is perhaps the trade-off for the added nuance it offers. As an illustration, for two groups each with 10 students and an equal spread of module marks (SD = 10.00), a difference of five marks equates to a DI of 1.12, and a difference of ten marks equates to a DI of 2.24. Another disadvantage of the DI is that it can only be used to compare two groups at a time, meaning separate calculations have to be performed for each pairing. Analyses involving multiple groups (or multiple combinations of groups) could thus quickly become unwieldy. In our case study, which we describe in the report linked above, we used regression modelling to investigate whether module characteristics (eg level, credit value, exam weight) could predict DI. This required us to compute one regression model for each ethnicity pairing (White v Asian, White v Black, White v Mixed, White v Other). One of our findings was that module characteristics significantly predicted DI only between White and Black students, which we note ‘highlight[ed] the importance of recognising the heterogeneity of student experiences and needs’ (Kwok & Alsop, 2021: 16). We hope to conduct similar analyses with a much larger dataset, ideally from multiple institutions, which would enable us to utilise multilevel regression techniques to more elegantly capture and explore granular differences in the marks awarded to different groups of students.

To our knowledge, there is no measure that is systematically being used to operationalise the ethnic awarding gap at a level more granular than final degree outcome. We argue that this limits universities’ abilities to understand the awarding gap and identify what can be done to address it at an institutional level. We believe that the DI offers a solution to this. It can be used by researchers, as we have done, to investigate what curriculum-related factors may be contributing to the awarding gap. Those who teach can use the DI to explore module mark differences in their programmes, both within and between cohorts. Those with access to institutional-level data can investigate these trends on an even larger scale, for instance to explore if there are particular areas or time points where modules have consistently high or low (or negative) DIs. The impact of the pandemic can also be explored in this way, for example by using student cohort data either side of Covid-19. Further, while this article has discussed the DI in the context of investigating the ethnic awarding gap, it can also be used to compare students grouped in other ways. It is important that institutions utilise the data they already have to understand awarding discrepancies in a clear and sufficiently granular way – using the DI would help accomplish this.

Kat Kwok is an Educational Researcher at the Oxford Centre for Academic Enhancement and Development at Oxford Brookes University. She is interested in using quantitative methods to investigate race and gender disparity in higher education, and the student experience. Kat recently started her PhD at Coventry University where she will be using a mixed methods approach to investigate the relationships between feedback and student characteristics, and the impact of feedback.

Dr Siân Alsop is Research Fellow in the Centre for Global Learning at Coventry University. She is a corpus linguist whose research areas include attainment disparities in higher education, feedback, and the language of lectures. Siân was previously a Lecturer in Academic Writing and has worked on a number of projects relating to academic discourse, including the development of the British Academic Written English (BAWE) corpus and the Engineering Lecture Corpus (ELC).


by Leave a comment

Understanding the value of EdTech in higher education

by Morten Hansen

This blog is a re-post of an article first published on universityworldnews.com. It is based on a presentation to the 2021 SRHE Research Conference, as part of a Symposium on Universities and Unicorns: Building Digital Assets in the Higher Education Industry organised by the project’s principal investigator, Janja Komljenovic (Lancaster). The support of the Economic and Social Research Council (ESRC) is gratefully acknowledged. The project introduces new ways to think about and examine the digitalising of the higher education sector. It investigates new forms of value creation and suggests that value in the sector increasingly lies in the creation of digital assets.

EdTech companies are, on average, priced modestly, although some have earned strong valuations. We know that valuation practices normally reflect investors’ belief in a company’s ability to make money in the future. We are, however, still learning about how EdTech generates value for users, and how to take account of such value in the grand scheme of things.


Valuation and deployment of user-generated data

EdTech companies are not competing with the likes of Google and Facebook for advertisement revenue. That is why phrases such as ‘you are the product’ and ‘data is the new oil’ yield little insight when applied to EdTech. For EdTech companies, strong valuations hinge on the idea that technology can bring use value to learners, teachers and organisations – and that they will eventually be willing to pay for such benefits, ideally in the form of a subscription. EdTech companies try to deliver use value in multiple ways, such as deploying user-generated data to improve their services. User-generated data are the digital traces we leave when engaging with a platform: keyboard strokes and mouse movements, clicks and inactivity.


The value of user-generated data in higher education

The gold standard for unlocking the ‘value’ of user-generated data is to bring about an activity that could otherwise not have arisen. Change is brought about through data feedback loops. Loops consist of five stages: data generation, capture, anonymisation, computation and intervention. Loops can be long and short.


For example, imagine that a group of students is assigned three readings for class. Texts are accessed and read on an online platform. Engagement data indicate that all students spent time reading text 1 and text 2, but nobody read text 3. As a result of this insight, come next semester, text 3 is replaced by a more ‘engaging’ text. That is a long feedback loop.


Now, imagine that one student is reading one text. The platform’s machine learning programme generates a rudimentary quiz to test comprehension. Based on the students’ answers, further readings are suggested or the student is encouraged to re-read specific sections of the text. That is a short feedback loop.


In reality, most feedback loops do not bring about activity that could not have happened otherwise. It is not like a professor could not learn, through conversation, which texts are better liked by students, what points are comprehended, and so on. What is true, though, is that the basis and quality of such judgments shifts. Most importantly, so does the cost structure that underpins judgment.


The more automated feedback loops are, the greater the economy of scale. ‘Automation’ refers to the decoupling of additional feedback loops from additional labour inputs. ‘Economies of scale’ means that the average cost of delivering feedback loops decreases as the company grows.


Proponents of machine learning and other artificial intelligence approaches argue that the use value of feedback loops improves with scale: the more users engage in the back-and-forth between generating data, receiving intervention and generating new data, the more precise the underlying learning algorithms become in predicting what interventions will ‘improve learning’.


The platform learns and grows with us

EdTech platforms proliferate because they are seen to deliver better value for money than the human-centred alternative. Cloud-based platforms are accessed through subscriptions without transfer of ownership. The economic relationship is underwritten by law and continued payment is legitimated through the feedback loops between humans and machines: the platform learns and grows with us, as we feed it.


Machine learning techniques certainly have the potential to improve the efficiency with which we organise certain learning activities, such as particular types of student assessment and monitoring. However, we do not know which values to mobilise when judging intervention efficacy: ‘value’ and ‘values’ are different things.


In everyday talk, we speak about ‘value’ when we want to justify or critique a state of affairs that has a price: is the price right, too low, or too high? We may disagree on the price, but we do agree that something is for sale. At other times we reject the idea that a thing should be for sale, like a family heirloom, love or education. If people tell us otherwise, we question their values. This is because values are about relationships and politics.


When we ask about the values of EdTech in higher education, we are really asking: what type of relations do we think are virtuous and appropriate for the institution? What relationships are we forging and replacing between machines and people, and between people and people?


When it comes to the application of personal technology we have valued convenience, personalisation and seamlessness by forging very intimate but easily forgettable machine-human relations. This could happen in the EdTech space as well. Speech-to-text recognition, natural language processing and machine vision are examples of how bonds can be built between humans and computers, aiding feedback loops by making worlds of learning computable.


Deciding on which learning relations to make computable, I argue, should be driven by values. Instead of seeing EdTech as a silver bullet that simply drives learning outcomes, it is more useful to think of it as technology that mediates learning relations and processes: what relationships do we value as important for students and when is technology helpful and unhelpful in establishing those? In this way, values can help us guide the way we account for the value of edtech.

Morten Hansen is a research associate on the Universities and Unicorns project at Lancaster University, and a PhD student at the Faculty of Education, University of Cambridge, United Kingdom. Hansen specialises in education markets and has previously worked as a researcher at the Saïd Business School in Oxford.


by Leave a comment

Pandemic and post-pandemic HE performance in Poland, UK and Ukraine

by Justyna Maciąg,  Mateusz Lewandowski, Tammi Sinha, and Tetiana Prykhodko

This is one of a series of position statements developed following a conference on ‘Building the Post-Pandemic University’, organised on 15 September 2020 by SRHE member Mark Carrigan (Cambridge) and colleagues. The position statements are being posted as blogs by SRHE but can also be found on The Post-Pandemic University’s excellent and ever-expanding website. The original  statement can be found here.

Higher Education Institutions (HEIs) have had to change their delivery and ways of working at incredible speed. The disruptive innovation caused by the COVID-19 pandemic is having profound impacts on all stakeholders of HEIs. This study reports on the results of the project which has brought together 3 perspectives from Poland, the United Kingdom and the Ukraine. The purpose was to evaluate and compare the performance of Universities during this period. Performance is understood as assessment of support given by a University to its stakeholders in the following spheres: organisational, technical, technological, competency and social. This study will contribute to better understanding the context of value creation by Universities during the pandemic and post-pandemic period.

We took the perspective of HEI stakeholders into consideration (students, academics and administrative staff). Their opinions and comments were collected by interviews in the form of an online questionnaire with some open questions. We intended to give them a space to share their perspectives, emotions and feelings caused by the lockdown. The questions carried out thematic analysis around the following issues: 1) organisational (planning and communication); 2) technical, (platforms available, teaching methods); 3) technological, (bandwidth, equipment); 4) competency (your own learning and comfort with online learning); 5) social conditions (your environment for study) of higher education experience within the current COVID-19 pandemic and follow up research post-pandemic. The surveys started in the middle of June 2020 and continued till October 2020. Sampling followed the snowball method. Participants were self-selecting with links shared for the online Microsoft forms and Google questionnaires. 

We collected 396 questionnaires, 296 students, 100 university staff and academics (240 in Poland, 133 in Ukraine, 24 in UK). We would like to thank all of our participants for their contribution and candour.

First we would like to start with some qualitative analysis of students and staff responses in the questionnaire. The open questions were used to diagnose their experiences related to measures taken by Universities during lockdown. They were also asked to highlight the most and the least effective  solutions offered. We decided to use an Ishikawa Diagram to analyse the possible causes for their most and least solutions identified. We analysed the factors around the COVI19  problem in order to provide insights and possible solutions for an effective and thriving  ‘post-pandemic University’.

We grouped responses under headings showing below in the Ishikawa Diagram.

Chart 1 Ishikawa Diagram

There are some obvious similarities between these countries and some differences. We draw a conclusion that in each country the situation was similar, the teaching-learning process was transferred into our Virtual Learning Environments (VLE), and staff started remote or hybrid work (both academics and admin staff). The difference was notably the mechanism of this change: in the UK it was deemed more incremental change, in Ukraine and Poland the change was more radical. The proof of this is that in the responses in Poland and Ukraine respondents indicated several solutions which aren’t coordinated and supported by the Universities (i.e. using a social media for teaching-learning process, lack of integration of different e-learning platforms). Whereas in the UK many Universities used VLEs as ‘business as usual’. 

The common themes identified for research in investigated countries were the expectation of support in different areas, not only in a teaching-learning process, but also in equipment (provision, repairs), financial aid, mental sphere, and competence development etc. The findings implied that the expectations of the students and staff support needs were not fully met at this time. Universities were in survival mode and the change management process was lacking in many areas.

Next, we analysed the background given by quantitative analysis of University performance in the technical, competence and organisational sphere (evaluation was carried out using a 5-point Likert scale). The results of research in each country are shown on Chart 2.

Chart 2 Evaluation of the support given by university during lockdown (Poland, Ukraine, UK)

We also investigated the need of support and help provided during the pandemic period studied. The results are presented in Chart 3.

Chart 3 Percentage of respondents who declare that they need support or help

The results of research showed that ‘University’ is mentioned the most, as the expected supporter for students and staff, both academics and administrators. The importance of social support also has appeared in our results. People are looking for assistance among colleagues, thus creating a proper, strong internal social relationship is valuable for them. 

Table 1 The frequently mentioned sources of support

Our key conclusion from this work, at this time, is the importance of support and setting expectations of what support is available in HEIs. We  draw a key finding that the understanding of value delivered by ‘the University’ has to change, and leave behind the neoliberal concept of value for money. We need to expand the understanding of value, taking into account the necessity of tolerance perceived inefficiencies within the university. University staff and students have had to adapt very quickly, and use all of their skills and tenacity to deal with this situation. Creating and co-creating value within universities has always been challenging, however the creativity of staff and students has pulled this sector through it. We have all had to become disruptive innovators.

Justyna Maciąg, PhD,  and Mateusz Lewandowski, PhD, are lecturers and researches in the Institute of Public Affairs at Jagiellonian University in Cracow, Poland. 

Tammi Sinha, PhD, is Senior Lecturer in Operations and Project Management, Director of the Centre for Climate Action, University of Winchester UK. Tetiana Prykhodko is Head of the Program of Analysis and Research,  City Institute at Lviv, and a PhD student at Ivan Franko National University of Lviv, Ukraine.


by Leave a comment

How likely are BTEC students to enter higher education?

By Pallavi Amitava Banerjee

 

Business and Technology Education Council (BTEC) qualifications are seen by some as prized qualifications for the labour market which draw on work-based scenarios. Providers claim these career-based qualifications are designed Continue reading →

Vicky Gunn


by 2 Comments

The latest TEF Assessment Framework, automated analysis of data, and some Scottish anxiety

By  Vicky Gunn

I had the good fortune to be in Rio for the Paralympics this September. My step-daughter was competing in an endurance road race. For her, the most important thing was improving on previous race times, but she’d hoped to get a medal as well (even though this was not predicted by her ‘metrics’). At the end of September, the Westminster Government through HEFCE, published the TEF2 Technical Specification[i] and I found, to my astonishment that the original differentiating phrases (meets expectations, excellent, outstanding) were to be replaced with medals: Bronze, Silver, Gold. All of this got me thinking about the Teaching Excellence Framework, built like British Cycling on the idea that we can differentiate excellence for competitive purposes and this is a good in itself. I find this comparison deeply troubling. I, like many involved in quality and teaching development in Scottish Higher Education, have invested several years of my professional life to fostering cultures of enhancement. Indeed, the distance travelled to improvement in teaching provision has been a mantra within the totality of Scottish higher education’s stakeholders (academic, government, student bodies alike). In the Quality Enhancement Framework (QEF), we have been more interested in seeing all Scottish institutions getting ‘personal bests’ (hence demonstrating continuous improvement from within their own context), rather than doing better than all the others (final outcome measure).

However, now that we have the first set of TEF indicative metrics, I (like a cycling coach) am assailed with a few doubts about the laudable concentration of raising the quality of the whole Scottish sector. This is an aim of the QEF. This resulted in engaged participants of a quality system which steadfastly refused the divisiveness associated with differentiated institutional quality review outcomes. Yet, if we individually enter it, the TEF will now demand this of us.  Should Scotland then change its QEF substantially with its aspirational collectivism to be consigned to being a phantasm of a previous era? How long can such a discourse last in the face of going for gold? Should I, as an institutional Head of Learning and Teaching, now focus on competing with HEIs, so my institution is seen as outstanding in comparison to all the others and place the sector’s aspirational culture in a box marked ‘soppy idealism’? To put it in British Cycling’s inelegant but superlatively economic phrasing: how will my small specialist institution medal when facing larger, wealthier institutions?  Continue reading →