SRHE Blog

The Society for Research into Higher Education

Monthly Archives: February 2020


by 1 Comment

Leadership in a Changing Landscape

by Jane Creaton

This introductory post is part of a series linked to a Symposium on Leadership in a Changing Landscape, which was held at the SRHE Annual Research Conference in December 2019. This symposium aimed to examine different dimensions of, and perspectives on, leadership in the changing landscape of higher education. Each of the contributions, and the reflections on the discussions that followed, will be summarised here on the SRHE blog over the next four weeks.  Drawing on a range of research projects and practice initiatives, the contributors will explore the career trajectories, motivations, challenges and identities of senior leaders in both research-intensive and teaching focussed universities.

The various projects sought to understand why people aspire to or take up senior leadership roles, how they manage different aspects of the work and the different approaches that are taken to the role. Aligning with the theme of the 2019 conference, the contributions also considered the potential for critical and creative leadership within the academy. In the increasingly measured and managed higher education sector, is it possible for leaders to develop distinctive approaches to leadership and/or to challenge the ideological underpinning of managerialism?

There are a number of key themes running through this diverse set of contributions, including what constitutes good leadership, how leaders can be supported and developed, and the affective dimensions of leadership. Some of the specific questions that we discussed in the symposium, which drew together findings from the projects and our own reflections on leadership from our perspective within higher education institutions, included:

1. What might ‘creative leadership’ or ‘critical leadership’ look like in higher education?

2. How can we challenge dominant discourses of leadership based on predominantly managerially based models and explore new, more flexible, human-focused and compassionate approaches to academic leadership?

3. How might aspiring professors be better prepared for professoriate leadership challenges?

Claire Gordon and Jane Creaton: The role of heads of departments

In our blog post, we will discuss an ongoing project that explores the working practices of heads of academic departments and the institutional policies and practices required to support them. Through interviews with HoDs across the sector, we analyse the key factors impacting on how the role is experienced and enacted, including disciplinary context, institutional structure and type of university. The project is also concerned with the extent to which current leadership and management programmes provide adequate preparation and support for a role which may be particularly vulnerable to work-related stress (Floyd and Dimmock, 2011; Creaton and Heard-Laureote, 2019). The initial analysis of interviews has produced a rich mix of metaphors and analogies to describe the role that have the potential for a more creative approach to leadership development.

Alan Floyd: Exploring notions of ‘good’ academic leadership in challenging times

Due to the nature of academic work, it is accepted that leaders cannot be effective without the support of their departmental colleagues (Floyd and Fung, 2017). Consequently, academic leadership is seen more as ‘the property of the collective rather than the individual’ (Bolden, Petrov, and Gosling, 2009: 259). Arguably, ‘distributing’ and sharing leadership is even more important in universities than in other organisations as academics are well educated, largely autonomous and trained to be highly critical. This means they are more likely to oppose and challenge more traditional leadership models and behaviours and may need a subtler form of leadership than other occupational groups (Bryman, 2007). In my blog post, I will draw on data from projects that have explored more flexible ‘distributed’ and ‘collaborative’ models of leadership, crucially focusing on data from both leaders and the led, to explore academics’ expressed notions of ‘good’ academic leadership in times of change and challenge.

Julie Hulme and Deborah Lock: Professors in preparation: supporting 21st century professorial leaders

Becoming a professor is not easy but for some reason becoming a professor in teaching and learning appears to be harder than most. Part of this is because there is no consensus about what a pedagogic professor looks like, and part of this is linked to uncertainty about appropriate selection criteria, and the type of evidence required to demonstrate professorial behaviours and activities (Evans, 2015). There is a lack of guidance and role (and real) models that aspiring professors (education, scholarship and/or professional practice) can turn to for advice about teaching and learning career pathways (Evans, 2017). The Professors in Preparation network is aimed at providing aspiring professors with a supportive community through which the pooling of knowledge through the sharing of ‘lived’ experiences, and identity stories aids successful applications (Waddington, 2016; Macfarlane and Burg, 2019). The network is based on the premise of a virtuous circle in which members that achieve professorship continue to contribute feedback and provide support to the next generation of professors. In our blog post, we will explore what we should expect from the 21st Century professoriate and how we could we reposition the status of educational and scholarship professors and help them become leaders of Trojan Mice instead of Cinderella followers.

Fiona Denney: What I wish I’d known” – academic leadership in the UK, lessons for the next generation

This blog post will discuss the results from a research project funded by the UK’s Leadership Foundation for Higher Education’s Innovation and Transformation Fund in 2015. 18 academics in leadership positions were interviewed about their leadership experiences and what they wished they had known before taking up their leadership posts. Eight themes and information about the context within which they lead were identified and are presented with a discussion of how this contributes to our understanding of the development of those who aspire to leadership positions in higher education. Literature has focused on the importance of prestige for promotion which can leave academic leaders unprepared for the other challenges of their role (Blackmore, 2015; Kandiko-Howson and Coate, 2015). I will also identify challenges and themes which can be used to better prepare the next generation of academic leaders.

Jane Creaton is Associate Dean (Academic) for the Faculty of Humanities and Social Sciences and a Reader in Higher Education at the University of Portsmouth. She has been a member of the SRHE Governing Council since January 2019.

References

Blackmore, P (2015) Prestige in universities: in tension with the efficiency and effectiveness agenda? Paper presented at the Society for Research into Higher Education Annual Research Conference, Newport, UK

Bolden, R, Petrov, G and Gosling, J (2009) ‘Distributed leadership in higher education: rhetoric and reality’, Educational Management Administration and Leadership https://doi.org/10.1177/1741143208100301

Bryman, A (2007) ‘Effective leadership in higher education: a literature review’, Studies in Higher Education, 32(6): 693-710, https://www.tandfonline.com/doi/abs/10.1080/03075070701685114?journalCode=cshe20  

Creaton, J and Heard-Laureote, K (2019) ‘Rhetoric and reality in middle management: the role of heads of academic departments in UK universities’, Higher Education Policy https://doi.org/10.1057/s41307-018-00128-8

Floyd, A and Fung, D (2017) ‘Focusing the kaleidoscope: exploring distributed leadership in an English university’, Studies in Higher Education https://doi.org/10.1080/03075079.2015.1110692

Evans, L (2015) The purpose of professors: professionalism, pressures and performance Stimulus paper commissioned by the Leadership Foundation for Higher Education, Leadership Foundation for Higher Education

Evans, L (2017) ‘University professors as academic leaders: professorial leadership development needs and provision’, Educational Management Administration and Leadership 45(1): 123–140

Floyd, A and Dimmock, C (2011) ‘‘Jugglers’, ‘copers’ and ‘strugglers’: academics’ perceptions of being a head of department in a post-1992 UK university and how it influences their future careers’, Journal of Higher Education Policy and Management https://doi.org/10.1080/1360080X.2011.585738

Kandiko-Howson, C and Coate, K (2015) The Prestige Economy and Mid-career Academic Women: Strategies, Choices and Motivation, Paper presented at the Society for Research into Higher Education Annual Research Conference, Newport, UK

Macfarlane, B and Burg, D (2019) ‘Women professors and the academic housework trap’, Journal of Higher Education Policy and Management 41(3): 262–274

Waddington, K (2016) ‘The compassion gap in UK universities’, International Practice Development Journal 6(1): 10


by Leave a comment

Guidance or interference? OfS under pressure

by GR Evans

The Office for Students received yet another ‘strategic guidance’ letter from the Secretary of State for Education, then Gavin Williamson, dated 1 January 2020.  This is the fourth in a year.  HEFCE used to receive just one, to go with the  annual statement of the ‘block grant’ figures covering both teaching and research.  This energetic approach recalls concerns about potential for future ministerial interference repeatedly expressed in the House of Lords during the debates before the passing of the Higher Education and Research Act 2017. The new legislation protects ‘the institutional autonomy of English higher education providers’ at s.2(1)(a) and s.2 (8) (b) and (c), and specified instances of institutional ‘academic freedom’ in ‘performing’ a provider’s ‘access and participations functions’ at s.36.  It defines the Haldane Principle at s.103 but in a curiously lop-sided way, in connection only in research and for UKRI not OfS.  So both the tone and the content of this series of letters of ‘guidance’ bear looking at closely for their implications.

OfS now receives only a Teaching Grant, because infrastructure funding for research now goes to Research England within UKRI. The same Minister was in charge of both – Chris Skidmore, one of the three who have gone in and out of that office since 2016. UKRI is in the Department of Universities, Science, Research and Innovation. So for research funding purposes the Minister of State operated in another Department of State altogether. Research England has taken over the infrastructure funding of research, the ‘R’ element of the old ‘block grant’.  Skidmore did not sign the latest letter to OfS,  though HEFCE often used to get its letters signed by both the Secretary of State and the Minister for Higher Education.

The ‘teaching funding’ element of the old block grant has now shrunk to a fraction of its earlier size.  In the latest OfS letter Gavin Williamson provides ‘some specific steers on funding priorities given the need to ensure we are spending public money in the most efficient and effective way’. There is to be a continuation of policy preferences tersely described, such as ‘allocations for high cost subjects’, ‘world leading small and specialist institutions’ and ‘supporting successful participation for underrepresented students’.  There is also to be a requirement to work ‘closely’ with  the DFE to ‘identify’ areas where the need is greatest, while ensuring ‘value for money’. A proposed review of ‘the funding method’ is strongly approved as a ‘move to evaluate value for money’.  ‘I know that  the OfS have been working closely with my officials on funding policy and I hope to see this continue’, Williamson concludes.

The tone is directive. Skidmore had been writing to Research England too, but in a rather different tone. On 2 October 2019 he wrote to David Sweeney, who had moved from HEFCE to head Research England to become its Executive Chair, to thank him for his outline of his ‘proposals’ for the development of the new Knowledge Exchange Framework (KEF), to be added to the TEF and the REF.  He also took the ‘opportunity’ to ‘share’ his ‘priorities’ for ‘the future of research and knowledge exchange’.

Among them was Open Access,  ‘a key feature of REF2021’.  Skidmore pressed this urgently, merely noting briskly ‘the implications for Learned Societies of this implementation’ and encouraging  ‘Research England to develop mechanisms which will support them in the transition’ and to engage in ‘dialogue with publishers’, for open access monographs (free books) are on their way. There is no mention of the consequences of the huge upheaval for institutions and academic authors, caused by authors having to pay for publication themselves and institutions having to fund those they choose to support. The heat of anxiety on all that has been growing.

The overriding purpose of research as described in Skidmore’s letter to Research England is to be ‘the creation, transmission and exploitation of knowledge for economic and social benefit’ with KEF in a prominent place and a Knowledge Exchange Concordat being framed, ‘ensuring that it effectively supports our shared priorities around research commercialisation and impact’.

It could of course be understandable that as a new entity the Office for Students  and Research England should both need a specially vigilant ministerial eye on the way they were shaping themselves and their work.  But the artificial separation of Government control of the T and R elements in the old block grant is creating new problems. A controversial Review of Post-18 Education and Funding (the Augur Review), was published in May 2019, proposing a reduction in undergraduate tuition fees from the level of £9,250 a year at which they then stood.  In the summer of 2019 the House of Lords Science and Technology Committee questioned Philip Augar and members of the Committee about the implications for the future of the ‘dual support’ system with its established division between infrastructure and project funding. The Committee was concerned that though ‘traditionally’ the dual-funding system had ‘supported the research community well’, the failure to increase the infrastructure component –  Quality Related (QR)  funding – since 2010, had ‘led to a deficit in funding which universities have had to plug through cross-subsidies’. In other words teaching and research cannot in practice be supported by quite separate funding streams within universities. For example, libraries serve both students and researchers.

Skidmore’s letter to Research England is not insensitive to this problem:

University partnerships with business will be a significant contributor to reaching the 2.4% target by leveraging additional private investment in research through schemes such as UK Research Partnership lnvestment Fund (UKRPIF).

He links that with the ‘impact agenda’, which will increase the benefits and effects from excellent university research for the economy and society, and in addressing key societal challenges such as climate change and ageing’.

The OfS has so far been noisier than UKRI in publishing policy objectives, many of them more ‘social and economic’ than academic or educational. That is unavoidable because the former Office for Fair Access created under the Higher Education Act 2004 ss.22-41, has been absorbed into the OfS. This has encouraged the OfS to launch many objectives which seem to belong in that area rather than in the purely academic. However, the Government’s locus in social and economic affairs is clearly of a different kind from its long-controversial place in controlling the way public funding for higher education is spent.  Those letters from Secretary of State and Minister to OfS and UKRI are beginning to form a corpus worth close study.

Meanwhile it looks as though teaching and research are to be prised even more decisively apart. The Government reshuffle removed Chris Skidmore but replaced him  with Michelle Donelan, who is to be a Minister only in the DfE.  Announcement of a Minister to take charge of research in BEIS was slow to emerge, but the eventual announcement led Nature’s news reporters to ask “Has the UK’s science minister been demoted? Amanda Solloway comes to the job with no ministerial experience, amid concern that the Prime Minister’s office is controlling the science agenda.” Clearly we must continue to watch this space …

SRHE member GR Evans is Emerita Professor of Medieval Theology and Intellectual History in the University of Cambridge, and CEO of the Independent Dispute Resolution Advisory Service for HE (www.idras.ac.uk).

Image of Rob Cuthbert


by Leave a comment

Digitalisation, Assetisation and the Future of Value in Higher Education

by Rob Cuthbert

Janja Komljenovic (Lancaster) led a seminar hosted by the SRHE South West Regional Network/International Centre for Higher Education Management at Bath University on Wednesday 19 February 2020.

The SRHE South West Regional network, convened by Rajani Naidoo (Bath) and Lisa Lucas (Bristol), never disappoints, and this seminar was the perfect antidote for a windy wet Wednesday in the West, with a brilliant presentation by Janja Komljenovic, co-Director of the Centre for Higher Education Research and Evaluation at Lancaster University.

The presenter declared her research interests in digitalisation and marketisation, and made a convincing case that these things should be seen as two aspects of the same HE phenomenon: digital infrastructure is “the hidden architecture of HE”, citing Ben Williamson (Edinburgh). An introductory tour d’horizon of educational technology in its manifold apps and applications in HE showed us the range of the possible, but this was no more than scene-setting, creating a platform for what was to come. Higher education has conceived of markets as if they were driving commodification and making the value of HE no more than something that can be measured in a price. Komljenovic wants us to achieve a radical reframing, in which commodities give way to assets, and price gives way to rent. Market-making in HE is, she argues, a process of assetisation, not commodification, drawing on a wide range of sources from many disciplines, not least Kean Birch’s ‘Towards a theory of rentiership’.

Assets differ from commodities in many respects, but in particular they change the way we should think about ownership, monetisation and value. Digital assets can indeed be owned, but are more likely to be licensed or rented out than to be purchased outright. Some have argued that digital data are the ‘hot’ 21st century product that occupies the place in the global economy which oil had in the last century. But the analogy is deeply flawed: monetisation of HE assets involves subscription not pricing, and the uses to which assets may be put are subject to contractual restrictions, quite unlike the buyer’s freedom to do as they please with a barrel of oil once purchased. And value is not backward-looking, bought and paid for, it must have a future orientation – higher education is not something that can be banked, its value lies in its potential to deliver in the future. Hence one direction for research is to explore the nature and value of emerging HE assets, who owns them, who can charge for their use, and on what terms.

Dynamic experimentation means that edtech may be oversold. Something touted as the new disruptive technology can prove to be overly ambitious when held up to the light, with the latest disappointments being MOOCs’ original claim of free access to high quality education and, it seems, the blockchain university. Digitalisation is different, and it indeed calls for new ways of understanding the higher education enterprise. The seminar challenged us to reconstruct our understanding of what a higher education market might mean in a digitalised world, to rethink what we understand by ‘value’, and to re-examine what we understand by ‘university’ – and whether the university itself is a sustainable platform for whatever HE may become in the 21st century. What a treat.

Rob Cuthbert is the editor of SRHE News and Blog.

Image of Rob Cuthbert


by Leave a comment

Sneering at ‘low quality’ universities and their students is not the way to equalise educational opportunities

by Rob Cuthbert

Jamie Doward reported for The Observer on 26 January 2020 on recent research by Stuart Campbell, Lindsey Macmillan, Richard Murphy and Gill Wyness of UCL’s Centre for Economic Performance (CEP Discussion Paper No 1647 August 2019 Inequalities in Student to Course Match: Evidence from Linked Administrative Data). Their report said: “We find sizeable socio-economic gaps in academic and earnings match across the attainment distribution, with low SES [socioeconomic status] students consistently undermatching, attending courses with lower attaining peers and lower expected earnings than their richer counterparts.” So far, so good, and no surprise. The paper is largely careful in using factual or objective descriptions, but then it explains its methods thus: “Calculate course quality: we rank each university-course combination in a distribution of course quality, based on either (i) The median of the best three age 18 exam results of students on the course (academic-based), or (ii) The median earnings outcomes of an earlier cohort of students on the subject 5 years after graduation (earnings-based).” So quality is deliberately aligned with difficulty of entry or future earnings potential – both of which are more likely to be associated with the prestige of the course and location of the university, and may have nothing to do with the educational or academic quality of the course. The argument is circular: “People with good A levels do high-quality courses. How do we know they’re high-quality? Because people with good A levels take them[1].” Worse, Gill Wyness is quoted in The Observer article as saying: “You’re much more likely to go to your local university if you are from a poorer background. But if you look at all the students who go to a university that is near them, the disadvantaged kids will still go to a lower-quality university than the advantaged kids.” She has no compunction in generalising from supposed ‘low quality’ courses to whole supposed ‘low quality’ universities.

The ‘low quality’ narrative is up and running, reinforced by right wing commentators like Iain Mansfield, a prolific tweeter now at the Policy Exchange think tank, previously a special adviser to Jo Johnson as Minister for Higher Education. He was quoted in Times Higher Education on 27 January 2020 as saying an Ofsted for universities regime could “come to be seen as needed if the major issue of low-quality provision isn’t tackled … I think it would be a last resort and it would be quite undesirable for our sector.” Mansfield, who developed the teaching excellence framework as a senior civil servant in the Department for Education, also said that “…’poor quality provision’ can already be identified via existing measures: dropout rates, the TEF and graduate employment data, as well as figures on grade inflation and unconditional offers”, as John Morgan reported. It is alarming that a recent senior DfE civil servant so readily accepts these as ‘measures’ of ‘quality’, when each is riddled with ambiguity. If DfE civil servants were drawn more from the ranks of people with actual teaching experience, they might have a better appreciation of what these ‘measures’ actually signify.

Drop-out rates: are higher for disadvantaged students, but many such students leave for non-academic reasons.If ‘course quality’ means high standards we might expect it to be correlated with comparatively high drop-out rates. Did you mean low drop-out rates imply lower quality? No, we thought not.

TEF: TEF has little to do with teaching or excellence: as SRHE Fellow Rosemary Deem (Royal Holloway) and Jo-Anne Baird (Oxford) argued in the Journal of Educational Change  in 2019: “… the English TEF is not about improving teaching but rather an endeavour to pit universities against each other in a highly marketised competitive system …”. Universities, of course, do little to help by trumpeting TEF Gold awards, bearing out the Deem/Baird argument.

Graduate employment data: can often be a function of postcode: the data “could be telling us that a public school dropout working at an upmarket estate agent in Kensington earns as much as a recent graduate working part-time in Bolton”, according to David Willetts, the Minister who commissioned the longitudinal educational outcomes project. He warned that: “Graduate earnings rarely afford good policymaking”.

Grade inflation: for many years Russell Group universities have led the way in grade inflation. Is this a marker of low quality? Some, perhaps most, of what the OfS likes to call ‘unexplained’ improvement in grades might actually be accounted for by long-term improvements in teaching, teachers and school leaver attainment.

Unconditional offers: If government exhorts universities over many years to behave as if they are in a market, they can hardly be blamed for trying to induce their potential student-consumers to choose them over their competitors, by making unconditional or ‘conditional unconditional’ offers. The evidence on unconditional offers is mixed (see Ratcliffe v Dandridge in January 2020) in terms of their impact on students’ motivation for A-levels. At the least, an unconditional offer of any kind is an indication that the university is confident this student will succeed. Harrumphing by Government ministers and the OfS is disingenuous and may work against diversity in the student population.

The ‘low quality’ narrative protects and extends the current stratification of prestige in HE, by abusing the universities which cater for the majority of UK HE students, and abusing their students for choosing to attend them. While that may be no more than we expect from some parts of the political spectrum, we are entitled to expect more concern for evidence and rigour from academic researchers. Lindsey Macmillan and Gill Wyness, two of the co-authors of the recent CEP paper, are respectively Director and Deputy Director of UCL’s Centre for Education Policy and Equalising Opportunities. ‘Impact’ seems to take precedence over rigour here, because this research has nothing to say about ‘course quality’. The message of their research is that high-achieving students from low SES backgrounds choose to attend universities other than those they might have considered and for which they were apparently qualified. Perhaps those students think the places they actually choose offer the best-quality education for them. What happened to informed student choice? Clearly, the UCL Centre for Education Policy and Equalising Opportunities thinks low SES students don’t know what’s best for them, and wants to change things by sneering at those students and the ‘low quality’ universities they persist in choosing. They think ‘student choice’ means forcing a larger number of disadvantaged students to go to ‘high-quality’ universities against their wishes. Is this what is meant by ‘high-quality’ research?  

Rob Cuthbert is editor of SRHE News and Blog. He has been a student, teacher, researcher and manager in a wide range of universities, chair of Aimhigher South West, a member of HEFCE’s Widening Participation Strategic Committee, adviser to the DES on the establishment of the Office for Fair Access, and a senior adviser at the Higher Education Academy.

[1] My thanks to Paul Temple for this observation.


by Leave a comment

The potential of automated text analysis for higher education research

by Stijn Daenekindt

Together with Jeroen Huisman, I recently published an article in which we mapped the field of research on higher education. In a previous blogpost we reflected on some key findings, but only briefly mentioned the method we used to analyze the abstracts of 16,928 research articles (which totals to over 2 million words). Obviously we did not read all these texts ourselves. Instead, we applied automated text analysis. In the current blogpost, I will discuss this method to highlight its potential for higher education research.

Automated text analysis holds tremendous potential for research into higher education. This because, higher education institutions—ie our research subjects— ‘live’ in a world that is dominated by the written word. Much of what happens in and around higher education institutions eventually gets documented. Indeed, higher education institutions produce an enormous amount and variety of texts, eg grant proposals, peer reviews and rejection letters, academic articles and books, course descriptions, mission statements, commission reports, evaluations of departments and universities, policy reports, etc. Obviously, higher education researchers are aware of the value of these documents and they have offered a lot of insightful case studies by closely reading such documents. However, for some types of research questions, analysing a small sample of texts just doesn’t do the job. When we want to analyse huge amounts of text data, which are unfeasible for close reading by humans, automated text analysis can help us.

There are various forms of automated text analysis. One of the most popular techniques is topic modelling. This machine learning technique is able to automatically extract clusters of words (ie topics). A topic model analyses patterns of word co-occurrence in documents to reveal latent themes. Two basic principles underlie a topic model. The first is that each document consists of a mixture of topics. So, imagine that we have a topic model that differentiates two topics, then document A could consist of 20% topic 1 and 80% topic 2, while document B might consist of 50% topic 1 and 50% topic 2. The second principle of topic modelling is that every topic is a mixture of words. Imagine that we fit a topic model on every edition of a newspaper over the last ten years. A first possible topic could include words such as ‘goal’, ‘score, ‘match’, ‘competition’ and ‘injury’. A second topic, then, could include words such as ‘stock’, ‘dow_jones, ‘investment, ‘stock_market’ and ‘wall_street’. The model can identify these clusters of words, because they often co-occur in texts. That is, it is far more likely that the word ‘goal’ co-occurs with the word ‘match’ in a document, then it is to co-occur with the word ‘dow_jones’.

Topic models allow us to reveal the structure of large amounts of textual data by identifying topics. Topics are basically a set of words. More formally, topics are expressed as a set of word probabilities. To learn what the latent theme is about we can order all the words in decreasing probability. The two illustrative topics (see previous paragraph) clearly deal with the general themes ‘sports’ and ‘financial investments’. In this way, what topic models do with texts actually closely resembles what exploratory factor analysis does with survey data, ie revealing latent dimensions that structure the data. But how is the model able to find interpretable topics? As David Blei explains, and this may help to get a more intuitive understanding of the method, topic models trade off two goals: (a) the model tries to assign the words of each document to as few topics as possible, and (b) the model tries, in each topic, to assign high probability to as few words as possible. These goals are at odds. For example, if the model allocates all the words of one document to one single topic, then (b) becomes unrealistic. If, on the other hand, every topic consists of just a few words, then (a) becomes unrealistic. It is by trading off both goals that the topic model is able to find interpretable sets of tightly co-occurring words.

Topic models focus on the co-occurrence of words in texts. That is, they model the probability that a word co-occurs with another word anywherein a document. To the model, it does not matter if ‘score’ and ‘match’ are used in the same sentence in a document or if one is used in the beginning of the document while the other one is used at the end. This puts topic modelling in the larger group of ‘bag-of-words approaches’, a group of methods that treat documents as …well … bags of words. Ignoring word order is a way to simplify and reduce the text, which yields various nice statistical properties. On the other hand, this approach may result in the loss of meaning. For example, the sentences ‘I love teaching, but I hate grading papers’ and ‘I hate teaching, but I love grading papers’ obviously have different meanings, but this is ignored by bag-of-words techniques.

So, while bag-of-word techniques are very useful to classify texts and to understand what the texts are about, the results will not tell us much about how topics are discussed. Other methods from the larger set of methods of automated text analysis are better equipped for this. For example, sentiment analysis allows one to analyze opinions, evaluations and emotions. Another method, word embedding, focusses on the context in which a word is embedded. More specifically, the method finds words that share similar contexts. By subsequently inspecting a words’ nearest neighbors — ie which are the words often occurring in the neighborhood of our word of interest — we get an idea of what that word means in the text. These are just a few examples of the wide range of existing methods of automated text analysis and each of them has its pros and cons. Choosing between them ultimately comes down to finding the optimal match between a research question and a specific method.

More collections of electronic text are becoming available every day. These massive collections of texts present massive opportunities for research on higher education, but at the same time they present us with a problem: how can we analyze these? Methods of automated text analysis can help us to understand these large collections of documents. These techniques, however, do not replace humans and close reading. Rather, these methods are, as aptly phrased by Justin Grimmer and Brandon Stewart, ‘best thought of as amplifying and augmenting careful reading and thoughtful analysis’. When using automated text analysis in this way, the opportunities are endless and I hope to see higher education researchers embrace these opportunities (more) in the future.

Stijn Daenekindt is a Postdoctoral Researcher at Ghent University (Department of Sociology). He has a background in sociology and in statistics and has published in various fields of research. Currently, he works at the Centre for Higher Education Governance Ghent. You can find an overview of his work at his Google Scholar page.


by 1 Comment

The (future) state of higher education research?

by Stijn Daenekindt and Jeroen Huisman

  • Stijn Daenekindt
  • Jeroen Huisman
    Jeroen Huisman

Parallel to the exponential growth of research on higher education, we see an increasing number of scientific contributions aiming to take stock of our field of research. Such stock-taking activities range from reflective and possibly somewhat impressionistic thoughts of seasoned scholars to in-depth reviews of salient higher education themes. Technological advancements (such as easy electronic access to research output and an increasingly broader set of analytical tools) obviously have made life easier for analysts. We recently embarked upon a project to explore the thematic diversity in the field of research in higher education. The results have recently been published in Higher Education. Our aim was to thematically map the field of research on higher education and to analyse how our field has evolved over time.

For this endeavour, we wanted our analysis to be large-scale. We aimed at including a number of articles that would do justice to the presumed variety in research into higher education. We did not, however, want the scale of our analysis to jeopardize the depth of our analysis. Therefore, we decided not to limit our analyses to, for example, an analysis of citation patterns or of keywords. Finally, to forestall bias (stemming from our personal knowledge about and experience in the field), we applied an inductive approach. These criteria led us to collect 16,928 journal articles on higher education published between 1991 and 2018 and to analyse each article’s abstract by applying topic modelling. Topic modelling is a method of automated text analysis and a follow-up blogpost (also on srheblog.com) will address the method. For now, it suffices to know that topic modelling is a machine learning technique that automatically analyses the co-occurrence of words to detect themes/topics and to find structure in a large collection of text.

In this blogpost, we present a glimpse of our findings and some additional thoughts for further discussion. In our analysis, we differentiate 31 research topics which inductively emerged from the data. For example, we found topics dealing with university ranking and performance, sustainability, substance use of college students, research ethics, etc. The bulk of these research topics were studied at the individual level (16 topics), with far fewer at the organisational (5) and system level (3). A final set of topics related either clearly to disciplines (eg teaching psychology) or to more generic themes (methods, academic writing, ethics). This evidences the richness of research into higher education. Indeed, our field of research certainly is not limited in terms of perspectives and unleashes “the whole shebang” of possible perspectives to gain new insights into higher education.

The existence of different perspectives also comprises potential dangers, however. Studies applying a certain approach on higher education — say, a system-level approach — may suffer from tunnel vision and lose sight of individual- and organization-level aspects of higher education. This may be problematic as processes on the different levels are obviously related to one another. In our analysis we find that studies indeed tend to focus on one level. For example, system-level topics tend to be exclusively combined with other system-level topics. This should not come as a big surprise, but there is potential danger in this and it may hamper the development of a more integrated field of research on higher education.

In our analysis, we also find a certain restraint to combine topics which are located at the same level. For example, topics on teaching practices are very rarely combined with topics on racial and ethnic minorities — even though both topics are situated at the individual level. To us, this was surprising as the combination of ethnicity and educational experiences is a blossoming field in the sociology of education. The fact that topics at the same level are only rarely combined is less understandable then the fact that topics on different levels are rarely combined. We hope that our analysis aids others researchers to identify gaps in the literature and that it motivates them to address these gaps.

A second finding we wish to address here relates to specialisation. Our analysis suggests that there is a trend of specialisation in our field of research. We looked at the number of topics combined in articles and we see that topic diversity declines over time. This is, on the one hand, not that surprising. Back in 1962, Kuhn already argued that the system of modern science encourages researchers towards further specialisation. So, it makes sense that over time, and parallel to the growth of the field of research on higher education, researchers specialise more and demarcate their own topic of expertise. On the other hand, it may be considered a problematic evolution as it can hamper our field of research to develop towards further maturity.

But what should we think of the balance between healthy expansion and specialization, on the one hand, and inefficient fragmentation, on the other? We lean towards evaluating the current state of higher education research as moving towards fragmentation. Other researchers, such as Malcom Tight, Bruce Macfarlane and Sue Clegg have similarly lamented the fragmented nature of our field of research. Our analysis adds to this by showing the trends over time: we observe more specialisation (not necessarily bad), but there are also signs of disintegration over time (not good). Other analyses we are currently carrying out also indicate thematic disintegration and suggest clear methodological boundaries. It looks like many researchers focusing on the same topic remain in their “comfort zone” and use a limited set of methods. For sure, many methodological choices are functional (as in fit-for-purpose), but the lack of diversity is striking. Moreover, we see that many higher education researchers stick to rather traditional techniques (survey, interviews, case studies) and that new methods hardly get picked up in our field. A final observation is that we hardly see methodological debates in our field. In related disciplines we often see healthy methodological discussions that improve the available “toolkit” (for example here). In our field, it appears that scholars shy away from such discussions and it suggests methodological conservatism and/or methodological tunnel vision.

There are still many things to investigate to arrive at a full assessment of the state of the art. One important question is how our field compares to other fields or disciplines. But if we were to accept the idea of fragmentation, it is pertinent to start thinking how to combat this. Reversing this trend is obviously not straightforward. But here are a few ideas. Individual scholars could try to get out of their comfort zone by applying other perspectives to their favourite research object and/or by applying their favourite perspective to new research topics. Related, researchers should be encouraged to use techniques less commonly used in our field and see whether they yield different outcomes (vignettes, experimental designs, network analysis, QCA/fuzzy logic, [auto-]ethnography and – of course – topic models). In addition, journal editors could be more flexible and inclusive in terms of the format of the submissions they consider. For example, they could explicitly welcome submissions in the format of ‘commentaries/ a reply to’. This would stimulate debate and open up the floor for increased cross-fertilisation of research into higher education and, in general, signal the maturity of research into higher education. Finally, there is scope for alternative peer review processes. Currently, only editors (and sometimes peer reviewers seeing the outcome of a peer review process) gain full insight in feedback offered by peers. If we would make these processes more visible to a broader readership – e.g. through open peer review, which still can be double-blind – we would gain much more insight in methodological and theoretical debates, that would definitely support the healthy growth of our field.  

This post is based on the article: Daenekindt, S and Huisman, J (2020) ‘Mapping the scattered field of research on higher education. A correlated topic model of 17,000 articles, 1991–2018’ Higher Education, 1-17. Stijn Daenekindt is a Postdoctoral Researcher at Ghent University (Department of Sociology). SRHE Fellow Jeroen Huisman is a Full Professor at Ghent University (Department of Sociology).

Paul Temple


by Leave a comment

Weirdos and misfits? I’ve met a few…

By Paul Temple

Perhaps, like me, you’ve had some harmless fun recently in drawing up a mental list of the “weirdos and misfits…with odd skills” you know in university life who might work with Dominic Cummings at Number 10. (In a few cases, I couldn’t decide who I’d feel sorriest for.) Now that Brexit has been “done”, it seems that Cummings plans to “turn the UK into a leading centre for science, putting it at the cutting edge of artificial intelligence, robotics and climate change” and needs some hired help. (This and other quotes come from a Financial Times profile of Cummings of 18/19 January 2020, said to have been fact-checked by its subject.)

The irony here, presumably unintended, would be almost funny if it wasn’t completely maddening. I’d be surprised if you could find a single working research scientist in the country who doesn’t view Brexit, so far as science is concerned, somewhere on a spectrum from “unfortunate” to “utter disaster”. Certainly, if there are any Brexiteer scientists working at UCL they’ve kept a very low profile indeed over the past few years. And now the man who has done as much as anyone to damage UK academic work by destroying our links with European partners calmly tells us that his “new agenda” – sensibly distancing himself from the tedious details of working out a new trade deal with the EU – is to achieve a scientific renaissance.

But Cummings, it seems, is thinking beyond the UK merely becoming better at science than it has so far managed when working collaboratively with European science networks. Cummings, an Oxford ancient and modern history graduate, clearly considers that he possesses the skills to apply science “to understanding and solving public policy problems”. This is probably what most social scientists, if pressed, would say they are trying to do, but I don’t think that the humdrum problems that most of us work on are what Cummings has in mind. Instead, “his inspiration is the US government’s Manhattan Project…[and how] the failing NASA bureaucracy [became] an organisation that could put a man on the moon…[he also plans] to set up a civilian version of the US Defense Advanced Research Projects Agency”. Big, shiny projects are what he wants.

I’ve used the Manhattan Project as a case study in my teaching, and I’ve no doubt that much can be learned from it. It helped that J Robert Oppenheimer was both a world-class physicist and, as it turned out, a world-class project director, who was able to work with a multi-national group of scientific egoists in a collection of army huts in the New Mexico desert and produce the world’s first atomic explosion within 28 months of starting work. But Oppenheimer knew what he had to do, had a fair idea about how to go about it, and could call on all the resources of the world’s scientific and engineering superpower. It doesn’t at all detract from his achievements to say that the Manhattan Project was in a certain sense straightforward compared to, say, improving health care or reducing crime for a large population. Leaving aside resource limitations, knowing “what works” in these and other areas of social policy has a different meaning to knowing “what works” in nuclear engineering or rocket design. Habermas described this difference in terms of “the ideology of technique”. Even defining what “improved health care” might look like will be contested, as will its measures of success. Nobody doubted that they’d know a nuclear explosion when it happened. (Actually, Oppenheimer might have agreed that quantum mechanics and problems in social policy do have something in common: if you think you understand what it is you’re observing, you’ve got it wrong.)

So my guess is that the clever Oxford humanities graduate, with no formal training in either natural or social science, is going to become very frustrated in attempting to apply methods from the former to try to solve complex problems in the domain of the latter. Paradoxically (or maybe not), this puts me in mind of the education research that I had some acquaintance with in the afterlife of the old Soviet Union. There, the necessary assumption was that if enough data were collected, and the precepts of scientific Marxism-Leninism were correctly applied to them, then a definitive solution to whatever the problem was would be found. There had to be a “scientific” answer to every question, if only you did enough work on it. To suggest otherwise would be, literally, unthinkable in a Marxist worldview.

Still, perhaps Cummings will show that answers to problems in big science do in fact read across to social policy: after all, compared to making Brexit the tremendous national success story that we’ve been assured it will be, it should be quite easy.

SRHE member Paul Temple is Honorary Associate Professor, Centre for Higher Education Studies, UCL Institute of Education, University College London. See his latest paper ‘University spaces: Creating cité and place’, London Review of Education, 17(2): 223–235 at https://doi.org/10.18546


by Leave a comment

Why we should care about comparative higher education?

by Ariane de Gayardon

In contrast to comparative education, whose history dates back to the beginning of the 19th century, comparative higher education is a relatively recent construct of research originating in the 1970-1980s. This early period gave us the first comparative instruments, still widely used today, as lenses to analyse national higher education systems. These include Clark’s triangle of coordination (1983), Altbach’s use of the concept of centre and periphery (1981) and Trow’s definition of elite, mass and universal systems (1973). Therefore, early on, comparative higher education proved very successful in increasing our understanding of higher education globally. But, since then, what has it accomplished?

While there are many users of comparative higher education – that is, researchers whose research could be considered comparative – there is still little written critically on comparative higher education research. The debate is alive, led by individual researchers, including Kosmützky, Bleiklie, and Valimaa. However, there is little acknowledgment of their efforts by users of comparative research, showing a clear divide between efforts to conceptualise and theorise comparative research in higher education and actual research practice. As a result, the field of comparative higher education is lacking rigour, as exemplified by the lack of appropriate rationales for sampling choices – why countries are included – in the vast majority of comparative papers (Kosmützky, 2016). This puts comparative higher education at odds with comparative studies in other disciplines, that have been focused on the comparative method as a way to reach causality or improve generalisation.

What researchers in comparative higher education have failed to achieve in the past 40 years is to elevate comparative studies in higher education to a (sub-)field of study. An academic field is built on the emergence of two dynamics: an intellectual debate and an institutional structure (Manzon, 2018). The debate around comparative higher education has been focused on proposing conceptual and theoretical frameworks, but it remains marginal. Additionally, questions that are still to be raised and answered include the objectives and purpose of comparative higher education, as well as what unites researchers undertaking comparative projects. At the same time, there is a lack of academic space for this debate to happen. Comparative higher education lacks specific journals – with the exception of the Journal of Comparative and International Higher Education, societies and associated conferences, and research centres. Unlike comparative education, it has not yet permeated into the teaching function of higher education, with an absence of textbooks and dedicated degrees (although some courses do exist). Comparative higher education therefore remains on the margins, a practice of research that is still to be properly understood.

This deficit of reflective and critical thinking on comparative higher education matters. The use of comparative higher education for cross-country comparisons remains essential in understanding higher education systems. It provides unique settings to deepen our knowledge of higher education phenomena through the way they manifest in different environments and in contact with different cultures. This leads to improved theorisation of higher education phenomena that transcends borders, helping to fight assumptions and opening new avenues for conceptualising higher education. Consequently, it helps us understand our own higher education system better, through knowledge of the ‘other’ and combatting “comparative chauvinism” and “comparative humility” (Teichler, 2014). And because comparative higher education is not limited to international comparisons, it provides an opportunity to increase our knowledge of within-system variations through tools to analyse both the local and the global in higher education.

Comparative higher education research is also of tremendous importance to evidence-based policy. Higher education policies remain decided at the country (state) level in most countries around the globe, which means that comparison is essential to understand the consequences of different policies. Policy evaluation in higher education needs comparative studies, internationally and historically in particular. Understanding higher education policies beyond the national context is also important in a world where policy-borrowing and lending is prevalent. Knowledge of the ways different policies adapt in different environments helps prevent the spread of seemingly successful policies that would have detrimental consequences if translated elsewhere.

Finally, higher education research already evolves in an international context. Higher education stakeholders – students and faculty in particular – are mobile beyond borders, while knowledge does not know national boundaries. As a result, the vast majority of researchers in higher education have frames of reference that extend beyond their national context. This means that most higher education research might be unintentionally comparative. This is problematic in two ways. First, the way you do research is important to recognise and understand to reach research rigor. Second, researchers might not be acknowledging properly their positionality and bias, by not reflecting on what they know and don’t know about higher education globally.

After 40 years of existence, it might be time to stop and reflect on comparative higher education research and decide what its mission is. To do so, we can rely on endless research and debate in the field of comparative education, as well as a robust literature on comparative studies, that would provide strong basis for the construction of a field of comparative higher education. This reflection will help strengthen the higher education research done comparatively, leading to a tremendous increase in our knowledge of higher education generally.

References

Altbach, PG (1981) ‘The university as center and periphery’ Teachers College Record, 82(4): 601-621

Clark, B (1983) The higher education system : Academic organization in cross-national perspective Berkeley, CA: University of California Press

Kosmützky, A (2016) ‘The precision and rigor of international comparative studies in higher education’ in Theory and Method in Higher Education Research (pp 199-221) Emerald Group Publishing Limited

Manzon, M (2018) ‘Origins and traditions in comparative education: challenging some assumptions’, Comparative Education, 54(1): 1-9

Teichler, U (2014) ‘Opportunities and problems of comparative higher education research: The daily life of research’ Higher Education, 67(4): 393-408

Trow, M (1973) Problems in the transition from elite to mass higher education Berkeley, CA: Carnegie Commission on Higher Education

Ariane de Gayardon is a Senior Research Associate in the Centre for Global Higher Education at the UCL Institute of Education and is Assistant Editor of the Journal of Studies in International Education