srhe

The Society for Research into Higher Education

Tag Archives: HE research and technology


by Leave a comment

Understanding the value of EdTech in higher education

by Morten Hansen

This blog is a re-post of an article first published on universityworldnews.com. It is based on a presentation to the 2021 SRHE Research Conference, as part of a Symposium on Universities and Unicorns: Building Digital Assets in the Higher Education Industry organised by the project’s principal investigator, Janja Komljenovic (Lancaster). The support of the Economic and Social Research Council (ESRC) is gratefully acknowledged. The project introduces new ways to think about and examine the digitalising of the higher education sector. It investigates new forms of value creation and suggests that value in the sector increasingly lies in the creation of digital assets.

EdTech companies are, on average, priced modestly, although some have earned strong valuations. We know that valuation practices normally reflect investors’ belief in a company’s ability to make money in the future. We are, however, still learning about how EdTech generates value for users, and how to take account of such value in the grand scheme of things.


Valuation and deployment of user-generated data

EdTech companies are not competing with the likes of Google and Facebook for advertisement revenue. That is why phrases such as ‘you are the product’ and ‘data is the new oil’ yield little insight when applied to EdTech. For EdTech companies, strong valuations hinge on the idea that technology can bring use value to learners, teachers and organisations – and that they will eventually be willing to pay for such benefits, ideally in the form of a subscription. EdTech companies try to deliver use value in multiple ways, such as deploying user-generated data to improve their services. User-generated data are the digital traces we leave when engaging with a platform: keyboard strokes and mouse movements, clicks and inactivity.


The value of user-generated data in higher education

The gold standard for unlocking the ‘value’ of user-generated data is to bring about an activity that could otherwise not have arisen. Change is brought about through data feedback loops. Loops consist of five stages: data generation, capture, anonymisation, computation and intervention. Loops can be long and short.


For example, imagine that a group of students is assigned three readings for class. Texts are accessed and read on an online platform. Engagement data indicate that all students spent time reading text 1 and text 2, but nobody read text 3. As a result of this insight, come next semester, text 3 is replaced by a more ‘engaging’ text. That is a long feedback loop.


Now, imagine that one student is reading one text. The platform’s machine learning programme generates a rudimentary quiz to test comprehension. Based on the students’ answers, further readings are suggested or the student is encouraged to re-read specific sections of the text. That is a short feedback loop.


In reality, most feedback loops do not bring about activity that could not have happened otherwise. It is not like a professor could not learn, through conversation, which texts are better liked by students, what points are comprehended, and so on. What is true, though, is that the basis and quality of such judgments shifts. Most importantly, so does the cost structure that underpins judgment.


The more automated feedback loops are, the greater the economy of scale. ‘Automation’ refers to the decoupling of additional feedback loops from additional labour inputs. ‘Economies of scale’ means that the average cost of delivering feedback loops decreases as the company grows.


Proponents of machine learning and other artificial intelligence approaches argue that the use value of feedback loops improves with scale: the more users engage in the back-and-forth between generating data, receiving intervention and generating new data, the more precise the underlying learning algorithms become in predicting what interventions will ‘improve learning’.


The platform learns and grows with us

EdTech platforms proliferate because they are seen to deliver better value for money than the human-centred alternative. Cloud-based platforms are accessed through subscriptions without transfer of ownership. The economic relationship is underwritten by law and continued payment is legitimated through the feedback loops between humans and machines: the platform learns and grows with us, as we feed it.


Machine learning techniques certainly have the potential to improve the efficiency with which we organise certain learning activities, such as particular types of student assessment and monitoring. However, we do not know which values to mobilise when judging intervention efficacy: ‘value’ and ‘values’ are different things.


In everyday talk, we speak about ‘value’ when we want to justify or critique a state of affairs that has a price: is the price right, too low, or too high? We may disagree on the price, but we do agree that something is for sale. At other times we reject the idea that a thing should be for sale, like a family heirloom, love or education. If people tell us otherwise, we question their values. This is because values are about relationships and politics.


When we ask about the values of EdTech in higher education, we are really asking: what type of relations do we think are virtuous and appropriate for the institution? What relationships are we forging and replacing between machines and people, and between people and people?


When it comes to the application of personal technology we have valued convenience, personalisation and seamlessness by forging very intimate but easily forgettable machine-human relations. This could happen in the EdTech space as well. Speech-to-text recognition, natural language processing and machine vision are examples of how bonds can be built between humans and computers, aiding feedback loops by making worlds of learning computable.


Deciding on which learning relations to make computable, I argue, should be driven by values. Instead of seeing EdTech as a silver bullet that simply drives learning outcomes, it is more useful to think of it as technology that mediates learning relations and processes: what relationships do we value as important for students and when is technology helpful and unhelpful in establishing those? In this way, values can help us guide the way we account for the value of edtech.

Morten Hansen is a research associate on the Universities and Unicorns project at Lancaster University, and a PhD student at the Faculty of Education, University of Cambridge, United Kingdom. Hansen specialises in education markets and has previously worked as a researcher at the Saïd Business School in Oxford.


by Leave a comment

Beware efficiencies! Assetisation as the future defraying of costs savings in the present

by Kean Birch

This blog is based on a presentation to the 2021 SRHE Research Conference, as part of a Symposium on Universities and Unicorns: Building Digital Assets in the Higher Education Industry organised by the project’s principal investigator, Janja Komljenovic (Lancaster). The support of the Economic and Social Research Council (ESRC) is gratefully acknowledged. The project introduces new ways to think about and examine the digitalising of the higher education sector. It investigates new forms of value creation and suggests that value in the sector increasingly lies in the creation of digital assets.

What makes learning more efficient? And what makes teaching more effective? According to EdTech providers and their champions, it is the digital transformation of higher education. The consulting company Gartner – which releases regular EdTech industry reports – defines this transformation as a shift from a ‘collectively-defined’ quality model in which universities provide their services – theoretically – to anyone, to a model in which quality is personally defined and delivered at scale through MOOCs or other means. In fact, Gartner emphasize the importance of EdTech providing scalable technologies for ensuring ‘cost effective education for the benefit of society’. And this seems to be the concern of many EdTech firms themselves; they aim to provide technologies that make life and work more efficient and effective for higher education institutions, managers, faculty, students, and staff.

But what does this actually mean?

I am part of a project, led by Dr Janja Komljenovic, looking at how value is increasingly being created in the higher education sector through the transformation of ‘things’ into digital and other assets – it could be students’ data, it could be research, it could be lectures, and so on. Part of our concern about these changes is the way they can end up reconfiguring societal, public, or commonly held resources as private assets from which companies can exact an economic rent. An important reason for examining this assetisation process is to analyse exactly how things are turned into private assets as a way to open them up to public scrutiny, and political intervention, should we so desire. While assets are constituted by legal forms, like property rights, and technical changes, like digital rights management, they are also the result of broader narratives about how we should or should not understand the world. Epistemic justifications matter. The World Economic Forum highlights what I mean here. They support the deployment of education technology as a way to “create better systems and data flows”. And this means more efficient and effective learning and teaching. But, what does efficiency and effectiveness mean in the case of higher education?

As we have interviewed EdTech providers in our project, we have noticed how they emphasize ‘efficiency’ as one of the key contributions of their technology, where this seems to be equated with producing an outcome at lower cost, whereas this is understood – in common sense terms – as doing something ‘better’ than before. It is important to see how the concept of efficiency is enrolled in the transformation of higher education into a range of assets. Assetisation in higher education depends on the development and promotion of a set of analytics that can identify efficiencies, understood as cost savings that someone or some institution can benefit from. Key to this assetisation process is the characterisation of efficiency as a common-sense goal for universities, managers, faculty, students, staff, and governments; in fact, efficiency can appear to be the very thing that education technologies are turning into an asset. For example, making it cheaper for students to study by enabling them to rent their textbooks, rather than have to buy them. Or making it cheaper for universities to pay subscription only for those electronic texts – or even parts of those texts – that are actually read and used by their staff and students. But this raises an important question: how do EdTech companies make money, if they are simply reducing costs all around?

EdTech companies look to the future for their success. Assets are temporal entities, entailing the creation of a stream of future revenues that can be capitalised in the present, thereby enabling investors to put a value to them that does not depend on being profitable now, or even generate significant revenues now. Efficiencies in the present often end up as defrayed costs in the future as those cost savings today compound into increased revenues for someone (eg EdTech) in the future. The future revenue expectations of EdTech companies come from the illusion of efficiency as cost savings at this point in time; for example, students can save on textbooks now but will be induced to subscribe to lifelong learning resources, or their personal data might be exploited in the future in multiple ways, or their reading habits will be used to sell something to universities, or any manner of revenue generating schemes. Someone is paying in the future.

EdTech companies have to make money somehow, and how they make money is the interesting question. Ideas about the current and future state of higher education and EdTech matter as they provide imaginaries of what is possible and desirable, which we discuss in this report. Claims to efficiency are part of how they make money; they are part of the way that EdTech companies construct new asset classes out of universities and their students, faculty, and staff. Interrogating how these supposed efficiencies are monetised is critical for getting a grip on the implications of EdTech for higher education in the longer term. It is essential we analyse this dynamic now to allow for timely public scrutiny, democratic debate and social intervention.

Kean Birch is Associate Professor at York University, Canada. He is particularly interested in understanding technoscientific capitalism and draws on a range of perspectives from science & technology studies, economic geography, and economic sociology to study it. More specifically, his research focuses on the restructuring and transformation of the economy & financial knowledges, technoscience & technoscientific innovation, and the relationship between markets & natural environments. Currently, he is researching how different things (e.g. knowledge, personality, loyalty, etc.) are turned into ‘assets’ & how economic rents are then captured from those assets – basically, in processes of assetisation and rentiership.


by Leave a comment

Mapping financial investment flows in digital higher education: a focus on data-rich operations

by Janja Komljenovic

This blog is based on a presentation to the 2021 SRHE Research Conference, as part of a Symposium on Universities and Unicorns: Building Digital Assets in the Higher Education Industry organised by the project’s principal investigator, Janja Komljenovic (Lancaster). The support of the Economic and Social Research Council (ESRC) is gratefully acknowledged. The project introduces new ways to think about and examine the digitalising of the higher education sector. It investigates new forms of value creation and suggests that value in the sector increasingly lies in the creation of digital assets.

Universities worldwide are increasingly interested in digital technologies and how they can support higher education. A recent study by the European University Association found that most European universities are already using or planning to use data-rich products and services, such as artificial intelligence, machine learning, learning analytics, big data, and the internet of things (see Figure 18 on page 36). Indeed, it is precisely these data-rich operations that are central to the idea of the disruptive potential of education technology (edtech), as argued by my colleague, Javier Mármol Queraltó, in the recent UU project report. The discourse of investors and edtech companies promises thoroughly improved higher education based on personalisation, automation and efficiency. But how deliverable are these promises? Who innovates in the space of data-rich operations, for which services and for which users? Who profits? These are some of the questions we address in the Universities and Unicorns project, which aims to understand forms of value and ways of creating it in digital higher education. In this blog post, I will address three possible trends that can be identified from the interim findings of our quantitative analysis. But before proceeding to discuss these trends, I will contextualise our analysis.

We used Crunchbase to build three databases covering 2,012 edtech companies, 1,120 investors in edtech, and 1,962 edtech investment deals. We identified those relevant to the higher education sector, and our data reflects the state of the sector as of July 2021. Based on this analysis, we identified four key service models in the higher education edtech industry. First, the business to business (B2B) model includes digital platforms serving universities and companies, such as virtual learning environments. Second, the business to customer (B2C) model includes platforms targeting individuals directly. Third, the business to business to customer (B2B2C) model serves institutions that use or further develop the platform to reach individuals, such as Massive Open Online Courses (MOOC) or Online Programme Management platforms (OPM). Finally, the business to the customer to customer (B2C2C) model includes platforms that connect individuals, such as skills and knowledge sharing platforms. B2B2C and B2C2C platforms, in particular, act as the kind of infrastructural intermediaries that are so popular in other sectors of our social and economic lives.

Our analysis found that half of all investment went into B2B platforms, followed by investment into B2C, while B2C2C and B2B2C together received just under a quarter of all investment. However, platforms with the fastest pace of increasing investment are those targeting individuals directly or through intermediation, ie B2C and B2C2C models. This might indicate emerging parallel or alternative higher education products and services that compete with traditional university provision, especially in the context of lifelong learning.

Digital platforms that say they incorporate data-rich operations in their products and services are not the priority area for investors. While we noticed an increasing investment in data-rich platforms, it was still only less than a quarter of all investment going into innovating such products. Nevertheless, we identified three possible trends that are especially worthy of our attention: (1) data-rich operations are being innovated largely in B2B platforms; (2) there is notable unevenness in terms of the location of edtech companies and investments in those platforms who innovate in data-rich operations; and (3) there might be potential for monopolies in data-rich innovation. Let’s delve into each of these possible trends.

Almost all investment in the companies developing data-rich operations in their platforms went to the B2B service model. Looking only at higher education institutions as the target customer, already half of the investment supports data-rich innovation. Most of that went into platforms that act as the institutional digital backbone, indicating that the intention might be to support all institutional functions beyond teaching with data-rich operations, such as artificial intelligence, machine learning and various kinds of analytics beyond learning analytics. There seems to be a trend towards data-rich digital ecosystems at universities that harvest all user and other data in the near future.

There is high unevenness in where the investment in data-rich platforms is allocated. Regarding the number of companies, 239 in our database declare that they offer data-rich operations on their platforms. Almost half of those (101) are based in the USA, 21 in the UK and 19 in India. Companies based in Africa are entirely missing from the list. In terms of investment amounts, 88% of all investment in companies offering data-rich services in their platforms went into companies based in the USA, 3% each to those based in Norway and the UK, and 6% to the rest of the world. The discrepancy between the number of companies and investment size indicates that investment amounts are higher in the USA than elsewhere in the world.

Finally, if we compare different indicators of investment in companies that innovate data-rich solutions for higher education institutions, we notice interesting dynamics. Looking at the money raised, half of B2B investment went into those companies with a platform that included data-rich operations. But this is only 30% of deals and 25% of companies. This indicates that the concentration of investment in data-rich operation platforms for higher education institutions goes into a smaller number of companies who get higher investments. We wonder if this signals potential for monopolies in the future. Moreover, if we compare granted patents, we notice that a higher percentage of companies offering data-rich solution platforms own patents (30%) versus those offering other kinds of service or product platforms (10%). Digital platforms are typically still protected by a licence, but that differs from a more restrictive patent protection. We wonder if such discrepancy in patent share might indicate black-boxing of data-rich operations in higher education?

Our research on digitalising higher education is showing the complex impact of digital technology and datafication on the sector. This impact includes potential positive and supportive measures, but also many potentially worrying trends. However, further research is needed into these trends and the role of different actors, particularly financial investors and edtech companies. Please follow our project in which we will share the findings from this further work as it unfolds.

Janja Komljenovic is a Senior Lecturer and co-Director of the Higher Education Research and Evaluation at Lancaster University in the UK. She is also a Research Management Committee member of the Global Centre for Higher Education with headquarters at the University of Oxford. Janja’s research focuses on the political economy of knowledge production and higher education markets. She is especially interested in the relationship between the digital economy and the higher education sector; and in digitalisation, datafication and platformisation of knowledge production and dissemination. Janja is published internationally on higher education policy, markets and education technology.


by Leave a comment

The potential of automated text analysis for higher education research

by Stijn Daenekindt

Together with Jeroen Huisman, I recently published an article in which we mapped the field of research on higher education. In a previous blogpost we reflected on some key findings, but only briefly mentioned the method we used to analyze the abstracts of 16,928 research articles (which totals to over 2 million words). Obviously we did not read all these texts ourselves. Instead, we applied automated text analysis. In the current blogpost, I will discuss this method to highlight its potential for higher education research.

Automated text analysis holds tremendous potential for research into higher education. This because, higher education institutions—ie our research subjects— ‘live’ in a world that is dominated by the written word. Much of what happens in and around higher education institutions eventually gets documented. Indeed, higher education institutions produce an enormous amount and variety of texts, eg grant proposals, peer reviews and rejection letters, academic articles and books, course descriptions, mission statements, commission reports, evaluations of departments and universities, policy reports, etc. Obviously, higher education researchers are aware of the value of these documents and they have offered a lot of insightful case studies by closely reading such documents. However, for some types of research questions, analysing a small sample of texts just doesn’t do the job. When we want to analyse huge amounts of text data, which are unfeasible for close reading by humans, automated text analysis can help us.

There are various forms of automated text analysis. One of the most popular techniques is topic modelling. This machine learning technique is able to automatically extract clusters of words (ie topics). A topic model analyses patterns of word co-occurrence in documents to reveal latent themes. Two basic principles underlie a topic model. The first is that each document consists of a mixture of topics. So, imagine that we have a topic model that differentiates two topics, then document A could consist of 20% topic 1 and 80% topic 2, while document B might consist of 50% topic 1 and 50% topic 2. The second principle of topic modelling is that every topic is a mixture of words. Imagine that we fit a topic model on every edition of a newspaper over the last ten years. A first possible topic could include words such as ‘goal’, ‘score, ‘match’, ‘competition’ and ‘injury’. A second topic, then, could include words such as ‘stock’, ‘dow_jones, ‘investment, ‘stock_market’ and ‘wall_street’. The model can identify these clusters of words, because they often co-occur in texts. That is, it is far more likely that the word ‘goal’ co-occurs with the word ‘match’ in a document, then it is to co-occur with the word ‘dow_jones’.

Topic models allow us to reveal the structure of large amounts of textual data by identifying topics. Topics are basically a set of words. More formally, topics are expressed as a set of word probabilities. To learn what the latent theme is about we can order all the words in decreasing probability. The two illustrative topics (see previous paragraph) clearly deal with the general themes ‘sports’ and ‘financial investments’. In this way, what topic models do with texts actually closely resembles what exploratory factor analysis does with survey data, ie revealing latent dimensions that structure the data. But how is the model able to find interpretable topics? As David Blei explains, and this may help to get a more intuitive understanding of the method, topic models trade off two goals: (a) the model tries to assign the words of each document to as few topics as possible, and (b) the model tries, in each topic, to assign high probability to as few words as possible. These goals are at odds. For example, if the model allocates all the words of one document to one single topic, then (b) becomes unrealistic. If, on the other hand, every topic consists of just a few words, then (a) becomes unrealistic. It is by trading off both goals that the topic model is able to find interpretable sets of tightly co-occurring words.

Topic models focus on the co-occurrence of words in texts. That is, they model the probability that a word co-occurs with another word anywherein a document. To the model, it does not matter if ‘score’ and ‘match’ are used in the same sentence in a document or if one is used in the beginning of the document while the other one is used at the end. This puts topic modelling in the larger group of ‘bag-of-words approaches’, a group of methods that treat documents as …well … bags of words. Ignoring word order is a way to simplify and reduce the text, which yields various nice statistical properties. On the other hand, this approach may result in the loss of meaning. For example, the sentences ‘I love teaching, but I hate grading papers’ and ‘I hate teaching, but I love grading papers’ obviously have different meanings, but this is ignored by bag-of-words techniques.

So, while bag-of-word techniques are very useful to classify texts and to understand what the texts are about, the results will not tell us much about how topics are discussed. Other methods from the larger set of methods of automated text analysis are better equipped for this. For example, sentiment analysis allows one to analyze opinions, evaluations and emotions. Another method, word embedding, focusses on the context in which a word is embedded. More specifically, the method finds words that share similar contexts. By subsequently inspecting a words’ nearest neighbors — ie which are the words often occurring in the neighborhood of our word of interest — we get an idea of what that word means in the text. These are just a few examples of the wide range of existing methods of automated text analysis and each of them has its pros and cons. Choosing between them ultimately comes down to finding the optimal match between a research question and a specific method.

More collections of electronic text are becoming available every day. These massive collections of texts present massive opportunities for research on higher education, but at the same time they present us with a problem: how can we analyze these? Methods of automated text analysis can help us to understand these large collections of documents. These techniques, however, do not replace humans and close reading. Rather, these methods are, as aptly phrased by Justin Grimmer and Brandon Stewart, ‘best thought of as amplifying and augmenting careful reading and thoughtful analysis’. When using automated text analysis in this way, the opportunities are endless and I hope to see higher education researchers embrace these opportunities (more) in the future.

Stijn Daenekindt is a Postdoctoral Researcher at Ghent University (Department of Sociology). He has a background in sociology and in statistics and has published in various fields of research. Currently, he works at the Centre for Higher Education Governance Ghent. You can find an overview of his work at his Google Scholar page.

Kelly Coate


by Leave a comment

Reflective teaching in higher education

By Kelly Coate

Those of us who research higher education, and universities in particular, are endlessly offered rich sources of data from one of the most enduring and fascinating institutions in the world. Higher education is an unusual site of research, given the wide range of disciplines that can be employed and the diversity of approaches that can be taken. It is unusual for other reasons too: here in the SRHE we continue to develop as a very strong community of higher education specialists, but we know that almost anyone who works in academia might fancy trying their hand at doing higher education research, most likely in their classrooms but increasingly with other groups such as administrators or managers. Some of us may despair at the lack of knowledge and depth that higher education research ‘amateurs’ bring to bear on the field, but others of us encourage novices to get involved, mainly through the postgraduate programmes in academic practice that have become embedded in many institutions. Therefore another distinctive feature of higher education research is that we speak to many audiences through our publications. Mainly – as in common with other disciplinary specialists – we like to talk to each other, but our books and articles are increasingly used in those academic practice programmes just mentioned, and so a wide range of other disciplinary experts are now engaging with our work. Continue reading →