srhe

The Society for Research into Higher Education


Leave a comment

Understanding the value of EdTech in higher education

by Morten Hansen

This blog is a re-post of an article first published on universityworldnews.com. It is based on a presentation to the 2021 SRHE Research Conference, as part of a Symposium on Universities and Unicorns: Building Digital Assets in the Higher Education Industry organised by the project’s principal investigator, Janja Komljenovic (Lancaster). The support of the Economic and Social Research Council (ESRC) is gratefully acknowledged. The project introduces new ways to think about and examine the digitalising of the higher education sector. It investigates new forms of value creation and suggests that value in the sector increasingly lies in the creation of digital assets.

EdTech companies are, on average, priced modestly, although some have earned strong valuations. We know that valuation practices normally reflect investors’ belief in a company’s ability to make money in the future. We are, however, still learning about how EdTech generates value for users, and how to take account of such value in the grand scheme of things.


Valuation and deployment of user-generated data

EdTech companies are not competing with the likes of Google and Facebook for advertisement revenue. That is why phrases such as ‘you are the product’ and ‘data is the new oil’ yield little insight when applied to EdTech. For EdTech companies, strong valuations hinge on the idea that technology can bring use value to learners, teachers and organisations – and that they will eventually be willing to pay for such benefits, ideally in the form of a subscription. EdTech companies try to deliver use value in multiple ways, such as deploying user-generated data to improve their services. User-generated data are the digital traces we leave when engaging with a platform: keyboard strokes and mouse movements, clicks and inactivity.


The value of user-generated data in higher education

The gold standard for unlocking the ‘value’ of user-generated data is to bring about an activity that could otherwise not have arisen. Change is brought about through data feedback loops. Loops consist of five stages: data generation, capture, anonymisation, computation and intervention. Loops can be long and short.


For example, imagine that a group of students is assigned three readings for class. Texts are accessed and read on an online platform. Engagement data indicate that all students spent time reading text 1 and text 2, but nobody read text 3. As a result of this insight, come next semester, text 3 is replaced by a more ‘engaging’ text. That is a long feedback loop.


Now, imagine that one student is reading one text. The platform’s machine learning programme generates a rudimentary quiz to test comprehension. Based on the students’ answers, further readings are suggested or the student is encouraged to re-read specific sections of the text. That is a short feedback loop.


In reality, most feedback loops do not bring about activity that could not have happened otherwise. It is not like a professor could not learn, through conversation, which texts are better liked by students, what points are comprehended, and so on. What is true, though, is that the basis and quality of such judgments shifts. Most importantly, so does the cost structure that underpins judgment.


The more automated feedback loops are, the greater the economy of scale. ‘Automation’ refers to the decoupling of additional feedback loops from additional labour inputs. ‘Economies of scale’ means that the average cost of delivering feedback loops decreases as the company grows.


Proponents of machine learning and other artificial intelligence approaches argue that the use value of feedback loops improves with scale: the more users engage in the back-and-forth between generating data, receiving intervention and generating new data, the more precise the underlying learning algorithms become in predicting what interventions will ‘improve learning’.


The platform learns and grows with us

EdTech platforms proliferate because they are seen to deliver better value for money than the human-centred alternative. Cloud-based platforms are accessed through subscriptions without transfer of ownership. The economic relationship is underwritten by law and continued payment is legitimated through the feedback loops between humans and machines: the platform learns and grows with us, as we feed it.


Machine learning techniques certainly have the potential to improve the efficiency with which we organise certain learning activities, such as particular types of student assessment and monitoring. However, we do not know which values to mobilise when judging intervention efficacy: ‘value’ and ‘values’ are different things.


In everyday talk, we speak about ‘value’ when we want to justify or critique a state of affairs that has a price: is the price right, too low, or too high? We may disagree on the price, but we do agree that something is for sale. At other times we reject the idea that a thing should be for sale, like a family heirloom, love or education. If people tell us otherwise, we question their values. This is because values are about relationships and politics.


When we ask about the values of EdTech in higher education, we are really asking: what type of relations do we think are virtuous and appropriate for the institution? What relationships are we forging and replacing between machines and people, and between people and people?


When it comes to the application of personal technology we have valued convenience, personalisation and seamlessness by forging very intimate but easily forgettable machine-human relations. This could happen in the EdTech space as well. Speech-to-text recognition, natural language processing and machine vision are examples of how bonds can be built between humans and computers, aiding feedback loops by making worlds of learning computable.


Deciding on which learning relations to make computable, I argue, should be driven by values. Instead of seeing EdTech as a silver bullet that simply drives learning outcomes, it is more useful to think of it as technology that mediates learning relations and processes: what relationships do we value as important for students and when is technology helpful and unhelpful in establishing those? In this way, values can help us guide the way we account for the value of edtech.

Morten Hansen is a research associate on the Universities and Unicorns project at Lancaster University, and a PhD student at the Faculty of Education, University of Cambridge, United Kingdom. Hansen specialises in education markets and has previously worked as a researcher at the Saïd Business School in Oxford.


Leave a comment

Pandemic and post-pandemic HE performance in Poland, UK and Ukraine

by Justyna Maciąg,  Mateusz Lewandowski, Tammi Sinha, and Tetiana Prykhodko

This is one of a series of position statements developed following a conference on ‘Building the Post-Pandemic University’, organised on 15 September 2020 by SRHE member Mark Carrigan (Cambridge) and colleagues. The position statements are being posted as blogs by SRHE but can also be found on The Post-Pandemic University’s excellent and ever-expanding website. The original  statement can be found here.

Higher Education Institutions (HEIs) have had to change their delivery and ways of working at incredible speed. The disruptive innovation caused by the COVID-19 pandemic is having profound impacts on all stakeholders of HEIs. This study reports on the results of the project which has brought together 3 perspectives from Poland, the United Kingdom and the Ukraine. The purpose was to evaluate and compare the performance of Universities during this period. Performance is understood as assessment of support given by a University to its stakeholders in the following spheres: organisational, technical, technological, competency and social. This study will contribute to better understanding the context of value creation by Universities during the pandemic and post-pandemic period.

We took the perspective of HEI stakeholders into consideration (students, academics and administrative staff). Their opinions and comments were collected by interviews in the form of an online questionnaire with some open questions. We intended to give them a space to share their perspectives, emotions and feelings caused by the lockdown. The questions carried out thematic analysis around the following issues: 1) organisational (planning and communication); 2) technical, (platforms available, teaching methods); 3) technological, (bandwidth, equipment); 4) competency (your own learning and comfort with online learning); 5) social conditions (your environment for study) of higher education experience within the current COVID-19 pandemic and follow up research post-pandemic. The surveys started in the middle of June 2020 and continued till October 2020. Sampling followed the snowball method. Participants were self-selecting with links shared for the online Microsoft forms and Google questionnaires. 

We collected 396 questionnaires, 296 students, 100 university staff and academics (240 in Poland, 133 in Ukraine, 24 in UK). We would like to thank all of our participants for their contribution and candour.

First we would like to start with some qualitative analysis of students and staff responses in the questionnaire. The open questions were used to diagnose their experiences related to measures taken by Universities during lockdown. They were also asked to highlight the most and the least effective  solutions offered. We decided to use an Ishikawa Diagram to analyse the possible causes for their most and least solutions identified. We analysed the factors around the COVI19  problem in order to provide insights and possible solutions for an effective and thriving  ‘post-pandemic University’.

We grouped responses under headings showing below in the Ishikawa Diagram.

Chart 1 Ishikawa Diagram

There are some obvious similarities between these countries and some differences. We draw a conclusion that in each country the situation was similar, the teaching-learning process was transferred into our Virtual Learning Environments (VLE), and staff started remote or hybrid work (both academics and admin staff). The difference was notably the mechanism of this change: in the UK it was deemed more incremental change, in Ukraine and Poland the change was more radical. The proof of this is that in the responses in Poland and Ukraine respondents indicated several solutions which aren’t coordinated and supported by the Universities (i.e. using a social media for teaching-learning process, lack of integration of different e-learning platforms). Whereas in the UK many Universities used VLEs as ‘business as usual’. 

The common themes identified for research in investigated countries were the expectation of support in different areas, not only in a teaching-learning process, but also in equipment (provision, repairs), financial aid, mental sphere, and competence development etc. The findings implied that the expectations of the students and staff support needs were not fully met at this time. Universities were in survival mode and the change management process was lacking in many areas.

Next, we analysed the background given by quantitative analysis of University performance in the technical, competence and organisational sphere (evaluation was carried out using a 5-point Likert scale). The results of research in each country are shown on Chart 2.

Chart 2 Evaluation of the support given by university during lockdown (Poland, Ukraine, UK)

We also investigated the need of support and help provided during the pandemic period studied. The results are presented in Chart 3.

Chart 3 Percentage of respondents who declare that they need support or help

The results of research showed that ‘University’ is mentioned the most, as the expected supporter for students and staff, both academics and administrators. The importance of social support also has appeared in our results. People are looking for assistance among colleagues, thus creating a proper, strong internal social relationship is valuable for them. 

Table 1 The frequently mentioned sources of support

Our key conclusion from this work, at this time, is the importance of support and setting expectations of what support is available in HEIs. We  draw a key finding that the understanding of value delivered by ‘the University’ has to change, and leave behind the neoliberal concept of value for money. We need to expand the understanding of value, taking into account the necessity of tolerance perceived inefficiencies within the university. University staff and students have had to adapt very quickly, and use all of their skills and tenacity to deal with this situation. Creating and co-creating value within universities has always been challenging, however the creativity of staff and students has pulled this sector through it. We have all had to become disruptive innovators.

Justyna Maciąg, PhD,  and Mateusz Lewandowski, PhD, are lecturers and researches in the Institute of Public Affairs at Jagiellonian University in Cracow, Poland. 

Tammi Sinha, PhD, is Senior Lecturer in Operations and Project Management, Director of the Centre for Climate Action, University of Winchester UK. Tetiana Prykhodko is Head of the Program of Analysis and Research,  City Institute at Lviv, and a PhD student at Ivan Franko National University of Lviv, Ukraine.


Leave a comment

How likely are BTEC students to enter higher education?

By Pallavi Amitava Banerjee

 

Business and Technology Education Council (BTEC) qualifications are seen by some as prized qualifications for the labour market which draw on work-based scenarios. Providers claim these career-based qualifications are designed Continue reading

Vicky Gunn


2 Comments

The latest TEF Assessment Framework, automated analysis of data, and some Scottish anxiety

By  Vicky Gunn

I had the good fortune to be in Rio for the Paralympics this September. My step-daughter was competing in an endurance road race. For her, the most important thing was improving on previous race times, but she’d hoped to get a medal as well (even though this was not predicted by her ‘metrics’). At the end of September, the Westminster Government through HEFCE, published the TEF2 Technical Specification[i] and I found, to my astonishment that the original differentiating phrases (meets expectations, excellent, outstanding) were to be replaced with medals: Bronze, Silver, Gold. All of this got me thinking about the Teaching Excellence Framework, built like British Cycling on the idea that we can differentiate excellence for competitive purposes and this is a good in itself. I find this comparison deeply troubling. I, like many involved in quality and teaching development in Scottish Higher Education, have invested several years of my professional life to fostering cultures of enhancement. Indeed, the distance travelled to improvement in teaching provision has been a mantra within the totality of Scottish higher education’s stakeholders (academic, government, student bodies alike). In the Quality Enhancement Framework (QEF), we have been more interested in seeing all Scottish institutions getting ‘personal bests’ (hence demonstrating continuous improvement from within their own context), rather than doing better than all the others (final outcome measure).

However, now that we have the first set of TEF indicative metrics, I (like a cycling coach) am assailed with a few doubts about the laudable concentration of raising the quality of the whole Scottish sector. This is an aim of the QEF. This resulted in engaged participants of a quality system which steadfastly refused the divisiveness associated with differentiated institutional quality review outcomes. Yet, if we individually enter it, the TEF will now demand this of us.  Should Scotland then change its QEF substantially with its aspirational collectivism to be consigned to being a phantasm of a previous era? How long can such a discourse last in the face of going for gold? Should I, as an institutional Head of Learning and Teaching, now focus on competing with HEIs, so my institution is seen as outstanding in comparison to all the others and place the sector’s aspirational culture in a box marked ‘soppy idealism’? To put it in British Cycling’s inelegant but superlatively economic phrasing: how will my small specialist institution medal when facing larger, wealthier institutions?  Continue reading