SRHE Blog

The Society for Research into Higher Education

Tag Archives: HE research methods


by 2 Comments

‘To diary or not to diary’? – lessons learned from the SRHE workshop ‘Using Diary Method in Social Research’

by Panagiota (Peny) Sotiropoulou

At the beginning of February 2024, I attended the in-person workshop on Using Diary Method in Social Research, organised by the SRHE and facilitated by Dr Emily Henderson, Dr Zoe Baker and Dr Ahmad Akkad.

Figure 1: Dr Zoe Baker presenting a group task during the workshop

As a reflective, life-long learner, I think there is no bigger satisfaction than sharing lessons learned from attending professional development opportunities like this with a wider audience. So, in this blog, I will present my main takeaways from the day and how I adapted what I learned to fit the diary method in a recent collaborative research proposal submitted for funding.

Using diary as a research method – my main takeaway messages from attending the workshop

  1. Using diary as a method for social research is perceived as a bit mysterious, “somewhat off the beaten path” (Hyers, 2018:vi), as it is still an underused method, without a big body of literature surrounding it. For this reason, researchers should be prepared to receive some initial hesitation from potential participants and be familiar with relevant existing studies that have used this method, to support and argue in favour of their choice.
  2. There is a difference between unsolicited (those pre-existing the research) and solicited diaries (those created for the purpose of the research). However, only the latter type of diary is relevant to researchers who are interested in incorporating diary entries in their research design to explore current topics.
  3. Simpler forms of diaries (e.g. paper templates or online documents) seem to work better than specialist diary apps in terms of facilitating participant engagement and retention.
  4. Two of the main advantages of the diary method are that it can provide access to settings that are hard for the researcher to enter (e.g. school classrooms) and that it provides the participants with the space, time and agency to think and decide what they want to share with the researcher, unlike an interview setting, for example, where participants have to react on the spot.
  5. Diaries are a good way to research marginalised groups and sensitive topics and provide a perfect way to explore lived, micro-level experiences.
  6. The success of a diary study is inextricably linked to the provision of clear instructions on when entries should be made and what participants should cover in those. In terms of the timing of entries, the main distinction is between interval-based (regular records kept over specific time intervals) and event-based sampling (records made every time the participant experiences something that qualifies as an event for the purpose of the research).
  7. The diary requirements should be in balance with participants’ availability, so that the diary does not become an onerous task.
  8. Diary studies come with a relatively high administrative burden, as researchers are required to sustain communications with their participants throughout the project to retain them and to monitor that their engagement is appropriate.
  9. Participation in diary studies might result in increased reactivity/self-awareness for participants (i.e. participants realise their circumstances better after keeping a diary). This might be either positive or negative, depending on whether or not participants take steps towards positive change.

And now what? Adapting diary methods to my research practice

At the time of the workshop I was involved in developing a collaborative research bid to examine the lived experiences of minority ethnic staff and students in Welsh higher education institutions (HEIs). My participation in the workshop consolidated my thinking that including diary elements in this project would be a perfect fit, for several reasons:

  1. Diaries would put the voices of minority ethnic staff and students at the forefront. By providing them with the space to create their own narratives, participants would be empowered to use their diaries to produce authentic and honest representations of their lived experiences.
  2. Diaries would allow participants to record their experiences in real time, providing detailed and context-rich data. This immediacy could capture nuances that might be missed by standalone interviews or surveys.
  3. The flexibility of diaries in terms of format (e.g. allowing for the inclusion of both written and audiovisual elements) and the ease with which they can be tailored to be submitted online would enable participants to adapt their diaries to what best suits their preferences and availability.
  4. Diary entries can complement other research methods, such as interviews and focus groups, to provide holistic exploration of lived experiences, with the latter acting as debrief opportunities to further explore the former.

As this bid came in response to a public tender there was a strict budget, a defined timeline and some specific methodologies requested that needed to be met for our proposal to be competitive. Diary methods were not amongst those requested and the time required for managing participants in such studies as well as analysing relevant findings made the inclusion of a pure diary study unsuitable for this specific bid. However, recognising the unique potential that incorporating diary elements would bring in exploring everyday experiences of minority ethnic staff and students, we decided to adopt participant-led multimedia-elicitation as a viable alternative.

Specifically, we thought that we would ask participants to use Padlet to capture multimedia depictions of their lived experiences accompanied by a short, explanatory commentary for their choice, with staff and students having separate Padlets to populate. We were particularly keen to allow for multimedia posts, so that we could capture more appropriately the various textures and facets comprising staff and students’ lived experience in HEIs (Metcalfe, 2016) (e.g. the sound of a spoken language -or the lack thereof- the places of inclusion/exclusion, as well as abstract concepts like ‘friendship’ etc).

Padlet perfectly served this purpose, as the platform affords for a variety of post types, such as audio recordings, website links, photos, songs. Padlet also allows for the creation of online peer communities, as it provides participants with the opportunity to interact with each other’s content. This further boosted our thinking to use the co-created Padlets of our participants as the basis for a subsequent online focus group discussion with them. This would enable us to better understand the meaning of the multimedia included as well as the interactions developed on the Padlets, as a means to shed more light on how these represent the lived experiences of minority ethnic staff and students.

Our choice was inspired by Keenan (2023), who reflects upon the diarying aspects of photo-elicitation, highlighting how this method is under-used in higher education and yet optimal for unravelling lived experiences. This is because it allows participants to be both creators and interpreters of the data, engaging them “in acts of diarying – both in terms of recording and reflexively interpreting everyday life” (Keenan 2023: 93).

Conclusions

Had I not attended the SRHE workshop, my understanding of the benefits of the diary method and its appropriateness for exploring lived experiences of marginalised communities in higher education would not have been so well-informed. It was learning from this workshop that prompted me to incorporate diary elements in the research bid on exploring minority ethnic staff and students’ lived experiences in higher education. Although the bid outcome is not yet published, I have already had a personal win, being able to include a new methodology in my practice.

So, many thanks to the SRHE for organising the session, to the facilitators for pitching it at such a perfect level, to my fellow participants, who were fully engaged and enriched the session even more by sharing their experiences, thoughts, and practices, and, last but not least, to my manager, who is always so supportive of me expanding my methodology repertoire and pursuing development opportunities. And here is my final lesson learned; do not hesitate to engage in professional development activities, as they are both educational and inspirational!

Dr Panagiota (Peny) Sotiropoulou is a mixed-methods researcher at Advance HE’s Insights Team. Her main interests lie in EDI considerations in HE, with a special focus on issues related to race and ethnicity. Her areas of specialisation involve mixed-methods research designs, impact and theory-based evaluations. Peny has extensive experience in programme evaluation, leading on Advance HE’s internal programme evaluations, in addition to those embedded to bespoke consultancy projects (read some of her recent work here). Peny has been involved with a wide array of projects, ranging from reviewing barriers to doctoral funding to institutional reporting and complaints processes. She has also been heavily involved in the production and dissemination of Advance HE’s annual Equality in Higher Education: Statistical Reports, as she loves to engage in outreach activities promoting EDI considerations to various audiences.

Get in contact with her on Twitter/X (@penpenwise) or LinkedIn.


by 1 Comment

The (future) state of higher education research?

by Stijn Daenekindt and Jeroen Huisman

  • Stijn Daenekindt
  • Jeroen Huisman
    Jeroen Huisman

Parallel to the exponential growth of research on higher education, we see an increasing number of scientific contributions aiming to take stock of our field of research. Such stock-taking activities range from reflective and possibly somewhat impressionistic thoughts of seasoned scholars to in-depth reviews of salient higher education themes. Technological advancements (such as easy electronic access to research output and an increasingly broader set of analytical tools) obviously have made life easier for analysts. We recently embarked upon a project to explore the thematic diversity in the field of research in higher education. The results have recently been published in Higher Education. Our aim was to thematically map the field of research on higher education and to analyse how our field has evolved over time.

For this endeavour, we wanted our analysis to be large-scale. We aimed at including a number of articles that would do justice to the presumed variety in research into higher education. We did not, however, want the scale of our analysis to jeopardize the depth of our analysis. Therefore, we decided not to limit our analyses to, for example, an analysis of citation patterns or of keywords. Finally, to forestall bias (stemming from our personal knowledge about and experience in the field), we applied an inductive approach. These criteria led us to collect 16,928 journal articles on higher education published between 1991 and 2018 and to analyse each article’s abstract by applying topic modelling. Topic modelling is a method of automated text analysis and a follow-up blogpost (also on srheblog.com) will address the method. For now, it suffices to know that topic modelling is a machine learning technique that automatically analyses the co-occurrence of words to detect themes/topics and to find structure in a large collection of text.

In this blogpost, we present a glimpse of our findings and some additional thoughts for further discussion. In our analysis, we differentiate 31 research topics which inductively emerged from the data. For example, we found topics dealing with university ranking and performance, sustainability, substance use of college students, research ethics, etc. The bulk of these research topics were studied at the individual level (16 topics), with far fewer at the organisational (5) and system level (3). A final set of topics related either clearly to disciplines (eg teaching psychology) or to more generic themes (methods, academic writing, ethics). This evidences the richness of research into higher education. Indeed, our field of research certainly is not limited in terms of perspectives and unleashes “the whole shebang” of possible perspectives to gain new insights into higher education.

The existence of different perspectives also comprises potential dangers, however. Studies applying a certain approach on higher education — say, a system-level approach — may suffer from tunnel vision and lose sight of individual- and organization-level aspects of higher education. This may be problematic as processes on the different levels are obviously related to one another. In our analysis we find that studies indeed tend to focus on one level. For example, system-level topics tend to be exclusively combined with other system-level topics. This should not come as a big surprise, but there is potential danger in this and it may hamper the development of a more integrated field of research on higher education.

In our analysis, we also find a certain restraint to combine topics which are located at the same level. For example, topics on teaching practices are very rarely combined with topics on racial and ethnic minorities — even though both topics are situated at the individual level. To us, this was surprising as the combination of ethnicity and educational experiences is a blossoming field in the sociology of education. The fact that topics at the same level are only rarely combined is less understandable then the fact that topics on different levels are rarely combined. We hope that our analysis aids others researchers to identify gaps in the literature and that it motivates them to address these gaps.

A second finding we wish to address here relates to specialisation. Our analysis suggests that there is a trend of specialisation in our field of research. We looked at the number of topics combined in articles and we see that topic diversity declines over time. This is, on the one hand, not that surprising. Back in 1962, Kuhn already argued that the system of modern science encourages researchers towards further specialisation. So, it makes sense that over time, and parallel to the growth of the field of research on higher education, researchers specialise more and demarcate their own topic of expertise. On the other hand, it may be considered a problematic evolution as it can hamper our field of research to develop towards further maturity.

But what should we think of the balance between healthy expansion and specialization, on the one hand, and inefficient fragmentation, on the other? We lean towards evaluating the current state of higher education research as moving towards fragmentation. Other researchers, such as Malcom Tight, Bruce Macfarlane and Sue Clegg have similarly lamented the fragmented nature of our field of research. Our analysis adds to this by showing the trends over time: we observe more specialisation (not necessarily bad), but there are also signs of disintegration over time (not good). Other analyses we are currently carrying out also indicate thematic disintegration and suggest clear methodological boundaries. It looks like many researchers focusing on the same topic remain in their “comfort zone” and use a limited set of methods. For sure, many methodological choices are functional (as in fit-for-purpose), but the lack of diversity is striking. Moreover, we see that many higher education researchers stick to rather traditional techniques (survey, interviews, case studies) and that new methods hardly get picked up in our field. A final observation is that we hardly see methodological debates in our field. In related disciplines we often see healthy methodological discussions that improve the available “toolkit” (for example here). In our field, it appears that scholars shy away from such discussions and it suggests methodological conservatism and/or methodological tunnel vision.

There are still many things to investigate to arrive at a full assessment of the state of the art. One important question is how our field compares to other fields or disciplines. But if we were to accept the idea of fragmentation, it is pertinent to start thinking how to combat this. Reversing this trend is obviously not straightforward. But here are a few ideas. Individual scholars could try to get out of their comfort zone by applying other perspectives to their favourite research object and/or by applying their favourite perspective to new research topics. Related, researchers should be encouraged to use techniques less commonly used in our field and see whether they yield different outcomes (vignettes, experimental designs, network analysis, QCA/fuzzy logic, [auto-]ethnography and – of course – topic models). In addition, journal editors could be more flexible and inclusive in terms of the format of the submissions they consider. For example, they could explicitly welcome submissions in the format of ‘commentaries/ a reply to’. This would stimulate debate and open up the floor for increased cross-fertilisation of research into higher education and, in general, signal the maturity of research into higher education. Finally, there is scope for alternative peer review processes. Currently, only editors (and sometimes peer reviewers seeing the outcome of a peer review process) gain full insight in feedback offered by peers. If we would make these processes more visible to a broader readership – e.g. through open peer review, which still can be double-blind – we would gain much more insight in methodological and theoretical debates, that would definitely support the healthy growth of our field.  

This post is based on the article: Daenekindt, S and Huisman, J (2020) ‘Mapping the scattered field of research on higher education. A correlated topic model of 17,000 articles, 1991–2018’ Higher Education, 1-17. Stijn Daenekindt is a Postdoctoral Researcher at Ghent University (Department of Sociology). SRHE Fellow Jeroen Huisman is a Full Professor at Ghent University (Department of Sociology).