SRHE Blog

The Society for Research into Higher Education

Tag Archives: research methods


by 2 Comments

‘To diary or not to diary’? – lessons learned from the SRHE workshop ‘Using Diary Method in Social Research’

by Panagiota (Peny) Sotiropoulou

At the beginning of February 2024, I attended the in-person workshop on Using Diary Method in Social Research, organised by the SRHE and facilitated by Dr Emily Henderson, Dr Zoe Baker and Dr Ahmad Akkad.

Figure 1: Dr Zoe Baker presenting a group task during the workshop

As a reflective, life-long learner, I think there is no bigger satisfaction than sharing lessons learned from attending professional development opportunities like this with a wider audience. So, in this blog, I will present my main takeaways from the day and how I adapted what I learned to fit the diary method in a recent collaborative research proposal submitted for funding.

Using diary as a research method – my main takeaway messages from attending the workshop

  1. Using diary as a method for social research is perceived as a bit mysterious, “somewhat off the beaten path” (Hyers, 2018:vi), as it is still an underused method, without a big body of literature surrounding it. For this reason, researchers should be prepared to receive some initial hesitation from potential participants and be familiar with relevant existing studies that have used this method, to support and argue in favour of their choice.
  2. There is a difference between unsolicited (those pre-existing the research) and solicited diaries (those created for the purpose of the research). However, only the latter type of diary is relevant to researchers who are interested in incorporating diary entries in their research design to explore current topics.
  3. Simpler forms of diaries (e.g. paper templates or online documents) seem to work better than specialist diary apps in terms of facilitating participant engagement and retention.
  4. Two of the main advantages of the diary method are that it can provide access to settings that are hard for the researcher to enter (e.g. school classrooms) and that it provides the participants with the space, time and agency to think and decide what they want to share with the researcher, unlike an interview setting, for example, where participants have to react on the spot.
  5. Diaries are a good way to research marginalised groups and sensitive topics and provide a perfect way to explore lived, micro-level experiences.
  6. The success of a diary study is inextricably linked to the provision of clear instructions on when entries should be made and what participants should cover in those. In terms of the timing of entries, the main distinction is between interval-based (regular records kept over specific time intervals) and event-based sampling (records made every time the participant experiences something that qualifies as an event for the purpose of the research).
  7. The diary requirements should be in balance with participants’ availability, so that the diary does not become an onerous task.
  8. Diary studies come with a relatively high administrative burden, as researchers are required to sustain communications with their participants throughout the project to retain them and to monitor that their engagement is appropriate.
  9. Participation in diary studies might result in increased reactivity/self-awareness for participants (i.e. participants realise their circumstances better after keeping a diary). This might be either positive or negative, depending on whether or not participants take steps towards positive change.

And now what? Adapting diary methods to my research practice

At the time of the workshop I was involved in developing a collaborative research bid to examine the lived experiences of minority ethnic staff and students in Welsh higher education institutions (HEIs). My participation in the workshop consolidated my thinking that including diary elements in this project would be a perfect fit, for several reasons:

  1. Diaries would put the voices of minority ethnic staff and students at the forefront. By providing them with the space to create their own narratives, participants would be empowered to use their diaries to produce authentic and honest representations of their lived experiences.
  2. Diaries would allow participants to record their experiences in real time, providing detailed and context-rich data. This immediacy could capture nuances that might be missed by standalone interviews or surveys.
  3. The flexibility of diaries in terms of format (e.g. allowing for the inclusion of both written and audiovisual elements) and the ease with which they can be tailored to be submitted online would enable participants to adapt their diaries to what best suits their preferences and availability.
  4. Diary entries can complement other research methods, such as interviews and focus groups, to provide holistic exploration of lived experiences, with the latter acting as debrief opportunities to further explore the former.

As this bid came in response to a public tender there was a strict budget, a defined timeline and some specific methodologies requested that needed to be met for our proposal to be competitive. Diary methods were not amongst those requested and the time required for managing participants in such studies as well as analysing relevant findings made the inclusion of a pure diary study unsuitable for this specific bid. However, recognising the unique potential that incorporating diary elements would bring in exploring everyday experiences of minority ethnic staff and students, we decided to adopt participant-led multimedia-elicitation as a viable alternative.

Specifically, we thought that we would ask participants to use Padlet to capture multimedia depictions of their lived experiences accompanied by a short, explanatory commentary for their choice, with staff and students having separate Padlets to populate. We were particularly keen to allow for multimedia posts, so that we could capture more appropriately the various textures and facets comprising staff and students’ lived experience in HEIs (Metcalfe, 2016) (e.g. the sound of a spoken language -or the lack thereof- the places of inclusion/exclusion, as well as abstract concepts like ‘friendship’ etc).

Padlet perfectly served this purpose, as the platform affords for a variety of post types, such as audio recordings, website links, photos, songs. Padlet also allows for the creation of online peer communities, as it provides participants with the opportunity to interact with each other’s content. This further boosted our thinking to use the co-created Padlets of our participants as the basis for a subsequent online focus group discussion with them. This would enable us to better understand the meaning of the multimedia included as well as the interactions developed on the Padlets, as a means to shed more light on how these represent the lived experiences of minority ethnic staff and students.

Our choice was inspired by Keenan (2023), who reflects upon the diarying aspects of photo-elicitation, highlighting how this method is under-used in higher education and yet optimal for unravelling lived experiences. This is because it allows participants to be both creators and interpreters of the data, engaging them “in acts of diarying – both in terms of recording and reflexively interpreting everyday life” (Keenan 2023: 93).

Conclusions

Had I not attended the SRHE workshop, my understanding of the benefits of the diary method and its appropriateness for exploring lived experiences of marginalised communities in higher education would not have been so well-informed. It was learning from this workshop that prompted me to incorporate diary elements in the research bid on exploring minority ethnic staff and students’ lived experiences in higher education. Although the bid outcome is not yet published, I have already had a personal win, being able to include a new methodology in my practice.

So, many thanks to the SRHE for organising the session, to the facilitators for pitching it at such a perfect level, to my fellow participants, who were fully engaged and enriched the session even more by sharing their experiences, thoughts, and practices, and, last but not least, to my manager, who is always so supportive of me expanding my methodology repertoire and pursuing development opportunities. And here is my final lesson learned; do not hesitate to engage in professional development activities, as they are both educational and inspirational!

Dr Panagiota (Peny) Sotiropoulou is a mixed-methods researcher at Advance HE’s Insights Team. Her main interests lie in EDI considerations in HE, with a special focus on issues related to race and ethnicity. Her areas of specialisation involve mixed-methods research designs, impact and theory-based evaluations. Peny has extensive experience in programme evaluation, leading on Advance HE’s internal programme evaluations, in addition to those embedded to bespoke consultancy projects (read some of her recent work here). Peny has been involved with a wide array of projects, ranging from reviewing barriers to doctoral funding to institutional reporting and complaints processes. She has also been heavily involved in the production and dissemination of Advance HE’s annual Equality in Higher Education: Statistical Reports, as she loves to engage in outreach activities promoting EDI considerations to various audiences.

Get in contact with her on Twitter/X (@penpenwise) or LinkedIn.


by 1 Comment

The (future) state of higher education research?

by Stijn Daenekindt and Jeroen Huisman

  • Stijn Daenekindt
  • Jeroen Huisman
    Jeroen Huisman

Parallel to the exponential growth of research on higher education, we see an increasing number of scientific contributions aiming to take stock of our field of research. Such stock-taking activities range from reflective and possibly somewhat impressionistic thoughts of seasoned scholars to in-depth reviews of salient higher education themes. Technological advancements (such as easy electronic access to research output and an increasingly broader set of analytical tools) obviously have made life easier for analysts. We recently embarked upon a project to explore the thematic diversity in the field of research in higher education. The results have recently been published in Higher Education. Our aim was to thematically map the field of research on higher education and to analyse how our field has evolved over time.

For this endeavour, we wanted our analysis to be large-scale. We aimed at including a number of articles that would do justice to the presumed variety in research into higher education. We did not, however, want the scale of our analysis to jeopardize the depth of our analysis. Therefore, we decided not to limit our analyses to, for example, an analysis of citation patterns or of keywords. Finally, to forestall bias (stemming from our personal knowledge about and experience in the field), we applied an inductive approach. These criteria led us to collect 16,928 journal articles on higher education published between 1991 and 2018 and to analyse each article’s abstract by applying topic modelling. Topic modelling is a method of automated text analysis and a follow-up blogpost (also on srheblog.com) will address the method. For now, it suffices to know that topic modelling is a machine learning technique that automatically analyses the co-occurrence of words to detect themes/topics and to find structure in a large collection of text.

In this blogpost, we present a glimpse of our findings and some additional thoughts for further discussion. In our analysis, we differentiate 31 research topics which inductively emerged from the data. For example, we found topics dealing with university ranking and performance, sustainability, substance use of college students, research ethics, etc. The bulk of these research topics were studied at the individual level (16 topics), with far fewer at the organisational (5) and system level (3). A final set of topics related either clearly to disciplines (eg teaching psychology) or to more generic themes (methods, academic writing, ethics). This evidences the richness of research into higher education. Indeed, our field of research certainly is not limited in terms of perspectives and unleashes “the whole shebang” of possible perspectives to gain new insights into higher education.

The existence of different perspectives also comprises potential dangers, however. Studies applying a certain approach on higher education — say, a system-level approach — may suffer from tunnel vision and lose sight of individual- and organization-level aspects of higher education. This may be problematic as processes on the different levels are obviously related to one another. In our analysis we find that studies indeed tend to focus on one level. For example, system-level topics tend to be exclusively combined with other system-level topics. This should not come as a big surprise, but there is potential danger in this and it may hamper the development of a more integrated field of research on higher education.

In our analysis, we also find a certain restraint to combine topics which are located at the same level. For example, topics on teaching practices are very rarely combined with topics on racial and ethnic minorities — even though both topics are situated at the individual level. To us, this was surprising as the combination of ethnicity and educational experiences is a blossoming field in the sociology of education. The fact that topics at the same level are only rarely combined is less understandable then the fact that topics on different levels are rarely combined. We hope that our analysis aids others researchers to identify gaps in the literature and that it motivates them to address these gaps.

A second finding we wish to address here relates to specialisation. Our analysis suggests that there is a trend of specialisation in our field of research. We looked at the number of topics combined in articles and we see that topic diversity declines over time. This is, on the one hand, not that surprising. Back in 1962, Kuhn already argued that the system of modern science encourages researchers towards further specialisation. So, it makes sense that over time, and parallel to the growth of the field of research on higher education, researchers specialise more and demarcate their own topic of expertise. On the other hand, it may be considered a problematic evolution as it can hamper our field of research to develop towards further maturity.

But what should we think of the balance between healthy expansion and specialization, on the one hand, and inefficient fragmentation, on the other? We lean towards evaluating the current state of higher education research as moving towards fragmentation. Other researchers, such as Malcom Tight, Bruce Macfarlane and Sue Clegg have similarly lamented the fragmented nature of our field of research. Our analysis adds to this by showing the trends over time: we observe more specialisation (not necessarily bad), but there are also signs of disintegration over time (not good). Other analyses we are currently carrying out also indicate thematic disintegration and suggest clear methodological boundaries. It looks like many researchers focusing on the same topic remain in their “comfort zone” and use a limited set of methods. For sure, many methodological choices are functional (as in fit-for-purpose), but the lack of diversity is striking. Moreover, we see that many higher education researchers stick to rather traditional techniques (survey, interviews, case studies) and that new methods hardly get picked up in our field. A final observation is that we hardly see methodological debates in our field. In related disciplines we often see healthy methodological discussions that improve the available “toolkit” (for example here). In our field, it appears that scholars shy away from such discussions and it suggests methodological conservatism and/or methodological tunnel vision.

There are still many things to investigate to arrive at a full assessment of the state of the art. One important question is how our field compares to other fields or disciplines. But if we were to accept the idea of fragmentation, it is pertinent to start thinking how to combat this. Reversing this trend is obviously not straightforward. But here are a few ideas. Individual scholars could try to get out of their comfort zone by applying other perspectives to their favourite research object and/or by applying their favourite perspective to new research topics. Related, researchers should be encouraged to use techniques less commonly used in our field and see whether they yield different outcomes (vignettes, experimental designs, network analysis, QCA/fuzzy logic, [auto-]ethnography and – of course – topic models). In addition, journal editors could be more flexible and inclusive in terms of the format of the submissions they consider. For example, they could explicitly welcome submissions in the format of ‘commentaries/ a reply to’. This would stimulate debate and open up the floor for increased cross-fertilisation of research into higher education and, in general, signal the maturity of research into higher education. Finally, there is scope for alternative peer review processes. Currently, only editors (and sometimes peer reviewers seeing the outcome of a peer review process) gain full insight in feedback offered by peers. If we would make these processes more visible to a broader readership – e.g. through open peer review, which still can be double-blind – we would gain much more insight in methodological and theoretical debates, that would definitely support the healthy growth of our field.  

This post is based on the article: Daenekindt, S and Huisman, J (2020) ‘Mapping the scattered field of research on higher education. A correlated topic model of 17,000 articles, 1991–2018’ Higher Education, 1-17. Stijn Daenekindt is a Postdoctoral Researcher at Ghent University (Department of Sociology). SRHE Fellow Jeroen Huisman is a Full Professor at Ghent University (Department of Sociology).


by 1 Comment

Redrawing research methods and rewriting data

by Kate Carruthers Thomas

The call for papers for the SRHE 2019 Conference slid into my inbox not so long ago, marking the point in the year when the mind must focus in the short-term, in order to benefit from all things Celtic Manor in the longer term! The conference theme: Creativity, Criticality and Conformity in Higher Education invites debate on transcending the traditional and building an innovative research culture. The theme is timely in view of my own recent experiments involving graphics and poetry in social sciences research.

One year ago, I sat with a mass of rich qualitative data I’d collected for Gender(s) At Work, a research project investigating gendered experiences of work and career trajectory in higher education (HE). I’d interviewed 50 members of staff, identifying as female, male and gender non-binary, working in academic and professional services roles within one UK university. I set about analysing the data using Massey’s theory of geographies of power operating within space. I wanted to explore ways in which gender operates as a ‘geography of power’ within HE and the extent to which participants’ diverse and complex lived experiences trouble the prevailing career narrative of linear, upward trajectory.

Clear space soon emerged between the rhetoric of gender equality and lived experiences in the workplace and throughout working lives. Despite decades of equal opportunities legislation and institutional equality policies, the glass ceiling remains a feature of our sector. Elements of less familiar career archetypes: the glass cliff (Ryan and Haslam, 2005; Bruckmuller et al, 2014); the glass escalator (Williams, 2013; Budig, 2002) and the glass closet (Merriam-Webster, 2018) also surfaced in the transcripts.  These metaphors, archetypal and architectural – were something of a gift to a researcher concerned with the relationship between space and power. I found myself experimenting – you might call it doodling – with visual representations of the glass ceiling,  escalator, cliff and closet.

Using the visual was not completely new territory for me; as a doctoral student I had employed visual mapping as a research tool (Carruthers Thomas, 2018a) and tentatively used abstract diagrams as aids to explaining my theoretical framework and findings (Thomas, 2016), but I hadn’t picked up a pencil with intent since school art lessons. Nevertheless, four cartoon characters emerged from my doodles; embodiments of gendered dis/advantage in the HE workplace.

Throughout the Gender(s) At Work project, I had been disseminating emerging findings through conference papers and PowerPoint presentations. I had written a chapter about my research methodology (Carruthers Thomas, 2019a). As academics we anticipate and reproduce such formats; they keep the academic wheels turning and form the building blocks of academic credibility. With data collection complete however, I was unsure that the temporal and structural constraints of these conventions were going to do justice to the volume of complex personal narratives entrusted to me by research participants. I was also becoming increasingly drawn towards McLure’s argument for

immersion in and entanglement with the minutiae of the data … an experimentation or crafting … a very different kind of engagement with data from the distanced contemplation of the table that is the arrested result of the process.

(McLure, 2013: 174-175).

In March 2018, the Sociological Review explicitly invited unconventional contributions to its conference: Undisciplining: Conversations from the Edges. Still enjoying my experimentation with cartooning, I decided to explore the possibilities of communicating my research findings through a ‘graphic essay’ entitled My Brilliant Career? An Investigation. This would be in the format of a large-scale, hand-drawn comic strip conforming to the structural conventions of an essay or article. My proposal was accepted and the work began! The learning curve was precipitous!

 In June 2018, I exhibited My Brilliant Career? An Investigation at Undisciplining (Carruthers Thomas, 2018b) in the impressive surroundings of BALTIC Gateshead. The four A2-sized panels remained on display throughout the three days of the conference. It was strikingly different, communicating my research this way rather than hothousing it in a 20 minute Powerpoint presentation. Many delegates returned to the exhibit several times to look, bring colleagues, take photographs, ask questions. I engaged in discussions not only about the medium, but about the research process and findings too. And I myself engaged anew with the work, as an exhibit, rather than a cherished work in progress. I later translated the four panels into an A1-sized academic poster, displayed at the SRHE 2018 Annual Conference.

Meanwhile, another call for unconventional conference contributions in the form of poetic and performative work, had come from the Art of Management and Organisation (AoMO). This triggered a second experiment in creative criticality resulting in Glass, a long poem also based on the Gender(s) At Work data. Unlike graphic art, in poetry I do have a track record (Carruthers Thomas, 2018c), but had not considered blurring the boundary between poetry and academia until this call. Yet, as an academic my research practice involves collecting, analysing, distilling and presenting data. My research is a form of enquiry seeking enhanced intelligence and evidence to advocate organisational, structural and cultural change. As a poet, I follow a similar process to create a poem. More, or less, consciously I collect data: ideas, questions, emotions, sense phenomena, then manipulate language and sound to distil the data into poetic form. Glass brings these practices together.

To write it, I returned yet again to the interview transcripts, creating a poem comprising four sections – ceiling, escalator, closet and cliff – using participants’ words and a narrative framework featuring the researcher’s voice, using original poetry. Glass was deliberately written as a piece to be performed, another first, as I had only previously written poems for the page.

Even now, even now in my meetings

I’m still faced with wall to wall suits.

And I still hear my colleagues repeating

the proposal I tried to get through weeks ago

Great idea!

                                                                                (extract from Cliff, Glass 2018)

Glass and My Brilliant Career were created independently of one another, in different media but they draw on the same research data. This is not all they share. Both involved an extended process of analysis and representation; repeated revisiting of the data and work in painstaking detail. Both explicitly draw on and draw in, the affective, bringing the potential for surprise, humour, anger and pain into the room without apology. Finally, both also required me to allow myself to be vulnerable to audience resistance, discomfort, critiques on multiple levels and questions of academic validity.

Largely positive responses to the graphic essay and the research poem at those conferences set me thinking about ways to signpost the potential of creative approaches in social science research more widely and led to another experiment in academic practice.  I designed a multi-modal dissemination programme to take the findings of Gender(s) At Work out to UK universities and research institutes.  The programme featured six ‘options’ from which host institutions could select, mix and match: the exhibit My Brilliant Career? An Investigation; the research poem Glass; a conventional Powerpoint presentation of the research findings: The Workplace Glassed and Gendered and another giving an illustrated account of my emerging graphic social science practice: The Accidental Cartoonist. Building on both the research findings and visual methods, I also designed two participative workshops. Mapping Career challenged participants to develop meaningful visual alternatives to the reductive metaphors of career ladder and pipeline and On The Page explored the way simple visual and graphic methods might be used in research and teaching. I publicised the programme via email across the UK HE sector.

The response was extraordinary. Since November 2018 I’ve visited universities and research institutes from Edinburgh to London; Cambridge to Bangor. Audiences have included academics in all disciplines, professional services staff, senior management, conference delegates, Athena SWAN teams, women’s networks and mentoring groups, postgraduate and undergraduate students. I called the initiative the ‘gword tour’ after my blog the g word (that’s g for gender).   Six months, 30 ‘gigs’ – all that’s missing is the T-shirt!

One day I might be discussing Gender(s) At Work aims, research methods and findings to Athena SWAN leads and women’s networks; on another I’ll be delivering the Mapping Career workshop at a staff conference. I’ve presented The Accidental Cartoonist to academic developers and EdD students and encouraged academics to experiment with visual methods in their research and teaching practices in the On The Page workshop. Glass has been performed at some unlikely venues, including the Wellcome Sanger Genome Campus, the Stansted Airport Novotel – and to audiences somewhat larger than those at the average poetry reading!

How will you crack the glass enclosing some,

exposing some, blinding others

to their privilege?

Reflect on it.

                                                                                (extract from Epilogue, Glass 2018)

Throughout the gword tour I have diligently handed out structured feedback forms (in return for a free postcard), providing me with a continuous feedback loop and resulting in adaptation and tweaking of individual sessions throughout. Now the tour has concluded, a large pile of completed forms await me and I’m looking forward to getting the bigger picture. Meanwhile I’m already musing on two questions which have arisen throughout the past year. Firstly, whether and how addressing familiar topics through unfamiliar media can disrupt audience expectations and dislodge habitual responses to tricky subjects such as gender equality; secondly, whether what I have described in this blog constitutes being ‘differently academic’. 

By ‘differently academic’ I mean taking the opportunity to sit with our data for longer, deliberately to approach it from different angles, to explore its creative dimensions. I mean bringing data to diverse audiences, in diverse ways over an extended period, a process which has only further energised and deepened my engagement in the original research questions. Audience after audience has grilled me on my research rationale, process, findings, limitations and implications. Each time, their questions, comments and challenges have pushed my analyses further and opened new lines of enquiry.

I fully intend to publish my reflections on these questions in conventional academic formats: papers, articles and chapters.It may be that creative, critical work in our field can only gain academic legitimacy through this route.Meanwhile, other opportunities have arisen. Glass was published in the Sociological Fiction Zine in May 2019 (Carruthers Thomas, 2019b). I am currently working on a set of visuals for a new academic research centre and will be poet-in-residence at an academic conference in November 2019. The SRHE call for papers defines creativity as ‘transcending traditional ideas, rules, patterns, relationships’. I hope to continue to be creative and critical in my academic work, not for transcending’s sake, but ‘to create meaningful new ideas, forms, methods, and interpretations’.

SRHE member Dr Kate Carruthers Thomas is Senior Research Fellow and Athena SWAN Project Manager at Birmingham City University  kate.thomas@bcu.ac.uk  @drkcarrutherst Blog: – the g word https://thegword2017.wordpress.com/

References

Bruckmüller, S, Ryan, M, Rink, F and Haslam, SA (2014) ‘Beyond the Glass Ceiling: The Glass Cliff and its Lessons for Organizational Policy’, Social Issues and Policy Review, 8(1): 202-232

Budig, M (2002) ‘Male Advantage and the Gender Composition of Jobs: Who Rides the Glass Escalator?’, Social Problems 49(2): 258-277

Carruthers Thomas, K (2018a) Rethinking Student Belonging in Higher Education: From Bourdieu to Borderlands, Abingdon: Routledge

Carruthers Thomas, K (2018b) My Brilliant Career? An Investigation. Graphic Essay exhibited at Undisciplining: Conversations from the Edges Sociological Review, Gateshead, BALTIC.  June 2018

Carruthers Thomas, K (2018c) Navigation, Blaenau Ffestiniog, Cinnamon Press. 

Carruthers Thomas, K (2019a) ‘Gender as a Geography of Power’ in G Crimmins (ed) Resisting Sexism in the Academy: Higher Education, Gender and Intersectionality, Basingstoke: Palgrave Macmillan

Carruthers Thomas, K (2019b) Glass. Sociological Fiction Zine, Edition #5 http://www.sofizine.com.

McLure, M (2013) ‘Classification or Wonder? Coding as an Analytic Practice in Qualitative Research’, In Coleman, R and Ringrose, J (eds) Deleuze and Research Methodologies, Edinburgh: Edinburgh University Press.  Chapter 9. pp.164-183.

Merriam-Webster (2019). [online] Available from https://www.merriam-webster.com/dictionary/glass%20closet Accessed 28 May 2019.

Ryan, M and Haslam, A. (2005) The Glass Cliff: Evidence that Women are Over-Represented in Precarious Leadership Positions, British Journal of Management, 16(2): 81-90

Thomas, K (2016). Dimensions of belonging: rethinking retention for mature, part-time undergraduates in English higher education, PhD thesis, Birkbeck, University of London

Williams, M (2013) ‘The Glass Escalator, Revisited. Gender Inequality in Neoliberal Times’, Gender & Society 27 (5): 609–629