SRHE Blog

The Society for Research into Higher Education


1 Comment

Doctoral progress reviews: managing KPIs or developing researchers?

by Tim Clark

All doctoral students in the UK are expected to navigate periodic, typically annual, progress reviews as part of their studies (QAA, 2020). Depending on the stage, and the individual institutional regulations, these often play a role in determining confirmation of doctoral status and/or continuation of studies. Given that there were just over 100,000 doctoral students registered in the UK in 2021 (HESA, 2022), it could therefore be argued that the progress review is a relatively prominent, and potentially high stakes, example of higher education assessment.  However, despite this potential significance, guidance relating to doctoral progress reviews is fairly sparse, institutional processes and terminology reflect considerable variations in approach, empirical research to inform design is extremely limited (Dowle, 2023) and perhaps most importantly, the purpose of these reviews is often unclear or contested.

At the heart of this lack of clarity appears to be a tension surrounding the frequent positioning of progress reviews as primarily institutional tools for managing key performance indicators relating to continuation and completion, as opposed to primarily pedagogical tools for supporting individual students learning (Smith McGloin, 2021). Interestingly however, there is currently very little research regarding effectiveness or practice in relation to either of these aspects. Yet, there is growing evidence to support an argument that this lack of clarity regarding purpose may frequently represent a key limitation in terms of engagement and value (Smith McGloin, 2021, Sillence, 2023; Dowle, 2023). As Bartlett and Eacersall (2019) highlight, the common question is ‘why do I have to do this?’

As a relatively new doctoral supervisor and examiner, with a research interest in doctoral pedagogy, in the context of these tensions, I sought to use a pedagogical lens to explore a small group of doctoral students’ experiences of navigating their progress review. My intention for this blog is to share some learning from this work, with a more detailed recent paper reporting on the study also available here (Clark, 2023). 

Methods and Approach

This research took place in one post-1992 UK university, where progress assessment consisted of submission of a written report, followed by an oral examination or review (depending on the stage). These progress assessments are undertaken by academic staff with appropriate expertise, who are independent of the supervision team. This was a small-scale study, involving six doctoral students, who were all studying within the humanities or social sciences. Students were interviewed using a semi-structured narrative ‘event-focused’ (Jackman et al, 2022) approach, to generate a rich narrative relating to their experience of navigating through the progress review as a learning event.

In line with the pedagogical focus, the concept of ‘assessment for learning’ was adopted as a theoretical framework (Wiliam, 2011). Narratives were then analysed using an iterative ‘visit and revisit’ (Srivastava and Hopwood, 2009) approach. This involved initially developing short vignettes to consider students’ individual experiences before moving between the research question, data and theoretical framework to consider key themes and ideas.

Findings

The study identified that the students understood their doctoral progress reviews as having significant potential for supporting their learning and development, but that specific aspects of the process were understood to be particularly important. Three key understandings arose from this: firstly, that the oral ‘dialogic’ component of the assessment was seen as most valuable in developing thinking, secondly, that progress reviews offered the potential to reframe and disrupt existing thinking relating to their studies, and finally, that progress reviews have the potential to play an important role in developing a sense of autonomy, permission and motivation.

In terms of design and practice, the value of the dialogic aspect of the assessment was seen as being in its potential to extend thinking through the assessor, as a methodological and disciplinary ‘expert’, introducing invitational, coaching format, questions to provoke reflection and provide opportunities to justify and explore research decisions. When this approach was taken, students recalled moments where they were able to make ‘breakthroughs’ in their thinking or where they later realised that the discussion was significant in shaping their future research decisions. Alongside this, a respectful and supportive approach was viewed as important in enhancing psychological safety and creating a sense of ownership and permission in relation to their work:

“I think having that almost like mentoring, which is like a mini mentoring or mini coaching session, in these examination spots is just really helpful”

“I’m pootling along and it’s going okay and now this bombshell’s just dropped, but it was helpful because, yeah, absolutely it completely shifted it.”

“It’s my study… as long as I can justify academically and back it up. Why I’ve chosen to do what I’ve done then that’s okay.” 

Implications

Clearly this is a small-scale study, with a relatively narrow disciplinary focus, however its value is intended to lie in its potential to provoke consideration of progress reviews as tools for teaching, learning and researcher development, rather than to assert any generalisable understanding for practice.

This consideration may include questions which are relevant for research leaders, supervisors and assessors/examiners, and for doctoral students. Most notably: is there a shared understanding of the purpose of doctoral progress reviews and why we ‘have’ to do it? And how does this purpose inform design, practice and related training within our institutions?

Within this study it was evident that in this context the role of dialogic assessment was significant, and given the additional resource required to protect or introduce such an approach, this may be an aspect which warrants further exploration and investigation to support decision making. In addition, it also framed the perceived value of the careful construction of questions, which invite and encourage reflection and learning, as opposed to seeking solely to ‘test’ this.

Dr Timothy Clark is Director of Research and Enterprise for the School of Education at the University of the West of England, Bristol. His research focuses on aspects of doctoral pedagogy and researcher development.


2 Comments

Can coaching bring back the joy to academic work?

By George Callaghan

Pause for a moment and jot down how many tasks and projects are currently at the front of your mind? You might already be thinking, “hold on, am I asked to pause, to stop thinking, stop doing, even for a moment? Does he not know how much I’ve got to do!” I would encourage you to give it a go.

Here are mine: write this blog, check work emails, check personal emails, re-read my Career Development Staff Appraisal Form for meeting later today, check train is going to be on time for said meeting, check if Waverley station has moved bike storage area since lock-down, check today’s to-do list I made yesterday, send the two qualitative interviews which have been transcribed to the printers…” OK, I will stop there – quite a long list which only took about 30 seconds to come up with. It also does not include other University work or general life stuff such as parenting, being in a relationship, owning pets, shopping and so on. The distinction between the private and professional life of academics is becoming increasingly blurred – and the pressure of work is becoming increasingly intense.

Then think back to when you embarked on your academic career, most likely full of excitement and joy at being able to pursue your intellectual passion for a subject, enthuse students, write papers, and successfully present at conferences.

What happened between the early excitement and present overload? How did our academic lives become so busy we barely have time for a coffee break, never mind time to think clearly and analytically? And crucially, what might we do about it?

While the answers to the changing nature of demands will be multi-factorial and include the marketisation of higher education and the pressure of research and teaching metrics, I argue in this blog that coaching offers a route-map to creating a more balanced and enjoyable professional life. It is an invitation to self-reflect, to recognise strengths, to develop insights, and to allow obstacles to be identified and overcome. This makes it a tremendously powerful staff development intervention.

Coaching can take several forms. For example, academic leaders and managers might use training to develop a coaching mindset. Here they would be using skills such as active listening and reflective inquiry to deepen the quality of their communication with colleagues. Alternatively, academic and professional staff might take dedicated one to one sessions with a trained and qualified coach.

Here, I begin to tell the story of how we are using coach training and coaching sessions to develop a coaching culture amongst academic staff within the Faculty of Arts and Social Sciences at the Open University. The project is still in its early stages but is showing great promise.

The initial idea was sparked by some coach training I engaged with as part of my professional development. I had a lightbulb moment when I realised that the constant curiosity, invitation to self-reflect and absence of judgement which underpin coaching conversations fit wonderfully well with the academic labour process. Many of us are drawn to work as university academics because we value agency, autonomy, and self-direction. As we know only too well, the current intensification of academic work militates against these, produces feelings of frustration and can be overwhelming. Coaching, with its focus on open questions and reflective inquiry, signposts new ways forward. Open questions and reflective inquiry may even lead to insights where we remember the joy and love of our work.

The project involves an external coach organisation providing introductory coaching skills training to academic leaders and managers. The positive early feedback led to expanding this offer of training coaching skills and to set up an internal coaching service where one to one coaching supports colleagues through career transitions.

We are presently working on an evaluation project using grounded theory methodology to analyse the impact of the coach skills training. The data is presently being collected and analysed and our aim is to offer a paper on this evaluation to Studies in Higher Education later in the year. Here, I offer my own reflections on what appears to be working – as well as some thoughts on what I might have done better.

In terms of what’s worked I am both refreshed and relieved to find that informal feedback and my own observations indicate that coaching adds value to the academic working life. One of these is the invitation to leaders and managers to self-reflect. To “listen more and talk less”.

As part of my own self-reflection, I began to pay attention to how I behave in meetings. Not how I thought I behaved, but what I do. I thought I consistently listened intently to others before making my own contribution. In fact, I was half listening to comments while internally formulating my own ‘excellent, articulate and very powerful’ contribution! I barely waited for others to stop speaking before I started. Acceptance of this embarrassing revelation led to a change in my listening. I began to concentrate on what others were saying. Not just to the words, but also the emotion behind the words. I began to pause before replying or I invited someone else to come in first. These are particularly challenging changes to make when one is chairing meetings or in a leadership and management position. Interestingly, once I let go of feeling responsibility for being the one with ‘the answer’ I felt more calm – and better ideas emerged.

In group or one to one meetings, taking the time to really listen generates new insights and opens the door to new possibilities. For leaders this can also be rather humbling as one realises others have equally (or more) valid ideas and solutions. This type of facilitative as opposed to directive leadership is particularly suited to academia, where the apprenticeship for the job involves independent thinking and the development of critical questioning.

This shift to leadership habits which draw on coaching, for example moving from ‘telling’ to ‘listening’, has the potential to motivate and energise colleagues. This takes time but offers substantial returns. Telling and directing is quicker in the short term – perhaps you are familiar with colleagues hesitating before making decisions, looking to first run it past a head of department, research lead or some other authority figure? While this style of management and leadership works to some extent (courses still get taught and research still gets done), it can create a dependent relationship. Leading through coaching invites colleagues to take more responsibility for their own – and consequently the university’s – development and growth.

What might I have done better? What immediately comes to mind is that I could have been much more patient. As I became convinced of coaching’s effectiveness, I set high expectations of uptake and the pace of change. The take up of coach training by leaders and managers did pick up, but over months and years as opposed to weeks. The habit of self-reflection I am (still) learning to practise has been of great assistance. The realisation that I must meet colleagues where they are now, not where I am.

Please consider how adopting a coaching mindset may be of service in improving the leadership and management in your own institution. You might reflect and think it is all working fine, but if you realise there is room for improvement then coaching may very well be of service. In the meantime, stay curious!

SRHE member George Callaghan is Professor of Personal Finance and Economics in the Faculty of Arts and Social Science at the Open University. He is also a qualified coach with the International Coaching Federation and the Institute of Leadership and Management. If you would like to discuss any points in this blog, please email George.callaghan@open.ac.uk