SRHE Blog

The Society for Research into Higher Education

Tag Archives: Assessment policy


by 3 Comments

The hidden layers of transparency in UK HE assessment practices

by Chahna Gonsalves and Zhongan Lin

Transparency in assessment practices is a critical component of the UK’s higher education sector, but it is a term that carries many layers of meaning. This blog post explores a study that examined how transparency is framed in assessment policies across 151 UK higher education institutions (HEIs). The findings reveal that while institutions strive for transparency, they often overlook the complexities and multidimensional nature of the concept.

Understanding transparency: more than just clear documentation

Transparency in assessment is often associated with clear documentation of criteria, grading practices, and feedback mechanisms. However, this techno-rational approach, which emphasizes explicit documentation and information dissemination, is just one facet of transparency. Our study highlights the need for a more nuanced understanding that includes socio-cultural practices and socio-material enactments.

Techno-rational approaches: the dominant paradigm

The study found that techno-rational approaches dominate the transparency discourse in HEI policies. These approaches focus on ensuring that assessment criteria, learning outcomes, and grading standards are clearly articulated and accessible. For example, many policies mandate the use of detailed assessment briefs, rubrics, and grade descriptors. While this approach aims to make evaluative processes clear and consistent, it often falls short in addressing the dynamic and interpretive nature of academic standards.

One of the most compelling findings was the over-reliance on explicit standards documents, which presume that written criteria can universally ensure fairness and consistency. This static view overlooks the reality that academic standards are co-constructed within specific social and cultural contexts. Without acknowledging this, policies may fail to convey the nuanced, tacit knowledge necessary for fully understanding and applying assessment criteria.

The limitations of techno-rational transparency

Simply providing clear documentation does not guarantee that all stakeholders will understand or effectively use the information. For instance, non-native English speakers and students with varying levels of academic literacy may struggle with the language used in assessment criteria. Moreover, policies often fail to specify effective methods for disseminating this information, relying heavily on static documents rather than interactive or diverse formats that could enhance understanding.

Socio-cultural practices: engaging stakeholders in meaningful dialogue

Beyond documentation, transparency also involves socio-cultural practices that engage stakeholders in ongoing dialogue and clarification of assessment criteria. Policies that promote discussion between educators and students, co-creation of assessment criteria, and collaborative marking processes can foster a deeper understanding and shared meaning of what is expected. For instance, involving students in the creation of rubrics and providing opportunities for mock marking can enhance their evaluative judgment and assessment literacy.

One interesting insight from the study was the importance of dialogue in building a shared understanding of assessment standards. Policies that encourage discussion about assessment criteria not only help students grasp what is expected but also allow educators to refine and clarify their expectations. This dynamic, interactive process contrasts sharply with the static dissemination of information typical of techno-rational approaches.

Socio-material enactments: the role of tools and artefacts

The study also highlights the importance of socio-material enactments, where transparency is realized through the interaction between social practices and material artifacts. This includes the use of digital platforms, rubrics, exemplars, and other assessment tools that facilitate a tangible understanding of assessment criteria. Effective use of these tools can bridge the gap between educators’ tacit knowledge and students’ understanding, fostering a more comprehensive view of transparency.

For example, the use of digital platforms to share assessment criteria and feedback can significantly enhance transparency. These platforms allow for continuous access and interaction with assessment materials, making it easier for students to understand and engage with the criteria. However, the study found that detailed guidance on such platforms is often scant in policies, pointing to a significant area for improvement.

Who benefits from transparency? A multifaceted audience

Transparency in assessment is not solely for students. It also encompasses other stakeholders, including markers, external examiners, tutors, and even employers. The study found that while most policies address the need for transparency for students and markers, they often neglect other crucial stakeholders. This oversight can lead to inconsistencies in how assessments are interpreted and applied, potentially undermining the fairness and effectiveness of the evaluation process.

A particularly intriguing aspect of the study was the identification of specific roles and responsibilities for promoting transparency. By clearly defining who is responsible for ensuring transparency – whether it be module leaders, programme teams, or tutors – institutions can better align their policies with the needs of various stakeholders. This clarity can help avoid the pitfalls of ambiguous roles and ensure a more consistent application of assessment criteria.

Methodology: building the framework

To develop a comprehensive framework we conducted a detailed content analysis of assessment policy documents from 151 UK HEIs. The data collection process involved systematically retrieving and examining these publicly accessible documents, which included academic manuals, assessment policies, feedback strategies, and codes of practice. We excluded documents that were outdated or inaccessible, resulting in a final corpus of 264 documents. Through both deductive and inductive coding methods, we analysed the texts to identify recurring themes and patterns related to transparency. This process involved categorising the data into three main discourses – techno-rational, socio-cultural, and socio-material – guided by Ajjawi, Bearman, and Boud’s (2021) framework. The iterative coding and categorization helped us build a nuanced understanding of how transparency is conceptualized and communicated in HEI assessment policies.

Towards a holistic framework for transparency

Our study proposes a holistic framework that integrates techno-rational, socio-cultural, and socio-material approaches to transparency. This framework emphasizes the need for clear, accessible documentation, active engagement with stakeholders, and effective use of assessment tools and artifacts. By recognizing the diverse needs of all stakeholders, HEIs can develop more inclusive and effective assessment policies.

One of the key contributions of this study is its challenge to the notion of transparency as a static attribute. Instead, transparency is presented as a dynamic, contextually situated practice that requires continuous negotiation and interaction among stakeholders. This perspective shifts the focus from merely providing information to actively engaging stakeholders in the assessment process.

Figure 1. Framework of assessment transparency in Higher Education

Implications for policy and practice

To improve transparency in assessment, HEIs must move beyond merely publishing information to actively engaging with stakeholders through dialogue and interaction. Policies should be clear about the roles and responsibilities of different stakeholders in ensuring transparency. Furthermore, the use of diverse and interactive dissemination methods can enhance understanding and support students’ academic success.

For policymakers, the study suggests that transparency should be explicitly defined within institutional contexts, with guidelines that emphasize both the dissemination of information and the engagement of stakeholders. Educational practitioners are encouraged to adopt participatory practices in assessment design, involving students in creating and understanding assessment criteria, which is pivotal in promoting transparency.

Conclusion: enhancing transparency for a fairer education system

Transparency in assessment is a complex, multifaceted concept that goes beyond clear documentation. By integrating techno-rational, socio-cultural, and socio-material approaches, HEIs can foster a more inclusive and effective assessment environment. This study underscores the importance of comprehensive policies that not only provide clear information but also engage stakeholders in meaningful ways, ultimately contributing to a fairer and more equitable higher education system.

Reflecting on our roles as stakeholders

As readers, it is crucial to reflect on our roles within the higher education assessment ecosystem. Whether we are students, educators, policymakers, or external examiners, we each play a part in fostering transparency. Understanding the nuances of transparency and actively engaging in dialogue and interaction can help us contribute to more equitable and effective assessment practices. By recognizing and fulfilling our roles, we can collectively enhance the transparency and quality of education in our institutions.

Reference

Gonsalves, C and Lin, Z (2024) ‘Clear in advance to whom? Exploring ‘transparency’ of assessment practices in UK higher education institution assessment policy’ Studies in Higher Education, 1-17. https://doi.org/10.1080/03075079.2024.2381124

Chahna Gonsalves is a Senior Lecturer in Marketing (Education) at King’s College London. She is Senior Fellow of the Higher Education Association and Associate Fellow of the Staff Educational Development Association. Her interest in rubrics and the language of assessment is an extension of her role as Department Education Lead.

Zhonghan Lin is a Doctoral Researcher based at the Center for Language, Discourse and Communication, the School of Education, Communication and Society, King’s College London. Her research interests include urban multilingualism, education in ethnically and linguistically diverse societies, and family language policy.

Marcia Devlin


by 1 Comment

Let’s not waste any more time

by Marcia Devlin

Last year, I was sent a satirical article about how to sabotage the productivity of your organisation by using a CIA manual from 1944.  It contained advice such as:

  • When possible, refer all matters to committees, for ‘further study and consideration’. Attempt to make the committee as large as possible — never less than five;
  • Haggle over precise wordings of communications, minutes, resolutions; and
  • Refer back to matters decided upon at the last meeting and attempt to re-open the question of the advisability of that decision.

There’s more and it’s worth a read.

Reflecting on my time in higher education over three decades, the article was both funny and depressing. Funny, because it precisely describes daily life working in a university, which is peculiar and amusing and many of us would have it no other way. Depressing, because it precisely describes daily life working in a university, which is highly bureaucratic and inefficient and often a profound waste of talent, goodwill and time.

The COVID-19 crisis has provided many opportunities to rethink what we have always done and to do things differently. I’m wondering whether universities might eventually benefit from this terrible crisis, including in ways that could be permanent.

Following the scrambling, adjustments and re-learning required for the global mass uptake of online and digital forms of education, our attention has begun to turn to the implications of this move.

Always of interest, academic integrity and the quality and standards of learning are now the subject of increased interest and scrutiny. As the ubiquitous, supervised, closed book exam en masse became impossible, along with other forms of assessment that require physical supervision of students, less frequently used assessment approaches have been considered and deployed.

Academic integrity is having a day or two in the sun as educators in universities consider how to ensure it, when they can’t always see what students are doing, including during electronic classes. Approaches that are being considered and used include: more gentle and/or educative interpretations of existing assessment and academic integrity policies; the use of technological proctoring tools, including homemade solutions using student phones; and so-called ‘alternative’ assessment.

In Australia, assessment policies have been changed, or waived in part, including through the granting of special power to a senior officer of the University in some places. All of these shifts have occurred with the aim of enabling to practical solutions to challenging and sudden changes.

New and streamlined governance arrangements have been created and enacted to ensure appropriate oversight of teaching, learning and assessment changes in the very short timeframes possible at the time. This one might confound any current CIA operatives in universities who are intent on slowing us down.

Considerations of academic matters that used to take one or two long Academic Board discussions and a fair amount of angst, not to mention tension and in some cases ongoing resentments between parties with different views, gave way, at least momentarily.  They were replaced by shorter, more focused considerations, often in single, brief meetings of key people. As far as I observed at my own University and elsewhere, we have made sensible and defensible positions and enacted them, with broad acceptance and little or no negative ripple.

Of course, we were forced to move quickly by the sudden onset of the COVID-19 crisis, which is the defining feature of most of the world’s experience in 2020. As we enter 2021, a ‘new normal’ is emerging in every aspect of human existence. It is understandable that we should hope things to return to the ‘old normal’, at least in some respects. But I’d argue that academic governance is one area in which we should try to retain the new normal, at least to some extent.

Imagine what might be possible if university staff were freed up, even just a little, from the at times tortuous dance of academic consideration that unnecessarily uses up precious talent, goodwill and time.

What if, post this terrible world crisis, we emerged with a commitment to do things differently in universities and in ways that maintained integrity, but did not steal our precious resources?

What if, instead of us doing as a CIA operative would recommend, such as: “Talk as frequently as possible and at great length. Illustrate your ‘points’ by long anecdotes and accounts of personal experiences”, we simply didn’t do that anymore?

What if we all became more conscious of the time of others that we use up in the pursuit of ancient but no longer purposeful traditions? And what if we all committed to stopping doing this and trying something more respectful instead?

That we have moved many millions of university students to profoundly different ways of learning, teaching and assessment, and created new and efficient ways to consider and govern effectively with academic integrity apparently intact, tells us that anything is possible in the university sector.

When we finally emerge from COVID-19, the world will be a different place and human contact will be more deeply appreciated in many ways. Why don’t we try to respect that contact in our universities by not wasting each other’s talent, goodwill and time any further?

Marcia Devlin is a Fellow of SRHE and former Senior Deputy Vice-Chancellor, now Adjunct Professor at Victoria University in Melbourne, Australia. An earlier version of this article appeared in Campus Review.