SRHE Blog

The Society for Research into Higher Education

Tag Archives: Assessment policy


by Leave a comment

When assessment goes local, who keeps what students bring back?

by Carlene Kyeremeh

A student sent me a photograph of a handwritten waybill. Dated November 2022, it records a unit price for printed sachet film. It also carries the names of a supplier and a customer and the given name of the person who issued it. The waybill appeared in a term paper on fiscal consolidation as evidence of a cost increase that no statistical series I have found records.

It is in my files. Not the department’s. Mine.

I describe the waybill here but do not reproduce it, identify those named, or give the figure it records. The students who wrote the paper agreed to this account. I have no record that the supplier, the customer or the person named agreed to the document’s retention or wider use, so I have minimised the identifying detail, including the price, which is the most commercially sensitive thing on the page.

I have argued elsewhere that the curriculum can help to thicken a thin documentary record. Ask each student to obtain one item of locally sourced evidence: an interview, a quoted price, a document from a firm or public office, and, over successive cohorts, a course can assemble material that no database previously held. This is no longer only my ambition. Programmes are already redesigning assessment in response to generative AI, and one attractive move is to require evidence a model cannot retrieve: a price obtained in a market, an interview conducted locally, a document held by a firm or public office. That redesign brings third-party material into course files at scale, often without a parallel decision about what should happen to it after assessment.

That is the promise. The waybill in my files is the problem that follows it.

What universities already know how to govern

Research ethics and commercial confidentiality are not routinely ignored in student work. Professionally oriented programmes have long had to decide what an organisation may disclose, who may see it and when it must be returned. For example, McMaster University’s course-based research guidance explicitly covers undergraduate, MBA and graduate courses. The University of Toronto provides both a course-based ethics route and a classroom confidentiality agreement for third-party proprietary information. In the UK, the University of Manchester’s ethics guidance asks researchers to establish first whether an activity is research training conducted as coursework, described as collecting and analysing a small amount of data to gain experience in method, or research proper. These arrangements show that collection, consent, confidentiality and use within a defined project can be governed. My argument is therefore not that universities lack such protections. It begins where a normal project would end.

The problem begins when the course ends

If material is returned, destroyed or retained only with the assessed work, its lifecycle has a clear end. But the design I am proposing depends on the evidence outliving the assignment. A document collected by one student in 2026 should be available to another cohort in 2028 or 2029, alongside prices, interviews and documents gathered in the intervening years. The course is not only assessing students. It is building a local source collection.

That changes the questions. What exactly does the university hold after the assessment period? Who is its custodian? How long may it be retained? May another cohort consult it? What fresh permission is needed if it is later used for research or publication? These procedures can help answer those questions, but not automatically. An agreement that properly confines information to one course may rule out the very reuse on which the proposed archive depends.

The evidence is not the assignment

The student’s analysis is coursework. The material used to produce it is not necessarily the same thing. A waybill may contain personal or commercially sensitive information. Interview notes are harder still: they are neither the source’s own record nor the student’s analysis, but one person’s written account of what another person said, made for an assignment and now held by a lecturer.

Where the activity is classified as teaching rather than research, this third-party material can still raise a lifecycle question: do institutional rules cover its custody, retention and reuse after the assessment for which it was collected? The Manchester distinction is instructive here. A course collection built to outlive its cohorts is not a small amount of data gathered to practise a method, but neither does it fit neatly within research as that definition intends. It is a lifecycle problem created by a particular curricular ambition.

The reuse is the point

Ghana’s Data Protection Act, 2012 (Act 843) requires personal data to be collected for a specific, explicitly defined and lawful purpose, limits retention and requires further processing to be compatible with the original purpose. Material obtained for one student’s term paper and later made available to another cohort may involve further processing. Nothing in the waybill I received establishes permission for it to become part of a standing course collection.

The Act contains provisions for historical, statistical and research uses where the relevant conditions are met. Those provisions offer a governed route, not a retrospective blank cheque. They do not turn a lecturer’s folder into an institutional archive. The underlying principle is not unique to Ghana: UK guidance likewise treats purpose, storage and research-related reuse as connected but distinct questions.

Ordinary assignment governance is adequate for ordinary assignments. The problem appears when coursework is asked to do something additional: to enlarge a locally available documentary record over time.

The archival value and the governance risk arise from the same feature. An archive that cannot be consulted next year is not an archive. A collection that can be consulted next year without the source having been told is not a defensible one.

A small institutional answer

The answer need not be a new bureaucracy. It could begin with a short institutional instrument that separates use in a student’s assessment from retention in a course source collection. Before collection begins, it should state the teaching purpose, what may be collected, who may later consult it, how long it will be kept, which unit is responsible and what choices the source has. Research use or publication should require a separate decision rather than being hidden inside the phrase ‘educational purposes’.

The remaining requirement is custody. What one lecturer can require, only an institution can keep. A named custodian, controlled access, a retention and disposal schedule, and a record of what the course has produced would convert a personal folder into an accountable institutional practice. Existing MBA, ethics and confidentiality processes provide models; the task is to extend their discipline to the intended afterlife of the evidence.

Proportionality matters. If the process becomes heavier than the assessment it supports, colleagues may stop asking students to gather local evidence. But a department unable to provide a clear collection notice, a retention rule and somebody accountable for what returns should not require the fieldwork. Otherwise, institutional risk is transferred to students and to the businesses that agree to speak with them.

Readers can test this in ten minutes. Take your own institution’s course-based ethics route or classroom confidentiality agreement and look for two things: whether either says anything about third-party material after the assessment period, and who is named as responsible for it. If neither appears, ask where else the institution governs the evidence after the assignment ends. If there is no answer, the arrangements may govern the assignment without governing its evidentiary afterlife.

Before we ask for more

The curriculum can enlarge the archive. The qualification is that the archival ambition creates an additional governance obligation. The question is not only whether the evidence was collected ethically and kept confidentially. It is whether its intended custody, retention and reuse were made explicit before collection began.

I have a waybill in my files and no good institutional answer to those questions. That is not an argument against asking students to go and find out. It is an argument for deciding what we will do with what comes back, before we ask them to go.

Dr Carlene Kyeremeh is Associate Professor and Vice President, University Advancement, Recruitment and Research at All Nations University, Koforidua, Ghana. Her research addresses curriculum decolonisation, higher education policy in Africa and the Caribbean, and the political economy of educational technology.This post develops a governance question raised in Issues 14 and 15 of The Decolonized Curriculum, her LinkedIn newsletter.


by 3 Comments

The hidden layers of transparency in UK HE assessment practices

by Chahna Gonsalves and Zhongan Lin

Transparency in assessment practices is a critical component of the UK’s higher education sector, but it is a term that carries many layers of meaning. This blog post explores a study that examined how transparency is framed in assessment policies across 151 UK higher education institutions (HEIs). The findings reveal that while institutions strive for transparency, they often overlook the complexities and multidimensional nature of the concept.

Understanding transparency: more than just clear documentation

Transparency in assessment is often associated with clear documentation of criteria, grading practices, and feedback mechanisms. However, this techno-rational approach, which emphasizes explicit documentation and information dissemination, is just one facet of transparency. Our study highlights the need for a more nuanced understanding that includes socio-cultural practices and socio-material enactments.

Techno-rational approaches: the dominant paradigm

The study found that techno-rational approaches dominate the transparency discourse in HEI policies. These approaches focus on ensuring that assessment criteria, learning outcomes, and grading standards are clearly articulated and accessible. For example, many policies mandate the use of detailed assessment briefs, rubrics, and grade descriptors. While this approach aims to make evaluative processes clear and consistent, it often falls short in addressing the dynamic and interpretive nature of academic standards.

One of the most compelling findings was the over-reliance on explicit standards documents, which presume that written criteria can universally ensure fairness and consistency. This static view overlooks the reality that academic standards are co-constructed within specific social and cultural contexts. Without acknowledging this, policies may fail to convey the nuanced, tacit knowledge necessary for fully understanding and applying assessment criteria.

The limitations of techno-rational transparency

Simply providing clear documentation does not guarantee that all stakeholders will understand or effectively use the information. For instance, non-native English speakers and students with varying levels of academic literacy may struggle with the language used in assessment criteria. Moreover, policies often fail to specify effective methods for disseminating this information, relying heavily on static documents rather than interactive or diverse formats that could enhance understanding.

Socio-cultural practices: engaging stakeholders in meaningful dialogue

Beyond documentation, transparency also involves socio-cultural practices that engage stakeholders in ongoing dialogue and clarification of assessment criteria. Policies that promote discussion between educators and students, co-creation of assessment criteria, and collaborative marking processes can foster a deeper understanding and shared meaning of what is expected. For instance, involving students in the creation of rubrics and providing opportunities for mock marking can enhance their evaluative judgment and assessment literacy.

One interesting insight from the study was the importance of dialogue in building a shared understanding of assessment standards. Policies that encourage discussion about assessment criteria not only help students grasp what is expected but also allow educators to refine and clarify their expectations. This dynamic, interactive process contrasts sharply with the static dissemination of information typical of techno-rational approaches.

Socio-material enactments: the role of tools and artefacts

The study also highlights the importance of socio-material enactments, where transparency is realized through the interaction between social practices and material artifacts. This includes the use of digital platforms, rubrics, exemplars, and other assessment tools that facilitate a tangible understanding of assessment criteria. Effective use of these tools can bridge the gap between educators’ tacit knowledge and students’ understanding, fostering a more comprehensive view of transparency.

For example, the use of digital platforms to share assessment criteria and feedback can significantly enhance transparency. These platforms allow for continuous access and interaction with assessment materials, making it easier for students to understand and engage with the criteria. However, the study found that detailed guidance on such platforms is often scant in policies, pointing to a significant area for improvement.

Who benefits from transparency? A multifaceted audience

Transparency in assessment is not solely for students. It also encompasses other stakeholders, including markers, external examiners, tutors, and even employers. The study found that while most policies address the need for transparency for students and markers, they often neglect other crucial stakeholders. This oversight can lead to inconsistencies in how assessments are interpreted and applied, potentially undermining the fairness and effectiveness of the evaluation process.

A particularly intriguing aspect of the study was the identification of specific roles and responsibilities for promoting transparency. By clearly defining who is responsible for ensuring transparency – whether it be module leaders, programme teams, or tutors – institutions can better align their policies with the needs of various stakeholders. This clarity can help avoid the pitfalls of ambiguous roles and ensure a more consistent application of assessment criteria.

Methodology: building the framework

To develop a comprehensive framework we conducted a detailed content analysis of assessment policy documents from 151 UK HEIs. The data collection process involved systematically retrieving and examining these publicly accessible documents, which included academic manuals, assessment policies, feedback strategies, and codes of practice. We excluded documents that were outdated or inaccessible, resulting in a final corpus of 264 documents. Through both deductive and inductive coding methods, we analysed the texts to identify recurring themes and patterns related to transparency. This process involved categorising the data into three main discourses – techno-rational, socio-cultural, and socio-material – guided by Ajjawi, Bearman, and Boud’s (2021) framework. The iterative coding and categorization helped us build a nuanced understanding of how transparency is conceptualized and communicated in HEI assessment policies.

Towards a holistic framework for transparency

Our study proposes a holistic framework that integrates techno-rational, socio-cultural, and socio-material approaches to transparency. This framework emphasizes the need for clear, accessible documentation, active engagement with stakeholders, and effective use of assessment tools and artifacts. By recognizing the diverse needs of all stakeholders, HEIs can develop more inclusive and effective assessment policies.

One of the key contributions of this study is its challenge to the notion of transparency as a static attribute. Instead, transparency is presented as a dynamic, contextually situated practice that requires continuous negotiation and interaction among stakeholders. This perspective shifts the focus from merely providing information to actively engaging stakeholders in the assessment process.

Figure 1. Framework of assessment transparency in Higher Education

Implications for policy and practice

To improve transparency in assessment, HEIs must move beyond merely publishing information to actively engaging with stakeholders through dialogue and interaction. Policies should be clear about the roles and responsibilities of different stakeholders in ensuring transparency. Furthermore, the use of diverse and interactive dissemination methods can enhance understanding and support students’ academic success.

For policymakers, the study suggests that transparency should be explicitly defined within institutional contexts, with guidelines that emphasize both the dissemination of information and the engagement of stakeholders. Educational practitioners are encouraged to adopt participatory practices in assessment design, involving students in creating and understanding assessment criteria, which is pivotal in promoting transparency.

Conclusion: enhancing transparency for a fairer education system

Transparency in assessment is a complex, multifaceted concept that goes beyond clear documentation. By integrating techno-rational, socio-cultural, and socio-material approaches, HEIs can foster a more inclusive and effective assessment environment. This study underscores the importance of comprehensive policies that not only provide clear information but also engage stakeholders in meaningful ways, ultimately contributing to a fairer and more equitable higher education system.

Reflecting on our roles as stakeholders

As readers, it is crucial to reflect on our roles within the higher education assessment ecosystem. Whether we are students, educators, policymakers, or external examiners, we each play a part in fostering transparency. Understanding the nuances of transparency and actively engaging in dialogue and interaction can help us contribute to more equitable and effective assessment practices. By recognizing and fulfilling our roles, we can collectively enhance the transparency and quality of education in our institutions.

Reference

Gonsalves, C and Lin, Z (2024) ‘Clear in advance to whom? Exploring ‘transparency’ of assessment practices in UK higher education institution assessment policy’ Studies in Higher Education, 1-17. https://doi.org/10.1080/03075079.2024.2381124

Chahna Gonsalves is a Senior Lecturer in Marketing (Education) at King’s College London. She is Senior Fellow of the Higher Education Association and Associate Fellow of the Staff Educational Development Association. Her interest in rubrics and the language of assessment is an extension of her role as Department Education Lead.

Zhonghan Lin is a Doctoral Researcher based at the Center for Language, Discourse and Communication, the School of Education, Communication and Society, King’s College London. Her research interests include urban multilingualism, education in ethnically and linguistically diverse societies, and family language policy.

Marcia Devlin


by 1 Comment

Let’s not waste any more time

by Marcia Devlin

Last year, I was sent a satirical article about how to sabotage the productivity of your organisation by using a CIA manual from 1944.  It contained advice such as:

  • When possible, refer all matters to committees, for ‘further study and consideration’. Attempt to make the committee as large as possible — never less than five;
  • Haggle over precise wordings of communications, minutes, resolutions; and
  • Refer back to matters decided upon at the last meeting and attempt to re-open the question of the advisability of that decision.

There’s more and it’s worth a read.

Reflecting on my time in higher education over three decades, the article was both funny and depressing. Funny, because it precisely describes daily life working in a university, which is peculiar and amusing and many of us would have it no other way. Depressing, because it precisely describes daily life working in a university, which is highly bureaucratic and inefficient and often a profound waste of talent, goodwill and time.

The COVID-19 crisis has provided many opportunities to rethink what we have always done and to do things differently. I’m wondering whether universities might eventually benefit from this terrible crisis, including in ways that could be permanent.

Following the scrambling, adjustments and re-learning required for the global mass uptake of online and digital forms of education, our attention has begun to turn to the implications of this move.

Always of interest, academic integrity and the quality and standards of learning are now the subject of increased interest and scrutiny. As the ubiquitous, supervised, closed book exam en masse became impossible, along with other forms of assessment that require physical supervision of students, less frequently used assessment approaches have been considered and deployed.

Academic integrity is having a day or two in the sun as educators in universities consider how to ensure it, when they can’t always see what students are doing, including during electronic classes. Approaches that are being considered and used include: more gentle and/or educative interpretations of existing assessment and academic integrity policies; the use of technological proctoring tools, including homemade solutions using student phones; and so-called ‘alternative’ assessment.

In Australia, assessment policies have been changed, or waived in part, including through the granting of special power to a senior officer of the University in some places. All of these shifts have occurred with the aim of enabling to practical solutions to challenging and sudden changes.

New and streamlined governance arrangements have been created and enacted to ensure appropriate oversight of teaching, learning and assessment changes in the very short timeframes possible at the time. This one might confound any current CIA operatives in universities who are intent on slowing us down.

Considerations of academic matters that used to take one or two long Academic Board discussions and a fair amount of angst, not to mention tension and in some cases ongoing resentments between parties with different views, gave way, at least momentarily.  They were replaced by shorter, more focused considerations, often in single, brief meetings of key people. As far as I observed at my own University and elsewhere, we have made sensible and defensible positions and enacted them, with broad acceptance and little or no negative ripple.

Of course, we were forced to move quickly by the sudden onset of the COVID-19 crisis, which is the defining feature of most of the world’s experience in 2020. As we enter 2021, a ‘new normal’ is emerging in every aspect of human existence. It is understandable that we should hope things to return to the ‘old normal’, at least in some respects. But I’d argue that academic governance is one area in which we should try to retain the new normal, at least to some extent.

Imagine what might be possible if university staff were freed up, even just a little, from the at times tortuous dance of academic consideration that unnecessarily uses up precious talent, goodwill and time.

What if, post this terrible world crisis, we emerged with a commitment to do things differently in universities and in ways that maintained integrity, but did not steal our precious resources?

What if, instead of us doing as a CIA operative would recommend, such as: “Talk as frequently as possible and at great length. Illustrate your ‘points’ by long anecdotes and accounts of personal experiences”, we simply didn’t do that anymore?

What if we all became more conscious of the time of others that we use up in the pursuit of ancient but no longer purposeful traditions? And what if we all committed to stopping doing this and trying something more respectful instead?

That we have moved many millions of university students to profoundly different ways of learning, teaching and assessment, and created new and efficient ways to consider and govern effectively with academic integrity apparently intact, tells us that anything is possible in the university sector.

When we finally emerge from COVID-19, the world will be a different place and human contact will be more deeply appreciated in many ways. Why don’t we try to respect that contact in our universities by not wasting each other’s talent, goodwill and time any further?

Marcia Devlin is a Fellow of SRHE and former Senior Deputy Vice-Chancellor, now Adjunct Professor at Victoria University in Melbourne, Australia. An earlier version of this article appeared in Campus Review.