SRHE Blog

The Society for Research into Higher Education


Leave a comment

Tell it slant: what universities miss in the joins

by Joanne Irving-Walton

Universities are increasingly good at recording fragments of student experience. But fragments do not automatically make a whole. The work of connecting them remains stubbornly human.

“Tell all the truth but tell it slant —” Emily Dickinson advises.

It is an odd instruction for universities increasingly committed to straight lines: lines of accountability, lines between strategy and delivery, pathways through programmes, and data intended to show, cleanly and conclusively, what is happening. Yet neither learning nor universities behave quite like this. Students move between modules, systems, academic staff, professional services, regulations, placements and increasingly distributed forms of provision. Each part of the institution sees something. None necessarily sees the whole.

At the same time, we are becoming increasingly accomplished at recording these fragments. Attendance is captured. VLE activity can be traced. Marks, referrals, extensions and interventions leave institutional records. Processes generate evidence that something has happened. Reliable systems, fair regulations, consistent decisions and clear routes to support all matter. The problem begins when increasingly detailed representations of student experience are mistaken for increasingly complete understanding of it. A university can know a great deal about what a student has done while understanding remarkably little about what is happening.

Students experience the joins

Students rarely experience universities as organisational charts. They encounter them at the joins: between module and course, academic and pastoral support, partner and university, attendance and wellbeing, policy and circumstance, one team’s responsibility and another’s. This is where seemingly small difficulties can accumulate.

A student is sent the correct information but does not understand what they need to do next. A concern is referred, but nobody knows whether they reached the service. Marks are recorded module by module, while the pattern across them goes unnoticed. One part of the university knows about caring responsibilities; another sees only absence. Nothing necessarily fails. The failure sits in the joins — and those are rarely what institutions measure.

Each part of the institution may have done what it was required to do. The record contains multiple fragments of truth, carefully documented and separately owned. The student lives them all at once. That mismatch will be familiar to anyone who has ever sat with a student while several perfectly reasonable institutional answers fail to add up. But for those fragments to become a coherent experience, someone has to connect them.

Course leaders, personal tutors, module teams, administrators and support staff notice when several small things form a pattern. They translate formal language, recognise when an apparently straightforward case is not quite straightforward, contact another team rather than simply redirecting the student, remember context that is not visible in the current system, and follow up after the formal handover.

This kind of work has a substantial intellectual lineage. Strauss described the ‘articulation work’ required to coordinate complex activity, while Star and Strauss (1999) examined how work essential to keeping systems functioning can remain invisible to formal accounts of what work is. More recent higher education research has similarly drawn attention to boundary-crossing work and the expertise required to operate across institutional divisions (Vere, Verney and Webster-Deakin, 2024).

Pugh (2024) uses the term connective labour for the relational work through which people recognise, read and respond to one another in professions such as teaching and care. The work I am interested in here operates at a different level. It is the interpretive and relational work of connecting the fragments of an institution so that students experience something more coherent than a series of separate systems, teams and processes. I call this institutional connective labour: recognising which elements belong together, what context matters, where apparently routine cases require another look, and when completing a process has not resolved the problem that prompted it.

The work that disappears when it succeeds

Institutional connective labour is remarkably difficult for organisations to see, particularly in the large and distributed environments of higher education. Successful repair tends to leave little trace. An experienced member of staff spots a problem before it escalates, finds the colleague who knows how to resolve it, explains an opaque process, reconciles contradictory information or prevents an administrative delay from affecting progression. Eventually, the student continues. On paper, very little happened. Indeed, the successful outcome can make the system itself appear to have worked. This creates an uncomfortable organisational paradox. The people who are best at compensating for fragmented systems may inadvertently make that fragmentation less visible. Their judgement, memory, relationships and persistence absorb the gaps.

Workload models record formal roles more readily than the accumulation of noticing, translating, connecting and following through around them. Restructures may identify apparent duplication without asking whether overlapping roles are providing connection or resilience. Processes can be streamlined without recognising the informal knowledge through which they were previously made workable.

The visible machinery remains. The less visible capacity that made it function is weakened. As universities become more distributed across campuses, partners, professional services, digital environments and different forms of provision, fragmentation is not necessarily evidence of poor organisation. Complex institutions will always distribute responsibility. But distribution creates a corresponding need for integration. We tend to design carefully for the former and assume the latter will happen.

Greater resolution is not greater understanding

Data and process offer partial institutional views. An attendance pattern can signal a problem without explaining it. A referral records movement through a system without showing whether useful support was reached. A completed process can demonstrate compliance while the original difficulty remains unresolved.

Learning analytics scholars have long warned against treating data as neutral representations of student experience. Signals require interpretation, context and attention to student agency (Slade and Prinsloo, 2013). The same problem extends beyond analytics. Rules, records and processes are representations of reality, not reality itself. Their application still requires decisions about relevance, context and meaning. Informal systems can reproduce inequality, make support dependent on encountering the right person and leave both students and staff unprotected. But neither can the need for context and interpretation simply be designed out.

The difficulty becomes clearest when several individually reasonable accounts do not quite align. Low attendance may indicate disengagement, but it may mean something else entirely. A missed deadline may be habitual delay or the point at which an otherwise successful student can no longer absorb a series of pressures. A student may appear to have been referred successfully while moving repeatedly between services without ever finding somewhere their problem fits. Greater resolution does not necessarily produce greater understanding. Sometimes it simply gives us a more detailed picture of the fragments. The difficulty is not that any one account is false. It is that each tells the truth from only one angle.

Seeing slant

Dickinson’s ‘slant’ offers a useful way of thinking about this. Seeing slant is not an argument for indecision, endless consultation or gathering every possible perspective before acting. It is the recognition that complex experience rarely arrives from one direction. For universities, two questions can help expose the joins:

  • What alternative stories could produce the pattern we can see?
  • Where are staff repeatedly having to translate, reconnect or repair what a formal process has separated?

These are questions about judgement rather than additional process. The aim is not to collect every possible view before acting. Universities must simplify, classify and make decisions. But credible judgement requires some awareness of what has been simplified in doing so. The more useful institutional question may therefore be not simply whether a process has been completed, but what had to happen around that process to make it work.

The institution between the boxes

The point is not simply that this work should be recognised. It should tell us something. If staff repeatedly have to reconnect the same processes, translate the same information or recover context lost at the same handover, those interventions are evidence. They show where the formal account of how the institution works diverges from how it works in practice.

That makes institutional connective labour potentially diagnostic. A workaround may solve an immediate problem, but repeated workarounds reveal something more useful: where information stops travelling, where responsibilities do not quite meet, or where a process produces completion without resolution. That does not mean turning connective labour into another metric. Formalising it too heavily would risk flattening the very context and interpretation that make it useful. The point is to create ways for recurring patterns noticed through this work to inform institutional learning without reducing them to another dashboard. Nor should institutions depend on a few experienced individuals to hold the system together. The challenge is to make the conditions for noticing, connecting and escalating less accidental — and the knowledge generated through that work less private.

A practical starting point is to ask, in course reviews, service reviews or handover discussions: where are staff repeatedly having to reconnect what the formal process separates? Then take one recently ‘resolved’ case and ask a simpler question: what had to happen outside the formal process for this to reach a workable outcome? If the same kinds of repair appear repeatedly, that is not merely evidence of helpful staff. It is evidence about where the institution itself needs attention. For the student, the difference between a fragmented institution and a coherent one may simply be whether somebody noticed the joins.

Joanne Irving-Walton is a Principal Lecturer at Teesside University, working across learning and teaching and international partnerships. She is particularly interested in how academic judgement and professional identity develop through the emotional realities of higher education.


Leave a comment

What is an academic judgement?

By Geoff Hinchliffe

Academics make academic judgements virtually every working day. But what exactly is an academic judgement? As a starting point, one might have recourse to appropriate statutory documents: for example, the 2004 Higher Education Act mentions that a student complaint does not count as a ‘qualifying’ complaint if it relates to matters pertaining to an ‘academic judgment’ (Higher Education Act, 2004, p5, Section 12). The Office of the Independent Adjudicator (OIA) helps to provide a gloss on the term:

“Academic judgment is not any judgment made by an academic; it is a judgment that is made about a matter where the opinion of an academic expert is essential. So for example a judgment about marks awarded, degree classification, research methodology, whether feedback is correct or adequate, and the content or outcomes of a course will normally involve academic judgment.” (OIA, 2018, Section 30.2)

But although it is heartening to see that some deference is paid to academic judgment, little light is thrown on what it actually is. This can, of course, be useful: for example, Cambridge University’s (2018) complaint procedure quotes the OIA definition without further elaboration. Providing no-one is prepared to question the nature of academic judgement, who are we to complain? But, at the risk of disturbing sleeping dogs, I propose to enquire more closely as to what constitutes an academic judgement.

Two points are worth making at the outset. The first is that academic judgements should not be construed as the special preserve of those designated as ‘academics’. Students also make academic judgements along the same lines as academics, so it’s not the case that academics make special judgements that students couldn’t possibly understand. The second point is that the object of judgements – what is being judged – may vary considerably, but the kind of judgement being made is still of the same type. The elements of judgement remain the same whether the object of scrutiny is a first year undergraduate essay or a paper in a leading journal.

It seems to me that there are four basic types of academic judgement and they frequently operate in combination – it is this that gives the whole business of judgement its mystique and rarefied quality.

1. Process judgements

When we evaluate any kind of process that has (or is supposed to have) a definable outcome we tend to use a set of criteria, although the latter may be carefully delineated or may operate as background guidelines. Examples are the following of some clinical procedure (where the criteria are strict) or following laboratory protocols (ditto). But they could also include the evaluation of the methodology in a piece of empirical research, in which we consider the suitability of the methodology used and its success or otherwise. Process judgements are also entailed in the evaluation of essays in terms of structure and argumentation leading up (hopefully) to a conclusion. In this case, we evaluate the process of writing and structuring an essay or report; we consider, for example,  whether the author has ‘signposted’ the argument so that its trajectory has some kind of sense and cohesion. Process judgements tend to be ‘rule-governed’. But as I have indicated, in some cases the rules are pretty clear and in other cases there is much room for flexibility. So, regarding essays, there are no fixed rules for demonstrating a process of argumentation; but neither are there no rules at all. In the film Pirates of the Caribbean it is explained that the Pirate’s Code is not to be adhered to strictly at all times because it is not so much a fixed code as ‘guidelines’. Very sound advice too.

For those interested in philosophy, process judgements are roughly akin to what Wittgenstein thinks of as ‘following a rule’ (see Wittgenstein, 1958, paras 201-2). In this case, what Wittgenstein has in mind are the rules for using words so that they have a meaning that is understood. But since meanings are never fixed (unless through prior stipulation) then they are indeed ‘guidelines’. Who would have thought a pirate would be reading Wittgenstein?

2. Epistemic Judgements

In the case of epistemic judgements we are assessing claims to knowledge. That is, we are assessing whether the claimant has sufficient evidence and reasons for making a knowledge claim. Of course, we are also interested in the context – that is whether and to what extent the claimant is aware of relevant context that may affect the claims they are making. Furthermore, we are often reluctant to make positive judgements if the claim simply asserts a proposition – to the effect that such-and such is the case – even if the proposition is true. We need to see the evidence and reasoning that back up the knowledge claim – bare assertions are usually not enough.

There are two further features of epistemic judgements: first they are objective, in the sense of being propositional – they purport to say ‘how the world is’. Second, they are universal in the sense that in making such a judgement I am claiming that everyone will reach the same conclusion as myself. Of course, others may disagree but the idea is that, in principle, these disagreements are adjudicable (Steinberger, 2018, p38).

Notice that if I am marking a student paper then I am assessing the kinds of epistemic judgements the student is making – whether the claims made are true and whether they are well founded. It’s not the case that the student is doing one thing and I am doing something else – the writer of the paper and the assessor are both making the same kind of judgements. That is, the student needs to be in the habit of assessing their own epistemic judgements as to evidence and reasoning in exactly the same way that I, the assessor, am doing. The process is the same: the only difference is that in the one case the outcome is a paper and in the other, the outcome is a mark.

3. Reflective Judgements

This kind of judgement is tricky to explain but I think readers will see its importance. By ‘reflective’ I don’t mean the reflective judgement beloved of writers of practitioner manuals where ‘reflective’ means ‘self-reflective’. Thus interpreted, reflection is usually reflection on a procedure and one’s part in it. In other words, practitioner self-reflective judgements are really a kind of process judgement.

What I have in mind as ‘reflective’ is when we think of a piece of data, a theory, or a concept in functional or relational terms. We look for a broader framework within which phenomena can be better understood. Thus, Kant thought that when we reflect on a natural phenomenon we situate nature in a purposive or teleological framework, in order to provide a kind of interpretive unity (Kant, 2000, p67).  More generally, we can think of reflective judgements as contextual: we look for links and relationships in order to make sense of the object of study, to bring some sense of order and unity to bear. Reflective judgements can be highly creative when links are made between phenomena that weren’t thought of before. Quite a lot of Foucault’s work was of this type – for example, the way in which he related different kinds of formative behaviours into the notion of the disciplinary: in seeing that behaviours in schools, prisons, hospitals and the like were produced and reproduced he was able to fashion a new concept – the ‘disciplinary’ – which gives us real insight into how modernity works.

Peter Steinberger (2018, pp47-50) explains the nature of reflective judgements rather well, in my view – he sees reflective judgements as different from what I have called epistemic judgements (following Kant, he calls the latter ‘determinate’ judgements). What a reflective judgement does is to provide an interpretive context in which different knowledge claims can be related and thus better understood. Reflective judgements operate at the level of meaning. When we ask our students to make the links both within and across modules we are asking them to think reflectively.

4. Normative Judgements

We need to be clear that normative judgements are not the same as ethical judgements. I see the latter as delivering a verdict on the worth or rightness of a person or action. As such, ethical judgements play (or should play) a minor role in academic research and production. It is irritating if a historian gives us ethical verdicts on her subject matter (Henry was a good king, but King John was a bad one) – ethical judgements are almost always unnecessary.

But normative judgements are something else. They involve according due sensitivity to the values and norms associated with the subject matter under consideration. By their very nature, normative judgements are contextual. For example, if a student fails to appreciate the nature of religiosity in fifteenth century England (for example, by seeing it in terms of ignorance and superstition because of a modernist, secular approach which the student brings to bear) then it is the normative judgement that has gone awry. Or, if a research methodology fails to take due account of the needs of confidentiality, again, it is a normative judgement that is deficient. Normative judgements often operate in combination with other judgements (especially with process and reflective judgements) and this is one of the reasons why academic judgement can be complex.

Conclusion

If I am right then there are four basic elements to an academic judgement. Typically in any assessment all four elements are operating together – process, epistemic, reflective and normative. The judgements we use are precisely those that we want our students to develop. We can see straight away that attempts to categorise and tabulate all of these elements may be helpful but are unlikely to be comprehensive. The precise nature of the judgement will vary according to subject matter and no set of assessment criteria that I have seen comes anywhere near to giving full justice to the complexity of the judgements involved. Moreover, most of those outside academia (government ministers, MPs, media people and the like) are just clueless regarding how much they know of this complexity.

Complex, yes: but not so complex that we can’t attempt to say what is involved in giving an academic judgement. But the above sketch cannot be the last word – if I have succeeded in suggesting some initial ‘guidelines’ then that is a start.

SRHE member Geoff Hinchliffe teaches undergraduates in the School of Education at the University of East Anglia. This blog is partly based on a paper he gave at the 2018 SRHE Research Conference.

Bibliography

Cambridge University (2018) Student Complaints https://www.studentcomplaints.admin.cam.ac.uk/general-points-about-procedures/academic-judgment

Higher Education Act (2004) http://www.legislation.gov.uk/ukpga/2004/8/pdfs/ukpga_20040008_en.pdf

Kant, I (1933) Critique of Pure Reason, trans. Kemp-Smith, N, London: Macmillan

Kant, I (2000) Critique of the Power of Judgement, trans. Guyer, P and Matthews, E, Cambridge: Cambridge University Press

OIA (2018) Guidance Note on the OIA Rules http://www.oiahe.org.uk/rules-and-the-complaints-process/guidance-note-on-the-oias-rules.aspx#para30

Steinberger, P.J (2018), Political Judgement, Cambridge: Polity Press

Wittgenstein, L (1958) Philosophical Investigations, Oxford: Blackwell