By Geoff Hinchliffe
Academics make academic judgements virtually every working day. But what exactly is an academic judgement? As a starting point, one might have recourse to appropriate statutory documents: for example, the 2004 Higher Education Act mentions that a student complaint does not count as a ‘qualifying’ complaint if it relates to matters pertaining to an ‘academic judgment’ (Higher Education Act, 2004, p5, Section 12). The Office of the Independent Adjudicator (OIA) helps to provide a gloss on the term:
“Academic
judgment is not any judgment made by an academic; it is a judgment that is made
about a matter where the opinion of an academic expert is essential. So for
example a judgment about marks awarded, degree classification, research
methodology, whether feedback is correct or adequate, and the content or
outcomes of a course will normally involve academic judgment.” (OIA, 2018,
Section 30.2)
But although it is heartening to see that some
deference is paid to academic judgment, little light is thrown on what it
actually is. This can, of course, be useful: for example, Cambridge
University’s (2018) complaint procedure quotes the OIA definition without
further elaboration. Providing no-one is prepared to question the nature of
academic judgement, who are we to complain? But, at the risk of disturbing
sleeping dogs, I propose to enquire more closely as to what constitutes an
academic judgement.
Two points are worth making at the outset. The
first is that academic judgements should not be construed as the special
preserve of those designated as ‘academics’. Students also make academic
judgements along the same lines as academics, so it’s not the case that
academics make special judgements that students couldn’t possibly understand.
The second point is that the object of judgements – what is being judged – may
vary considerably, but the kind of judgement being made is still of the same
type. The elements of judgement remain the same whether the object of scrutiny
is a first year undergraduate essay or a paper in a leading journal.
It seems to me that there are four basic types of
academic judgement and they frequently operate in combination – it is this that
gives the whole business of judgement its mystique and rarefied quality.
1. Process judgements
When we evaluate any kind of process that has (or
is supposed to have) a definable outcome we tend to use a set of criteria,
although the latter may be carefully delineated or may operate as background
guidelines. Examples are the following of some clinical procedure (where the
criteria are strict) or following laboratory protocols (ditto). But they could
also include the evaluation of the methodology in a piece of empirical research,
in which we consider the suitability of the methodology used and its success or
otherwise. Process judgements are also entailed in the evaluation of essays in
terms of structure and argumentation leading up (hopefully) to a conclusion. In
this case, we evaluate the process of writing and structuring an essay or
report; we consider, for example, whether
the author has ‘signposted’ the argument so that its trajectory has some kind
of sense and cohesion. Process judgements tend to be ‘rule-governed’. But as I
have indicated, in some cases the rules are pretty clear and in other cases
there is much room for flexibility. So, regarding essays, there are no fixed
rules for demonstrating a process of argumentation; but neither are there no
rules at all. In the film Pirates of the
Caribbean it is explained that the Pirate’s Code is not to be adhered to
strictly at all times because it is not so much a fixed code as
‘guidelines’. Very sound advice too.
For those interested in philosophy, process
judgements are roughly akin to what Wittgenstein thinks of as ‘following a
rule’ (see Wittgenstein, 1958, paras 201-2). In this case, what Wittgenstein
has in mind are the rules for using words so that they have a meaning that is understood.
But since meanings are never fixed (unless through prior stipulation) then they
are indeed ‘guidelines’. Who would have thought a pirate would be reading
Wittgenstein?
2. Epistemic Judgements
In the case of epistemic judgements we are
assessing claims to knowledge. That is, we are assessing whether the claimant
has sufficient evidence and reasons for making a knowledge claim. Of course, we
are also interested in the context – that is whether and to what extent the
claimant is aware of relevant context that may affect the claims they are
making. Furthermore, we are often reluctant to make positive judgements if the claim
simply asserts a proposition – to the effect that such-and such is the case –
even if the proposition is true. We need to see the evidence and reasoning that
back up the knowledge claim – bare assertions are usually not enough.
There are two further features of epistemic
judgements: first they are objective,
in the sense of being propositional – they purport to say ‘how the world is’.
Second, they are universal in the sense that in making such a judgement I am
claiming that everyone will reach the same conclusion as myself. Of course,
others may disagree but the idea is that, in principle, these disagreements are
adjudicable (Steinberger, 2018, p38).
Notice that if I am marking a student paper then I
am assessing the kinds of epistemic judgements the student is making – whether
the claims made are true and whether they are well founded. It’s not the case
that the student is doing one thing and I am doing something else – the writer
of the paper and the assessor are both making the same kind of judgements. That
is, the student needs to be in the habit of assessing their own epistemic
judgements as to evidence and reasoning in exactly the same way that I, the
assessor, am doing. The process is the same: the only difference is that in the
one case the outcome is a paper and in the other, the outcome is a mark.
3. Reflective Judgements
This kind of judgement is tricky to explain but I
think readers will see its importance. By ‘reflective’ I don’t mean the
reflective judgement beloved of writers of practitioner manuals where
‘reflective’ means ‘self-reflective’. Thus interpreted, reflection is usually reflection
on a procedure and one’s part in it. In other words, practitioner self-reflective
judgements are really a kind of process judgement.
What I have in mind as ‘reflective’ is when we
think of a piece of data, a theory, or a concept in functional or relational
terms. We look for a broader framework within which phenomena can be better
understood. Thus, Kant thought that when we reflect on a natural phenomenon we
situate nature in a purposive or teleological framework, in order to provide a
kind of interpretive unity (Kant, 2000, p67). More generally, we can think of reflective
judgements as contextual: we look for links and relationships in order to make
sense of the object of study, to bring some sense of order and unity to bear.
Reflective judgements can be highly creative when links are made between phenomena
that weren’t thought of before. Quite a lot of Foucault’s work was of this type
– for example, the way in which he related different kinds of formative
behaviours into the notion of the disciplinary: in seeing that behaviours in
schools, prisons, hospitals and the like were produced and reproduced he was
able to fashion a new concept – the ‘disciplinary’ – which gives us real
insight into how modernity works.
Peter Steinberger (2018, pp47-50) explains the
nature of reflective judgements rather well, in my view – he sees reflective
judgements as different from what I have called epistemic judgements (following
Kant, he calls the latter ‘determinate’ judgements). What a reflective judgement
does is to provide an interpretive context in which different knowledge claims
can be related and thus better understood. Reflective judgements operate at the
level of meaning. When we ask our
students to make the links both within and across modules we are asking them to
think reflectively.
4. Normative Judgements
We need to be clear that normative judgements are
not the same as ethical judgements. I see the latter as delivering a verdict on
the worth or rightness of a person or action. As such, ethical judgements play
(or should play) a minor role in academic research and production. It is
irritating if a historian gives us ethical verdicts on her subject matter
(Henry was a good king, but King John was a bad one) – ethical judgements are
almost always unnecessary.
But normative judgements are something else. They
involve according due sensitivity to the values and norms associated with the
subject matter under consideration. By their very nature, normative judgements
are contextual. For example, if a student fails to appreciate the nature of religiosity
in fifteenth century England (for example, by seeing it in terms of ignorance
and superstition because of a modernist, secular approach which the student
brings to bear) then it is the normative judgement that has gone awry. Or, if a
research methodology fails to take due account of the needs of confidentiality,
again, it is a normative judgement that is deficient. Normative judgements
often operate in combination with other judgements (especially with process and
reflective judgements) and this is one of the reasons why academic judgement
can be complex.
Conclusion
If I am right then there are four basic elements to
an academic judgement. Typically in any assessment all four elements are
operating together – process, epistemic, reflective and normative. The judgements
we use are precisely those that we want our students to develop. We can see
straight away that attempts to categorise and tabulate all of these elements
may be helpful but are unlikely to be comprehensive. The precise nature of the
judgement will vary according to subject matter and no set of assessment
criteria that I have seen comes anywhere near to giving full justice to the
complexity of the judgements involved. Moreover, most of those outside academia
(government ministers, MPs, media people and the like) are just clueless
regarding how much they know of this complexity.
Complex, yes: but not so complex that we can’t
attempt to say what is involved in giving an academic judgement. But the above
sketch cannot be the last word – if I have succeeded in suggesting some initial
‘guidelines’ then that is a start.
SRHE member Geoff Hinchliffe teaches
undergraduates in the School of Education at the University of East Anglia.
This blog is partly based on a paper he gave at the 2018 SRHE Research
Conference.
Bibliography
Cambridge University (2018) Student
Complaints https://www.studentcomplaints.admin.cam.ac.uk/general-points-about-procedures/academic-judgment
Higher Education Act (2004) http://www.legislation.gov.uk/ukpga/2004/8/pdfs/ukpga_20040008_en.pdf
Kant, I (1933) Critique of Pure
Reason, trans. Kemp-Smith, N, London: Macmillan
Kant, I (2000) Critique of the
Power of Judgement, trans. Guyer, P and Matthews, E, Cambridge: Cambridge
University Press
OIA (2018) Guidance Note on the
OIA Rules http://www.oiahe.org.uk/rules-and-the-complaints-process/guidance-note-on-the-oias-rules.aspx#para30
Steinberger, P.J (2018), Political
Judgement, Cambridge: Polity Press
Wittgenstein, L (1958) Philosophical
Investigations, Oxford: Blackwell