By Paul Temple

Peer review has been in the news recently (well, what counts as news in our business): which perhaps isn’t surprising considering the effect it can have on academic careers – and much more besides.

Richard Smith, when editor of the BMJ, conducted an experiment by deliberately inserting errors into a paper (presumably one written specially for the occasion – this isn’t made clear!) and sending it to reviewers who were in the dark about what was going on. (A university ethics committee would have had fun with this.) None of the chosen reviewers apparently spotted all the errors: from which (along with other findings) Smith concluded that “peer review simply doesn’t work” (THE, 28 May 2015). But one of the reviewers, Trisha Greenhalgh of Oxford University, presents the same facts in an interestingly different light (THE, 4 June 2015). She spotted a couple of serious errors early in the paper, concluded it was rubbish, told the BMJ so, and read no further. So, for her, peer review was working just fine.

This is an interesting methodological point – opposite conclusions being drawn from the same events – but in our field, unlike natural science, I think the typical problem is a little different. I’m guessing that the errors that Smith dreamed up were essentially factual ones, allowing Greenhalgh and others quickly to dismiss the paper. The papers submitted to higher education journals are (mostly but not always) unlikely to contain glaring factual errors (thinking that the Robbins report argued against expansion, say): instead, in my experience as a journal editor, the problems will be more complex – perhaps an unclear argument, or a poor research design, or a limited grasp of the relevant literature. Other than at the extremes, these are matters on which reasonable people can disagree – thus creating the main challenge of editorship.

I once received a review of a paper that had been submitted to us which said, in effect, “This is a weak paper because it doesn’t take proper account of the work of X, who is the leading scholar in this field. The author should study X’s work before having another go.” You’ll have already guessed that the paper was, in fact, by X. I think that this mildly amusing story does actually show peer review working. I had myself thought that the paper was rather poorly put together and it certainly needed, and received, more work before it was published. The reviewer had studied it with an open mind, which I suspect wouldn’t have been the case if the identity of the author was known – the strong temptation would have been to think, “Here’s X writing on their specialist topic: has to be good.”

The serious downside of peer review is the scope it offers anonymous reviewers to make unnecessarily wounding comments about the work of a perhaps young, inexperienced writer, whose self-confidence may be badly dented by a savage review. (The blood pressure of old, experienced writers may also suffer.) Learning to handle criticism is a skill that academics need to develop early on (who teaches them?), but journal editors also have a responsibility to think about the possible impact of a harsh (as distinct from simply critical) review before passing it on. As our late colleague David Watson sometimes remarked, too many academics think that courtesy is a one-way street.

SRHE member Paul Temple, Centre for Higher Education Studies, UCL Institute of Education, University College London.