SRHE Blog

The Society for Research into Higher Education


1 Comment

The ‘unpoliceability’ of AI journal policies

by Dirk Lindebaum

Core to the mission of SRHE is the commitment to high-quality research and scholarship to advance understanding of higher education. This focus makes the conditions under which research is produced, including editorial and journal policies concerning the publication process, a topic of natural interest to readers of the SRHE blog.

The publication process, and how stakeholders interact in that process, however, is profoundly changing due to the widespread use of generative AI (hereafter AI). Consider the irony that authors can now offload their writing to AI; that reviewers can use AI to perform reviews on AI-produced text; and that editors can use AI to make decisions on papers written by AI, and grounding decisions in AI-produced reviews. It is machines chatting with machines, and machines reacting to machines, just without the informed doubt, the willingness of daring to know, the realization that we as social scientists always condition and are conditioned through our work and social practices that follow from our work.

Notwithstanding this development, the use of AI in the research process is routinely and boldly argued to be ‘inevitable’. What is less often understood is that this narrative likely accelerates an already existing crisis of credibility in some fields (eg in management research). A sobering look at the available empirical evidence (summarized here) underlines that the crisis is real.

First, empirical studies suggest that, in 2025, more than 140,000 non-existing references have infiltrated scientific records. Second, while the quality of scholarship is decreasing (as indicated through a measure of language complexity), the use of AI helps researchers to produce more papers and faster (with direct effects on individual careers). All this unfolds against an already disadvantageous baseline, where disruptive and innovate research occurs less often. Third, while many AI journal polices exhort scholars to be the ‘human-in-the-loop’ to check the accuracy of AI outputs, empirical studies suggest that AI biases (caused by human biases in the training data) transfer and magnify in humans through epistemic feedback loops in Human-AI interactions. Fourth, many authors neglect to disclose AI-use, the consequence being that many AI journal policies fail to ensure transparency or constrain AI adoption. Hence, many AI journal policies concerning the use of AI may attempt to ‘police the unpoliceable’, despite the well-intending motives behind such policies.

After years of editorial frontline experience, I believe two policy moves are necessary to counter these developments which, if unaddressed, likely compromise the integrity of science. On the one hand, qualitatively speaking, I propose a set of evaluative criteria concerning how to evaluate (as editors and reviewers), and prepare submissions (as authors), in ways that mobilise judgment to strengthen the voice of authors in terms of mastery of the literature, method, analysis, contribution and recommendations for practice. Developing editor and reviewer judgment to strengthen author voices, and for authors to force themselves to nurture said judgment to inform their own voices distinctly and uniquely, is a matter of paramount importance when the defining scarcity of our time is not access to information or even knowledge, but rather the capacity to develop and apply sound judgment in the presence of seemingly boundless AI slop. I treat judgment here as epistemic judgment concerning reflexive justifications for truth claims – to oneself and others under conditions of uncertainty – and as expressed through the authors’ voice.

This effort has to be earned. This means persisting with developing judgment over time, not despite the struggle involved, but because of it. This is exactly why this struggle matters for AI advocates. While some argue that one needs to be an expert to use the AI in research, this point is often based on decades of building expertise before the arrival of AI. Yet, current socialization practices are such that junior colleagues and PhD students may not find the conditions in which to develop expertise. This does and will negatively affect expertise levels now and in future. The result is that the conditions of re-building expertise shrink, and that means the possibility to recover lost expertise decreases. It seems to me a caution that needs to be put plainly – this should concern usall. However, these ‘qualitative’ criteria are unworkable without also creating the conditions in which editors and reviewers can manage the current deluge of journal submissions.

Therefore, the second, quantitative, proposal is to introduce a submission cap per authors per annum (as assessed by ORCID IDs), given record numbers of submissions to many journals across science subjects. This would provide necessary ‘breathing space’ for often overstretched (volunteering) editors and reviewers, while promoting equitable access to editorial resources, reduce congestion in the peer review pipeline, and discourage the use of the journal as a preliminary check on manuscript submission readiness. Lastly, a submission cap can also counter the creeping ‘junkification’ of science – the mounting volume of low-quality content that permeate academic publishing, which will only intensify in the age of Gen-AI.

Subject to amendments to suit the needs of a particular journal, the proposal would work like this: each author can submit three new submissions per year to a journal, but with a twist to encourage authors to improve the quality of first submission. For each invitation to revise and resubmit a paper, authors could submit one additional paper to a journal (3 + 1). And if they are lucky to get a second (or third) invitation to revise and resubmit a paper, then that number would be 3 + 2 etc. The expansion is author-specific; if submission A gets a revise and resubmit, every author listed on submission A receives +1 to their personal limit at this journal. This expansion would apply to new submissions, and not to invited revisions.

Some colleagues may have reservations about the proposal due to a perceived limit to their productivity. However, let’s appreciate that even if a prolific scholar manages to publish three articles in each of the best five journals in their fields per year, these 15 articles can hardly be said to be detrimental to the scholar’s career prospects.

A submission cap is consistent with scientific progress because it helps editors and reviewers regain vital space to write quality letters and reviews based on the proposed new set of evaluative criteria. Hence, it is key to consider the complementary nature of both proposals, with the benefit of enabling the conditions under which published works can be trusted (more) in the age of Gen-AI. Epistemic agency would flow back to journals and their editors, for academics would set the agenda (rather than big tech).

Rather than advocating contra AI in the research process, my stance here is pro developing human judgment. If developed and assessed well in the research process, the focus on judgment will render the debate around AI journal policies of secondary relevance. Why try to police the unpoliceable if investing in human judgment is one of the best ways to safeguard the integrity of science?

Dirk Lindebaum is Professor of Management & Organisation at the University of Bath, and advocate and owner of “AI-Free Scholarship” which is a registered (noncommercial) trademark that applies to his writing and teaching.

Image of Rob Cuthbert


Leave a comment

SRHE News on Publishing: reports from April 2023

by Rob Cuthbert

One of the benefits of SRHE membership is exclusive access to the quarterly newsletter, SRHE News, www.srhe.ac.uk/publications/srhe-newsletter. SRHE News typically contains a round-up of recent academic events and conferences, policy developments and new publications, written by editor Rob Cuthbert. To illustrate the contents, here is part of the April 2023 issue on recent developments in Publishing. If you would like to see a sample issue just email rob.cuthbert@uwe.ac.uk or rob.gresham@srhe.ac.uk.

Open access

John Sherer (North Carolina) blogged for The Scholarly Kitchen on 23 March 2023 about a recent initiative to publish open access monographs in history, reporting technical problems, author resistance but also much greater take-up/use, with about three times as many reported individual engagements as even a successful paywalled monograph.

An article on 6 March 2023 by Alexander B Belles and colleagues from Penn State in the Journal of Science Policy and Governance made recommendations about how to handle  the US Office of Science and Technology Policy requiring that all federally funded scholarly research be accessible to the public immediately upon publication. The article said: “While this open access policy will ultimately benefit society by increasing the availability of data and research outputs, it could place a heavy burden on researchers due to the relatively high cost of open access alongside an academic culture that tends to favor publishing in high impact subscription journals. We … offer recommendations for agencies, universities, and publishers to mitigate the impacts on researchers.” One recommendation was to consider cancelling publisher subscriptions and divert funds to author processing charges.

Jack Grove reported for Times Higher Education/insidehighered.com on 16 March 2023 on the suspiciously remarkable expansion of Swiss open-access publisher MDPI, which published no fewer than 240,500 articles in 2021, “just slightly fewer than Springer Nature and Elsevier’s combined open-access total that year, levying an average article processing charge of 1,258 Swiss francs ($1,364) per paper.” Jack Grove had reported for Times Higher Education on 15 March 2023 that analysis by economist Paolo Crosetto (National Research Institute for Agriculture, Food and Environment, France) showed “the number of MDPI’s special issues continued to rise sharply in 2022. Focusing on 98 MDPI journals with an impact factor, there were 55,985 special issues with a closing date in 2023, as of 23 February, Dr Crosetto told Times Higher Education. That compares with 39,587 open special issues identified at the end of March 2021, although only 10,504 of these eventually published anything. In 2022, 17,777 special issues published content.” Mark Hanson (Exeter) blogged about the predatoriness of MDPI on 25 March 2023.

Web of Science reported on 20 March 2023 that it had this year already disqualified some 50 journals, including an MDPI flagship journal, from having an impact factor in future. Christos Petrou of Scholarly Intelligence blogged for The Scholarly Kitchen on 30 March 2023 about the recent delisting of 50 journals, its implications for publishers, including MDPI, Hindawi and Wiley (which recently acquired Hindawi), and the consequences of the ‘guest editor’ model which underpins the recent growth of MDPI and other journals.

Shaping the field of lifelong education

The editors of theInternational Journal of Lifelong Education looked back on 40 years of the journal  to develop themes which had shaped the field. They chose “citizenship and its learning; learning in, through and for work; and widening participation and higher education”. The article by John Holford (Nottingham) and his co-editors was part of the journal’s retrospective issue 41(6) (2 November 2022).

Books with DOIs are more discoverable on Google Scholar

Lettie Y Conrad (independent) and Michelle Urberg of EBSCO blogged for The Scholarly Kitchen about their funded study to find how metadata contributes to the successful discovery of academic and research literature via the mainstream web. “Initial results indicated that DOIs have an indirect influence on the discoverability of scholarly books in Google Scholar — however, we found no direct linkage between book DOIs and the quality of Google Scholar indexing or users’ ability to access the full text via search-result links. Although Google Scholar claims to not use DOI metadata in its search index, the results of our mixed-methods study of 100+ books (from 20 publishers) demonstrate that books with DOIs are generally more discoverable than those without DOIs.

Why journal submissions get rejected

Alex Edmans (London Business School) reflected on his experience as editor of the Review of Finance and analysed his reasons for rejecting nearly 1000 submissions, for SSRN on 9 February 2023.

The ethics of peer review

The endless lament of journal editors about finding reviewers continued, as Dirk Lindebaum (Grenoble Ecole de Management) and Peter J Jordan (Griffith) mused in Organization (30(2) 396-406) on reviewer disengagement: “… an audit culture in academia and individual incentives (like reduced teaching loads or publication bonuses) have eroded the willingness of individuals to engage in the collective enterprise of peer-reviewing each others’ work on a quid pro quo basis. … it is unethical for potential reviewers to disengage from the review process … we aim to ‘politicise’ the review process and its consequences for the sustainability of the scholarly community. We propose three pathways towards greater reviewer engagement: (i) senior scholars setting the right kind of ‘reviewer’ example; (ii) journals introducing recognition awards to foster a healthy reviewer progression path and (iii) universities and accreditation bodies moving to explicitly recognise reviewing in workload models and evaluations. … the latter point … aligns individual and institutional goals in ‘measurable’ ways. In this way, ironically, the audit culture can be subverted to address the imbalance between individual and collective goals.”

Identity theft prompts scientists worldwide to contemplate legal action

Jack Grove reported for Times Higher Education/insidehighered.com on 10 February 2023 that many leading scientists had been wrongly named as authors or editors on AI-generated papers and predatory journals. Some were considering legal action, which might be supported by UKRIO.

The gaming of citation and authorship

Stuart Macdonald (Leicester) wrote a truly terrifying analysis of the extent of misrepresentation in academic publishing, in Social Science Information (online 7 February 2023): “Many authors in medicine have made no meaningful contribution to the article that bears their names, and those who have contributed most are often not named as authors. Author slots are openly bought and sold. The problem is magnified by the academic publishing industry and by academic institutions, pleased to pretend that peer review is safeguarding scholarship. In complete contrast, the editors of medicine’s leading journals are scathing about just how ineffectual is peer review in medicine. Other disciplines should take note lest they fall into the mire in which medicine is sinking.”

APCs are a heavy burden for middle-income countries

Alicia J Kowaltowski (São Paolo) and colleagues from Brazil blogged for The Scholarly Kitchen on 9 March 2023 about the way author processing charges can be a major problem for middle-income countries like Argentina, Brazil, India, Mexico, and South Africa.

Predatory journals and the mislocated centres of scholarly communication

Franciszek Krawczyk and Emanuel Kulczycki (both Adam Mickiewicz University in Poznań, Poland) argued in their article in Tapuya: Latin American Science, Technology and Society (2021, 4(1)) that so-called predatory journals may have a significant role in enabling otherwise marginalised scholars to maintain their academic careers despite a location on the periphery of mainstream academic debate. “Knowledge production is an important factor in establishing the geopolitical position of countries … we introduce the term “mislocated centres of scholarly communication” to help better understand the emergence of predatory journals, and journals that bear similarities to them, in geopolitical peripheries. Mislocated centers of scholarly communication are perceived in the peripheries as legitimized by the center but are in fact invisible or illegitimate in the center. Thus, we argue the importance of viewing these mislocated centers as the result of unequal power relations in academia. … predatory journals are a geopolitical problem because the geopolitical peripheries of science are much more often harmed by them than the center. Unlike predatory journals, mislocated centers of scholarly communication are not necessarily fraudulent but rather they are geopolitical roles imposed on some journals by a dynamic between center and peripheries.”

Routledge/Taylor & Francis acquire US publisher Stylus

The founder of Stylus Publishing announced in an email to authors on 2 March 2023 that the publisher will be sold to Taylor & Francis and operate as part of its Routledge division, as Doug Lederman reported for insidehighered.com on 3 March 2023. “Founded in 1996, Stylus’ publishing focuses on higher education, covering such areas as teaching and learning, student affairs, professional development, service learning and community engagement, study abroad, assessment, online learning, racial diversity on campus, women’s issues, doctoral education, adult education, and leadership and administration.” The publisher seems mainly to produce practical guides for US HE, with no obvious impact more widely.

Rob Cuthbert is the editor of SRHE News and Blog, emeritus professor of higher education management, Fellow of the Academy of Social Sciences and Fellow of SRHE. He is an independent academic consultant whose previous roles include deputy vice-chancellor at the University of the West of England, editor of Higher Education Review, Chair of the Society for Research into Higher Education, and government policy adviser and consultant in the UK/Europe, North America, Africa, and China. He is current chair of the SRHE Publications Committee and of the Editorial Advisory Board for Studies in Higher Education.

Paul Temple


2 Comments

End-of-the-peer review?

By Paul Temple

Peer review has been in the news recently (well, what counts as news in our business): which perhaps isn’t surprising considering the effect it can have on academic careers – and much more besides.

Richard Smith, when editor of the BMJ, conducted an experiment by deliberately inserting errors into a paper (presumably one written specially for the occasion – this isn’t made clear!) and sending it to reviewers who were in the dark about what was going on. (A university ethics committee would have had fun with this.) None of the chosen reviewers apparently spotted all the errors: from which (along with other findings) Smith concluded that “peer review simply doesn’t work” (THE, 28 May 2015). But one of the reviewers, Trisha Greenhalgh of Oxford University, presents the same facts in an interestingly different light (THE, 4 June 2015). She spotted a couple of serious errors early in the paper, concluded it was rubbish, told the BMJ so, and read no further. So, for her, peer review was working just fine.

This is an interesting methodological point – Continue reading →