Study points out evolution of peer review in academic publishing, problems and alternatives


Library stacks. Credit: KU Marketing Communications

LAWRENCE — Ben Merriman has experienced all sides of academic publishing as an author, journal editor and reviewer. That experience, coupled with his research in administrative structures’ role in resolving policy conflicts, led the University of Kansas assistant professor public affairs & administration to explore the history of peer review in social sciences in a study for The American Sociologist. Merriman's study explores the historical emergence of major elements of editorial peer review, its inherent problems and frustrations, and how a more multifaceted publication model could address many of those issues.

Ben Merriman, KU assistant professor of public affairs & administrationIn his study, Merriman analyzed written annual reports of the editors of American Sociological Association journals, which started in 1952. Those reports spell out efforts taken to manage the publication process and how they led to much of today’s peer-review model being implemented by the early '80s. The process of blind review, external review, exclusive submission to one journal, revise and resubmit all were adopted over a period of 30 years of gradual, mostly unplanned change documented in the annual editorial reports. Journal editors, who originally guided content and operations of their publications, sought assistance in reviewing large loads of work and eventually came to a much different job.

“The role of editor seems to have evolved into the job of managing reviewers, as opposed to overseeing content,” Merriman said. “There are several prevailing ways of managing the problem of what content to publish. The most common way is by restricting the amount of stuff to review.”

In the study, Merriman compares the academic publishing model to those of literary and scholarly book publishing, literary periodical publishing, grant peer review and other models of scholarly journal editing.

“Compared to these alternatives, the success of editorial peer review in identifying outstanding work is open to debate, and it is without question a notably labor-inefficient method of evaluation,” he wrote.

The review process frequently takes more than a year from submission to publication and often longer. That can be a problem for young authors completing a doctorate and hoping to land a job, which can depend on having a good publication to the candidate’s name. It can also be a hindrance to young researchers advancing their careers in which publications are an essential accomplishment. Attempts to speed up the process have also led to increasing numbers of rejections of submissions without review, known as desk rejections. Editors can reject a piece for any reason, and the review process can lead to the rejection of ideas that don’t align with prevailing thoughts in the social sciences, at times stifling disagreement and debate and making otherwise good work unpublishable, Merriman said.

“I think there is a nonideological conservatism to the review process that discourages disagreement. Also, it’s called peer review, but the people doing the reviewing have often been in the field longer than the author and are more established. There has been Nobel-level research that has been rejected as well. I think disagreement can be very healthy for a field,” he said.

While Merriman said the process has several other problems, including labor inefficiency, long delays in getting findings published, difficulty detecting fraud and predicting impact of work, Merriman points out that it does have several merits as well, such as ensuring the basic soundness of the research. And unlike other models, modern peer review allows for nearly anyone to have their work considered without regard to status or renown.

“There is strong evidence that blind review protects, at least partly, against some of the biggest problems, like gender bias, for example,” Merriman said. “But you can have blind review without the extensive, lengthy process we have now. This basic model does many things quite well. It has definitely improved my work. There is something to it morally that I agree with as well. But it would be healthy if there were more of a variety of review methods.”

Problems presented by the current publication model in the social sciences could largely be addressed by more robust and varied means of publishing findings, Merriman wrote. Allowing people to make good faith arguments and to present findings quickly would be advantageous, and discovered errors could be corrected later. Additionally, allowing authors’ versions of articles to appear online before, or concurrently with reviewed versions, allows for more ideas to be shared widely. He also advocates for open access journals to be expanded to democratize knowledge in opposition to the privatization of research.

Above all, adopting more editorial approaches in academic publishing in the social sciences could address many of the current issues while retaining what the process gets right, Merriman said. While double-blind peer review is justified as one method to produce knowledge, understanding that it is a recent development and not a centuries-honored gold standard can help new ideas for sharing scholarship come to light.

“I think it’s easy to become accustomed to the notion of being frequently evaluated without reflection on if it’s truly the best method, or why it came to be,” Merriman said. “Knowing about the historical accidents could make it clear that this method is not inevitable. There are parts worth defending, and I think at least having more editorial approaches could help. Academia is full of smart people who I’m sure have excellent ideas on how to share knowledge with the world.”

Image credit: Library stacks. KU Marketing Communications

Thu, 09/16/2021

author

Mike Krings

Media Contacts