Brain power

Does peer review in a competitive environment result in important research going unpublished?

Researchers face significant pressure to ‘publish or perish’, with the expectation that they will publish frequent articles in academic journals, and preferably in high-impact journals.

However, the top journals have extremely high paper rejection rates – above 80% – which creates a highly competitive publishing environment. A recent paper1 looks at the effects of this competition in a peer review system, using an experiment that the authors have called the ‘Art Exhibition Game’.

As discussed in The Conversation by one of the paper authors, the Art Exhibition Game is a simplified version of the academic journal publishing system, simulated in an artistic context. Instead of writing academic papers, participants used a computer interface to draw images, and instead of choosing an academic journal for publication, participants chose an available exhibition to display their work. As is done in most academic journals, double-blind peer-review was used to decide if images were good enough to display.

Through the game, the paper authors wanted to investigate three fundamental aspects of competition:

  • Does competition promote or reduce innovation?
  • Does competition reduce or improve the fairness of the reviews?
  • Does competition improve or hamper the ability of reviewers to identify valuable contributions?

Results

  1. Competition is a double-edged sword: whereas on the one hand, competition fosters innovation and diversity of products, on the other hand, it also leads to more unfair reviews and to a lower level of agreement between reviewers.
  1. Competition does not improve (nor worsen) the validity of the outcomes of peer review. In fact, under competition the rejection rate increases by 20%, meaning that both more low-quality and more high-quality work is eliminated.
  1. Even if outcomes of peer review are generally valid, competition increases the rejection of high-quality items and encourages self-interested referees to behave strategically. The results are consistent with other empirical studies that found that the most competitive journals in the field of medical sciences failed to accept some of the most cited articles, which later appeared in lower-tier journals.

In response, the article authors make recommendations for reform:

In times where science is increasingly witnessing peer review rings, scientific fraud, and plagiarism, the results of our study suggest a redesign the scientific incentive system such that sustainable forms of competition are promoted. For example, career schemes that tolerate early failure and focus on long-term success (50) could be the best way to guarantee high levels of responsible innovation.

Reference:

  1. Balietti, S., Goldstone, R. L., & Helbing, D. (2016). Peer review and competition in the Art Exhibition Game. Proceedings of the National Academy of Sciences, 201603723.

Also published on Medium.

Bruce Boyes

Bruce Boyes (www.bruceboyes.info) is editor, lead writer, and a director of the award-winning RealKM Magazine (www.realkm.com), and a knowledge management (KM), environmental management, and project management consultant. He holds a Master of Environmental Management with Distinction, and his expertise and experience includes knowledge management (KM), environmental management, project management, stakeholder engagement, teaching and training, communications, research, and writing and editing. With a demonstrated ability to identify and implement innovative solutions to social and ecological complexity, Bruce's many career highlights include establishing RealKM Magazine as an award-winning resource, using agile and knowledge management approaches to oversee an award-winning $77.4 million western Sydney river recovery program, leading a knowledge strategy process for Australia's 56 natural resource management (NRM) regional organisations, pioneering collaborative learning and governance approaches to support the sustainable management of landscapes and catchments, and initiating and teaching two new knowledge management subjects at Shanxi University in China.

Related Articles

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Back to top button