Does peer review in a competitive environment result in important research going unpublished?

Researchers face significant pressure to ‘publish or perish’, with the expectation that they will publish frequent articles in academic journals, and preferably in high-impact journals.

However, the top journals have extremely high paper rejection rates – above 80% – which creates a highly competitive publishing environment. A recent paper¹ looks at the effects of this competition in a peer review system, using an experiment that the authors have called the ‘Art Exhibition Game’.

As discussed in The Conversation by one of the paper authors, the Art Exhibition Game is a simplified version of the academic journal publishing system, simulated in an artistic context. Instead of writing academic papers, participants used a computer interface to draw images, and instead of choosing an academic journal for publication, participants chose an available exhibition to display their work. As is done in most academic journals, double-blind peer-review was used to decide if images were good enough to display.

Through the game, the paper authors wanted to investigate three fundamental aspects of competition:

• Does competition promote or reduce innovation?
• Does competition reduce or improve the fairness of the reviews?
• Does competition improve or hamper the ability of reviewers to identify valuable contributions?

Results

1. Competition is a double-edged sword: whereas on the one hand, competition fosters innovation and diversity of products, on the other hand, it also leads to more unfair reviews and to a lower level of agreement between reviewers.

2. Competition does not improve (nor worsen) the validity of the outcomes of peer review. In fact, under competition the rejection rate increases by 20%, meaning that both more low-quality and more high-quality work is eliminated.

3. Even if outcomes of peer review are generally valid, competition increases the rejection of high-quality items and encourages self-interested referees to behave strategically. The results are consistent with other empirical studies that found that the most competitive journals in the field of medical sciences failed to accept some of the most cited articles, which later appeared in lower-tier journals.

In response, the article authors make recommendations for reform:

In times where science is increasingly witnessing peer review rings, scientific fraud, and plagiarism, the results of our study suggest a redesign the scientific incentive system such that sustainable forms of competition are promoted. For example, career schemes that tolerate early failure and focus on long-term success (50) could be the best way to guarantee high levels of responsible innovation.

Reference:

Also published on Medium.