This article is part 3 of a series reviewing selected papers and associated commentary from Altmetric’s list of the top 100 most discussed and shared research and commentary of 2020.
The #9 most discussed and shared article1 in Altmetric’s top 100 list for 2020 is a little different because it’s a retracted article, with the cause of the retraction being the reason why the article has been so widely discussed and shared.
Titled “The association between early career informal mentorship in academic collaborations and junior author performance,” the article reports on a study of mentoring in scientific collaborations, where junior scientists receive support from potentially multiple senior collaborators who don’t necessarily have a formal supervisory role. The study involved identifying 3 million mentor–protégé pairs, and then conducting a survey with a random sample of protégés.
Mentoring is considered a valuable knowledge management (KM) tool2 because it can very effectively transfer knowledge between highly experienced and less experienced organizational members. While typically seen as facilitating the transfer of knowledge from senior or older organizational members to their junior or younger colleagues, mentoring can also involve younger staff assisting older staff.
The scientific collaboration mentorship study findings support the effectiveness of mentoring, with the researchers stating that “We find that mentorship quality predicts the scientific impact of the papers written by protégés post mentorship without their mentors.”
However, the researchers also drew other conclusions that provoked strong criticism from the scientific community through numerous Tweets and open letters to Nature Communications, the journal that had published the article. The researchers had stated that:
We also find that increasing the proportion of female mentors is associated not only with a reduction in post-mentorship impact of female protégés, but also a reduction in the gain of female mentors. While current diversity policies encourage same-gender mentorships to retain women in academia, our findings raise the possibility that opposite-gender mentorship may actually increase the impact of women who pursue a scientific career. These findings add a new perspective to the policy debate on how to best elevate the status of women in science.
As Retraction Watch reports, an example of the criticism is the following Tweet from Dr. Daniela Witten, a biostatistician who is a professor and the Dorothy Gilford Endowed Chair of Mathematical Statistics at the University of Washington.
The recent @NatureComms paper doesn’t tell us much about the impact of gender on mentorship but it sure does tell us that the statistics community needs to do a better job teaching scientists about correlation, causation, and confounding
— Dr. Daniela Witten (@daniela_witten) November 19, 2020
In response to the criticisms, Nature Communications added a note to the article on 19 November 2020 stating that the journal editors were investigating the concerns raised, and then following the investigation, formally retracted the article on 21 December 2020. An editorial3 announcing the retraction states:
The criticisms from readers revolved around the validity of the conclusions in light of the available data, assumptions made and methodology used. In particular, readers criticised the use of co-authorship as a measure of mentorship, and citations as a measure of success of the mentoring relationship…
We followed our established editorial processes, which involved recruiting three additional independent experts to evaluate the validity of the approaches and the soundness of the interpretation. They supported previous criticisms and identified further shortcomings in relation to the use of co-authorship as a measure of informal mentorship. They also noted that the operationalisation of mentorship quality, based on the number of citations and network centrality of mentors, was not validated.
According to these criticisms, any conclusions that might be drawn on biases in citations in the context of co-authorship cannot be extended to informal mentorship. As such, the paper’s conclusions in their current form do not stand, and the authors have retracted the paper…
Simply being uncomfortable with the conclusions of a published paper, would and should not lead to retraction on this basis alone. If the research question is important, and the conclusions sound and valid, however controversial, there can be merit in sharing them with the research community so that a debate can ensue and a range of possible solutions be proposed. In this case, the conclusions turned out not to be supported, and we apologise to the research community for any unintended harm derived from the publication of this paper.
As part of our investigation, we also reviewed our editorial practices and policies and, in the past few weeks, have developed additional internal guidelines, and updated information for authors on how we approach this type of paper. As part of these guidelines, we recognise that it is essential to ensure that such studies are considered from multiple perspectives including from groups concerned by the findings. We believe that this will help us ensure that the review process takes into account the dimension of potential harm, and that claims are moderated by a consideration of limitations when conclusions have potential policy implications.
What does this mean for knowledge management?
Scientific research literature is a key evidence source in evidence-based management. However, prior to use, all scientific research evidence must be critically appraised to judge its trustworthiness, value, and relevance in a particular context. The scientific collaboration mentorship study highlights the importance of this.
This particular study was retracted because its sexist bias triggered widespread concern, exposing serious flaws in the research. However, while academic journals are purging sexist and also racist work, there’s a lot of other less obvious flawed research still in circulation, so vigilance is needed. Further examples and case studies of flawed research and how to identify it can be found in our evidence-based KM resources and our “Critical eye” and “Quality of science and science communication” series.
- AlShebli, B., Makovi, K., & Rahwan, T. (2020). RETRACTED ARTICLE: The association between early career informal mentorship in academic collaborations and junior author performance. Nature communications, 11(1), 1-8. ↩
- Young, R. (ed)(2010). Knowledge management tools and techniques manual. Asian Productivity Organization, Tokyo. ↩
- Editors. (2020). Regarding mentorship. Nature Communications, 11, 6447. ↩
Also published on Medium.