Archives
- Home
- Blog
- Posts
- ReimagineReview news
- Community call recap: revealing quality of peer review through increased transparency
Community call recap: revealing quality of peer review through increased transparency
The theme of Peer Review week in 2019 was quality in peer review. However, the quality of most reviews are obscured to researchers because traditional peer review processes are conducted behind closed doors. We identified five active approaches to improving peer review quality through increased transparency and brought practitioners of each together in a community call. Interesting questions were raised during a subsequent discussion, regarding who truly values crediting peer review, and we invite your response in our Twitter poll.
View the recording of the call here and read on for our summary.
Five approaches to increase transparency
Signing peer review can foster a positive spirit in the research community
Our first speaker, Chris Jackson (professor and geologist at Imperial College of London) has signed all of his peer reviews in his career. Signing can help the reviewer maintain a constructive tone, leading to a positive mindset during review that can foster collective scholarship and collaborative spirit. He has not experienced any obvious retaliation, but instead has had many positive exchanges after the peer review. Chris encourages more of us to share our experiences of signing peer reviews, but acknowledges that the prevalence of signing peer review—and the culture surrounding this practice—may be highly varied between research disciplines.
Peer review transparency should serve to improve science
Next, Quincey Justman (Editor-in-Chief of Cell Systems) reminded us to keep in mind the young scientists who depend on robust dissemination systems in choosing a research direction. At Cell Systems, Quincey is leading two efforts at Cell Systems to encourage both transparency and peer review quality. First, they are publishing selected Peer Reviews as an article type, which recognizes the reviewers’s scholarship on their CV’s. These Peer Review articles serve as examples of good peer review and encourage trainees to peer review together with their mentors. Second, Cell Systems offers a transparent peer review process. Authors choosing transparent peer review will have their article published alongside a link to the preprint (if applicable), the reviews, editorial decisions letters, and previous versions of the manuscript, enabling a complete reconstruction of the peer review process from preprint to publication, which is missing in many other transparent peer review implementations.
Independent tags can signal level and type of review
Peer review is an important measure of validity for many academic bodies, in evaluation and funding, therefore it is critical to reveal the type and the level of review conducted. Our next speaker Amy Brand (Director of the MIT Press and co-lead of the Peer Review Transparency(PRT) Project) emphasized that using the reputation or particular branding of journals to credential work will only contribute to the dominance of established journals. This misses the opportunity to indicate the rigor of peer review of less established journals. PRT aims to distinguish rigor in peer review from journal brand, therefore removing some disincentives from publishing open-access platforms. PRT is starting to provide standardized tagging for aspects of the review process. These could serve to increase transparency as independent signifiers of quality.
Peer review policies should be clear
As a community, we aspire for changes in journal practices, however, the journal policies that underlie the practices are unavailable. Tony Ross-Hellauer (Leader of the Open and Reproducible Research Group at Graz University of Technology) and the Transpose team is countering this with a journal database on peer-review, co-reviewing and preprinting. In a landscape study, Transpose found that very often, journal policies are variable, hidden, undefined and unclear. This database is valuable to many types of users. For example, authors can identify journals that have more transparent practices in their field, editors that can understand the changing landscape in their area, and meta-researcher to monitor the changing statistics in certain policies.
Shared publisher data can probe the efficacy of peer review
Bahar Mehmani (Reviewer Experience Lead at Elsevier) joined Elsevier with goals of improving the peer review process from within the publishing company. Around the same time, she participated in PEERE, an organization of researchers who study peer review. Peer review data is needed to enable this research, but are often not available to publishers and society journals. Bahar established the protocols between various publishing stakeholders with submission systems to obtain the data. Together with PEERE, they analyzed an Elsevier pilot of five journals that switched to publishing peer review reports and the impact of this change on peer review performance. They found that reviewers’ acceptance rate and review time didn’t significantly change. However, the changes are not even across all demographics: Professors are more likely to decline, where early stage researchers (ESRs) are more likely to accept. A small group—young male ESRs write review reports with more objective language after the switch. These results should reassure those who are concerned that publishing peer review reports will damage peer review, in exchange for an increase in transparency.
Who really cares about review credit?
Credit for peer review is often brought up as a potential benefit to increased transparency. Whether published or not, review activity can be tracked systematically with Publons and ORCID and included in a CV. However, during the webinar, Rich Abdill (PhD student at the University of Minnesota and a creator of Rxivist) raised a question:who truly values reviewer credit?. Currently, few if any funders and institutions track peer review activity, so tangible benefits for researchers are scant. Do reviewers care about getting credited as a way to document their scholarship? Is recognition by the community sufficient incentive to change reviewer motivation? We encourage you to weigh in with the polls below.
Is it important to you to receive recognition for performing journal peer review?
— ASAPbio (@ASAPbio_) October 24, 2019
Do you think your contributions as a peer reviewer are valued by your funder(s) and institution(s) in funding, hiring, and promotion?
— ASAPbio (@ASAPbio_) October 24, 2019
Victoria Yan, ReimagineReview Coordinator
victoria.yan@asapbio.org
Header photo from Unsplash
