- Website
- Leave a comment
- prev
- next
In a nutshell
For detailed article please check or OA article and its supplementary data: https://www.nature.com/articles/s41467-018-08250-2
In November 2014, five Elsevier journals agreed to be involved in the Publication of Peer Review reports as articles (from now on, PPR) pilot. During the pilot, these five journals openly published typeset peer review reports with a separate DOI, fully citable and linked to the published article on ScienceDirect. Review reports were published freely available regardless of the journal’s subscription model (two of these journals were open access, while three were published under the subscription-based model). For each accepted article, all revision round review reports were concatenated under the first round for each referee, with all content published as a single review report. Different sections were used in cases of multiple revision rounds. For the sake of simplicity, once agreed to review, referees were not given any opt-out choice and were asked to give their consent to reveal their identity. In agreement with all journal editors, a text was added to the invitation letter to inform referees about the PPR pilot and their options. At the same time, authors themselves were fully informed about the PPR when they submitted their manuscripts. Note that while one of these journals started the pilot earlier in 2012, for all journals the pilot ended in 2017.
Goals and intentions
Our aim was to understand whether knowing that their report would be published affected the referees’ willingness to review, the type of recommendations, the turn-around time and the tone of the report. These are all aspects that must be considered when assessing the viability and sustainability of open peer review. By reconstructing the gender and academic status of referees, we also wanted to understand whether these innovations were perceived differently by certain categories of scholars
Types of outputs
Review process
-
Review requested byAuthors
-
Reviewer selected byEditor, service, or community
-
Public interactionNo
-
Author responseYes
-
DecisionBinary decision
Review policy
-
Review coverageComplete paper
-
Reviewer identity known toEditor or service
-
Competing interestsChecked
Gallery
Review features
-
Manuscript hostingYes
-
Review of code or dataYes
-
Eligible reviewers/editorsEditors are appointed for the journals. They choose reviewers and manage the peer review process.
-
Tags or badgesNo
Transparency
Results
-
Number of scholarly outputs commented on10,000+
-
MetricsYes, article page usage data on Science Direct shows one out of three click on the pilot journal article pages is on the peer review reports. Surveying editors of participating journals shows editors are using published peer review reports as a resource material for training young reviewers.
-
Results summary
we measured changes both before and during the pilot and found that publishing reports did not significantly compromise referees’ willingness to review, recommendations, or turn-around times. Younger and non-academic scholars were more willing to accept to review and provided more positive and objective recommendations. Male referees tended to write more constructive reports during the pilot. Only 8.1% of referees agreed to reveal their identity in the published report. These findings suggest that open peer review does not compromise the process, at least when referees are able to protect their anonymity.
-
Results URL
Add a comment