Building on ideas presented by Erin O’Shea and colleagues (ASAPbio meeting, February 2018), once an editor has invited a paper for peer review, eLife is committed to publishing the work (https://elifesciences.org/inside-elife/2905802e/peer-review-elife-trials-a-new-approach). 1) New submissions were evaluated by Senior Editors to identify the papers to be invited for peer review. 2) After peer review and consultation between the reviewers, a Reviewing Editor compiles the feedback and peer reviews. 3) The authors decide how to respond, and submit a revision and their response to the reviews (unless they wish to withdraw their submission). 4) The Reviewing Editor evaluates the revised submission and responses, and the paper is published along with the decision letter, peer reviews, author responses, and an editorial rating (to indicate whether all issues had been addressed, or whether major or minor issues remain unresolved). The trial was closed once 300 authors had opted in.
Goals and intentions
Three main motivations:
Builds on the consultative approach to peer review that eLife has developed by removing the gatekeeping role of peer review
Could reduce waste and unnecessary work when articles are submitted and reviewed by multiple journals before publication
Encourages evaluation of research based on the more than just the journal name in which the work is published
Examples of data we hope to gather:
percentage of authors who undertake the trial process
percentage of reviewers who agree to participate, and who sign their reviews
percentage of revised papers that address all the concerns raised during review
In “Peer Review: First results from a trial at eLife” (linked below), we compare the outcomes of 313 trial submissions with 665 regular submissions received during the same period of time. That is, almost a third of authors opted in to the trial approach. So far we have evaluated the first part of the editorial process (whether a paper is sent for in-depth review). The success rates at this first step for male and female last authors in the trial were similar (22.6% and 22.1% respectively), but late-career last authors fared better than their early- and mid-career colleagues. Further data will be posted and discussed covering the rest of the trial process.