Explore
    Explore

    Effect of open peer review on quality of reviews and on reviewers'recommendations: a randomised trial

    Randomised trial of open review

    In a nutshell

    In 1998 we conducted a randomised trial at The BMJ, a large general medical journal, to examine the effect on peer review of asking reviewers to have their identity revealed to the authors of the paper. Outcome measures included review quality (using the validated Review Quality Instrument), the recommendation regarding publication, the time taken to review.

     

    Overview
    Goals and intentions

    Previous RCTs at The BMJ failed to confirm that blinding of reviewers to authors' identities improved the quality of reviews so we decided to experiment with open identities peer review. To us it seemed unjust that authors should be “judged” by reviewers hiding behind anonymity: either both should be unknown or both known, and it is impossible to blind reviewers to the identity of authors all of the time. We therefore conducted a randomised trial to confirm that open review did not lead to poorer quality opinions than traditional review and that reviewers would not refuse to review openly (because open review would then be unworkable).

    What is reviewed
    • Review of data or code

      No

    Format of review
    Transparency
    Review features
    • Eligible reviewers/editors

      Consecutive manuscripts received by the BMJ and sent by editors for peer review during the first seven weeks of 1998 were eligible for inclusion. Four potential clinical reviewers were selected by one of the BMJ's 13 editors. Two of these four reviewers were chosen to review the manuscript. The other two were kept in reserve in case a selected reviewer declined. The selected reviewers were randomised either to be asked to have their identity revealed to authors (intervention group) or to remain anonymous (control group), forming a paired sample.

    Results
    • Number of scholarly outputs commented on

      100-1,000

    • Metrics

      A questionnaire survey was undertaken of the authors of a cohort of manuscripts submitted for publication to find out their views on open peer review.

    • Results summary

      Asking reviewers to consent to being identified to the author had no important effect on the quality of the review, the recommendation regarding publication, or the time taken to review, but it significantly increased the likelihood of reviewers declining to review. Most authors were in favour of open peer review. (van Rooyen S, Godlee F, Evans S, Black N, Smith R. Effect of open peer review on quality of reviews and on reviewers' recommendations: A randomised trial. BMJ 1999; 318: 23 – 27.)

    • Results URL

      https://www.bmj.com/content/318/7175/23

    mood_bad
  • No comments yet.
    • Add a comment

    Leave a Reply ·

    Your email address will not be published. Required fields are marked *