In a nutshell

In 1998 we conducted a randomised trial at The BMJ, a large general medical journal, to examine the effect on peer review of asking reviewers to have their identity revealed to the authors of the paper. Outcome measures included review quality (using the validated Review Quality Instrument), the recommendation regarding publication, the time taken to review.

 

Overview
Goals and intentions

Previous RCTs at The BMJ failed to confirm that blinding of reviewers to authors' identities improved the quality of reviews so we decided to experiment with open identities peer review. To us it seemed unjust that authors should be “judged” by reviewers hiding behind anonymity: either both should be unknown or both known, and it is impossible to blind reviewers to the identity of authors all of the time. We therefore conducted a randomised trial to confirm that open review did not lead to poorer quality opinions than traditional review and that reviewers would not refuse to review openly (because open review would then be unworkable).

What is reviewed
  • Review of data or code

    No

Format of review
Transparency
Review features
  • Eligible reviewers/editors

    Consecutive manuscripts received by the BMJ and sent by editors for peer review during the first seven weeks of 1998 were eligible for inclusion. Four potential clinical reviewers were selected by one of the BMJ's 13 editors. Two of these four reviewers were chosen to review the manuscript. The other two were kept in reserve in case a selected reviewer declined. The selected reviewers were randomised either to be asked to have their identity revealed to authors (intervention group) or to remain anonymous (control group), forming a paired sample.

Results
  • Number of scholarly outputs commented on

    100-1,000

  • Metrics

    A questionnaire survey was undertaken of the authors of a cohort of manuscripts submitted for publication to find out their views on open peer review.

  • Results summary

    Asking reviewers to consent to being identified to the author had no important effect on the quality of the review, the recommendation regarding publication, or the time taken to review, but it significantly increased the likelihood of reviewers declining to review. Most authors were in favour of open peer review. (van Rooyen S, Godlee F, Evans S, Black N, Smith R. Effect of open peer review on quality of reviews and on reviewers' recommendations: A randomised trial. BMJ 1999; 318: 23 – 27.)

  • Results URL
mood_bad
  • No comments yet.
  • Add a comment