- Profile
- Comments 0
- prev
- next
- Website
- Leave a comment
- prev
- next
In a nutshell
We conducted a randomised controlled trial to see whether telling peer reviewers that their signed reviews of original research papers might be posted on The BMJ's website would affect the quality of their reviews.
Goals and intentions
Traditional peer review for scientific and medical journals has major flaws. Although the peer reviewer knows the identity of the author of the manuscript being assessed, the author doesn’t know the identity of the peer reviewer. Over recent years this form of peer review has come under increasing criticism, largely for its lack of accountability. Unscrupulous reviewers can delay or prevent the publication of work with which they disagree, or promote inappropriately the work of likeminded researchers. Worse still, reports exist of anonymous reviewers appropriating ideas and words from the manuscripts they have been reviewing. Although the peer review process has changed and reviewers for some journals have become used to more openness, evidence about the effect of opening review further is still required. Trust in the processes of science is still relevant, and reluctance remains, at least in some areas of medical research, to fully open up peer review. We set out to investigate the effects of reviewers’ foreknowledge that their signed review might be shared not just with co-reviewers and authors, but also with any interested reader.
Project status
Types of outputs
Review process
-
Review requested byAuthors
-
Reviewer selected byEditor, service, or community
-
Public interactionNo
-
Author responseYes
-
DecisionBinary decision
Review policy
-
Review coverageComplete paper
-
Reviewer identity known toPublic
-
Competing interestsChecked
Review features
-
Notes
Consecutive eligible papers were randomised either to have the reviewer’s signed report made available on the BMJ’s website alongside the published paper (intervention group) or to have the report made available only to the author—the BMJ’s normal procedure (control group). The intervention was the act of revealing to reviewers—after they had agreed to review but before they undertook their review—that their signed report might appear on the website. The main outcome measure was the quality of the reviews, as independently rated on a scale of 1 to 5 using a validated instrument by two editors and the corresponding author.
-
Review of code or dataNo
-
Eligible reviewers/editorsThe first of the editors’ four potential reviewers was contacted with details of the study, including the title and author of the paper, and asked whether he or she was willing to review the paper. At this point, no information was given about the study arm into which the paper had been allocated. If the first reviewer declined, the next was contacted, continuing until a consenting reviewer was found. If none of the four potential reviewers consented, the paper was excluded.
-
Criteria for inclusion
Consecutive research manuscripts chosen by editors for external review between November 1999 and July 2000 were eligible for inclusion.
Transparency
Results
-
Number of scholarly outputs commented on100-1,000
-
Results summary
Telling peer reviewers that their signed reviews might be available in the public domain on the BMJ’s website had no important effect on review quality. Although the possibility of posting reviews online was associated with a high refusal rate among potential peer reviewers and an increase in the amount of time taken to write a review, we believe that the ethical arguments in favour of open peer review more than outweigh these disadvantages.
-
Results URL
Add a comment