Glossary

Glossary

Below is a list of the terms used to describe listings on ReimagineReview, some with additional descriptions and definitions.

Types of project

  • Trials: Trials are time-limited experiments conducted by preexisting entities, such as journals and platforms.
  • Projects: Projects are platforms or services.
  • Observational studies: Observational studies are studies in which no intervention is made, but peer review is studied. Can include meta-analysis and modeling.

Status

  • Proposed: Not yet under active development
  • In preparation: Under development, but not operating
  • Active: Currently operating
  • Complete: No longer active

Review or curation initiated by

  • Author initiates review: The author decides to initiate the peer review process. Eg, traditional journal review is initiated by the authors’ submission.
  • Editor or moderator initiates review: Editor or the review service moderators initiate review or curation. 
  • Reviewer initiates review: The review process is initiated by reviewers, such as in a comment left on a website.

Post-review decision

  • Binary decision: Reviewers and/or editors make a decision to accept or reject the article for certification or publication.
  • No decision: No decision or certification is made post-review.
  • Point-based evaluation: The manuscript is evaluated by voting or point-based system.

Functions of the peer review

  • Curation of interesting work
  • Feedback to authors
  • Validation of soundness

Areas of innovation

  • Bias in review
  • Incentives and recognition for reviewing
  • Reviewer training
  • Transparency
  • Costs: in money and/or time and resources
  • Quality of review: rigor, thoroughness, or tone of the review
  • Speed

Types of outputs

The type(s) of scientific product that is reviewed or evaluated by the project. For example, journals typically only evaluate manuscripts submitted directly (regardless of whether those manuscripts have been preprinted), so they should select “Privately shared manuscripts” unless preprints are specifically important to the evaluation process.
  • Journal accepted manuscripts: The evaluation occurs on papers that have already been published/accepted by a journal (post-publication evaluation).
  • Preprints
  • Other scholarly outputs: Data, protocol, code etc.
  • Privately shared manuscripts:  Note that a typically journal process operates on privately shared manuscripts. 

Transparency

  • Double blind: Reviewers are unaware of the authors’ identities, and vice versa
  • Open interaction: Reviewers communicate with one another during the review process
  • Open reports: The content of peer review reports is published
  • Open identities: The identities of peer reviewers are published
  • Open participation: Anyone can participate in the peer review process
  • Single blind: Authors are unaware of reviewers’ identities, but NOT vice versa

Format

  • Annotations
  • Quantitative scores
  • Free-form commenting
  • Structured review form

Process

  • Automation: Filtering or evaluation of scientific output using automated technologies and programs. 
  • Comment moderation: Comments are screened or moderated before or after posting.
  • Journal integration: (For review happening outside the traditional journal format) editors at one or more journals integrate the content of the review into their own processes or the final presentation of the paper, or do journals send information to the project.
  • Pre-publication review: Review prior to public disclosure (of a preprint or other output).
  • Comment indexing: Comments are given a DOI or included in disciplinary databases
  • Comment rating (meta-evaluation): Comments can be up- or down-voted, highlighted, or reviewed themselves.
  • Post-publication review: Review after public disclosure (or a preprint or other output)
  • Professional editors
  • Preregistration: A proposed research plan that tests a predicted relationship, with a defined research timeline. De