April 2, 2012 Leave a comment
Marc Bellemare with a post on adding a crowd-sourcing element to peer review. Marc highlights an editorial in Review of Financial Studies that asserts that what is often viewed as the last step–the publication of a paper in a peer review journal–should be seen as the first step, with the real value of a paper determined by the give and take that comes after publication.
It worries me to move away from the peer review system we have without having something better to replace it, especially the statement in the editorial that we shouldn’t worry about whether a paper is correct or not, the crowd sourcing will sort that out. Evidence is very important to policy discussions (or at least it should be….) and one of the reasons I blog is to try and lengthen the ‘shelf life’ of good, peer reviewed research, so we need to strive for identifying ‘correct’ research, with the crowd sourcing perhaps helping to identify how important a study may be. I stand by my ideas of how to change peer review (along with the uncertainties) that were published in January, 2012, and which are summarized below.
- The identity of reviewer and reviewee should be known to one another
- The title of manuscripts under review should be public, along with the authors of the manuscript and the identity of the reviewers
- How long the reviewers have been reviewing the manuscript should be public
- How long authors have had a request for revision should be public
- Upon publication, the correspondence between reviewers/editors/authors should be public (this is important because often people say “why didn’t you do this subanalysis”; often it was done, but cut from a published paper due to length restrictions)
- The use of online early publication is a good thing; I wonder if it will eventually become the only modality? (I only take one journal in hard copy now, Health Affairs, and otherwise utilize Duke University’s global subscription service)
- Gated papers hinder academic investigation and discourse, but I am unsure of how to fund journals without subscriptions
This is an important conversation for researchers interested in policy, and one that seems to produce more questions than answers.