By enabling the publication of these peer review documents, we take a step towards opening up our editorial decision-making process. We hope that this will contribute to the scientific appreciation of the papers that we publish, in the same way as these results are already discussed at scientific conferences. We also hope that publication of the reviewer reports provides more credit to the work of our reviewers, as their assessment of a paper will now reach a much larger audience.I wonder, though, what impact this decision is liable to have on the peer review process. As I've blogged before (here and here), there are certainly benefits and drawbacks to peer review. Though the editor has the final decision of whether to publish a paper, peer reviewers can provide important feedback the editor may not have considered, and can be selected for their expertise in the topic under study (expertise the editor him/herself may not have). At the same time, reviewer comments are not always helpful, accurate, or even professional in their tone and wording.
The question, then, is whether peer review transparency will change any of that. Will reviewers be on better behavior if they think their comments may be made public? Perhaps, though they can still remain anonymous. One hopes it will at least encourage them to be more clear and descriptive with their comments, taking the time to show their thought process about a certain paper.
Of course, the impact of peer review transparency wouldn't just stop there. How might these public reviews be used by others? One potential issue is when news sources cover scientific findings. When I used to teach Research Methods in Psychology, one of my assignments was to find a news story about a study, then track down the original article, and make comparisons. Spoiler alert: many of the news stories ranged from being slightly misleading to completely inaccurate in their discussions of the findings. Misunderstanding of statistics, overestimating the impact of study findings, and applying findings to completely unrelated topics were just some of the issues.
Probably part of the issue is lack of scientific literacy, which is a widespread problem. There is no training for journalists covering research findings, though one wonders, if there were, whether it would look something like this:
|Source: SMBC Comics (one of my favorites)|
I'm not suggesting we hide this process. In fact, my past blog posts on peer review were really about doing the opposite. I just wonder what ramifications this decision will have on publishing. It should be noted that Nature Communications is planning to assess after a year whether this undertaking was successful. But if it is successful, is this something we think we'll see more of at other journals?