Peers Review

GG has been skeptical of many suggestions about peer review over the years, things like post-publication review, public peer review, publishing via blogs, or outright elimination of peer review. But the latest wrinkle might bear some thought.

Editors for the journal Synlett decided to try something a bit different, something they describe as intelligent crowd review.  In essence, this is creating a forum populated by some range of experts (they recruited 100) and then tossing a submission into the forum and letting the experts do what they will.  Overall they got results faster and with greater insight than traditional peer review.

Why is this?  As the Ars Technica article on this suggests, in this environment, reviewers can just focus on what they know backwards and forwards.  Yeah, that is a fair introduction, or, no, that equation is inappropriate. So you dive in (maybe as lunchtime entertainment), shoot all the fish in your barrel and leave the rest of the manuscript for others.

The biggest advantage of this proposal is in reviewing complex multi-disciplinary papers where a reviewer either has to say “I can’t review this part of the paper because it is too far out of my expertise” or has to bone up on material he or she is unfamiliar with. Either of these tends to slow the review process down. Given the increased emphasis and visibility of such research, embracing such an approach might be a boon to editors and authors alike.

Of course there are problems that would have to be solved. Avoiding conflicts of interest could get challenging; this might get harder too if an author specifically requests certain individuals not be granted access to a submission. The system apparently requires the editor to assemble the resulting crowd review, which could in some cases require the editor to fill in gaps. Whether such a system would breed a new kind of burnout remains to be seen.

But this might be one of the better hopes for getting out of the peer-review rut we are presently in.  It is certainly worth careful consideration.

Tags: ,

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: