The Impending Collapse of Peer Review

GG has defended peer review a few times as a means of limiting the damage from flawed papers.  It is a positive good for science despite its limitations.  But we are quite possibly in the waning days of peer review.

What has inspired this dystopian view? GG is an associate editor and has been for many years, and it has gotten ridiculously hard to get reviews.  The growth of multi-author papers means that the number of conflicted potential reviewers has grown, limiting the reviewer pool. More and more, potential reviewers are choosing to not even respond to a request for a review–which eats up more time than a simple “no”.  Others are giving the quick “no,” which is better, but still extends the reviewing process. Others are agreeing, but then decide the task is more onerous than cleaning up after a sick dog; months can go by with no response or just a hurried “getting to it now”. Sometimes there is never a review. Meanwhile, authors justifiably fume at the long times their papers are in review. At some point, the system will simply break down: authors will opt for venues not requiring review or using some form of post-publication review.

There are two culprits: the tremendous volume of papers, and the increasing demands on the time of reviewers.  The first is driven by a mindset that every grant must yield papers–and in some circles, that is, every grant must yield at least a paper a year.  Incrementalism drives identification of the least publishable unit.  Toss in a growing trail of reviews as papers pinball down from the most prestigious journals to less desirable destinations and you seem to have an unending stream of requests for review.  The final straw is the decreasing availability of funds, which paradoxically drives an increasing number of proposals which, once again, demand review.

On the flip side are the demands on reviewers’ time.  First, these are the same people writing all those proposals and papers, and that takes a lot of time. But as universities have tightened their belts, more menial tasks are foisted on the faculty, from dealing with routine administrative paperwork to emptying their trash cans. Also, there is more pressure for public outreach, which takes time.  Not to mention the allure of social media like Twitter and Facebook and (um) blogs. (GG views blogging as recreational, FWIW).

The obvious solutions are unlikely. Odds are low that more money will soon fill the coffers at NSF, or that outreach components for grants will be reduced. Once a university operates without trash collection in offices, it is unlikely to restore it when it can instead invest in amenities to attract the shrinking number of undergraduates.

There is only one knob really in our control, and it is our expectations of what our colleagues should be doing. Ask “what impact has your science had?” rather than “how many papers did you publish last year?” and maybe we could stem the tide.

Of course, that might require reading your colleagues’ papers.  Which (sigh) takes time….

Tags: ,

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: