Is Peer Review Better Anonymous or Signed?

A line in a lengthy post considering PubPeer (where we shall not go at the moment–PubPeer seems not to have gotten much of a foothold in the corner of earth science GG plays in) got GG a bit grumpy:

Those who critique science need to be protected, otherwise they will never be able to speak freely about a paper[‘]s problems. This is why scientific peer review is conducted anonymously, and why any website attempting post publication review also needs to guarantee the anonymity of its user base.

Actually, in earth science peer review anonymity is optional (Science and Nature are journals that make it very hard not to be anonymous).  More to the point, GG has always signed reviews. It would seem then that GG is a bit of an idiot.

So why sign reviews?  Here’s one reason: so people know when you were the one who suggested rejection of their paper (and yes, GG has signed such reviews).  Authors have the habit of trying to guess who wrote reviews; GG’s experience as an associate editor suggests that such guesses are typically wide of the mark, even for experienced senior scientists. By consistently not being anonymous, GG doesn’t take the blame for stuff he didn’t write. And you should take the blame for stuff you wrote that was bad.

Second, anonymity breeds contempt. Flame wars on websites make this obvious, but it extends into professional work as well. A review should be written as though you were on the opposite side of a table from the author, explaining what is unclear, what is demonstrably wrong, etc.  You are criticizing the work, not the worker, and there should be a common goal of making the work the best it can be. With only the very rarest of exceptions, this in the end improves the work the author is seeking to publish. Also, it means you have to be careful not to be flippant: if you sound off about something and the author shows you were fantasizing about how you think things should be, you look like an idiot to both the editor and the author. That extra sense of possible mortification can mean that you will construct your review more carefully.

Third, this blunts ad hominum attacks by the author in responding.  Occasionally GG has seen authors respond with vitriol to a review, claiming profound ignorance of the anonymous reviewer, utterly unaware that they are demonstrating a greater ignorance than the reviewer, who is often an equally accomplished scientist with strong grounding in the questions being discussed.  Knowing who was writing would force the author to be more contrite and careful in rebutting arguments.

Fourth, this provides the author an opportunity to note a potential conflict that the AE might have not known about.  GG has yet to see such an issue, but it isn’t impossible.

Fifth, the author will often acknowledge a reviewer by name if known.  This makes reviewers somewhat more responsible participants in the publication process (and also provide readers with some additional information as to how thoroughly a paper might have been reviewed).

The usual reason presented for protection by anonymity is to prevent retaliation.  For tenured professors, this is ridiculous; the only position where realistically somebody can cause you serious damage (provided they aren’t homicidal maniacs) is if they are a government grants officer (e.g., NSF program manager). Even if somebody really hates you and writes evil reviews on all your proposals, you know what? The panels or program officers evaluating those reviews will learn very quickly to discount them. Admittedly there is a greater risk for junior faculty and students, but there is also a greater upside that usually gets overlooked.  A careful and thorough review reflects a careful and thorough scientist; if your review is well done and persuasive, the author is apt to note who you are in a good way.  Now there are awards for reviewing that are ways for some individuals to gain recognition while preserving their anonymity on the particular review they wrote, but these are more impersonal and less apt to really make a strong impression.

Anonymity might bring out the full extent of the disdain of a reviewer for a piece of work, but it isn’t clear to GG that this is actually the best means of obtaining the desired results.  Maybe another day we shall revisit just what reviewing should and shouldn’t be.

Tags: , ,

14 responses to “Is Peer Review Better Anonymous or Signed?”

  1. Oregonians in Grenoble says :

    I know a few people who sign all their reviews, but not many. The fact is that most people take even the most honest and constructive criticism very poorly. Authors may benefit from negative but constructive reviews, and they may acknowledge that fact to themselves, but such reviews rarely result in the author’s gaining a favorable opinion of the reviewer.

    Someone who really hates you and wants to sabotage your grant proposals is probably smart enough to write reviews that are critical without being malevolent. One “good” rating is enough to doom just about any NSF proposal. These days, it’s hard even to survive “very good.”

    You’re a brave man.

    Like

    • cjonescu says :

      Brave? No. Stupid? Perhaps. Nobody likes criticism when they get it; it isn’t how they take it when they get it, it is how they deal with it by the time the process is done. GG has appreciated a “reject” review more than a “publish as is” review because the first made the paper better, while the second didn’t (yes, both were on the same paper). That doesn’t mean there wasn’t some serious eye-rolling when the “reject” review rolled in. It is kind of sad that most of us presume the worst of our fellow scientists (and even sadder that sometimes that presumption is justified).
      As for an NSF proposal being doomed by a good, well, it depends. In the panel GG participated in, a review that stood out as unusually severe in its judgement often was discarded (and we never were working off of the label the reviewer assigned but instead the material in the review itself). Someone who truly hates a fellow scientist over a review is probably sufficiently petty to show their true colors in several ways. And someone that petty is apt to try to guess an anonymous reviewer’s identity (especially if, in being anonymous, the reviewer was unnecessarily rude or pretentious)–but usually they guess wrong and then it is somebody else who gets flamed without having done anything. GG will take the heat for what he did and isn’t interested in either having his comments injure others or have others’ comments injure him.

      Like

  2. Terry McGlynn says :

    Signed reviews that do not recommend publication are rarely, if ever, in the interest of junior scientists.

    http://smallpondscience.com/2014/03/26/why-i-prefer-anonymous-peer-reviews/

    Like

  3. Israel Vaughn says :

    As a graduate student who has reviewed papers, this advice is not very thoughtful. Yes, if you have tenure perhaps it is fine, but it can be a very bad idea for junior scientists. Just being human precludes you to irrational bias, that you may not even be aware of.

    In my narrow field, if I give a senior research a less than favorable review, then it will likely affect my future grant prospects, my future post-doc prospects, and my future tenure prospects.

    Like

    • cjonescu says :

      Well, GG signed negative reviews as a grad student and as a non-tenure track researcher and survived to be a tenured professor. So while the caution is understandable (risking one’s career is obviously a hard risk to accept), there is some evidence that in earth science, at least, it is possible to accept this risk.

      Like

  4. Anthony C. Tweedale says :

    As to argument #4: “…an opportunity to note a potential conflict that the AE might have not known about. GG has yet to see such an issue.” :
    Here it is:
    my Commentary (to a high quality j, ‘Env Health’ on chemical risk assessment’s failure was recommended to be stripped of all major arguments (in my view) by an reviewer, who because he signed it, I discovered had extensive industry support for his research (in all pre-market RA, industry performs the tests on the safe dose is determined; and they use very insensitive test methods). ‘Twas later accepted by Elsevier’s Env Res. (also w/ respected editors) without such evisceration,

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: