Blinded by information

An interesting summary of ideas on what information should be hidden from reviewers or other scientists is at STAT. Among the suggestions is to withhold author names from editors and/or published papers and withholding journal names from cvs.  The justifications are to try to remove biases that work against some segments of the scientific community, and there is a lot to be said in favor of that. Now the particular discussion online is aimed more at the medical/biological science, so there are some differences for earth science. Feeling, as usual, a bit contrary, and having argued against anonymity in the past, GG would like to ask, what would we lose?  Are there positive uses of this information?

First, the only thing editors might lose if they were unaware of the identities of authors of a paper might be wariness for plagiarism or fraud and a decreased ability to identify reviewers unconnected to the authors.  So perhaps some awkward moments might emerge (imagine if the editors asked an author to review the paper); these perhaps could be mitigated by some automated system.

What if papers were published without author names? You have to wonder just how long it would take for somebody to take credit on their internal university report of activities for a paper that wasn’t theirs. Perhaps journals could issue certificates of authenticity. But how about for the consumers of this work?

Its hard to imagine anonymity surviving within a community.  You see somebody give a talk, they show the same material as a paper, their talk has the same title, etc. So this might be wishful thinking anyways. But what do we gain from author lists on papers?

Here GG suspects personal behavior differing from others.  GG will look over papers from particular authors with a history of advancing interesting ideas (e.g., Brian Wernicke or Gene Humphreys) even if well away from GG’s ongoing research interests; if author names were missing, these might be overlooked.  But when looking for papers providing material necessary for a research project, it is the material and not the authors GG hunts for. [Of course, previous familiarity with works by those certain authors is likely to bias reference hunts]. Authorship can also inform us of an intellectual lineage; there is less point in citing five papers from the same research group than citing five papers from separate research groups to support some point. If we didn’t know authors, we might find ourselves occasionally thinking we were relying on independent lines of logic when that might not be true.

How about citations without author names?  This might hold some real promise (it would be a whole lot easier to criticize your own previous paper if you didn’t have to write that your own name after “earlier work was botched by”) but probably doesn’t get you very far for earth science publications.  In Science and Nature, where you have to go to the end to see the citation anyways, this is probably OK, but a lot of us like the (Author, year) style of citation in text–we often know what that paper is, for good or ill, and so this saves time flipping to the references section. Its not as likely that we’ll know a paper by its DOI (arguably the most likely thing you’d replace author, year with) and going to a footnote style at the bottom of a page ala most history books probably won’t enthuse anybody.

Perhaps the oddest suggestion was removing journal names either from citation lists or cvs. GG has a lot of sympathy here: ideally your choice of journal should just reflect the audience that journal serves, not be a proxy for the significance of your science.  But a lot of folks mistakenly equate a journal’s (perceived) prestige with the quality of the science, so removing this information in publication lists could make more sense than it initially seems (though the DOI will effectively contain the information). Frankly, there are a lot of bad Science papers and a lot of good JGR papers (and there are even decent Tectonophysics papers); these journals serve different purposes. Putting a new seismological processing algorithm in Science or Geology would be insane; those are there wrong audiences even if your paper could fit in these letter journals. Journal choice should not be affecting the perception of significance of scientific work.

For all the good of dropping journal names, you do lose some information. Some journals allow the authors to pick the (associate) editor that handles the paper.  Others are known to have pretty lax acceptance policies; some journals are well-enough edited that you could probably accept the analysis in a paper without reproducing it yourself; conversely, some journals allow total dreck in, and you’d probably want to reconfirm results yourself.  Now for papers that are essential to the underpinning of a research project, the journal again shouldn’t matter as you should really have a pretty good feel for the substance of the work in those papers. On a cv, you’d worry if all the papers were in dreck journals (why would an author do that?), but then if you couldn’t rely on the journal names to evaluate the cv, you might do what you are supposed to do: read the bloody papers. So maybe the losses here are pretty minor.

All this is probably discussing numbers of fairies on pinheads.  Most authors aren’t eager to have their intellectual products go out anonymously; journals are unlikely to want to lose their own perceived niche. So the noble quest to try and minimize bias by scientists is likely going to have to march to a different field of battle….

Tags:

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: