Is science suffering from ADD?

GG is beyond grumpy, having been nixed on an NSF proposal.  This happens (all too often), but one sentence in the panel summary and a mail review is the main cause of grumpiness.  This sentence said that the single published paper from a previous grant was unacceptably poor production.

Since when was one paper (in this case, a rather thick paper that took more than three years to assemble) on a grant unacceptable? Zero is certainly unhappy, but one? Now if the reviewer or panel had read the paper and said that it was not worth the grant money, that would be fine (well, GG would argue with them, but at least there was a true evaluation of merit and not a lazy accounting a second grader could manage).

It seems that science is going the way of newspapers, TV shows and interpersonal communication: we are headed for byte-sized fragments. Papers have gone to stories that can be read aloud in a minute.  TV shows, especially some reality shows, like to show you what is coming up, have a commercial, show you what you just saw, show you a little new stuff, and then show you what is coming up again.  Interpersonal communications used to be lengthy letters, then it became phone calls or shorter emails, and of course now we are down to texts and tweets.  Does science benefit from this?

In a word, no. If anything, most scientific advances require more explanation because the stuff that might have fit into a short paper has been done. GG could illustrate how transform faults fit into plate tectonics in a couple minutes with a napkin and pen (an advance that was a significant milestone in the 1960s), but explaining the relationship between supercontinent cycles and reversals of the earth’s magnetic field might take a bit longer.  Science is an intellectual operation requiring logic, analysis and data.  Omitting some of that makes the product more an op-ed piece than a scientific contribution. This isn’t to deny that there are instances where short papers make a lot of sense, and some fields generate a fair number of papers in short time periods (e.g., earthquake seismology often has a spurt after some charismatic earthquake, like Tohuku or Sumatra as new data is rapidly collected, analyzed and shared), but by and large it takes some noticeable space to really show what new advances really are made of.

So GG bridles at the “one grant = multiple papers” sensibility.  To keep a ‘normal’ earth science research program going, we might expect to have about 3 students, and usually we are funding those students off of grants.  Usually a grant might support a single student; two years is a pretty typical number.  Right now the batting average is about .250 for NSF grants in earth science (younger PIs are favored, as are PIs from states with a poor research record (EPSCoR states), so the average must be lower for senior scientists from states with healthy research programs), so figure that to have 3 grants in place it took writing 12 grant applications over 2 years, or 6/year.  Now, if each grant is to have multiple publications, it seems you’d have to have 3 papers/year.  So you’ve now spent your year writing 9 rather significant items; that is a lot (especially if you are teaching and doing normal service activities, which, in theory, take up at least half of the year and often manage more than that).  How significant are these papers going to be? It is worth noting that most earth science is not done the way a lot of biology currently does: the researchers writing the grants are deeply involved in doing the research and writing the papers [in many biology labs but few U.S. earth science labs, there is a more severe pecking order, with the PI getting grants, the postdocs supervising the research being done by the grad students].

While scientific papers have always been progress reports in the sense that they don’t usually end all discussion on a topic, they hadn’t usually been progress reports in the sense of students telling teachers how many pages of a book they’ve read so far. Why should a research grant be producing multiple papers?  Wasn’t there some strong motivating question behind the grant application that was to be addressed? Do we need separate papers saying data was collected, and then another on its analysis, and another on its interpretation?  GG thinks not, but our short attention span culture seems to be pushing us to this tiny papers with little individual significance.

Is there a downside to a lot of little papers instead of longer comprehensive papers?  Well, yes.  Each tiny paper will need peer review, which means an associate editor (AE) and two mail reviewers will look at each one.  Say that we have three papers instead of one, this means that 9 people are being asked to look over this body of work instead of 3.  And, quite frequently, those nine will need to see the other papers in order to make sense of the fragment that they have to review.  But the number of reviewers is finite, so what happens is that it gets harder to get reviews, so those AEs will have to beat the bushes for reviewers.  And some times, the additional papers are not available to the mail reviewers, so their reviews are incomplete and flaws that could be corrected are left to fester in the literature.  What is more, usually of necessity there will be a lot of repetition of material between the papers (background, motivation, often even data collection approaches) so for somebody to really comprehend what was done, they will have to read significantly more in the little paper milieu.

So, dear NSF project managers, please rebuke your panels and mail reviewers for equating numbers with quality or success. Other than zero, numbers could be misleading….

Advertisements

Tags:

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: