Failure should be an option

What, precisely, should be the criteria that successful NSF grant applications meet?  There are a constellation of possibilities: whether the research will investigate problems of broad interest in science, or whether the results will open a new field of research, or whether the results will overturn long-held beliefs, etc.  And of course NSF has added things like educating more STEM majors at different levels.

One wonders, though, if failure is not an option at NSF.

Now, GG is well aware that one magic phrase at NSF is supposed to be “high risk, high reward.”(Another is “transformative”, unless a new catchphrase has emerged). But with a congressional committee keeping exceptionally close scrutiny on NSF, especially in earth science, you have to think that funding something that could fail–or is likely to fail–might cause a program manager to quail. So you have to wonder, how often is NSF funding things that could fail?

Feel free to wander through the list of 1888 active awards in solid earth geoscience and see how many feel like “high risk, high reward” or are likely to fail. [Good luck discerning that through the publicly-available abstracts].

Here’s the rub.  Work that challenges current views runs the risk of challenging reviewers who simply don’t think that their current understanding is wrong; these reviewers will tend to study the proposal closely and point out and magnify flaws. (This is not the reviewers being mean or Machiavellian, it is them being human.  Proposals embracing a popular or pet hypothesis simply won’t encounter the scrutiny ones attacking said hypotheses will). So what if a proposal is capable of failing, or is even likely to fail? Well, those reviewers are certain to point out that possibility and downgrade the proposal for it. It is something of the nature of scientists–we are quick to look for flaws; sometimes it becomes something of a game to see who can find the most flaws and lord over them. This is a useful (if maddening) trait when reviewing papers, but proposals should be different. Are we as good at recognizing promise of really moving forward as we are of finding nits to pick?

So, ironically, the funded proposals most likely to fail are, probably, those looking to confirm or refine current understanding.

All of this means that proposal writers, desperate to pass through the ever-narrowing gates of funding, will avoid risking the possibility of failure (or even its appearance).  Incremental science might not be NSF’s goal, but it can pass through the system far more easily than more dangerous stuff. In avoiding risk, investigators are being steered away from the problems probably most worth their time.

Long ago, GG was told that Eldridge Moores had a policy back when he edited the journal Geology; his preference (so it was told) was to publish papers producing one review recommending “accept without changes” and another recommending rejection.  Although this occasionally produced papers that, in retrospect, fully justified the “reject” recommendation, the basic idea had merit: it could be good to focus on the papers that so challenged part of the community that they felt those papers should be squashed. Perhaps this is apocryphal, but it seems worth considering.  Maybe program managers, instead of having panelist vote on a grade for a proposal, should simply time how long they want to argue about any given proposal and fund the ones producing the longest arguments.

Hey, it can’t be worse than what we have now.

Advertisements

Tags:

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: