Another day, another piece in the scientific literature arguing that we need to stop counting publications and instead focus on quality-a noble goal. The latest, in Nature, is focused more on the biomedical research literature, but the recommendations sound familiar:
Politicians must understand that job creation is not — and furthermore, should not be — a primary goal of the NIH or any other science-funding agency. Funds should be distributed on the basis of merit alone, not geopolitical considerations and interests. Institutions need to realign their mentality with their original academic mission, and reduce soft-money positions. Publishers should care less about publishing flashy stories and more about disseminating solid science. Individual scientists should emphasize excellence and rigour over stockpiling more and more papers and grants.
Quite the wish list. This paper is kind of interesting for two reasons. One is it does make you ask, just who is counting? And two, it claims to identify the moment that the scientific endeavor went off the rails.
OK, so who counts, and what do they count? The most impactful counters are faculty promotion committees and the external writers on tenure cases. If they don’t care, gratuitous publication would stop. They mainly count publications (and to a lesser extent, citations) but also count grants and/or grant dollars. And the question then is, why?
This is where this particular paper gets interesting, as it suggests there was a major turning point for NIH-funded fields:
Between 1998 and 2003, the annual budget of the US National Institutes of Health (NIH) almost doubled, soaring from US$13.7 billion to $27.2 billion. This applauded increase was supposed to reward good work and reduce the time scientists spent applying for grants, but it had unintended consequences.
The infusion of funds increased the scientific workforce and the number of products marketed to scientists, but did little to boost excellence. It brought about a ‘quantity, not quality’ approach that is the antithesis of what science should be about. It fuelled an idea that academic science is a business that has to sustain constant yearly growth. This ‘businessization’ of science, with its emphasis on job creation and translatability, is undermining the freedom of ideas that allows huge, often unpredictable, progress.
The paper goes on to note the feeding frenzy that followed, with institutions expanding to try to capture some of this money and numbers of students, postdocs and faculty increasing as well, returning grant success rates to pre-increase levels or lower. (Other science funding agencies did not see an equivalent increase at this time, so there is a great natural experiment sitting out there for some social scientists). Basically, the author argues that pursuit of the almighty dollar destroyed research.
(Now, the author is being a bit naive here, as others as far back as 1965 argued that science was being diluted by becoming a comfortable profession due to government support. And GG has groused about publication counting and grantsmanship and growth of research institutions and numbers of scientists. So this isn’t as new as the author represents.)
Just why do those academic groups count pubs and dollars? Well, in part because setting up a science lab can cost a lot of money; universities that pony up multimillion dollar startup packages are looking to get that money back from overhead that is brought in by researchers and tuition dollars of graduate students. Those funds then support the startup for the next new hire, and so on. Breaking this cycle would change how laboratory science is done (especially in biomed). Similarly, grant agencies are scrutinized by Congress and they have to go before congressional committees to argue for their budgets. Sharing the results of a blue-sky science breakthrough are difficulty enough to make to an academic audience; for a roomful of politicians, more tractable measures are desirable. Basically, if you have money coming from the government, it will be hard to avoid some crappy proxies for success.
What of the solution suggested? Would politicians accept spending money that might only flow to a few congressional districts? Where it might not produce jobs? And where it might not produce any results whatsoever? Where it might not produce a workforce useful to the broader economy? How would we pare back overgrown science departments at unworthy institutions? Who determines the unworthy institutions?
Yes, we need to move away from the orgy of incremental science that surrounds us, but it isn’t clear that the path advocated is a workable one. We can and should fix the metrics used within academia that incentivize faculty in a bad way, but the money side of things is going to continue to distort science.