The Nature Index stuff made GG wonder just how highly cited are the best geoscience papers from those prestigious journals? And how do they stand up to some other journals? So here are some results from Web of Science.
For ease of calculation, we’ll just look at the journals that were all earth science, and let’s limit things to since 1960. So of the prestigious journals in the Nature list, here are the number of citations of the top three papers:
- Earth and Planetary Science Letters (1966 start): 6600, 5835, 2482
- Geochimica et Cosmochimica Acta: 8167, 3035, 2348
- Geology (1972 start) 1562, 1158, 1113
- Geophysical Research Letters (1974 start) 2342, 2241, 1239
- Journal of Geophysical Research: Solid Earth (1991; 1985-1991 Solid Earth and Planets, before that just JGR B) 2583, 2529, 2516.
- Nature Geoscience (2008 start) 1374, 1017, 931
The four bold faced citations are those above 3000. Now here are some other reasonably prominent geoscience journals with their top 3 citations.
- Applied Geochemistry (since 1987) 3593, 776, 692
- Bulletin of the Seismological Society of America 3237, 2388, 1898
- Chemical Geology (since 1968 WoS) 5846, 2639, 2087
- Contributions to Mineralogy and Petrology (since 1969 in WoS) 2210, 2090, 2042
- GSA Bulletin 2143, 1560, 1246
- Geophysical Journal RAS (to 1987)/GJI (1989-) 3454, 2763, 1910
- Journal of Geology 2105, 1905, 1850
- Lithos (1975 start in WoS) 1226, 1147, 976
- Tectonics (1981 start) 1263, 1021, 785
- Pure and Applied Geophysics (1964 start in WoS) 2342, 610, 599
Several of the most highly cited papers are in review journals not listed here (which are quite prestigious and often the good review papers carry more than just a review). But looking at this list it is hard to say that this second list is really all that different in producing extremely highly cited papers, and you could argue that this list might be just as important a set of journals as that used in Nature Index. Even a journal as uneven as Tectonophysics occasionally has a gem in it (1756 citations) and specialty journals like Quaternary Research (2253) and Precambrian Research (1267) often produce influential papers. Even some of the new electronic journals (G^3, Geosphere) have some well cited papers despite starting this century.
The message? Prestige is earned by what you say, not where you say it.
And now, for your enjoyment of the ability to place your institution above others, we introduce yet another metric! (Applause, hosannas, people falling to the floor in ecstasy). And so, as is usually the case, the promotion people at the relevant universities (like GG’s) push any favorable numbers out the door, like here.
The new metric? Well, OK, technically it is now 4 years old but it seems to have gained some prominence with a recent modification: Nature Index. And just what does it measure? It is simply counting the number of articles in a subset of “prestigious” journals over the past year affiliated with institutions. Which journals are prestigious? You wouldn’t be wrong to say Nature journals, many of which make the cut. In earth and environmental science (where CU ranked in the top 10, much to the pleasure of the university’s promotors) the list is:
Awhile back GG groused about why journals continue to make electronic versions of their publications look exactly like the paper copies. Tiny strides are made from time to time (Geosphere, the journal GG has worked with the most, changed its layout awhile back from portrait to landscape and got rid of the awful, awful, AWFUL practice of having material on a single page split between stuff in both landscape and portrait orientations), but by and large materials remain static images. While GG has been focused on trying to get things to work within the current structure of pdf files (which do allow some interactivity), others over the years have advocated totally different means of distributing science. For instance, Jon Claerbout years ago advocated for the “reproducible paper” (which we previously discussed when considering issues with the “geophysics paper of the future”).
This brings us to a new Atlantic article titled The Scientific Paper is Obsolete, which outlines two major efforts to get around the limitations of paper by making a totally new format. As spelled out in the article, it is like the battle between Apple and Google over phone operating systems, or probably more accurately like Windows and Linux battling over how to make a desktop operating system. The article’s author, James Somers, adopts Eric Raymond’s earlier characterization of desktop OS strategies as a battle between cathedral builders versus the bazaar. Heading Team Cathedral is Stephen Wolfram and Mathematica, where a replacement for the scientific paper would be a nicely prepared Mathematica notebook. Team Bazaar is all open-source, having elevated Python to the point of a new system termed Jupyter which also makes notebooks. [Oddly missing from the article is any mention of Matlab, which shares many of the same traits with Mathematica and is far more popular in engineering and much of earth science].
Why hasn’t one or both of these taken over the world of science? In the article, Bret Victor is quoted as saying it’s because this is like the development of the printing press, which merely reproduced the old way books looked for quite awhile until newer formats were recognized as better and adopted. Sorry, but he is wrong. This is like the invention of paper. And this is why there is so much uncertainty about adopting these technologies.
GG has pretty adamantly argued that you shouldn’t be measuring a scientist’s worth from where she publishes and thus rewards based on journal metrics are misguided and ethically wrong. So it was gratifying in one sense to see Sylvia McLain’s post on publishing in lower impact journals. But then GG read the comments, which included these:
I value what you say here and I thank you for expressing this important perspective. I, too, made similar choices early in my faculty career. I had several very high impact publications from my PhD and postdoc years, but during my time as an assistant professor I focused on just putting out good solid science…The tenure committee called this a “downward career trajectory” and sent me packing.
Sure, people in your specific subfield can appreciate high quality j Chem Phys, but everyone else only has brand name to go on. If you are expecting to look [for?] a job some day, high profile journal articles are pretty much the only ones that count.
Low IF journals are not going to get you tenure, so I sure hope assistant professors don’t take this advice to heart.
Indeed there might be gems in low impact journals and those profs might be excellent scientists, but why not settle for excellent work published at high impact journals?
Is it true there are places where “everyone else only has brand name to go on” and so decides on tenure based on the journal’s impact factor? Really? People are sent packing solely because of where they publish? GG is beyond appalled and only hopes this misrepresents what some schools are doing.
Look, ideally tenure should reflect the impact a faculty member has had on their community. That should be measured by what the leaders in the field say along with an evaluation by the faculty of what the main contributions are and what influence they have had. This requires departments to read the candidate’s work and to solicit useful reference letters (i.e., ignoring ones that simply count publications or citation indices). While citation numbers and publication history can provide some information, it should be tiny and used more to better understand where the candidate’s work resides. GG feels that this is what we look at in our department at CU Boulder, so this isn’t pie-in-the-sky.
That said, it appears that some places are blinded by the editorial whims of the tabloid journals (the greatest barrier to publication in Science or Nature is not peer review or the quality of the work, it is having enough of a hook so that the discipline editor can sell the paper to the main editorial group–and this might depend on what the journal has recently published or just how many news stories some other work got). Read More…
The noble case being made for such services is that they can allow for broader peer-review, they can avoid embargoing good science by evil journal gatekeepers, and they can accelerate the pace of science. All three are misguided, at least in earth science. Many if not most earth science articles are unlikely to attract much attention (most academia.edu earth science papers are unread, let alone “reviewed”). Delays in publication are as much the difficulty in getting reviewers as anything else; outright obstruction can be avoided by going to another journal (there are quite a few) or complaining to a society president. And arguably the pace of science is a bit too fast, judging by the sloppy citations of the literature and piecemeal publications from some corners of the field.
If not for noble reasons, what is pushing this? It appears part of this is a desire by some to get something “citable” as soon as possible–for instance, Nature Geoscience editorialized “In an academic world structured around short grant cycles and temporary research positions whose procurement depends on a scholarly track record, there is room for a parallel route for disseminating the latest science findings that is more agile, but in turn less rigorously quality controlled.” [This is hilarious coming from a publisher whose lead journal actively quashes public or even professional discussion of papers prior to publication].
Let’s be clear: this is a crappy excuse that opens the door to “fake science.”
Sorry, had to point this out. Anderson Cooper was here at CU and said that…
…he doesn’t know of another profession that tries as hard to “get it right,” and correct itself when it makes mistakes.
(This according to a Daily Camera story, which we know must be right because they are journalists and try to “get it right”….)
Oooh, ooh, let GG try to see if there might be other such professions!
Um, science, maybe? ‘Cause that is kind of the definition in many ways.
Probably a lot of engineering (hey, let’s rebuild the Tacoma Narrows bridge the same way!).
Bet airline pilots try hard to get it right.
How about rocket science? When was the last time NASA blew up a rocket because of frozen O-rings? Just once, right?
Sure, there are lots of fields where the same mistake happens over and over (can we say “food poisoning” and “Chipotle” and find them together in news stories from multiple years?). But you do sometimes wonder how much journalists actually notice about occupations other than politician and commentator….
GG was a bit disturbed in reading an Atlantic story describing how PLOS Biology and a few other journals were relaxing the rules on publishing papers that were scooped by rivals publishing the same results earlier (once again, pointed out by Retraction Watch). Responses from scientists as quoted in the article included “This changes EVERYTHING” and “This will change the career trajectory for many a disillusioned scientist.”
Really? What is going on?
It isn’t that allowing for parallel studies to make publication is bad, what bothered GG was the perception that this is righting an awful wrong that destroys many careers–that this is a BIG DEAL. From the way this is covered, you get the feeling that a lot of scientists are doing the exact same experiments with the exact same goals, but only the first to the finish line gets a publication. Now everybody gets a participation ribbon, right? If this truly reflects the state of biology these days, GG wonders if we have too many biologists. When considered in tandem with the “replication crisis” that also seems most focused in the biosciences along with the substantial number of retractions and fraud in that field, it feels like we are staring at an entire discipline in crisis, but not for the reasons they perceive.
It’s not like other fields don’t see work getting scooped, but it is far too rare to merit the kinds of plaudits this PLOS Biology practice garnered (In fact, lines of research in earth science are often distinct enough that truly being scooped so as to make a study unpublishable inspires questions of academic theft). So GG has to wonder if (1) certain lines of research are simply far overpopulated with researchers who (2) rush to get incremental results for publication, and in that rush (3) the least careful (or (4) most dishonest) are the most rewarded, which means that a significant fraction of new results are, in fact (5) non-reproducible. Now maybe GG misunderstands biology (well, OK, no “maybe”)–perhaps the bar on being novel is much higher in bioscience such that a paper reaching the same result from a different pathway becomes unpublishable. This is not the case in earth science: a paper determining, say, the slip history of an earthquake from seismological observations won’t preclude another paper getting the slip history from geodesy–or even other flavors of seismological data. Yes, you may have trouble getting your later geodesy paper in Science or Nature, but there won’t be barriers in discipline journals.
GG recently was reading the GSA Presidential Address of M. King Hubbert from 1963, and his complaint (he was an earlier grumpy geoscientist) feels relevant:
Within the university it is easily seen that such a system [of “publish or perish”] strongly favors the opportunist capable of grinding out scientific trivialities in large numbers, as opposed to the true scholar working on difficult and important problems whose solutions may require concentrated efforts extending over years or even decades. It took Kepler, working on the lifetime of astronomical observations of Tycho Brahe, 19 years to solve the puzzle of planetary motions, but the results were the now celebrated Keplerian Laws of Planetary Motion. Newton, with few intervening scientific publications, spent altogether some 20 years studying the mechanics of moving bodies before writing his great treatise Philosophiae Naturalis Principia Mathematica (1686) in which are derived the Newtonian Laws of Motion and the Law of Universal Gravitation. Twenty-two years of work, the last 11 essentially free of other writings, preceded Charles Darwin’s publication of On the Origin of Species by Means of Natural Selection (1859). How long could any of these men survive in an American university of today?*
If everyone is pursuing the same results with the same techniques over and over and over, it speaks to an absence of imagination, an unwillingness to explore, a pressure to indulge in groupthink–in essence, a desire to “grind out scientific trivialities in large numbers.” What all this noise and fury truly represents is not a terrible wrong committed against nobel scientists, but a gross corruption of the goals of science and a massively unhelpful and misaligned reward structure. Letting the late comers publish is hardly the big solution the Atlantic article claims; it is, in fact, helping to hide the main problem by rewarding the losers in a race that never should have been run.