A thread on an AGU mail group lately has gone back and forth on whether peer-review of proposals by U.S. federal agencies is fair. Some have asserted that retribution exists in the system, but many of those who have participated have argued it is about as fair as any other activity involving humans, downplaying the possibility of massive collusion to punish an individual. It would not surprise GG if on a few occasions some kind of retribution tipped the scales against a proposal, but it is far more likely in most cases that a combination of other factors doomed a proposal. What emerged in this thread was an interesting thought, namely the reemergence of the idea of a double blind (or at least single blind) review system.
One fundamental premise, as noted by one writer, is “past performance is no indication of future success.” Basically, somebody who has generated something good might well lay an egg, while somebody whose last project failed could be on to something good. There are two issues here GG would like to contemplate: what does it mean to “succeed” and “fail,” and what components of an individual’s scientific reputation might be relevant.
First, failure is always possible. In trying to gain knowledge previously inaccessible to humanity, a scientist is venturing into the unknown. Things not going as planned is not particularly unusual. But what does it mean to fail? Read More…
GG was a bit disturbed in reading an Atlantic story describing how PLOS Biology and a few other journals were relaxing the rules on publishing papers that were scooped by rivals publishing the same results earlier (once again, pointed out by Retraction Watch). Responses from scientists as quoted in the article included “This changes EVERYTHING” and “This will change the career trajectory for many a disillusioned scientist.”
Really? What is going on?
It isn’t that allowing for parallel studies to make publication is bad, what bothered GG was the perception that this is righting an awful wrong that destroys many careers–that this is a BIG DEAL. From the way this is covered, you get the feeling that a lot of scientists are doing the exact same experiments with the exact same goals, but only the first to the finish line gets a publication. Now everybody gets a participation ribbon, right? If this truly reflects the state of biology these days, GG wonders if we have too many biologists. When considered in tandem with the “replication crisis” that also seems most focused in the biosciences along with the substantial number of retractions and fraud in that field, it feels like we are staring at an entire discipline in crisis, but not for the reasons they perceive.
It’s not like other fields don’t see work getting scooped, but it is far too rare to merit the kinds of plaudits this PLOS Biology practice garnered (In fact, lines of research in earth science are often distinct enough that truly being scooped so as to make a study unpublishable inspires questions of academic theft). So GG has to wonder if (1) certain lines of research are simply far overpopulated with researchers who (2) rush to get incremental results for publication, and in that rush (3) the least careful (or (4) most dishonest) are the most rewarded, which means that a significant fraction of new results are, in fact (5) non-reproducible. Now maybe GG misunderstands biology (well, OK, no “maybe”)–perhaps the bar on being novel is much higher in bioscience such that a paper reaching the same result from a different pathway becomes unpublishable. This is not the case in earth science: a paper determining, say, the slip history of an earthquake from seismological observations won’t preclude another paper getting the slip history from geodesy–or even other flavors of seismological data. Yes, you may have trouble getting your later geodesy paper in Science or Nature, but there won’t be barriers in discipline journals.
GG recently was reading the GSA Presidential Address of M. King Hubbert from 1963, and his complaint (he was an earlier grumpy geoscientist) feels relevant:
Within the university it is easily seen that such a system [of “publish or perish”] strongly favors the opportunist capable of grinding out scientific trivialities in large numbers, as opposed to the true scholar working on difficult and important problems whose solutions may require concentrated efforts extending over years or even decades. It took Kepler, working on the lifetime of astronomical observations of Tycho Brahe, 19 years to solve the puzzle of planetary motions, but the results were the now celebrated Keplerian Laws of Planetary Motion. Newton, with few intervening scientific publications, spent altogether some 20 years studying the mechanics of moving bodies before writing his great treatise Philosophiae Naturalis Principia Mathematica (1686) in which are derived the Newtonian Laws of Motion and the Law of Universal Gravitation. Twenty-two years of work, the last 11 essentially free of other writings, preceded Charles Darwin’s publication of On the Origin of Species by Means of Natural Selection (1859). How long could any of these men survive in an American university of today?*
If everyone is pursuing the same results with the same techniques over and over and over, it speaks to an absence of imagination, an unwillingness to explore, a pressure to indulge in groupthink–in essence, a desire to “grind out scientific trivialities in large numbers.” What all this noise and fury truly represents is not a terrible wrong committed against nobel scientists, but a gross corruption of the goals of science and a massively unhelpful and misaligned reward structure. Letting the late comers publish is hardly the big solution the Atlantic article claims; it is, in fact, helping to hide the main problem by rewarding the losers in a race that never should have been run.
*-An aside. Of course, if we gave all faculty 20 years to publish something, for every Darwin or Newton, we’d have hundreds of deadwood research faculty. Hubbert was in fact arguing that research as supported by the government should be moved out of the university.
Perhaps the most bizarre aspect of the migration to electronic media is continued use of one of the most annoying aspects of the original print-electronic split, namely the electronic supplement. Originally intended as a way to keep things like data tables from overwhelming print copies of journals, this appendage to publications has become more a scourge than a help.
The first attempt in the geosciences to do something like this was the ill-fated split of GSA Bulletin papers into a paper version and a microfiche version (ick). This actually devastated the journal as authors fled to safer harbors. As more and more papers are being accepted with small stubs as the “article” and the main text as the “supplement”, journals seem hellbent on replaying GSA’s old blunder. The only difference is that this time, there is no real excuse for doing so.
Basically, at this point, why would any supplement that is either text or pdf format be excluded from being part of the electronic version of a paper? These are often no larger files than the “main” article, and increasingly these parts of papers are where the real nuts and bolts are that you need to really evaluate a paper.
There might be reason to keep supplements for non-readable materials (binary files, mainly), but it is time to bury them for stuff that is human-readable. Call these things appendices, put them at the end of the paper, and include them directly when you go to download the article. Enough of this business of realizing later, after you’ve pulled down the article and started to go through it thoroughly, that the guts are still sitting somewhere online. Worse, all too often the outline of the logic is in the main paper while the supplement contains the pieces of the body, often tossed together in a disorganized heap that was spared any copy editing. Reassembling the true logic of the scientific work is harder than it would have been had the paper been written as (gasp!) a single, long paper.
Why are we continuing to allow paper journal formats to mangle our science?
A good quick primer on how different Americans view the Trump Presidency is a summary of year-end summaries at electoral-vote.com. Lots of things get mentioned (nobody is fond of his Twitter feed, but otherwise views are all over the place), but one that doesn’t is science. Even if you drill into the rather lengthy reviews (e.g., Politico’s very long one), science only shows up as a sidelight to the decision to withdraw from the Paris climate accord and is absent entirely from several others. So where does science stand?
An op-ed in the New York Times by a former science advisor tells us that the guy in the picture below (whom we saw at the March for Science in Denver) might just as well be the real deal: there is still no science advisor. And that is a bellwether for the whole of federal government.
It is worth recalling that the science advisor does not make policy and does not administer anything; the advisor provides advice on science that might be relevant to government activities. And there are lots of things that really are not controversial where having insight from scientific work might be useful. And even the controversial stuff, it is just advice. Until now, having an advisor has not been controversial, but the consistent, seeming across-the-board, attempt to flush science from any role within government is not likely to make for more effective government.
Nor does it generally please scientists. An intriguing question is, will this purge also prove to be bad politics? There are plenty of conservative scientists out there (in the oil industry, in the national labs, in the defense industry, and there are a significant number in academia, too). Will this repeated snub for them have an impact? We are not talking about large numbers of votes (this isn’t like improving or suppressing minority turnout by a few percentage points, which can turn elections). But we are talking about a community that is, still, by and large viewed in a positive light by most Americans. Might their voices be influential? A number of scientists–people who by and large walked away from popularity contests long ago–are volunteering to run for office in response to the dismissal of science at the national level. There are enough that a group modeled after Emily’s List (314 Action) was formed to help scientists considering entering politics as candidates. Odds are, most are not entering as Republicans, even if that had been their inclination in the past.
Science was proud to stand apart from the political fray, ready to help any legislators understand the scientific dimensions of policy issues. That behavior may well be over.
“Publish or perish.” So goes the mantra supposedly defining academic life. The implication is that publication is enough. We can all agree that unpublished science is not going to have much of a chance of influencing the course of scientific thought, but is the act of publication really enough?
There are a number of instances of science rediscovering things. Perhaps one of the more familiar stories is that of Gregor Mendel, whose work, goes the story, went unread by Darwin and unnoticed by others, leaving Mendel to die in obscurity, his work only appreciated years later when it was rediscovered. While this story is largely wrong, it does resemble some lesser known cases. That blog post listing some of the rediscoveries concludes by saying, in essence, this won’t happen in the future. Our literature discovery tools are too good.
Nonsense. (At least for now). Read More…
GG has been rather abrupt in some previous posts with some authors over choices made in making maps for publication, and some of the authors have been insulted, in part because GG seems to have mistaken a conscious choice for laziness or thoughtlessness. Its not like GG is free and clear on making mistakes, so today let’s review a few boo-boos of GG.
First up is one we discussed before, the use of gradient color:
GG was the thesis advisor and second author here, and there are two main problems. One is the rainbow color map, which is widely panned for the challenges it places before those with some level of color blindness. [Alternate color schemes are nicely described here and you can test how images look to the color-blind here. The only good news for this figure is that a continuum of color is more decipherable than separated color blobs]. The other problem is that the colors are continuous. Can you tell -2.2 from -2.6? This probably would have been better with discrete color steps. But that’s probably not the complete answer:
This is from Jones et al., 1994, showing travel time residuals with a discrete color bar. On the left is the original, on the right as viewed by a red-blind viewer (as created by Coblis). While the Levandowski et al. image could be deciphered because of smoothly continuous colors, this figure becomes hopeless. (Plate 2 in Jones et al., 1994, is even worse, but GG can’t recover the original file to show this cleanly, and plate 3 is an utter disaster for red-green colorblind). There are similar examples in later tomography papers. Frankly, using some of the better diverging color maps out there would be a better practice.
In looking at the little advertisements (“press releases”) for newsworthy new science that is the website SciTechDaily, GG found this stunning assertion:
First-of-Its Kind Seismic Study Challenges Concepts of Geology
Wow! A first-of-its-kind study and challenging some unnamed concepts of geology. Not every day that happens. What was more, the study was authored by well-respected scientists like Vadim Levin, who was quoted in the puff piece saying “The upwelling we detected is like a hot air balloon, and we infer that something is rising up through the deeper part of our planet under New England.”
Frankly, this is a case of university promotion run amok, and Vadim has to take at least partial ownership.
First, the study is hardly the first of its kind. It compares tomographic wave speeds with measurements of shear-wave splitting, stuff that has been done now for decades. What is new are some SKS splitting measurements from some sites that hadn’t been included in previous regional studies. The splitting magnitudes were small, suggesting that the regionally present transverse [horizontal] anisotropy was damped or reoriented in this region. Yet we get quotes from Vadim (who certainly should know better) like this: “Our study challenges the established notion of how the continents on which we live behave.”
Oh, be real. This study is not about to rewrite the textbooks despite Levin’s statement that “It challenges the textbook concepts taught in introductory geology classes.”
Look, the paper is perfectly fine. But it was not the work that originated the idea that this body under New England was a convective upwelling; in fact, those papers don’t challenge any notion about continents, instead suggesting that the trailing edges of continents might generate convective motions in the mantle. (Vadim was a coauthor on at least one of these papers published a year ago).
Clearly the hype with the press release is way out of proportion to the significance of the paper. This is not how we should be promoting science; in fact, it is just the kind of press release that can torque other workers in the field. GG’s view is that scientists need to control their message–not only in their papers but in the press releases they contribute to.
As an aside, how believable is this interpretation? Read More…