Although GG has repeatedly flogged various meta-indices as lousy ways of measuring science, let’s take a moment to celebrate what it does do. In places where administrations were lazy, they used to value faculty on the number of papers in prestige journals and then the number in quality journals (um, well, guess this is still a thing in some quarters). Whether these papers had any impact was unclear. So citation indices tell us that somebody was paying attention (or not). So while the use of citations as a means of evaluating scientists is a poor substitute for really knowing what the scientist did, it is an improvement.
But there are all kinds of issues with the citation indices in use. So what would be the best citation index?
First issue is pretty obvious and one that is kind of addressed in some existing measures. That is self-citation. If you churn out enough papers citing your own work, you can look pretty good. This works really well if you have a circle of collaborators playing the same game (some journals got kicked out of Science Citation Index for this sort of thing). So dropping self-citations is a start (though how you define a self-citation needs work–to GG it should be a common author on both original and citing papers).
Second is the common complaint that you can get cited for the wrong reason–namely as an example of what not to do. This is a bit of a red herring–scientists are loathe to directly criticize others in print (even as they might badmouth them in private). A quick search of papers GG has for “wrong” turns up few cases of a specific paper being cited for being incorrect (and most of those cases are comments on the original paper). No, the actual snubbing is usually in not citing a paper that arguably is relevant because you think they made a mistake in some way. So what you really want to count are the non-citations in relevant papers. These snubs carry plausible deniability (“I didn’t cite you? Shoot, I meant to, but I forgot / the journal limits citations / it got taken out by an editor / I hadn’t seen the paper”) but realistically are telling you that the uncited paper is not so well-received after all. So a better citation index would subtract the appropriate non-citations.
Third is the reverse problem: the over-citation of some papers that are sort to community totems for certain ideas. These can be review papers or textbooks, and those kinds of publications are pretty recognizable for this thing, but it will often happen that a paper will be one everybody thinks of as the origin of some class of ideas or one that just happened to review enough material that it acts as sort of a meta-citation. The citations to such a paper are frequently gratuitous and sometimes reveal that the authors have not read the classic paper in question. These citations are often a simple placeholder in response to an editor or reviewer asking for a citation. So an even better citation index would downright or remove placeholder citations.
But at the very bottom of the problems with citation metrics is that not all citations are equal. A citation that provides the motivation or techniques central to the citing paper is far more valuable than, say, a citation for some peripheral background material. Perhaps the number of times a citation appears in a paper would give you a clue, but just a clue, as to the importance of the earlier work to the current paper. Realistically you’d have to understand the paper and the material being cited to gauge how central that citation was. The best citation index would somehow weight all the citations for that measure.
Of course, that is really what you are asking those experts providing external letters of evaluation for. So maybe we already have the best citation index–we just didn’t recognize it for what it is….
Heated rhetoric seems the tone of the day, but it is interesting to see where the battle lines get drawn.
On the national stage, nobody wants to see more gun deaths–there is no disagreement about the end goal. Homelessness? Would like to see less of that, along with poverty and malnutrition. Education? Most Americans want their kids to get the best education possible. Even something more polarizing is probably not as extreme as it seems–nobody wants there to be lots of abortions, though there is a serious disagreement on how close to zero is fair and appropriate. What there is for all these topics is a massive disconnect on the means. Usually when you agree on a goal there is room to compromise on means (say by some combination of gun purchasing checks and school security or mental health outreach). What is amazing about modern American culture is that so many of the hot-button arguments are over means and not ends.
Its not clear to a mere grumpy geophysicist why this should be. Certainly there are cases where there are some actors for whom the means are ends (gun manufacturers, say, in a gun control debate have little interest in shrinking their market; public school teacher unions are not eager to support private school vouchers). So it might not be a surprise that some will be acting to make a debate over means as polarized as possible, but why does this seem to push policy makers to the fringes of the debate? Most of us are in the middle somewhere, we all want to get going in more or less the same direction. As Rodney King said long ago (and had placed on his tombstone), can we all just get along? Most of these questions are not winner-take-all; compromise and examining the outcomes of different blends of solutions can move us toward a better world. As a scientist, a field where we like to see some experimental results to guide us, GG is frustrated by the lack of ability to focus on solving a problem rather than simply arguing over methods and getting nowhere….
That said, there are a few winner-take-all arguments out there. Most of them involve land, and one of them got GG thinking on this. Read More…
Update 4/16/19: Seems people normalize to about 8 year sliding windows according to a study of tweets. Worth noting that there is no attempt to examine whether this depends on age (are more tweets from 20-somethings and so 2-8 years is their youth? Or do grumpy old people really forget how things were?).
Science fiction stories and some prospective science fact articles suggest that much longer lifespans are just around the corner. GG wonders how things would be different now if we had natural lifespans of, say, a couple centuries. On the face of it you can imagine some fun meetings (say, Albert Einstein chatting with Stephen Hawking, or John Muir swapping stories with David Brower or Edward Abbey; seeing Harry Truman dress down Donald Trump might be entertaining). It is hard to imagine that today’s political perspectives would remain the same were Eisenhower and Marshall still around to defend NATO. But GG’s point here is one of societal perspective on what is normal. This is most clearly discussed in the context of climate and the natural environment.
Most of us tend to define as normal weather the kinds of weather we grew up with. A period of exceptionally bland weather like the late 1950s and 60s tuned a generation into thinking that weather should be pretty calm, but those people who grew up a few decades earlier saw far more extreme weather events. The result of baby-boomer misapprehension in some cases was to underestimate the chances of major floods or droughts, oversights that have proven costly to correct. Would having living voices from different periods recalibrate how society views weather and climate? Would having, say, a 200 year old resident of an area telling of how winters were in the 1890s and 1880s when blizzards could isolate communities for months affect younger people enough to realize that things really have changed a lot?
Perhaps more telling might be that testimony on how the land has changed. Most people today think of woods in the northeastern U.S. as wild and primitive where the reality is that nearly all are second or third growth over old farmlands. How might we view those woods if the last farmers on those lands reminded us how they came to be? People driving across Nevada today might see an empty natural environment, but the folks who lived there or passed through with wagons in the 19th century might recall the grasses that fed their animals being wildly different from the plants there now. Fishermen in California might drool at the stories of the fish that were in the rivers just over 100 years ago. Farmers in some areas might be startled to hear how much soil used to be present in their fields. Would this inform us at a gut level on how deeply altered the physical landscape is? Would it make us better stewards of the land? Or might these same voices express even greater satisfaction at having made such profound alterations? Would an old beaver trapper look out over Salt Lake City or Denver with remorse or pride?
We as a society technically know a lot about these changes. Field biologists can point out the plants that don’t belong because of the descriptions and samples made by those who encountered these lands far earlier. Historians can point out where the old fields were, geologists can document the rapid erosion in some places. Maybe we can imagine standing on the Santa Monica Mountains and looking out over an oak-dotted grassland that is now the totally urban San Fernando Valley, but how much more would we truly embrace that knowledge were we standing next to one whose eyes had actually seen that sight?
The point? Humans constantly reset their collective view of normal by simply dying off. And in some ways, that is probably for the best. We don’t need old Germans advocating for the return of the greater Germany of 1900 any more than we need a return of the USSR or the colonialism of the British Empire. But when it comes to the planet as a whole, the lack of perspective hurts. Efforts to preserve nature can be misdirected to preserve “natural” places that are are utterly unnatural while neglecting ecological treasures that don’t fit in with our evolving sense of normal nature. We get unglued about addressing global climate change because we don’t remember such a big snowstorm from our youth. In short, we have trouble placing our own limited experiences within a broader context; we often instead replace that broader context with those experiences. Frankly, we need help to do this better, and maybe longer lifespans might help. Too bad it will be too late for much of the natural world…
One of the fixtures of modern life seems to be the hearty embrace of uninformed certainty. People who just know that certain things are an unqualified bad and will go to any lengths to fight those things seem to make up the vast majority of social media contributors. Although there are many fine examples of this on the political right, let’s complain about some on the political left.
Two such issues are centerpieces of complaints here in Boulder. One is the presence of genetically modified organisms used in crops (GMOs) and the other is the practice of fracking. Neither warrants the blanket condemnation they receive.
Most opponents of GMOs know little about how we’ve ended up with the food crops we have now, though occasionally you get clues, like if you stumble on wild strawberries and wonder why they are so tiny. Our food crops are the products of generations of hit or miss efforts of artificial selection (picking the outcomes you like best) and crossing of different plants to get useful hybrids. The genetic tools now available remove a lot of the hit and miss part of the effort allowing scientists to directly target the aspects of a plant that are causing trouble.
When you say that all GMOs are bad, you might as well say all spot welds in a car are bad and you only want a car assembled with no welds. The use of genetic tools is a technique and not an end per se. A spot weld might make a tougher car, but it will not make a better computer. It is what you do with the tool that matters.
Does this mean all GMOs are good? Hardly, if for no other reason than the law of unintended consequences. For instance, there was a desire to have a variety of common golf green grass be resistant to Roundup; as High Country News tells the story, the new variety was successful–but when it escaped from where it was being grown, it became a troublesome weed along irrigation ditches in eastern Oregon. Human endeavors are filled with such mistakes, many having nothing to do with GMOs (think of all the times an exotic species was introduced and found to be a pest, and then the effort to use the pest’s natural enemy simply created another problem). Just as we recognize that bringing exotic species into someplace requires some forethought, development of GMOs needs to face similar scrutiny.
Fracking is a slightly different issue, though it shares the same blanket opposition that has little to do with what it is and does. Most of the concerns with fracking have nearly nothing to do with the actual process of fracturing rock deep in the earth to release hydrocarbons. Instead when you hear the actual harms people complain about, it is the industrial noise and associated air pollution of the drilling and fracking operations, the greater density of drill pads often needed for the current “non-traditional” horizontal drilling, surface water pollution from spills, aquifer contamination from improperly sealed wells, earthquakes from injection wells disposing of accessory fluids from production, or even the antiquated forced pooling laws that greatly limit the options for those holding both surface and mineral rights. When people talk of banning fracking, it would be like a city banning a car company from using welds–it is not the welding that is the problem, it would be the noise and impacts of the car factory that are being opposed. Fracking is really being used as a proxy for resurgent oil and gas development.
Is fracking then an unalloyed good? Well, no. There are some very positive aspects of it: by increasing the recovery of hydrocarbons from an existing field, it can slow the desire to expand production into virgin areas. The recent application in associated with horizontal drilling has opened up a lot of natural gas, which has been replacing dirtier coal in electricity generation as a result. But there are some instances where fracking is indeed a direct evil. In a few places, it has indeed caused larger earthquakes (though far, far fewer than injection wells). There is an indication that fracking in some shallow rocks immediately below an aquifer in Wyoming has indeed directly contaminated fresh water. And no doubt a few fracking operations have spilled fracking fluids into surface waters. And, of course, the application of the technique has opened up areas that previously were uneconomic (which is a mixed bag depending on where you are and what the land use looks like).
Most folks would probably like the world to be black and white, good or bad. But there is gray all over the place, and GG earns his nom de plume when encountering absolutism. This desire to polarize to the extreme removes all sensible middle ground. We would all win if GMOs were not so misrepresented but also if the regulation on their development made more sense. We would all win if oil and gas development was throttled back by a more driven effort to move on to renewable energy sources. Recognizing the strengths and weakness of things like GMOs and fracking could focus our attention on the specific instances that are most troublesome. But when you just paint the whole thing one color, you lose the ability to separate the dangerous from the innocuous.
“We have a crisis in higher education in this country today. Our faculties are not reflecting the diversity of thought in America” -CU Regent John Carson (via Daily Camera)
Is this what we have come to? Are we supposed to reflect the diversity of thought in America? So we should have people who think homeopathy is solid science? That the Earth is 6000 years old? That astrology works? That evolution does not occur? That the Earth is flat, even? How about that Barack Obama was born in Kenya? That “Ronald Wilson Reagan” having 6 letters in each name was a mark of the devil? That the moon landing was faked? That LSD is a great cancer cure? America is absolutely overflowing with ‘diversity of thought’, but a lot of that thought is lazy, unsupported or easily refuted from evidence. Do we really want that in the university?
To be fair, Regent Carson wasn’t advocating for cranks on the faculty, he was arguing that political conservatives are underrepresented on the faculty. But that line of argument can be used to support almost anything (astrologers are even more underrepresented than conservatives).
What seems to be missing from these concerns is evidence that being a conservative results in discrimination in either hiring, promotion, or salary increase. Read More…
Certainly one of the most striking things about modern American political discourse is the magnitude of outright lying going on. While misdirection and obfuscation were not uncommon in political speech, outright provable lying wasn’t. And yet now we have a President who Politifact says has made statements that are either false or “pants on fire” 47% of the time and who has inspired the Washington Post fact checker to keep a running count of lies. This follows years of internet chain emails and conspiracy theorists that have made Snopes expand rapidly to capture and review all the questionable stuff circulating on the internet. Needless to say, this tends to encourage others to play equally fast and loose with truth. For a scientist, this is a distressing trend–but it isn’t really that new.
Now to be clear, big lies have made the circuit before, being a staple of the Nazi government, for instance; the related game of “whataboutism” was a favorite of the old Soviet state. Some might point to McCarthyism in the US as a domestic episode, though the Red Scare had less questioning of objective truth and more vilification by insinuation. Here GG refers to outright misrepresentations of is going on. And as science’s goal is to discern the nature and rules of the reality we inhabit, it has a habit of landing in the crosshairs of those whose interests conflict with reality.