A piece in the New York Times by the authors of a paper in press in the Journal of Experimental Psychology: General has kind of baffled the Grumpy Geophysicist for a few days now. It argues that confirmation bias is not a problem, but desirability bias is. In essence, you favor new information that aligns with what you want to happen rather than what you think is true.
Now, had you asked GG before this to define confirmation bias, he might have said “favoring information that says what you want it to say.” What the paper says, though, is that this describes desirability bias. To tease the two apart, you need a situation where what you want and what you expect are two different things [frankly, though, this is a capsule definition of a pessimist].
The experiment described used the desired and anticipated results of the last U.S. presidential election as expressed by 811 participants (89 others were disqualified, 48 for saying they made a mistake or were dishonest). If you believed candidate A was likely to win but wanted candidate B and you were given information that indicated that candidate A was ahead in the polls, you didn’t change your estimate of who would win by much. If you saw information that candidate B was ahead, you gave candidate B a substantially greater chance of winning. The authors then assert that confirmation bias isn’t an issue, but desirability bias is.
It isn’t hard to expect a disconnect on other topics. Do you want climate change to occur and likely to lead to societal disruption? Probably not, yet many exposed to evidence that climate is warming increase their belief that the climate is warming, no? This possible conundrum didn’t slip by the study’s authors, who wrote (R1 version of their manuscript):
When confronted with new information regarding global temperature increase, strong believers updated their beliefs more upon receipt of ostensibly undesirable information (i.e., a faster temperature increase than expected), whereas weak believers updated their beliefs more upon receipt of ostensibly desirable information (a slower increase than expected). Though this pattern appears consistent with an independent confirmation bias, such an outcome may emerge when individuals are personally invested in “being right”—indeed, for many climate change activists a belief that the world is warming constitutes a core part of their identity (Stern et al., 1999). For such people, objectively undesirable (but confirming) information about the rate of global warming may be subjectively desirable: vindicating their commitment to combatting climate change (Sunstein et al., 2016) and affirming their cultural group identity (Kahan et al., 2012).
In other words, those anticipating climate change want to be proven right, so their acceptance of evidence confirming their evaluation that climate change is occurring is because they desire to be right more than they desire the climate to not change. Um, precisely how is this different from confirmation bias again? Is confirmation bias supposed to be free of emotions? It feels like you can always make it seem as though desirability bias is at the root, making the term confirmation bias irrelevant.
The op-ed closes with this summary: “Our study suggests that political belief polarization may emerge because of peoples’ conflicting desires, not their conflicting beliefs per se. This is rather troubling, as it implies that even if we were to escape from our political echo chambers, it wouldn’t help much. Short of changing what people want to believe, we must find other ways to unify our perceptions of reality.”
Sorry, but this is feeble. It implies we are prisoners of our present beliefs. This is the precise mindset underlying the mantra that science advances one funeral at a time–a mindset precisely contrary to what science should be. If this is so, how exactly did same-sex marriage advance when the echo chambers kept up their respective drumbeats? A lot of folks who are OK now with same-sex marriage don’t personally want it or even like it, yet they have come to feel that is the fair thing to do. How did they come to change their minds if they didn’t favor it all along? At a more general level, how could we ever recognize hazards? Why would you want to believe that DDT killed birds? Why would you want to believe that humans created an ozone hole?
Hell, how did all those study participants ever reach the point where they expected their preferred candidate to lose? Doesn’t the existence of those folks somehow disprove the rigidity of this hypothesis?
GG’s feeling is that in parsing the question of belief versus desire, the study’s authors have made a distinction with little value. The cute example they used feels artificial.
Maybe instead of studying why we don’t change our minds, we need to study why we do. There are folks working on this (GG has noted one example before). Hopefully all they desire is to get a correct answer–then their desirability bias will work for us all.
Paul Braterman points out the difficulty of the straitjacket of middle school “scientific method” and how that runs into challenges with historical science (something GG approached more timidly in an old post “Can you falsify an earth science hypothesis?“)
Science does not have a separate special method learning about world, the “scientific method” as taught in schools is a damaging illusion, and the falsifiability criterion has itself been falsified
Below, R: How not to; “The Scientific Method”, as inflicted on Science Fair participants. Click to enlarge
Consider this, from a justly esteemed chemistry text:
Scientists are always on the lookout for patterns.… Once they have detected patterns, scientists develop hypotheses… After formulating a hypotheses, scientists design further experiments [emphasis in original]
Or this, from a very recent post to a popular website:
The scientific method in a nutshell:
1. Ask a question
2. Do background research
3. Construct a hypothesis
4. Test your hypothesis by doing experiments
5. Analyze your data and draw conclusions
6. Communicate your results [emphasis in original]
Then, if you find yourself nodding in agreement, consider this:
Since a scientific theory, by…
View original post 2,414 more words
GG attended a workshop here at CU on lessons from mining that could help guide oil and gas development (since the conveners encouraged outcomes to be shared on social media, figure this is OK). In kind of an odd way, the focus was more on what happens at the end of mining or oil development more than what happens at the start, so that will tend to be the focus here.
So a quick summary of points GG noticed.
- Mining is highly focused, oil and gas far more distributed with a web of infrastructure.
- Mines active today have to meet bonding requirements and increasingly have to have reclamation plans; oil and gas wells have far less specific requirements (e.g., bonding is not by well but by state or even nation).
- Mines are a single use of the land; oil and gas production often shares the land with other rural uses.
- Problem mines are problems for thousands of years–there is no true long term remediation. They can foul a lot of water for a long time. Wells are more insidious, typically failing silently until you know groundwater is compromised or a house blows up.
- Mines these days are rarely totally shut down; they frequently are mothballed and then brought back online. Oil and gas wells are frequently plugged and closed.
- Mining’s main impact seems to be contaminating surface waters. Oil and gas activity mainly affects subsurface waters.
- Modern mining remains dominantly rural [save for mining towns!], but oil and gas has moved into suburbia. However, old mines are around a lot of western towns and there is renewed activity (and opposition) from time to time.
So what would the public want for post-development lands? In both cases, one can presume a safe environment available for any subsequent use. In many cases, they might want something resembling the pre-mining landscape. How realistic is this?
For mines, it depends on the mine. Big, modern open pit mines with sulfides are likely nearly hopeless. Strip mines for coal probably can be reclaimed provided they are not in areas where erosion is likely. Many small legacy mines can be shuttered to have an acceptably low level of impact. You can probably tell when a mine can be safely shut down.
For oil and gas wells, there is a surprisingly high level of uncertainty. Modern plugging procedures will usually work for the near term, but if gas continues to migrate up the well bore, any weaknesses that develop in the well plug or around the outside of the well bore will allow the gas to vent to the surface. Degradation of the well materials will connect shallow and deep aquifers, which can be troublesome if the deep aquifers have sufficient pressure to invade a shallower drinking water aquifer. Or if the deep aquifer has negative pressure, you can lose drinking water to the deeper aquifer. That oil and gas wells are not of the same material as the surrounding rock means that it is likely over long periods of time that some kinds of failures of the well’s plug will occur (chemistry and stress will focus on that interface). How often is this likely? How often will a failure produce surface problems? We really aren’t certain.
One suggested solution for problem mines is to make use of the waste material. This might help for acid mine drainage, but is less helpful for some other environmental hazards from mines. It is unlikely that a plugged oil or gas well that leaks has any economic utility.
So at the end, what does full closure of mines or wells look like? Mines are unlikely to have their footprint totally erased, and some will be problems for centuries, but many others will be available for other uses. Oil and gas wells are tougher. Most rules require a plugged well’s pad to be returned to something looking like the original landscape. When bonding is insufficient (as has been the case in Wyoming, for instance), failed companies’ wells might not be reclaimed. But even where surface reclamation is done (and oil and gas companies like to show pictures of old well sites to show they don’t look particularly bad), the well below is still subject to failure and leaking. While some mine sites might well be safe to build on (and many mountain resort towns are in fact built on old mine sites), building on an old well is playing a bit of Russian roulette. Shallow aquifers could fail as well. Perhaps monitoring for natural gas and pollutants in the water would permit full reoccupation of well sites, but it seems just as likely that rules will prevent building on or too near old well sites.
What do local communities need to know? They should probably understand that oil and gas wells are forever–plugged wells in most cases will cause no problems, but given that we haven’t watched a bunch of wells plugged with modern techniques for a really long time, that there is a non-zero risk of future leakage, and so monitoring appropriate for the subsequent use of the land should be required. Ripping out as much of the oil and gas infrastructure as possible is wise. For mines, it kind of depends. Any mine with underground workings can later collapse, so building on top of such mines should be considered with caution. If a mine is leaking colorful water into streams, odds are this will continue for centuries and some kind of action is desirable, but know there are not, at present, permanent fixes.
Well, it is that time of year when we send off freshly minted graduates off into the real world. They have sat through speeches imploring them to go out and make the world a better place from their elders and others reminiscing on their times in college before marching to a podium, getting a piece of paper, and discovering that the alumni association is really interested in them.
While the speeches heard quickly fade from memory, GG would like to take a stab at some advice for science graduates….without the need to sit under a hot sun wearing a giant trash bag and the most ill-fitting and unflattering hat on earth.
Congratulations New Scientists! You have completed a degree program viewed as Important by Important People like politicians (very few of whom have completed such a degree) and placement officers (ditto) and your professors (who generally do have such degrees). So you must have done something significant.
Why might this be so significant? It is because you are now armed with a powerful weapon, a sword of science, if you will. With this, you can cut through bias to find truth, you can drop superstition in its tracks, drive rumor into retreat and determine how the world really works. You have encountered and hopefully mastered a mode of thinking that helps you to penetrate thickets of ignorance.
Others will defer to your better judgement because you wield this weapon. Some of you will even command comfortable salaries. Having passed through the travails of an academic program in science, you may find the way forward easier for having suffered to this point.
But don’t pat yourself on the back just yet–remember you are holding that sword of science. It might hurt.
One of the frustrations students sometimes have is a feeling that their perception of the quality of instruction is ignored. Some will complain that some faculty got a promotion or tenure or didn’t get fired despite getting a scathing review from students in some form of student review of a course (here at CU these are faculty course questionnaires, or FCQs, a term we’ll use as a stand-in for all the variants out there).
There is some truth to this. Faculty at a tier 1 research university almost never are denied tenure because a course was poorly taught. And unless it becomes a tradition, it will rarely affect a faculty member’s salary. Why is this? After all, teaching is a significant part of the job. And so what impact, if any, do these surveys have?
The first problem is that things like FCQs are only one rather imperfect measure of quality of instruction. They are, for instance, easily manipulated by giving higher grades (the most sadistic trick is to give high grades on a midterm, then the FCQ is administered before the final, where the instructor lowers the boom). At CU these questionnaires are administered the last couple weeks of class, when students are most stressed about completing the course with a good grade, so how a course fits in with the general level of stress can color evaluations. Occasionally even the best instructor will get sideways with a class, perhaps for a joke that falls flat or because of some misbehavior from a student that leads to disharmony. Students’ self-perception of the fraction of material they have mastered fits into this. And for non-major courses, there is much less interest in mastering the material, so a poorly taught intro non-majors course might get high FCQs because it was easy (this is not as common for majors courses, where students tend to recognize that there is stuff they need to learn that didn’t get taught).
What FCQs don’t measure is how much students learned, and how capable they are of completing tasks taught in the course. It is possible to have an ambitious class get low FCQs despite students actually knowing more that those completing a less ambitious section of the same course. One approach to measure what students learned is a concept inventory: a set of questions, usually given at the start and end of a class, that reflects understanding of key concepts being taught in a class. If students don’t improve, poor teaching; if they do, better teaching. These work really well in courses with very fixed academic goals, like intro math and physics, but creating such inventories is difficult and time consuming; courses like intro geology, which might have goals varying somewhat between instructors, can only give an incomplete picture of the success of instruction.
A more common attempt to gauge instruction quality is peer review–having other faculty come in and observe the class and, ideally, interview it. This is most common for pre-tenure professors where a lot of mentoring is possible. But your teaching might seem quite good to peers but lousy to students, and observing one or two classes will often only reveal the most flamboyant of transgressions.
Ideally you’d like to see what students retain 4 or 5 years after completing a course. This isn’t ever done. GG’s one experience was encountering a student in a science museum who had taken his intro course. Asking him if the course helped him at all working in a science museum, the answer was “No, not at all.” Evidently for that student, that course was a disaster.
So FCQs maybe aren’t a great measure of teaching, but then what good are they?
Traditionally, the Laramide Orogeny starts around 75 million years ago. Probably most geoscientists would agree with the overall analysis of Dickinson et al. (1988), which is mainly based on sedimentary rocks preserved from that time. So their criteria were that marine sedimentation (diagonal hatch) had ended prior to the Laramide, individual basins shifted from sharing facies with adjacent areas (black square) to having distinctly thicker deposits (circle) and coarse clastic detritus derived from nearby uplifts (black triangle) as the Laramide started:
It would be hard to argue that the Laramide Orogeny started later than the kinds of dates that Dickinson et al. proposed–but could it be earlier? If you had shallow sea floor covered in muds and parts started to rise up, might the muds simply get entrained in the existing current systems and be scoured down, creating a lacuna that, later on, would erased by even deeper erosion? In other words, is it possible that there was early deformation that wasn’t vigorous enough to overcome the broad subsidence of the region and so failed to produce positive topography? And if so, would subsurface loads have started to create local depocenters that perhaps have escaped recognition?
When you look back to find when the Old West died, GG would like to nominate 1906 as that magic year.
In 1906, the last of the classic gold rushes of the West reached its peak. Goldfield, having been found just a couple years earlier, became the most populous city in Nevada on the basis of its considerable bonanza gold deposits. It and its companion silver boom town of Tonopah represented the last gasp of big finds by miners wandering the west. As these towns faded out, the state of Nevada would try to find a new economic base. First they encouraged travel for getting a divorce, and then they removed the restrictions on gambling. That transition from a mainly extractive economy to a mainly tourism based economy began as Goldfield started to empty out. The memory of the mining heritage would live on: nearly every Nevada town seems to have a casino named the Nugget (and most others have some mining theme, like “Bonanza” or “Silver strike”), but it would increasingly be tourists and not mineral veins that would be mined.
Another tourism related event–one most folks overlook these days–occurred in 1906. The first park set aside by the nation was Yosemite Valley; in 1864 it was transferred to the state of California to be protected in perpetuity. In 1890, advocates for protecting the surrounding high country had given up on the state, feeling it had mismanaging the park, and worked to get a federally managed national park created. Thus Yosemite National Park (the federal version) was created as the third national park behind Yellowstone and Sequoia. The state, however, continued to manage the valley. Continued agitation by park advocates finally led the state to relinquish control of the valley in 1906, in essence declaring an end to any possible equivalence of state and federal control of parklands. The transfer to the federal government would also end the state’s practice of allowing Native Americans to continue to live in the valley; though it would take the Park Service decades, they finally removed the last descendant of the Ahwahneechee from the valley. For most of the following century, Native Americans would be denied a modern presence in federal parks; instead they were relegated to colorful descriptions of their ancestors’ historic occupation of the land.
And then in 1906 the San Andreas Fault, only recently named at that point, failed in the catastrophic San Francisco Earthquake. Between the quake and the fire, much of the city’s Gold Rush heritage was lost–not only buildings but photographs, written records and other memorabilia of a city that grew from a small trading post to an international metropolis on the back of the riches that passed out of the Sierra. As the city rebuilt, it would not be in the mold of the old Gold Rush town but would be the new financial and trade capital of the West Coast, one stylistically different from the city that had just been demolished.
So 1906 saw the loss of much memory of the Gold Rush, both in records in San Francisco and in activity as Goldfield began its decline. The era of modern tourism, with federally managed playgrounds and locally permitted houses of various sins, was grafted onto declining mining camps and previously state-managed land.
A coda helps to illustrate the transition. The 1906 quake triggered avalanches in the Sierra Nevada, including in the remote Mineral King valley high in the southern Sierra, where many of the buildings of a small resort were smashed. The resort’s owner was seeking a patent on land being claimed as a mill site for associated mineral claims, a request opposed by the Sierra Forest’s supervisor, who pointed out that no mining was actually occurring. Despite the destruction of much of the resort, Arthur Crowley pushed together remains of two buildings to continue operations as he continued his quest for a paten. A court held that Crowley’s claim was valid, and the patent was granted. Mining law had opened up a tourism future; the driver of the West in the 19th century was giving way to that of the 20th century.