Well, it is that time of year when we send off freshly minted graduates off into the real world. They have sat through speeches imploring them to go out and make the world a better place from their elders and others reminiscing on their times in college before marching to a podium, getting a piece of paper, and discovering that the alumni association is really interested in them.
While the speeches heard quickly fade from memory, GG would like to take a stab at some advice for science graduates….without the need to sit under a hot sun wearing a giant trash bag and the most ill-fitting and unflattering hat on earth.
Congratulations New Scientists! You have completed a degree program viewed as Important by Important People like politicians (very few of whom have completed such a degree) and placement officers (ditto) and your professors (who generally do have such degrees). So you must have done something significant.
Why might this be so significant? It is because you are now armed with a powerful weapon, a sword of science, if you will. With this, you can cut through bias to find truth, you can drop superstition in its tracks, drive rumor into retreat and determine how the world really works. You have encountered and hopefully mastered a mode of thinking that helps you to penetrate thickets of ignorance.
Others will defer to your better judgement because you wield this weapon. Some of you will even command comfortable salaries. Having passed through the travails of an academic program in science, you may find the way forward easier for having suffered to this point.
But don’t pat yourself on the back just yet–remember you are holding that sword of science. It might hurt.
One of the frustrations students sometimes have is a feeling that their perception of the quality of instruction is ignored. Some will complain that some faculty got a promotion or tenure or didn’t get fired despite getting a scathing review from students in some form of student review of a course (here at CU these are faculty course questionnaires, or FCQs, a term we’ll use as a stand-in for all the variants out there).
There is some truth to this. Faculty at a tier 1 research university almost never are denied tenure because a course was poorly taught. And unless it becomes a tradition, it will rarely affect a faculty member’s salary. Why is this? After all, teaching is a significant part of the job. And so what impact, if any, do these surveys have?
The first problem is that things like FCQs are only one rather imperfect measure of quality of instruction. They are, for instance, easily manipulated by giving higher grades (the most sadistic trick is to give high grades on a midterm, then the FCQ is administered before the final, where the instructor lowers the boom). At CU these questionnaires are administered the last couple weeks of class, when students are most stressed about completing the course with a good grade, so how a course fits in with the general level of stress can color evaluations. Occasionally even the best instructor will get sideways with a class, perhaps for a joke that falls flat or because of some misbehavior from a student that leads to disharmony. Students’ self-perception of the fraction of material they have mastered fits into this. And for non-major courses, there is much less interest in mastering the material, so a poorly taught intro non-majors course might get high FCQs because it was easy (this is not as common for majors courses, where students tend to recognize that there is stuff they need to learn that didn’t get taught).
What FCQs don’t measure is how much students learned, and how capable they are of completing tasks taught in the course. It is possible to have an ambitious class get low FCQs despite students actually knowing more that those completing a less ambitious section of the same course. One approach to measure what students learned is a concept inventory: a set of questions, usually given at the start and end of a class, that reflects understanding of key concepts being taught in a class. If students don’t improve, poor teaching; if they do, better teaching. These work really well in courses with very fixed academic goals, like intro math and physics, but creating such inventories is difficult and time consuming; courses like intro geology, which might have goals varying somewhat between instructors, can only give an incomplete picture of the success of instruction.
A more common attempt to gauge instruction quality is peer review–having other faculty come in and observe the class and, ideally, interview it. This is most common for pre-tenure professors where a lot of mentoring is possible. But your teaching might seem quite good to peers but lousy to students, and observing one or two classes will often only reveal the most flamboyant of transgressions.
Ideally you’d like to see what students retain 4 or 5 years after completing a course. This isn’t ever done. GG’s one experience was encountering a student in a science museum who had taken his intro course. Asking him if the course helped him at all working in a science museum, the answer was “No, not at all.” Evidently for that student, that course was a disaster.
So FCQs maybe aren’t a great measure of teaching, but then what good are they?
GG recalls leaving undergrad school and getting offers from graduate programs that included financial support–tuition paid plus a stipend. “Wow,” exclaimed a fellow senior “Getting paid to get an education!” It seemed incredible to us. That the salary was maybe just enough to live on, probably while dipping into some savings or a loan, hardly seemed a problem. Grad school was only a way station, not a destination.
And yet today there are graduate students at Yale engaged in a hunger strike to get that university to recognize their union, which was formed to increase wages and benefits. What gives?
It would be easy to brush this off as entitled students demanding a full wage while getting a free education or to vilify universities as sweat-shop employers (which is the point of view of the linked op-ed). The fact is, higher-ed has been jammed into an uncomfortable spot for number of reasons and the result, not surprisingly, is inequity. The answer might be a bit of a shock. [None of this is to defend Yale’s refusal to negotiate, which in a way is contrary to academia’s predilection to talk. At length. Not act, but talk.]
[But before moving on to the big picture, a gripe. The author of the op-ed, a history professor, contends that the appropriate job market for PhD history or English students is a professorship and, noting that there are more graduates than positions, infers that these students are destined for careers as adjunct faculty. If this professor’s instruction is so narrow that all her students can do with a PhD is teach history, then she should do what a Princeton professor once stated: I’ve trained my replacement, now I’m done. GG has groused about this blinkered view of the utility of a PhD before.]
Honestly, why this needs to be said, again and again, gets the Grumpy Geophysicist aggravated. Bibliographic metrics are no substitute for reading science, whether reviewing proposals, reviewing papers, or granting tenure. So the latest paper to make this point also makes the nice point that papers that step away from incremental science tend not to have great short-term metrics. Basically, it takes the field awhile to recognize something new and worthwhile. Even more infuriating is that journal impact factors are being used to evaluate individual pieces of science! This is insane.
Maybe all evaluators should be given the same advice accompanying investments: “Past performance is no guarantee of future success.”
Years ago proposals to the National Science Foundation were simply to conduct scientific research. But, as has often been the case when science is funded from the public, there were complaints about the relevance of science, etc. And so NSF rolled out a requirement that proposals also include a section addressing the broader impacts of research grants, probably to make science more criticism-proof. At first, this seemed mainly an information request–what other good might incidentally arise from doing this science. But as time has gone on, this has become grounds for rejecting otherwise acceptable scientific proposals. And GG will argue that this is mistaken (though he has suggested some more realistic claims that could be listed as broader impacts).
Now some will defend the broader impacts requirements and argue that they are in fact too weak (Small Pond Science, for instance, delves into broader impacts quite a bit). In a way, we are agreeing on impact while disagreeing on a cure. Certainly there are lots of reasons to encourage the dissemination of science and to encourage scientists to interact with communities. But the reality is that an awful lot of broader impact work is, well, awful. For instance, many scientists decided that a good outlet would be to train teachers–but not knowing how to train teachers, not having run workshops or prepared educational materials, having no concept of how to measure success, their efforts fell flat on their face. Other efforts had outreach in the forms of unread blogs or poorly attended lectures (full disclosure: this blog has never been claimed as a part of broader impacts in any proposal).
So what is the result? Read More…
No, not changes to the college, changes by it. For years and years now, higher education has been viewed as the perverter of young minds even as it is lauded as the gateway to upward mobility. Although this is usually portrayed as fine upstanding youth becoming leftist socialists, some of us remember the preppie phase where leftist parents lamented the materialistic impulses of their college offspring.
Does this have any meaning? Is it that the teachers at colleges and universities are brainwashing students? Does this reflect a cocoon where disagreement with the party line is squashed?
There are of course many divergences of current college students from GG’s time, but some continue to surprise a little. One such is that it seems nearly all college students don’t keep their textbooks.
We aren’t talking about the texts from breadth requirement courses (though GG kept his texts from those too), we are talking about the textbooks used in upper division major courses. One might think that these could prove helpful references in professional life, but many students sell their texts back to the bookstore. The odd thing is that many students want a textbook when taking a class (GG had dropped requiring a text in a 1000-level class years ago and students kept asking for a text, so he listed readings to go with the web materials provided). Publishers have noticed and now offer “rental” versions of e-book versions of texts that self-destruct after the course is over; similarly, campus bookstores are now aggressively seeking and marketing used books (and harassing faculty to let them know of future text requirements so they can try to only repurchase the texts that they might need).
This has to be more than simple economics–yes, books can be expensive, but they are still a fraction of the cost of the course and can still provide education long after the course is completed. Is this a case of penny-wise and pound-foolish? Or is it some feeling among modern students that the internet will always have what they need? Or that a textbook will go out of date in short order?
As a textbook coauthor (well, GG wrote software), GG is curious. Declining sales of texts will mean there will be fewer texts going forward and they will cost more, which presumably will result in fewer sales (the textbook death spiral).
Students often wonder about the cost of textbooks (well, so do authors and professors, to be honest). In some ways this parallels the arguments over the value of academic journal publishers. Texts usually are reviewed (first as a proposal, again when nearly completed); they have copy editors and graphic designers to try and make the book pleasant to read. And the potential financial reward for some flavors of text (mainly the intro texts–upper division geology texts are not huge money makers) encourages authors to work to make material more accessible and better illustrated than their own course notes. So there is real value added, though there is a question of that value versus the cost.
Maybe it is just that modern students are wiser. Of the texts GG has saved, probably half have never been reopened and most of the rest just a couple more times. But there are a few that are now falling apart from repeated use. Maybe modern students aren’t simply cheap, maybe they are better about divining which are those go-to books.