The Impact of Student Teaching Evaluations
One of the frustrations students sometimes have is a feeling that their perception of the quality of instruction is ignored. Some will complain that some faculty got a promotion or tenure or didn’t get fired despite getting a scathing review from students in some form of student review of a course (here at CU these are faculty course questionnaires, or FCQs, a term we’ll use as a stand-in for all the variants out there).
There is some truth to this. Faculty at a tier 1 research university almost never are denied tenure because a course was poorly taught. And unless it becomes a tradition, it will rarely affect a faculty member’s salary. Why is this? After all, teaching is a significant part of the job. And so what impact, if any, do these surveys have?
The first problem is that things like FCQs are only one rather imperfect measure of quality of instruction. They are, for instance, easily manipulated by giving higher grades (the most sadistic trick is to give high grades on a midterm, then the FCQ is administered before the final, where the instructor lowers the boom). At CU these questionnaires are administered the last couple weeks of class, when students are most stressed about completing the course with a good grade, so how a course fits in with the general level of stress can color evaluations. Occasionally even the best instructor will get sideways with a class, perhaps for a joke that falls flat or because of some misbehavior from a student that leads to disharmony. Students’ self-perception of the fraction of material they have mastered fits into this. And for non-major courses, there is much less interest in mastering the material, so a poorly taught intro non-majors course might get high FCQs because it was easy (this is not as common for majors courses, where students tend to recognize that there is stuff they need to learn that didn’t get taught).
What FCQs don’t measure is how much students learned, and how capable they are of completing tasks taught in the course. It is possible to have an ambitious class get low FCQs despite students actually knowing more that those completing a less ambitious section of the same course. One approach to measure what students learned is a concept inventory: a set of questions, usually given at the start and end of a class, that reflects understanding of key concepts being taught in a class. If students don’t improve, poor teaching; if they do, better teaching. These work really well in courses with very fixed academic goals, like intro math and physics, but creating such inventories is difficult and time consuming; courses like intro geology, which might have goals varying somewhat between instructors, can only give an incomplete picture of the success of instruction.
A more common attempt to gauge instruction quality is peer review–having other faculty come in and observe the class and, ideally, interview it. This is most common for pre-tenure professors where a lot of mentoring is possible. But your teaching might seem quite good to peers but lousy to students, and observing one or two classes will often only reveal the most flamboyant of transgressions.
Ideally you’d like to see what students retain 4 or 5 years after completing a course. This isn’t ever done. GG’s one experience was encountering a student in a science museum who had taken his intro course. Asking him if the course helped him at all working in a science museum, the answer was “No, not at all.” Evidently for that student, that course was a disaster.
So FCQs maybe aren’t a great measure of teaching, but then what good are they?
Well, they do form one measure of teaching quality, but they are more apt to help show where an instructor’s strength might be. At a school with a graduate program, most faculty will teach at the lower division, upper division, and graduate levels. Ideally you teach at all levels, but if a faculty member is consistently doing poorly at one level but OK or better at others, a departmental chair might redirect him or her to teach at the levels he or she is more successful. So a series of lousy FCQs could result in an instructor no longer teaching a certain course.
Truly awful or truly exceptional FCQs will get noticed if maintained. No faculty member wants to teach a class the students hate, and no administration wants courses taught that aggravate students.
Of course, students look at these ratings to gauge which class to take (when there is an option). Is this helpful? A course with low ratings might be low because the instructor really doesn’t cover material at all, or because there is a lot more material than usual–and depending on the quality of the student, one of these might be a better choice than the other, but it might not be obvious from the FCQs.