The Impact of Student Teaching Evaluations

One of the frustrations students sometimes have is a feeling that their perception of the quality of instruction is ignored.  Some will complain that some faculty got a promotion or tenure or didn’t get fired despite getting a scathing review from students in some form of student review of a course (here at CU these are faculty course questionnaires, or FCQs, a term we’ll use as a stand-in for all the variants out there).

There is some truth to this.  Faculty at a tier 1 research university almost never are denied tenure because a course was poorly taught.  And unless it becomes a tradition, it will rarely affect a faculty member’s salary. Why is this?  After all, teaching is a significant part of the job. And so what impact, if any, do these surveys have?

The first problem is that things like FCQs are only one rather imperfect measure of quality of instruction. They are, for instance, easily manipulated by giving higher grades (the most sadistic trick is to give high grades on a midterm, then the FCQ is administered before the final, where the instructor lowers the boom). At CU these questionnaires are administered the last couple weeks of class, when students are most stressed about completing the course with a good grade, so how a course fits in with the general level of stress can color evaluations. Occasionally even the best instructor will get sideways with a class, perhaps for a joke that falls flat or because of some misbehavior from a student that leads to disharmony. Students’ self-perception of the fraction of material they have mastered fits into this. And for non-major courses, there is much less interest in mastering the material, so a poorly taught intro non-majors course might get high FCQs because it was easy (this is not as common for majors courses, where students tend to recognize that there is stuff they need to learn that didn’t get taught).

What FCQs don’t measure is how much students learned, and how capable they are of completing tasks taught in the course. It is possible to have an ambitious class get low FCQs despite students actually knowing more that those completing a less ambitious section of the same course. One approach to measure what students learned is a concept inventory: a set of questions, usually given at the start and end of a class, that reflects understanding of key concepts being taught in a class. If students don’t improve, poor teaching; if they do, better teaching.  These work really well in courses with very fixed academic goals, like intro math and physics, but creating such inventories is difficult and time consuming; courses like intro geology, which might have goals varying somewhat between instructors, can only give an incomplete picture of the success of instruction.

A more common attempt to gauge instruction quality is peer review–having other faculty come in and observe the class and, ideally, interview it. This is most common for pre-tenure professors where a lot of mentoring is possible. But your teaching might seem quite good to peers but lousy to students, and observing one or two classes will often only reveal the most flamboyant of transgressions.

Ideally you’d like to see what students retain 4 or 5 years after completing a course. This isn’t ever done.  GG’s one experience was encountering a student in a science museum who had taken his intro course.  Asking him if the course helped him at all working in a science museum, the answer was “No, not at all.” Evidently for that student, that course was a disaster.

So FCQs maybe aren’t a great measure of teaching, but then what good are they?

Well, they do form one measure of teaching quality, but they are more apt to help show where an instructor’s strength might be. At a school with a graduate program, most faculty will teach at the lower division, upper division, and graduate levels. Ideally you teach at all levels, but if a faculty member is consistently doing poorly at one level but OK or better at others, a departmental chair might redirect him or her to teach at the levels he or she is more successful. So a series of lousy FCQs could result in an instructor no longer teaching a certain course.

Truly awful or truly exceptional FCQs will get noticed if maintained. No faculty member wants to teach a class the students hate, and no administration wants courses taught that aggravate students.

Of course, students look at these ratings to gauge which class to take (when there is an option). Is this helpful?  A course with low ratings might be low because the instructor really doesn’t cover material at all, or because there is a lot more material than usual–and depending on the quality of the student, one of these might be a better choice than the other, but it might not be obvious from the FCQs.

Tags: , ,

2 responses to “The Impact of Student Teaching Evaluations”

  1. Paul Braterman says :

    IMO,, the most important single role of evaluations is as feedback for the individual instructor, and any good evaluation process will invite open-ended comment from the students (these of course will need to be typed to conceal identity before the instructor gets them).

    I first met the FCQ process in midcareer, when I moved from the UK to the US. The results were a painful shock, and forced me to improve my comprehensibility. If I had received similar feedback earlier in my career, I would have been a much more effective instructor and communicator.

    A lot depends on exactly what questions are asked,, and there is a need for serious study of this. When, as Departmental Chair, I tried to improve the evaluation questionnaire along the lines of that used by another Department, I met massive opposition based on self-interest. Some faculty had learnt how to game the system, and did not want to change the one that gave more information.

    Like

    • cjonescu says :

      Thanks as always for the comment. I am curious what kinds of questions you found that would improve the feedback. We are allowed to individually add questions to our FCQs, so could be some of us could add questions you found to be helpful. I can imagine opposition, though exposing opposition based on self-interest in a semi-public way (e.g., faculty meeting) would presumably disarm it.

      You are correct that I inappropriatly overlooked the impact on the individual instructor, who generally is the only one to read the comments, though in my experience many faculty members won’t bother, having been brutalized in the past (e.g., best way to improve the class? “Fire the instructor”). I personally look at the comments from upper division classes but have given up on lower division ones where both snide comments and conflicting recommendations both fail to get at core problems in my teaching. I have implemented suggestions from upper division FCQs, and so arguably that has been one of the more concrete outcomes from that exercise.

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: