Much to my surprise, when I came onto campus this morning I found my student evaluations from last fall waiting in my mailbox. As I wrote back in November when the evaluations were distributed, in the past professors don’t get the evals until May. Things have been sped up — perhaps because unlike in the past, no one bothered to type up the written comments. I was simply given all of the evaluations in a manila envelope.
A summary, prepared by the division dean, was attached to the front. Five of my seven classes were evaluated; a total of 211 students participated by turning in evaluations. Students were allowed to rate their profs as “Outstanding”, “Good”, “Average”, “Poor”, or “Failing.” My ratings (and you’ll just have to take my word for it) were
Outstanding: 84%
Good: 15%
Average: 1%
Poor: 0%
Failing: 0%
Now, lest you think I write only to brag, note what else was in my summary: the details of the college and departmental averages for all full-time faculty. The college reports the following ratings for some four hundred professors evaluated campus-wide last fall:
Outstanding: 65%
Good: 28%
Average: 6%
Poor: 1%
Failing: <1%
Clearly, grade inflation works both ways! 93% of the faculty ranks above average. 65% of us are outstanding, which raises an obvious question about what it is that so many of us can be standing out from! What on earth does “average” mean when only 6% of full-time faculty fall into that category?
The evaluations are anonymous and not given to the professors until after the course is over. Clearly, students aren’t praising their professors merely in order to curry favor in the hopes of receiving a better grade in return. I know full well Pasadena City College is not uniquely superb; my colleagues who teach on other community college campuses report similar evaluation results. Are our students worried about the fragility of our egos? Are they incapable of distinguishing what is genuinely average, or do they see “average” as a term of vile opprobrium? We can’t all be this good, can we?
I do note that the ratings for faculty as a whole have gone up, college-wide. I can’t find my 2004 evaluations, but I did dig up my old 1997 evals (which were done right before I was awarded tenure). The summary report that year for the college was as follows:
Outstanding: 56%
Good: 25%
Average: 15%
Poor: 3%
Failing: 1%
Our averageness is declining, and our outstandingness is improving! The Lake Wobegon effect is clearly reciprocal.
Sigh.
Hugo,
A couple possibilities:
1. Teachers are being graded against the teachers students had in High School.
2. Teachers are being graded against a level of acceptability. Outstanding means “Very Good,” Good means “Good,” Average means “Acceptable,” Poor means “Poor” and Failing means “Failing.”
3. The methodology isn’t valid for a statistical analysis. Human brains simply don’t work that way. Just because something gives you repeatable results doesn’t mean it has any significance.
BTW: Have a “class” of students grade a teacher from a 10 minute sample of video of the teacher’s teaching. Compare the evaluations to the evaluations the class submits after the class is over. The results are almost identical.
Either intuition about instructors is quite good or evaluations are meaningless. Or, quite possibly intuition about instructors is quite good AND evaluations are meaningless.
When I’d teach the Stress Management section to paramedics, I could tell you beforehand who would threaten to kill me in the evaluation and who would think I was Socrates and Richard Feynman combined.
Highly specific comments are far more useful than these numerical ratings, although as long as some wise-butted Western Civ. teacher doesn’t look at the statistics, it might go a long way toward boosting instructor morale and causing them to live up to the study.
You do know that getting a better evaluation will cause you to do a better job teaching, right?
Ah, the Lake Wobegone effect. I’ve said the same thing here:
http://philosophy.wisc.edu/hunt/wasgradehuntartfin.doc
I have another hypothesis to add to Rob’s three:
4. An overwhelming majority of students don’t really care about teacher evaluations, and will automatically rate all their teachers `Outstanding’ unless they (the teachers) turn out to be spectacularly bad and/or give the student an especially low grade.
As evidence, I offer RateMyProfessor, and similar sites. At least from what I’ve seen, the typical professor rated on such a site is likely to fall in the high middle range of whatever scales have been set up — on 10-point scales, between 5 and 7. (The exceptions seem to be organised smear/promotion campaigns.) The site I was looking at yesterday also allowed evaluators to write a short paragraph on each teacher; while the paragraphs covered exactly what you’d expect (difficult or easy assignments/grading, boring or interesting lecturer, and other such things that aren’t closely related to the quality of instruction), it still showed students thoughtfully using criteria to evaluate their instructors.
When evaluation is seen as a chore — and rushing through an evaluation means you get to go home and sleep or eat fifteen minutes early — students won’t bother to put any serious thought into the evaluation. When evaluation is voluntary, students actually put some thought into the evaluation, and I’m guessing they spend more time working on them.
To further expand the opinions above: I’m a graduate student myself, and I’m of two minds about professors’ evaluations. On the one hand, I want to give them an honest idea what I thought of their teaching — even as a TA, I can see that it’s awfully hard to teach when you don’t know what effect you’re having on your students or why.
On the other hand, I know that student evaluations, like any consumer evaluations, can be used in bizarre and not always fair ways when it comes time for raises, promotions, and so on. For example, my dad, who’s a doctor, gets some kind of performance bonus if he gets more “Outstanding” ratings than the other doctors in his office — but getting “Very Good” from every single patient is worth less, in this contest, than getting “Outstanding” from one patient and “Totally Sucks” from all the rest.
So, between that and simple politeness (surprisingly hard to get past!), I find myself very rarely going below “Very Good,” and essentially never below “Acceptable”, on my evaluation forms. I do try to restrict “Outstanding” to those teachers I really think were well above average, but I doubt I’m keeping my curve very well balanced.
I was really annoyed when I went to JiffyLube and the manager told me that if headquarters called me for a customer satisfaction survey, I should give them an “excellent” rating. Then friend told me they were probably like the place she works (a pharmacy), which uses the system Hypatia mentions — they only count the top ratings. It makes me think that in addition to driver’s licenses and gun licenses, we should have statistics licenses.
I usually give my professors high ratings overall, but in the part where we get to break down specifics - organized, contributed to the course, listened to questions, etc. - I am sure to distinguish areas where the professor is good from those where they need improvement.
I’ve always disliked the notion of balanced grading curves, either for students or for their teachers. In an ideal world, every single student in a class would be engaged, motived, and performing very well — i.e., all getting good grades. It’s extremely rare that this actually happens, of course, but it should be the ideal, and for professors as well. If this survey is indicative (which might be impossible to know), rejoice!
For my part, I always seem extremely hesitant to give professors poor marks. Maybe it’s partly because I was taught that “If you can’t say something nice, don’t say anything at all”, maybe it’s because I respect that it takes a lot of work to just get up in front of students day after day, but I don’t like hitting “average”.
In a few cases, there hasn’t really been a good way to critize a professor. I couldn’t very well write “was hard to hear because he had the exact same pitch and volume as the air conditioner” on an evalution.
Umm, maybe we are getting better? Ten years ago, I don’t remember seeing half the teaching workshops and pedagogy meetings I see today. It seems like, in the past few years, so many faculty members, at least where I’ve been, are really working hard to become better teachers. We’re writing about it on blog. We’re giong to more conferences and meetings. A lot of us are really trying.
I’ve seen professors lose out on tenure solely because of what the teaching evaluations say. Maybe it’s just here, but students know they count. Students know they are read. And they use them to be heard. And they are.
In my little world, at least.
Nels, that’s possible — but no one in my department received anything less than 50% outstanding. (And some of us never go to these pedagogy workshops to listen to the imploring voices of the Ed.D. crowd). Still you may be right….
I too just got my evaluations from last semester and was not surprised to discover that my students failed me (our evaluations use ‘very poor’, but given the context, I like fail as the bottom option). It was my first semester teaching at this institution and I agree with the students that it went badly, but on reflection (& discussion with others) I suspect that in reality I was probably just average. For all our American enthusiasm for equity, we hate average.
Among the vitriol in the written comments, there are some useful suggestions but so far the majority of these propose that I do exactly the things that I and my teaching mentors identified as ways to improve the course. This makes me deeply suspicious of the value of student evaluations for anything other than a pedagogical strategy of inclusiveness. The students get more out of the impression that their opinion matters than the instructors get out of the opinions provided.
Or maybe this is why some create their own evaluations in addition to the institutional ones - perhaps an issue of survey design?
I have my own suspicions about this, and they relate to mandatory evaluation. The more colleges make students write evaluations, the more students who don’t care fill them out, and then it’s all just noise. Fairly easy to test empirically…
It seems that the words switch scale there, at ‘average’. The rest are absolute terms: this class was good, the teaching was poor, etc. The place between ‘good’ and ‘poor’ is ‘acceptable’, not ‘average’. It’s the same issue that has people talking past each other about grade inflation; is it a relative or an absolute scale?
For me, filling out evaluations, I would say 4/5 means I’m glad I took the class (but it wasn’t life-changing), and 3/5 means I at least didn’t suffer from it but would have been at least as happy in any other decent class. ‘Average’ here is distinctly not a compliment.
(I realize this is almost a week after posting, but it’s my first visit back here after your vacation)