Tuesday, November 17, 2009

Harry Potter and the Totally Arbitrary Evaluation

Last week I was observed by my principal and a master educator. I haven't had my meeting with the master educator yet to discuss my evaluation (sorry -- no juicy numerical details), but I did meet with my principal to discuss my scores and I left that meeting even more convinced that some administrators in this school system just don't have what it takes.

The lesson I got observed on went OK, but not great. I had split the class up into two groups based on their performance on a mini-assessment, and was doing differentiated lessons based on those groupings. While there were no major problems, the class wasn't terribly well organized. Essentially, whenever I left one group to work with the other, the first group didn't really accomplish much. Nothing terrible happened, but I hadn't set up structures to ensure that the groups continued working even when I walked away. All this to say that I expected my evaluation to be OK, but not stellar.

Wrong. I received a score of 3.8 (out of a perfect 4), which puts me in the "highly effective" category. Now, if I'd actually earned that score, I'd be pleased. But I didn't. My lesson showed me to be effective, but not outstanding. So why did I get the score I got? Because my principal has decided that she likes me. Of course, this isn't really a problem for me (except that I'm not really getting any feedback for improvement, I suppose). But it is a problem for the people she's decided she doesn't like. Some teachers at my school are unhappy with their scores, and for some I don't really doubt that it's because they're not based in reality.

I firmly believe in accountability for teachers. Teachers should be held to high standards of excellence. Someone should be able to walk into your classroom at any time and see what you're doing, and you should be doing your job reasonably well. I firmly believe that teachers who aren't meeting an acceptable level of performance should be put on an improvement plan and, if that doesn't work, transitioned out of the classroom (read: fired). Kids deserve that much. But I'm also coming to realize that such a practice won't ever happen fairly until we have administrators who are willing and able to do that job. If teachers are the single biggest factor in improving student achievement (as I -- and Michelle Rhee -- think they are), then aren't administrators the single biggest factor in improving teacher effectiveness?

(P.S. I think some will read this post and say, "That's just what we've been saying forever! Rhee is terrible! That's why the union protects us from arbitrary firings!" Well, I still disagree with those statements. There are way too many teachers in this system who are grossly incompetent, and I applaud Michelle Rhee's fervent attempts to rid our system of them. I'm just saying we won't be able to do that until administrators are on board with doing their jobs well. That is all.)


Kat said...

Ha! Great post about how the unearned good scores are just as suspect as the unearned bad ones, and further evidence why we all need to set our own bars...and set them higher than what is expected of us.

It'll be interesting to see what your ME feedback and scores are. I hope you consider sharing that experience, and I applaud you for sharing this one.

KM said...

I had the opposite experience. My principal can't stand me for reasons I still don't know and my scores on her evaluation pretty clearly reflect that. Furthermore, she doesn't seem to understand the rubric. For instance, she wrote that I reached three different learning styles successfully but then scored it as a two. When we met to discuss it I questioned that and she ultimately ended up changing it, but still she's the evaluator doing the evaluation-- shouldn't she know the scoring system as well if not better than me?

Anonymous said...


You agree with Rhee that teachers are the single biggest factor in student achievement. So does that make parents the second biggest factor?

I get that teachers are extremely important, but are you seriously saying they are more important than parents?

Pollack said...

In order to get rid of the incompetent teachers you need not only administrators who are "on board and doing their jobs well," but a realistic expectation of replacing them with better candidates, which in most districts—as in DC, which relies upon programs like TFA to fill positions with idealistic college grads who more than half the time are burnt out in two years—is not a realistic expectation. There's remarkable job security in teaching, not because of powerful unions or incompetent administrators, but because so few qualified people are willing to do the job for the pay, since if they're competent and qualified they can surely get paid more to work less elsewhere; and since so many who try (in fact a majority, by most estimates) decide after a year or two that it isn't worth it. A bad teacher is better than no teacher in almost every case, and these are often the only realistic alternatives.

I'm wary of calls for "teacher accountability" because I have never yet seen anyone use a method of evaluating teachers that convincingly measured their quality, since that is an intrinsically subjective evaluation, and public education is horrified of subjective evaluations and can't trust anyone to make them. The obsession with making everything objective results in evaluation criteria that are effectively semi-arbitrary checklists. (Name on the board? Check. Date? Check. Day's agenda? Check. Walking around the room? Check.)

(The way they evaluate teachers at places like St. John's is by having a rotating committee of senior faculty hold interviews and have conversations and make a decision. And sometimes the decision sucks, or seems political, or whatever, but that's not the way they do it because they're silly and old-fashioned; it's because that's pretty much the only way to do it. And they don't typically have unqualified substitutes filling the vacancies that result.)

And lastly, although we surely should not lose sight of the value of good teachers, or the importance of their influence, there is either some extremely careful parsing of words or extremely careless use of them when someone says that teachers "are the single biggest factor in improving student achievement" who presumably would not tell their own child (or any student) that the single biggest factor in their achievement is the quality of their teachers.

Pollack said...

I realize now that my reference to "St. John's" might be unclear, since I wrote this comment originally as a response to a shared item on Google Reader, shared by someone who would get that reference, and I pasted here without editing.

I was referring to St. John's College, which is a place I attended, but the same principle applies to lots of other places besides.

Kelly said...

KM, while MEs have had extensive (one hopes) training in rubric scoring, I don't believe administrators have had the same level of training around the use and application of the rubric's domains.

Theoretically, yes, she should know the scoring system better than you.

Realistically, this is not such an easy rubric to use. There are still some "holes" and/or grey (subjective) areas.

I would encourage you to take the rubric and visit a colleague's classroom. Spend 30 minutes in there and score what you see. It's the best way to understand the strong and weak points of the rubric, and also to get an idea what it looks like from an evaluator's point of view...something teachers need to do more of.

It's not as easy as it looks. Yeah, you shouldn't be correcting your principal on what amounts to a high-stakes test for you. But be proactive about addressing it (which you were) and know the tools being used to judge you.

Mr. Potter said...

I wrote that teachers are the single biggest factor in improving student achievement. This is (I'm fairly certain) what Michelle Rhee says too. I do not think that teachers are the cause of the achievement gap. The single biggest determining factor for student achievement (according to research I've read) is the level of academic achievement of the parents. However, I think the only way to close the achievement gap (i.e., improve student achievement) is to have an entire school system of educators that are doing their jobs extremely well. As a school system, we can't change the parents kids have. But we can change the teachers.

An elementary school teacher said...

I haven't got my rating yet from my principal who just recently observed me. And if I, like you, got a 3.8, I will thank God and I'm an atheist. I mean it.

Pisces said...

I understand your frustration. I used to teach in Louisiana public schools. Once, my assistant principal gave me low marks on an observation because my stapler ran out of staples and "this caused some confusion." It only took me 10 seconds to reload it! Then someone from the school board came to observe me and gave me very high marks because I spoke French to the students. I'm a French teacher. I never got any useful feedback.

Kings said...

Research shows that teachers are the most important “in school” factor in improving student performance. That phrase “in school” is often left out, making it appear that academic research shows beyond a shadow of a doubt that good teaching is all that is needed for kids to achieve at the highest levels.

And of course you are right, Mr. Potter, “As a school system, we can't change the parents kids have. But we can change the teachers.” But what does that mean REALLY? Will changing teachers close the achievement gap? Have you seen that happening. In the twenty year history of TFA, what significant gains have been made?

Can you think of anything else, besides changing teachers and changing parents that might help? Like better discipline? More supplies? Smaller class size? Better in school nutrition? Longer school days? Summer enrichment? I think it’s hubris for teachers to think they can singlehandedly make incredible changes and I think it’s reprehensible for administrators and policy makers to take advantage of youthful idealism to make you think you can perform miracles. They know you can’t, but they enjoy seeing you try. It keeps them from looking for serious, complex solutions and helps them break the union and get rid of veteran (i.e. expensive) teachers.

So while you’re pouring your guts out for the kids, they’re saving money and waiting for you to burn out so they can send the next batch of idealists in.

Meanwhile, what about the kids?

Kings said...

Analogy: Let's say you can't change the mountain, but you can change the mountain climbers. What does that mean? What if the mountain is so high and the weather conditions notoriously so bad that it's rare that anyone has made it to the top? Do you keep recruiting mountain climbers or do you look for another way to get to the top - helicopters for instance, or perhaps gauging weather conditions to climb at only the most opportune times, and using the the very best gear and giving climbers the very best training in advance.

Compare this to TFA - picking academically strong novices, giving them a few weeks of advanced training and throwing them in the classroom to see how they do. If/when they burn out or simply don't work out, get a new batch to try their hand at teaching. To me, this shows little respect for the teacher and even less for the kids. It plays into youthful idealism and adult self-serving interests.

Sara said...

All evaluations -- in education and outside of it -- are largely arbitrary: subject to the whims, moods and personal preferences of the evaluator at the time of the evaluation. Even a rubric can't be normed for human nature, try as Jason Kamras might.

Maybe the answer is to discount your principal's eval and put more stock in the master teacher's. Yet how many people will cry "unfair!" if their principal's eval is high yet the master teacher's is low?

It's not a perfect world. It will never be a perfect world. At best, the most we can hope for are slightly imperfect solutions in this not-so-perfect world.

Kings said...

a main problem with the IMPACT evaluation is that, regardless of its effectiveness, its main purpose is not to help teachers, it's to decide who will be fired at the end of the year. It's not about educating teachers about a new methodology, it's about weeding people out who are not already familiar with it. Teachers who have not been exposed to the methodology in the past are now expected to figure it out on their own it by the end of the year, or be fired.

I have now heard of two much-in demand teachers who have received less than a 1.5 on their master teacher evals. These are teachers who have parents and kids fighting to get into their classes because of their strong reputations. Oh yes - they're old and they didn't do "information chunking" and in the 30 minutes the master teachers were there, they saw no evidence that the teachers wanted their kids to succeed.

Toby said...

I just had my IMPACT eval by my principal. She was pleased with my lesson, but as the internet was down in my school, the actual score was unavailable. I was told that it was over 3 points. Heck, I want what Harry got, as I think my lesson was good, by IMPACT and any other standards. I had never heard of IMPACT or any other tool besides the PPEP until it was introduced earlier this SY. I did make it a point to at least read over the rubric and all the Ts so i could be somewhat prepared for it. Most teachers in my school, according to our principal got 2.5 and above, many getting 3 and above. I do feel a little relieved that this first observation is over, as I know my principal is tougher than the ME.

Pollack said...

Mr. Potter,

I did mean to suggest what I took to be two possibilities: (1) very careful parsing of words, and (2) very careless use of them. Your emphasizing that you were only talking about factors in IMPROVING achievement suggests number (1).

I am still fairly skeptical of the use of such language, which is why I lumped together the very careful parsing and the careless use. I'm of the opinion that what our public schools are, and what they're supposed to do, and how they work, and what the best role of the "educators" is, need to be considered as fundamental and still unanswered (mostly unasked) questions. Harping on teachers is facile — many of them are incompetent, to be sure, and some are great, but we actually don't have enough of the latter to replace the former, or even the means of developing them.

Teachers are a significant variable, and conveniently one of the easiest ones to fuss with. Very many other variables are in fact virtually impossible to fuss with. But I'm not convinced that they are the "single biggest factor," or even that such a factor is identifiable. The reality is much muddier than that. (For many of my own students—and from the looks of one of your recent posts, for many of yours as well—the single biggest and most pressing factor in improving their achievement would probably be getting them to show up to class.) My point was just that the "teachers are the most important thing" claim is overly simplistic; however you or Rhee parsed the language, you would probably not accept such an account when speaking to a single child in isolation, because it's obviously too reductive of the countless factors affecting that child's improvement (or lack of improvement). When we speak in the aggregate, the reality surely gets no less complex, but we're more prone to use less complex accounts.

Also, though it's a minor nitpick, I don't believe that parental academic achievement is the single biggest (or even a) "determining factor." It is among the most predictive, though. The distinction is very important.

And finally, I was not "anon," but had as my identity my real surname, and a link to my own site.

Good luck with all the holiday interruptions.

edlharris said...

Jay Mathews discusses IMPACT today:



Anonymous said...

The Gods send clues about this situation frequently.
Eddie of Iron Maiden.
Closure of Fitsgeralds Reno/getting kicked out of Fits LV for "counting cards".
WWII's other holocaust:::Unit 731:::tsushogo. And I get blamed because the Gods used me as an excuse to bury it in history, an act positioned with the Japanese and their importance in society.
Stop obsessing. Let it go.

Kidney poison cut into my methanfetamine. The "free" crank had a connection to the employer::::Severe seizures to punish me for stealing. Both punishment for the same offense, one decades later.
And they used Artificial Intelligence to "push" me into stealing to sell the destruction of my health and life in this very public situation, for without it required resistance would have caused great problems in their positioning.
It's just a matter of getting the pieces of the puzzle in place so their tools in highly visible, respected positions don't look bad in the process.
Praying for the end of your wide-awake nightmare.

"You're not going to be able to find work." because the Gods are positioning the Jews preventing me from finding a job.
Sad thing is these people won't have ANYTHING to do with it. The god's positioning enables them the freedom to sell this telepathically. The actual people in these clone hosts have learned far too much to do anything foolish enough like participate. It will set them back and amplify the extreme punishment already in store for them.
If there is ever people willing to participate they are either the youngsters who have yet to learn or a fresh new clone host, some kid freshly in, grumbling about going into a 60, 65 year old decrepid man.

They have shared that when the change is initially made they alter the DNA with Artificial Intelligence, ensuring it is not the same person.
Now depending on their level of activity the Gods switch out for new individuals, ie presidents switched frequently. They say they bring the (m)father's DNA when it is time to conceive, so it is strategically important who is in there for the conception of the children. The same may not be true for typical activity:::the Gods may alter the DNA upon initial clone hosting but it may be a general change. Conception requires specific DNA.
And? So?
So there are no Rockafellers. This was how the Gods eliminated their true importance, for now they are little more than lowly English peasants, ironically.
Now the case can be made that far more men are invited into clone hosting than women, and since lineage is matrilineal it would rather follow that blood line instead.
All it takes is one.
But I wonder if it even matters. Those who learn and are wise enough to refuse the offer of clone hosting still have to repair their relationship with the Gods and hope for a better placement in their next life, one which certainly won't involve being a billionaire. Whether real Rockafellers or DNA alternates is of no concequence. Their goal is still ascention, which takes work and dedication. Being/thinking you're capitalism royalty invites a host of temptations, including grandeur and self-importance, which are all damaging to a good relationship with the Gods.
If only they can overcome the temptations that money brings, like ample stunning women and Scarface piles of cocaine.