The other problem, one that won't ever be solved perfectly, is that principals are seeing growth scores that occasionally defy what they personally know about the skills of certain teachers. That's going to happen with sample sizes as small as single classrooms, but hopefully not too often, and teachers who do well on the other 60 to 80 percent of their evaluation shouldn't have much to fear from it. Experts, even those who aren't big on judging teachers this way, agree the system New York is putting in place to measure student achievement year-over-year is "state of the art."
"State of the art?"
Based on what?
The tests they're using for this system are an error-riddled embarrassment.
The system is set up on a bell curve so that every year, 10%-20% of teachers are going to be ranked substandard no matter what.
And as the geniuses at NEWSDAY themselves even note, there are teachers coming up substandard on the "performance-based" part of the evaluation system who administrators, students, parents and other teachers know to be excellent educators.
The same happened with New York City's vaunted TDR system - teachers who were known by administrators, students, parents and other teachers to be excellent educators came up "ineffective."
Michael Winerip covered the story of one last year in his Times column, actually.
Under the new APPR system, this kind of teacher will be slated to be fired after two years of "I" ratings.
Gary Rubenstein pointed out that the VAM the city used on 4th-8th grade ELA and math teachers was essentially random - one third came up "effective," one third above, one third below. But the clicker was, many teachers switched where they came up each year, so that an "effective" teacher often came up "ineffective" the year after and vice versa for the teacher found "ineffective" the year before.
And as Carol Burris has pointed out, the system is set up so that you can be ranked "effective" on the three different evaluation parts - the state assessment, the local assessment and the classroom observations - and STILL be given an "I" rating.
Whether you are talking about the state's APPR system or the city's TDR system, there is nothing "state of the art" here, unless by state of the art, you mean how ingeniously dishonest these systems are.
They are rigged against teachers.
They are meant to be rigged.
That's why they've set it up on a bell curve, that's why they're hiding the tests used to generate the "performance data," that's why the governor, the Regents and the NYSED made it so that a teacher whose students show no growth on the tests must be rated "ineffective" automatically no matter how she/he scores on the rest.
They want to fire teachers - lots of them - and now they've got a couple of systems in place (the state APPR and the local assessment data systems) that will provide them with the "data" to support the firings.
Even though the data generated from these systems has large margins of error, wide swings in stability and is based upon error-riddled tests.
The only thing "state of the art" here is the dishonesty with which the governor, the NYSED. the Regents and the editorial writers at the Times, the Post, the News, Newsday and the Journal cheer this evaluation system while claiming it is so "scientific" and excellent.
They know this thing is half-baked (as even NEWSDAY admitted today) they know it's untested, they know it's rigged to smear a certain percentage of teachers as "ineffective" every year no matter what - but they're going to use it on teachers and fire the "bad" ones anyway.