The Irrationality Of New York State's VAM Scores

Carol Burris has a piece in the Washington Post about Sheri Lederman's case against NYSED over her VAM score, which went from 14 out of 20 one year to 1 out of 20 the next despite her students' test scores remaining stable.

Burris points out Lederman is not the only irrational VAM score around the state:

Lederman was not the only teacher in the school to get a poor score. In 2014, 21 percent of the staff at E.M. Baker School received a score of “ineffective,” 21 percent “developing” and 57 percent were “effective.”  Just the year before, not one teacher received an “ineffective” score.

The irrationality was not limited to the teachers of Sheri Lederman’s school, one of the highest performing elementary schools in the state. In 2014, 44 percent of the teachers of the Fox Meadow School in Scarsdale received growth scores that said they were not “effective” teachers with 22 percent rated “ineffective.”  Yet 61 percent of the school’s students were proficient in English Language Arts, and 75 percent were proficient in math—more that double the state’s proficiency rate.   Similar results were found at the high-achieving Harbour Hill School in Roslyn, where 36 percent of its teachers received growth scores that labeled them “ineffective.”

The Lincoln School in Rochester, is a school designated as a priority/failing school by the state. Its proficiency rate was less than 3 percent. In 2014, 100 percent of its teachers received “effective” state scores, with 7 percent being rated “highly effective.”  At another school facing receivership, The Martin Luther King Jr School in Utica, New York, 60 percent of the teachers received “effective” VAM scores and 40 percent were given VAM scores of “highly effective .”

NYSED never showed cause for why Lederman received the 1 out of 20 on her VAM score the year after she received 14 out of 20.  They instead tried to have the Lederman case dismissed twice (here and here), arguing there was no harm to Lederman because she was rated "effective" overall despite the "ineffective" VAM score.

However the case did go to trial and the judge didn't sound like he was buying the "Because We Said So" bell curve argument NYSED showed up with for why Lederman was deemed "ineffective" on her VAM:

The more Ms. Galligan tried to defend the bell curve of growth scores as science, the more the judge pushed back with common sense. It was clear that he did his homework. He understood that the New York State Education Department’s VAM system artificially set the percentage of “ineffective” teachers at 7 percent. That arbitrary decision clearly troubled him. “Doesn’t the bell curve make it subjective? There has to be failures,” he asked.

...

“Did her students learn nothing?” Justice McDonough asked. “How could it be that she went from 14 out of 20 points to 1 out of 20 points in one year?” He noted that the students’ scores were quite good and not that different from the year before.

Back behind the bell curve Ms. Galligan ran. As she tried to explain once again, the judge said, “Therein lies the imprecise nature of this measure.”

Burris writes that a decision is expected in two to three months - the Lederman's have argued that not only Sheri Lederman's rating should be overturned but the court should  “declare the measure an abuse of discretion” because “the State Education Department does not get a pass on unreasonable and irrational actions."

Of course the evaluation system that the Lerdermans have out on trial has been made that much worse by Governor Cuomo and the Legislature this year, with test scores now increased to about 50% of a teacher's evaluation rating, so the "irrationality" of the state's APPR VAM matters greatly and a decision by the judge to declare the measure "an abuse of discretion" will have serious consequences for the governor, the Legislature, the Board of Regents, NYSED and teachers and schools all across the state.