Perdido 03

Perdido 03

Friday, November 30, 2012

More On APPR And Merryl Tisch

Regents Chancellor Tisch wrote a misleading piece about APPR today in which she claimed the new evaluation system based upon high stakes test scores is essential to improving teacher quality:

Some of the debate has been over the use of student test scores to measure teacher performance. The complaint is that test scores don’t give full measure of a teacher’s performance. That’s true: Student test scores are only one measure of an effective evaluation system. But they are an essential measure.
The new law limits the use of state student-growth scores to 20 percent of evaluations, with another 20 percent of the score derived from locally negotiated objective measures. The remaining 60 percent is negotiated between the district and the local unions, and there are a number of options (e.g., supervisor observation, peer and student review, professional development) that can be adopted.

The student-growth scores provided by the state for teacher evaluations are adjusted for factors such as students who are English Language Learners, students with disabilities and students living in poverty. When used right, growth data from student assessments provide an objective measurement of student achievement and, by extension, teacher performance.

We should never judge an educator solely by test scores, but we shouldn’t completely disregard student performance and growth either.

 Actually the new law is written in such a way that a teacher can be rated "developing" or "effective" in all three components of the evaluation that Tisch describes above and still get rated "ineffective" overall.

Carol Burris demonstrates how that works here:

Now take a look at the chart below, which will be used in New York to evaluate teachers. It is similar to a chart I explained here.This is what was decided as part of last week’s grand bargain; it’s what NY lawmakers will be asked this spring to put into law to sort and select public school teachers, with those deemed ineffective for two years to be fired.

Regulation/Student Growth/Local Measures/Other 60/Composite
Ineffective/..........0-2........../.........0-2.........../.............../.......0-64
Developing/.........3-8........./..........3-8........../.locally..../.......65-74
Effective/.............9-17......./..........9-17......./developed/.......75-90
Highly Effective/.18-20...../..........18-20.... /................/.......91-100

Now let’s go back to my first cafeteria scenario, applying it to the chart.

Ms. Alvarez is a second-year teacher. Her diverse third-grade class, which includes English language learners, takes the state tests. In the first category, ‘student growth,’ the teacher’s students show average growth. She is rated effective and earns 9 points. In the second column, again she is rated effective based on student work and gets 9 points again. Her principal critiques her lessons and there is room to grow, so she assigns her 46 out of the possible 60 points in category three, ‘other 60’. Although the state does not provide ranges for the ‘other 60,’ we can see that a score of 46 based on the proportions in the first two columns, would be effective. Now let’s add the numbers up and look at the final column: 9+9+46=64. Overall, Ms. Alvarez is rated ineffective. She decides that maybe teaching is not for her.

Hard to argue that this APPR system is fair when it tags a teacher as "ineffective" overall even when she is rated "effective" on all three components - and yet it does.

It would be nice if Ms. Tisch explained how such a system is fair, but she doesn't - she's too busy criticizing the UFT for not agreeing to the local part of the system as quickly as possible so the state can start firing teachers immediately.

As for the sophisticated model of student growth that the state is going to use on teachers, School Finance 101 takes a closer look as that here:

Setting aside this long list of concerns about the NYC VAM results, I now turn to the NYSED – state median growth percentile data (which actually seem inferior to the NYC VAM model/estimates). In her editorial, Chancellor Tisch proclaims:
The student-growth scores provided by the state for teacher evaluations are adjusted for factors such as students who are English Language Learners, students with disabilities and students living in poverty. When used right, growth data from student assessments provide an objective measurement of student achievement and, by extension, teacher performance.
Let me be blunt here. CHANCELLOR TISCH – YOU ARE WRONG! FLAT OUT WRONG! IRRESPONSIBLY & PERHAPS NEGLIGENTLY WRONG!

[now, one might quibble that Chancellor Tisch has merely stated that the measures are "adjusted for" certain factors and she has not claimed that those adjustments actually work to eliminate bias. Further, she has merely declared that the measures are "objective" and not that they are accurate or precise. Personally, I don't find this deceptive language at all comforting!]

Indeed, the measures attempt – but fail to sufficiently adjust for key factors. They retain substantial biases as identified in the state’s own technical report. And they are subject to many of the same error concerns as the NYC VAM model.  Given the findings of the state’s own technical report, it is irresponsible to suggest that these measures can and should be immediately considered for making personnel and compensation decisions.

Finally, as I laid out in my previous blog post to suggest that “growth data from student assessments provide an objective measure of student achievement, and, by extension, teacher performance” IS A HUGE UNWARRANTED STRETCH! 

While I might concur with the follow up statement from Chancellor Tisch that “We should never judge an educator solely by test scores, but we shouldn’t completely disregard student performance and growth either.” I would argue that school leaders/peer teachers/personnel managers should absolutely have the option to completely disregard data that have high potential to be sending false signals, either as a function of persistent bias or error. Requiring action based on biased and error prone data (rather than permitting those data to be reasonably mined to the extent they may, OR MAY NOT, be useful) is a toxic formula for public schooling quality.


School Finance 101 posted earlier about the NY State evaluation system and found it wanting in a number of other areas as well.  You can see that post here.

It would be nice if Merryl Tisch told the truth about the new APPR system.

But like many education reform proponents, she's given to lies and deception in order to push through very radical changes to the school system.

Maybe she believes her own bullshit and really thinks this system is going to improve teaching and learning.

Or maybe she knows she's full of shit and says what she says anyway so she can make money for her brother at K12 Inc. and her pal Rupert and Joel at Wireless Gen and the rest of the ed reform entrepreneurs she hangs with.

Frankly it doesn't matter what her motivation for spewing bullshit is.

The fact is, she is full of shit, what she's writing about this evaluation system is pure deception and misinformation and she ought to be relieved of her chancellorship and replaced with somebody who actually cares about schools, students and teachers rather than just test makers, data trackers and education entrepreneurs.

No comments:

Post a Comment