Enough With The Metrics

The Guardian editorial writers wrote the following on August 15, 2008:

In this week's Guardian, a former aide of Tony Blair admitted that Sats risked turning schools into "drab, joyless assessment factories" where preparation for tests crowded out real learning.

Instead of using Sats as a snapshot assessment of how pupils are doing, schools end up teaching to the test because of the confused double purpose of the exams. As well as checking on the progress of individual children, the results end up defining the standing of the school - once they are collated into league tables. What gives the tables their power with parents is the seeming precision of the numerical rankings. But that precision is entirely spurious. Academic analysis suggests that year-by-year chance fluctuations in pupils' ability overpower any real differences in performance for the majority of schools. And much of the real variation that can be discerned is down to the social mix of the pupils, as opposed to the quality of teaching. There have been worthy attempts to recognise good teaching in tough areas by creating new tables which adjust the figures to take into account, among other things, the number of children entitled to free school meals. But such approaches are inescapably arbitrary, not to mention hard to understand.

Earlier this year ministers floated plans for a new battery of metrics, covering everything from bullying to drugs. They would do better to reflect that there should be more to education than arithmetic alone.

The geniuses at the NY Times wrote today

Education Secretary Arne Duncan has been pushing the states to create rigorous teacher evaluation systems that not only judge teachers by how well their students perform but also — when the results are in — reward good teachers while easing chronic low performers out of the system. More than half the states have agreed to adopt new evaluation systems in exchange for competitive grants from the federal Race to the Top program or greater flexibility under the No Child Left Behind law.

These incentives are long overdue. As things stand now, according to a study by the New Teacher Project, a Brooklyn-based policy group, many school managers make no distinction between high-performing and low-performing teachers. The result is that poor teachers stick around while good teachers go elsewhere or leave the profession, frustrated because they are not promoted, rewarded with better pay, or even simply acknowledged.

That clearly needs to change if the new evaluation systems are to have any impact on the quality of the teacher corps.

The study covered four large urban school districts consisting of more than 2,100 schools and nearly a million and half students. It measured about 20,000 teachers by how much academic growth students showed in a given year. On average, the highest-performing teachers — about one-fifth of those studied — helped students learn two to three additional months’ worth of math and reading, compared with the average teacher, and five to six months more compared with low-performing teachers.

No questioning of the metrics by the geniuses at the Times - of course the tests measure what we say they measure, of course the scores mean teachers who "add value: to their students' scores are "good" and teachers who do not are "bad," of course the "bad" teachers should be fired and the "good" ones rewarded with extra money and of course "incentives" will make teachers work harder and improve the test scores (and thus learning) of students.

All based on false premises of course - teachers are not working hard enough now but if you incentivize them, they will work harder. Test scores measure good teaching and can be used to evaluate teachers. Merit pay will keep the "good teachers" around and demoralize the "bad ones (which may push them out of the system.)

Sigh - most teachers do not want these things. Most teachers want Washington and the states to stay out of their classrooms and let them teach the way they know how to teach. They want the micromanagement of teaching from the districts to stop. They want the states and districts to stop changing policies every two years so they can actually see if something is working as opposed to just changing stuff for change's sake. They want to stop being demonized in the media and by the politicians for the things that are out of their control.

Most of all, teachers want to be evaluated rigorously but fairly. Value-added measurements with high margins of error and wide swings in stability based upon error-riddled standardized tests are NOT rigorous or fair ways to evaluate teachers. Basing teacher pay on these will not incentivize the "good" teachers to stay and the "bad" ones to go.

Most of the teachers I know who have left over the last five years have been fed up with the jive coming out of first NCLB and now RttT - the endless testing, the overreliance on metrics, the endless reforms that "hold teachers accountable" but let students and parents slide on accountability, and the refusal of the people in charge at the city, state and federal level to take responsibility for their policies - whenever something goes wrong, it's always the teachers' fault.

In short, teachers want to be treated with respect and appreciated for the hard work they do day in and day out in some very challenging circumstances.

And they do not want to be evaluated using metrics that do not measure what the people running things say they measure.

The geniuses at the Times (the $88 million dollar losing Times, btw) believe metrics can be used to quantify "good teaching" and "bad teaching," reward the "good teachers" and push out the "bad" ones and make the system a better one.

If metrics were the sole measurement of quality, the $88 million dollar losing NY Times is clearly a failure in the world of newspapers.

But the Timesmen and women do not see it this way, of course - metrics apparently matter in education, but not so much in journalism.

We would be better off if we got some of the The Guardian writers to speak truth to the "geniuses" at the Times about the value of metrics.

Or if we just solely judged the $88 million dollar losing Times by metrics and shut the whole thing down - like a "failing school" with "bad test scores.