The Next Time You Hear Some Edubabble Starting With The Words "Research Shows..."

...You can cut the person off by asking if they're sure the "research" they're citing isn't horseshit:

A painstaking yearslong effort to reproduce 100 studies published in three leading psychology journals has found that more than half of the findings did not hold up when retested. The analysis was done by research psychologists, many of whom volunteered their time to double-check what they considered important work. Their conclusions, reported Thursday in the journal Science, have confirmed the worst fears of scientists who have long worried that the field needed a strong correction.

The vetted studies were considered part of the core knowledge by which scientists understand the dynamics of personality, relationships, learning and memory. Therapists and educators rely on such findings to help guide decisions, and the fact that so many of the studies were called into question could sow doubt in the scientific underpinnings of their work.

...

More than 60 of the studies did not hold up. The project began in 2011, when a University of Virginia psychologist decided to find out whether suspect science was a widespread problem. He and his team recruited more than 250 researchers, identified the 100 studies published in 2008, and rigorously redid the experiments in close collaboration with the original authors.

Only 36% of the studies held up to scrutiny.  If you add in the so-called research in other fields, even that kind of accuracy is probably too high:

Dr. John Ioannidis of Stanford, who predicted in the early 2000s that about half of the findings across medicine were inflated or wrong, said that the 36 percent replication rate for psychology might even be high once the results for other disciplines, like economics, animal research, cell biology and other areas of biomedicine, were also tested.

“Many of the biases found in psychology are pervasive,” Dr. Ioannidis said.

It would be interesting to see a painstaking yearslong effort to reproduce 100 studies of education research to see what the replication rate is.

You hear people on all sides of education reform issues throw out the words "Research shows..."

Our culture has fetishized "research" and "science," as if to say if something has been found by "research" in a "scientific study," it must be so.

This is not so, as one commenter at the NY Times notes:

I applaud the researchers ability to recognize the limitations of their research and their standard of peer review. There is an old saying in medical research: "half of what is published is worthless; the challenge is figuring out which half." The measures adopted help figure that out.

The bigger danger, however, lies not with the scientists but by the talking heads that like to cite (often incorrectly) these studies. President Obama famously said he would "restore science to its rightful place", implying that science was the highest level of understanding. These findings prove what every honest researcher knows- that published studies are often incorrect, that established but untested norms of behavior are often correct, and claiming science as an authority is limiting. Placing blind trust in science is just as foolish as placing blind trust in faith.

The take home message is simple- if a single scientific study claims something different than experience suggests, it needs to be re-tested before it is claimed to be true. Remember that the next time someone proposes a sweeping social change based on shoddy and poorly explained science.

Gee, re-testing scientific research before proposing sweeping social change based on shoddy and poorly explained science - that would never happen in education, would it?

Nahh - the accuracy and validity of the studies on VAM, the Common Core State (sic) Standards, the small schools initiative, teacher evaluations based upon test scores, students who have teachers who raise test scores make more money over their lifetimes, et al. were all tested and re-tested and have been found to be "objective science," as Governor Cuomo is fond of saying.

Right?

Sure.

And "untested norms of behavior" that have stood the test of time are always suspect until Raj Chetty or some other "researcher" gets his/her hands on them for some of their rigorous, scientific research to test them.

Right?

No, not right.

As the commenter at the Times writes, blind trust in so-called science is no better than blind trust in faith.

When you get back to school in a week or so, you can bet you'll hear the words "Research shows..." at some PD meeting sooner rather than later.

It is high time to challenge the research fetishizers when they throw out their "Research shows..." jive to prove that the "research" they're citing is part of the 36% that can be replicated and not the 64% that can't.

Joel Klein's School Of One Doesn't Work

So reports the Daily News:

Former NYC Chancellor Joel Klein's highly touted School of One math project dropped by 2 of 3 schools in pilot program
Initiative was hailed by Time magazine as one of the 50 best inventions of 2009, but NYU study shows $9 million effort failed to raise test scores more than old-school math classes.

But the DOE is forging ahead with plans to utilize the program anyway - even though all it is, really, is a glorified test prep program.

UFT Takes Ads Out Against Data Report Publication But Inexplicably Supports Using A Similar Method For Teacher Evaluations

What jokers we have at the UFT:

The New York City Education Department will release the ratings of thousands of teachers on Friday, ending a nearly year-and-a-half-long legal battle by the teachers’ union to keep the names confidential.

The ratings, known as Teacher Data Reports, grade nearly 18,000 of the city’s 75,000 public school teachers based on how much progress their students have made on standardized tests. The city developed these so-called value-added ratings five years ago in a pilot program to improve instruction and has factored them into yearly teacher evaluations and tenure decisions.

Even before their release, the ratings have been assailed by independent experts, school administrators and teachers who say there are large margins of error — because they are based on small amounts of data, the test scores themselves were determined by the state to have been inflated, and there were factual errors or omissions, among other problems.

The union, the United Federation of Teachers, is responding to the release with a $100,000-plus newspaper advertising campaign starting on Friday. With the headline “This Is No Way to Rate a Teacher,” the advertisements will feature an open letter from the union president, Michael Mulgrew, that displays a complex mathematical formula followed by a checklist of reasons why the ratings are problematic.

“The Department of Education should be ashamed of itself,” Mr. Mulgrew said Thursday. “It has combined bad tests, a flawed formula and incorrect data to mislead tens of thousands of parents about their children’s teachers.”

Mr. Mulgrew is absolutely correct on all three counts - the tests were badly designed, the value-added formula has a high margin of error and wide swings in stability, and some of the data attributed to individual teachers is incorrect.

Those are some damned good reasons to oppose the publication of the Teacher Data Reports.

They're also some damned good reasons to oppose the new teacher evaluation system that bases 40% of a teacher's evaluation on similar data and value-added measurements to the TDR's.

Somehow Mulgrew and Company can run an assault against the TDR's as flawed and error-riddled, but defend their agreement with Cuomo to let the state grade every teacher in New York with a very similar method without any irony.

The teachers quoted in the NY Times article understand the problem with basing a teacher's evaluation on test scores and value-added measurements, however:

The New York Times, one of a number of media organizations that had requested the records, plans to publish the ratings on its education blog, SchoolBook, and has asked teachers to respond online. On Thursday, several posters on SchoolBook called the reports deeply flawed and criticized the city as well as the news media for making them public.

Marie Kallo, a sixth-grade English and social studies teacher at Intermediate School 234 in Brooklyn, said that even though she had received an above-average rating, she was troubled by a significant error in her report: It said she had taught 120 students in 2007-8 when she had actually taught more than 200.

“That makes me question the accuracy of all the data reports,” Ms. Kallo said, adding that she also did not understand how the ratings were calculated. “How is it fair to be judged on information that is not accurate?”

Karen Fine, a third-grade teacher at Public School 134 in Manhattan who previously taught fifth grade, said she and her colleagues believed that the ratings were an unfair and inaccurate measure of a teacher’s performance because they used an unreliable methodology that had been criticized by many respected researchers and statisticians, and because they did not account for factors that could affect students on the day of testing, like being tired, nervous, or scared.

“For many of us who teach in N.Y.C., this has been our life’s calling,” she said. “We are constantly attacked on so many levels for what ails education in our country when we know that it takes a community to help children learn: principals, administrators, parents, lawmakers, and yes, teachers. The responsibility cannot lie solely on us.”

Indeed.

But the new teacher evaluation system agreed to last week by the NYSUT and the UFT will help politicians, so-called education reformers, media elites and others continue to attack and scapegoat teachers for all that ails public education.

What's worse, because the unions failed to make Cuomo impose this piece of garbage evaluation system himself and instead held his hand onstage when he announced it, they co-own the evaluation system along with Cuomo, the NYSED, and the Regents.

No wonder they're sending out Leo Casey, the UFT's resident shill and used car salesman, to fool people into thinking the new evaluation system will be nothing like the madness that is the Teacher Data Reports.

But Leo can sling his bullshit all he wants and Mulgrew can take out all the ads he wants on the TDR publication - in the next year when every teacher gets the memo that states 40% of their evaluation is based upon test scores (20% from the state tests, 20% from the city tests), that over 20 high stakes standardized tests are going to be added to the school year in order to pull the new eval system off, and that if a teacher comes up "ineffective" on the 40% based upon test scores, they will be declared "ineffective" overall and slated to be fired in a year, the UFT won't be able to hide their capitulation.

Same goes for when teachers discover that they can be declared "effective" in all three components of the evaluation system yet still come up "ineffective" overall as a teacher and thus be slated for firing in a year, or come up "effective" in all three components and still come up "developing", which means hours of garbage PD to "improve" before the next eval.

Just wait, fellas.

You can fool people with bullshit for a long time, but eventually even the blind notice the overwhelming stench of fecal matter with the moniker UFT stamped on it when they're up to their necks in it.

New York Times Spins Teacher Data Reports Publication As A Positive Development

The publication of 12,700 Teacher Data Reports from 2008-2010 is imminent and the New York Times is dying to get their hands on the names and numbers even though

The ratings are imperfect, according to independent experts, school administrators and teachers alike. There are large margins of error, because they are generally based on small amounts of data. And there are many other documented problems, like teachers being rated even when they are on maternity leave.

Sure, the numbers are completely flawed, the algorithms that were used to come up with those numbers have margins of error bigger than the cocaine bags under Cathie Black's eyes, and some of the ratings will rank English teachers as math teachers or attribute the wrong numbers to teachers, but you know, it's the public's right to know this information.

Oh, plus the Times claims they've solved all of the problems inherent in the flawed Data Reports:

With SchoolBook’s partners at WNYC, The Times has developed a sophisticated tool to display the ratings in their proper context, a hallmark of our journalism.

And just in case the "sophisticated tool" created by the Times and WNYC STILL doesn't give enough "context" to the numbers and you feel slandered by the story, the Times invites you to become your own lawyer:

But we want to take that a step further, by inviting any teacher who was rated to provide her or his response or explanation. We are seeking those responses now, so they can be published at the same time as the data reports.

We plan to include those responses alongside the ratings themselves, so readers can consider them together.

If there were special circumstances that compromise the credibility of the numbers in particular cases, we want to know.

What self-serving horseshit the Times is spinning here.

They are publishing data that they know to be flawed, no "sophisticated tool" they develop is going to make that data any less flawed, and the Times and its outgoing education editor and soon to be Times Jerusalem bureau chief Jodi Rudoren (who stirred up her own bit of controversy a few weeks ago with a tweet to a Palestinian-American author that ensures her stay at the Times Jerusalem news bureau may be brief and she'll be back covering teacher data reports before she knows it) can rationalize this all they want.

The fact is, the numbers are horseshit, the reports themselves are horseshit, the Times is horseshit for publishing them and Jodi Rudoren is horseshit for spinning the publication as some kind of public service.

Chris Christie Brags About Austerity Measures

Raising health care costs on public employees by 22%, slashing Medicaid, refusing to fund the pension plan unless the unions agree to sharp cuts in benefits and hikes in retirement ages - and he was swaggering and bragging as he announced all of this, according to the NY Times:

TRENTON — A pugnacious and boastful Gov. Chris Christie proposed a no-growth budget for New Jersey on Tuesday, saying he had inspired a legion of copycat governors from Albany to Sacramento and was not about to let up in his efforts to shift money from public workers to property-tax payers and businesses that create jobs.

Declaring that his deep cuts last year had “established the new normal,” Mr. Christie, a Republican, carved out a modest increase for public-school aid, kept municipal aid flat, offered small increases for indigent care at hospitals and financial aid for college students, and set aside a bigger amount to support improvements in the state’s vital transportation system.

But to pay for his priorities, Mr. Christie, who has waged a 14-month rhetorical war against the teachers’ and other public-sector unions, stepped up the pressure to slash their members’ health and retirement benefits.

The governor conditioned a doubling of the middle-class property tax rebate, for example, on the passage of changes that would make public employees pay 30 percent of the cost of their health insurance, compared with an average of 8 percent that Mr. Christie said many now pay.

He also offered to make a partial payment to the state’s woefully underfunded pension system, but only if the system was overhauled to require bigger payments from workers, smaller payouts and an increase in the retirement age.

“Please, let’s not pick the special interests over our overburdened taxpayers,” Mr. Christie said during his budget address.

Mr. Christie said that even in Wisconsin, where Gov. Scott Walker’s proposal to cut the benefits and collective bargaining rights of public workers had set off a firestorm of protests, officials “have decided there can no longer be two classes of citizens: one that receives rich health and pension benefits, and all the rest who are left to pay for them.”

Meanwhile the Times points out that post crisis, austerity measures seem very sexy, but they are actually harmful.

But you know, when you're talking about raising taxes on hedge fund managers, there can never be enough political courage in the form of austerity measures to lay off workers, squeeze their wages and slash their benefits.

Remember the lines the Founding Fathers wrote: Government by the hedge fund manager, for the hedge fund manager, of the hedge fund manager...