Sunday, February 23, 2014

How The Chinese Rig Their PISA Scores - And Why It Doesn't Matter Anyway

The next time some education reformer type spews alarmist clap trap around the PISA scores, you can lay this on them:

Chinese experts are also less impressed than Truss by the Pisa scores. "Even though Shanghai students scored well on the test, this doesn't mean that Shanghai's education system doesn't have any problems," said Lao Kaisheng, a professor in the education department of Beijing Normal University. "In fact, it's the opposite."

As long as China's education system remains vast but resource-constrained, Lao added, its schools will default to testing as a reliable indicator of competence. "The education system here puts a heavy emphasis on rote memorisation, which is great for students' test-taking ability but not for their problem-solving and leadership abilities or their interpersonal skills," he said. "Chinese schools just ignore these things."

According to an analysis of the rankings, the children of Shanghai's cleaners and caterers are three years more advanced than UK lawyers' and doctors' children in maths. Yet the figures are an unreliable measure of equality. Although Shanghai's 23 million people make up less than 2% of China's population, its per capita GDP is more than double the national average; its college enrolment rate is four times as high.

Furthermore, nearly half of Shanghai's school-age children belong to migrant families and were effectively barred from taking the test: because of China's residence registration system, these students are forced to attend high school in their home provinces, where schools are often debilitatingly understaffed. Although students from 12 provinces took the test in 2009, the government only shared Shanghai's scores.

"The OECD has not disclosed if other Chinese provinces secretly took part in the 2012 assessment. Nor have Pisa officials disclosed who selected the provinces that participated," wrote Tom Loveless, an education expert at Harvard University, on a Brookings Institute blog. "There is a lack of transparency surrounding Pisa's relationship with China."

In short, the Chinese PISA scores are rigged and the Organization for Economic Co-operation and Development (OECD) help them to hide the chicanery - here's more from Tom Loveless:

Shanghai is portrayed as a paragon of equity in PISA publications. A 2010 OECD publication, Strong Performers and Successful Reformers in Education, highlights model systems that the world should emulate. Shanghai is singled out for praise. One section on Shanghai is entitled, “Ahead of the pack in universal education.” The city is described as an “education hub,” and the only discussion that even remotely touches upon migrants is the following:
 “Graduates from Shanghai’s institutions are allowed to stay and work in Shanghai, regardless of their places of origin. For that reason, many  ’education migrants now move to Shanghai mainly to educate their children.’ “[2]
That description is surreal. PISA’s blindness to what is really going on in Shanghai was also evident in the official release of PISA’s latest scores. The 2012 data appear in volumes organized by themes. Volume II is entitled, PISA 2012 Results: Excellence through Equity, Giving Every Student the Chance to Succeed. Shanghai is named as one jurisdiction where schools “achieve high mathematics performance without introducing greater inequities in education outcomes (p. 28)” and one with “above average socio-economic diversity (p. 30).” In the 336 pages of this publication on equity, the word “migrant” appears only once, in a discussion of Mexico. The word “hukou” does not appear at all.

The OECD responded to Loveless' criticism by saying he is relying on old stereotypes of Shanghai, that things have changed there and migrant families are able to send their children to high school in Shanghai rather than having to send them back to schools in their home provinces.

But Valerie Strauss of the Washington Post countered that rebuttal with this:

I asked Post reporters who have covered China about Shanghai, and they said that despite the 2008 change in Shanghai policy, things have improved slightly for migrant families but not very much. Children who do not have Shanghai registration can now in theory get into Shanghai public schools, but only through very complicated and often arduous means that require either a brilliant kid, valuable inside connections or bribe money. (See this Washington Post story that talks about how Chinese parents pay bribes to get their children into top schools.) And while  the government recently announced that the hukou registration system is going to be reformed in the next few years, this does not mean families can suddenly get registered in big cities such as Shanghai.

So bear all this in mind the next time you see a statement like this:

If we don’t emulate China by establishing common nationwide education standards and implementing even more standardized testing, we will fall behind internationally. So goes the argument of Common Core advocates like Michelle Rhee.

“You need to reframe the debate,” Rhee said, speaking to Florida lawmakers in May. “This is about China kicking our butts. Do you want China to kick out butts? No!”

Even if the Chinese PISA scores were accurate - which they do not seem to be  - writer Mitchell Blatt calls into question the lessons Rhee claims we can take from them:

China’s education system is uniquely positioned to produce students who will do well on tests. It’s based on national standards, the knowledge of which is evaluated in a series of nationwide tests. Before entering high school, students take the zhong kao — the high school entrance exam — and before entering college, students study non-stop for a year to take the gao kao — the college entrance exam. Schooling is based heavily on tests and rote memorization.

So it’s no surprise that China did well on PISA.

The important question to ask is, Is China creating a competitive, innovative economy?
China has 19 percent of the world’s population, but in 2011, China only accounted for 9 percent of the world’s share of patent applications. The United States, by contrast, has 4 percent of the world’s population but accounted for 26.7 percent of the world’s patent applications in 2011, according to the World Intellectual Property Organization.

In China, the use of standards has fostered a high-pressure system that kills creativity.
“When [my son] Star was young, he was very imaginative,” Frank Chen, director of Asian operations for OnSpeX, said. “Now, he appears to be losing his creativity.”

“If he writes whatever he wants in an essay, he will get a bad grade, because the teachers do not like it. But there’s nothing to do about it. I want him to go to a good college. The teachers get bonuses if he goes to a good college.”

Because a student’s performance on the college entrance exam determines which college he goes to, teachers teach to the test, and there is a greater emphasis on memorization than on learning.

This is stifling creative innovation. After Steve Jobs died, Jobs’ biography became a best-seller in China. Chinese people asked the question, “Why doesn’t China have its own Steve Jobs?”

Former Google China CEO Kai-Fu Lee said on Sina Weibo, China’s most popular microblog, that China’s education system has prevented China from developing such innovators.

It is a folly to think that raising America’s international test scores will automatically improve America’s education quality or economic competitiveness. A paper by Christopher H. Tienken, an assistant professor at Seton Hall University, notes: “According to the World Economic Forum (Schwab 2009), the United States has ranked either 1st or 2nd consistently in economic competitiveness since 1998. … Keep in mind that that U.S. students never scored in the top two spots on any international test during this same period or any prior time.”

An analysis by Keith Baker, a retired researcher for the Department of Education, found that when comparing the United States to countries that outscored us on the FIMS test, the U.S., on average, outranked those countries in the categories of economic growth, quality of life, livability, democracy, and creativity.

Moreover, even if we do make scoring higher on international tests a goal of education reform, Common Core isn’t the right way to do it.

“There is no strong, or even mild, correlation — and certainly not a cause-and-effect relationship — between national standards and national performance on international tests,” Tienken’s report notes.

Right now, the Chinese are discussing how to make China’s education system less reliant on standardized tests and rote memorization. If Michelle Rhee really wants to learn from China, she should look to Chinese reformers. As they know, standardization doesn’t build a globally competitive workforce.

Whether China is cheating on the PISA or not, the United States should not be trying to emulate an education system that is plagued by inequity, stifles creativity, and creates such high pressure on students, parents and teachers that nobody feels safe enough to, as the cliche goes, "think outside the box" and try something different, innovative or new.

1 comment:

  1. Which might be another good reason why we don't want to mimic Shanghai!

    ReplyDelete