U.S. Math Education Still in the Doldrums

(Photo: arjin j)

Every three years, the OECD, in the PISA assessment, studies 15-year-olds around the world to measure performance in reading, mathematics, and science. The results of the 2012 PISA assessment, which had a particular focus on mathematics, just came out and the United States does not fare well: “Among the 34 OECD countries, the United States performed below average in mathematics in 2012 and is ranked 26th.” I worry not so much about the rank, but about the low absolute level of proficiency to get this rank.

The U.S. students’ particular strengths and weaknesses are even more distressing:

Students in the United States have particular strengths in cognitively less-demanding mathematical skills and abilities, such as extracting single values from diagrams or handling well-structured formulae. They have particular weaknesses in items with higher cognitive demands, such as taking real-world situations, translating them into mathematical terms, and interpreting mathematical aspects in real-world problems.

Thirty-five years ago, the situation was similar. A large study of 13- and 17-year-old students across the United States, the second National Assessment of Education Progress (1977-1978), concluded: “Students appear to be learning many mathematical skills at a rote manipulation level and do not understand the concepts underlying the computation” [Carpenter, et al., Mathematics Teacher 73:329-338 (1980)]. One example was the question: “Estimate the answer to 12/13 + 7/8.” The choices were 1, 2, 19, 21, and “I don’t know.” Among 13-year-olds, 14 percent chose “I don’t know,” and only 24 percent chose 2. Among 17-year-olds, 16 percent chose “I don’t know,” and only 37 percent chose 2. The correct percentages are not much higher than the monkey line.

A frequent explanation for the U.S.’s poor PISA results is poverty — for example, by Daniel Wydo or (after the similar 2009 PISA results) by Stephen Krashen. That explanation is based on comparing wealthy U.S. schools (where fewer than 10 percent of students get free or reduce-price lunches) against the average score in countries with less poverty, such as Finland. The logic is superficially plausible, because Finland has only a few percent of students in poverty.

However, comparing wealthy students to averages of a country is a stacked comparison. The same authors who argue for the poverty explanation concede this point in other arguments, saying that it is unfair to let Shanghai’s PISA scores represent all of China, given that most of China is less wealthy than Shanghai. A fairer comparison is to compare comparable segments of the population.

That is what PISA did in its analysis of the effects of poverty. For example, see Figure II.2.6 (“Mean mathematics performance, by national quarter of socio-economic status”) on page 43 of Volume II of the PISA results (here is the data as a spreadsheet). Each country was divided into quartiles by socioeconomic status (SES), the average mathematics score computed by quartile, and then ranks computed for the top and bottom quartiles. The comparison is of top quartiles against each other, or of bottom quartiles against each other, rather than top quartiles against averages.

If poverty explained the U.S.’s poor performance, with scores of the bottom quartile dragging down and obscuring the alleged great performance of the U.S. top quartile, then the U.S. top quartile should do well internationally. However, when compared against the top quartiles in other countries, the U.S. rank drops from 26th to 32nd. Similarly, U.S. students with a SES at the OECD average do slightly worse than the U.S. average (Figure II.2.5 of the same report).

As a final example, look at the Finland and U.S. by-quartile comparison. In Finland, the bottom quartile has a SES of -0.68 (0.00 is the OECD average), and an average PISA math score of 488 (500 is the OECD average). In the United States, the second-highest quartile has an SES of 0.60 — they are much better off than the worst-off Finns — but almost the same average score, 494. Even though Finland has poor students, they do reasonably well.

In short, the “it’s poverty” explanation is not convincing. The usual U.S. math curriculum is simply weak, even in well-off schools. After decades of reform in the mathematics curriculum, students in the United States are still unprepared for full participation in society.