U.S. Math Education Still in the Doldrums

(Photo: arjin j)

(Photo: arjin j)

Every three years, the OECD, in the PISA assessment, studies 15-year-olds around the world to measure performance in reading, mathematics, and science. The results of the 2012 PISA assessment, which had a particular focus on mathematics, just came out and the United States does not fare well: “Among the 34 OECD countries, the United States performed below average in mathematics in 2012 and is ranked 26th.” I worry not so much about the rank, but about the low absolute level of proficiency to get this rank.

The U.S. students’ particular strengths and weaknesses are even more distressing:

Students in the United States have particular strengths in cognitively less-demanding mathematical skills and abilities, such as extracting single values from diagrams or handling well-structured formulae. They have particular weaknesses in items with higher cognitive demands, such as taking real-world situations, translating them into mathematical terms, and interpreting mathematical aspects in real-world problems.

Thirty-five years ago, the situation was similar. A large study of 13- and 17-year-old students across the United States, the second National Assessment of Education Progress (1977-1978), concluded: “Students appear to be learning many mathematical skills at a rote manipulation level and do not understand the concepts underlying the computation” [Carpenter, et al., Mathematics Teacher 73:329-338 (1980)]. One example was the question: “Estimate the answer to 12/13 + 7/8.” The choices were 1, 2, 19, 21, and “I don’t know.” Among 13-year-olds, 14 percent chose “I don’t know,” and only 24 percent chose 2. Among 17-year-olds, 16 percent chose “I don’t know,” and only 37 percent chose 2. The correct percentages are not much higher than the monkey line.

A frequent explanation for the U.S.’s poor PISA results is poverty — for example, by Daniel Wydo or (after the similar 2009 PISA results) by Stephen Krashen. That explanation is based on comparing wealthy U.S. schools (where fewer than 10 percent of students get free or reduce-price lunches) against the average score in countries with less poverty, such as Finland. The logic is superficially plausible, because Finland has only a few percent of students in poverty.

However, comparing wealthy students to averages of a country is a stacked comparison. The same authors who argue for the poverty explanation concede this point in other arguments, saying that it is unfair to let Shanghai’s PISA scores represent all of China, given that most of China is less wealthy than Shanghai. A fairer comparison is to compare comparable segments of the population.

That is what PISA did in its analysis of the effects of poverty. For example, see Figure II.2.6 (“Mean mathematics performance, by national quarter of socio-economic status”) on page 43 of Volume II of the PISA results (here is the data as a spreadsheet). Each country was divided into quartiles by socioeconomic status (SES), the average mathematics score computed by quartile, and then ranks computed for the top and bottom quartiles. The comparison is of top quartiles against each other, or of bottom quartiles against each other, rather than top quartiles against averages.

If poverty explained the U.S.’s poor performance, with scores of the bottom quartile dragging down and obscuring the alleged great performance of the U.S. top quartile, then the U.S. top quartile should do well internationally. However, when compared against the top quartiles in other countries, the U.S. rank drops from 26th to 32nd. Similarly, U.S. students with a SES at the OECD average do slightly worse than the U.S. average (Figure II.2.5 of the same report).

As a final example, look at the Finland and U.S. by-quartile comparison. In Finland, the bottom quartile has a SES of -0.68 (0.00 is the OECD average), and an average PISA math score of 488 (500 is the OECD average). In the United States, the second-highest quartile has an SES of 0.60 — they are much better off than the worst-off Finns — but almost the same average score, 494. Even though Finland has poor students, they do reasonably well.

In short, the “it’s poverty” explanation is not convincing. The usual U.S. math curriculum is simply weak, even in well-off schools. After decades of reform in the mathematics curriculum, students in the United States are still unprepared for full participation in society.


caleb b

As a volunteer math tutor (and student tutor in college, and Big Brother), all of my anecdotal experience (including my own HS education) has confirmed that US students are simply not receiving the rigorous math necessary to truly understand the subject. My current 9th grade students are paralyzed by fractions and do not understand order of operations. Isolating a variable is a constant struggle as they cannot remember the single most basic principle of algebra (just do the same thing to both sides of the equation).

It is hard to know where to place the blame. I imagine that No Child Left Behind is a likely culprit, as well as school policies that overwelming favor leniency. Teachers cannot give zeros. They must ALWAYS allow work to be turned in, no matter how late. Teachers are required BY LAW to call parents twice in order to give a student a failing grade…despite the fact that parents might not have a working number or speak english.

I also imagine that part of the problem is the cultural incentive structure of acquiring good math teachers. If I were to win the lottery, I’d drop out of the corporate world and teach algebra to inner-city kids. But since I won’t win the lottery, I’m not willing to take a massive pay cut and lose future earnings just to go back and teach. So the field of math teachers is comprised of those that are smart enough to make solid corporate money, but choose to forgo all that money for the love of teaching. I can imagine that this pool is faily limited.

One final thought…although each generation believes the next is the ruin of society, I think this generation of kids really ARE the ruin of society. They can’t stay off their phones for even a few minutes and their attention spans and long-term memory are almost non-existent. The smart kids are still smart, but the average kid tries 1/3rd as hard as 20 years ago and expects nothing lower than a B in every class.

Read more...

Taylor S. Marks

It's been my observation that nearly nobody who owns a smart-phone has the ability to leave it down - there is no generational divide in this, except that fewer elderly citizens own smart phones than younger citizens.

It seems to me what we need are better teachers. Which I would guess means being quicker to fire teachers that perform poorly and increasing the salary we offer to teachers, so that we might lure in better ones.

josh

The SES lower quartile of students in Finland really can't be compared to the SES lower quartile of students in the U.S. I don't have data in front of me, but I'd hypothesize the variability of poverty is the largest in the world so it's misleading to compare the countries this way. It's better than just looking at the countries but it's still flawed.

Chris Chan

I agree with what Josh said. There are several ways to measure poverty; what the OECD uses is the relative distance from a percentage of the median household income of a particular country. And there are several factors that contribute to well-being - and by extension, academic performance - that vary greatly among countries.

Dr. Bob

It's amazing what folks will publish and not carefully look at the research on poverty and achievement. You would think they would have learned their lesson on data analysis after the last presidential election.

James

I suggest that there may be a confusion of cause and effect: rather than poverty causing poor math scores,poor math scores cause poverty.

SC

Seems more like a "chicken or egg" situation to me...

James

Not really, because the ability to for instance comprehend simple* interest rate calculations can save you a whole bunch of money. Which, invested wisely, can make the difference between continued proverty and future prosperity.

*I mean that the calculations are simple, not that people shouldn't understand compound interest as well :-)

Julien Couvreur

Russ Robert of EconTalk had a recent episode on education, where he made an interesting observation about the expectations and culture of parents.
In the US, how do most parents supplement their kids education? Extra-curricular sports, arts or maths?
That is an indication of what they feel their child can benefit most from.

kip

a^4 + b^4 + c^4 = 37

Solved by an American!

RGJ

We are probably the most culturally mixed country on that list. It is much easier to educate a homogenous population of children than it is one from a mixed base of ethic and cultural influences. Just is. Mutts don't win the dog shows.

Paul Miller

I am a junior college mathematics instructor. It seems as though more and more colleges are adding remedial math courses to their curricula because fewer students are proficient enough to get into college level courses. One of the biggest discussions in community colleges is how we can increase retention without decreasing rigor. But as the skills of incoming freshmen seem to decline, that retention battle becomes harder and harder.

College enrollment data indicates that students who begin in remedial courses have a much lower graduation rate. Could it be that some colleges will go the other direction and cut remedial programs alltogether in favor of boosting retention numbers?

Many of my students ask why they have to learn math or comment that they won't remember any of it in a few years. Maybe so...but will there come a day that basic arithmetic is a skill ONLY possessed by STEM'ers? What would that look like? Does the ability to add fractions translate into real life success & prosperity?

Read more...

Jason

Number nitpick. Shouldn't the discussion of the change when top quartiles are compared read

… the US rank changes from 26th to 32nd ….

The 26th rank overall was mentioned in the first paragraph as was the fact that the study only compared 34 countries.

TedS

The poverty explanation is not exactly wrong - the data in this study show that poverty makes a difference - however, it is not as simple as people often assume. The United States has a problem with concentrated poverty - in certain areas everyone is poor and those who rise economically tend to move away. This has led to a subculture in those areas where education and other forms of investment in children becomes less common. The poor in Finland, by contrast, tend to be surrounded by more of a socioeconomic range, to their benefit. That was previously true in the US as well, but in the last several decades socioeconomic differences have increasingly been mirrored by physical separation (think urban vs. suburban).

Also, the author is correct that poverty can't explain the whole gap. Children in middle class American families tend to spend fewer and shorter days in school and complete less homework than children in other advanced first world countries. Unfortunately, increasing time spent on schoolwork costs money and parents don't particularly want it, so it's unlikely to change.

Read more...

mark

As a high school math teacher for the past 12 years, these findings are not surprising in the least. My students can plug numbers into formulas and memorize stuff all day long; unfortunately that is not what math is. In that respect, my coworkers and I have failed them as educators.

On the other hand though, we don't really have a lot of options. Teachers aren't free to teach what they feel is in the best interests of the students or even teach so that the students really understand concepts fully. Everything is test-score based these days, and these tests are not concept-based at all. They're multiple choice tests that for the most part test whether students can regurgitate information.

So as a public schoolteacher, I have two choices: I can teach what material will be covered on those tests and make sure my students do well on them, or I can spend a few months covering things that people really NEED to know (i.e. basic number sense, like the 12/13 + 7/8 example). I would prefer to do the latter, but if I did, my students would bomb the tests, and I would be out of job.

And if you think it's possible to do both, then you're overestimating the length of our school year, the attention span of today's youth, or both.

Read more...

Steve Nations

I'd say that the problem with all this is the last sentence of Sanjoy Mahajan's post: ". . . students in the United States are still unprepared for full participation in society." This is wrong. Clearly U.S. students are prepared. Because society doesn't care if they can do math -- at least society doesn't care about most of them doing math. Society -- the economy -- cares about other things.

Craig O'Brien

Sanjoy starts off by saying "I'm not really concerned about the rank," but then supports his argument by the rank. How did the performance of the top US quartile compare to the average performance? I'd download the data to check, but I don't have Excel on my home machine (it's always nice when data is provided in a non-proprietary format...) And is comparing the top quartile of a rich country to the top quartile of a poor country any better than comparing an entire rich country to an entire poor country? Doing so seems to imply that poverty is a relative phenomenon that respects political boundaries.

PittCaleb

So I call BS on this study. Rather, I want to change the e) I Don't Know to e) I don't care.

I have a 3rd grader who just turned 9 and an 12 year old 7th grader. I just performed this single test question for both of them. My 3rd grader too a few seconds, but once he reasoned it out, got the answer correct, with flawless logic. My 12 year old answered it immediately without hesitation.

So if my 9 year old can answer the question relatively quickly, the problem is not the public school system, but the nature of testing in our schools and how this study was performed. Yeah, my kids have it better than those in the inner cities, but seriously, we're going to say my 3rd grader is smarter than the average American HS graduate? This has as much to do with kids not caring, about these tests, their education, their society, as anything else to be honest.

The first commenter, sharing my name interestingly, claims HS students are "frozen by fractions." I will grant that possibly being true, but if it can be "mastered" by a 3rd grader, there is an issue with wanting to learn this rather than a complete break-down of the American Educational System.

Read more...