More Evidence on the (Lack of) Impact of School Choice

There is no policy economists love more than school choice. Milton Friedman was an early proponent. The idea certainly makes sense: if parents have the ability to choose the best schools for their children, outcomes should improve through both the better matching of kids to specific schools and the resulting competition that would force schools to develop their programs.

The theory sounds great, but evidence confirming it has been hard to find. Julie Cullen and Brian Jacob, my good friends and co-authors, haven’t done school choice proponents any favors with their latest paper (the full version of which can be found here). Using kindergarten lottery outcomes that determine which kids get into the most sought-after schools, they are able to compare the outcomes of those who win the lottery versus those who lose. The students who win the lotteries go to “better” schools and have “better” peers, but they don’t have better outcomes. These results confirm the earlier findings that Julie, Brian, and I obtained when we examined the impact of lotteries on high school outcomes in Chicago.

Why don’t the kids who get access to “better” schools do better? That is a difficult question. Part of the answer is likely that the definition of “better” is based on outputs, like how high the test scores are at the school or what fraction of its students attend good colleges. That sort of metric ignores the fact that “better” schools tend to attract “better” kids. These are kids with strong families and good academic backgrounds. So even if the school is not at all good at adding value, it will still have the best outputs, because it had the best inputs. If the school does not have high value added, there is no reason to expect that a child who transfers there will do better than she did at her previous school. Parents don’t have good information on the inputs to a school, only the outputs, so it is difficult for them to accurately assess value added.

Also, I believe that people tend to systematically overstate the importance of peer effects, plus many parents are not choosing schools based on academics, but based on other criteria like convenience.

In thinking about the broader implications of this research, it is important to bear in mind that the school choice program that Julie and Brian analyze is just one kind of school choice (albeit the most common one), operating within a single public school system. It differs from voucher programs or school choice across school districts, and increased competition may be more effective in those settings.


David

Sorry, Johhny. You have to stay in the crumbling, failing, dangerous public school in your neigborhood because your freedom of choice is not statistically justified.

Aaron

I don't think it matters. Monopolies are bad. Perhaps merely having the "choice" available (in your lottery example above) raises the bar for everybody.

Competition in any space isn't about making one product obviously better than the other, it's about better serving customers specific needs and improving the quality of the entire market.

Ryan Parks

I would argue that if the net cost to tax payers doesn't increase substantially then parents and students should be afforded choice on principal alone. Also, I remain skeptical of these studies ability to measure non-academic dimensions of one school versus another.

Roger

When do we start holding these school systems accountable, by privatizing them and running them like businesses. How can we expect to hold students accountable each and every day if we don't do the same with the school board and administrators?

Chris

David, Johnny's parents have absolute freedom of choice to not live in the district that their child will have to attend. Who is forcing them to live there? Move. You're creating a restriction that doesn't exist.

Walter Wimberly

I've never like lotto type selection systems, because it can negatively effect the selected school...e.g. someone wants their kid to go to "The Good Silver Bullet Solution High School". Now Good High happens to have parents who know that staying in school, preparing for college, personal discipline etc. are all important to having good jobs and being successful, and have been training their kids in such manner since they could walk and talk. Enter into that mix a group of kids who don't have the discipline, but have the luck of the lotto - then what happens. Either a) the new kids can't keep up and still continue to do poor, b) the new kids "slow down" all of the other kids, or c) both.

I agree that the parents/guardians influence on the kids to get them to do well has more effect than the school as a whole - and without that influence, the kids won't do well regardless of what school they go to.

So has any studies been done to show what happens to overall "achievement" when unprepared students are added to the population?

Read more...

Dan

Just a quick thought on this: what if "better" was measured differently? What if "better" was measure by the type of wages earned in job or likelihood of attending college?

The reason I say this is that, while the better school may not directly affect the short term metrics like test score, it may have effect on longer term metrics. I can envision someone who, through attending a better school, gets a better job or goes on to higher education, because of the experience in a better school, regardless of whether their test scores increase.

a teacher

Choice is made-- by choosing the community a parent lives in. Communities are a reflection of common values and expectations and schools reflect the community. Usually, one chooses to remain in a community of shared values or move to a new one-- and his or her children will be indoctrinated in the same values. Believe it or not, not all parents care to give their children the best opportunities for a stellar education. It's the elephant in the room when discussing school choice.

Donut

Rich people have school choice: They pick which private school to send their children.

Middle class people have school choice: They choose where they live in a city based on the quality of the school system, or suck it up and eat beans and drive old cars so the kids can go to private school. Ahem.

Poor parents (who care) lack both those choices. They cannot afford private school, and cannot afford to live in the nice neighborhoods where the schools are better. They are disproportionately stuck with poor performing schools. I am amazed and frustrated at how this voting bloc allows their elected officials to do this to them.

So, is school-choice opposition based on economic class bias? I see it above, in the "just move" arguments...

BTW, every educator knows that the quality of parents is THE single indicator for educational success. The act of caring enough to participate in a lottery signals that the parents involved care more than normal.

Read more...

jon

Ah, education. This is one of my hobby horses.

In my opinion, the lack of universally available , high quality education, is the main problem with Market driven society. I have no data for this, but my logic is:

A market is great, because a more effective worker gets more money, and therefor more choice in society (money == Unit of social choice as my engineering Philosophy lecturer told me).

However, assuming that an equal student with a better education is a more effective worker (no proof, but is 'seems right'), then better education gives better social choice.

IF better education can be bought by the social winners (rich) then there becomes a tendency for families and areas to accumulate social choice.

This means that the origional statment above changes to 'those born to families with more social choice will have more social choice'.

If you think about it, it was the ultimate free market (no governments) in early times that led to the rise of the upper classes, and the monarchy.

So, to stop this happening, and to maintain a fair society, this inter-generational accumulation of social choice needs to be limited, mostly by having the government provide 'the best' education to everyone. I imagine that a higher inheritance tax could be used to do this.

Would never get voted in of course.

Read more...

tkw

let's say you wanted to do a study on entree choice. you get a restaurant full of people, hold a lottery. the winners get the lobster, the losers get the pasta, and everyone's satisfaction is measured.

is this a fair way to study the effect of choice in restaurants?

Marco

I think Dan put his finger on a bigger problem ... how do you measure success?

Test scores seem like a bad proxy; people score well and don't succeed at life; and people score poorly and 'succeed'. Then again, you can't easily measure success.

I believe you'd find a low correlation between primary/secondary school test scores and income/success/'happiness' (if such a thing can really be measured). So why are we testing these things anyway? Shouldn't we first establish (if possible), what the influencers are of happiness/success?

Dave

What I don't understand is why they don't give tests before and after each semester? Seems like you could do a pretty good job of assessing how every teacher does.

My brother teaches and said from year to year his students differ substantially in their abilities; yet when you only test outcomes you never capture the effectiveness of the teacher/school.

Am I missing something ... besides that fact that this type of law being passed (against teacher union objections) stands about the same chance as @10's inheritance tax being passed.

Steven

But doesn't this experiment violate your hypotheses? You propose that it is matching students to schools that affords a benefit, the example you give of lotteries doesn't really afford parents choice, it's more like a random controlled trial assigning participants to one condition or another. Furthermore, this system fails to foster competition in the market due to the same reason, parents aren't really choosing the school of their choice.

Hence, I don't think this example (albeit I haven't read the article, just going on your summary) refutes the argument at all.

David

Amen Donut (#9).

Poverty is a restriction that exists. "Just move" is a callous and selfish response.

Dr. Troy Camplin

Well, what do we mean by a "better" school. Different does not mena better. Here in Texas, where there are charter schools, students do have a choice, but the fact is that many of the charter schools are not in fact better precisely because they still have to abide by all the laws that help to make our schools miserable failures -- like the kinds of standardized testing that results in students learning how to pass the test rather than actually learning anything of substance. My wife is a K teacher, and she has to get the students to simply repeat back to her what will be on the test, whether they understand what they are doing or now. And she tells me, with great frustration, that they do not. I ran into the same problem when I taught at a charter school for a while. I taught English, and we were told that we had to teach test-taking strategies, and that we couldn't teach either poetry or novels because they wouldn't be on the test. So as long as the alternative schools have to abide by the same rules as the public schools, we should not be surprised to see little difference.

That having been said, I read several months ago in the journal Science about a study done of Montessori schools which showed that those students did considerably better than public school students in just about every area, from academics to behavior. The difference? Curriculum.

Read more...

Mike

The real problem with school choice is that it doesn't fix the biggest barriers to learning (and thus performance): parents, discipline, and lack of resources. Before people jump down my throat about the last point, just answer the question, at what other professional job (most states require a college education for a teaching position) do you have to pay for your own materials?

Also, no matter what a school does, the biggest indicator of child success is where they are even BEFORE school starts. Expecting magic from schools is a genuinely unreasonable expectation. The problem is that most schools end up expending an extreme amount of time, money and effort on remedial work to the detriment of everyone else.

Will

Boy, so much hating on the data!

Commenteers, rather than trying to find flaws in the studies that can allow you to continue holding your treasured policy position, why don't you keep an open mind and think through the possibility that, in fact, "school choice" might not be a panacea for public education?

I don't know what's the right policy answer (I could hardly know less about this subject) but it's very un-freakonomics to close one's mind to what appears to be reliable data. My question is "knowing this, what can we learn and what can researchers further study to learn even more?"

Josh

Steven, it's unfortunate that you mention this study and Milton Friedman's name in the same post. While the study provides useful information in some contexts, it does not include any of the things Friedman talked about while fighting for school choice. There is no competition among schools; all schools will be fully funded regardless of outcomes; private schools were not included; parents do not get to choose their top school for their child.

All of these things were critical to Friedman's position, which is based on the premise that competition (rather than monopoly) improves the product.

Silvanus

What's interesting to me... is that most people who tend to argue for privatization also argue for individual responsibility (at least in my lived experience with people I actually meet)... yet they don't recognize the disconnect.

Education is primarily self motivated. You can lead a kid of Yale, which has the best resources of any university in the land- with a great library, great faculty, etc... but you can still get a really sloppy thinker and someone who isn't intellectually curious.

The school is a collection of educational resources- it is up to the student to make use of those resources. I say this with the knowledge that I attended both public and private schools for my K-8, 11-12, undergraduate, masters and now PhD education. I "succeeded" in all my classes despite the business model of the institution. Why? Because schools are not factories... education is not a tradeable commodity.

Read more...