More Evidence on the (Lack of) Impact of School Choice

There is no policy economists love more than school choice. Milton Friedman was an early proponent. The idea certainly makes sense: if parents have the ability to choose the best schools for their children, outcomes should improve through both the better matching of kids to specific schools and the resulting competition that would force schools to develop their programs.

The theory sounds great, but evidence confirming it has been hard to find. Julie Cullen and Brian Jacob, my good friends and co-authors, haven’t done school choice proponents any favors with their latest paper (the full version of which can be found here). Using kindergarten lottery outcomes that determine which kids get into the most sought-after schools, they are able to compare the outcomes of those who win the lottery versus those who lose. The students who win the lotteries go to “better” schools and have “better” peers, but they don’t have better outcomes. These results confirm the earlier findings that Julie, Brian, and I obtained when we examined the impact of lotteries on high school outcomes in Chicago.

Why don’t the kids who get access to “better” schools do better? That is a difficult question. Part of the answer is likely that the definition of “better” is based on outputs, like how high the test scores are at the school or what fraction of its students attend good colleges. That sort of metric ignores the fact that “better” schools tend to attract “better” kids. These are kids with strong families and good academic backgrounds. So even if the school is not at all good at adding value, it will still have the best outputs, because it had the best inputs. If the school does not have high value added, there is no reason to expect that a child who transfers there will do better than she did at her previous school. Parents don’t have good information on the inputs to a school, only the outputs, so it is difficult for them to accurately assess value added.

Also, I believe that people tend to systematically overstate the importance of peer effects, plus many parents are not choosing schools based on academics, but based on other criteria like convenience.

In thinking about the broader implications of this research, it is important to bear in mind that the school choice program that Julie and Brian analyze is just one kind of school choice (albeit the most common one), operating within a single public school system. It differs from voucher programs or school choice across school districts, and increased competition may be more effective in those settings.


severin

I am not sure how much of a "choice" we are talking about, it is a choice to move from one government school to another, but they use the same incentives, same curriculum, the teachers have the same training, the formula is the same from one government school to the next. It is not surprising in that situation the results indicate that the school you chose makes little impact.

mfw13

I'm working on my doctorate in Education and have done a fair amount of research on this topic, and of all the comments so far, Mike @ 12:30 hits closest to the truth.

For starters, research shows that the most accurate predictor of student achievement is parental education level. Children who come from families where education and knowledge are valued and prioritized achieve at much higher levels that children who do not.

This leads to the one major complaint that most professional educators have with politicians trying to make education policy; politicians overemphasize the importance of the school and unemphasize the importance of strong parenting. Even the best schools have great difficulty effectively teaching students who suffer from poor parenting. Everyone seems to forget that children actually spend more time each day out of school than in school. Yet when students fail to achieve at expected levels, it's usually the school system that is faulted, not the parents.

I would also like to point out that the whole argument behind school choice rests on a bit of a fallacy, namely that poor schools will only try to improve if the free market forces them to. This completely ignores the fact that there are many other factors besides systemic willpower that keep poor schools from improving, the primary ones being a lack of quality teachers nationwide, lower levels of funding for schools with low property tax bases, the huge numbers of ESL students straining instructional resources, and the huge number of children being raised by single parents.

On a larger level, this all is reflective of the fact that we place primary responsibility for educating children on the school system, and to a large extent ignore the importance of good parenting.

Read more...

Adolfo

The idea to choose schools does not require an improvement in the output... the simple fact that parents have freedom to choose improve the welfare.

E Olson

So if the kids are less affected, what about the schools?

Can we infer that these kindergarten lotteries would lower the scores of "good" schools and maybe raise the scores of the bad schools - pulling the overall school ratings toward the average?

Ron

Sorry, Stephen, but these and other papers don't have anything to do with school choice. They DO relate to the idea that what we think of as better schools may not have better results.

You are smart enough to know that you are taking data on one subject and trying to infer that it implies another. It doesn't.

I lose respect for people who distort data for their own preconceived ideas, particularly academics. I have completely lost respect for you.

On the other hand, you are now particularly qualified to work as a commentator for Fox News.

Bob.

School choice is only part of the solution. Improve education with merit based pay for teachers based on how well they teach their students.

Test the students at the beginning of the year, and at retest them on the same stuff at the end of the year. Find the delta, and I bet after a few years you'd have a pretty good idea of who the great teachers are and who the poor ones are and then you pay them accordingly.

me

I am thinking that readers might have had a clearer understanding of the outcome of the studies if you placed your last paragraph up a bit more in your article.

Opinions would not have been formed by then.

htb

It's more complicated than most of the respondents are willing to admit. For example, one of the most important predictors of academic success for a poor child (qualifies for reduced/free lunch) is -- how few other poor kids there are. Apparently if you put a poor kid (e.g., me, 20 years ago) in a school with a bunch of middle-class kids, then the poor kids do better. If you put the same kid in a school with a bunch of poor kids, then the poor kids do much, much worse. The wealthier school's average might decline very slightly, but on the whole, everyone in the state ends up doing better. Apparently it's much easier to bring three kids out of 25 up to speed than to do exactly the same thing with 22 kids out of 25. The concentration effect really matters (and hugely for English language learners).

Allen

Dear Ron (1:57pm), you spelled Stephen (sic) incorrectly. It is spelled with a v.

You are smart enough to see the difference between the ph and the v.

I lose respect for people who distort data for their own preconceived ideas. I have completely lost respect for you.

My argument makes as much sense as yours does, so don't try and sit in your ivory tower and try to claim that you are free from preconceived ideas.

mathking

Ah, an education debate on the Freakonomics blog. We haven't had one of those in such a long time.

While I agree with much of what mfw13 says about setting policy, as a teacher myself I also have to say "We work with what we have." As a teacher, I need to do the best with what I have.

I also note that proponents of school choice who think it is the key to solving all of the nation's educational ills are all about test scores showing how bad public schools are. But the minute test scores fail to show that chareter schools or private schools are better, all the excuses about the limitations of testing come out.

Dr. Camplin and I went back and forth a while on the subject of improvement in a previous thread (which lead to some interesting ideas for me to ponder) but I want to bring it up here in the context of measuring good schools. We do a LOT of testing now and we should be using some of that data to see which schools do a good job of adding value to student's educations. Improvement is not enough to insure that a student is educated, but the schools that consistently show the most value added are clearly where we should be looking to find out what works.

As for the argument that schools should be run like a business, well, it's just wrong. The purposes are different. The incentives for the involved parties are different. This is not to say that many school districts couldn't benefit from some business management skill and fiscal knowledge.

Read more...

David

mfw13 (#22),

Poor schools can either improve, or they can close. There goes your "fallacy."

I agree with you and everyone who says that good parenting is the number one factor. So why strip them of their freedom to pursue the best options for their kids?

misterb

Education is one of those goods that may or may not be best served by a free market. I'll admit that I don't know what's best for the kids. Certainly, nowadays even Communists believe that lead-covered toys are best served by a free market, but education has traditionally been considered a public good. The real question is: what are the "goods" in education? Is society's main benefit that little hooligans are kept off the street? Do citizens of a successful democracy require knowledge, skepticism and a full knowledge of how the world works to choose our best representatives?
Is education's responsibility to turn out good little worker bees to fill all the mcJobs that robots can't replace? Until we can decide what we are trying to achieve, we can't decide what we are testing for. And if we're testing for the wrong things, then constructing an educational system around passing tests is subverting our society's goals.

Read more...

Martin

Why choosing your child's school doesn't result in the 'better grades' you expected:

Think about it... any teacher will tell you that smaller class sizes are the key; allowing for more individualized instruction and a higher percentage of resource-usage per student (at least in that classroom).

One thing parents look for... from elementary school to college... the teacher-to-student ratio. The higher the number, the less appealing to both parent and student... and teacher, for that matter. So, if the 'pro-choice' (sorry, but that term seems appropriate here) crowd sends Johnny and Janie to the school with the lower ratio, suddenly the mix is thrown off. Teachers have more students in the room... and less time to spend with each... resulting in... you guessed it... lower than anticipated grades.

Forget about picking the 'right' school... what we need is a balance of funding for all public schools. And it's not just a problem in low-income areas; two separate school buildings within the same district often have significant differences in both class-size and resources. If we could come up with a plan that results in equitable funding for all students... then we'll see grades go up.

Read more...

Charlie Evett

Clearly we should be paying the students for achieving high test scores, that would give them and their parents incentive to succeed.

Ann

I can see what htb is saying with my own child's school. The free/reduced lunch rate is up to 20% which seems pretty high to me, but I guess that's really pretty good. The school is clean and well-kept, with lots of involved parents, a strong teaching staff and a good administration. So we're lucky. My little 2nd grader has been innundated with books for her entire life and her reading lexile (some sort of text comprehension measurement) puts her into about the mid-fourth grade. So her teacher paired her with a little girl who was struggling so much in reading that she hadn't been able to be scored. My daughter was so excited the other day when she told me that her reading buddy, Mariah, now had a score of over 150 points and her teacher was so proud of them that she gave them both popcorn as a special treat! I'm proud of my daughter too and it's absolutely wonderful that the world of reading is opening up for Mariah. But what about all those classrooms that are full of Mariahs? Where are their reading buddies?

Read more...

David

Regarding the comments about teaching only to take the test ... I think many things could be done to improve the quality of tests, for example adding writing sections for English. Math isn't as open to memorization, and where it is, it's not all bad ... I assume everyone still uses their times tables.

Tests work you just need to improve on them; let's not throw out the baby with the bath water.

Mike

IQ?

scott cunningham

In Maria Ferreyra's June 2007 AER article "Estimating the Effects of Private School Vouchers in Multidistrict Economies," she finds somewhat different results. The abstract reads,

"This paper estimates a general equilibrium model of school quality and household residential and school choice for economies with multiple public school districts and private (religious and nonsectarian) schools. The estimates, obtained through full-solution methods, are used to simulate two large-scale private school voucher programs in the Chicago metropolitan area: universal vouchers and vouchers restricted to nonsectarian schools. In the simulations, both programs increase private school enrollment and affect household residential choice. Under nonsectarian vouchers, however, private school enrollment expands less than under universal vouchers, and religious school enrollment declines for large nonsectarian vouchers. Fewer households benefit from nonsectarian vouchers."

Has anyone (on here) familiar enough with the school choice literature to talk about these differing results? I heard Ferreyra give this paper many years ago and found it very compelling and interesting. I was not, at the time, familiar with the empirical methods she used, though. Can others who work in this area possibly speak to whether these are in tension with the Cullen piece that Dr. Levitt is referring to (as well as his Levitt's own Econometrica piece recently published finding no impact of lottery-based voucher systems on the educational outcomes)?

For instance, Ferreyra's is, ultimately, a long-run model - in her model, because school choice unhinges the residential choice from the school choice, there's a sorting effect that occurs, which might not occur in the kinds of school choice experiments that Cullen and Levitt have analyzed.

Read more...

Penny

My children went to elementary school in a system that allowed school choice, Richmond Unified School District in Richmond, California (it later went bankrupt and is now called West Contra Costa USD).

This school district ecompassed some areas of urban poverty and some areas of suburban middle class. The idea was to get the kids from poor areas into better schools. It didn't work out that way.

The practical effect of school choice was that all the students with parents who could and did research schools ended up in the good schools, and all the kids with parents who couldn't be bothered or didn't know how to find and evaluate school performance data ended up in the cruddy schools.

I spent a couple of hours on the bus every day taking my kids to a good school. When my son was in kindergarten I went to the school and back on the bus three times a day. How many parents can or will do that?

Parent involvement is very important, and better schools often have parents who have the time, inclination, and know-how to help out at school and help their kids be good students.

The poor kids will still have parents who are too tired, busy, or uneducated to help out no matter what school they go to.

School choice is a a red herring. The real determining factor in a kid's outcome is not the quality of the school, it's the quality of the parents. I know that sounds horrid of me to say, but unless we face the reality and find a way to make up for the deficits of some parents, nothing will change. Maybe extra classroom staff for schools with poor test grades? Give bonuses to good teachers who transfer to those schools?

Read more...

loBob

Value Added was mentioned several times -
are there any cities or or states that really do value added calculations for their schools ? I don't mean in a half arsed way, I mean keeping track of each student's progress and comparing it to that of their peers - their peers being the group of students in the state that are matched academically, socially, economically, age within 6 months, diagnosed disabilities - ALL of the factors they can get data on that can be shown to affect the variance in scores. When this is done year after year, you can find schools and teachers whose students do better than expected after controlling for factors outside of school - and find those whose students do worse. I know some schools do it, but I don't think the results would be much better than noise unless you were following at least several thousand kids at each grade level.

As long as we are stuck with NCLB, this is the type of score each school should get.

Read more...