Evidence on School Choice

Economists, long inspired by Milton Friedman and others, generally embrace the concept of school choice. But actual evidence on its efficacy has been thin.

A new working paper by Justine S. Hastings, Christopher A. Neilson, and Seth D. Zimmerman, using data from a low-income urban school district, offers some encouraging news for choice advocates:

[W]e use unique daily data on individual-level student absences and suspensions to show that lottery winners have significantly lower truancies after they learn about lottery outcomes but before they enroll in their new schools. The effects are largest for male students entering high school, whose truancy rates decline by 21% in the months after winning the lottery.

How do the authors interpret this finding?

We interpret this as students exerting more effort towards academics at their current school due to an increase in intrinsic motivation from knowing that they will be able to attend a school of their choice in the subsequent school year.

Furthermore, test scores seem to bolster the argument:

We then examine the impact attending a chosen school has on student test score outcomes. We find substantial test score gains from attending a charter school and some evidence that choosing and attending a high value-added magnet school improves test scores as well. Our results contribute to current evidence that school choice programs can effectively raise test scores of participants. Our findings suggest that this may occur both through an immediate effect on student behavior and through the benefit of attending a higher-performing school.


Bill

If there's a lottery involved, it's not choice. It's chance.

So lottery winners do better after winning the lottery? How about lottery losers? I don't think anyone doubted that winning the lottery wasn't beneficial to the winners.

Paul

There is only a lottery because there are not enough slots for all the students who applied. The lottery method is a better one to study to look for this effect since it is less biased. In other words, the charter schools are not just taking the "best performing" kids from the regular school. They are taking a random group from those who have applied. The effect here is not of the "lottery". The effect here is having one's choice confirmed, and then BELIEVING that you have a shot at improving yourself, so you work harder.

It seems same motivation as later in life. People work harder when they believe that their hard work will pay off with a chance to improve their situation (whether a promotion, a raise, or whatever).

It seems a good allegory between a "dead-end job", and a "dead-end education". Neither of these prospects inspire people to do their best.

Mike B

That's not entirely true. Students still have to have parents motivated enough to enroll them in the lottery which means said parents also care about that student's education. Charter schools don't siphon off the "best" students, but the students who have a commitment to education. The truth that school choice masks is that under the current system of crippling poverty and dysfunctional homes is that for some greater proportion of students to succeed, others must be left behind.

A system in which those that want to learn can learn is low hanging fruit that simply cannot scale beyond the point when the students who care about their education is exhausted. At that point you will either have the non-caring students concentrated in the failing schools or of said failing schools are closed, the problem children will be moved to non-failing schools prompting a new round of exodus.

Schools can't solve the myriad social problems that plague our poor communities. How much our society wants to spent to combat poverty is a matter for debate, but we shouldn't fool ourselves thinking that letting the deck chairs choose where they want to be shuffled will stop the boat from sinking.

Read more...

Alex in chicago

Hmm. More evidence that teachers really don't affect student performance that much.

Is anyone surprised?

Steve S.

No, choice is one of many factors - teachers included.

nobody.really

Haven’t read the paper, but the data in the abstract does not obviously support the conclusion in the abstract.

Classic problem: When two groups behave differently, is it because of selection bias (i.e., Members of each group were not picked randomly)? Or differential role expectations (e.g., A group of people picked up in a church may well be dressed differently than a group of people picked up at the beach – not because the two groups differ in any meaningful way, but only because people play different roles at church than at the beach)? Or because there is some causal difference between the two groups (e.g., Test scores for some schools are higher because the teachers are better)?

The abstract says that there was a lottery to get into a school. But where ALL people in the comparison group entered into the lottery, or (as is commonly the case) only the most motivated? That would create selection bias.

The abstract pretty clearly finds a change in role expectations, with absenteeism declining for kids that win the lottery – even though they had not yet entered the new school. And this complicates the abstract’s conclusion that “Our findings suggest that this [improved test scores] may occur both through an immediate effect on student behavior and through the benefit of attending a higher-performing school.” Perhaps another way to state the conclusion is that improved scores may occur from EITHER the immediate effect on student behavior (reduced absenteeism, etc.) OR attendance at the new school OR some combination of the two.

In short, it’s unclear from the abstract that the “higher-performing school” actually demonstrated that it is performs higher. For all I know, the increased test scores result purely from selection bias and/or changed role expectations.

Read more...

Steve S.

This is interesting, as it suggests that choice affects student motivation. Its important to consider the ways that choosing ones school would also affect the behaviors, dispositions and attitudes of parents whose children were accepted into the school of their choice.

The counter argument is that all schools are schools of choice; assuming families can choose where they live (thus, move to the neighborhood where the school matches your educational philosophy, curriculum, values, etc.). There's also the consumer psychology paradox between choice and happiness; surely a happy student would have greater motivation and achievement gains.

For full disclosure: I'm a choice advocate:)

Andreas

Free schools in sweden has lead to increased segregation.

http://www.guardian.co.uk/world/2011/sep/10/sweden-free-schools-experiment

Tom Fid

It seems like this could be evidence that choice works for the students who enroll in magnet schools, but it's not clear that that translates to a performance improvement for the system as a whole. There are certainly some reasons to think that it might, e.g., if the system tailors to individual needs better, but there are also reasons to think that this could be a selection effect, and that performance of non-participating students or schools would fall to compensate.

Paul M.

Never mind the fact that the lottery is a self-selecting system. These 14 year-olds aren't making a choice for a different school; These parents are choosing to put their child in another school. You mean when parents start participating in a child's education, truancy rates go down and test scores go up? Shocker.

So, while a lottery may remove selection bias on the part of the student, it exemplifies a selection bias towards parents. The parents most likely to be involved in their child's education are the most likely to enter the lottery. These means that every one of those lottery winners has a parent who entered them into that lottery and participates in their child's education more so than parents who didn't enter them in the lottery.

This doesn't show that charter/voucher/magnet schools work better, it just shows that increased parent participation in education generates better results. Not exactly a mind-blowing idea.

Read more...

EmW

Paul M. argues that the increase in child performance was due to parental involvement. However, the study showed the student scores went up after winning the lottery, but before entering the new school. Parental involvement was the same before and after this effect. It cannot be explained by parental involvement. The only thing that changed was they were going to a new school.

Paul M.

What changed was that the parents had a choice in where they sent their child to school. Parents made an investment of time and effort into their child's education and saw that it "paid off" by a lottery win. While the most involved would enter the lottery, it can be reasonably assumed that parents, encouraged by their "victory" in winning the lottery, would increase their participation because they felt like they were having an effect on their child's performance and opportunities.

This study was in low-income urban areas, meaning that these parents are often the ones who feel like they have the least control over the direction of their child's lives due to meager resources in both their homes and school districts (property tax funding of school districts is nearly universal in the US). With this victory in the lottery these parents are getting a "See, you can make a difference!" pat on the back and are being encouraged through this to involve themselves more.

Also, many of these charter schools that open spots through lotteries have attendance and performance guidelines to stay enrolled in those schools. If a parent worked to get their child into a school, and knew that they had to meet performance guidelines, lest their work be undone, these parents now have an active investment in their child's performance. It's in the best interest of the parents now to ensure that their investment of energy in this endeavor pays out its returns.

Maybe this means we need to find ways to encourage parental involvement into education, so that parents have a real investment in it, but it doesn't mean that just getting some good news is the game-changer.

Read more...

Andy

Likewise, I have not read the full paper (charging for access to academic papers is for a separate comment thread), but I don't see the clear evidence for school choice here.

The "winners" - whether through a lottery or selection criteria - do better than they would otherwise have done...seems intutive. However, does this mean better outcomes across the population? And what constitutes the best outcomes across a population. In the very long term, you could argue that the supply of schools ends up matching the range of demands, but the market here will be slow to respond and many kids would fall through the cracks when a poor quality school "fails".

Tony

At the risk of adding little except a "me too," I am also not surprised that the performance of lottery "winning" students improves (though I guess it's interesting that the performance gains start before the change in school), but I wonder if there is a net benefit to the system.

Eric M. Jones.

"... effects are largest for male students entering high school, whose truancy rates decline by 21% in the months after winning the lottery..." I suppose the real paper would clear this up, but ~5 truancies reduced to ~4 would not be very significant, now would it?

BTW, Did anyone else notice the grammatical errors in the abstract? Where did these guys go to school?

Stephen

Based on the marked improvements in measurements that quantify effort mentioned in the first section, I think you could reasonably argue that finding ways to increase motivation should be the goal. Simply reducing absenteeism without changing anything else about a school would almost certainly increase academic performance among those attending class more regularly.

It would appear that the sense of hope gained from seeing positive things in your future increases the effort put forth by the subject, even if you are still in the "bad" school. To me, this is evidence in support of hope more broadly. Give a person tangible hope for a better future and they are willing to work harder to make it happen. It's possible that the lottery-school was a better school, but it's known from the figures that the better school was receiving students with higher levels of motivation and effort, which would definitely help.

While school-choice isn't necessarily a bad thing, I think this paper shows that beyond finding good schools to replace bad ones, we can create better students (and in turn, better schools) if we can find a way to provide better incentives for kids and parents in under-performing schools.

Read more...

triclops

better bury this one quickly, the effectiveness of school choice gives the hoi polloi too many unseemly ideas.

Let's just pay teachers more and have more computers in classrooms, and keep doing what we are doing.

The Wall

Dismantling the public education system is intended to bolster the standings of failing charter schools whose dirty money has bought both fallen and falling politicians. This nationwide attack on schools is not about Unions, Wages, Benefits or Politics. It is about rewarding businesses which will soon be leaching from the taxpayers to reap profits while providing a dismal educational experience.

Clearly Temporary Governor Scott Walker defunded education to bolster the standings of charter schools, school choice and school vouchers (championed by Wisconsin's defrocked politician Scott Jensen) Yes that is what he is doing now - Fact, not wet dream. while charter schools fail to compete in results with fully funded public schools, the remedy was easy to implement, just cut funding to public schools until scores declined.

"School Vouchers", "School Choice" and "Charter Schools" are programs created to justify using tax dollars to pay for religious classes with tax dollars which is directly blocked by the constitution. They should call it "indirectly funding school prayer with tax dollars" and be truthful for once about their motives. Run by the evidence burning defrocked politician Scott Jensen whose stench permeates this movement.

Motive? Stripping tax dollars out of the education process.

Not to provide a better education for anyone - except the taxpayer who better be noticing by now. Quoting stacked "studies" and manufactured statistics hiring a slew of bloggers these tax siphoners have overplayed their hand and taxpayers and voters are shutting them down.

I knew, most now know - the rest might learn.

Read more...