Evidence on School Choice

Economists, long inspired by Milton Friedman and others, generally embrace the concept of school choice. But actual evidence on its efficacy has been thin.

A new working paper by Justine S. Hastings, Christopher A. Neilson, and Seth D. Zimmerman, using data from a low-income urban school district, offers some encouraging news for choice advocates:

[W]e use unique daily data on individual-level student absences and suspensions to show that lottery winners have significantly lower truancies after they learn about lottery outcomes but before they enroll in their new schools. The effects are largest for male students entering high school, whose truancy rates decline by 21% in the months after winning the lottery.

How do the authors interpret this finding?

We interpret this as students exerting more effort towards academics at their current school due to an increase in intrinsic motivation from knowing that they will be able to attend a school of their choice in the subsequent school year.

Furthermore, test scores seem to bolster the argument:

We then examine the impact attending a chosen school has on student test score outcomes. We find substantial test score gains from attending a charter school and some evidence that choosing and attending a high value-added magnet school improves test scores as well. Our results contribute to current evidence that school choice programs can effectively raise test scores of participants. Our findings suggest that this may occur both through an immediate effect on student behavior and through the benefit of attending a higher-performing school.

Leave A Comment

Comments are moderated and generally will be posted if they are on-topic and not abusive.

 

COMMENTS: 25


  1. Bill says:

    Hidden due to low comment rating. Click here to see.

    Disliked! Like or Dislike: Thumb up 14 Thumb down 25
    • Paul says:

      There is only a lottery because there are not enough slots for all the students who applied. The lottery method is a better one to study to look for this effect since it is less biased. In other words, the charter schools are not just taking the “best performing” kids from the regular school. They are taking a random group from those who have applied. The effect here is not of the “lottery”. The effect here is having one’s choice confirmed, and then BELIEVING that you have a shot at improving yourself, so you work harder.

      It seems same motivation as later in life. People work harder when they believe that their hard work will pay off with a chance to improve their situation (whether a promotion, a raise, or whatever).

      It seems a good allegory between a “dead-end job”, and a “dead-end education”. Neither of these prospects inspire people to do their best.

      Well-loved. Like or Dislike: Thumb up 18 Thumb down 0
      • Mike B says:

        That’s not entirely true. Students still have to have parents motivated enough to enroll them in the lottery which means said parents also care about that student’s education. Charter schools don’t siphon off the “best” students, but the students who have a commitment to education. The truth that school choice masks is that under the current system of crippling poverty and dysfunctional homes is that for some greater proportion of students to succeed, others must be left behind.

        A system in which those that want to learn can learn is low hanging fruit that simply cannot scale beyond the point when the students who care about their education is exhausted. At that point you will either have the non-caring students concentrated in the failing schools or of said failing schools are closed, the problem children will be moved to non-failing schools prompting a new round of exodus.

        Schools can’t solve the myriad social problems that plague our poor communities. How much our society wants to spent to combat poverty is a matter for debate, but we shouldn’t fool ourselves thinking that letting the deck chairs choose where they want to be shuffled will stop the boat from sinking.

        Well-loved. Like or Dislike: Thumb up 26 Thumb down 8
      • Enter your name... says:

        “Students still have to have parents motivated enough to enroll them in the lottery which means said parents also care about that student’s education…” but in this experiment, 100% of the students had that kind of parent.

        The two groups in this study are:

        (1) Students who have parents motivated enough to enroll them in the lottery because they care about that student’s education AND who happened to win the enrollment lottery, and

        (2) Students who have parents motivated enough to enroll them in the lottery because they care about that student’s education AND who did NOT happen to win the enrollment lottery.

        Since this is a straightforward, single-variable, randomized, controlled experiment, we assume that winning the enrollment lottery is responsible for the differences seen between the two groups.

        Well-loved. Like or Dislike: Thumb up 29 Thumb down 1
      • Mike B says:

        I didn’t say it wasn’t, the problem is the conclusion that allowing all students to choose their schools will increase the results for all students. When you allow students who want a better education to get a better education and escape those they don’t care about education they do better. Duh!

        What the study masks is that the improvement is caused by one group of students being allowed to self separate from another group of students. I don’t necessarily want to pass judgement on this approach, but people need to realize that it amounts to a “Some Children Left Behind” policy.

        Thumb up 4 Thumb down 6
  2. Alex in chicago says:

    Hmm. More evidence that teachers really don’t affect student performance that much.

    Is anyone surprised?

    Hot debate. What do you think? Thumb up 14 Thumb down 17
  3. nobody.really says:

    Haven’t read the paper, but the data in the abstract does not obviously support the conclusion in the abstract.

    Classic problem: When two groups behave differently, is it because of selection bias (i.e., Members of each group were not picked randomly)? Or differential role expectations (e.g., A group of people picked up in a church may well be dressed differently than a group of people picked up at the beach – not because the two groups differ in any meaningful way, but only because people play different roles at church than at the beach)? Or because there is some causal difference between the two groups (e.g., Test scores for some schools are higher because the teachers are better)?

    The abstract says that there was a lottery to get into a school. But where ALL people in the comparison group entered into the lottery, or (as is commonly the case) only the most motivated? That would create selection bias.

    The abstract pretty clearly finds a change in role expectations, with absenteeism declining for kids that win the lottery – even though they had not yet entered the new school. And this complicates the abstract’s conclusion that “Our findings suggest that this [improved test scores] may occur both through an immediate effect on student behavior and through the benefit of attending a higher-performing school.” Perhaps another way to state the conclusion is that improved scores may occur from EITHER the immediate effect on student behavior (reduced absenteeism, etc.) OR attendance at the new school OR some combination of the two.

    In short, it’s unclear from the abstract that the “higher-performing school” actually demonstrated that it is performs higher. For all I know, the increased test scores result purely from selection bias and/or changed role expectations.

    Thumb up 6 Thumb down 7
  4. Steve S. says:

    This is interesting, as it suggests that choice affects student motivation. Its important to consider the ways that choosing ones school would also affect the behaviors, dispositions and attitudes of parents whose children were accepted into the school of their choice.

    The counter argument is that all schools are schools of choice; assuming families can choose where they live (thus, move to the neighborhood where the school matches your educational philosophy, curriculum, values, etc.). There’s also the consumer psychology paradox between choice and happiness; surely a happy student would have greater motivation and achievement gains.

    For full disclosure: I’m a choice advocate:)

    Thumb up 3 Thumb down 1
  5. Andreas says:

    Free schools in sweden has lead to increased segregation.

    http://www.guardian.co.uk/world/2011/sep/10/sweden-free-schools-experiment

    Thumb up 2 Thumb down 0
  6. Tom Fid says:

    It seems like this could be evidence that choice works for the students who enroll in magnet schools, but it’s not clear that that translates to a performance improvement for the system as a whole. There are certainly some reasons to think that it might, e.g., if the system tailors to individual needs better, but there are also reasons to think that this could be a selection effect, and that performance of non-participating students or schools would fall to compensate.

    Thumb up 2 Thumb down 1
  7. Paul M. says:

    Never mind the fact that the lottery is a self-selecting system. These 14 year-olds aren’t making a choice for a different school; These parents are choosing to put their child in another school. You mean when parents start participating in a child’s education, truancy rates go down and test scores go up? Shocker.

    So, while a lottery may remove selection bias on the part of the student, it exemplifies a selection bias towards parents. The parents most likely to be involved in their child’s education are the most likely to enter the lottery. These means that every one of those lottery winners has a parent who entered them into that lottery and participates in their child’s education more so than parents who didn’t enter them in the lottery.

    This doesn’t show that charter/voucher/magnet schools work better, it just shows that increased parent participation in education generates better results. Not exactly a mind-blowing idea.

    Thumb up 8 Thumb down 8
    • EmW says:

      Paul M. argues that the increase in child performance was due to parental involvement. However, the study showed the student scores went up after winning the lottery, but before entering the new school. Parental involvement was the same before and after this effect. It cannot be explained by parental involvement. The only thing that changed was they were going to a new school.

      Well-loved. Like or Dislike: Thumb up 16 Thumb down 2
      • Paul M. says:

        What changed was that the parents had a choice in where they sent their child to school. Parents made an investment of time and effort into their child’s education and saw that it “paid off” by a lottery win. While the most involved would enter the lottery, it can be reasonably assumed that parents, encouraged by their “victory” in winning the lottery, would increase their participation because they felt like they were having an effect on their child’s performance and opportunities.

        This study was in low-income urban areas, meaning that these parents are often the ones who feel like they have the least control over the direction of their child’s lives due to meager resources in both their homes and school districts (property tax funding of school districts is nearly universal in the US). With this victory in the lottery these parents are getting a “See, you can make a difference!” pat on the back and are being encouraged through this to involve themselves more.

        Also, many of these charter schools that open spots through lotteries have attendance and performance guidelines to stay enrolled in those schools. If a parent worked to get their child into a school, and knew that they had to meet performance guidelines, lest their work be undone, these parents now have an active investment in their child’s performance. It’s in the best interest of the parents now to ensure that their investment of energy in this endeavor pays out its returns.

        Maybe this means we need to find ways to encourage parental involvement into education, so that parents have a real investment in it, but it doesn’t mean that just getting some good news is the game-changer.

        Well-loved. Like or Dislike: Thumb up 7 Thumb down 2
      • Steve S. says:

        Who is to say that the parents didn’t become more involved upon the results of the lottery?

        Thumb up 0 Thumb down 1
  8. Andy says:

    Likewise, I have not read the full paper (charging for access to academic papers is for a separate comment thread), but I don’t see the clear evidence for school choice here.

    The “winners” – whether through a lottery or selection criteria – do better than they would otherwise have done…seems intutive. However, does this mean better outcomes across the population? And what constitutes the best outcomes across a population. In the very long term, you could argue that the supply of schools ends up matching the range of demands, but the market here will be slow to respond and many kids would fall through the cracks when a poor quality school “fails”.

    Thumb up 2 Thumb down 2
  9. Tony says:

    At the risk of adding little except a “me too,” I am also not surprised that the performance of lottery “winning” students improves (though I guess it’s interesting that the performance gains start before the change in school), but I wonder if there is a net benefit to the system.

    Thumb up 1 Thumb down 1
  10. Eric M. Jones. says:

    “… effects are largest for male students entering high school, whose truancy rates decline by 21% in the months after winning the lottery…” I suppose the real paper would clear this up, but ~5 truancies reduced to ~4 would not be very significant, now would it?

    BTW, Did anyone else notice the grammatical errors in the abstract? Where did these guys go to school?

    Thumb up 1 Thumb down 2
  11. Stephen says:

    Based on the marked improvements in measurements that quantify effort mentioned in the first section, I think you could reasonably argue that finding ways to increase motivation should be the goal. Simply reducing absenteeism without changing anything else about a school would almost certainly increase academic performance among those attending class more regularly.

    It would appear that the sense of hope gained from seeing positive things in your future increases the effort put forth by the subject, even if you are still in the “bad” school. To me, this is evidence in support of hope more broadly. Give a person tangible hope for a better future and they are willing to work harder to make it happen. It’s possible that the lottery-school was a better school, but it’s known from the figures that the better school was receiving students with higher levels of motivation and effort, which would definitely help.

    While school-choice isn’t necessarily a bad thing, I think this paper shows that beyond finding good schools to replace bad ones, we can create better students (and in turn, better schools) if we can find a way to provide better incentives for kids and parents in under-performing schools.

    Thumb up 2 Thumb down 0
  12. triclops says:

    better bury this one quickly, the effectiveness of school choice gives the hoi polloi too many unseemly ideas.

    Let’s just pay teachers more and have more computers in classrooms, and keep doing what we are doing.

    Thumb up 3 Thumb down 5
  13. The Wall says:

    Dismantling the public education system is intended to bolster the standings of failing charter schools whose dirty money has bought both fallen and falling politicians. This nationwide attack on schools is not about Unions, Wages, Benefits or Politics. It is about rewarding businesses which will soon be leaching from the taxpayers to reap profits while providing a dismal educational experience.

    Clearly Temporary Governor Scott Walker defunded education to bolster the standings of charter schools, school choice and school vouchers (championed by Wisconsin’s defrocked politician Scott Jensen) Yes that is what he is doing now – Fact, not wet dream. while charter schools fail to compete in results with fully funded public schools, the remedy was easy to implement, just cut funding to public schools until scores declined.

    “School Vouchers”, “School Choice” and “Charter Schools” are programs created to justify using tax dollars to pay for religious classes with tax dollars which is directly blocked by the constitution. They should call it “indirectly funding school prayer with tax dollars” and be truthful for once about their motives. Run by the evidence burning defrocked politician Scott Jensen whose stench permeates this movement.

    Motive? Stripping tax dollars out of the education process.

    Not to provide a better education for anyone – except the taxpayer who better be noticing by now. Quoting stacked “studies” and manufactured statistics hiring a slew of bloggers these tax siphoners have overplayed their hand and taxpayers and voters are shutting them down.

    I knew, most now know – the rest might learn.

    Thumb up 2 Thumb down 3
  14. Jason says:

    Does anyone know if the lotteries being studied were accompanied by a big event? Imagine a room full of parents and kids, with the parents saying to the kids, “If we win, you will finally have a chance in life. If we lose, your life is over.” Lots of cheers, coupled with tears of sadness for the losers. I imagine that those events have a profound effect on both winners and losers.

    My conclusion about this is both positive and negative. We need more opportunities for kids and parents to feel ownership over the education process. If choice matters, but it is not within our reach to give every kid the choice of an elite magnet/charter/private school, then let’s figure out a way to incorporate the same elements into our traditional schools. Let kids choose a career academy. Let parents influence selection of school resources. Like an employee stock ownership program – it’s not the ultimate solution, but a good tool to help improve morale related to schools.

    Thumb up 1 Thumb down 0
  15. RGJ says:

    This is a subject I have studied a great deal. One can not make blanket statements about charter schools or lotteries or testing outcomes or attrition rates because there are 1000 flavors of school choice enacted around the country, with various levers and bells and switches designed in toss-in political concessions by prostit…I mean politicians. And those results are cherry-picked and twisted and spun by a hundred expert studies with right wing think tanks zealots battling .edu vested interests.

    At the end of the day, it is a civil rights topic. In many American cities, an inner city parent (read minority) has no economic choice but to send their child to the failing, drug ridden, dangerous government-run building down the street. If they don’t do so, the government will take their child away and prosecute them.

    Fixes for these schools, even if they could be wrested away from the adult monopolies benefiting from their current structure will take decades to see affect. In Newark, NJ, the per student cost in government-run schools is more than the cost of sending a child to the most prestigious private school in the state. Throwing money at the problem does not work.

    It is outrageous, and it is a civil rights issue.

    Well-loved. Like or Dislike: Thumb up 5 Thumb down 0
  16. Jodie Whiteley says:

    I don’t disagree that school choice could be good for some students. Obviously there are students whose potential is limited due to their surroundings, and improving those surroundings can allow for more advantages and increased potential for those students’ futures. However I think the way we select students to participate and the reasons for schools to participate in these programs is greatly flawed and needs improvement.

    An example of my point is from my own experiences with this program. My children went to a school in a suburb of Milwaukee, Wisconsin. Milwaukee schools tend to have a lot of students who come from poverty, and the schools in this district are greatly lacking in funding & quality teachers, programs & supplies. Students in the MPS district tend to live in high crime neighborhoods & their schools have many issues which require police presence or arrests at the schools.

    My family was a struggling middle class family; it was of extreme importance to me, to live in a safe neighborhood with neighbors who had similar values & interests to mine and to have good schools. After all I was going to be raising my family here and these neighbors would be the kids my kids went to school with and became friends with. I wanted to be comfortable allowing them to play at their friends’ homes & not fear for their safety or what habits they may pick up from their friends or at school. I was stretching my budget to the limit to afford housing & taxes in this community, but it was worth it to me. Then came the 220 program.

    At the time (not sure if it has changed, since we moved) the 220 program allowed student school choice. All the communities which were willing to participate in the program (accepting students from other districts) received a 13% tax break, for this reason many of the wealthier parents in my community were all for it and gladly voted in support of participation of the program. As for the students my understanding was it was participation to ANY school participating in the program (your choice) on a first serve first come basis. So at the start of the new year parents could stand in line to add their child’s name to the list of participants – until all spots were filled (which happened quickly). The district then provided transportation for the students from their neighborhood to their school of choice at no charge.

    The problem I have with this program is the selection process. Any student could volunteer to go to any school, as long as there was a spot available. There were no criteria for selection process. Some very high income communities gladly volunteered to have their schools participate for the tax credit. No one thought about the effects on the students involved. No one thought about the whether or not the program was being used to actually benefit the neediest children that it should have been intended for.

    By allowing students to enroll simply on a first come first serve basis – you were inevitably bringing the crime from the inner city to the suburbs, rather than bringing the advantages of the suburban schools to the disadvantaged lower income students not meeting their own potential because of limited resources.

    Students should have had to APPLY to participate in the program. Students with the highest potential, good grades & attendance, the least behavioral problems should have had the first opportunity to participate. But instead those families who stood in line first were taken. Where they first in line because most likely the parents were unemployed & on welfare and were readily available to do so, over those that were poverty level, working parents, without transportation who were trying to juggle their job, kids & public transportation in order to go sign up? (That is my opinion only – but it seems like a likely scenario) What ended up happening is kids who did not want to be in school (any school) in the first place participated. They brought their bad attitudes, poor GPA & many times illegal activity to a good school. Criminals (in many cases) were put in school with naive kids who had never been exposed to these types of behaviors and were not prepared to deal with how to respond. I paid alot of money to live in a more expensive SAFER part of the city, with like-minded neighbors for my children to make friends with, and instead my children were going to class with children from the worst neighborhoods, who had nothing in common with my family & who were gaining nothing from going to this school other than a new place to sell drugs or unexpecting targets to steal from. Then you had kids from good families who lived in bad neighborhoods because that is all they could afford. They were trying their best to raise their kids to be good people in a bad environment, they were trying to keep their children safe in an unsafe neighborhood, or hoping their bright student would not be doomed to the same lifestyle they currently had due to poor schools holding them back rather than raising them up to meet their own potential. Nothing was helping these students they were still left behind & let down by the system.

    My bright young blond haired, blue eyed daughter went thru a lot of changes when she started high school. (this was her first year w/the 220 program) She began to realize that she no longer knew were she fit in. She was not one of the rich kids, from a childs perspective she felt like she did not fit in, because of money & clothes. Rather than always feeling like the loser in the group, she went in another direction. She hung out with the kids that would have been lower in the hierarchy of high school social status than she was, then rather than feeling like the loser in the group, she felt like the queen of the group. She was in many cases more wealthy, smarter, lived in a nicer house & in some cases even prettier than the friends she chose. It also just so happened that in a predominately white school all the friends she chose were black kids from the 220 program. So she was the only white girl in the group, if we tried to talk with her about the negative influence her new friends had on her, she accused us of being racist. Then even when I sat at the kitchen table with her doing her homework each night, her grades were slipping more and more (this was a child who had the opportunity to skip 2nd grade) turns out, she didn’t like being so much smarter than her new friends, it was uncool, so even when the assignment was done she chose to not turn it in – to be cool like her friends who never did it, because they came from homes with many problems, and parents who were not involved.

    The problem just continued to get worse for our daughter, soon the “rich” kids saw her as just another one of the bad kids they avoided to stay out of trouble. Then we had the constant fights about racism. I explained we did not care about skin color we caried about the quality of the person, we did not want her hanging out with bad influences regardless of color. She then did start to make friends with a little higher quality character (still from the program). Then we had new issues. When your child has a friend who lives in a crack house in the hood – but its not the kids fault then what. We invited her friend over a lot, sleep overs or whatever, until there came the day her mother showed up at my house extremely angry. The Mom was a drug addict, she told her daughter to call as soon as she got home from the movies, the kids had not even been in the house 5 minutes they were telling me about the movie when the Mom rang the door bell. She went to the door we do not use, we have dogs who could get out, so I told the girls to tell her just a minute thru the window and I went out the side door to show her in. By the time I got around the front of the house, she had convinced her child to open the door, and she had grabbed her by the hair and pulled her out the door and threw her into her car. I tried to go talk to her to see what the issue was, but she just sped off. My daughter was in tears seeing something like that happen. This childs mother was apparently so unhappy because my daughter never slept at their house, she had told the girl, she could not come to our house again until my daughter went to her house. She lived in a neighborhood with drive by shootings all the time. She smoked & sold crack at her home. This was never going to happen. The girls Mom had made her a promise, she told the girl if she behaved and did not get into any trouble, her reward would be she would be allowed to drop out of school when she turned 16. HOW DOES A PARENT COMPETE WITH THAT?

    Another example, my daughter went to homecoming with a very nice boy who was half white & half black also part of the program. I gave them a ride home. The boy begged me not to take him home. He wanted me to drop him off at midnight to ride the bus. His reason was he lived in a very bad neighborhood, his Mom was white and even she did not leave the house after dark. He said being a white woman in a new SUV it was not safe for me to take him home. Again as a parent what do you do in these situations.

    I just feel like the program is set up for more failures then successes. It has potential – but would first need to use he program to help the kids with potential and not waste time & money on kids who have no desire to learn regardless of their environment. I also think they need to have some conversations with both students & parents about the program and the conflicts that could arrise, such as those I mentioned and how best to deal with those situations. You don’t want to give kids the opportunity to go to a better school & then tell the other kids to stay away from them because they are bad kids from bad neighborhoods. But you also do not want kids putting themselves in harms way, in situations they are not prepared to handle just to make everything feel fair.

    Thumb up 0 Thumb down 0