Episode Transcript
We begin today on the South Side of Chicago, in Dawes Park. It’s summertime. A bunch of teenagers are playing basketball.
Some younger kids are hanging around nearby. They’re just goofing off and, when they see a microphone, they do what kids often do — they yell into it.
KID: It’s all about the music, boy!
It’s the kind of neighborhood where growing up can be very hard. Just ask the guys on the court.
Deonte GRIFFIN: Living in Chicago, it’s a whole bunch like, you know what I’m saying, you young and you black, you gotta watch your back at all times. But at the same time, you know, you got to always just try to be on something positive because there’s gonna always be something negative out here. That why everybody be trying be coming to the court, do something positive, some…
Bill HEALY: Can I ask your name?
D. GRIFFIN: Yeah, my name Deonte “De Great” Griffin. D-E-O-N-T-E, De Great D-E-G-R-E-A-T, Griffin, you know that’s worldwide.
Deonte “De Great” Griffin is 18 years old. He recently graduated from high school — but not in Chicago. He grew up here but a few years ago he left for Atlanta when one of his friends was shot and killed. The friend was 13.
D. GRIFFIN: My dad didn’t want me to go to high school down here just because it’s just so crazy.
That’s the thing about Chicago. It’s pretty dangerous. How dangerous is it, compared to say, New York?
Steven LEVITT: Relative to New York, it is amazingly dangerous. The homicide rates in Chicago are roughly 3 times higher per capita than the homicide rates in New York, believe it or not.
That’s Steve Levitt, my Freakonomics friend and co-author. He’s taught economics for years at the University of Chicago, with a particular focus on crime. According to FBI data, the homicide rate in Chicago is 18.5 per 100,000 people. In New York, it’s just 5.1.
LEVITT: The big difference when you look at who’s murdering and being murdered is that Chicago has very active gangs, very violent gangs, and for reasons I can’t say I understand very well, New York has never really had gangs the way that Chicago has gangs.
Indeed, while it’s famously hard to get good data on gang numbers, law enforcement groups estimate that Chicago has more than 100,000 gang members while New York, a much larger city, has only around 20,000. So it’s easy to believe that, that has something to do with Chicago’s higher crime rate — although we should say that Chicago isn’t even in the Top 10 when it comes to the most violent big American cities. There’s Detroit, Oakland, Memphis, St. Louis. In any case, Chicago is plenty dangerous — so much so that the Chicago Public Schools, the C.P.S., asked Levitt and some colleagues for help.
LEVITT: They were extremely concerned about the pattern of violence against Chicago Public School students. Students were being shot. Almost never in school, but when they weren’t at school. It’s remarkable how many of these students were being shot. It was something like 250 Chicago Public School students were being shot each year — of which about 20 or 30 were dying.
The CPS had launched a program in 2009 designed to prevent violence and keep kids in school. It was considered particularly innovative. It offered full-time mentoring; it gave students part-time jobs. It was, however, an expensive program — about $15,000 per student, per year. So, the C.P.S. wanted Levitt and his colleagues to help figure out how to allocate this precious resource. The plan was to pay special attention to roughly 200 kids who were considered most at-risk.
LEVITT: What they asked us to do was to try to come up with a statistical model that would help them figure out who they should target this program for. They gave us a really unique data set. We had the identities of every student in the Chicago Public Schools and we knew a lot about them — about their grades, whether they came to school or not, and even about their criminal background; whether they had been sent to juvenile detention.
So Levitt got to work, along with a couple of colleagues, Dana Chandler and John List. They analyzed the identifying characteristics of students who’d been shot in the past, in order to help predict which kids might be targets in the future. Some of the characteristics jumped right out at them.
LEVITT: The single best predictor of being shot overall, as is so often the case with crime, turns out to be male. It was almost exclusively boys who got shot.
Okay, no big surprise there. There were some other unsurprising traits.
LEVITT: Having been to juvenile detention, having dropped out of school, having low G.P.A., all of those things predicted who would get shot.
Race was a very prominent factor — or so it seemed.
LEVITT: An African-American boy was 20 times more likely to be shot than a white boy who’s going to the Chicago Public Schools. Yet once you controlled for information about the neighborhoods and the grades and the past history and things like that, the role of race really disappeared. The race of a kid was more a symptom of some of the behaviors that might be involved, rather than actually the race itself was determining the being shot.
Stephen J. DUBNER: How well did your predictive model seem to work?
LEVITT: It worked okay. But not fantastically well. We could isolate a group of kids out of a sample who were about 20 times as likely to be shot as the average kid. That sounds impressive when you think about it, but the likelihood of being shot is so low in the sample that actually we were only able to predict one or two shootings over this set of kids. It’s an interesting case of statistics, of how if you present it one way, “Wow that’s really impressive.” But if you really dig down into it, you realize, “Wow, this is such an unlikely and semi-random act that we really didn’t have much predictive power at all.”
Alright, so Levitt and his colleagues weren’t much help on that dimension. But they also tried to figure out the efficacy of the program the school was already running, the program that tried to keep kids in school and away from violence. One component of this program was run by a national nonprofit called Youth Advocate Programs, or YAP.
LEVITT: It was in one of these urban programs where they had mentors and they came in and they tried to coach the kids and teach them to do the right thing. That sort of stuff.
DUBNER: You sound so enthusiastic about it, yeah?
LEVITT: Once you know the results it’s hard to be enthusiastic about the program.
DUBNER: Alright, give me the results then.
LEVITT: We really didn’t see any effect at all. Now it wasn’t a perfect study because, as much as we pushed and prodded on the Chicago Public Schools to run it as a randomized experiment — which you and I and a lot of other people have been advocating when it comes to doing social science — they weren’t willing to do that.
That said, the study was good enough to learn what effect, if any, the program had.
LEVITT: The program didn’t seem to have really on any dimension, seem to have changed the lives of these kids.
DUBNER: What is a better model, knowing what you know about these interventions and how they didn’t work?
LEVITT: The conventional wisdom is that there’s almost nothing that you can do to these kids that will help. That there’s been a long history of various interventions and by and large what we’ve found is that once you get to be a teenager, there’s not much you can do that will be very helpful for this population.
The idea is that a lot of our bad habits are pretty fixed by adolescence, and it’s really hard to rewire a teenager who’s already in trouble or heading for trouble. But is that really true? Or is it just a way to explain why society hasn’t done a better job of helping at-risk kids and preventing crime? Today on Freakonomics Radio, we’ll hear from a criminologist who did get to run a randomized experiment — the results of which just might rewire your thinking about crime, and a lot of other things too:
Sara HELLER: The first time we saw these effects, we thought “Wow, can this be right?”
* * *
Now we’re in downtown Chicago, 9:30 on a Friday morning. Tony DiVittorio is running a meeting with about a half dozen men, sitting on folding chairs.
Tony DIVITTORIO: Let’s do a little housekeeping. First of all, welcome. Couple things going on. Number one: we’re scheduled to be here until four.
DiVittorio works for a Chicago organization called Youth Guidance — not to be confused with YAP, or the Youth Advocate Program. Different people. But Youth Guidance is also in the business of trying to keep Chicago Public School students out of trouble. Today, DiVittorio is training a group of counselors.
DIVITTORIO: What’s the goal for today? Today is to introduce the value of accountability. Our fourth goal.
DiVittorio’s background is in psychology. He’ll drop phrases like “hostile-attribution theory” and “positive-thought replacement.” But he also knows what it’s like to grow up in a rough neighborhood — like the students he’s trying to help.
DIVITTORIO: Born and raised on the southwest side of Chicago, so I had that experience of being an at-risk youth myself.
DiVittorio also calls himself a “rites-of-passage elder.” He’s thought a lot about what it means to be a man, to become a man. He and his father had a difficult, distant relationship. DiVittorio has worked through a lot of old emotions, poked a lot of old wounds. And over time, he incorporated this thinking into his work with teenagers, and he built a program he calls Becoming a Man, or BAM. He’d get young men to sit down in a circle and talk about the authority figures in their lives; about why they shouldn’t drop out of school.
DIVITTORIO: I was working with these youth who were referred to me for anger issues, cutting class, yelling at their teachers, what have you. I was operating within these different modes. I was a clinician; I’m a psychologist, doing counseling, having these boys explore the men they want to become and passing on this knowledge and getting them to do what we call “deep dive.” I realized that whatever I was doing was effective, because the teachers kept referring the boys, they kept seeing some emotional regulation with the boys. The boys kept coming back, opening up in these circles. I sat down in 2001 and said, “What is it that I’m doing? What’s happening here?”
It was only when he set his method down on paper, DiVittorio says, that he realized what he was doing.
DIVITTORIO: Oh, I see. I’m doing cognitive behavioral therapy.
Cognitive behavioral therapy, or C.B.T., is a form of psychotherapy that’s pretty common in mental health treatment, although not in the kind of social work that DiVittorio does. It’s meant to be a short-term, pragmatic approach to helping people solve problems. As the name implies, it’s meant to change behaviors without necessarily dwelling on the underlying psychological sources of those behaviors, as talk therapy does. C.B.T. tries to get people to think differently about behaviors that have become practically automatic; to respond more thoughtfully to stress — and not just in a purely therapeutic realm.
HELLER: C.B.T. has actually, interestingly, been proven effective in lots of other domains.
That’s Sara Heller.
HELLER: For depression and anxiety, for medication adherence, for smoking cessation, we know that C.B.T. is a strategy that works in a short amount of time to change behavior. What we didn’t know is that it could apply to these other types of policy-relevant behaviors like dropout and crime.
Dropout and crime – that’s exactly what Heller wanted to learn about. She’s a professor of criminology at the University of Pennsylvania. She also participates in the University of Chicago’s Crime Lab, which is a network of researchers who try to find empirical solutions to crime and violence.
HELLER: The Crime Lab developed because the University of Chicago was desperate to try to help the city come up with ways to solve the gun-violence problem in Chicago and elsewhere. One of the first things the Crime Lab did was to run a design competition and announce to the city: “Give us all your best ideas for reducing gun violence. We’re going to pick the best idea, raise some money for it, and make it happen.”
And that’s how Heller and her colleagues got acquainted with Tony DiVittorio and his BAM project. In 2009, the Youth Guidance organization won the design competition. Now, this was a major victory — the Crime Lab helped Youth Guidance raise nearly $1 million in funding. But it was also a challenge. DiVittorio and BAM had been operating in just a single public school in Chicago. Now, they’d be in over a dozen schools working with a lot more kids.
DIVITTORIO: In 2009, I was the only BAM staff. For ten years, it was just me. Now we have 60 BAM counselors, or something like that.
Here’s how it would work. DiVittorio and the other BAM counselors would meet with students inside their schools during regular class periods. That way, the kids wouldn’t have to get themselves somewhere else after school for the therapy. It was a one-hour session, once a week. Each student had the chance to take up to 27 sessions during the school year. The Crime Lab’s generous funding came with just one catch: BAM had to let Sara Heller and the other researchers evaluate the program to see whether it actually worked. The key was to set up the project as a randomized control trial. The population of eligible students would be divided into a treatment group, who’d get the C.B.T. intervention, and a control group, who wouldn’t. The random division was accomplished by lottery.
HELLER: Everyone appreciates the idea there’s no nepotism involved. You don’t have to pick and choose which kids deserve to be in the program or not. On the more challenging end, the youth don’t feel like they’ve been singled out because they’re in trouble, right? It’s just the luck of the draw.
The kids who did get drawn were now placed in the BAM program. The idea was to teach them to slow down their thinking when they ran into challenging situations; to control their immediate impulses.
HELLER: The basic theory rests on something that we’re calling automaticity, or automatic behavior. Automatic behavior is something we all engage in all the time, right? It’s just behaving without consciously thinking about it.
DUBNER: There are probably all kinds of behaviors that we all do, whether bad behaviors or otherwise, that we probably think we’re deciding to do but we’re really just doing purely out of habit or automaticity, yeah?
HELLER: Exactly. It’s a good thing we do, because it’s actually very cognitively costly to think about every action.
DUBNER: Right, it would take forever. We’d never get past breakfast if we had to make every decision from scratch, right?
HELLER: Think about even opening a door, right? If you have to consciously think, “Okay, there’s the doorknob. Now I’m going to reach out and I’m going to clench my fingers around it.” That’s a very simplistic example, but there’s all sorts of things that require a lot of automaticity in everyday life. Oftentimes, automatic behavior is very adaptive. But in some circumstances, it can be maladaptive, especially when your automatic response is ill-suited to the situation that you’re in.
Heller worked with several colleagues on this study, from a variety of disciplines — Jens Ludwig, Sendhil Mullainathan, Harold Pollack, and other academic heavyweights. The notion of automaticity was drawn in large part from the worldview of the psychologist Daniel Kahneman, whose work we often discuss on this show. In fact, the research paper that Heller and the others would write is called, “Thinking, Fast and Slow”– which is the same name as Kahneman’s 2011 book. It describes the two avenues of thought we all engage — System 1 and System 2. System 2 is a more deliberative style; System 1 is so fast and emotional as to be automatic. But people from different backgrounds have different automatic responses.
HELLER: Think about a youth walking down the street in an upper-middle class neighborhood. If someone comes up and demands his cell phone, probably his automatic response is going to be to hand it over and then go run and tell an adult. He’s built up that automatic response over many years of being in situations where that has become very adaptive, right? Where adults are reliable, where compliance is a constructive behavior. When that kid then shifts over to the classroom and the teacher says, “All right, everyone sit down. It’s time to start class.” Compliance might be the automatic behavior. It’s very adaptive.
DUBNER: Okay, that’s that kid, but not all kids.
HELLER: Right, so now think about a youth who’s growing up in a much poorer neighborhood where there’s a lot more chaotic environments, where compliance is probably not the adaptive thing, right? There’s a lot of evidence about street life in poorer neighborhoods and how important it is to stand up for yourself to show that you’re not a victim, right? Because if you just comply with every request that comes your way, you’re going to get beaten up quite a lot. There you might develop an automatic response that’s a little bit more aggressive, more assuming that other people are hostile, trying to avoid conflict until it comes and then really strongly standing up for yourself. If someone on the street says, “Hey, give me your cell phone.” Your response is probably going to be at least a little bit less polite than, “Here you go.”
DUBNER: Right.
HELLER: If you think about what that person might do, there might be shoving or fighting, there might be some profanity. If you then think about that youth in a classroom where the teacher says, “Everyone sit down so we can start class.” That might feel like automatically the same situation, right? Someone is challenging you. You might feel like something is at stake for you. But if you have that same aggressive reaction in front of the teacher, you’re going to get kicked out of class. You can see there that it’s not that the youth in a poor area is any more automatic, or that the rich youth are behaving less automatically. Everyone’s applying this automatic behavior, but youth in poor neighborhoods face so many different situations. Their situations are more variable and when the context varies, that’s particularly hard for automatic responses. You’re going to be more likely to have a maladaptive response.
So how do you learn to interrupt that automatic response? That’s what Tony DiVittorio and the BAM program are all about. The students in the program would participate in a variety of C.B.T. routines. One of them is known as the “fist exercise.”
HELLER: ‘The fist’ is really interesting. It’s one of the first things that youth do when they’re starting the BAM program.
It works like this: DiVittorio or another BAM counselor would bring roughly a dozen teenage boys into a classroom and invite them — not tell them — invite them to sit in a circle. More often than not, the boys do it.
DIVITTORIO: Before I say anything, I say, “Stand up, find a partner, go anywhere in the room you’d like. Just don’t stand on any tables. Don’t go by the teacher’s desk.”
HELLER: They pair off.
DIVITTORIO: I say, “Listen, pay attention to what I’m about to say: One person be letter A and the other person be letter B in that partnership. Go ahead and decide.”
The kids decide who’s who.
DIVITTORIO: Then I say, “Letter A, go ahead and make a fist.”
Now all the A’s make a fist.
DIVITTORIO: “Letter B partner, you have thirty seconds to open up letter A’s fist. But hang on, let me give you the rules.”
HELLER: There’s only one rule.
DIVITTORIO: There are no rules.
HELLER: “Go.” You can imagine what happens in a room full of adolescent boys who have just been told there are no rules. All hell breaks loose.
DIVITTORIO: They start beating the crap out of each other.
HELLER: Everyone’s chasing after each other trying to get the fist open.
DIVITTORIO: Ten out of ten times they’re physically fighting. Sometimes they do headlocks. Teachers are walking by thinking that we’re out of control.
HELLER: It goes on for about 30 seconds.
DIVITTORIO: Timeout, timeout.
HELLER: They switch partners. The other person closes his fist, so there’s equal-opportunity bashing going on.
DIVITTORIO: You do it again. Same thing happens.
It’s utter chaos.
HELLER: Then Tony will blow the whistle and bring everyone back for the debrief.
DIVITTORIO: Their hearts’ elevated; they’ve got a smile on their face; they’ve had some fun.
HELLER: He’ll say, “Okay, how many of you succeeded in opening your partner’s fist?”
DIVITTORIO: “Raise your hand.” Maybe three people.
HELLER: Tony will say, “That’s interesting, it doesn’t seem like a very difficult task. Tell me the things that you tried.”
DIVITTORIO: “How did you get the fist open or how did you try?” You go around the circle.
HELLER: They’ll say, “First, I grabbed his wrist and twisted, but he got away. Then I tried a kidney punch.” Tony is saying, “Interesting, interesting.”
DIVITTORIO: I say, “It’s interesting not one of you asked your partner if he could open up his fist.”
HELLER: Right, no one ever asks the partner to open his fist. Tony will say, “Why not?” They’ll say things like, “He would have thought that I was a punk. He never would have done it.”
DIVITTORIO: Or, “It wouldn’t work.”
HELLER: Tony will say, “How do you know?”
DIVITTORIO: “How do you know it wouldn’t work?”
HELLER: This starts a conversation about how you know what other people are thinking and how you know how they’ll react to things unless you sort of try and ask.
DIVITTORIO: Now I go into lecture mode. They’re at the edge of their seat, and I talk about, “Did you know that a powerful man, an exceptional man, is a man who is willing to ask for what he wants? Listen, there’s times I have got to take things. I know being on the streets. Sometimes I have to be in that kind of thinking. But there’s other times, man, where that ain’t going to work. I have to know how to ask for what I want.” They’re quiet. Their eyes are real big. They’re hearing a soft skill.
HELLER: It’s a very engaging way to get youth to start to think about their own thought processes, to think about other people’s thought processes, without having to sit down and give them instruction about what metacognition and thinking about their own thinking means.
We wondered how the fist exercise would work with the kids we heard from earlier, playing basketball in Dawes Park.
HEALY: Can I try this game with you guys?
We sent producer Bill Healy there. Healy modified the fist exercise. Instead, he used a water bottle. He gave the bottle to Deonte Griffin and told another kid he had to get it out of Griffin’s hand.
HEALY: You got 30 seconds, all right? You’ve got to get this bottle away from him and there’s only one rule, okay? And that’s that there are no rules.
D. GRIFFIN: He’s got to get the bottle outta my hand?
HEALY: Uh – huh.
Again, the only rule was that there are no rules.
D. GRIFFIN: This will be fun.
They waited for the sirens to pass.
D. GRIFFIN: I hear this almost every morning. I woke up hearing sirens.
And then came the countdown.
Joseph HAMPTON: Want me to count out loud?
HEALY: Sure. Actually, let’s do 20 seconds.
HAMPTON: Twenty.
HEALY: Ready, set, go.
HAMPTON: 20, 19, 18, 17, 16 …
D. GRIFFIN: He got it at 18.
HEALY: Here, now you’ve got to get it from him.
D. GRIFFIN: I want another countdown.
HAMPTON: All right: 20, 19, 18, 17, 16 …
D. GRIFFIN: It took me longer.
HEALY: The question that they wanted me to ask then is, “Why didn’t you just ask him for it?”
D. GRIFFIN: That is a good question. I don’t know. I guess that’s just a …
Tajaelien GRIFFIN: That was just a game you just told us.
D. GRIFFIN: But you know what? It’s crazy because if you think about it mentally, I could have just asked for it. It’s like your subconscious tell you to take it, instead of doing the right thing; which is asking for it, which would be something simple. You don’t have to waste no energy. It’s like you gotta take it. It’s ironic because it almost relates to Chicago.
HEALY: How?
D. GRIFFIN: Instead of doing something simple, keeping stuff simple, they try to just take everything. Even when people can’t get jobs, what they do? They try to rob, steal, just take instead of doing the right thing and try to work for what they want and just ask for it. That’s what we need in Chicago because it’s a lot of the men who grow up not around real men. They don’t know how to carry themselves as real men. When they grow up, they grow up trying to be the people who they see. Those are their team, their peers, and they ain’t going to do nothing but try to do things that make their peers feel accepted. Because a lot of kids down here don’t grow up with father figures who [are] going to teach them how to be a man. That’s why, when they go out here in these streets, they try to do things like shoot somebody to make them feel like, “I’m a man.” But that’s not what it is. Anybody can kill somebody. Real men move their mom out the hood. That’s what real men do.
Deonte Griffin hasn’t ever been to BAM. He’s never met Tony DiVittorio. But the lesson of the fist exercise — or, in this case, the bottle exercise — seems to hold real promise. So, coming up on Freakonomics Radio: will the program really work? Will it actually help prevent these kids from getting violent, from dropping out of school?
HELLER: The fact that we see this over and over in different settings starts to build the case that there might be really something going on here.
* * *
As you have likely gleaned by now, the Chicago youth counselor Tony DiVittorio has a pretty dynamic personality.
DIVITTORIO: Guys, we’re going to start up again.
He’s sincere and he’s funny. You can tell he loves working with young people and teaching others how to work with young people. During the training session for BAM counselors, he connects easily with the other men.
DIVITTORIO: We’re going to have some fun now.
In addition to the fist exercise and drills like that, DiVittorio likes to use role-playing. He wants to get kids to think critically about their own behavior.
DIVITTORIO: My favorite one, I’ll share with you. My favorite one is called High School Day.
High School Day is about learning to take an emotion like anger and channel it into something more positive. DiVittorio talks to the kids about the difference between a “savage” energy and a “warrior” energy. The savage energy, he says, is a completely normal emotion.
DIVITTORIO: Warrior energy is taking normal savage energy and making a choice to express it positively.
He has them act out situations they’re likely to encounter in their own lives. One kid will play the role of a high-school security officer, who’s filled with “warrior” energy. And another kid plays himself — a student on his way to class.
DIVITTORIO: “You’re the student on your cell phone walking through the hall and you’re filled with savage energy today. No one tells you what to do. You come from the streets, man. You’re badass.” Then you have them act out these scenarios. They have a lot of fun with it. Then you process it. What was it like to be in savage energy? What was it like to be in warrior energy? Did you feel like a punk? You get them reflecting upon that. Then you go into act two, where now the security officer’s having a bad day. He or she is filled with savage energy, but the student’s coming to BAM and he’s trying to do this warrior energy thing. He’s full of respect and now the security officer is like, “You’ve got a suspension, you’ve got this, you’ve got that.” The student has to, in the role-play, stay in warrior energy.
The boys learn what it feels like to not lash out, to not act on impulse. Then DiVittorio sits them in a circle and they dig deep. They talk about their strong points; they talk about what trips them up.
DIVITTORIO: This is when the boys really open up. “Man, every time I think about my father,” or something. We start to see why it’s difficult for this youth to come into the school and go to algebra class and sit there and learn algebra. Because all the truth comes out in that circle. We’re able to really now get to some real concrete solutions on how to handle this.
What DiVittorio and BAM don’t do, says Sara Heller, is tell kids how they should and shouldn’t behave.
HELLER: They’re not teaching youth never fight. That would be a stupid thing to teach youth in these neighborhoods. It’s trying to teach youth to slow down a little bit, be a little bit more reflective rather than reflexive in their thinking, and think for five seconds, “What kind of situation am I in? Do I need to tamp down this automatic response or not?”
So this all sounds good, right? C.B.T. with at-risk Chicago teenagers, getting them in touch with their emotions and their automatic responses. But, did it work? It’s all well and good to run crime-prevention programs and stay-in-school programs, but if they don’t prevent crime and keep kids in school, aren’t they kind of a waste of time and money? Fortunately, Sara Heller and her colleagues now had the data to answer this question. The Becoming a Man study covered more than 2,700 boys in 18 public schools in Chicago, from seventh grade through tenth grade. A lot of these kids were already in trouble — roughly a third of them had been arrested at least once before the study began. Now remember, the researchers randomized the study so that half these kids entered the BAM program and the other half didn’t. This allowed Heller and her colleagues to look at arrest and school dropout data in the 12-plus months after the BAM program began and compare the boys who got the treatment to those who didn’t.
HELLER: The first year of BAM, we saw a 44 percent decline in violent crime arrests, and a 36 percent decline in other nonviolent non-property, non-drug arrests. We see an increase in school engagement that we think might end up eventually, once the kids are old enough, improving graduation rates by maybe 7 to 22 percent.
DUBNER: The improvement is wildly large, at least to my eyes. I don’t want to say, “Too good to be true.” I don’t mean to imply that I don’t believe it, but wow, it seems as though it really works.
HELLER: I agree with you. The first time that we saw these effects we thought, “Wow, can this be right?”
So, just to be sure, Heller and her colleagues did a second randomized study of another season of BAM counseling, fairly similar in scope to the first. The effects weren’t as large as the first study — but still, large.
HELLER: Same program, different set of kids are also finding about a 30 percent decline in overall arrests.
Even so, Heller and the other researchers wanted to make sure they were measuring the efficacy of CBT generally and not just the BAM program. After all, Tony DiVittorio is a really likable guy. Plus, he’s built like a boxer. He just looks like someone you want on your side.
HELLER: One of the things you might worry about with the BAM program, “Is it just that Tony — who developed the program — is so charismatic and so good that he’s having these effects? Could this possibly work in other settings with different types of youth?
Even DiVittorio says it’s hard to know what the magic ingredient is. Yes, BAM uses a lot of C.B.T., but also some mentoring and “rites of passage” work.
DIVITTORIO: BAM, since it’s such a mix of these different theories, it’s trying to figure out what are we really measuring here?
So the University of Chicago Crime Lab researchers ran a third experiment, this one having nothing to do with BAM. This one was a rather accidental experiment that happened at the Cook County Juvenile Temporary Detention Center, the J.T.D.C.
HELLER: Which is the youth detention center where youth go between the time they get arrested and when the court decides what they want to do with them. On average, youth are there for only three weeks, although there’s a lot of variability in how long youth stay. Historically, the J.T.D.C. has been a very challenging place. There was an A.C.L.U. lawsuit that ended up with the federal government appointing an administrator there who started a series of reforms. Some of these reforms involve cognitive behavioral therapy, or C.B.T. The youth engage in C.B.T. every day. For various reasons, those reforms got frozen halfway through, such that half of the residential units within the center were operating under this new C.B.T. system, and half weren’t. They were operating under the status quo.
DUBNER: Perfect for you, then, right?
HELLER: Exactly. If you’re a researcher, your spidey sense should be tingling right now. We talked to the administrator and we said, “How do you decide which youth to place in which centers? Do you try to keep youth separate by gang affiliation?” He says, “No, I like to really mix them up.” We say, “Do you try to place youth who you think are going to benefit most from the C.B.T. program?” He says, “No, it’s practically random.” We said, “Really? Random, you say? How interesting.” We talked to him about the benefits of making it actually random so that we could learn from what was going on there, and in fact, the whole nation could learn what’s happening in Chicago’s detention center. He was excited and agreed. We worked out a system with the staff to randomly assign whether youth went to the C.B.T. centers or the non-C.B.T. centers, which sets us up for an evaluation. There again, we’re seeing declines in return rates to the detention center 12 to 18 months later that are about 20 percent lower than in the control group. The fact that we see this over and over in different settings starts to build the case that there might be really something going on here.
Sara Heller and her colleagues wrote up the two BAM studies and the J.T.D.C. study in a research paper. “Given the evidence of sizable behavior change across all three interventions,” they pointed out, “It’s worth noting what these interventions are not. They do not involve academic remediation or vocational education, or job training, or paid temporary jobs, or internships, or early childhood education, or cash, or in-kind transfer to reduce poverty.” In other words, these interventions don’t involve any of the conventional — and often expensive — treatments that are typically used. Cognitive behavioral therapy is relatively dirt-cheap.
HELLER: That’s right. It’s very low-cost, especially in the detention center where the staff are already there, the building’s already there.
DUBNER: The kids are already there.
HELLER: Right. All you have to do is change what you’re doing for a half hour a day. That is incredibly low-cost. The BAM program happens in schools. When you’re training mentors, bringing them into schools and you have some administrative costs, there’s still a cost to the program. In that first year, it was about $1,100 per participant. But relative to the other types of interventions we think about, they’re incredibly inexpensive. The benefit-to-cost ratio is really high, which is good news, right? Because if you’re thinking about trying to eradicate poverty or solve the failing school problem in urban areas, those types of tasks, not that they aren’t worth doing, but [they’re] incredibly expensive. They take a lot of time. Here we have interventions where, with an average of 13 sessions, you can change future behavior by quite a lot.
This is the sort of cost-benefit analysis that economists and other academics are fond of, but which isn’t always welcome among the people who are actually responsible for trying to solve problems. There are politics to be considered; there are vested interests. Sara Heller says that the City of Chicago and its mayor, Rahm Emanuel, were supportive of this research, and they ought to be commended for it.
HELLER: Not every policymaker wants to know if the way he’s spending money is the right way. People have programs that they like, that they want to support. It’s a scary thing to open them up to evaluation. We might have evaluated these programs and found out that they didn’t work at all, or even that they had harmful effects. The fact that Mayor Emanuel and all the people who run the programs were so open and supportive, not just of their programs, but also of their data, right? We had access to public school records, to administrative arrest records. That’s a really remarkable thing, and the studies wouldn’t have been possible without that support. I’m in Philadelphia now, and the city here is very interested in rigorous evaluation. It’s a good lesson for other cities who are thinking about how to improve outcomes for their city’s youth. It doesn’t have to be scary to work with researchers. Some of us understand the challenges and the limitations that they face. I hope that this doesn’t just talk about these programs and these ideas, but also helps people think about the benefits of incorporating rigorous evaluation into what they’re already doing.
DUBNER: Your evidence is compelling that these forms of C.B.T. work, that these interventions work. What do you know about the lasting effect? In other words, do you teach people how to make a different, less-automatic decision that will help them in a number of ways, but that if they don’t keep getting the C.B.T., then they’ll kind of forget or revert? Or is there evidence that it lasts?
HELLER: We have some short-term evidence on that right now. For the first BAM study, the violent crime decline happened in the first year, the year that the program was going on, and faded out in the second year. But the schooling engagement increase lasted through the year following the program. Similarly, for the detention center, we see reduced return rates 12 to 18 months later. We don’t have enough follow up data to see what happens after that. It’s a really important question to think about how lasting these effects are going to be. Would a booster shot of C.B.T. down the line help make the effects last longer? In fact, a follow-up study is trying to test some of those questions: does it make a difference if it’s one or two years of the program? We have this idea in our head that the only way a program is good and works is if it has effects that last forever. Especially when you’re thinking about something like violence, that’s really the wrong way to think about it. Violence is incredibly socially costly. If there’s a way to reduce violent crime, even for just a year at a pretty low cost, we should do it, right? You’re having a cost effective change, you’re getting more back in sort of social benefits than you’re spending. In fact, because criminal behavior declines very quickly with age — violence peaks in the late teens, fades out very quickly in the early 20s — if you’re making a difference for just a year in someone’s late teens, then you really might be making a difference in the lifetime number of violent incidents that’s going to occur in that youth’s life. It’s important to push back a little bit against this idea that things only matter if they last forever, although, obviously, we would love the effects to last forever. We’re working to figure out what combination of interventions and for how long, can help the effects last.
I don’t know about you but it’s hard for me to not be intrigued and impressed with the results of this intervention — especially considering how relatively simple and cheap it is. So next week on Freakonomics Radio, we wonder: what happens if you do choose to spend a bit more money — if you keep up the cognitive behavioral therapy but also throw in some cash incentives?
Chris BLATTMAN: It was literally ten Andrew Jacksons in the palm of your hand.
This wasn’t in Chicago, or even the U.S., but in Liberia. And if you think it’s tough getting a teenager in Chicago to stay in school, think about what it takes to turn around a former Liberian child soldier.
Klubusumo Johnson BORH: Child soldiers were always used in the instances to torture. Even if there was a case wherein such a person needed to be executed, you always would want to use child soldiers to do that.
C.B.T and cash payments in Liberia. That’s next time, on Freakonomics Radio.
Comments