“How Is a Bad Radio Station Like the Public School System?”: We’ve all gotten used to the thrill of customization — Pandora Radio lets anyone customize the music he or she wants to hear. Could a New York City pilot program called School of One do the same thing for education?
We’ve just released the latest episode of our Freakonomics Radio podcast. It’s called “How Is a Bad Radio Station Like the Public School System?” (You can download/ subscribe at iTunes here, get the RSS feed here, or listen live via the link at right.)
This episode is a full 30 minutes long, and it’s about what we call “the thrill of customization” — that is, how technology increasingly enables each of us to get what we want out of life, whether it’s a consumer experience or a religious experience. The main focus of the episode is a fascinating New York City Department of Education pilot program called School of One.
You’ll hear from its founder, Joel Rose, as well as its co-founder and tech guru, Chris Rush; you’ll also hear from Joel Klein, the city’s schools chancellor, and Arne Duncan, the U.S. Secretary of Education (whom some of you may remember as the former head of Chicago Public Schools who worked with Levitt to get rid of cheating teachers).
And, because we tend to never follow a straight line in our podcasts, you’ll also hear some interesting stuff about Pandora Radio from its founder, Tim Westergren, who has just been named to the Time magazine list of 100 influentials (congrats!).
The School of One tries to take advantage of technology to essentially customize education for every kid in every classroom and help teachers do their job more effectively. That is of course a daunting task — and perhaps, some might argue, unnecessary — but the amount of thought and analysis that have so far gone into the program is impressive. Furthermore, the enthusiasm it has generated from people like Duncan and Klein make it a program to watch. And the early results are promising.
You’ll hear about School of One’s conception, its potential pitfalls, and most of all how it works day-to-day. You’ll spend some time in a classroom in I.S. 339 in the Bronx, hearing from kids like Lionel (at right), whose daily “playlist” — in this case, his math lessons — are chosen in part by an algorithm that is designed to learn how Lionel learns best.
And you’ll hear how Chris Rush and others track and analyze the schoolwork that Lionel is doing to make sure he’s not just doodling away his time (like Levitt did in the third grade).
Below you’ll find some more images, including screen shots of the School of One software that helps handle the various analyses. Hope you enjoy the episode; it was an interesting one to make.
Stephen J. DUBNER: You've been there: it's impossible to find a decent song on the radio. When you do, and it ends, the next song stinks. Wouldn't it be nice if the radio only played the songs you want to hear? The rest of the world has if figured out. Amazon does it. Netflix does it. Your airline does it, even your credit card company. Just go buy some baby diapers, see how long it takes before someone mails you a prospectus for a college-savings plan. If your personal data is like a fingerprint, then you've left big, greasy smudges all over the universe. The key is an algorithm, a formula to harness that data and customize the world for you. To give you the things you need to make you smarter, richer, happier. To give you only the songs you want to sing. There's actually a solution to that radio problem. And it's so easy that even Steve Levitt can use it.
Steven D. LEVITT: About three years ago, on our blog -- the Freakonomics blog -- I admitted to people that I'm a Luddite and that I don't know anything about technology. And I admitted to people that I'm a very late adopter when it comes to technology. And I asked our blog readers, "Tell me what cutting edge technologies I should adopt in my life?” I got a lot suggestions and I think the only one that stuck with me was Pandora.
DUBNER: Levitt's my coauthor, an economist at the University of Chicago
LEVITT: And it was absolutely stunning to me that there was a technology where you could tell it the music you liked, and it would play other music you had never even heard of but would like, and it would do it all for free, and if you didn’t like a song, you’d say “don’t play that song ever again.” And it would just go away. I mean what an incredible invention that was.
DUBNER: Now if you had never gotten on Pandora and were just doing it today, what’s a song you would say “God, this is the song I love and I want 100 more like it, and I don’t know where to find it. What’s the song you’d put in today?
LEVITT: So if I had to put in a song today I would put in a very obscure song, by a group called Iglu and Hartly, called “In This City.”
DUBNER: OK so to start this off we’ll play a song that exemplifies the musical style of Iglu and Hartly, which features basic rock song structures, call and answer vocal harmony, also known as antiphony, major key tonality, and a vocal-centric aesthetic and acoustic rhythm guitars. So that’s what you like.
LEVITT: I love – actually I love all of those things.
DUBNER: So the next time somebody asks you “Steve Levitt, what kind of music do you like?”
LEVITT: I'm going to say, "I like antiphony."
ANNOUNCER: Today, on FREAKONOMICS RADIO, the thrill of customization -- and a question: what does a bad radio station have in common with the public-school system? Here's your host Stephen Dubner.
DUBNER: Imagine you fell asleep 150 years ago, and you woke up today. There aren't many things you'd recognize. But one of them is a classroom -- one teacher in a box with 25 students. Why hasn't it changed?
Joel KLEIN: My name is Joel Klein and I'm the chancellor of the New York City public school system. In a prior life I used to represent the American Psychiatric Association, and it was during an era which there were a lot of light bulb jokes. My favorite light bulb joke ever was, how many psychiatrists does it take to change a light bulb? And the answer is, only one, but the light bulb really has to want to change. And the answer to your question is, the school system does not want to change. It wants more resources.
DUBNER: Grades are up, graduation rates are up, but when you look across the board in New York City, when you look at the absolute numbers, not relative to where they were, but absolute, you can’t be too happy, right? You’re graduating about 59 percent of students now, so that’s still about 10 percent below national average. I hope I’m right on that. So what’s the problem? Are New York City kids dumb? Are the teachers who come here not good enough?
KLEIN: There’s a lot of questions there. So let me start with for example the graduation rates. If you count August graduates, which I think we should count, we're at 63 percent, still below where we need to be. But quite frankly, in the last four years alone that’s gone up about three points a year. And that’s huge. No other city, I think, has seen that kind of gain. But, in the end two things, one has to be candid, looking at America generally, as you said, about 70 percent graduate high school. So 30 percent don’t graduate high school. Out of the 70 percent who graduate, probably 30, 35 percent of them are not prepared for college. So, what’s the point?
DUBNER: The problem isn't only in New York City, of course. The national drop-out rate has been declining -- but still, nearly three out of 10 kids never finish high school. Test scores have essentially been flat for a couple decades; other countries are passing us by. I asked Arne Duncan, the U.S. Secretary of Education, how big a concern these numbers are.
Arne DUNCAN: This is a huge deal, Steve. We are very concerned that if we are going to remain economically competitive as a country, if we are going to give our children a chance to compete in a global economy we have to dramatically improve the quality of education. All of us as students want to be treated as individuals and we've had this sort of factory model of education in which everyone was treated the same. The best teachers are the ones that understand who you are, who find skills and abilities in yourself that you didn’t know you had in yourself. When it doesn’t work, those teachers are just relating to a mass of faces in front of them, they don’t know you as an individual, you get this sense that they don’t care about you quite frankly.
DUBNER: So that's how a classroom is like the radio -- a factory model, a big fat effort to pitch right down the middle. But things are changing in schools -- I mean, people have been talking about education reform forever -- but lately, things really are changing. There’s a proliferation of charter schools, some of them excellent. The Teach for America program. Parenting workshops and pre-school programs. In New York City, there's a small band of reformers pushing a gutsy program with a catchy name: it's called "School of One." They want to dump the factory model altogether. Secretary Duncan is wild about School of One, and Joel Klein's the man who gave it the go-ahead. Here's the program's founder, Joel Rose.
Joel ROSE: So in New York City, even before the recession we had six applicants for every teacher that we hired. We have worked incredibly hard to that our highest performing teachers are rewarded for the work that they do. And we can tell you on a relative basis who our stronger teachers are and who our weaker teachers are. But one of the things that I was interested in was looking at that information on an objective basis. In other words, if we picked a particular standard, like what percent of eighth grade math teachers in New York City get 80 percent of their students to proficiency by the end of the year, that number came in at 13 percent.
DUBNER: Thirteen percent!
ROSE: Right. And when we asked the question a little differently, what percent of eighth grade math teachers can get one years worth of growth out of 80 percent of students in the class that came in at 12 percent.
DUBNER: Yikes! How about reading?
ROSE: In reading, the numbers are about half that.
DUBNER: Half that!
ROSE: Right. Now, we have some incredibly hard working and talented teachers in New York City, and there are thousands of students who graduate college that would love to move to the city and work here. This is not an issue of recruitment or even of talent. It’s an issue of design. I mean, if you think about any other industry, if we create an objective standard of success and only 13 percent of a particular classification of employee was successful we would change the job. We would never have that kind of conversation in education.
DUBNER: So you’re saying that if plumbers or IT designers or engineers, or writers, or presidents, performed at the level, at the objective level that New York City teachers did, we’d either get rid of all of them, or radically change the job?
ROSE: If we set the bar at how successful do we need plumbers to be, and that standard is a reasonable objective standard, and only 13 percent of plumbers can actually hit that standard, then we have to think differently about how we’re organizing the work of plumbing.
DUBNER: How should the work of teaching be organized? Rose started out as a classroom teacher, three years in a fifth grade in Houston, through Teach for America. He went to law school, then got back into education, with Edison Learning, the for-profit schools company. Eventually he landed in New York City's Department of Education. That's when he had a revelation: even the most talented teachers in New York were operating in the old 25-kids-in-a-box, factory mode; how well can one person possibly teach math to 25 different brains at the same time?
ROSE: So I’m originally from South Florida, and I was visiting my family in Miami. I went to go see a friend who runs something called a New Horizons learning center. This is a franchise that provides technical training to people on a particular area of technology. So if you want to be certified in something from Microsoft, or Cisco or something like that you go to one of these centers, you take a test, you get a certificate, and you can use that to get a job. So I went to meet him for lunch and I walk into the office, and there’s a big sign right when you walk in and it says “choose your modality.”
DUBNER: Choose your modality?
ROSE: Choose your modality.
DUBNER: What’s that mean?
ROSE: That a student can choose to come and take live instruction from a teacher, they can choose to learn at their own pace at home online, or they can do what New Horizons calls mentored learning, which is a student can come in whenever they like and there is a teaching assistant type person that can help them as they work through the software.
DUBNER: So I can eat in, take out, or have delivery food?
DUBNER: And did a gigantic light bulb flash off in your head?
ROSE: A gigantic light bulb flashed off in my head. What I immediately saw when I saw that sign is that is what we’ve been looking for in K-12 education, the introduction of other modalities. That live, teacher-led instruction, the kind of instruction we’re all familiar with, that we all grew up with, is one way, it may be the best way, but it’s not the only way that kids can learn.
DUBNER: So Joel Rose thought: what if, instead of all 25 kids in a classroom trying to learn math from one teacher -- in one modality at a time -- what if you could take the classroom, divide it into smaller groups, and have each kid get her own playlist of different modalities every day, kind of like the playlists that Pandora makes for Steve Levitt? With Klein's blessing, School of One quickly moved into pilot phase. This spring, it's being used as an afterschool program in three New York City schools, teaching 6th grade math to 270 students. Let's go to one of these schools, I.S. 339 in the Bronx. Here's Kirsten Shy, from School of One.
Kirsten SHY: So what we see here on the screens, the first thing when a student walks in, they see their schedule for the day so right now it shows period 1. Kids are divided up by their homeroom teams and they can see which area of the room they are supposed to be in. So at this table they are doing virtual tutoring, at the far table over there, there’s kids that are working on a variety of different online math programs and at this closer table, there’s kids that are working on independent practice. Which is another modality that we have
DUBNER: Can you quickly name all the modalities for me?
KIRSTEN: Ooh! Quiz me. Large live instruction, small live instruction, virtual live instruction, which we call virtual tutoring, colloquially, independent practice, small group collaboration and independent virtual instruction.
DUBNER: So these guys are getting, these guys are doing math homework with headphones at a computer and there is somebody is on the other line. Who is that person and what are they teaching them?
KIRSTEN: The other person on the line is a virtual tutor. I’m not sure exactly where they are in the United States right now, and they are working on the individual skills that they need to be working on, so it could be any number of 5th or 6th grade math skills that a kid needs at that time.
STUDENT: I’m talking to the teacher and doing his work he's telling us, he's telling us to add any number with two in it.
TEACHER: Now your question is, add three to any number.
STUDENT: Add three to any number? Three plus forty.
TEACHER: Suppose you wrote any number as 40, I could write it as 35…
DUBNER: Small-group instruction? Independent practice? A virtual tutor? It isn't what these kids are used to. Here's Patrona Hudson, 12 years old.
Patrona HUDSON: I was failing math because I didn’t understand it. Nobody sat down with me and helped me like work. Like, my teacher - her name is Ms McKiff – and if you ask her for help, she be like, that you have to do it on our own, try to figure it out. She'll explain it like once and if you don’t understand it, she'll quit. She’ll be like, oh, I give up, like too bad.
LIONEL: My name is Lionel, and I’m 11.
DUBNER: And what are you doing here in the School of One?
DUBNER: What about the kids who are in your class in math who don’t come to school of one? Do you feel like you’re brushing past them?
LIONEL: Yes, cause I’m learning a lot more than what they’re learning. They’re probably at home playing video games.
DUBNER: Every lesson, every quiz, every keystroke -- it's all fed into the School of One algorithm. Here's Joel Rose again.
ROSE: At the end of every day, each kid takes a test. Now the algorithm can learn what the kid learned today. Which lessons stuck, which topics need more attention, how each kid learns the best. Every day, as the kid gets a little smarter, the algorithm does too.
DUBNER: Far from I.S. 339 in The Bronx, all the way down in Lower Manhattan, in the old Tweed Courthouse, the School of One team sits in a big, open room. There are laptops, whiteboards on wheels, an iPad that everyone's been testing out, smudged with a million fingerprints. Chris Rush is a co-founder of the project. If Joel Rose is the education-theory guy for School of One, Rush is the tech savant. In some ways, He’s the algorithm. He's sitting in front of two computer screens.
Chris RUSH: OK, so what I’m doing is looking at the results from the entire week, and I have a grid here that has every student and has every skill that they could be learning: a skill such as adding fractions of circumference of a circle. I can see how many lessons they’ve had on each skill. I can see how fast they’re moving through those skills. So what I’m monitoring is, I’m looking through those skills where kids are getting stuck so we can target those to improve the program around adding fractions or circumference of a circle. And I can also make sure that any kids that are getting stuck get the proper attention that they need. So I tally how fast kids are moving through the skills and give them a rating, sort of like par in golf. If they’re beating par, the average, or they’re ahead of it, and we take the kids that are falling behind, and we say what’s happening here, how can we do something better for them, and update our algorithm, and pull them out to make sure they’re getting the proper attention.
DUBNER: OK so we talked to a pretty enthusiastic boy name Lionel who was happy to be in the program, who was so happy that he was telling his friends they should stop going home after school and playing video games and should start coming here to play math computer games, he’s successfully recruited one or two kids. So let’s look at Lionel. Tell us what he’s been doing, how he’s been doing, what direction he’s moving in.
RUSH: So I’m going to pull up Lionel here, and what I really see, Lionel is one of those great case studies actually. The nice story about Lionel is that Lionel was really struggling in the month of March, it was taking him ten exposures, twelve exposures to learn certain skills, and now Lionel is moving through certain things with two exposures, three exposures. That’s really exciting, that’s what we want to see--not just learning the skills, but he’s learning how to learn the skill faster. We have one skill here that’s on hold, because Lionel needs to come back to it, and that’s dealing with, this particular skill has to do with algebraic expressions, and that seems to be where he’s having more trouble. Where geometry seems to be something that he’s doing much better in.
DUBNER: So what do you do with that information? Where does that steer you to help steer him?
RUSH: So we’d look a little bit more closely. Is it geometry that he’s doing well with, or is it the way that we’re teaching him geometry? And in this particular case it looks like we were actually teaching him geometry a bit differently than we were teaching him how to deal with algebraic expressions. The first thing we would do is try to take the methods that we taught him geometry and try to apply those to how we taught him algebraic expressions.
DUBNER: So the idea I’m guessing is that you will find out which approach works best for each kid and that you will end up with a manageable mix, because if everybody needs the live teacher instruction then you’re back to 28 kids in a box, yeah?
RUSH: Correct. It’s very much like the Pandora music service online, where you enter in a song name, or an artist and it tries to guess the next best song for you. And maybe it guesses right the first time, but two or three songs in, it got something wrong. Basically it works the same way here. If a student tries small group collaboration and they don’t do well that day, we give that a thumbs-down. If they do small group collaboration and they do well, we give it a thumbs up. So we can rate our schedules everyday for each student so we get smarter and smarter at choosing the next days schedule for each student.
DUBNER: Ok so Tim you’ve let me hijack your Pandora account. Were listening to the Pandora account of the Pandora guy. We’re on Ben Folds Radio, that’s your station here.
Tim WESTERGREN: That’s the heart of my genome
DUBNER: Is that right?
WESTERGREN: Yeah, I dig all that stuff.
DUBNER: And how popular are you?
WESTERGREN: We have a little over 50,000,000 listeners in the US.
DUBNER: Fifty million listeners! That’s one out of every six human beings in the United States.
WESTERGREN: That’s right, it’s grown like a weed.
DUBNER: Wow, so more people are Pandora listeners than are let’s say users of Reynolds Wrap probably.
WESTERGREN: Not the first correlation I would have thought of, but that may be true. You know, we’ve had fifty million people who have registered and created stations. That translates into a little bit north of twenty million people come each month. So, it’s not a full fifty million every day.
DUBNER: OK, but that’s still a lot of Reynolds Wrap.
WESTERGREN: Yeah, yeah, that’s right.
DUBNER: So what Pandora does is you customize a radio station for anyone who asks you for as many radio stations, as they want. So, do you feel a bit like we’ve entered a new world where essentially everything is customizable?
WESTERGREN: Yeah, I think that’s basically true. I think what the web did at first was made everything available. That was kind of the first phase of the web. And the second phase is to kind of repair that and make all that stuff that’s so voluminous actually navigable. And I think that’s where Pandora sits. You know, we look at music and say you’ve got a tyranny of choice on the web, like walking into a record store and you’ve got eighty thousand CDs to choose from and you need to figure out where to start. We’re trying to do that with music, and the music genome project’s sort of purpose is to make that easy for someone.
DUBNER: Our public schools are longing for a tyranny of choice -- the misery that comes from having TOO MANY options. Is School of One the answer? Right now, it's just an experiment. If it works, it'll go into more New York City schools -- and if it works REALLY well, maybe it'll spread further. Maybe it'll even be a big moneymaker for New York, exporting customized education around the country, around the world! Now if and when education is customized, there'll be fortunes made, and maybe lost. There'll be competition among technology vendors and content providers and a million others; there'll be turf wars with teachers unions. Already, Joel Klein has said he wants to cut the number of teachers in New York City by 30 percent -- and pay teachers 30 percent more. But would School of One require fewer teachers -- or maybe more? Less money -- or more? Joel Rose says it's simply too early to know that. What we DO know is that a future with something like School of One will be different. Very different. Technology, instead of being discouraged in schools, would move to the head of the class. Teachers would have to be trained differently. Here's Blair Hyser, the math department chair at I.S. 339 and a teacher at School of One.
Blair HYSER: The fear of School of One is just that students might get lost, given that there isn’t that traditional 25 kids, one teacher and that teacher works with those students every day. So I think that is the one hesitation or fear, it’s different, it’s new, its something that hasn't really happened before. As a teacher, if you’ve taught fifteen years or twenty years in one way, it’s very challenging to kind of think outside the box in terms of your instruction.
DUBNER: Arne Duncan, the Secretary of Education, thinks that most of what's happened in the past isn't worth hanging onto.
DUNCAN: I think our Department of Education has been part of the problem historically. I’ve been very honest about that. And as hard as we’re pushing everybody else to move outside their comfort zone and do more, I promise you that we’re trying to be very self critical, look ourselves in the mirror every single day. And historically, we’ve been this compliance driven, big bureaucracy. And we are fundamentally trying to change what we do from being this big bureaucracy to be this engine of innovation and scaling up what works.
DUBNER: So far, the School of One numbers look good. Here's Joel Rose.
ROSE: Before the program began, every student took a pretest that was aligned to the skills on their playlist. The average score was 42 percent. At the end of the program, roughly 50-60 hours of instruction later, they took a post-test on the same skills. Post-test scores average 70 percent. We asked the DOE research and policy team, is that good? Is that not good? How do we contextualize that? They did an analysis and found that these gains were, depending on what we consider a control, 4-8 times the gains we would see in traditional schools in roughly a third the time.
DUBNER: But let's not get too excited. A lot of experiments look good in the lab. New York City has 1.1 million schoolchildren: can they really be taught one at a time? Can an algorithm, a playlist, help teachers do their jobs better? Can that algorithm really help a kid, like Lionel, find his own music? The odds seem long -- but just think how long the odds must have seemed 10 years ago flailing around at the radio dial, hoping to find JUST the right song.
DUBNER: Have you heard from anyone for whom Pandora really changed the shape of their life in some way, maybe it resulted in a courtship and a marriage, a divorce, a murder.
WESTERGREN: We had someone whose father was ill, or aging and was in a home kind of reaching the end of his life. In the last few months, they discovered Pandora and they shared it with him, and he really took to it. So when they visited him everyday he would listen to Pandora all day long, and it became kind of a companion. He eventually passes away, but he passed away listening to Pandora. And because when you die you are given a time of death, the family wrote us to thank us, and told us the time of death and we actually wrote them and told them which song was playing when he died. And we sent them a CD and they played that song at his funeral.
ANNOUNCER: This episode of Freakonomics Radio was produced by Stephen Dubner with Aimee Machado. Subscribe to this podcast on iTunes, and the next episode will show up automatically on your playlist. Go to freakonomics.com to read more about the hidden side of everything.