Episode Transcript
You’ve been there: it’s impossible to find a decent song on the radio. When you do, and it ends, the next song stinks. Wouldn’t it be nice if the radio only played the songs you want to hear? The rest of the world has if figured out. Amazon does it. Netflix does it. Your airline does it, even your credit card company. Just go buy some baby diapers, see how long it takes before someone mails you a prospectus for a college-savings plan. If your personal data is like a fingerprint, then you’ve left big, greasy smudges all over the universe. The key is an algorithm, a formula to harness that data and customize the world for you. To give you the things you need to make you smarter, richer, happier. To give you only the songs you want to sing. There’s actually a solution to that radio problem. And it’s so easy that even Steve Levitt can use it.
Steven D. LEVITT: About three years ago, on our blog — the Freakonomics blog — I admitted to people that I’m a Luddite and that I don’t know anything about technology. And I admitted to people that I’m a very late adopter when it comes to technology. And I asked our blog readers, “Tell me what cutting edge technologies I should adopt in my life?” I got a lot suggestions and I think the only one that stuck with me was Pandora.
Levitt’s my co-author, an economist at the University of Chicago.
LEVITT: And it was absolutely stunning to me that there was a technology where you could tell it the music you liked, and it would play other music you had never even heard of but would like, and it would do it all for free, and if you didn’t like a song, you’d say “don’t play that song ever again.” And it would just go away. I mean what an incredible invention that was.
DUBNER: Now, if you had never gotten on Pandora and were just doing it today, what’s a song you would say, “God, this is the song I love. I want 100 more like it, and I don’t know where to find it.” What’s the song you’d put in today?
LEVITT: If I had to put in a song today, I would put in a very obscure song by a group called Iglu and Hartly, called “In This City.”
DUBNER: To start this off, we’ll play a song that exemplifies the musical style of Iglu and Hartly, which features basic rock song structures, call and answer vocal harmony, also known as antiphony, major key tonality, and a vocal-centric aesthetic and acoustic rhythm guitars. So that’s what you like.
LEVITT: Actually, I love all of those things.
DUBNER: So the next time somebody asks you, “Steve Levitt, what kind of music do you like?”
LEVITT: I’m going to say, “I like antiphony.”
* * *
Imagine you fell asleep 150 years ago, and you woke up today. There aren’t many things you’d recognize. But one of them is a classroom — one teacher in a box with 25 students. Why hasn’t it changed? Awhile back, I put that question to Joel Klein, who at the time was chancellor of the New York City public school system.
Joel KLEIN: In a prior life, I used to represent the American Psychiatric Association, and it was during an era which there were a lot of light bulb jokes. My favorite light bulb joke ever was, “How many psychiatrists does it take to change a light bulb?” And the answer is, “Only one, but the light bulb really has to want to change.” And the answer to your question is, the school system does not want to change. It wants more resources.
DUBNER: Grades are up. Graduation rates are up. But when you look across the board in New York City, when you look at the absolute numbers, not relative to where they were, but absolute, you can’t be too happy, right? You’re graduating about 59 percent of students now, so that’s still about 10 percent below national average. I hope I’m right on that. So what’s the problem? Are New York City kids dumb? Are the teachers who come here not good enough?
KLEIN: There’s a lot of questions there. So let me start with for example the graduation rates: if you count August graduates, which I think we should count, we’re at 63 percent, still below where we need to be. But quite frankly, in the last four years alone, that’s gone up about three points a year. And that’s huge. No other city, I think, has seen that kind of gain. But, in the end, one has to be candid. Looking at America generally, as you said, about 70 percent graduate high school. So 30 percent don’t graduate high school. Out of the 70 percent who graduate, probably 30, 35 percent of them are not prepared for college. So, what’s the point?
The problem isn’t only in New York City, of course. The national drop-out rate has been declining — but still, nearly three out of 10 kids never finish high school. Test scores have essentially been flat for a couple decades; other countries are passing us by. I asked Arne Duncan, the U.S. Secretary of Education, how big a concern these numbers are.
Arne DUNCAN: This is a huge deal, Steve. We are very concerned that if we are going to remain economically competitive as a country, if we are going to give our children a chance to compete in a global economy, we have to dramatically improve the quality of education. All of us, as students, want to be treated as individuals and we’ve had this factory model of education in which everyone was treated the same. The best teachers are the ones that understand who you are, who find skills and abilities in yourself that you didn’t know you had in yourself. When it doesn’t work, those teachers are just relating to a mass of faces in front of them, they don’t know you as an individual, you get this sense that they don’t care about you, quite frankly.
So that’s how a classroom is like the radio: a factory model, a big fat effort to pitch right down the middle. But things are changing in schools. I mean, people have been talking about education reform forever. But lately, things really are changing. There’s a proliferation of charter schools, some of them excellent. The Teach for America program. Parenting workshops and pre-school programs. In New York City, there’s a small band of reformers pushing a gutsy program with a catchy name: it’s called “School of One.” They want to dump the factory model altogether. Secretary Duncan is wild about School of One, and Joel Klein’s the man who gave it the go-ahead. Here’s the program’s founder, Joel Rose:
Joel ROSE: In New York City, even before the recession, we had six applicants for every teacher that we hired. We have worked incredibly hard to that our highest performing teachers are rewarded for the work that they do. And we can tell you on a relative basis who our stronger teachers are and who our weaker teachers are. But one of the things that I was interested in was looking at that information on an objective basis. In other words, if we picked a particular standard, like what percent of eighth grade math teachers in New York City get 80 percent of their students to proficiency by the end of the year, that number came in at 13 percent.
DUBNER: Thirteen percent!
ROSE: Right. And when we asked the question a little differently, what percent of eighth grade math teachers can get one years worth of growth out of 80 percent of students in the class that came in at 12 percent.
DUBNER: Yikes! How about reading?
ROSE: In reading, the numbers are about half that.
DUBNER: Half that!
ROSE: Right. Now, we have some incredibly hard working and talented teachers in New York City, and there are thousands of students who graduate college that would love to move to the city and work here. This is not an issue of recruitment or even of talent. It’s an issue of design. I mean, if you think about any other industry, if we create an objective standard of success and only 13 percent of a particular classification of employee was successful we would change the job. We would never have that kind of conversation in education.
DUBNER: So you’re saying that if plumbers or IT designers or engineers or writers or presidents, performed at the level, at the objective level that New York City teachers did, we’d either get rid of all of them, or radically change the job?
ROSE: If we set the bar at how successful do we need plumbers to be, and that standard is a reasonable objective standard, and only 13 percent of plumbers can actually hit that standard, then we have to think differently about how we’re organizing the work of plumbing.
How should the work of teaching be organized? Rose started out as a classroom teacher, three years in a fifth grade in Houston, through Teach for America. He went to law school, then got back into education, with Edison Learning, the for-profit schools company. Eventually he landed in New York City’s Department of Education. That’s when he had a revelation: even the most talented teachers in New York were operating in the old 25-kids-in-a-box, factory mode. How well can one person possibly teach math to 25 different brains at the same time?
ROSE: I’m originally from South Florida, and I was visiting my family in Miami. I went to go see a friend who runs something called a New Horizons learning center. This is a franchise that provides technical training to people on a particular area of technology. If you want to be certified in something from Microsoft, or Cisco or something like that you go to one of these centers. You take a test, you get a certificate, and you can use that to get a job. So I went to meet him for lunch and I walk into the office, and there’s a big sign right when you walk in and it says, “Choose your modality.”
DUBNER: Choose your modality?
ROSE: Choose your modality.
DUBNER: What’s that mean?
ROSE: That a student can choose to come and take live instruction from a teacher, they can choose to learn at their own pace at home online, or they can do what New Horizons calls mentored learning, which is a student can come in whenever they like and there is a teaching assistant type person that can help them as they work through the software.
DUBNER: So I can eat in, take out, or have delivery food?
ROSE: Exactly.
DUBNER: And did a gigantic light bulb flash off in your head?
ROSE: A gigantic light bulb flashed off in my head. What I immediately saw when I saw that sign is, “That is what we’ve been looking for in K-12 education: the introduction of other modalities.” That live, teacher-led instruction, the kind of instruction we’re all familiar with, that we all grew up with, is one way, it may be the best way, but it’s not the only way that kids can learn.
So Joel Rose thought: what if, instead of all 25 kids in a classroom trying to learn math from one teacher in one modality at a time — what if you could take the classroom, divide it into smaller groups, and have each kid get her own playlist of different modalities every day, kind of like the playlists that Pandora makes for Steve Levitt? With Klein’s blessing, School of One quickly moved into pilot phase. This spring, it’s being used as an after-school program in three New York City schools, teaching 6th grade math to 270 students. Let’s go to one of these schools, I.S. 339 in the Bronx. Here’s Kirsten Shy, from School of One:
Kirsten SHY: So what we see here on the screens [is] the first thing when a student walks in. They see their schedule for the day. So right now, it shows period 1. Kids are divided up by their homeroom teams and they can see which area of the room they are supposed to be in. So at this table, they are doing virtual tutoring. At the far table over there, there’s kids that are working on a variety of different online math programs. And at this closer table, there’s kids that are working on independent practice, which is another modality that we have.
DUBNER: Can you quickly name all the modalities for me?
KIRSTEN: Ooh! Quiz me. Large live instruction, small live instruction, virtual live instruction — which we call virtual tutoring, colloquially — independent practice, small group collaboration and independent virtual instruction.
DUBNER: So these guys are doing math homework with headphones at a computer and there is somebody is on the other line. Who is that person and what are they teaching them?
KIRSTEN: The other person on the line is a virtual tutor. I’m not sure exactly where they are in the United States right now. They are working on the individual skills that they need to be working on, so it could be any number of 5th or 6th grade math skills that a kid needs at that time.
STUDENT: I’m talking to the teacher and doing his work. He’s telling us to add any number with two in it.
TEACHER: Now your question is, add three to any number.
STUDENT: Add three to any number? Three plus forty.
TEACHER: Suppose you wrote any number as 40. I could write it as 35 . . .
Small-group instruction? Independent practice? A virtual tutor? It isn’t what these kids are used to. Here’s Patrona Hudson, 12 years old.
Patrona HUDSON: I was failing math because I didn’t understand it. Nobody sat down with me and helped me like work. My teacher — her name is Ms. McKiff and if you ask her for help, she be like, “That you have to do it on our own. Try to figure it out.” She’ll explain it once and if you don’t understand it, she’ll quit. She’ll be like, “I give up. Too bad.”
LIONEL: My name is Lionel, and I’m 11.
DUBNER: And what are you doing here in the School of One?
LIONEL: Math.
DUBNER: What about the kids who are in your class in math who don’t come to School of One? Do you feel like you’re brushing past them?
LIONEL: Yes, because I’m learning a lot more than what they’re learning. They’re probably at home playing video games.
Every lesson, every quiz, every keystroke — it’s all fed into the School of One algorithm. Here’s Joel Rose again.
ROSE: At the end of every day, each kid takes a test. Now, the algorithm can learn what the kid learned today. Which lessons stuck, which topics need more attention, how each kid learns the best. Every day, as the kid gets a little smarter, the algorithm does too.
Far from I.S. 339 in The Bronx, all the way down in Lower Manhattan, in the old Tweed Courthouse, the School of One team sits in a big, open room. There are laptops, whiteboards on wheels, an iPad that everyone’s been testing out, smudged with a million fingerprints. Chris Rush is a co-founder of the project. If Joel Rose is the education-theory guy for School of One, Rush is the tech savant. In some ways, he’s the algorithm. He’s sitting in front of two computer screens.
Chris RUSH: Okay, so what I’m doing is looking at the results from the entire week. I have a grid here that has every student and has every skill that they could be learning; a skill such as adding fractions of circumference of a circle. I can see how many lessons they’ve had on each skill. I can see how fast they’re moving through those skills. I’m looking through those skills where kids are getting stuck, so we can target those to improve the program around adding fractions or circumference of a circle. I can also make sure that any kids that are getting stuck get the proper attention that they need. I tally how fast kids are moving through the skills and give them a rating, sort of like par in golf. They’re beating par, the average, or they’re ahead of it. And we take the kids that are falling behind, and we say, “What’s happening here? How can we do something better for them, and update our algorithm, and pull them out to make sure they’re getting the proper attention?”
DUBNER: Okay, so we talked to a pretty enthusiastic boy name Lionel who was happy to be in the program, so happy that he was telling his friends they should stop going home after school and playing video games and should start coming here to play math computer games. He’s successfully recruited one or two kids. So let’s look at Lionel. Tell us what he’s been doing, how he’s been doing, what direction he’s moving in.
RUSH: I’m going to pull up Lionel here. Lionel is one of those great case studies actually. The nice story about Lionel is that Lionel was really struggling in the month of March. It was taking him ten exposures, twelve exposures to learn certain skills, And now, Lionel is moving through certain things with two exposures, three exposures. That’s really exciting. That’s what we want to see. [He’s] not just learning the skills, but he’s learning how to learn the skill faster. We have one skill here that’s on hold, because Lionel needs to come back to it, and this particular skill has to do with algebraic expressions. That seems to be where he’s having more trouble. Where[as] geometry seems to be something that he’s doing much better in.
DUBNER: What do you do with that information? Where does that steer you to help steer him?
RUSH: We’d look a little bit more closely. Is it geometry that he’s doing well with, or is it the way that we’re teaching him geometry? And in this particular case, it looks like we were actually teaching him geometry a bit differently than we were teaching him how to deal with algebraic expressions. The first thing we would do is try to take the methods that we taught him geometry and try to apply those to how we taught him algebraic expressions.
DUBNER: So the idea is that you will find out which approach works best for each kid and that you will end up with a manageable mix, because if everybody needs the live teacher instruction, then you’re back to 28 kids in a box, yeah?
RUSH: Correct. It’s very much like the Pandora music service online, where you enter in a song name, or an artist and it tries to guess the next best song for you. And maybe it guesses right the first time, but two or three songs in, it got something wrong. Basically, it works the same way here. If a student tries small group collaboration and they don’t do well that day, we give that a thumbs-down. If they do small group collaboration and they do well, we give it a thumbs up. So we can rate our schedules everyday for each student, so we get smarter and smarter at choosing the next days schedule for each student.
DUBNER: Okay. Tim, you’ve let me hijack your Pandora account. We’re listening to the Pandora account of the Pandora guy. We’re on Ben Folds Radio, that’s your station here.
Tim WESTERGREN: That’s the heart of my genome.
DUBNER: Is that right?
WESTERGREN: Yeah, I dig all that stuff.
DUBNER: And how popular are you?
WESTERGREN: We have a little over 50 million listeners in the US.
DUBNER: 50 million listeners! That’s one out of every six human beings in the United States.
WESTERGREN: That’s right. It’s grown like a weed.
DUBNER: Wow. More people are Pandora listeners than are, let’s say, users of Reynolds Wrap, probably.
WESTERGREN: Not the first correlation I would have thought of, but that may be true. We’ve had fifty million people who have registered and created stations. That translates into a little bit north of twenty million people come each month. It’s not a full fifty million every day.
DUBNER: Okay, but that’s still a lot of Reynolds Wrap.
WESTERGREN: That’s right.
DUBNER: What Pandora does is you customize a radio station for anyone who asks you, for as many radio stations as they want. Do you feel a bit like we’ve entered a new world where essentially everything is customizable?
WESTERGREN: I think that’s basically true. What the web did, at first, was made everything available. That was the first phase of the web. And the second phase is to repair that and make all that stuff — that’s so voluminous — actually navigable. That’s where Pandora sits. We look at music and say, “You’ve got a tyranny of choice on the web. [It’s] like walking into a record store and you’ve got eighty thousand CDs to choose from and you need to figure out where to start.” We’re trying to do that with music, and the Music Genome Project’s sort of purpose is to make that easy for someone.
Our public schools are longing for a tyranny of choice — the misery that comes from having too many options. Is School of One the answer? Right now, it’s just an experiment. If it works, it’ll go into more New York City schools. And if it works really well, maybe it’ll spread further. Maybe it’ll even be a big moneymaker for New York, exporting customized education around the country, around the world! Now, if and when education is customized, there’ll be fortunes made, and maybe lost. There’ll be competition among technology vendors and content providers and a million others. There’ll be turf wars with teachers unions. Already, Joel Klein has said he wants to cut the number of teachers in New York City by 30 percent and pay teachers 30 percent more. But would School of One require fewer teachers, or maybe more? Less money, or more? Joel Rose says it’s simply too early to know that. What we do know is that a future with something like School of One will be different. Very different. Technology, instead of being discouraged in schools, would move to the head of the class. Teachers would have to be trained differently. Here’s Blair Hyser, the math department chair at I.S. 339 and a teacher at School of One.
Blair HYSER: The fear of School of One is just that students might get lost, given that there isn’t that traditional 25 kids, one teacher and that teacher works with those students every day. I think that is the one hesitation or fear. It’s different, it’s new, its something that hasn’t really happened before. As a teacher, if you’ve taught fifteen years or twenty years in one way, it’s very challenging to think outside the box in terms of your instruction.
Arne Duncan, the Secretary of Education, thinks that most of what’s happened in the past isn’t worth hanging onto.
DUNCAN: Our Department of Education has been part of the problem, historically. I’ve been very honest about that. And as hard as we’re pushing everybody else to move outside their comfort zone and do more, I promise you that we’re trying to be very self critical, look ourselves in the mirror every single day. Historically, we’ve been this compliance-driven, big bureaucracy. We are fundamentally trying to change what we do from being this big bureaucracy to be this engine of innovation and scaling up what works.
So far, the School of One numbers look good. Here’s Joel Rose.
ROSE: Before the program began, every student took a pre-test that was aligned to the skills on their playlist. The average score was 42 percent. At the end of the program, roughly 50-60 hours of instruction later, they took a post-test on the same skills. Post-test scores average 70 percent. We asked the D.O.E. research and policy team, is that good? Is that not good? How do we contextualize that? They did an analysis and found that these gains were, depending on what we consider a control, 4-8 times the gains we would see in traditional schools in roughly a third the time.
But let’s not get too excited. A lot of experiments look good in the lab. New York City has 1.1 million schoolchildren: can they really be taught one at a time? Can an algorithm, a playlist, help teachers do their jobs better? Can that algorithm really help a kid like Lionel find his own music? The odds seem long, but just think how long the odds must have seemed 10 years ago flailing around at the radio dial, hoping to find just the right song.
DUBNER: Have you heard from anyone for whom Pandora really changed the shape of their life in some way, maybe it resulted in a courtship and a marriage, a divorce, a murder.
WESTERGREN: We had someone whose father was ill, or aging and was in a home, reaching the end of his life. In the last few months, they discovered Pandora and they shared it with him, and he really took to it. When they visited him, everyday he would listen to Pandora all day long, and it became kind of a companion. He eventually passes away, but he passed away listening to Pandora. And because, when you die, you are given a time of death, the family wrote us to thank us, and told us the time of death. We actually wrote them and told them which song was playing when he died. And we sent them a CD and they played that song at his funeral.
Comments