Episode Transcript
Hey everybody. Happy Christmakwanzaakkah, and thanks for downloading the Freakonomics Radio podcast. Our last episode was about “weird recycling” — reusing stuff that most people didn’t think was reusable. Today, we’re taking our own advice. This episode is an encore of a show we did a while back, which we thought was worth updating. Hope you agree.
You’ve been there: It’s impossible to find a decent song on the radio. When you do, and it ends, the next song stinks. Wouldn’t it be nice if the radio only played the songs you want to hear? The rest of the world has if figured out. Amazon does it. Netflix does it. Your airline does it, even your credit card company. Just go buy some baby diapers — see how long it takes before someone mails you a prospectus for a college-savings plan. If your personal data is like a fingerprint, then you’ve left big, greasy smudges all over the universe. The key is an algorithm, a formula to harness that data and customize the world for you. To give you the things you need to make you smarter, richer, happier. To give you only the songs you want to sing. There’s actually a solution to that radio problem. And it’s so easy that even Steve Levitt can use it.
Steven D. LEVITT: About three years ago, on our blog — the Freakonomics blog — I admitted to people that I’m a Luddite and that I don’t know anything about technology. And I asked our blog readers, “Tell me what cutting edge technologies I should adopt in my life?” I got a lot suggestions and I think the only one that stuck with me was Pandora.
Levitt’s my co-author, an economist at the University of Chicago.
LEVITT: And it was absolutely stunning to me that there was a technology where you could tell it the music you liked, and it would play other music you had never even heard of but would like, and it would do it all for free, and if you didn’t like a song, you’d say, “Don’t play that song ever again.” And it would just go away. I mean what an incredible invention that was.
DUBNER: Now if you had never gotten on Pandora and were just doing it today, what’s a song you would say, “God, this is the song I love and I want 100 more like it, but I don’t know where to find it.” What’s the song you’d put in today?
LEVITT: So if I had to put in a song today, I would put in a very obscure song, by a group called Iglu and Hartly, called “In This City.”
DUBNER: O.K., so to start this off we’ll play a song that exemplifies the musical style of Iglu and Hartly, which features “basic rock song structures, call and answer vocal harmony, also known as antiphony, major key tonality, and a vocal-centric aesthetic and acoustic rhythm guitars.” So that’s what you like.
LEVITT: I love – actually I love all of those things.
DUBNER: So the next time somebody asks you, “Steve Levitt, what kind of music do you like?”
LEVITT: I’m going to say, “I like antiphony.”
* * *
Imagine you fell asleep 150 years ago, and you woke up today. There aren’t many things you’d recognize. But one of them is a classroom — one teacher in a box with 25 students. Why hasn’t that changed? A while back, I put that question to Joel Klein. At the time, he was chancellor of the New York City public school system.
Joel KLEIN: In a prior life I used to represent the American Psychiatric Association, and it was during an era which there were a lot of light bulb jokes. And your people might be too young, but my favorite light bulb joke ever was, “How many psychiatrists does it take to change a light bulb?” And the answer is, only one, but the light bulb really has to want to change. And the answer to your question is, the school system does not want to change. It wants more resources.
DUBNER: Grades are up, graduation rates are up, but when you look across the board in New York City, when you look at the absolute numbers, not relative to where they were, but absolute, you can’t be too happy yet, right? You’re graduating about 59 percent of students now, so that’s still about 10 percent below national average. I hope I’m right on that. So what’s the problem? Are New York City kids dumb? Are the teachers who come here not good enough?
KLEIN: There’s a lot of questions there. So let me start with, for example, the graduation rates. If you count August graduates, which I think we should count, we’re at 63 percent, still below where we need to be. But quite frankly, in the last four years alone that’s gone up about three points a year. And that’s huge. No other city, I think, has seen that kind of gain. But, in the end two things, one has to be candid, looking at America generally, as you said, about 70 percent graduate high school. So 30 percent don’t graduate high school. Out of the 70 percent who graduate, probably 30, 35 percent of them are not prepared for college. So, what’s the point?
The problem isn’t only in New York City, of course. The national drop-out rate has been declining — but still, nearly three out of 10 kids never finish high school. Test scores have essentially been flat for a couple decades; other countries are passing us by. I asked Arne Duncan, the U.S. Secretary of Education, how big a concern these numbers are.
Arne DUNCAN: This is a huge deal, Steve. We are very concerned that if we are going to remain economically competitive as a country, if we are going to give our children a chance to compete in a global economy, we have to dramatically improve the quality of education. All of us as students want to be treated as individuals and we’ve had this sort of factory model of education in which everyone was treated the same. The best teachers are the ones that understand who you are, who find skills and abilities in yourself that you didn’t know you had in yourself. When it doesn’t work, those teachers are just relating to a mass of faces in front of them, they don’t know you as an individual, you get this sense that they don’t care about you quite frankly.
So that’s how a classroom is like the radio — a factory model, a big fat effort to pitch right down the middle. Isn’t it time to change the classroom model? I mean, people have been talking about education reform forever — and lately, things really are changing. There’s a proliferation of charter schools, some of them excellent. The Teach for America program. Parenting workshops and pre-school programs. And in New York City, a small band of reformers has been pushing a gutsy program with a catchy name: It’s called “School of One.” They want to dump the factory model altogether. Secretary Duncan is wild about School of One, and Joel Klein’s the man who gave it the go-ahead. Here’s the program’s founder, Joel Rose.
Joel ROSE: So in New York City, even before the recession, we had six applicants for every teacher that we hired. We have worked incredibly hard so that our highest-performing teachers are rewarded for the work that they do. And we can tell you on a relative basis who our stronger teachers are and who our weaker teachers are. But one of the things that I was interested in is looking at that information on an objective basis. In other words, if we picked a particular standard, like what percent of eighth grade math teachers in New York City get 80 percent of their students to proficiency by the end of the year. That number came in at 13 percent.
DUBNER: 13 percent!
ROSE: Right. And when we asked the question a little differently — what percent of eighth grade math teachers can get one year’s worth of growth out of 80 percent of students in the class — that came in at 12 percent.
DUBNER: Yikes! How about reading?
ROSE: In reading, the numbers are about half that.
DUBNER: Half that!
ROSE: Right. Now, we have some incredibly talented and hard-working teachers in New York City, and there are thousands of students who graduate college that would love to move to the city and work here. This is not an issue of recruitment or even of talent. It’s an issue of design. I mean, if you think about any other industry, if we create an objective standard of success and only 13 percent of a particular classification of employee was successful, we would change the job. We would never have that kind of conversation in education.
DUBNER: So you’re saying that if plumbers, or if IT designers or engineers, or writers, or presidents, performed at the level, at the objective level that New York City teachers did, we’d either get rid of all of them, or radically change the job?
ROSE: If we set the bar at how successful do we need plumbers to be, and that standard is a reasonable objective standard, and only 13 percent of plumbers can actually hit that standard, then we have to think differently about how we’re organizing the work of plumbing.
How should the work of teaching be organized? Rose started out as a classroom teacher, three years in a fifth grade in Houston, through Teach for America. He went to law school, then got back into education, with Edison Learning, the for-profit schools company. Eventually he landed in New York City’s Department of Education. That’s when he had a revelation: Even the most talented teachers in New York were operating in the old 25-kids-in-a-box, factory mode; how well can one person possibly teach math to 25 different brains at the same time?
ROSE: So I’m originally from South Florida, and I was visiting my family in Miami. I went to go see a friend who runs something called a New Horizons learning center. This is a franchise that provides technical training to people on a particular area of technology. So if you want to get certified in something from Microsoft, or Cisco, or something like that you go to one of these centers, you take a class, you take a test, you get a certificate, and you can use that to get a job. So I went to meet him for lunch and I walk into the office, and there’s a big sign right when you walk in and it says, “Choose your modality.”
DUBNER: Choose your modality.
ROSE: Choose your modality.
DUBNER: What’s that mean?
ROSE: That a student can choose to come and take live instruction from a teacher, they can choose to learn at their own pace at home online. Or they can do what New Horizons calls “mentored learning,” which is a student can come in whenever they like and there is a teaching assistant–type person that can help them as they work through the software.
DUBNER: So I can eat in, take out, or have delivery food?
ROSE: Exactly.
DUBNER: And did a gigantic light bulb flash off in your head?
ROSE: A gigantic light bulb flashed off in my head. What I immediately saw when I saw that sign is that is what we’ve been looking for in K-12 education, the introduction of other modalities. That live, teacher-led instruction, the kind of instruction we’re all familiar with, that we all grew up with, is one way, it may be the best way, but it’s not the only way that kids can learn.
Coming up, “one teacher in a box” versus one algorithm for as many kids as you want. I can see how many lessons they’ve had on each skill. I can see how fast they’re moving through those skills. And … we meet the man behind Pandora.
* * *
So Joel Rose thought: What if, instead of all 25 kids in a classroom trying to learn math from one teacher — in one modality at a time — what if you could take the classroom, divide it into smaller groups, and have each kid get her own playlist of different modalities every day, kind of like the playlists that Pandora makes for Steve Levitt? With Klein’s blessing, School of One quickly moved into pilot phase. In early 2010, it began as an after-school program in three New York City schools, teaching 6th grade math to 270 students. I visited one of these schools, I.S. 339 in the Bronx, and met Kirsten Shy, from School of One.
Kirsten SHY: So what we see here on the screens, the first thing when a student walks in, they see their schedule for the day. So right now it shows Period 1. Kids are divided up by their home room teams and they can see which area of the room they are supposed to be in. So at this table they are doing virtual tutoring, at the far table over there, there’s kids who are working on a variety of different online math programs and at this closer table, there’s kids that are working on independent practice. Which is another modality that we have.
DUBNER: Can you quickly name all the modalities for me?
SHY: Ooh! Quiz me! Large live instruction, small live instruction, virtual live instruction (which we call virtual tutoring, colloquially), independent practice, small group collaboration, and independent virtual instruction.
DUBNER: So these guys are getting, these guys are doing math homework with headphones at a computer and there is somebody is on the other line. Who is that person and what are they teaching them?
SHY: The other person online is a virtual tutor. I’m not sure exactly where they are in the United States right now, and they are working on the individual skills that they need to be working on, so it could be any number of 5th or 6th grade math skills that a kid needs at that time.
STUDENT: I’m talking to the teacher and doing his work. He’s telling us to add any number with two in it.
TEACHER: Now your question is, add three to any number.
STUDENT: Add three to any number? Three plus 40.
TEACHER: Suppose you wrote any number as 40, I could write it as 35 …
Small-group instruction? Independent practice? A virtual tutor? It’s not what these kids are used to. Here’s Patrona Hudson, 12 years old.
Patrona HUDSON: I was failing math because I didn’t understand it. Nobody sat down with me and helped me, like, work. Like, my teacher, she’ll explain it like once and if you don’t understand it, she’ll quit. She’ll be like, “Oh, I give up, like, too bad.”
LIONEL: My name is Lionel, and I’m 11.
DUBNER: Lionel. And what are you doing here in the School of One?
LIONEL: Math.
DUBNER: What about the kids who are in your class in math who don’t come to School of One? Do you feel like you’re brushing past them?
LIONEL: Yes, ‘cause I’m learning a lot more than what they’re learning. They’re probably at home playing video games.
Lionel. Patrona. Middle school math. Every lesson, every quiz, every keystroke — it’s all fed into the School of One algorithm. At the end of every day, each kid takes a test. Now, the algorithm can learn what the kid learned today. Which lessons stuck, which topics need more attention, how each kid learns the best. Every day, as the kid gets a little smarter, the algorithm does, too. Far from I.S. 339 in The Bronx, all the way down in Lower Manhattan, in the old Tweed Courthouse, the School of One team sits in a big, open room. There are laptops, whiteboards on wheels, the standard intensity of a startup. Chris Rush is a co-founder of the project. If Joel Rose is the education-theory guy for School of One, Rush is the tech savant. In some ways, he’s the algorithm. He’s sitting in front of two computer screens.
RUSH: Okay, so what I’m doing is looking at the results from the entire week, and I have a grid here that has every student and has every skill that they could be learning: a skill such as adding fractions of circumference of a circle. I can see how many lessons they’ve had on each skill. I can see how fast they’re moving through those skills. So what I’m monitoring is, I’m looking through those skills where kids are getting stuck so we can target those to improve the program around adding fractions or circumference of a circle. And I can also make sure that any kids that are getting stuck get the proper attention they need. So I tally how fast kids are moving through the skills and we give them a rating, sort of like par in golf. If they’re beating par, the average, or they’re ahead of it, and we take the kids that are falling behind, and we say, “What’s happening here?” How can we do something better for them? And update our algorithm, and pull them out to make sure they’re getting the proper attention.
DUBNER: Okay so we talked to a pretty enthusiastic boy name Lionel who was happy to be in the program, who was so happy that he was telling his friends they should stop going home after school and playing video games and should start coming here to play math computer games. He said he’s successfully recruited one or two kids. So let’s look at Lionel. Tell us what he’s been doing, how he’s been doing, what direction he’s moving in.
RUSH: So I’m going to pull up Lionel here, and what I really see, Lionel is one of those great case studies, actually. The nice story about Lionel is that Lionel was really struggling in the month of March, it was taking him 10 exposures, 12 exposures to learn certain skills, and now Lionel is moving through certain things with two exposures, three exposures. That’s really exciting, that’s what we want to see – not just learning the skills, but he’s learning how to learn the skill faster. We have one skill here that’s on hold, because Lionel needs to come back to it, and that’s dealing with, this particular skill happens to do with algebraic expressions, and that seems to be where he’s having more trouble. Where geometry seems to be something that he’s doing much better in.
DUBNER: So what do you do with that information? Where does that steer you to help steer him?
RUSH: So we’d look a little bit more closely. Is it geometry that he’s doing well with, or is it the way that we’re teaching him geometry? And in this particular case it looks like we were actually teaching him geometry a bit differently than we were teaching him how to deal with algebraic expressions. The first thing we would do is try to take the methods that we taught him geometry and try to apply those to how we taught him algebraic expressions.
DUBNER: So the idea I’m guessing is that you will find out which approach works best for each kid and that you will end up with a manageable mix, because if everybody needs the live teacher instruction then you’re back to 28 kids in a box, yeah?
RUSH: Correct. It’s very much like the Pandora music service online, where you enter in a song name, or an artist and it tries to guess the next best song for you. And maybe it guesses right the first time, but two or three songs in, basically it works the same way here. If a student tries small group collaboration and they don’t do well that day, we give that a thumbs down. If they do small group collaboration and they do well, we give it a thumbs up. So we can rate our schedules every day for each student so we get smarter and smarter at choosing the next day’s schedule for each student.
DUBNER: O.K., so Tim, you’ve let me hijack your Pandora account. We’re listening to the Pandora account of the Pandora guy. We’re on Ben Folds Radio, that’s your station here.
Tim WESTERGREN: That’s the heart of my genome.
DUBNER: Is that right?
WESTERGREN: Yeah, I dig all that stuff.
Tim Westergren is the founder of Pandora. It’s built on what he calls the “Music Genome Project,” whereby a bunch of musician/analysts painstakingly break down each song into hundreds of qualities – from rhythmic patterns to guitar style to … antiphony. In the summer of 2011, the company went public, and was capitalized at nearly $3 billion. Pandora’s database now contains nearly a million songs by 100,000 artists; it has more than 100 million registered users.
WESTERGREN: That’s right, it’s grown like a weed.
DUBNER: Wow, so more people are Pandora listeners than are, let’s say, users of Reynolds Wrap probably.
WESTERGREN: Not the first correlation I would have thought of, but that may be true.
DUBNER: So what Pandora does is you customize a radio station for anyone who asks you for as many radio stations, as they want. So, do you feel a bit like we’ve entered a new world where essentially everything is customizable?
WESTERGREN: Yeah, I think that’s basically true. I think what the web did at first was made everything available. That was kind of the first phase of the web. And the second phase is to kind of repair that and make all that stuff that’s so voluminous actually navigable. And I think that’s where Pandora sits. You know, we look at music and say you’ve got a tyranny of choice on the web, like walking into a record store and you’ve got 80 thousand CDs to choose from and you need to figure out where to start. We’re trying to do that with music, and the Music Genome Project’s sort of purpose is to make that easy for someone.
Our public schools are longing for a tyranny of choice — the misery that comes from having too many options. Is School of One the answer? That’s impossible to say. So far, the early returns look good. Here’s Joel Rose again:
ROSE: Before the program began, every student took a pre-test that was aligned to the skills on their playlist. The average score was 42 percent. At the end of the program, roughly 50 – 60 hours of instruction later, they took a post-test on the same skills. Post-test scores average 70 percent. We asked the DOE research and policy team, is that good? Is that not good? How do we contextualize that? They did an analysis and found that these gains were, depending on what we consider a control, 4 – 8 times the gains we would see in traditional schools in roughly a third the time.
If and when education is customized, there’ll be fortunes made, and maybe lost. There’ll be competition among technology vendors and content providers and a million others. There’ll be turf wars with teachers unions. Would School of One require fewer teachers — or maybe more? Less money — or more? Joel Rose says it’s simply too early to answer these questions. What we do know is that a future with something like School of One will be different. Very different. Technology, instead of being discouraged in schools, would move to the head of the class. Teachers would have to be trained differently. Here’s Blair Hyser, the math department chair at I.S. 339 and a teacher at School of One:
Blair HYSER: The fear of School of One is just that students might get lost, given that there isn’t that traditional 25 kids/one teacher, and that teacher works with those students every day. So I think that is the one hesitation or fear, it’s different, it’s new, it’s something that hasn’t really happened before. As a teacher, if you’ve taught 15 years or 20 years in one way, it’s very challenging to kind of think outside the box in terms of your instruction.
Arne Duncan, the Secretary of Education, thinks that most of what’s happened in the past isn’t worth hanging onto.
DUNCAN: I think our Department of Education has been part of the problem historically. I’ve been very honest about that. And as hard as we’re pushing everybody else to move outside their comfort zone and do more, I promise you that we’re trying to be very self-critical, look ourselves in the mirror every single day. And historically, we’ve been this compliance-driven, big bureaucracy. And we are fundamentally trying to change what we do from being this big bureaucracy to be this engine of innovation and scaling up what works.
Instruction already is changing. Across the country, there’s been a big spike in students enrolled in full-time virtual schools. The Khan Academy — with its simple, do-it-yourself video lectures — has started distributing its material to school systems. And since we first talked to Joel Rose, there have been big changes at School of One. He left the project to start a new non-profit organization that hopes to take the mixed-modality model national. But let’s not get too excited. A lot of experiments look good in the lab. New York City alone has 1.1 million schoolchildren: Can they really be taught one at a time? Can an algorithm, a playlist, help teachers do their jobs better? Can that algorithm really help a kid like Lionel find his own music? The odds seem long — but just think how long the odds must have seemed 10 years ago flailing around at the radio dial, hoping to find just the right song. Have you heard from anyone for whom Pandora really changed the shape of their life in some way, maybe it resulted in a courtship and a marriage, a divorce, a murder.
WESTERGREN: We had someone whose father was ill, or aging, and was in a home kind of reaching the end of his life. In the last few months, they discovered Pandora, and they shared it with him, and he really took to it. So when they visited him everyday, he would listen to Pandora all day long, and it became kind of a companion. He eventually passes away, but he passed away listening to Pandora. And because when you die you are given a time of death, the family wrote us to thank us, and told us the time of death and we actually wrote them and told them which song was playing when he died. And we sent them a CD and they played that song at his funeral.
* * *
Freakonomics Radio is produced by WNYC, A-P-M, American Public Media and Dubner Productions. This episode was produced by Collin Campbell and Aimee Machado. Our staff includes Suzie Lechtenberg, Diana Huynh, Katherine Wells, Bourree Lam, David Herman and Chris Bannon. This episode was mixed by Michael Raphael and John DeLore. Special thanks to Jacob Berman. If you want more Freakonomics Radio, subscribe to our free podcast on iTunes and go to Freakonomics-dot-com where you’ll find lots of radio, a blog, the books and more.
Comments