Extra: An Update on the Khan World School

A year ago, back in episode 82, I talked to Sal Khan, the founder of Khan Academy, one of the most visited online education sites. We were talking back then about his big new idea, the Khan World School. It’s a fully online high school with a self-paced curriculum; content delivered largely through videos; and a weekly deep dive into one big important issue. But the last time we talked a year ago, the model was unproven. Here’s Sal at that time, talking about his hopes for what the school could be.

KHAN: And so we said, “Let’s do the world’s best high school that happens to be online.”

Welcome to People I (Mostly) Admire, with Steve Levitt.

Today in this special bonus episode, I sit down with Sal to hear how the first year turned out, and I chat with Chloe Peterson, one of the students attending the school.

*      *      *

LEVITT: Can you just explain the key concept behind the Khan World School? What makes it different? What makes it better?

KHAN: In 2012, I wrote a book, The One World Schoolhouse. And in that was: How did the education system that most of us have lived in and still exists — how did that come to be? Mass public education, it was a hugely positive thing for society, but it had to make some compromises. It borrowed tools of the Industrial Revolution. It batched students together, put them essentially on kind of a standards-based assembly line where the teacher might give some lectures and at some point there’s a check to see if the students got things. And if they didn’t, too bad; the assembly line keeps moving on. A lot of students start accumulating gaps and you get to the world that we get to today where the average American student learns about 0.7 grade levels per year. Now you have tools like Khan Academy. At least in certain domains, you can start learning at your own time and pace. The last third of the book was: “What could a school of the future look like if you have tools like this? Can we use technology tools for more personalized learning?” Which is just learning at your own time and pace. If you haven’t learned something yet, you can keep working on it. That’s mastery learning. But also this notion of: if you’re able to do some of this core knowledge and skills more efficiently, can you free up more time for more project-based learning, Socratic dialogue, just getting kids out there into the real world. It’s one thing to write all of these ideas. It’s obviously a whole other thing to implement it.

LEVITT: That’s for sure.

KHAN: In 2014, based on that, we started a lab school out here in Northern California. I was somewhat selfishly motivated. My oldest child at the time was entering kindergarten and I also wanted to test these ideas and I wanted to show the world that it’s not just theory, that it actually could be done at a reasonably cost effective way. Khan Lab School is now K through 12. It’s had some really promising results. So that’s been going on. And then, you know, in parallel, the pandemic happened. And in the pandemic, as we know, the whole world had to transition to some form of online learning. I think most people had a bad experience when the traditional academic model that was essentially developed in the Industrial Revolution was just transplanted to Zoom. And now kids were just on Zoom for hours a day, eyes glazed over. So it left a bad taste in folks’ mouth. But at the same time, based on our experience at Khan Lab School, we said, “No, there’s another way of doing this, where you could actually make an online school that can lean into best practices of connecting humans, having them discuss things, having them do things, but also leverage tools like Khan Academy and others to give students more autonomy where you can.” And this could also be an opportunity to scale, because obviously one lab school out here in Northern California — it has 300 kids. It’s not going to scale to hundreds of thousands. And so that’s where we said, “Well, maybe we should start an online school to show how this could be done well.” We got connected with A.S.U. Arizona State University is pretty consistently on the cutting edge of, like, just trying new things out in the education world. They already had A.S.U. Prep, which is one of the largest online high schools that people respect. There’s some other online high schools that a lot of people don’t respect. And I just called them up just for advice, and Amy McGrath, who leads that effort, said, “You want to do this together?” ‘Cause this is what they always believed as well. And I’m like, “That would be amazing.” So we launched Khan World School, as you mentioned, almost a year ago. We had 50 students from around the world. The school is actually free to anyone in Arizona. Khan Academy’s mission is free world-class education for anyone, anywhere. And it’s obviously — it’s an aspiration. But now, actually, Khan World School literally is a free world-class education for anyone in Arizona. But, outside of Arizona, we’ve tried to make it as approachable and as cost effective as possible. On top of that, some very generous folks — especially yourself, Steve — have made donations to support scholarship students.

LEVITT: If you don’t live in Arizona and you don’t have a scholarship, it’s $10,000 a year. Not much, really, compared to the typical private school.

KHAN: That’s right. Khan World School — one of the things I’m most proud of, and this is something we’ve worked on with you and your team, is: Yes, we use Khan Academy and other tools for students to be able to learn at their own time and pace. But they get supported Oxford style with tutorial instructors and they have a lot of community, so it isn’t that you’re just working in isolation. But most days for these students are anchored by a Socratic seminar. And these seminar topics are ones that we’ve developed with your group. And they’re topics that are multidisciplinary that you actually care about. Is A.I. going to benefit or hurt humanity? Should we be allowed to use CRISPR to modify the human genome? Plus all of the traditional philosophical debates or hypotheticals from history. What if Caesar never crossed the Rubicon? Really interesting things that take ideas from philosophy, from history, from civics, from science, and put it all together. And so the students are able to connect with each other, learn to communicate and collaborate, and we really haven’t seen that in the online school world before.

LEVITT: And one other thing I want to add, Sal, to the seminar concept you’re talking about is that my team made sure to embed data science in it because I’m incredibly passionate about the idea that kids should know how to deal with data. And I think the best way to learn about data is not to be given a data science course, it’s to have it integrated into everything you do. And I think these 50 kids really got a quadruple dose of data as they went through their first year. Sal, I know neither you nor I thinks of standardized test scores as the be-all and end-all of education, but they are a form of feedback on whether a school is working. And the test score results for Khan World School — they are so good that when the Khan World School people sent me the test results, I actually thought that they must have made some kind of mistake. I have literally never seen results like this from a school in 25 years of research I’ve been doing in education. And I actually insisted that they send me the raw data and let me look at myself before I would believe it. And I’m happy to say, everything checked out. Were you as surprised as I was by the test scores?

KHAN: Simple answer is: yes. And to your point, we know no test, especially no standardized test, is going to be perfect. But when I saw those results, I was both excited and I, myself, was skeptical. I actually haven’t been telling a lot of people about it because they almost seem too good to be true.

LEVITT: The Khan World School students, they took a test called Exact Path, which is used by about 8,000 school districts nationwide. So this is a pretty widely known and respected test. Okay, so they took the test, the kids, at the start of the school year and at the end of the school year. And the key question is how much does an individual student’s score grow relative to their starting value? It’s what education researchers call the “value added.” So let’s start with math scores. The typical American ninth grader on this test sees his or her math scores go up by 19 points, on average, over the course of ninth grade. In this context, 19 is a benchmark against which to compare the Khan World School students. So at Khan World School, the average increase in scores — it wasn’t 19 points, it wasn’t 29 points, it wasn’t 49 points. It was 83 points! That’s more than four times the gains of a typical student. I mean, my god, that’s absolutely incredible. And honestly, it would have been even higher, except about a fourth of all the Khan World School students, they hit the maximum. They got perfect scores on the math test at the end of the year. They couldn’t increase anymore because the test wasn’t hard enough for them.

KHAN: It’s weird because the broader education establishment would cut off their left arm for 4 percent. We’re talking about a 400-percent improvement. You’re right. I actually think we’re facing this ceiling effect of the test itself because a lot of these students, you can imagine if you learned at 4x the pace in ninth grade, you’re actually already doing like 11th or 12th grade math, which these tests don’t test.

LEVITT: I might have expected that the Khan World School model with a lot of independence for the students and the need for self motivation, it would be great for some students and lousy for others. But by my calculations, about 90 percent of the students at Khan World School had above-average test score gains in math. So the K.W.S. approach to math, it seemed to work for almost everybody, at least this first class.

KHAN: Yeah. And one thing to be clear, a lot of folks might say, “Oh, well, you know, this is a school that maybe there’s some self-selection for kids who are motivated, who want to do this.” But to your point, these are growth measures. This is wherever your starting point was. Usually it’s a case where students who are more ahead actually have trouble growing faster because they’re already to some degree starting to hit like a ceiling effect. And on top of that, Khan World School, it’s not that every student that joined was a superstar in math. We were definitely looking for students who we think could thrive in this environment. So there was some selection involved for sure, but it was not that every student is some type of a math prodigy. That’s far from what’s happening.

LEVITT: Now, people might reasonably say, “I’m not surprised that kids did well in math. Khan Academy math is widely known to be excellent.” But I don’t think anyone, even you, definitely not me, expected the kind of gains the students experienced in reading. Okay, so the typical American ninth grader gained 17 points in reading over the course of the school year on the same brand of tests that you used. And the K.W.S. kids, they gained, on reading, 58 points. It’s more than three times bigger than what’s expected. In the research studies that I’ve been part of, reading scores tend to be much tougher to affect than math scores. This is just, again, mind boggling.

KHAN: I agree. And if I had to guess why we’re seeing that, I think a large part of it has to do with the seminar, where students know that they’re going to have a rich discussion about interesting things, and we give them a good amount of reading. And, when you’re motivated to read and dig into some sometimes difficult texts, that’s going to improve your reading. Then on top of that, they have some choice about when they do it and how they do it. I think we all remember being in school, sometimes reading a book that you don’t necessarily get into it. But when you get into it, when you enjoy it, when you have agency and you have choice and you have either the seminar, where you can show what you know and argue and really dig into these ideas, or the students have to make videos of themselves talking about different aspects of the book. I also think that helps you process things when you have to talk about it orally — but I was surprised as well that we saw results that good.

LEVITT: I also suspect that this school has created a culture around kids being expected to be self-motivated and to learn. And I think it was just contagious.

KHAN: I think you just hit the nail on the head on probably the most important thing, which is the environment in which something is happening. If you can get the culture going where the dominant culture is one of learning; where it’s safe to explore ideas, share ideas, not be judged; one of creativity and positivity — you almost at that point just have to get out of the way.

After the break, I’ll talk with one of the students at Kahn World School.

*      *      *

PETERSEN: My name is Chloe Chung Peterson. I’m 15 years old. I grew up in Shanghai and right now I’m in the process of moving to Hong Kong.

LEVITT: So tell me, how did you first hear about Khan World School.

PETERSEN: Through this podcast, actually. My mom is a big fan and so she was listening to it and she was like, “Chloe, oh my God. I found out about this school and it seems really interesting. You should apply.”

KHAN: And so were you a willing participant? Or did you get dragged along kicking and screaming?

PETERSEN: I mean, at first I was a little bit unsure because it’s such a new concept and I didn’t really know what I was getting into.

LEVITT: Were you apprehensive because it was the first year of the school — it must have been hard to even have an idea of what it would be like. I’m not sure anyone knew what it would be like. It was being made up as people went along.

PETERSEN: Yeah. The main thing for me was the idea of it being an online school because I had some bad experiences with online school at my old school with Covid and everything, and so I didn’t really want to go through that again and sit through hours and hours of just lectures on a computer. But after figuring out how my school day would be structured and how they teach and do their lessons and their approach to that, I started liking the idea more and became more open to the school.

LEVITT: So let’s talk about math. What is math at Khan Word School?

PETERSEN: So math, we use the Khan Academy courses. In each unit you have videos, you have practices, and then you have quizzes, and then you have the unit test. I usually prefer to watch some of the videos to get a general concept of what the math is. And if I understand it, then I might do a quiz and see if I know it. But if I don’t, then I’ll go back, watch the videos, take some notes. Once you finish that in Khan Academy, you record yourself taking the test, and then the teacher double checks it and you pass.

KHAN: Who decides what you should study? ‘Cause all the kids aren’t studying the same math because it’s based on what you already know, right?

PETERSEN: At the beginning of the school year we had an Exact Path test for math, which shows the teacher where your level is at, and then she would assign you courses. So last year I took Algebra one and Geometry, and next year I’m going to do Algebra Two and Pre-Calculus. But some people whose math was already pretty good, they did Algebra Two and Pre-Calculus this year and next year they’re going to do Calculus as a unique course.

LEVITT: Let me just stop you, ‘cause you just said without skipping a beat that you did Algebra One and Geometry. You covered two years of math in one year in what would typically happen in a U.S. classroom. It’s exactly the point that Sal Khan has made, which is that if you let kids who have a little bit of grit and self-motivation learn math in their own way, it’s just so much more efficient than sitting a bunch of kids in a classroom and having a teacher lecture them all on the same subject. It’s what economists call a different production function for knowledge. And I think what we saw this year in the test scores at Khan World School really reinforces the idea. And it sounds like for you it was pretty painless. Is that your experience?

PETERSEN: I guess so. I was never really bad at math, but I wasn’t good at it. But I think when I was at Khan World School, it was a nice subject just because of the way the lesson was structured and just made me like math.

LEVITT: Would you call yourself a math person now?

PETERSEN: Uh, I wouldn’t say I’m a math person, that’s a little bit too far, but I don’t dislike it.

LEVITT: Did you get a sense of the culture? What was the vibe that the school gave off?

PETERSEN: Well, from my interactions with the students, they were all really nice and supportive. I think that people here really do value hard work because if you’re signing up for school that’s completely self-paced, you’re not going to look down upon people who are trying to do their best and stay motivated. Before this, I would procrastinate quite a bit, so I would sometimes be doing homework late at night because I got distracted. But here, since there’s no, like, real structure or deadlines, you have to learn to plan out your day and stick to that plan. So I definitely learned how I do work best and I figured out a way to manage my time in getting all of my schoolwork done.

LEVITT: In the U.S. we have a huge mental health crisis with teenagers. Covid obviously contributed to it, but it’s deeper than that. There’s just a sense that things are going wrong. There’s too much stress, too much competition. What do you see as the pros and cons of Khan World School when it comes to your own mental health?

PETERSEN: As I’m getting older, I’m getting a lot more stressed about college and my grades and my extracurriculars and there’s this pressure to do well and do as much as you can. Especially in Asian culture, there’s such a pressure to be the best that you can and get extra training. I know some people who have, like, extracurriculars that go until 9:00 PM and at my old school, I would also get home at like 8:00 PM because I was doing piano classes or dance classes and my school would end at 5:00. With Khan World School, I don’t have homework, so that’s less pressure, because I do everything in my school work independently, so I don’t have to have this sort of rigid structure that gets in the way of me wanting to do everything that I want to do. It does take away from that competition between your peers and trying to impress them or the stress of failing in front of them. Because it’s self-paced, it’s a lot more relaxed. A lot of it is depending on how you feel and your mindset. So if you’re not feeling well, there’s no pressure to always be on and always try your best because people get burnt out and sometimes they need to take a slower week. And so Khan World School lets you do this.

LEVITT: Yeah, it sounds like the way you’re describing it, your competition in some sense is yourself at the beginning of the year. That’s what you’re benchmarked against is: what do I know and how do I learn more? At least from my vantage point, it seems like they’ve done a really good job of not pitting you against the other students, of making it about your own journey. In my own experience in school, it was always about the other kid. “Oh, am I better than the other kids? Am I losing to the other kids?” Which in retrospect doesn’t seem that healthy because when you leave school and you go to a workplace, it’s very cooperative. The whole norm of a job is: everybody’s working together as part of a team. When I was growing up, I really had the idea that cutthroat was the right way to do it, and I had to unlearn that. So it sounds like Khan World School is doing a good job in that regard. Would you agree?

PETERSEN: Yeah, definitely. At the end of the day, the school isn’t about the test score, it’s about whether you know the knowledge. So you can retake a quiz if you didn’t do well on it, and you can fix your essay if you didn’t get the score that you wanted or you messed up a little. And I think that’s pretty good because in a real-life environment, it’s not like you submit a paper or you submit like a business idea and it’s not good and they just fail you and they fire you. They’ll give you feedback and they’ll help you improve it or you go back and improve it yourself. But in a real school, that’s not something that you necessarily learn because a bad score is just a bad score and it’s not an opportunity to do better.

*      *      *

LEVITT: So if you stop for a minute and you just pause and you reflect on how much you’ve accomplished in just a year on the Khan World School, it must feel pretty good, no?

KHAN: It does. But this is probably a personality issue, that as far as any of my projects go, I’m always thinking about, “Well, this is nice, but wouldn’t it be more amazing if we could get this to more students and make it even better?” I think to your point, the model has already surprisingly gone better than I expected. My mission at Khan Academy is I want to move the dial for everyone, but I also know there’s some kids who just — their potentials there, but just because of the context that they’re in, they’re not able to tap into that incredible potential and I hope that Khan World School can be a life preserver, give that chance to hopefully, over time, many more students. And those kids are going to be the ones that cure cancer and address global warming and be leaders who actually have our common interest at heart.

When Sal Khan first told me about Khan World School, I was so inspired that I offered to help create parts of the curriculum free of charge, and I provided scholarship money to pay the tuition for five students. Having watched how things played out in the first year, my enthusiasm has only grown. If you know of a self-motivated, rising ninth or 10^th grader who’s bored or frustrated at his or her current school, tell them about Khan World School. There’s still time to apply for this fall, but you have to move quickly. If you Google “Khan World School,” that’s K-H-A-N World School, you can find all the relevant information. And if you’re an adult who is as inspired as I am by this new vision of what high school could and should look like, tax deductible scholarship donations are greatly appreciated. I’m providing five scholarships again this year, and if you’re interested in providing financial support, reach out to me directly at pima@freakonomics.com and I can put you in touch with the right people. Next week, we’ll be back with a brand-new episode featuring artist Wendy MacNaughton. I’m definitely stepping out of my comfort zone on this one. Wendy will be the very first artist I’ve had on the show.

LEVITT: But I bet you don’t talk to a lot of economists either, though, right? When’s the last time you talked to an economist?

MACNAUGHTON: That’s a really good point. No, I don’t. We can be each other’s exceptional friends. How about that?

Thanks for listening and we’ll see you next week.

*      *      *

People I (Mostly) Admire is part of the Freakonomics Radio Network, which also includes Freakonomics Radio, No Stupid Questions, and The Economics of Everyday Things. All our shows are produced by Stitcher and Renbud Radio. This episode was produced by Morgan Levey with help from Lyric Bowditch, and mixed by Jasmin Klinger.  Our theme music was composed by Luis Guerra. We can be reached at pima@freakonomics.com, that’s P-I-M-A@freakonomics.com. Thanks for listening.

KHAN: I combed my hair.