Episode Transcript
Steven LEVITT: I am especially excited about today’s episode because my guest is Sendhil Mullainathan, one of my absolute favorite people in the world. Sendhil is a world-class economist, a MacArthur “Genius” recipient, and holds a university professorship at the University of Chicago. That’s a special position reserved for only the most highly esteemed faculty. Only 10 of the university’s 3,000 faculty members have that title. What makes Sendhil so special? His mix of brilliance and childlike curiosity make him unlike anyone I’ve ever known. His research reflects his curiosity. He’s written about everything from racial bias and artificial-intelligence algorithms to the psychological effects of poverty, from the determinants of C.E.O. pay to corruption, and who gets a driver’s license in India.
Welcome to People I (Mostly) Admire, with Steve Levitt.
LEVITT: I’ve known Sendhil for 25 years, but it wasn’t until he left Harvard to come to the University of Chicago a few years back that I came to fully appreciate him. Lots of people are interesting if you only talk to them once a year. But not many people can make everyday conversation fascinating. He just never fails to amaze me with his creativity and breadth of knowledge. And I’m quite confident that once you hear him talk, you’ll agree.
* * *
LEVITT: Sendhil, I always eagerly look forward to our conversations and two reasons I love talking to you are, first, there’s always something that has you so excited you can barely contain yourself. And second, what that something will be is completely unpredictable. So the last time we talked, it was an Oculus virtual reality headset that had transformed your life. So is your love for the Oculus still going strong?
Sendhil MULLAINATHAN: Absolutely. I have to say it is one of the more amazing inventions. I’m sure that lots of people out there who’ve been using it for years are like, “How is this an insight?” But these media for communication, by which we can translate ideas to each other, they may start off as novelties but they have profound consequences on how we live our lives.
LEVITT: So, I was convinced when we talked and you were so excited. I went right home and I immediately bought an Oculus for my research center because one of our goals is to transform the educational system. And the way you described your experiences, all I could think about is that the possibilities, if we got this technology into classrooms, must be infinite. And I must say the workers in my center, I’ve never seen them tackle a project with such gusto. Our Oculus has been in use, I’d say 10 to 12 hours per day, every day. And all of that time, I’m sure, is devoted to exploring the educational potential of the device. One thing I know about you is you don’t think small, you always think big. What’s your big vision for what happens with V.R.?
MULLAINATHAN: Let me throw out an idea for you and see what you think. Why do we go into offices at all? I mean, at all. Why do we fly across the country? Why do we do any of it? And it’s because there is still a reality that somehow when you and I talk on the phone or when you and I even talk on Zoom, it’s not quite the same as being in person. Now, it’s entirely possible that we can recreate that experience 100 percent with virtual reality, which if you think of the knock-on consequences — let me give you an example. One of the things that people say is that the cell phone decoupled a phone number from a location. Suddenly the cell phone made it so that a phone number referred to you, not the address at which you were sitting. Now that has profound consequences. If similarly your office location was not a physical place, but was you — you can work from anywhere. You can do anything. It totally transforms our capacity to interact with each other and how we interact. So, I’m curious what you think of that one.
LEVITT: I actually thought you were going to say something different. I thought you were going to emphasize the fact that with virtual reality, you can bring any place to you.
MULLAINATHAN: Oh, I love that.
LEVITT: As well as you bring you to any other place —.
MULLAINATHAN: I love that.
LEVITT: So, you can be in the Grand Canyon or under the water or inside a molecule — you can be miniature, now, moving around, NaCl —.
MULLAINATHAN: Yeah.
LEVITT: Sodium and chlorine atoms.
MULLAINATHAN: Yeah. That’s a great idea. Including places that aren’t really places, per se — you could be inside of an atom, as you said, you could be inside of the brain watching neurons fire You’ve seen this, right? Like a good figure, a good drawing of some effect suddenly changes your understanding of it. So, people say Maxwell, for example, came up with his equations for electricity in large part because he came up with these good diagrams and the same is true of Feynman diagrams, et cetera. Imagine what it’s going to do for science. If you can put yourself in a high-fidelity simulation of the thing that you’re studying. Bring the place to you is a terrific way of putting it.
LEVITT: What’s the path? The one problem is you’ve got this two-sided market issue, which is that let’s just say, schools are the target, if schools don’t have the technology and adopt it, then it doesn’t make sense to develop the programs for it. But the schools won’t adopt until you have the programs.
MULLAINATHAN: If you look at a lot of the early-day uses of computing, they weren’t blow-you-away. They’re just things that really got some people excited. Like I remember calling up Bulletin Boards to do virtual Dungeons and Dragons with — like there was a modem. I still remember it going “churchurchur.” Now you would have said, “We built all this technology so you can play a role-playing game?” Well — whatever. But it’s related to your research assistants who work at your center. They’re playing and play is so underrated because it just fuels this first phase of all sorts of infrastructure building. And so, I think even if 10 years out, we imagine all these grand changes. It heartens me that right now people who start playing with it, enjoy playing, and then start tinkering, and make stuff they enjoy. Because that gives you that kind of base-layer of stuff and just creates demand. And from it, you get all this other cool stuff.
LEVITT: And never having done it myself. What makes it so special?
MULLAINATHAN: Here’s a psychological fact that I think many people may not have realized. It’s like when someone says to you optical illusion, you think, “Oh, I know that it’s like this image where it’s an old lady or a young woman — you can look at it both ways.” But they forget that like every photograph, every T.V. screen is an optical illusion because it is truly two-dimensional, but your mind creates three dimensions. There’s no three dimensions. It’s just two. But that optical illusion is actually relatively low fidelity. You put on the V.R. set and it feels 100 percent like 3-D. So for example, there’s this little thing with Jurassic Park and a dinosaur. And I was like, oh, let me play this video. Your heart is beating. And when the thing’s tail comes at you, you say to yourself, “It’s not a real tail. Do not duck.” And then you’re like, sh*t, I’m ducking before I know it. It taps into this very basic sensory system and that’s just amazing.
LEVITT: This is going to seem off point, but about 20 years ago in a seminar, you said something, Sendhil — and I remember virtually nothing from 20 years ago. It’s incredible that I remember this. You were giving a seminar. We were both just young professors getting started. And you said something like, “A stranger tells you some random fact, you’re far more likely to remember that fact if the stranger is wearing a bright orange clown wig then if she’s average looking, even though it’s the same fact,” and your point was about salience and how things that are unrelated to the actual content have a huge impact on how people perceive it. Now, I find it so ironic that I can remember this point you made about salience — somehow you turned into the clown. Do you remember saying that?
MULLAINATHAN: I don’t remember the clown specifically, but it is something that’s stuck with me. The mind doesn’t code information because it’s important or you really want to remember it, it codes information because of these kinds of salient things. And it reminds me of — I’m not gonna ask if you read it, but have you at least pretended to read Marcel Proust’s, Remembrance of Things Past?
LEVITT: No, I’ve never even pretended to read it.
MULLAINATHAN: Some of us have pretended to read it. It’s six volumes or something so at a minimum you can say things like, “I read volume one,” even though you basically bought it. So, the opening scene in there is exactly this. He’s in bed and he bites into this madeleine, this pastry, and suddenly he’s flooded with this memory of being at his aunt’s house. And he realizes it’s because the flavor of this pastry is strongly associated with that time at his aunt’s house. And something as simple as the smell and flavor brings with it these evocative beautiful memories. And you’ll notice this in yourself, like smell — so salient — brings with it so many things. The memory system doesn’t work the way you want it to work.
LEVITT: And so is it your hypothesis that lessons taught through virtual reality might be much stickier than other means of learning?
MULLAINATHAN: Absolutely. Yeah. I’d say that in general, we learn so much better when we’re doing things than when we’re just listening to things. I’ll give you an example. If I say to you, “Think of one of your favorite meals from a past vacation five years ago,” you can probably remember — like you had it exactly once, but you can probably remember the dish and you can probably remember what was in it. And you remember the feeling you had. That’s astonishing. If I said to you, “Remember a good conversation,” I’d be like, “I don’t know. We talked, it was nice. It was fun.” But taste, smell — these things, they’re just hard wired back in.
LEVITT: So, along those lines, I told you something I remembered from 20 years ago, but I also want to make a confession to you. You wrote a book and — I don’t know, it must’ve been almost a decade ago and the book was called Scarcity and I love the book. I read it in two or three sittings. My blurb is on the cover of the book and I’ve raved about it to anyone who would listen. So, as preparation for this conversation today, I went back and I looked at the book for the first time in ages. Obviously, I remember the key thesis of the book. Like you have a really simple point that has stuck with me, but I would have sworn I have never read the book before. It’s loaded with these stunning experimental results and these interesting stories. And as I read them, I felt like I had literally never read about this experiment before, when obviously I had — And I would have thought sure, I couldn’t reproduce if someone said, “What’s on page seven of Sendhil’s book?” I won’t be able to do it. But I would have thought that oh yeah, of course this is the study when the people weren’t allowed to have lunch and then they had to do the word search. Oh yeah. I can’t believe — nothing. The good news is I enjoyed it just as much a second time, because it was all new to me. But the bad news was, it really scared me because I would have expected that seeing the stories I would remember them. I wonder if my chronic lack of sleep from having too many kids really left me somehow deeply, mentally impaired. What do you think?
MULLAINATHAN: So, that’s at the heart of this app that we’re releasing that I want to tell you about in a second, but I also want to tell you that your experience is the experience everybody has. Lately, I’ve asked people the following question: “Hey, what’s a novel that you really like?” And they’ll tell me the novel. And I’m like, “Great, can you tell me how it ends?” And they’re like, “Oh…” I say, “Okay, fine, you can’t do that. Can you tell me two of the main characters’ names?” They’re like, “I remember one kind of…” and it’s amazing how quickly things that you’ve read, they come in and they go out the other ear. It’s just astonishing.
LEVITT: Do you think that’s particular to reading? Is reading an especially bad way to gather information or do you think it’s more generic?
MULLAINATHAN: I think it’s two things. We imagine the mind to be this sort of warehouse of information, but don’t appreciate how much it’s just open doors. It’s like a train station that people walk through. Like ideas are just going through the train station of our mind — not many things stay behind. But you’re onto something. I think reading is just such a bad way to learn because in many ways, it’s passive. It’s not experiential. Like you remember your experiences, you remember the feeling of being on a rollercoaster, those physical highs, those ahas that you yourself have experienced, but not the things that you’ve read about. One of the things therefore I try to do in class is rather than teach people things, I try to create little experiences for them. You’ll like this one, Steve. So do you know this $20 auction? You basically say I’m going to auction off this $20 bill.
LEVITT: The both pay version? Where both people have to pay?
MULLAINATHAN: Both people have to pay.
LEVITT: I love that. Yeah. Tell people about it. It’s incredible.
MULLAINATHAN: Yeah. In class, I hold it up. I say, “Here’s a $20 bill, I’m gonna auction it off, and so I’m going to give it to the winner just to make things interesting, the highest bidder wins the bill and pays. The second highest bidder doesn’t get anything, but they also pay.” So, at first everyone’s a little hesitant and you’re like, okay, $20 — and minimum increments of bids of a dollar. So of course somebody says a dollar and you say, “Okay, a dollar going once, going twice…” — I’m not gonna let this guy get $20 for a dollar. So someone else bids two. And that’s all you need.
LEVITT: So, the key is that unlike every other auction in the world, both the high bidder and the second-place bidder, both have to pay, they pay whatever their bid was.
MULLAINATHAN: Exactly. And the highest bidder wins the bill.
LEVITT: Yup.
MULLAINATHAN: Okay. So, once you have one person bid a dollar and somebody else has bid $2, think of what is about to happen. I say, “Okay, going once, going twice…” the dollar bidders like, well I’m not going to let this guy get it for two — to get to three. And they just keep leapfrogging each other. A crucial thing happens when someone has bid 10 and nine, the $9 bidder says, “I’ll bet 11.” At that point, I’m making a profit. I’ve now just auctioned off a $20 bill, at a minimum, I make a dollar profit because someone’s bid 11, someone’s bid 10, I’m getting 21, I’m giving up $20. Cause it seems absurd that I’m making a profit off of this. And the second crucial thing that happens when the highest bid is at 19, because at that point, both players know this is not going to end well, but they don’t want to be the one for whom it ends badly. So the $18 bidder’s — “Wow, I’m going to lose 18 anyway, let me bid 20.”
LEVITT: I’ve seen this happen, it’s always the same. Until that point, they don’t feel that bad because they feel like this is going to end at 20, just because I think people so often, they bring rules of thumb with them to problems without thinking. So they think, “Okay, fine. I’ll bid 20. I’ll break even, the other guy will bid 19. He’ll lose 19. We’ll be done.” But then when you get to that point, something awful sets in. Cause what happens then?
MULLAINATHAN: The $19 bidder’s like, “I’m losing 19 and I’m not getting anything. If I just bid $21 for this $20 bill, I can at least recoup some of my losses.” Once you cross that boundary, there is no stopping this freight train. It’s especially awesome doing classes when you have these sort of more bro-ish — because they just won’t let go. You have to just pull the plug. You have to say, “I am calling it here at $42, highest bid.” It’s absurd. Obviously the two people involved feel the lesson, but so does everybody else having been part of it. And I’d say, “Now let’s talk about the sunk-cost fallacy. The idea of throwing good money after bad.” Now, when you talk about the sunk-cost fallacy, it has real personal resonance. Like people will remember this experience and that lets them understand sunk-cost fallacy more than if I just talked it through — don’t throw good money after bad, etc.
LEVITT: I never thought about that in terms of sunk costs. What’s always been to me so interesting is it’s a game that once you start it, doesn’t have an obvious equilibrium because if I’ve bid 40 and you’ve bid 41, each time I’m faced with roughly the same problem, which is for $2 more —.
MULLAINATHAN: Yup.
LEVITT: I get a chance at winning 20. And so if the other guy will quit with a 10-percent probability, it makes sense to do it.
MULLAINATHAN: Yeah, exactly. I should walk through how I do it. I say here’s a sunk-cost fallacy throwing good money after bad. Here’s something called “escalation of commitment” where you consistently keep ignoring — you’re like we’ve already done it. It’s only a little bit more. And I think a lot of sunk-cost fallacies play out like this $20 bill auction. They’re like these little escalation of commitments. If that makes sense. It’s like you’re in a bad relationship. Let’s give it another week. Okay. But a week from now, where are you going to be that’s that different than today? This project doesn’t look that good. Let’s give it another $10,000 investment and let’s see how much — so there’s this element of the thing where you’ve spent some resources, put yourself, actually, ironically, not that differently from where you started and you don’t step back and say, “Where does this whole thing end?” Locally, every single extra dollar seems to make sense. But if you step back you’re like, “Where does this thing end up?”
LEVITT: Yeah. The fact that every time you’ve spent the extra $2, the other guy’s topped you — over time, that should change your probability about what’s going to happen next. Related to that decision-making I woke up in the morning for maybe five straight years. And the first thing I thought about every morning is should I quit my job as a professor? And every morning I thought, “God, I would so like to quit, but I can always quit tomorrow. And maybe something will happen today that will make me change my mind.” And so I literally delayed my decision because I had this third option, which is well, as long as I can wait till tomorrow, the cost of waiting is really small. Like a good example in this game is if you weren’t given the chance to go up by $2, which is essentially like pushing the decision until tomorrow, if you just had to make a decision, look, I’m either going to play this game until I’m dead or I’m going to stop now, everybody would stop now. It’s the fact that you introduced this third option, which is like a wasting-away option that completely defines my life. I would say my entire life, that has been my rule of decision-making.
MULLAINATHAN: Your point about deferring to tomorrow is so profound because I think there’s a deep, psychological bias that you tend to think tomorrow will somehow be different. It shows up in so many aspects of life where if you just say to yourself, “If tomorrow is a repeat of yesterday, how would I behave differently?” And so many things change. There’s an employee. They’re not that good. They haven’t done very well. And you’re like, “Well, let’s give them a chance.” The first time giving them a chance may make sense. But after four times you’re like, “If tomorrow is going to be a repeat of yesterday, what am I doing?” And that simple heuristic, tomorrow is a repeat of yesterday, really cuts through a lot of clutter, like so much decision clutter.
LEVITT: Do you use that in your own life?
MULLAINATHAN: I try to use that in my own life. The place where I’ve gotten better and better at it is in trying to decide what I should do — what should I work on? And it’s tempting to work on things that give you some pleasure in the future, but they’re painful right now. And you ended up with so many of those things you say, “Wait a minute, will tomorrow be like today and yesterday, which is just a lot of grinding out in the hope of some big payoff in the future? That doesn’t make any sense.” It’s like, I’m lifting weights for some weightlifting competition that doesn’t seem to ever appear. I loved your coin experiment.
LEVITT: The coin experiment that Sendhil refers to is a research project I did to learn about whether or not people make good choices. I built a web page for people who were struggling with the hard decisions. So, should they quit their job or not? Maybe should they end a relationship or stick with it? And I invited those people to be part of an experiment and if they agreed, I did something really simple — I flipped a coin. And if the coin toss came up heads, I encouraged them to make a change, to end that relationship, to quit the job. And if it came up tails, I encouraged them to stay the course. And the amazing thing is almost 20,000 people flipped coins and the outcome of the coin toss actually affected the way people behaved. And six months later, I surveyed those people to figure out how happy they were with their choices.
MULLAINATHAN: I’ve taken away the fact that when you’re near indifference, which is a lot of the people in your coin experiment, they’re like, “Hey, I’m indifferent. I’ll let you decide.” And when you’re near indifferent, you’re not actually near indifferent, so —.
LEVITT: Exactly.
MULLAINATHAN: Right? You should be able to figure out which way the bias goes. You’re like, “I’m indifferent.” Guess what? You have status quo bias. So if you’re near indifferent, it’s easy. Change. And there’s so many things like that. You’re like if you’re near indifferent, what’s your bias? You tend to pick the better known option. Great. You’re near indifferent, pick the less known option. I found that just a very powerful situation, a powerful tool.
LEVITT: Yeah, for me, it’s just a great heuristic because every problem ends up turning into the same problem. As long as I have strong preferences, I follow my preferences when I don’t know what to do, I know that I’m just messed up because history tells me that I’m always stuck with the status quo. And so I should always make the change. Now, even knowing that, I don’t make enough changes. It’s interesting that I’m the author of a study that says when you’re indifferent, you should make a change. And my whole body convulses at the thought of change and I don’t do it, like incredible. I completely and utterly know that I’m doing something crazy and I do it anyway. That’s how powerful my status quo bias is.
MULLAINATHAN: Oh my God. Status quo bias is so powerful. I don’t know if you’re into food, but one thing I’ve noticed with food is if you say to somebody, “Try this,” and they’re like, “Oh, it has mushrooms. I don’t like mushrooms.” “Go ahead. Just give it a taste.” “No, I couldn’t. I don’t like …” “What are you allergic? Or poisonous to mushrooms?” “No, no, no.” And I am completely like this, so much so that my ex-wife — I didn’t like soy sauce. I was like, “I don’t like soy sauce. I don’t like that at all.” So, she would secretly cook me things with soy sauce that I actually liked. It’s this thing of like, why am I so wedded to this status quo that I don’t even experiment a tiny bit? What does one mouthful cost me? Nothing.
LEVITT: Right, right.
MULLAINATHAN: And it has gotta be a given that I don’t dislike everything with mushrooms. That is absurd, right? I’m sure I disliked some mushrooms, but not all. And so in food, I really see it. Because the cost of experimentation is zero. And yet we do so little of it. I often say to people or they’ll say, “Let’s go to this restaurant. I have a favorite dish here.” “Okay. great. Have you tried any of the other dishes?” “No.” “How many times have you been here?” “Like 30 times. Each time though, I have my favorite. I don’t want to give up on my favorite.” It’s like you tried one thing, try a few others.
LEVITT: I have a rule of thumb at restaurants, which is I try one thing. If I like it, I’ll go back there over and over. And if I don’t like it, I will never go back. I almost never sample two things on a menu.
MULLAINATHAN: Go back a second time — do you try something else or you just…
LEVITT: Oh, never. No, I wouldn’t even think about it. Almost every restaurant I frequent, I hit the jackpot the first time and I can’t imagine how anything could be better than the thing that I got the first time. Ever. But I’m happy about that. Actually, that one doesn’t bother me. I do a lot of things that bother me, but that one seems to me, that’s sensible. Once you hit something good, why would you ever mess with success?
MULLAINATHAN: Next time you go to your favorite restaurant, order the thing you like, and since I know you have the cash, order something else on top. It’s like costless experimentation. Just see what it tastes like. Who knows if they did one thing — maybe they do a second thing right.
You’re listening to People I (Mostly) Admire with Steve Levitt, and his conversation with economist Sendhil Mullainathan. After this break, they’ll return to talk about Sendhil’s new app that helps people learn new ideas.
* * *
LEVITT: So, if there’s one thing we value at People I (Mostly) Admire, it is the power of ideas and experimentation for making things better. And honestly, most of the good ideas we have on this show come from my producer, Morgan. And so, I want to introduce her to everyone. Morgan, can you please say hello?
LEVEY: Hi, Steve. How’s it going?
LEVITT: It is going great. Except that we put this new segment into the show, which has me answering listeners’ questions and you don’t think I’m very good at it. And so my view is, look, if you don’t think I’m doing a good job, why don’t you come out here with me and help me do a better job? And so we’re going to give that a shot today.
LEVEY: I didn’t say you were doing a bad job, but I do think that you answer questions that are easy for you to answer.
LEVITT: Oh my God. Absolutely.
LEVEY: I also just don’t want to hear the answer to any more golf questions. So I thought I’d come on here and start helping you choose them.
LEVITT: Okay, fine. I hand over the reins. What are you going to make me answer today?
LEVEY: Okay. This is one that we get from a lot of listeners. And this question specifically came from someone named Erik and he wanted to know what books do you recommend for someone interested in economics or books that you’d recommend in general?
LEVITT: Morgan, that’s exactly the kind of question I would never choose to answer on my own because I’m just going to say all the books that we’ve already covered on the show.
LEVEY: No, I’d like to know what else you are reading besides the books of our guests.
LEVITT: So really, I don’t read anything else besides the books of our guests, cause it takes forever to read all those books. Okay. So I’m going to talk about books I’ve read in the past.
LEVEY: Okay.
LEVITT: So, the first one is a book called Fist, Stick, Knife, Gun, and it was written by Geoffrey Canada, and it’s an old book. I must have read it 25 years ago, and it is a very personal account of violence in his life and in the communities around him. And it’s just incredibly thoughtful and it’s changed the way I think about crime, which is something that I’ve thought a lot about. Now, it’s not an economics book. It’s a sociology book. When it comes to economics, I’m going to go back even further to two books that I read in college. The first one is a book called The Evolution of Cooperation by Robert Axelrod. And it is this amazing book on game theory — it’s actually a book we’ve talked about on an earlier episode because Yul Kwon, one of my guests, had used that book —.
LEVEY: Oh, right.
LEVITT: As the cornerstone of his strategy in Survivor. It’s about how cooperation emerges in the prisoner’s dilemma. It inspired me more than anything I had ever read in economics, except for the last book I’m going to mention, which is a book from 1960 and it is by a professor I had at Harvard, Thomas Schelling, he won the Nobel prize. This book is called Conflict and Strategy, and it is a book of game theory with no math. It’s just a really smart guy who thought deeply about the implications of strategic behavior. And I couldn’t believe how big and deep the ideas were. It was the thing that pushed me to want to go get a Ph.D. in economics.
LEVEY: Okay. Those are great. But those were all written over 25 years ago. Do you think you’ve read a book in the past two decades that you could recommend?
LEVITT: Oh, if I had to pick one, I guess I would say Nudge by Richard Thaler and Cass Sunstein, and the basic idea is so simple and so obvious that I kick myself and I say, “How did I never think of that before?” And that to me is the best advertisement you can make for a book.
LEVEY: Okay. I think I let you off the hook a little bit with this question, especially since I know you’re good friends with Richard Thaler so, of course, you’re going to recommend Nudge, but hopefully our listeners will enjoy hearing those recommendations. And I think now we should get back to the interview.
LEVITT: All right, let’s do that. As always, we love to get listener emails. The email address is pima@freakonomics.com. Morgan and I both read every email.
* * *
LEVITT: This has been such a typical Sendhil conversation. In 15 minutes he managed to go from Feynman diagrams to Marcel Proust, to the sunk-cost fallacy to not liking soy sauce. In the second half, I want to hear more about this new app. Since when do econ professors develop apps? And I also want to hear about his experiences in the Obama administration. I just cannot imagine the chaos that must have ensued when Sendhil was put in charge of a government bureaucracy.
LEVITT: So earlier in our conversation, you referenced an app that you developed, it’s a learning app. What does it do?
MULLAINATHAN: It’s called Pique — P-I-Q-U-E, like “pique your interest,” and at its core it’s trying to say, “Look, we have a fundamental problem in learning new ideas. We read about them, but reading is a bad way of learning.” So, what it tries to do is it takes books and the ideas in books and says, “We’re going to turn those into experiences.” So like the dollar auction, you’re actually going to go and experience something in the app and then we’ll say, “Look, you’ve learned about escalation of commitment.”
LEVITT: You say books? What do you mean by books? Like real books?
MULLAINATHAN: Yeah. Like real books. The first set we’re doing right now is behavioral economics, behavioral science books. I’ll give you an example. There’s this great book by Leidy Klotz. It’s called Subtract. It just came out. It’s awesome.
LEVITT: Oh, yeah, yeah. I’ve heard of that. I haven’t read it.
MULLAINATHAN: It’s really nice. And it’s one of the books in our app and what we do is he has an experiment where there’s a grid. Okay. So, people come into the app, they see a grid, and the grid has different colors. And so your job is when you click on a tile, it changes the color and flips it. It’s a little game. And what you want to do is you want to try to make the grid symmetric color-wise. Okay. And you go, you play with it. And you find that in about 12 flips, you can make it symmetric. And there’s actually a much easier way to do it than 12. I think it’s four or eight steps — much fewer. But when you do the 12 it’s because you’re looking to add colors to create symmetry. The way you do four is you remove colors to create symmetry. And Leidy’s point in the book is that in many situations, we look to create good changes by addition — we rarely look to get to where we want through subtraction. Reading Leidy’s book is a delight, but playing games like this, where you kind of struggled with the game, you’re like I can’t get it. I can’t get to the best — and then you have the aha moment. You’ve learned the subtract lesson by doing it. And you remember that far more. And so the app Pique takes these books and creates these experiences. Have I ever told you the story of the Stephen Hawking book, A Brief History of Time?
LEVITT: No. Nuh uh.
MULLAINATHAN: So, this is a book where you see it on everybody’s shelf. But nobody’s actually read this book, right? I’m sure Stephen Hawking has read this book. I’m hoping his editor has —
LEVITT: Kind of like Thinking Fast and Slow.
MULLAINATHAN: Yeah.
LEVITT: I didn’t say that — Danny‘s our friend. I would never say that about Danny.
MULLAINATHAN: Right. Exactly. Exactly. Oh yeah. So…
LEVITT: A Brief History of Time.
MULLAINATHAN: A Brief History of Time. I’m convinced nobody has read A Brief History of Time. And I was like maybe I’m being cynical. And then I realized, no, the publishers know this and they’re even more cynical than I am. So, I think it was 20th-year anniversary of the book, they published a new book: An Even Briefer History of Time. Because they realized surely people have bought that 200-page one and feel a little guilty not having read it — let them buy a hundred-page version and feel guilty a second time around. It sold pretty well, I think.
LEVITT: Yeah, I bet. But I wonder if it doesn’t work, if the point is to impress people —.
MULLAINATHAN: Yeah, exactly.
LEVITT: The thickness might actually be really important to them…
MULLAINATHAN: Exactly. It’s one thing to say, “Have you read A Brief History of Time? It’s another thing to say, “Have you read An Even Briefer History of Time?” Yeah, so what the app does is, it’s our ambitious attempt to change the way people absorb and learn information. And so, the activities are just fun and interesting and they’re illustrations of stuff so that you can go through them and then you walk away having learned it.
LEVITT: Is your thought that people will read the books first and get the lessons reinforced through the app or that the app potentially becomes a very time-efficient way to avoid having to read 300-page books that only make a couple points?
MULLAINATHAN: Yeah. I imagine that for a lot of people they will do the app and then for some it’ll motivate them to read more detail in the book, because it’s also more interesting to read a book when you’ve got a lesson from it motivating you and pushing you through. And I do think there’s a time efficiency gain of doing the experience. It’s like teaching. You do the $20 bill exercise. It’s just so much more efficient than sitting there hearing about escalation of commitment or whatever it might be.
LEVITT: So, take a book that many people would know…
MULLAINATHAN: We have one that I like, which we can try on your listeners and I can try on you as well, Steve. It’s a memory experiment. Okay? I’m going to list a bunch of words. I don’t want you writing anything down. And then I want you to, after I list the words, write down whatever you remember. I’m going to list about 10 words. So it’s pure memory. I’ll say them out loud. Don’t do anything. And then after I say, “Go,” write down as many as you can remember. Okay, here we go. Bed, rest, awake, nap, dream, wake, doze, snore, slumber, blanket, snooze, tired. Okay. Go ahead. Just write down as many of these words as you can remember.
LEVITT: Did you write them down? So you know what you actually said?
MULLAINATHAN: I did. Yes. Okay. Do you have it?
LEVITT: All right. So I got seven written down.
MULLAINATHAN: Okay. So let’s see. Do you have the word “bed”?
LEVITT: Yes.
MULLAINATHAN: “Tired”?
LEVITT: Nope. Oh yeah, I do. Yep.
MULLAINATHAN: “Dream”?
LEVITT: Nope.
MULLAINATHAN: “Slumber”?
LEVITT: Yep.
MULLAINATHAN: “Sleep”?
LEVITT: Nope.
MULLAINATHAN: Oh. So for your listeners, let’s just say you did, Steve. Let’s just pretend you had “sleep.”
LEVITT: Okay.
MULLAINATHAN: Okay. So, this experiment, over 50 percent of people remember the word “dream” and over 50 percent of people remember the word “sleep,” but there’s a big difference between those two words. “Dream” was on my list, “sleep” was never on my list.
LEVITT: Ah.
MULLAINATHAN: You can reliably induce a false memory of the word “sleep.” And it happens very consistently. This is one of the things that we have in the app, and I’m sure in the audience, some people listening to this podcast will remember having heard the word “sleep” and they can go back and listen, I never said the word. And you can see what’s happening here — sleep is in this sort of network of these words. It’s like that party where your friend says to you, “Hey, you remember that party you were at?” And you’re like, “I wasn’t at that party.” They’re like, “No, you were there.” And it’s because every one of your other friends was there, they’re like, “Well, you were there too.” And “sleep” is like that. And so this is a false memory inducement, and you can give all sorts of lectures about people’s memory is fallible, don’t trust your memory. But you do an experiment like that and suddenly people are like, “I really remember the word ‘sleep.’” And in fact, you did not remember the word “sleep” and it teaches you a lesson around the fallibility of memory.
LEVITT: You live and breathe ideas like few people I’ve ever known and academics was made for you and vice-versa. But the app won’t be the first time you’ve stepped foot outside of academia. You spent some time as the research lead for the Consumer Finance Protection Bureau during the Obama administration. I mean, the federal government is notorious for red tape, wasn’t that a nightmare for you?
MULLAINATHAN: Honestly, at first it was because the professional world is so complicated, you know what I mean? The toughest part is that the ratio of innovation to execution is so different than what we’re used to.
LEVITT: Mm hmm. It’s true.
MULLAINATHAN: And so I’ve gained a lot of respect for people who get sh*t done, because it is insanely hard. And we somehow put a lot of weight on ideas, but execution is a whole other thing. So, in that sense you’re exactly right. There was a nightmare element to it. The part of it that was really interesting though, was — huh, how would you actually meaningfully protect consumers against a lot of these sh*tty products while also preserving innovation? These problems are complicated when you start really getting down into them.
LEVITT: So, I’m just imagining you, with an idea an hour, each idea requires three years of execution — what happened next?
MULLAINATHAN: That was not my finest moment. I learned the hard way there that if I want to get things done, I have to figure out how to become a manager, which I was not, or figure out how to get a structure in which I can play a role. It’s one thing to not be good at something and to fail at something. But it’s another thing to not even know why you’re not good at something. Like the thing about school, if someone gives you an exam, you do badly, they tell you what you got wrong and they tell you the book you need to read to get better. Unfortunately, life does not have any of that. You don’t even get a score, you just kinda know you’re doing badly. If there’s one thing that would be interesting to figure out is, how do you improve learning through the rest of life? School is such an artificial way to think about learning. It’s just too much very directed feedback. And then you’re turned out into the world where you get zero directed feedback. And somehow we’re not preparing people. I know I was doing badly, but I didn’t know what I was supposed to do to fix it.
LEVITT: I think you’re right. I’ll give you an example where I stumbled in the same way you did. So after Freakonomics came out, all sorts of companies would call me up and say, could I do consulting? And I’d say, “Sure.” And they’d say how much you charge? I’d say, “Well, I’ll do it for free. If you give me the data, I can write my academic papers.” And so I would go to these companies and I had been very successful in academics. So, I had a set of rules of thumb about what made for a good question, how you analyzed it. And it turned out that all those rules of thumb, which made a ton of sense in academics, which would take a very small problem because the reward in academics comes from answering a relatively boring question, a hundred percent right, over the time span of two or three years, you take your time, you finish — but businesses needed me to answer interesting questions quickly with a 80 percent likelihood that I’m right. And I was not at all suited. And I gave the worst advice. And I always had made fun of other people because what I’ve seen too often is people just use whatever rules of thumb have worked on the last problem. And they apply that to the new one, without thinking. Hey, that’s exactly me. That’s exactly what I’m doing. I think it also takes — to learn you have to really be willing to admit your own flaws. And I think so much in society pushes us against that. So you think that school should be more like life or the point of school should be to teach people to be learners?
MULLAINATHAN: That’s the idea I’m throwing around in my head these days — you said it perfectly, Steve. Could school teach us to be learners, but learners in environments — and in psychology, there’s this notion of a wicked environment. A wicked learning environment is one where there’s very little feedback. And how would you arm people to be learners in a wicked environment? I don’t have an answer to it, but that’s the kind of question that I’m throwing around in my head because that seems like it would have very high return — if we could even make a little bit of progress on that, given that everyone is about to be thrown out into this world. But the second thing we can do is we can ask the question, why are learning environments so wicked? Why is feedback so bad? Jens Ludwig, one of our friends, he’s always pointing out to me that if you look at how much feedback judges get, it’s basically zero. So you’re a judge, and you do pretrial, which means you’re just trying to decide whether the person should stay at home and wait for trial or whether they should wait for trial in jail. It’s a simple decision. In Cook County, you might make it a hundred times a day. Now we have the data, we know what happens to those people you send home, and how many of them didn’t come back. And how much feedback do judges get? Nothing. Like nothing. They don’t get a report that says here’s how your cases did after the people were sent home. And if you look at how many situations the decision-maker doesn’t get much feedback, it’s a ton. Like we teach, right? So what feedback do we get? Some stupid teaching evaluations, which are nothing more than at the end of the quarter, what did the person think? There’s no, “Hey, we went and surveyed your students five years later and here are the concepts they remember from your class.” I would love to know what that is. Maybe they don’t remember any of the concepts from my class — that would be worth knowing. Or which ones stuck. Like you just don’t get that much feedback on any of the things that you do all the time. And it just makes for such a terrible learning environment.
LEVITT: Yeah, I agree a hundred percent. And I also think though, in many settings, the actors do their best to avoid feedback. Because in complex settings, like a business environment, I think it’s better for most of the people in the firm that nobody has any idea whether their decisions paid off or not. When I’ve gone to try to run experiments in firms, and I know you’ve run experiments in firms, it’s hard to get them to run. And I used to think the obstacle is that they’re expensive and they take a lot of human capacity, a lot of resources. And I’ve realized over time that nobody actually wants to know whether the decision they made was a good one or a bad one. I don’t know if it’s because the payoff to good decisions is smaller than the cost of bad decisions. Maybe it’s risk aversion but the bigger obstacle isn’t even that. The bigger obstacle is in order to run an experiment, you have to admit upfront that you don’t know the answer and in business, I’m supposed to know whether this advertising works or not, but actually, even though we spend a hundred million dollars a year on advertising, I have no idea if it works. To admit that and to say, “I’m going to learn from an experiment.” That’s the thing which is really interesting to me that so differentiates academics from business. As academics, we always start from the principle that we don’t know anything. But I think any setting in which people have to pretend like they have expertise, that gets in the way of experimentation and feedback more than anything.
MULLAINATHAN: And this is deeply related to the point you said earlier as well, which is like, at an individual level, it’s just very hard to admit that you’re bad at something, that you’re wrong — not wanting to admit upfront and not wanting to learn afterwards. Even in academia when we’re exploring new ideas we say, “We don’t know,” but if a Ph.D. student comes to me and says, “Hey do these slides make a good talk?” A part of me always wants to say, “I don’t know. It’s as hard for me as it is for you. I don’t know, I’m just — I’m just groping around here,” But you can’t, right? Imagine how disheartening it would be for this student to be like, “Hey, look, we’re all throwing darts. If you want me to throw the darts for you, I can do it too.” They don’t want to hear that. And that’s what’s happening in organizations too. At least we imagine leadership involves putting this mantle of “I know exactly what I’m doing.” To a degree it’s true. No one wants to follow the general — who’s like, “We’re going to go this way. I don’t know if it’s the right way guys, but it’s a good guess. Let’s go.” Society would be so different if we could just own up to the sheer level of ignorance that we as experts have about a huge number of things.
LEVITT: So, you were in Cambridge, back and forth between Harvard and M.I.T. for over two decades. Why did you decide to move to the University of Chicago after all these years?
MULLAINATHAN: The honest answer is actually very related to your flipping a coin study, actually. One of the things that was most influential was when I was trying to decide what to do, somebody said to me, “Sometimes change is good just for changes sake.” That really stuck with me because on my pro-con list, nowhere was there written “change.”
LEVITT: Mm hmm.
MULLAINATHAN: It’s like, they’re material aspects, but when you reflect on it, what’s the biggest aspect of this decision? Change. I’m going to be in a whole new city, whole new environment. And so, once someone gave me that advice, I was like, “Oh, this seems like a relatively straightforward thing. Change by itself is just good.” It’s related to where we started, we ridiculously underweight the value of change. You know what I mean? Like you think you’re the same person wherever you go. That is absolutely not true. You’re exposed to new things. Your mind changes. In many ways. I think people become stagnant as they get older because they’re not doing enough to expose themselves to truly new situations.
LEVITT: Do you have advice for young people who are trying to figure out their place in the world?
MULLAINATHAN: I think the one piece of advice I would give comes from this awesome paper. I think it’s Dan Gilbert. It’s called the end of history illusion. If you ask people, how much have they changed in the last five years? Most people say a lot, especially young people like, take a 22-year-old: “Oh my God. Who was I when I was 17? My God, 17 to 22 — I changed so much.” Then you ask people, how much will you change in the next five years? They’re like, “A little bit.” Pick any age, you always act as if history has ended. All the change you’re going to do is done, which is absurd because from 17 to 22, you changed a lot, 22 to 27 you changed a lot. So, I think the biggest error people make is they think they are choosing for who they are right now. What they’re actually choosing is for this person five years from now, who’s going to be very different from them. Very different from them. So, if you say “I’m deciding whether to go work at company A or company B,” if you think you’re set, what you’re choosing is between two companies. If you think that you’re changing, what do you change towards? You change towards the people around you. So, what you should ask yourself is, “I am going to become like the people at company A or like the people at company B. That’s who I’m actually going to become. Which of these kinds of people do I want to be as a person?” And so, that puts a whole different perspective on it because now you no longer think of yourself, you’re actually choosing versions of you and you really have to accept that. Whenever I try to tell students this, it’s amazing how much they resist it. People say this to me, “I’m going into consulting, but I’m not going to be the stereotypical consultant.” I’m like, “My best guess is you are going to be the stereotypical — what do you want me to tell you?”
LEVITT: Yeah. I’ve never heard anyone say what you just said, it’s so interesting to hear it.
MULLAINATHAN: I want to hear your advice.
LEVITT: And It’s actually evolved a lot. It’s been affected by this podcast and hearing what other people say, but I have come to believe that the single most important thing to recognize when you’re young is that life is long and it’s not a race and there’s this sense of urgency of not getting off the track, of having to do something tomorrow. So before I went back to get a Ph.D., I spent two years doing consulting. And I was panicked that I was behind the other people. Look, it made no sense to be panicked about it. So that’s really, to me, not being in a hurry and the luxury of knowing that you can make mistakes, you can experiment, you can dabble and still have all the time in the world to be what you want to be.
MULLAINATHAN: The most interesting people we know didn’t know at 22 what they wanted to do. And some of the most interesting people don’t know at 40 what they want to do. And that’s good. If you’re embracing life, that’s what’s going to happen. Yet, somehow you feel like you ought to know, there’s this fixation on it. It’s okay. There’s a lot of other stuff coming down the road.
LEVITT: And I think society’s gotten much worse on that. As I look at my teenage kids, the amount of focus they have about what college will they go to, what activities am I doing when I’m 13 that will positively influence my chances of getting into college. And that is a kind of rat race that I think we’ve done such a disservice to our kids.
MULLAINATHAN: I feel like kids get less and less play. I mean, I went to Cornell — very good school. And I remember being stressed about it, but I also didn’t think to myself, I needed to check off a bunch of boxes to try and get into the best school. Like I just felt like I just had to be myself. Like a ton of time to just explore and just play and acquire interesting ideas and things. It made you able to enjoy and really become intellectual in a way that I couldn’t imagine doing if my only goal were to get good grades and check the right boxes. It goes back to your Oculus point. I really liked it when you said these guys were playing with it. Like play is ridiculously underrated. Every time I’ve licensed myself to mess around, great things have come, because that’s how you get into really good ideas — is you mess around.
LEVITT: What a great place to stop, talking about play. Because I really do believe that Sendhil’s willingness to play is a huge part of what makes him special. And I have to admit, I used to love play. But wow, do I find it hard to play these days with all my other obligations. But I’m making a promise to myself — at least for one week, inspired by Sendhil, I’m going to make play a priority.
So this would have been a natural place to stop our conversation, but with Sendhil, we always end up talking for hours and canceling whatever other meetings we had planned. And that’s exactly what happened here. We just kept talking. So, next week’s episode will be the second half of our conversation where we discuss the implications that artificial intelligence will have on society, his secrets for generating so many ideas, and the radical ideas in the book he titled Scarcity.
LEVITT: I saw the title and it’s called Scarcity. And I thought to myself, this is so arrogant because economics is the study of scarcity. So, who does Sendhil think he is that he’s going to have something new to say?
* * *
People I (Mostly) Admire is part of the Freakonomics Radio Network, which also includes Freakonomics Radio, No Stupid Questions, and This Won’t Hurt a Bit. This show is produced by Freakonomics Radio and Stitcher. Morgan Levey is our producer and Jasmin Klinger is our engineer. Our staff also includes Alison Craiglow, Greg Rippin, Joel Meyer, Tricia Bobeda, Emma Tyrrell, Lyric Bowditch, and Jacob Clemente. All of the music you heard on this show was composed by Luis Guerra. To listen ad-free, subscribe to Stitcher Premium. We can be reached at pima@freakonomics.com. That’s P-I-M-A at Freakonomics-dot-com. Thanks for listening.
LEVITT: If somebody provides me detail about anything, and I have no recollection. I believe it’s true.
MULLAINATHAN: Steve, do you remember that $10,000 bet that we had? I was trying to figure out when — when you were going to pay me.
LEVITT: Nah, it’s not good, ’cause there’s not enough detail.
Comments