Our latest Freakonomics Radio podcast is called “Would You Let a Coin Toss Decide Your Future?” (You can download/subscribe at iTunes, get the RSS feed, or listen via the media player above. You can also read the transcript below; it includes credits for the music you’ll hear in the episode.)
In it, Stephen Dubner grills Steve Levitt about a new project called Freakonomics Experiments. (Levitt blogged about it here and stopped by Marketplace last week.) The basic idea is to learn more about how people make decisions, especially when they’re on the margin. So: if you’re struggling with a decision, large or small, you can bring your question to FreakonomicsExperiments.com, where a random coin flip will help solve your dilemma. You also become part of the scientific experiment by taking follow-up surveys and letting the research team know how the decision turned out.
The idea originated from the flood of emails Dubner and Levitt received in response to our “Upside of Quitting” podcast. Many listeners said the show had emboldened them to quit something they no longer wanted to do. In this podcast, you’ll hear from two of those quitters: former racecar driver Daniel Herrington and ex-runner Serra Mentessi.
But other listeners wrote in with ongoing issues:
LEVITT: So many people said, “Hey should I quit this?” or “Should I do that?” And how would we know? I mean, we have no idea, we don’t know these people, we don’t know anything about them. There’s no way we could give an intelligent answer to that question, but it got me thinking. And really what’s the best we can do is we can give people who are having trouble deciding a framework for thinking about those decisions. And in certain circumstances, maybe we could do some experiments which would both help us and our research and also help the people.
Thus was born Freakonomics Experiments. Here’s what Levitt hopes to learn from the data:
LEVITT: What I hope to learn is whether or not there are any maybe big, systematic rules that we could tell people about decision-making. So for instance, it may just be the case that people rarely make the big changes, but the people who make the big changes are much happier in their lives after they make those changes. So there’s a status quo bias. So let’s just say over thousands of people making real-world big decisions we find that the changers, the ones who shake up the status quo, do better. Well if that’s true, then that’s a really important message, because what that means is whenever you’re on the margin, you should have a default rule, which is I go for the change. Okay, do I know that will happen? I have no idea. It could be just the opposite. It could be that we should never do any big changes, we’re always better off leaving things the way they are. We might learn that as well. But it’s the fact that I have no idea what the right answer is which gets me so interested in going out and for the first time trying to measure this and learn about it.
You’ll also hear about the study’s design, what kinds of questions you can’t ask the coin, and why this whole idea isn’t quite as ridiculous as it sounds.
[MUSIC: Ed Hartman; “Happy Marimba”]
Stephen J. DUBNER: If you ask Daniel Herrington to introduce himself, tell you what he does, this is what he says:
Daniel HERRINGTON: Uh, Hi, this is Daniel Herrington, and I work for ARPA-E, the Advanced Research Projects Agency for Energy within the Department of Energy.
DUBNER: But if you’d asked him that question two years ago, you would have gotten a different answer.
[MUSIC: Omnivoid; “Stuck In The Race” (from Combustion)]
[ANNOUNCER: Here we come off of turn number 4 for the final time, to the line, it’s going to be Daniel Herrington. He wil win his first race at Firestone Indy Lights series...]
HERRINGTON: I would have said, “Hi, this is Daniel Herrington, and I’m a professional racecar driver.”
DUBNER: Herrington is 26. Two years ago, he was a racecar driver. He was doing alright: a few good finishes in Indy Car races, a few million dollars in sponsorship. But he wasn’t setting the world on fire. He started thinking about other options. It was a hard decision.
[MUSIC: Ben Siems; “When The Horizon Bursts” (from Empire at Twilight)]
HERRINGTON: I felt indecisive the whole time. I didn’t know on a day-to-day basis what I should be doing. And that’s kind of a terrible feeling.
DUBNER: So during this period of Herrington’s indecision, we put out a podcast called “The Upside of Quitting.” Herrington listened to it, he stared his indecision in the face and he quit driving his racecar. For the most part, at least. He went back to school, at Duke, got a master’s in engineering management. And that lead to a very good job at the Department of Energy in Raleigh, North Carolina. He advises teams who receive government money to develop energy technologies, and he likes his work. But a familiar feeling has already come back to him.
HERRINGTON: I know, it’s the same thing. Actually this is kind of funny, this is revealing about me. I’m trying to decide whether I should leave what I’m doing now, and go do something else where I’m more hands-on, or not.
DUBNER: That’s right. Daniel Herrington is thinking about quitting, again. Now to his credit, he acknowledges he’s a pretty indecisive guy. Not just with big things like his career, but even simple decisions like where to eat, what movie to see. For those smaller decisions, though, he’s come up with a solution.
HERRINGTON: I’ve actually started flipping a coin or doing Rock Paper Scissors. Actually, my girlfriend and I are both pretty indecisive. So we start coming up with a list of a few restaurants and Rock Paper Scissors it out until we’ve got a winner. It’s actually, it’s beautiful, it works so well. I don’t know why I wasn’t doing this for years.
DUBNER: When the stakes are low, like picking a restaurant or a movie, flipping a coin is easy. But you’d never do that for a really important decision, like a big move or a spouse or a new profession. Would you? Or would you?
[ANNOUNCER: From WNYC and APM: American Public Media, this is Freakonomics Radio, the podcast that explores the hidden side of everything. Here’s your host, Stephen Dubner.]
[MUSIC: The Diplomats of Solid Sound; “El Corazon Negro” (from Instrumental Action Soul)]
DUBNER: Hey so Levitt, you wanted to try something a little bit different for the podcast today, yeah?
Steve LEVITT: Yeah, I think I’m more excited for this podcast than for any of the hundred plus that we’ve ever done.
DUBNER: Which isn’t saying much. You’re never been very excited have you?
LEVITT: I know, but this time I’m really excited.
DUBNER: Steve Levitt is my Freakonomics friend and co-author. He is the first to admit that most of our stories are fact-based entertainment; they’re not really meant to help people.
LEVITT: I think in general we don’t think we know the answers, that in order to be a helper, you really have to think you’ve got the magic serum that will get people to the right answer. And that’s not really our style, It’s not very often that what we do translates directly into real, positive changes in people’s lives.
DUBNER: But then there was that one episode, “The Upside of Quitting.” We raised the possibility that the old conventional wisdom – you know, “a winner never quits, and a quitter never wins” – that it might be, for some people and in some circumstances, just wrong. We talked about the sunk-cost fallacy and opportunity cost. And we suggested that, overall, quitting has a bad rap, but that strategic quitting can actually be a great thing. We heard from a man who quit his dream job running the U.S. Department of Labor:
Robert REICH: I made the decision shortly after the election that I would leave. And then one day I went into the Oval Office and explained to the president that I just felt that I had no choice. He was very understanding about it.
DUBNER: We heard from a woman who quit a good IT job to become an escort:
ALLIE: You know, of course it’s always scary to leave behind something that’s legit. And go with something that maybe isn’t considered that. You know, I know that it was the right decision for me. For me, I don’t have a problem with having sex with strangers.
DUBNER: And from a woman who quit her religion:
Saloma FURLONG: The upside of it is, there have been so many times, so many moments in my life, when I knew that quitting the Amish was the right thing to do.
DUBNER: Even Steve Levitt is a big fan of quitting:
LEVITT: A lot of people -- you make choices without a lot of information and then you get new information. And quitting is often the right thing to do. I try to talk my kids into quitting soccer, baseball if they’re not good at it. I mean, I’ve never had any shame in quitting. I’ve pretty much quit everything that I’m bad at.
DUBNER: We got a lot of e-mails after that quitting episode. From people like Daniel Herrington, the racecar driver. From people who were thinking about quitting their job, their marriage, their passions – or what they had thought were their passions.
[MUSIC: Vunt Foom; “Grease” (from Sub Valve Release)]
Serra MENTESSI: Hi, I’m Serra Mentessi, I’m from Sacramento, CA, and I guess I’m an ex-runner.
DUBNER: Mentessi wrote to us about her running habit.
MENTESSI: I’ve always been active. I still am active. But after awhile, I just kind of fell out of love with running, but I really just found it was very hard to quit. Because what else are you going to do? You bought the shoes, you’ve got the clothes, and you belong to all the running clubs, your friends all think of you as a runner, and I think you can so define yourself that way that you just kind of fall into the habit, even though you’re not kind of in it anymore.
One day, she went out for a run. With her iPod.
MENTESSI: That run...I still think about that run. I was headed out, I’d made it out to the end of the block. It was a beautiful day. It was July here in California. And I just wasn’t in it. I’d had a fall a couple of weeks before, my ankle was still a little swollen. And I just did not want to do this run, but it was on my schedule to do it, and so I headed out. And “The Upside of Quitting” came on, and I didn’t even put it together at the time. And I probably was about 15 to 20 minutes into the podcast, when I realized this podcast was sent to me on this day, and this was going to be my last run. They’d given me permission to quit, which I think I needed. I needed someone to tell me it was okay. And I got home and took a shower, and I quit.
DUBNER: So Levitt, that “Upside of Quitting” episode, that turns out to be probably your favorite that we’ve ever done, wouldn’t you say?
LEVITT: Yeah, I have to admit I thought it was a terrible idea when you brought it up. It was totally your idea, I thought it would be a total bomb. I humored you. I said that’s fine, you do all the work, that’s great. But it was really…I was stunned. I’m willing to listen to evidence and have my mind changed. And the flood of emails we got of people who heard that podcast and were prompted to make a choice, to actually quit something that they had somehow known inside them they should quit for a long time but hadn’t been able to do it until that podcast freed them up, it really got me thinking, when it comes to these big decisions that people face in their lives, should they quit their job, or should they, what college should they go to, or, you know, should they go on a second date with a certain person, or end a relationship, that would we, could we possibly have something to offer that could really be useful to the people who listen to this show? And I think the answer is yes.
DUBNER: Okay, so as you may have sensed by now, this conversation is leading somewhere. To a real thing. Or at least a real experimental thing. That will take real people, with hard decisions to make, and will help them make those decisions.
LEVITT: So many people said, “hey should I quit this?” or “should I do that?” And how would we know? I mean, we have no idea, we don’t know these people, we don’t know anything about them. There’s no way we could give an intelligent answer to that question, but it got me thinking. And really what’s the best we can do is we can give people who are having trouble deciding a framework for thinking about those decisions. And in certain circumstances, maybe we could do some experiments which would both help us and our research and also help the people. And so out of this was born a new website that we have that you can link to either through the Freakonomics web page, or go directly to FreakonomicsExperiments.com. And essentially, what this page will do is if you’ve got a tough question, and you can’t figure out what the answer is, we will walk you through a few steps, and then if you’re still undecided as to what to do, we will do you a huge favor, and that huge favor is we will flip a coin for you. Okay, and if you’re really uncertain, you have the option of flipping best two out of three if you really want to make sure that the gods are giving you the right answer. And all we ask is that in return for us helping you try to come to a decision on whatever important problem you’re facing that you help us out by filling out a few short surveys. And what we want to do is conduct real research in the real world to figure out in the end yes or no, quitting or not quitting, leaving or not leaving, starting or not starting turns out to be good or bad for the people who make those decisions.
DUBNER: So what we’re looking for is people who, we should clarify a little bit, they’re on the brink, or at least they’re entertaining a pretty big decision, right? It’s not like whether to buy, you know, the little iPad or the regular-size iPad right?
LEVITT: Oh that’s okay; we’ll take anything. We take all comers. It’s cheap to flip the coin, so if you have any decision you want. But what we care about are the big decisions, because what’s interesting to me is that what economists are always trying to figure out is at the margin, so when you’re just a little bit to one side of a decision, or a little bit to the other, would it have been better to go one way or the other? And the problem is in the real world you don’t get to live two lives. I mean, all of us I think would like to say, you know, I wish that I could see what life would be like if I broke up with this girl or if I didn’t break up with this girl. And then I want to see my life play out and I’ll be able to make the right decision. Of course you can’t. There’s a lot of uncertainty; you’re not sure. The kind of problems that people have the hardest time deciding about, typically, are important problems, problems in which, depending on what path you take, your life will be very different, but as much as you think about it, you just can’t figure out whether your life is going to be better or worse. You know, it’s going to be different. I think those are the exact kind of problems where in my own life as I’ve tried to make decisions I’ve really come to the point where I throw up my hands and how do you decide?
DUBNER: It’s a coin flip.
LEVITT: You don’t know what to do. It’s a coin flip.
[MUSIC: Josh Bernasconi; “Cops In The Water”]
DUBNER: If you have even a shred of common sense, you’re probably thinking … What? I’m going to base one of the most important decisions of my life on a coin flip? When we come back, Steve Levitt explains why this isn’t quite as ridiculous as it may sound.
[ANNOUNCER: From WNYC and APM: American Public Media, this is Freakonomics Radio. Here’s your host, Stephen Dubner.]
[MUSIC: The Rosewood Project; “Drift Away” (from Under The Midnight Stars)]
DUBNER: So Steve Levitt has a brazen idea. He wants to turn your life into an experiment. Wait, there’s probably a better way of putting that. How about this: If you’ve got a hard decision in your life, something you just can’t quite commit to, Levitt wants to help you make that decision. By flipping a coin for you. On a website. Seriously. It’s called FreakonomicsExperiments.com.
LEVITT: When you’ve come to a point in life where you have to make a decision and you don’t know what to do, it’s costly. Right? You spend enormous amounts of time fretting about it, but you don’t know the answer. You’ve already proven to yourself that you don’t know which is better and so consequently, the release that comes with having someone else, in particular this coin, which I want to emphasize is truly a random coin. Some people might think oh this is some kind of trick or they’re up to some kind of hijinks, absolutely not, we have a truly randomized coin based on, you know, this very complicated Swiss algorithm that’s been developed to do randomization. And the release that comes when you are able to simply say, okay, the die is cast this is what I have to do, I think that will be a real benefit for people who are tied up in knots.
DUBNER: OK, so let’s say that I’m thinking about leaving a relationship, or leaving a job, or quitting college, or moving out of the country?
LEVITT: Stopping smoking.
DUBNER: Stopping smoking.
LEVITT: Going on a diet.
DUBNER: Okay. And I hear you talking about this and I say I like the idea, I like the idea that there’s a different way to think about this, that I can think about these things that economists talk about, the sunk cost fallacy, and opportunity cost, and yeah, I should reassess that, but then I hear you talk about that you flip the coin for me to make that decision. I think that sounds a little bit ridiculous, doesn’t it Levitt? Can you see, can you understand how people might think that sounds ridiculous?
LEVITT: Okay, we’re not going to flip the coin right away. First, before we flip the coin we’re going to make you think a little bit about why you might want to stay at your job or leave your job. And we’re going to ask you some questions. And…
DUBNER: For instance?
LEVITT: We’re going to ask you about what kind of job it is, and how long you’ve been there, and how you feel on various levels about the job, and what emotions are elicited when you think about going back to grad school. And I think for a lot of people, by the time they get done with this very short survey, maybe it’ll take you five or ten minutes, I think you may have figured out for yourself what you want to do. And in that case, forget it. You don’t need to flip any coins. Just go and do what you want to do. But for those of you who go through that process and still are completely perplexed as to what to do, we will flip a coin. And one of the great virtues, I think, of having us flip the coin is that the biggest fear people have about making the decision is regret. Oh, I wish I hadn’t done that. And I actually think there’s pretty good psychological evidence which suggests that if you let us flip the coin for you, you will feel less regret whatever happens. You’ll say, well, boy this didn’t turn out great, but look, I’m part of a research project, I’m helping other people by doing this, and look, I don’t know what would have happened if I had done the other thing. And I sure would have felt a lot worse if I hadn’t let someone else flip the coin for me. And we ask you when we flip the coin to make a pledge that you will say, when quitting your job, that if the coin comes up heads that in your best intentions you’ll actually try to leave that job within a couple months. And then if it doesn’t come up heads you’ll try to commit to us that you will stay at that job for a couple months.
DUBNER: Okay, and then you stay in touch with me in either case, yes, whether I stay or leave?
LEVITT: Yeah, absolutely. So we’ll flip the coin for you, and you’re going to go away and you're going to live your life. And then a month or two later we’re just going to send you an email that says hey would you mind coming back and in return for some super cool Freakonomics swag, would you take a five minute survey and just tell us a little bit about your life now and what you’re doing, whether you did quit your job, if you did how you feel, and then you know, maybe six months later, we might send you another five minute survey, and then maybe a year later another survey. And if people keep on answering our surveys, heck we’ll send you another one ten years from now, would be great if we’re still alive and people are still answering our emails. That would be wonderful.
DUBNER: What do you do if someone writes in and says that they’re thinking about killing someone else or maybe killing themselves?
LEVITT: Oh, that’s a hard one. I think, I think I would have to talk to our Institutional Review Board about that. I don’t know the rules. I’m not sure. But, you know, we have an FAQ on the website, and I think it would be smart of us to put up an FAQ that says if I am about to kill someone and I ask you to toss the coin, what will you do, and then we’ll actually know what we’ll do after that. But I’m going to figure it out because I hadn’t really…As demented as I am, that hadn’t occurred to me until you just brought it up right now.
DUBNER: What if, let’s go down the moral ladder then a few rungs and say I write in to you and say, I work for this company, and I’m a good worker there, but they are exploitative in a number of ways, and they make a lot more money than they should be making because of my labor and for a variety of reasons they don’t compensate me or other employees well. And therefore I found a kind of quasi-legal was to embezzle. What do you…Can you help that person with a decision?
LEVITT: You know, the way the software works, I think it probably will help with the decision. The real question is are we required by law to report people who say they want to do illegal activity? And I think the answer is definitely no. I think it’s only if you’re going to kill someone that we’re required to tell somebody. So I think anything short of killing is fair game on the site.
DUBNER: I’d be very curious, in the “The Upside of Quitting” podcast that we made, we talked to some, a couple, two women, one kind of middle-aged, one younger, who had left their religion, they happened to be Amish. And that’s a kind of quitting that doesn’t really get discussed that much. And it’s interesting because it’s got so many dimensions to it. It’s not just a religion, but it’s often your family, and community, and so on. Do you expect that you’ll encounter that? Would you like to encounter that? Do you think that’s something you can be helpful with?
LEVITT: I think that’s a great question, and in fact that is one of the very questions that you will find on the webpage is, you know, are you thinking of leaving a religion? And we’ll flip the coin for you there as well.
DUBNER: Now, just to be clear, if you do go to FreakonomicsExperiments.com, your data will be anonymized. And any kind of decision is fair game, anything you’re having a hard time with – short of harming yourself, or somebody else. As the results start to come in, you’ll hear about them in podcasts, maybe in books we write down the line. Levitt’s academic partner in this project is John List, another economist at the University of Chicago.
LEVITT: And John List, for those of you who read SuperFreakonomics, you’ll know, is without question in my mind the greatest experimental economist who does experiments in the real world. He’s really completely transformed the way economists think about doing research and going from being collectors and analyzers of data to actually creators of data. And what we’re doing here with Freakonomicsexperiments.com is something that I don’t know that anyone’s ever done it before, which is to take the experimental approach of economics to really important, real world problems, with real world people living their everyday lives, and then try to measure whether those decisions turn out to be good or bad decisions. So it’s really for me, what gets me excited, and I’m sure you can sense my excitement, is the idea that we’re taking economic research to a place it’s never been done before and really in some ways democratizing it to the extent that anyone can now be part of this process.
DUBNER: So what do you expect to learn?
LEVITT: Well, what I hope to learn is whether or not there are any maybe big, systematic rules that we could tell people about decision making. So for instance, it may just be the case that people rarely make the big changes, but the people who make the big changes are much happier in their lives after they make those changes. So there’s a status quo bias. So let’s just say over thousands of people making real world big decisions we find that the changers, the ones who shake up the status quo, do better. Well if that’s true, then that’s a really important message, because what that means is whenever you’re on the margin, you should have a default rule, which is I go for the change. Okay, do I know that will happen? I have no idea. It could be just the opposite. It could be that we should never do any big changes, we’re always better off leaving things the way they are. We might learn that as well. But it’s the fact that I have no idea what the right answer is which gets me so interested in going out and for the first time trying to measure this and learn about it. The beauty of the coin toss is because we’re isolating it down to the people who truly could go either way and are happy to do either one, they just don’t know what to do, that we’re really going to get something that’s like a causal effect. I mean, as you know, I’ve spent my entire career trying to distinguish between correlation and causality. And through this coin toss, we are going to have the best mechanism that I’ve ever heard of for figuring out whether big decisions are made properly or not.
DUBNER: How many people would you like to get involved in these experiments? How many people would you need to make it accomplish what you want it to accomplish?
LEVITT: That’s a great…It’s hard to know. It partly depends whether people really will live by the coin toss or not. But, my hope is that it’s fun for people on top of everything else so people will want to do it. So I hope thousands of people will want to do it. I hope it kind of goes viral and everybody, it becomes a thing that when you’re so tired of hearing your friend go on and on and on, days and weeks at a time about should I do this, wishy-washy, that you say, dude, just go to Freakonomics Experiments, they’ll take care of it for you.
[MUSIC: Spencer Garn; “Money Situation”]