Episode Transcript
Stephen J. DUBNER: Um, so Levitt, you’re a fairly successful fellow.
Steve LEVITT: Thank you.
DUBNER: I’m just curious — have you succeeded at everything you’ve ever done?
LEVITT: No, I’ve mostly failed at everything I’ve ever done.
DUBNER: Particulars please?
LEVITT: Well, definitely golf. Golf has been a lifelong failure. You know that when I was a kid, all I wanted to be was a professional golfer. And thank God that I was so bad at golf that I couldn’t even imagine trying to be a pro golfer. I think one of the worst lives you can have in some sense in a modern world is thinking you’re good enough to play pro golf, not being good enough to play pro golf, struggling for a decade or twenty years and then you got nothing to show for it. But luckily I was so bad I couldn’t even pretend to follow that path.
DUBNER: Now, you also failed at becoming the kind of economist that you thought you should become when you first started studying economics, right?
LEVITT: Yeah, I started out as a macroeconomist wannabe and very quickly I realized that modern macro is all about solving dynamic optimization models. That was hopeless. Then I wanted to be an economic theorist. And actually, that was more dangerous because I—
DUBNER: You were closer, almost good enough.
LEVITT: I actually was able to publish some papers that were pretty good in pretty good journals. But the thing that saved me on being a microeconomist is actually a funny story. Which is that one of my advisors at M.I.T., I walked into his office, and he said, “So, what have you been working on, Dave?” Now, any normal person would have said “oh actually, my name is Steve.” But I didn’t do it. And he called me Dave for literally weeks and months. It got so bad that at the annual party of the faculty and students, I gathered around all my friends, I said, Look, he’s gonna call me Dave, just don’t worry about it. And finally, it was not that I thought I couldn’t be a theorist, it was that I was so embarrassed and traumatized by the idea that at some point I would have to say to my main advisor that my name was actually Steve not Dave after six months of pretending, that I decided I should forget about theory and go do something different.
* * *
In last week’s episode, we talked about quitting – specifically, the upside of quitting. Why are so many people so reluctant to quit? One reason is that we tend to equate quitting with failure, and there’s a huge stigma attached to failure. But … should there be? In our new book, Think Like a Freak, we make the argument that perhaps we’re not thinking clearly about failure. Maybe failure can be your friend.
LEVITT: I always tell my students — fail quickly. The quicker you fail, the more chances you have to fail at something else before you eventually maybe find the thing that you don’t fail at.
Here’s the thing: when failure is stigmatized – demonized – people will try to avoid it at all costs, even when it represents nothing more than a temporary setback. Steve Levitt, my Freakonomics friend and co-author, he does some consulting on the side. He tells a story about one of his earliest consulting jobs, with a big multinational retailer.
LEVITT: This was a firm like many firms that had a very particular culture. And this was a culture in which if you ever said anything bad about the firm or the prospects of the firm you were a culprit, you were not a team player.
This firm was about to open its first store in China, in Shanghai. It was a pretty big deal. And for all kinds of reasons, it was really important that the store open on-time. So, about two months before the scheduled opening, the CEO summoned the leaders of the seven teams that were involved in the store’s opening. And he asked each of them for a detailed status report. All the reports were positive. Then the CEO asked all seven team leaders to pick one of three signals—a green light, yellow light, or red light—to show how confident they were that the Shanghai store would indeed open on time. All seven went with the green light.
LEVITT: Everyone was so excited, they were completely on track for this opening two months later. Now, there was a very smart set of people at this firm who completely understood how this culture — while helpful in some ways – was potentially detrimental in others. So they had set up an internal prediction market. And they wanted people to be able to express their true opinions, and the market — this sort of prediction market is a great way to get people to tell you the truth because its anonymous. And because there are actual incentives, money incentives tied to it. And it’s fun because people like to win. It was a competition to see who could amass the most money by buying and selling securities tied to the outcome of the firm.
And unlike the stock market, which is not supposed to allow insider knowledge, this is actually the opposite, right? In a firm you are trying to get the best knowledge you can from people who actually are working on the project,
LEVITT: Exactly. The firm had put this internal prediction market into place exactly with the hope that they could learn the truth about what was going on, because they weren’t sure they were learning it otherwise.
This prediction market offered one bet on whether the Chinese store would open on time. Considering that all seven team leaders involved in the opening had given it the green light, you might [expect] everybody at the company to be bullish as well. They weren’t. The company’s internal prediction market showed a 92 percent chance the store wouldn’t open on time. So guess who was right—the anonymous bettors in the prediction market? Or the team leaders who had to stand in front of their bosses.
LEVITT: So even though the seven teams had all said we’re sure we’re on track, everybody basically knew almost for sure that there was almost no chance the store would open on time. Indeed the store did not open on time.
What does that say about the seven team leaders who had to present to the CEO, and their fear of failure? Was it fear of failure? Was it just fear of appearing that they are not doing their job well? How does failure factor in?
LEVITT: It’s funny because in this case failure can be defined in different ways. Ultimately they failed — the store did not open. Could you pin that failure on any individual? Not really. But. It’s tricky because of the incentives. We talked so much about incentives over the years. And what kind of incentives. Obviously there were some financial incentives associated with their jobs. That wasn’t it. I think that what really was most costly is for you to stand up in the room and be the only one willing to say, “I’m really sorry and I know we’ve been planning this launch for five years, but we botched it and we’re not going to not make it.” But there was no place for that.
It’s easy, especially from a distance, to criticize the seven team leaders who told their CEO they were sure the store would open on time. From the sound of it, the CEO had what’s called “go fever.” Ever heard of “go fever”? That’s a phrase associated with NASA, having to do with rocket launches. When a boss catches “go fever,” it can take a lot of courage to focus on potential failures. Just think of all the factors working against you: momentum, institutional politics, ego. And the consequences of “go fever” can be far more tragic than the delayed opening of a retail shop in Shanghai …
NASA ARCHIVAL TAPE: This is Shuttle Launch Control at T minus 2 hours, 28 minutes and counting. Here comes the 51-L flight crew boarding the elevator, ready to depart the ONC building for the launch pad.
On January 28, 1986, NASA was planning to launch the space shuttle Challenger from Kennedy Space Center in Cape Canaveral, Florida. The mission had drawn massive public interest, largely because the crew included a civilian, a schoolteacher from New Hampshire named Christa McAuliffe.
MCAULIFFE ARCHIVAL TAPE: Well, I am just so excited to be here. I don’t think any teacher has ever been more ready to have 2 lessons in my life…
The launch had already been delayed a few times. On the night before the new launch date, NASA held a long teleconference call with engineers from Morton-Thiokol, the contractor that built the Challenger’s solid-rocket motors.
Allan McDONALD: Ok, well, I‘m Allan McDonald.
McDonald was one of the Thiokol engineers, on site in Cape Canaveral.
McDONALD: At the time, I was the director of the space shuttle solid rocket motor project for my company Morton-Thiokol.
It was unusually cold in Florida.
McDONALD: There is a meteorologist in Orlando that’s saying that these strong winds we saw today are being followed by [an] extreme cold front moving toward the Cape and they’re expecting to see temperatures maybe as low as 18 degrees Fahrenheit by tomorrow morning at the opening of our launch window. And I says, good grief. I says, I’m really concerned that our o-ring seals in these big joints will operate properly at those kind of temperatures.
These o-ring seals kept the extremely hot gasses – like 6,000 degrees hot – from escaping the shuttle boosters. You do not want those gasses to escape the shuttle boosters. But, as McDonald later described in his book Truth, Lies, and O-Rings, the boosters had never been tested below 53 degrees. And the forecast for launch morning was calling for temperatures much lower than that. So on that teleconference call the night before the launch, McDonald and his team recommended the launch be postponed again.
McDONALD: And NASA immediately challenged the basis for such a recommendation, which really surprised me. Because prior to that time, I had given every flight readiness review for every flight since I had been…was put on the program. And I was challenged at every one of them of making sure that I could say it was safe enough to flight. And here all of a sudden we recommended not flying and they challenged the basis for that.
McDonald couldn’t believe that NASA wanted to launch despite what he saw as an obvious risk.
McDONALD: I was amazed that they made such statements, because we always erred on the side of safety. In fact, the engineers got put into a position to prove that it would fail and they could not do that; that’s a whole different question.
According to McDonald, his boss, back at Morton-Thiokol headquarters in Utah, got off the NASA conference call for about thirty minutes to talk about the situation with other Thiokol executives. When he got back on the line, McDonald says, the recommendation to postpone the launch had been reversed. McDonald was angry, but he had been overruled. The launch was back on. Now NASA requested that the “responsible Morton-Thiokol official” sign off on the decision to launch.
McDONALD: I knew who that responsible Thiokol official was. That was me, that was my job, that was my responsibility. That’s where I was at. I did the smartest thing I ever did in my entire lifetime – and that was, I refused to sign the launch recommendation. As a result of that my boss had to sign it and send it down to me.
Which he did — by fax machine. As the engineers were waiting for the fax, McDonald spoke up again:
MCDONALD: So then I got more frustrated. I says, well let me tell you something. I certainly hope nothing happens tomorrow, but if it does, I’m not going to be the person to stand before a board of inquiry and explain why I gave you permission to fly these solid-rocket motors in a condition that I knew they were never qualified to do.
Coming up on Freakonomics Radio … you know what happens the next morning, don’t you?
NASA ARCHIVAL TAPE: Flight controllers here looking very carefully at the situation. Obviously a major malfunction.
McDONALD: The only thing I saw flying were the solid rocket boosters – they were still going. Everything else had disappeared in this huge explosion cloud. In teeny pieces.
And … what if you could learn how you might fail without going to the trouble of actually failing?
Gary KLEIN: So lean back in your chair. Get yourself calm and just a little bit dreamy. I don’t want any daydreaming but I just want you to be ready to be thinking about things.
* * *
So, Allan McDonald was the director of the space shuttle solid-rocket motor project for Morton-Thiokol, the company working with NASA on the Challenger shuttle launch. The night before the launch, McDonald recommended the launch be delayed, out of concern that the shuttle’s o-ring seals might fail in the unseasonably cold Florida weather. But McDonald was overruled. The next morning, he walked to launch control.
MCDONALD: Carrying my briefcase in one hand and my headset in the other. And it was like 22 degrees. And I saw all these icicles hanging all over the place – and I said to myself, they obviously aren’t going to launch this thing today. And I was amazed they finally came on and said they were gonna send a team — an ice team out — to see if they could knock the ice down as much as possible to reduce the debris risk. And they did that. They ended up doing that actually twice before they finally went into internal count and launched the Challenger.
NASA ARCHIVAL TAPE: 10, 9, 8, 7,6, we have main engine start…
MCDONALD: And I remember I held my breath…
NASA ARCHIVAL TAPE: 4, 3, 2, 1… and liftoff. Liftoff of the 25th space shuttle mission and it has cleared the tower…. Velocity 2900 feet per second, altitude 9 nautical miles, downrange distance 7 nautical miles…
MCDONALD:…73 seconds later to see this thing blow up in the sky.
MCDONALD: I was in shock, I said: it blew up. My heart sank just like everybody in that control room and I could hear people sobbing in the background.
NASA ARCHIVAL TAPE: Flight controllers here looking very carefully at the situation. Obviously a major malfunction.
MCDONALD: And I kept hearing this voice over the network saying RTLS RTLS, which is return to launch site, and not getting a response back from the orbiter. The only thing I saw flying were the solid rocket boosters – they were still going. Everything else had disappeared in this huge explosion cloud. In teeny pieces.
NASA ARCHIVAL TAPE: We have a report from the flight dynamics officer that the vehicle has exploded. Flight director confirms that, we are looking at checking with the recovery forces to see what can be done at this point.
MCDONALD: No way they could have survived that in my opinion, and they didn’t, obviously.
At that point, McDonald says, the cause of the explosion was unknown.
MCDONALD: We were told to not copy anything off the screen, they were all frozen, they locked the doors, they cut all the telephone lines so nobody could talk to anybody. They held a meeting saying all the data would be considered secret. And it really wasn’t until the next day that I really realized what had happened.
In the days after the accident, McDonald says, a review of the data showed the cause of the explosion to be what he had feared might happen: the o-rings failed to hold their seal in the cold temperature.
McDONALD: That’s exactly what happened.
This cause was later made official by a Presidential Commission, to which Allan McDonald contributed testimony. In the televised portion of the hearings, the physicist Richard Feynman famously demonstrated the cause of the failure by dropping the o-rings into some ice water:
FEYNMAN: I took this stuff I got out of your seal and I put it in ice water, and I discovered that when you put some pressure on it for a while and then undo, it maintains, it doesn’t stretch back. It stays the same dimension. In other words, for a few seconds at least, and more seconds than that, there is no resilience in this particular material when it is at a temperature of 32 degrees. I believe that has some significance for our problem.
Of course Allan McDonald already knew that, or at least that is what he was worried might happen. That’s what makes the Challenger explosion so remarkable—and even more tragic that it was. The people in the know – at least some of them – had foreseen the exact cause of failure. So why, even with that warning, did NASA push on?
McDONALD: What really happened was typical I think in large bureaucratic organizations, and any big organization where you’re frankly trying to be a hero in doing your job. And NASA had two strikes against it from the start, which — one of those is they were too successful. They had gotten by for a quarter of a century now and had never lost a single person going into space, which was considered a very hazardous thing to do. And they had rescued the Apollo 13 halfway to the moon when part of the vehicle blew up. Seemed like it was an impossible task, but they did it. So how could this cold o-ring cause a problem when they had done so much over the past years to be successful? So it gives you a little bit of arrogance you shouldn’t have. And, huge amount of money involved. But they hadn’t stumbled yet and they just pressed on. So you really had to, quote, “prove” that it would fail, and nobody could do that.
You might think it’s a rare case when a group of decision-makers know with such precision just what will go wrong with a project. But is it? What if there were a way to peek around the corner on any project to see if it’s destined to fail — that is, if you could learn how you might fail without going to the trouble of actually failing? I’d like you to meet Gary Klein:
KLEIN: I’m a cognitive psychologist. I got my PhD back in 1969 in experimental psychology.
Klein’s most recent book is Seeing What Others Don’t: The Remarkable Ways We Gain Insights. He studies decision-making — especially how people make decisions in real settings. Like the Challenger launch.
KLEIN: The Challenger incident has been intensely studied and serves as a wonderful object lesson. What we saw here was repression of consequences that you didn’t want to have to deal with because it would’ve been so inconvenient.
So, we make the argument in our book that there’s a general reluctance to quit or to untangle ourselves from projects or relationships or whatnot, and that that’s in part because failure is seen as such a stigma. No one wants to—we don’t want to quit because we don’t want to be seen as having failed. And so let’s talk, first of all, about the stigma of failure. Do you think it deserves it? Or do you think we should rethink the way we consider failure generally?
KLEIN: Well, the standard line is that you fail often to get smarter. And you fail in productive ways, and it’s a way of building mental models. And all of those things are true.
So how can failure be made more fruitful? Gary Klein has one idea – which I think is pretty brilliant. It’s called the pre-mortem. You’ve surely heard of a post-mortem. That’s the process by which you figure out what, exactly, killed, say, a hospital patient. Or maybe a big project. Theoretically, everybody benefits from the knowledge you gather during a post-mortem.
KLEIN: Everybody except the patient, because the patient is dead. The idea of a pre-mortem is to try to run through that process before the patient dies.
But the patient, in this case, we should say, is not a human. But it’s a project or a product or something like that, correct?
KLEIN: Exactly, exactly.
With a pre-mortem, you try to find out what might go wrong before it goes wrong. How does this work? Gary Klein will walk us through it. Imagine you work for a company that is about to launch this big project that you and a lot of others have been working on for months, maybe years. But before you launch, Klein gathers all the important team members in a room …
KLEIN: I need you to be in a relaxed state of mind. So lean back in your chair. Get yourself calm and just a little bit dreamy. I don’t want any daydreaming but I just want you to be ready to be thinking about things. And I’m looking in a crystal ball. And uh, oh, gosh…the image in the crystal ball is a really ugly image. And this is a six-month effort. We are now three months into the effort and it’s clear that this project has failed. There’s no doubt about it. There’s no way that it’s going to succeed. Oh, and I’m looking at another scene a few months later, the project is over and we don’t even want to talk about it. And when we pass each other in the hall, we don’t even make eye contact. It’s that painful. OK. So this project has failed, no doubt about it.
KLEIN: Now, for the next two minutes — and I’m gonna time this — for the next two minutes, I want each of you to write down all the reasons why this project has failed. We know it failed. No doubts. Write down why it failed. I keep a strict clock because we don’t want to chew up too much time on this. But I see that everybody is writing and I say there’s 20 seconds to go, and they’re writing faster because they’re trying to get everything in. And now, five seconds. And then, okay, finish the sentence and pencils up.
Now Klein would go around the room and ask for one item from each person’s list. They compile a catalog of all the ways the project might fail. Klein calls this “prospective hindsight.” What does it accomplish?
KLEIN: It really reduces over-confidence. And people are usually way too confident at the beginning of a project. So the pre-mortem tempers that over-confidence.
Now Klein would ask everyone in the room to think up one thing they could do to help the project.
KLEIN: And then we’d go around the room to see what people are planning to do that they hadn’t thought of before to try to make it more successful.
So you put a happy ending on a potentially sad story?
KLEIN: Exactly. That’s what we’re trying to do. Without sugarcoating the problems that face us.
Now, here’s an obvious question. If you’re running a pre-mortem and everybody involved with the project is able to come up with a few ways in which the project might fail – why are these potential problems only coming up now, when the project is about to launch? Gary Klein says that’s because the pre-mortem liberates people who might otherwise be afraid of looking like they’re not a team player. Now everybody is being asked to think about failure. So instead of looking like a bad teammate, you’re pulling in the same direction as everyone else.
KLEIN: Now the demand characteristic is: show me how smart you are. Show me how clever you are and how experienced you are by identifying things that we need to worry about.
DUBNER: I’m curious to know how well the pre-mortem is catching on. Because I feel like, whenever I describe it to a group of people, small or large, they’re blown away by it. They love the idea. And very few of them have heard of it. So I’m curious, for an idea that sounds so great, sounds so practical, why hasn’t it taken over the world a bit more?
KLEIN: That’s very kind of you. I’m not entirely sure. I think one reason is, we have not gone out of our way to publicize it. This was just a kind of activity that grew up out of our own practice. And we were just doing it internally. After, we would do it with a few project managers and sponsors, and they said, ‘Can you use it with some of our other projects, even though you’re not involved?’ So I sort of see it the other way. I’m surprised at how many people are using it. I hear about people using it and nobody ever mentioned it to me. So I’m impressed that it’s gotten as much mileage as it has.
I went back to Steve Levitt with a couple of questions. Why do you think there is such a stigma associated with failure, generally?
LEVITT: I think to be willing to accept failure, you have to have self-confidence. That, um, you have to be accepting of the idea that failing isn’t, doesn’t define who you are. Failing gets something out of the way that keeps you from finding the thing that you’re actually going to be good at. I mean, look, we’re spoiled, you and I both have both stumbled onto a lot of success, and it’s so much easier to be terrible at things and to admit you are terrible when in other parts of your life you get rewarded and people write you letters and say you’re great. I think it’s easy for us to accept failure. And honestly, you know, I think I’ve gotten much better as I’ve gotten older at being able to laugh at myself. I was really insecure for a long time. I didn’t want to show people I was bad. But now I think I’ve actually gotten…I mean, I often will laugh the hard- When I do stupid things, I laugh the hardest.
DUBNER: So, what’s the strategy? If you’re the CEO or if you’re another CEO? Even if you’re the CEO of a two person operation listening to this? What’s the strategy to encourage people to acknowledge that failure may happen, and we’d be much better of[f] dealing with the failure before and not after?
LEVITT: The only credible way to make failure acceptable is to celebrate failure. I know at my little firm, TGG, The Greatest Good, we had our greatest failure actually turned out to be our greatest success in terms of morale building. I got asked to give a speech to one of our clients, and I told the truth in that speech and it led to our being fired the very next day. The executives of the company it turned out weren’t that interested in hearing the truth. And unceremoniously through email, we were fired from one of our biggest accounts. And the thing was that we celebrated it. Not just pretend celebrated but really celebrated. Our view was, there is no point in staying up all night working on projects for clients if they don’t appreciate it. If they don’t want to hear the truth then how in the world do we want to spend our life doing this work. And what happened was, people at our little firm, I think they would have thought we’d be upset. And it wasn’t just me, the people who ran the firm, we couldn’t have been happier. We broke out beer and wine and celebrated the fact that one of the biggest companies in the world had decided to fire us because we told the truth. And I think, one of the things you and I have really come to believe over time: it’s not for most people about money; there’s something bigger. That if you’re going to work hard and do something, it’s about the pride, and the joy of feeling like you’re part of something bigger. Something exciting. And the folks who worked at our little consulting firm actually felt that day it was about something bigger. We were about the truth, about doing good work, and that’s hard to do in a business setting. But it was, you know, I don’t want to get all misty, but it was kind of magical.
Comments