Survivor Bias on the Gridiron

The concept of survivor bias, if you don’t know it, is well worth being aware of. It’s most often used in finance, where it refers to a “tendency for failed companies to be excluded from performance studies” (thanks, Wikipedia). Think of the Dow Jones Industrial Average, which indexes the stock prices of 30 of the largest and most important U.S. companies — until, that is, one of said companies does so poorly that it is booted from the index and is replaced by a company that’s doing better.

Over time, therefore, the DJIA reflects a different reality than many people presume. It is biased toward survivors — or, if you want to think of the concept more broadly, toward winners.

This winner’s bias, if you will, shows up in pretty much every realm imaginable: academics, medicine, politics, etc. I don’t mean to sprinkle skepticism all over your inherently positive thoughts about the world, but I do think it’s worth keeping winner’s bias in mind whenever you read (or write) something about the performance of a given group or institution or coalition.

Winner’s bias is perhaps especially pronounced in sport. The behaviors of winners are remembered and dissected far more thoroughly than those of losers, and given greater weight, even if the outcome was decided by a tiny margin.

In the sports section of yesterday’s USA Today, I came across not one but two subtle examples of this phenomenon, in two articles on the very same page. Both articles discussed the fourth-quarter comeback of a much-favored N.F.L. team in a Monday night game; both contained quotes about a star player urging his team on to success. First, the Patriots came back to beat the Bills:

“When Tom [Brady] came in the huddle at 5:32 and said, ‘Guys, let’s get it going. We’re going to win this game,’ everything just went positive from there,” [Randy] Moss said.

And then the Chargers came back to beat the Raiders:

“L.T. [LaDainian Tomlinson] came over and said, ‘We’re going to win this football game,'” right tackle Jeromey Clary said.

“That kind of got us going again,” [Darren] Sproles said.

I have a simple question: does anyone really think that Brady and Sproles don’t say that kind of the thing in the huddle when their teams are behind and fail to come back?

Attributing causal effect to such a statement might seem natural for a player whose team just earned a big win, or for a sports reporter soaking up the happy locker-room vibe. But if you think about it, this kind of thinking is essentially nothing more than superstition. I’m not saying that people shouldn’t traffic in superstition if it turns them on (yeah, I’ve got a few of my own hoodoos, thanks). But when you’re trying to measure performance, we should remember to not actually believe in it.


I think people realize that stars like Tom Brady do say these things even if they end up failing. That's part of their appeal: the never-say-die confidence they have in themselves, no matter how dire the circumstances.


Dubner wrote: "does anyone really think that Brady and Sproles [he actually meant LT] don't say that kind of the thing in the huddle when their teams are behind and fail to come back?"

The idea isn't whether the Pats (or Chargers) win EVERY time Brady (or LT) says something "inspirational" in the huddle. That would be a high bar for this type of rah-rah talk. However, a potentially more interesting question might be whether their teams are more likely, at the margin, to win when they display such confidence. Given that these players do probably say this every time they are behind, it would be hard to construct a good counterfactual.

Go Caps

Would this apply to guarantees of victory before a game begins? I'm thinking of Mark Messier guaranteeing a win in game 6 of '94 East finals, scoring a hat trick to keep the Rangers alive. You put significantly more on the line with pre-game guarantees in public than locker-room, huddle pep talks.

K Smith

Granted the DJIA is biased toward survivors, as the losers are dropped from time to time, but it doesn't quite have the same survivorship bias that is most often used in the finance industry. While the losers are dropped, if you do an analysis on the DJIA in aggregate, those losers remain a part of the final index price and performance until they are dropped. In the finance industry, oftentimes the opposite is true - such that the losers that have closed or been dropped from the analysis group disappear from the record, almost as if they were never there. This will theoretically bias performance results to the upside.


I think that's more of an issue of hindsight bias, rather than survivor bias. Perhaps the players remember their stars saying that, due to hindsight bias, and the news reporter reports on it due to survivor bias!


My favorite example of this is when famous/successful people preach 'reach for your dreams and they'll come true' to kids. It is a nice sentiment but obviously it omits the people who have reached for their dreams and it didn't come true.
Most times hard work and dedication does not amount to stardom and financial success.


I think Buffalo's kick return fumble had more to do with the Patriots win than Tom Brady's pronouncement.

Joe D

I'm not sure this demonstrates your point. No, no one "really think[s they] don't say that kind of thing" every time. And I imagine only the players themselves are at all apt to see causality.

I've heard that athletes are coached to say just a few meaningless things to reporters, who then dutifully report them as if they were insights.


If I may theorize why this type of survivor bias is so common; it's because I believe that there is a causal effect between a negative attitude and a negative performance. Actually, I haven't seen any data in the sporting world confirming this, but Robert Sutton (Stanford prof) does highlight this effect for organization change efforts (see his book,"Hard Facts, Dangerous Half-Truths, and Total Nonsense"). Sutton's summary of the studies on change is that negative attitudes towards change efforts tend to be a self-fulfilling prophecy. So while the opposite may not be true, it's easy to see why people draw a linkage between attitude and performance.


Yea agreed, I don't really think this should be classified as survivorship bias at all.


You mean the people who lose games or wars weren't defeated because our side believed more in God?


This is definitely survivor bias. You don't get to the level of professional football -- the top thousand people in the country at it -- without (among other things) being really, really confident that you're always going to win, that you're always going to come back, that you're always going to be better the next time. No one in the NFL slumps to the huddle when his team is down by less than two touchdowns and says, "Geez, guys, I dunno... feel like packing it in? I mean, it's a long bus ride back to the hotel. We could beat the crowds."


IMO, the most famous example of survivor bias is evolution. People say, "it's impossible that the elephant evolved without help from a higher power!" Yet how many species died out? We see only the ones that made it. The dead ends are not counted, although they are far more numerous. In that context, the elephant is not amazing at all.

Also concur with #5. How about some statistics on how many make it/fail? My guess is less than 5% get to the top.

David LaMar

While I'm a fan of Dubner, he's wrong on this point. DJIA is not a case of survivor bias as the performance of the index reflects stocks that are falling until they are removed.

A better example of survivor bias is a mutual fund company that brags that 90% of its funds beat their respective index over the last 10 years, perhaps ignoring that fact that underperforming funds were closed or merged into outperforming funds.


Has anyone ever done a study to look at the performence of the companies listed in the DOW of any particular year?


Not to be picky, but since Moss and Brady have been on the same team (excluding last season when Brady was injured all year) the Patriots have won 19 out of 20 games. The one loss, the Super Bowl against the Giants, was lost by the Patriots defense after Brady seemed to win the game late (with a TD pass to Moss no less). From this perspective, in Moss' mind, Brady has never failed to follow through on winning the game!


@Cola - So are you saying that in order to reach your dreams you shouldn't work hard?

What makes LT & Brady the superstars that they are? If every leader says the same thing when losing, then the teams that win the game must be better leaders. I wonder what a football couch would have to say about this article and the importance of a "vocal" leader on a football team (especially at the Quarterback position).


Related to those who show confidence in the huddle are those who, after winning, thank God that he helped them succeed in such a big game.

1. Did the losing team *fail* to ask for God's help?
2. Was the losing team somehow less deserving of divine intervention?
3. Doesn't God have more important things to do on Sundays than to watch football?

tom p

The Brady remark is particularly galling. While he scored a late touchdown he threw an interception on the two point conversion which would have put them 3 points down (and only needing a field goal to tie.) Then after scoring another TD (after the Bills committed the most bone-headed play of the new season) he blew the two point conversion thereby giving the Bills the chance to tie with just under a minute left. As the cliche goes, "Success has many fathers, Failure is but an orphan."


Who still wonders why the news is so quick to report the Dow and why anybody cares about it for market analysis?