Survivor Bias on the Gridiron

The concept of survivor bias, if you don’t know it, is well worth being aware of. It’s most often used in finance, where it refers to a “tendency for failed companies to be excluded from performance studies” (thanks, Wikipedia). Think of the Dow Jones Industrial Average, which indexes the stock prices of 30 of the largest and most important U.S. companies — until, that is, one of said companies does so poorly that it is booted from the index and is replaced by a company that’s doing better.

Over time, therefore, the DJIA reflects a different reality than many people presume. It is biased toward survivors — or, if you want to think of the concept more broadly, toward winners.

This winner’s bias, if you will, shows up in pretty much every realm imaginable: academics, medicine, politics, etc. I don’t mean to sprinkle skepticism all over your inherently positive thoughts about the world, but I do think it’s worth keeping winner’s bias in mind whenever you read (or write) something about the performance of a given group or institution or coalition.

Winner’s bias is perhaps especially pronounced in sport. The behaviors of winners are remembered and dissected far more thoroughly than those of losers, and given greater weight, even if the outcome was decided by a tiny margin.

In the sports section of yesterday’s USA Today, I came across not one but two subtle examples of this phenomenon, in two articles on the very same page. Both articles discussed the fourth-quarter comeback of a much-favored N.F.L. team in a Monday night game; both contained quotes about a star player urging his team on to success. First, the Patriots came back to beat the Bills:

“When Tom [Brady] came in the huddle at 5:32 and said, ‘Guys, let’s get it going. We’re going to win this game,’ everything just went positive from there,” [Randy] Moss said.

And then the Chargers came back to beat the Raiders:

“L.T. [LaDainian Tomlinson] came over and said, ‘We’re going to win this football game,’” right tackle Jeromey Clary said.

“That kind of got us going again,” [Darren] Sproles said.

I have a simple question: does anyone really think that Brady and Sproles don’t say that kind of the thing in the huddle when their teams are behind and fail to come back?

Attributing causal effect to such a statement might seem natural for a player whose team just earned a big win, or for a sports reporter soaking up the happy locker-room vibe. But if you think about it, this kind of thinking is essentially nothing more than superstition. I’m not saying that people shouldn’t traffic in superstition if it turns them on (yeah, I’ve got a few of my own hoodoos, thanks). But when you’re trying to measure performance, we should remember to not actually believe in it.

TAGS: ,

Leave A Comment

Comments are moderated and generally will be posted if they are on-topic and not abusive.

 

COMMENTS: 46

View All Comments »
  1. Min says:

    I think people realize that stars like Tom Brady do say these things even if they end up failing. That’s part of their appeal: the never-say-die confidence they have in themselves, no matter how dire the circumstances.

    Thumb up 0 Thumb down 1
  2. David says:

    Dubner wrote: “does anyone really think that Brady and Sproles [he actually meant LT] don’t say that kind of the thing in the huddle when their teams are behind and fail to come back?”

    The idea isn’t whether the Pats (or Chargers) win EVERY time Brady (or LT) says something “inspirational” in the huddle. That would be a high bar for this type of rah-rah talk. However, a potentially more interesting question might be whether their teams are more likely, at the margin, to win when they display such confidence. Given that these players do probably say this every time they are behind, it would be hard to construct a good counterfactual.

    Thumb up 1 Thumb down 0
  3. Go Caps says:

    Would this apply to guarantees of victory before a game begins? I’m thinking of Mark Messier guaranteeing a win in game 6 of ’94 East finals, scoring a hat trick to keep the Rangers alive. You put significantly more on the line with pre-game guarantees in public than locker-room, huddle pep talks.

    Thumb up 0 Thumb down 0
  4. K Smith says:

    Granted the DJIA is biased toward survivors, as the losers are dropped from time to time, but it doesn’t quite have the same survivorship bias that is most often used in the finance industry. While the losers are dropped, if you do an analysis on the DJIA in aggregate, those losers remain a part of the final index price and performance until they are dropped. In the finance industry, oftentimes the opposite is true – such that the losers that have closed or been dropped from the analysis group disappear from the record, almost as if they were never there. This will theoretically bias performance results to the upside.

    Thumb up 0 Thumb down 0
  5. Mike says:

    I think that’s more of an issue of hindsight bias, rather than survivor bias. Perhaps the players remember their stars saying that, due to hindsight bias, and the news reporter reports on it due to survivor bias!

    Thumb up 0 Thumb down 0
  6. Cola says:

    My favorite example of this is when famous/successful people preach ‘reach for your dreams and they’ll come true’ to kids. It is a nice sentiment but obviously it omits the people who have reached for their dreams and it didn’t come true.
    Most times hard work and dedication does not amount to stardom and financial success.

    Thumb up 2 Thumb down 0
  7. Jeff says:

    I think Buffalo’s kick return fumble had more to do with the Patriots win than Tom Brady’s pronouncement.

    Thumb up 1 Thumb down 0
  8. Joe D says:

    I’m not sure this demonstrates your point. No, no one “really think[s they] don’t say that kind of thing” every time. And I imagine only the players themselves are at all apt to see causality.

    I’ve heard that athletes are coached to say just a few meaningless things to reporters, who then dutifully report them as if they were insights.

    Thumb up 0 Thumb down 0