Picking the NFL Playoffs: How the Experts Fumble the Snap

Photo: Ed Yourdon

Our latest Freakonomics Radio podcast, “The Folly of Prediction,” is built around the premise that humans love to predict the future, but are generally terrible at it. (You can download/subscribe at iTunes, get the RSS feed, or read the transcript here.) But predictions about world politics and the economy are hard — there are so many moving parts. What about something with less variables, like football for example?

In the podcast, you’ll hear from Freakonomics researcher Hayes Davenport, who ran the numbers for us on how accurate expert NFL pickings have been for the last 3 years. He put together a guest post for us on football predictions.

Picking the NFL Playoffs: How the Experts Fumble the Snap

As careers in journalism go, making preseason NFL predictions is about as safe as they come these days. The picks you make in August can’t be reviewed for four months, and by that time almost nobody remembers or cares what any individual picker predicted. So when Freakonomics asked me to look at the success rate of NFL experts in predicting division winners at the beginning of the season, I was excited to look back at the last few years of picks and help offer this industry one of its first brushes with accountability.

For my sample data, I looked at a total of 101 sets of predictions over the course of three seasons from pundits representing the three major sports journalism outlets: ESPN, Sports Illustrated, and USA Today. Overall, I found that NFL pickers chose the division winner correctly about 36% of the time. If they were picking at random, they’d get it right 25% of the time, so they’re certainly performing above the level of pure chance. Some years were better than others, of course: in 2010, the pickers actually did predict division winners with 25% accuracy, exactly as well as they’d do with a blindfold and a dartboard.

But how good is that 36% overall figure, really? Think of it this way: if the pickers were allowed to rule out one team from every division and then choose at random, they’d pick winners 33% of the time. So if you consider that most NFL divisions include at least one team with no hope of finishing first (this year’s Bengals, Chiefs, Dolphins, Panthers, Broncos, Vikings, and Manning-less Colts, for example), the pickers only need a minimum of NFL knowledge before essentially guessing in the dark.

How is it possible, then, that paid experts pick at about the same level of accuracy as any armchair prognosticator? Looking deeper into the predictions, one statistically derived explanation stands out: widespread risk aversion. See, football pickers tend to rely rather heavily on the previous year’s playoff picture when making their predictions: on average, they include 8.13 teams that made the playoffs the year before, and only 3.83 new teams. You can see the appeal of a strategy like that: if a team was good last season, shouldn’t they be good again?

But compared to any other major professional sport, NFL standings are the most volatile from year to year, and picking eight-plus teams to return to the playoffs just doesn’t make sense in a league with such parity. Over the last fifteen seasons, the NFL has averaged exactly six new teams in the playoffs every year, meaning that half of the playoff picture is completely different from the year before. The turnover is pretty consistent from year to year: not once in that decade-and-a-half have there been fewer than five new playoff teams from the season before.

Given that information, a savvy picker relying on statistical precedent would choose six new teams when predicting the playoffs. But in the 105 sets of picks we looked at over the last three years, pickers chose six or more new teams only 11 times. Some pickers actually predicted that there would be only one new team in the playoffs, which has never happened in the modern NFL.

There are plenty of individual cases that demonstrate how picking based on the previous season has led predictors astray. The Dallas Cowboys, for example, were a “hot” team in 2008. They’d won the hugely competitive NFC East the season before, and were selected by 29 of 35 pundits to repeat this. They ended up finishing third. So the next season, the pundits turned their back on the Cowboys, reluctant to get burned again. Only six of the 35 chose the Cowboys to win the division…which, of course, they did. In 2010, the pundits leapt back onto the bandwagon, with 26 of the 35 picking the Cowboys to take the NFC East. Once again, they were a year behind the trend: the Cowboys finished third again.

There is, of course, an obvious reason why pickers are inclined to play it safe: if you go with the crowd and pick a team that was successful last year, you’re unlikely to get completely embarrassed. Fans won’t remember if you picked the failed 10-6 Giants to make the playoffs, but they might remember the guy who picked the 2-14 Panthers to take their division. And if you do miss on all your picks for a season, you’ve got an easy out if everyone else was picking the same way you did. The football prediction industry, like the game it covers, is a lot easier when all the players are working together.

Hayes Davenport is a writer for Allen Gregory, which premieres in October on Fox. He also blogs about basketball at CelticsHub.com.

Leave A Comment

Comments are moderated and generally will be posted if they are on-topic and not abusive.



View All Comments »
  1. Mitch says:

    I think that predictions are bad so much as some things are impossible to predict. There’s too much randomness, particularly star players getting injured, to foresee the narrative that actually plays out, and sometimes the successful prediction isn’t the right one, given the information that’s available at the time.

    Well-loved. Like or Dislike: Thumb up 7 Thumb down 0
  2. Peter says:

    I think your assertion that “a savvy picker relying on statistical precedent would choose six new teams when predicting the playoffs” is silly. That’s kind of like saying that if Tiger Woods had a 49% chance of winning golf tournaments a few years ago, if you had to pick one player you would pick someone in the field because the field wins 51% of the time.

    Well-loved. Like or Dislike: Thumb up 27 Thumb down 7
    • Jesse says:

      Hidden due to low comment rating. Click here to see.

      Disliked! Like or Dislike: Thumb up 9 Thumb down 23
      • Peter says:

        Can you explain why you think it’s a bad analogy?

        Well-loved. Like or Dislike: Thumb up 9 Thumb down 0
      • Jon says:

        Hidden due to low comment rating. Click here to see.

        Disliked! Like or Dislike: Thumb up 3 Thumb down 16
      • Peter says:


        The two outcomes analogy is exactly the point. The situation the author presents is a two-outcomes situation: previous winners vs. new teams, and he suggests picking one team from the group of new teams, which would be equivalent to in the Tiger Woods example choosing one player from the group of other players (the field). While everyone else in the field together have a better shot of winning, any player in the field individually would be a worse choice than Tiger. In the NFL example while the group of new teams together have a better shot of winning than the previous winner, any individual new team would likely be a worse choice than the previous winner.

        Well-loved. Like or Dislike: Thumb up 20 Thumb down 3
    • Steve says:

      Exactly. It really depends on if you are rewarded for a better average, or if you are only rewarded for getting 100%. If you know that approximately half the teams repeat their division, your best strategy is to pick every team to repeat, that way you will get 50%. Much better than picking 50% of teams to repeat (of which you only get 50% right), and then picking another team for the remaining spots, which you will presumably get less than 50% right. However, if you are only rewarded for a 100% accurate prediction, then yeah, you need to take some chances.

      Well-loved. Like or Dislike: Thumb up 12 Thumb down 1
      • Joshua Northey says:

        Well the smart move really depends on if you are trying to maximize: the accuracy of your prediction, or its uniqueness, or its shock value (to drive traffic), or more likely what weight you put on those three things.

        Honestly most of them are just relatively unsophisticated and are trying to predict something that is relatively unpredictable. Case close.

        If you want to ding them on something I would say they are weakest on differentiating between which things in the previous year’s performance that led to wins and losses tend to correlate with the the following year’s performance, and which do not.

        As an example: It is a very bad habit to predict good teams to repeat who recovered say 75% of opponent fumbles when opponent fumble recovery is in fact very random and there is no reason to expect a repeat of that good fortune.

        So you re-evaluate the teams based on what elements of their performance tend to correlate year over year and then use last years performance on those measures as your guide, instead of using the aggregate performance on all measures which includes a lot of statistical noise.

        Well-loved. Like or Dislike: Thumb up 6 Thumb down 1
  3. YX says:

    I think most of the expert would do much better picking teams if their goal is to get as many right as possible.
    As Vegas sharps who bring money in year after year shown, it is possible to be right more often than wrong.
    But since their goal is to generate viewer-ship (interesting but not controversial) rather than be correct, the result is greatly skewed.

    Thumb up 4 Thumb down 0
    • ktb says:

      I was thinking the exact same thing! It’s not necessarily an issue of correctly determining the winner(s) – it’s an issue of determining which team has the most followers who want to hear that you think their team will win.

      It would be interesting to see a study that compared playoff predictions to demand for articles about those teams. The Cowboys example above would probably fit that very well.

      Thumb up 2 Thumb down 0
  4. RickD says:

    If you know that 6/12 playoff teams will return and 6/20 non-playoff teams will make it to the following year’s playoffs, it’s not necessarily savvy to pick a slate of playoff teams including the previous year’s non-playoff teams. Your likelihood of error increases every time you include a non-playoff team to join the set.
    Of course, this is an overly simplistic way to view the problem.

    Well-loved. Like or Dislike: Thumb up 18 Thumb down 1
    • Howard says:

      True, but if you’re trying to paint an accurate picture of the upcoming season’s playoffs, you should produce one that accounts for the high parity of the NFL. If you choose the entire playoff slate from the previous season your odds max out at 60% or so.

      Thumb up 1 Thumb down 2
    • Will W says:

      If you eliminate 8 teams that you conclude don’t really have a shot at the playoffs (this is subjective but like he mentioned there are some that don’t have chance), you’re really just picking 12/24 with six old teams and six new ones. Then it really just becomes about risk aversion: if you assume that about six teams are going to turn over, then you have a chance at getting zero playoff teams correct but you also increase your odds to a possible 100% accuracy, which you almost definitely don’t have if you just pick the entire slate from the year before.

      Thumb up 1 Thumb down 1
  5. Adam Y says:

    Playoff predictions are bad for the same reason that soccer players kick to the side on penalty kicks way more often than they should. Its because a successful penalty kick is not the only objective. The objective is to have a successful penalty kick, but also not look stupid if you fail. If you kick it down the middle and the goalie stops it, then you’re perceived as an idiot. In the same way, if you go out on a limb and you’re really wrong with your picks then you look like an idiot.

    Thumb up 3 Thumb down 2
    • alex in chicago says:

      As a soccer player I never understood this sentiment. While there are a great many statistical analyses about Penalty Kicks, they all ignore reality. The fact is that if you kick to the side, and kick it well, you are guaranteed a goal. The only reason to kick to the middle is if you aren’t good enough to not hit your spot.

      I’m 4/5 on PKs in my career, the only “miss” was when I went “down the middle” and luckily it was during the game and the keeper deflected it right to me…I didn’t make the same mistake on the rebound shot.

      Thumb up 3 Thumb down 2
      • Kyle says:

        While I understand your reasoning of why you shouldn’t kick a penalty kick right down the middle, a statistical sample of 5 kicks hardly seems like a reliable sample size from which to draw a conclusion. I’m willing to stick with statistical analysis based on all penalty kicks over multiple seasons.

        Thumb up 1 Thumb down 0
    • Meghan says:

      The intention of a PK is to score…missing makes you look like an idiot UNLESS the goalie makes a save that seems impossible…kicking the ball furthest from a standing goalie makes them choose the correct direction, height, and speed at which a ball is traveling…kicking it at a goalie makes it easier to save…getting a goalie to have to leave their feet removes control and strength…

      Thumb up 1 Thumb down 0
  6. Greg says:

    I like the article! But I’m not sure that picking 6 last-year playoff teams and 6 last-year non playoff teams for this year’s playoffs is “savvy”. If 6 of each (say) make the playoffs this year, then each of last year’s playoff teams have a 50% chance of making this year’s playoffs, while only 30% of last year’s non-playoff teams will make the playoffs this year. If the goal is to maximize the number of correct predictions this year, why not just go with last year’s entire playoff slate – knowing that there will be mistakes but also knowing that trying to predict the mistakes will probably result in a worse showing?

    Well-loved. Like or Dislike: Thumb up 8 Thumb down 0
  7. Caleb b says:

    It works the same way in finance too. Being the hero vs the goat. If you buy Apple and get burned, no big deal, everyone liked Apple. But go out on a limb with some Brazilian copper mine and you had better be right or you’re gone.

    Thumb up 2 Thumb down 0
  8. Caleb b says:

    Sorry, should have mentioned…if everyone bets on that obscure Brazilian copper mine and blows up, the Fed sweeps in with a bunch of free money and you don’t have to cancel that fancy European ski vacation you had planned.

    For example: see Sub-prime MBS/ CDOs.

    Thumb up 2 Thumb down 0