Picking the NFL Playoffs: How the Experts Fumble the Snap

Photo: Ed Yourdon

Our latest Freakonomics Radio podcast, “The Folly of Prediction,” is built around the premise that humans love to predict the future, but are generally terrible at it. (You can download/subscribe at iTunes, get the RSS feed, or read the transcript here.) But predictions about world politics and the economy are hard — there are so many moving parts. What about something with less variables, like football for example?

In the podcast, you’ll hear from Freakonomics researcher Hayes Davenport, who ran the numbers for us on how accurate expert NFL pickings have been for the last 3 years. He put together a guest post for us on football predictions.

Picking the NFL Playoffs: How the Experts Fumble the Snap

As careers in journalism go, making preseason NFL predictions is about as safe as they come these days. The picks you make in August can’t be reviewed for four months, and by that time almost nobody remembers or cares what any individual picker predicted. So when Freakonomics asked me to look at the success rate of NFL experts in predicting division winners at the beginning of the season, I was excited to look back at the last few years of picks and help offer this industry one of its first brushes with accountability.

For my sample data, I looked at a total of 101 sets of predictions over the course of three seasons from pundits representing the three major sports journalism outlets: ESPN, Sports Illustrated, and USA Today. Overall, I found that NFL pickers chose the division winner correctly about 36% of the time. If they were picking at random, they’d get it right 25% of the time, so they’re certainly performing above the level of pure chance. Some years were better than others, of course: in 2010, the pickers actually did predict division winners with 25% accuracy, exactly as well as they’d do with a blindfold and a dartboard.

But how good is that 36% overall figure, really? Think of it this way: if the pickers were allowed to rule out one team from every division and then choose at random, they’d pick winners 33% of the time. So if you consider that most NFL divisions include at least one team with no hope of finishing first (this year’s Bengals, Chiefs, Dolphins, Panthers, Broncos, Vikings, and Manning-less Colts, for example), the pickers only need a minimum of NFL knowledge before essentially guessing in the dark.

How is it possible, then, that paid experts pick at about the same level of accuracy as any armchair prognosticator? Looking deeper into the predictions, one statistically derived explanation stands out: widespread risk aversion. See, football pickers tend to rely rather heavily on the previous year’s playoff picture when making their predictions: on average, they include 8.13 teams that made the playoffs the year before, and only 3.83 new teams. You can see the appeal of a strategy like that: if a team was good last season, shouldn’t they be good again?

But compared to any other major professional sport, NFL standings are the most volatile from year to year, and picking eight-plus teams to return to the playoffs just doesn’t make sense in a league with such parity. Over the last fifteen seasons, the NFL has averaged exactly six new teams in the playoffs every year, meaning that half of the playoff picture is completely different from the year before. The turnover is pretty consistent from year to year: not once in that decade-and-a-half have there been fewer than five new playoff teams from the season before.

Given that information, a savvy picker relying on statistical precedent would choose six new teams when predicting the playoffs. But in the 105 sets of picks we looked at over the last three years, pickers chose six or more new teams only 11 times. Some pickers actually predicted that there would be only one new team in the playoffs, which has never happened in the modern NFL.

There are plenty of individual cases that demonstrate how picking based on the previous season has led predictors astray. The Dallas Cowboys, for example, were a “hot” team in 2008. They’d won the hugely competitive NFC East the season before, and were selected by 29 of 35 pundits to repeat this. They ended up finishing third. So the next season, the pundits turned their back on the Cowboys, reluctant to get burned again. Only six of the 35 chose the Cowboys to win the division…which, of course, they did. In 2010, the pundits leapt back onto the bandwagon, with 26 of the 35 picking the Cowboys to take the NFC East. Once again, they were a year behind the trend: the Cowboys finished third again.

There is, of course, an obvious reason why pickers are inclined to play it safe: if you go with the crowd and pick a team that was successful last year, you’re unlikely to get completely embarrassed. Fans won’t remember if you picked the failed 10-6 Giants to make the playoffs, but they might remember the guy who picked the 2-14 Panthers to take their division. And if you do miss on all your picks for a season, you’ve got an easy out if everyone else was picking the same way you did. The football prediction industry, like the game it covers, is a lot easier when all the players are working together.

Hayes Davenport is a writer for Allen Gregory, which premieres in October on Fox. He also blogs about basketball at CelticsHub.com.


I think that predictions are bad so much as some things are impossible to predict. There's too much randomness, particularly star players getting injured, to foresee the narrative that actually plays out, and sometimes the successful prediction isn't the right one, given the information that's available at the time.


I think your assertion that "a savvy picker relying on statistical precedent would choose six new teams when predicting the playoffs" is silly. That's kind of like saying that if Tiger Woods had a 49% chance of winning golf tournaments a few years ago, if you had to pick one player you would pick someone in the field because the field wins 51% of the time.


Worst analogy ever…


Can you explain why you think it's a bad analogy?


I think most of the expert would do much better picking teams if their goal is to get as many right as possible.
As Vegas sharps who bring money in year after year shown, it is possible to be right more often than wrong.
But since their goal is to generate viewer-ship (interesting but not controversial) rather than be correct, the result is greatly skewed.


I was thinking the exact same thing! It's not necessarily an issue of correctly determining the winner(s) - it's an issue of determining which team has the most followers who want to hear that you think their team will win.

It would be interesting to see a study that compared playoff predictions to demand for articles about those teams. The Cowboys example above would probably fit that very well.


If you know that 6/12 playoff teams will return and 6/20 non-playoff teams will make it to the following year's playoffs, it's not necessarily savvy to pick a slate of playoff teams including the previous year's non-playoff teams. Your likelihood of error increases every time you include a non-playoff team to join the set.
Of course, this is an overly simplistic way to view the problem.

Adam Y

Playoff predictions are bad for the same reason that soccer players kick to the side on penalty kicks way more often than they should. Its because a successful penalty kick is not the only objective. The objective is to have a successful penalty kick, but also not look stupid if you fail. If you kick it down the middle and the goalie stops it, then you're perceived as an idiot. In the same way, if you go out on a limb and you're really wrong with your picks then you look like an idiot.

alex in chicago

As a soccer player I never understood this sentiment. While there are a great many statistical analyses about Penalty Kicks, they all ignore reality. The fact is that if you kick to the side, and kick it well, you are guaranteed a goal. The only reason to kick to the middle is if you aren't good enough to not hit your spot.

I'm 4/5 on PKs in my career, the only "miss" was when I went "down the middle" and luckily it was during the game and the keeper deflected it right to me...I didn't make the same mistake on the rebound shot.


I like the article! But I'm not sure that picking 6 last-year playoff teams and 6 last-year non playoff teams for this year's playoffs is "savvy". If 6 of each (say) make the playoffs this year, then each of last year's playoff teams have a 50% chance of making this year's playoffs, while only 30% of last year's non-playoff teams will make the playoffs this year. If the goal is to maximize the number of correct predictions this year, why not just go with last year's entire playoff slate - knowing that there will be mistakes but also knowing that trying to predict the mistakes will probably result in a worse showing?


"I was excited to look back at the last few years of picks and help offer this industry one of its first brushes with accountability."

You must not read ESPN's TMQ. Specifically his annual bad predictions review:

Caleb b

It works the same way in finance too. Being the hero vs the goat. If you buy Apple and get burned, no big deal, everyone liked Apple. But go out on a limb with some Brazilian copper mine and you had better be right or you're gone.

Caleb b

Sorry, should have mentioned...if everyone bets on that obscure Brazilian copper mine and blows up, the Fed sweeps in with a bunch of free money and you don't have to cancel that fancy European ski vacation you had planned.

For example: see Sub-prime MBS/ CDOs.

Joshua Northey

"Given that information, a savvy picker relying on statistical precedent would choose six new teams when predicting the playoffs. "

This whole section shows you don't understand statistics, which is kind of sad and shocking for someone posting on this site.

Even if on average half of the teams each year are new, the best prediction could very well be that all of last year's teams repeat under a wide array of assumptions.

NFL Pundits are bad at picking NFL division winners mostly because it is unpredictable. That is all that really needs to be said on that subject.


And how about all those idiots who pick all the one seeds in the NCAA tournament? Don't they know how unlikely it is for that to happen?


"Given that information, a savvy picker relying on statistical precedent would choose six new teams when predicting the playoffs. "

No... Given that information, a savvy picker would predict all 12 playoff teams to repeat their playoff appearances, giving them 1 in 2 rather than 1 in 3 like the other "experts".


Well, I challenge you Freakonomics to set up the challenge to choose the playoff teams for next season. Compare who is the most accurate...NFL analysts, statisticians, economists, and a set of fans...I'm in as a fan!!!!