More Predictions, From Bad to Worse

fortune teller

(Photo: Josh McGinn)

Our “Folly of Prediction” podcast made these basic points:

Fact: Human beings love to predict the future.

Fact: Human beings are not very good at predicting the future.

Fact: Because the incentives to predict are quite imperfect — bad predictions are rarely punished — this situation is unlikely to change.

A couple of recent cases in point:

The National Oceanic and Atmospheric Administration predicted a particularly bad Atlantic hurricane season this year but, thankfully, were wrong, as noted by Dan Amira in New York magazine. It is hard to imagine that many people are unhappy about that. 

Here, as noted by Ira Stoll in the New York Sun, are the picks by ESPN experts at the start of the 2013 baseball season. How bad were their picks?

There were 43 experts in the poll. Not a single one of them picked either the Cardinals or the Red Sox to make it to the World Series. Granted, both teams were a bit of a surprise — but that’s my point: you would have done better placing one bet on each of the 30 MLB teams to make the Series rather than rely on the experts’ picks.

What is amazing is how much consensus there was among the picks — the Washington Nationals and Detroit Tigers, for instance, were very popular picks to make or win the Series. Sure, both teams looked good on paper but the degree of consensus in these picks should remind us all of just how much herding there is when experts make predictions.

I didn’t run all the numbers — you are free to do so — but the first example I looked at is pretty sobering. Twenty of the 43 experts picked the Toronto Blue Jays to win the A.L. East. How’d the Jays do? Dead last in their division, with a 74-88 record, 23 games behind the Red Sox — who, by the way, were picked by zero of the 43 pundits to win even their division, much less the A.L. or the World Series.

Question: how much do you think this poor outcome will discourage the same pundits from making their picks next spring?

Note: PunditTracker is keeping track of some of this mess.

Leave A Comment

Comments are moderated and generally will be posted if they are on-topic and not abusive.

 

COMMENTS: 12

View All Comments »
  1. matt says:

    Stephen, the Pundit Tracker is one of the coolest websites I have ever visited! Thanks for the link!

    Thumb up 3 Thumb down 0
  2. Chad C says:

    You cannot judge predictors based upon one prediction, especially one where any particular outcome has a low probability of occurring. Just because a lot of experts say the same thing does not necessarily imply herding. It is certainly possible that the Tigers and Nats had the best chance of making the World Series, but maybe those probabilities were less than 20%.

    Say I ask people to predict the outcome of the sum of two die rolls; every person should predict 7, but that has less than a 17% chance of happening. People can make the best prediction possible and still be wrong most of the time, and they can all be making the same prediction without herding if it is the most likely outcome.

    Well-loved. Like or Dislike: Thumb up 32 Thumb down 4
    • texagg04 says:

      Then what is missing with predictions are numbers to support those predictions as well as temper their likelihood.

      Thumb up 4 Thumb down 0
    • meanonsunday says:

      Not really a valid analogy since the predictions in question involve estimating unknown probabilities, and the uncertainty in any prediction is certainly large relative to the differences among teams.

      Thumb up 2 Thumb down 1
  3. Robert Childress says:

    I find sports predictions to be a mixed bag. On the one hand, they are made by people who supposedly lack the regional and local biases endemic in many teams’ hometown news outlets. However, this lack of local knowledge leads to overreliance on statistics which have still not quantified the intangibles that make sports so much fun to follow. If anyone asked me if I expected my hometown Nationals to repeat as AL East champs, I would have laughed. They didn’t do enough to shore up a power-lacking lineup, and their fielding remained sub-par. The same applies to my Redskins, who rode a wave of momentum on a rickety Steinbeckian wagon to reach the playoffs … only to have their defensive shortcomings and one-dimensional offense exposed in the playoffs.

    The point of these examples is this: Any non-biased person with adequate knowledge of a specific team can give a better prediction than a stats-reliant “expert”. The experts see a large forest from 30,000 feet, but the individual fan sees the dying, undernourished trees that constitute the forest.

    Thumb up 3 Thumb down 2
    • Eric R says:

      ‘If anyone asked me if I expected my hometown Nationals to repeat as AL East champs, ”

      I agree, the odds of the Nationals ever winning the *AL East* is fairly remote… winning two in a row? Even more so… :)

      Thumb up 2 Thumb down 0
  4. James says:

    “Fact: Human beings are not very good at predicting the future.”

    Falsehood. Humans are quite good at predicting those aspects of the future that are predictable. As for instance, notice that every one of the pundits who predicted the wrong World Series teams still came through on predicting that there actually would be a World Series.

    Indeed, I wouldn’t be at all surprised to discover that people could buy tickets to the event well in advance of knowing which particular teams would play, and if that’s not predicting the future, what is it?

    Thumb up 2 Thumb down 5
  5. Steve Nations says:

    I think a better question than how much this poor outcome will discourage the same pundits from making their picks next spring is how much this poor outcome will discourage anybody from asking for their predictions next spring.

    Thumb up 2 Thumb down 0
  6. Matt W. says:

    With the Flu season approaching this got me to thinking about how accurate they are in predicting what the dominant strains will be for a particular season. These predictions are the basis for the vaccine in a given year. How accurate have the been in the past? Anybody know?

    Thumb up 2 Thumb down 0
    • Enter your name... says:

      I don’t have the numbers any longer, but the answer, last I looked, was “pretty good—not great”. They give you three, and one of the three is almost always relevant. Some of it depends a bit on location, which makes it harder to assess. So, for example, you might have one strain more significant in the north than in the south: whether you think that one was a “good prediction” kind of depends on whether or not you live in the area that saw the most of it.

      Thumb up 0 Thumb down 0
  7. Bryan R says:

    I believe that despite the difficulty in predicting sports outcomes, people will continue to predict. The reason is because of the attention enjoyed by those who “get it right”. The large number of incorrect predictions has allowed those who fail to go by unnoticed, but those who succeed to enjoy attention, even if their predictions were due to chance. Sports predictions may be far from perfect, but the incentive to guess is so high that people will continue to speculate.

    Thumb up 0 Thumb down 0
  8. G says:

    Chad c has all you really need to know about this particular instance of prediction and its probable accuracy.

    I will also note that this highlights the need of managing or hedging your predictions or those that you follow. As an example a friend bet a long shot on a Stanley Cup winner once and it nearly paid off. Nearly. They lost in the finals. I asked him why he didn’t hedge. He said how and I told him by betting some of his possible windfall on the other/ remaining team(s). The same could have been done for a bet on the Jays early in the season when they were hot. It’s one thing to make a prediction but then you need to manage that prediction through to its conclusion. Just an aside.

    Thumb up 0 Thumb down 0