Bring Your Questions for FiveThirtyEight Blogger Nate Silver, Author of The Signal and the Noise

Nate Silver first gained prominence for his rigorous analysis of baseball statistics. He became even more prominent for his rigorous analysis of elections, primarily via his FiveThirtyEight blog. (He has also turned up on this blog a few times.)

Now Silver has written his first book, The Signal and the Noise: Why So Many Predictions Fail — But Some Don’t. I have only read chunks so far but can already recommend it. (I would like to think his research included listening to our radio hour “The Folly of Prediction,” but I have no idea.)

A section of Signal about weather prediction was recently excerpted in the Times Magazine. Relatedly, his chapter called ”A Climate of Healthy Skepticism” has already been attacked by the climate scientist Michael Mann. Given the stakes, emotions, and general unpredictability that surround climate change, I am guessing Silver will collect a few more such darts. (Yeah, we’ve been there.)

In the meantime, he has agreed to field questions about his new book from Freakonomics readers. So feel free to post your questions in the comments section below, and we’ll post his replies in short course. Here, to get you started, is the book’s table of contents:

 

1. A CATASTROPHIC FAILURE OF PREDICTION

2. ARE YOU SMARTER THAN A TELEVISION PUNDIT?

3. ALL I CARE ABOUT IS W’S AND L’S

4. FOR YEARS YOU’VE BEEN TELLING US THAT RAIN IS GREEN

5. DESPERATELY SEEKING SIGNAL

6. HOW TO DROWN IN THREE FEET OF WATER

7. ROLE MODELS

8. LESS AND LESS AND LESS WRONG

9. RAGE AGAINST THE MACHINES

10. THE POKER BUBBLE

11. IF YOU CAN’T BEAT ’EM . . . 

12. A CLIMATE OF HEALTHY SKEPTICISM

13. WHAT YOU DON’T KNOW CAN HURT YOU

 This post is no longer accepting comments. The answers to the Q&A can be found here.

COMMENTS: 54

View All Comments »
  1. David L says:

    Do you see a media bias toward (presidential) polls that demonstrate the narrowest results? (in other words, would an EIC/ME prefer to publish a 45-45 poll over a 48-42 poll)?

    The cynic in me argues that keeping the rhetorical election close will attract more readers/viewers.

    Well-loved. Like or Dislike: Thumb up 8 Thumb down 1
  2. paul clark says:

    There are so many polls produced during the two-year-plus American presidential election cycle. Do the polls really help educate voters or are they mainly an easy prop for media and pundits to create a short-term news burst?

    Thumb up 4 Thumb down 0
  3. Camille Sweeney says:

    What’s the biggest mistake that major pollsters make?
    Camille Sweeney
    Twitter @artofdoingbook

    Thumb up 4 Thumb down 0
  4. Quin says:

    How would you compare the accuracy of polls vs. prediction markets for American elections? Why don’t you include the latter in your estimates on FiveThirtyEight?

    Well-loved. Like or Dislike: Thumb up 7 Thumb down 1
  5. Seminymous Coward says:

    Which party would win a baseball game with only its national-level politicians as players?

    Well-loved. Like or Dislike: Thumb up 15 Thumb down 2
  6. Ryan says:

    What’s your response to the “attacks” by Michael Mann on your climate science chapter?

    Well-loved. Like or Dislike: Thumb up 12 Thumb down 1
    • James says:

      Hidden due to low comment rating. Click here to see.

      Disliked! Like or Dislike: Thumb up 3 Thumb down 9
      • J1 says:

        Orbital mechanics problems aren’t predictions; they’re (not particularly complicated) math problems that assume compliance with known laws of physics. Determining the path of a spacecraft after launch is not analogous to predicting the weather.

        Well-loved. Like or Dislike: Thumb up 8 Thumb down 1
      • James says:

        They certainly are predictions: at time t you predict an event that will take place at time t+dt.

        You’re right that they’re predictions made by doing computations based on known laws of physics. So are many other sorts of predictions, for instance those involving weather and climate. Indeed, weather prediction is not particularly complicated, physically or mathematically. The difficulty in making long-term weather predictions stems (as Silver himself discusses in the linked NYT article) from the inherent sensitivity to initial conditions of the physics involved.

        That is where the Freakonomics guys display their ignorance: they make blanket statements about the impossibility of prediction, when in fact many problems (like orbital mechanics) are quite predictable, others (like weather) are predictible within calculable limits, still others may not be predictable at all. Rather than dismiss all prediction as folly, it would seem sensible to try to understand what can and can’t be predicted, and why.

        Thumb up 5 Thumb down 6
      • Ryan says:

        I think this is a very important distinction that Mann bring up. Making predictions based on fundamental physical laws is very different than making predictions about, say, economic activity.

        However, it should be pointed out that the calculations behind weather prediction are not as settled as James says. While its true that we (physicists) have a very good handle on particle-particle interactions, weather simulations are not done by simulating trillions (or whatever) of individual atoms. They are done, out of necessity, by utilizing approximations to get the simulation down to a manageable scale.

        It’s these approximations, rather than the underlying laws of nature, that add an element of uncertainty to the predictions. The physics is well understood and relatively simple. The approximations are neither.

        I do agree though that to lump scientific weather models in with “less scientific” predictions and then try to make blanket statements about the entire group is folly.

        Thumb up 3 Thumb down 0
      • James says:

        Sorry, but it is the exponential nature of the physical laws that makes long-term weather prediction fundamentally impossible. Silver’s linked NYT article has a good explanation. The approximations would be good enough, if the physical laws allowed it. After all, we don’t have to simulate every atom of a space probe to predict its trajectory: the physical laws are such that we can approximate it very well as a single particle.

        Now in theory it would be predictible by particle simulation IF we could know the exact initial conditions of every single atom, but we can’t. Not can’t as in we don’t have the instruments & data storage needed, but can’t as in Heisenberg’s Uncertainty Principle is a fundamental law of nature.

        Thumb up 1 Thumb down 0
      • Ryan says:

        Surely chaos plays a role in the difficulty of long term weather prediction. But to pretend that there aren’t modeling issues or that the computational methods are somehow “settled” is disingenuous.

        Right now instrumentation, data storage and computational power are many orders of magnitude larger impediments to attempting such an ‘atomic’ calculation than fundamental limits are, so Heisenberg is irrelevant.

        Even without Heisenberg, such a simulation wouldn’t work because of finite precision. There are many sources of inaccuracy in these simulations.

        Thumb up 2 Thumb down 0
  7. Scott Blackburn says:

    The argument for aggregation is that if you ask individuals to estimate something, the average of their results is more accurate than any one individual predictor. This has led to successful poll based models.

    But polls are not completely independent variables like estimates. Do you think its possible that we will see an election, where aggregation will build in some methodology bias and lead to vastly inaccurate predictions?

    Thumb up 1 Thumb down 0
  8. Quin says:

    In the primaries, my preferred candidates never seem to win, but as I talk to people in my circle, they say they like my candidate best also but feel compelled to vote for one of the front-runners so as not to waste their vote. How would (did?) elections fare without polling? How do published polling results bias future polls? Can anything be done about it?

    Well-loved. Like or Dislike: Thumb up 18 Thumb down 1