Bring Your Questions for FiveThirtyEight Blogger Nate Silver, Author of The Signal and the Noise

Nate Silver first gained prominence for his rigorous analysis of baseball statistics. He became even more prominent for his rigorous analysis of elections, primarily via his FiveThirtyEight blog. (He has also turned up on this blog a few times.)

Now Silver has written his first book, The Signal and the Noise: Why So Many Predictions Fail — But Some Don’t. I have only read chunks so far but can already recommend it. (I would like to think his research included listening to our radio hour “The Folly of Prediction,” but I have no idea.)

A section of Signal about weather prediction was recently excerpted in the Times Magazine. Relatedly, his chapter called “A Climate of Healthy Skepticism” has already been attacked by the climate scientist Michael Mann. Given the stakes, emotions, and general unpredictability that surround climate change, I am guessing Silver will collect a few more such darts. (Yeah, we’ve been there.)

In the meantime, he has agreed to field questions about his new book from Freakonomics readers. So feel free to post your questions in the comments section below, and we’ll post his replies in short course. Here, to get you started, is the book’s table of contents:

 

1. A CATASTROPHIC FAILURE OF PREDICTION

2. ARE YOU SMARTER THAN A TELEVISION PUNDIT?

3. ALL I CARE ABOUT IS W’S AND L’S

4. FOR YEARS YOU’VE BEEN TELLING US THAT RAIN IS GREEN

5. DESPERATELY SEEKING SIGNAL

6. HOW TO DROWN IN THREE FEET OF WATER

7. ROLE MODELS

8. LESS AND LESS AND LESS WRONG

9. RAGE AGAINST THE MACHINES

10. THE POKER BUBBLE

11. IF YOU CAN’T BEAT ’EM . . . 

12. A CLIMATE OF HEALTHY SKEPTICISM

13. WHAT YOU DON’T KNOW CAN HURT YOU

 This post is no longer accepting comments. The answers to the Q&A can be found here.


David L

Do you see a media bias toward (presidential) polls that demonstrate the narrowest results? (in other words, would an EIC/ME prefer to publish a 45-45 poll over a 48-42 poll)?

The cynic in me argues that keeping the rhetorical election close will attract more readers/viewers.

paul clark

There are so many polls produced during the two-year-plus American presidential election cycle. Do the polls really help educate voters or are they mainly an easy prop for media and pundits to create a short-term news burst?

Camille Sweeney

What's the biggest mistake that major pollsters make?
Camille Sweeney
Twitter @artofdoingbook

Quin

How would you compare the accuracy of polls vs. prediction markets for American elections? Why don't you include the latter in your estimates on FiveThirtyEight?

Seminymous Coward

Which party would win a baseball game with only its national-level politicians as players?

Ryan

What's your response to the "attacks" by Michael Mann on your climate science chapter?

James

I too would be interested in seeing this response, and also in a discussion of how you (and these Freakonomics guys) reconcile their claims of unpredictability with e.g. NASA's ability to predict the paths of space probes to accuracies of better than one part in 100 million (Curiousity rover)?

J1

Orbital mechanics problems aren't predictions; they're (not particularly complicated) math problems that assume compliance with known laws of physics. Determining the path of a spacecraft after launch is not analogous to predicting the weather.

Scott Blackburn

The argument for aggregation is that if you ask individuals to estimate something, the average of their results is more accurate than any one individual predictor. This has led to successful poll based models.

But polls are not completely independent variables like estimates. Do you think its possible that we will see an election, where aggregation will build in some methodology bias and lead to vastly inaccurate predictions?

Quin

In the primaries, my preferred candidates never seem to win, but as I talk to people in my circle, they say they like my candidate best also but feel compelled to vote for one of the front-runners so as not to waste their vote. How would (did?) elections fare without polling? How do published polling results bias future polls? Can anything be done about it?

Dave M

If the public had no knowledge of poll results would the election outcome be any different? Is seeing "your guy" falling behind a motivation to vote? Perhaps seeing him too far behind causes people to stay home since their vote won't matter?

Quin

Seems like we were thinking along the same lines and at almost the same time! Good show, old chap!

Jeff Bladt

What industries do you see as next to revolutionized by data? I keep hearing that PR is particularly vulnerable, in the emperor-has-no-clothes sense.

Nick

Who should I bet on to win the world series?

frankenduf

do u vote?- and if so, does it produce cognitive dissonance if ur stats lead u to believe that ur vote wont change the outcome?

rempart

I have the book on order, can't wait for the 27th. How about some about a fantasy baseball question? In a draft league, do you think it is best to use a good prediction model (Pecota) for example and let it stand so to speak. I mean of course, just line them up top to bottom and only use minor manipulations ( something like inside info on an injury). The question implies the use of intuition comapred to the model. Daniel Kahneman refers to this in his book "thinking slow and fast". The notion of models outperforming intuition. He also talks about checklists as being useful, if applied in a disciplined manner. What I'm getting at it is, a baseline Pecota type projection, followed up by a checklist of the top 5-6 things that move a player off his projection. Do you think this could help identify the Jose Bautista, Cliff Lee errors earlier. Or is this just noise?

Owinok

Nate, could you let readers here know what statistical analysis package you use? Secondly, have any firms or collaborators asked you to evaluate any packages that they have made?

Lenny

It appears to me that prediction markets such as Intrade are continuing to grow, so I'm assuming that the insight that one can get from them is changing as more people use them. What's your opinion of the usefulness of prediction markets and how has that changed over the past few years? How do you feel they will change moving forward?

Dutch

What's your take on the public equity markets--are they efficient or nearly-efficient? How much predictive power does a change in a company's stock price have on future earnings annoucement?

Dutch

Pardon me if you mind me asking, but what do you do with your personal investments? Do you buy any individual stock names, or do you stick with indexing? Overall, how do you apply your work in predictions to your personal finances? To me, the prices of securities have some predictive power and I'd love to hear your take on how you think about this and apply it to yourself.

Devin

For anyone interested in the subject of predictions, I'd recommend Future Babble by Dan Gardner. I haven't read Silver's book, but I would imagine that it would be similar to Gardner's.

Has Silver read Future Babble? Has anyone at Freakonomics read it?

Does Silver's book contain anything about Philip Tetlock?

Eric M. Jones.

Anyone interested in predictions MUST READ "The Book of Predictions" by Amy Wallace Irving Wallace David Wallechinsky 1980. You can always pick up a used copy someplace. I keep mine in a velvet-lined box inside a safe.

BTW:

I always like to do surveys by asking 1000 people with IQs of 150 and 1000 people with IQ's of 50. That way I get a population average IQ of 100. My surveys describe the way the world works very well.

RGJ

Hi Nate,

Baseball question, if you don't mind.

Why would a National League team, particularly one with financial constraints, not consider the following approach?

Sign no high priced starting pitchers. Fill your pitching roster with middle reliever/reliever pitchers with different attributes -- junk ballers, power pitchers, sidearmers -- with a lefty/righty balance.

Balance your roster with righty and lefty position players.

In the course of a game, substitute (except for blowouts) for the pitcher when his place comes up in the order, and do so with a righty or lefty batter depending on the pitcher.

Advantages:

-- You bring an American League offense to the National League.
-- In key situations, you have a disproportionate number of righty/lefty pitching matchups.
-- In key situations, you have a disproportionate number of righty/lefty fielding matchups.
-- Rather than seeing the same pitcher four or five times in a row, an opposing batter may face batters of different styles and handedness (sic?) in a game.
-- Keep payroll down by avoiding high priced starting pitchers.
-- Arguable, but may have an injury benefit by avoiding 100 plus pitch outings to pitchers.
-- keeps the entire roster active

Disadvantages/Challenges

-- will have trouble attracting pitchers focused on traditional counting stats and Ws.
-- will pose a pitching management challenge in rest and IP.

Naturally, in a blow out situation, you could simply have a young pitcher stretch out for 7 or 8 innings and save everyone else's arm. And it is important to note that now, when there is an upside blowout with a team's ace, those usual innings are pitched by him in search of the almighty W.

I ran this by Bill James on his website. His comment was that the only downside he saw was the part about attracting top starting pitchers. He also said that, in reality, baseball has been moving this way, albeit glacially, with the growth of middle relievers and closers and emphasis on righty/lefty situational matchups.

Thoughts?

Read more...

Laura Harrison McBride

Have you done any research on homeopathy...that is, why the allopathic medical community believes it doesn't work--despite more anecdotal evidence than you could fit in the Bodleian Library--insisting that only highly fallible so-called double-blind studies (the things that have given us at least six "miracle" drugs recalled for killing people at alarming rates) can prove a medicine's effectiveness?