FiveThirtyEighter Nate Silver Answers Your Questions About Politics, Baseball, and The Signal and the Noise
We recently solicited your questions for Nate Silver regarding his new book The Signal and the Noise: Why So Many Predictions Fail — But Some Don’t. Not too surprisingly, a lot of the questions were about politics and baseball. Below are Nate’s answers to some of them. Thanks to him for playing along and to all of you (as always) for sending in the excellent questions.
Q. Under what circumstances will a voter actually change his/her mind about whom to vote for? I understand that this rarely happens (this study for example), and that most of the action involves undecided voters deciding whom to vote for.
Also, if political scientist are right that voters rarely change their minds, how can a large swing in the polls ever occur? A classic example that your briefly mention in your book is that of Michael Dukakis, who was ahead of GHW Bush by 10% at one point in 1988. –Alan T
A. We see more big shifts in the primaries, when voters don’t have that much information about the candidates. Dukakis was a relative unknown at the start of the 1988 race, before the two parties could advance their own narratives. You rarely see big swings in voter conversion in late stage presidential races, though. If I knew how to cause such a swing, I’d be drawing a big salary from one of the campaigns right now.
Q. On basis of what methodology criteria do you rate the polls on bias? Which kind of specific methodology features of the polls do you take into account? Data collection, data extraction and sample size? How willing are all the polls to share this with you? –Robin Vernooij
A. We evaluate polls based on their sample size, how recently they were conducted (though this can be overrated in terms of importance), and qualitative rating of the poll based on results and methodology (belonging to a professional polling organization can serve as a proxy here.) Not including cell phone data is a big issue, as you can miss 1/3 of the population now. We only “punish” a poll for bias if it leads them to make errors.
Q.Do you have any idea as to the scope of the possible effect of the alleged voter suppression laws? Does the 538 model takes this into account? –Martin Schwimmer
A. In states with the new laws, we made small changes to the model based on work from political science. Of course, a lot of these laws are now neutered, as PA’s was this week. From a forecasting (not a democratic) perspective, the concerns are overblown because of the reaction of the courts. Studies say the narrative of voter turnout decline is exaggerated. Yes, laws that affect more poor people will keep Democrats from the polls, but they will also keep some Republican voters away. The Obama campaign has a tremendous get out the vote machine, in any event. We hope that the winner is decided by at least a few percentage points so that we don’t have to relive all of these scenarios.
Q. When predictions involve human ‘systems’ & behavior (social, economic, political etc) that are by their very nature ‘adaptive’, how do you deal with the tricky “Heisenberg Principle” — like effect where the very act of predicting itself becomes a factor that adds information that alters the system and influences individual and/or collective behavior? –John
A. This is a gigantic problem. In the book, we discuss how consumers, politicians, and businesses make plans based on economic forecasts that can have a host of problems. We also look at how this manifests in disease modeling. If you accurately forecast a very bad flu, it may cause people to stay home, which is good but cancels your forecast. But, the forecast served its purpose because it made people aware of their circumstances.
Q. Way back in 2005, Steven Levitt said: “My contention is that the secret to Oakland’s success has little to do with the things described in Moneyball
To support this statement he said: “The reason the A’s win, year after year, is because they have better pitchers than anyone else. the 2004 season is typical: the A’s were ninth out of fifteen teams in the American League in scoring runs, but had the second lowest ERA.”
Did Levitt forget about park adjustments? Who was right Michael Lewis or Levitt? – J. Cross
A. I talk to Billy Beane in the book. He says now that these issues are under the same larger umbrella philosophy: you want more information and you want to use it accurately. Scouts can be Moneyball-like in their thinking, too, particularly if they have the incentive to find the players who will produce in the Majors. The A’s never de-emphasized scouting, but they created a culture to properly evaluate statistics. Moneyball may have simplified the debate, but Beane sees the value in the different approaches.
Q. What’s your assessment of economics as a discipline, judged in terms of its ability to make politically useful predictions? For example, can economists predict with any reliability what the economic impact of a tax cut or a government spending program will be? –Dan Schroeder
A. The view of macroeconomic prediction in the book is pretty harsh. Economists have shown no real ability to predict a recession more than six months out. See the Wall Street Journal panel that predicted there would be no recession in December, 2007. It’s hard to measure the economy. Revisions can be as substantial as 5% in some quarters. Therefore, it is hard to predict and judge what the right policy is and what the implications of any policy are. So, we should be skeptical of anyone who predicts the impact of policy with a high degree of certainty. Humility is key.
Q. Is there any research on whether partisan political campaigning can prevent or delay an economic recovery? My theory is that “confidence” in the economy may have weakened as a direct consequence of the Republican primary season and heightened election-year politics in general. Republicans are very eager this year to tell us that the economy is bad under President Obama. Does that rhetoric actually cause the economy to be worse? Is economic growth harder to come by in election years in general? Maybe due to uncertainty? –James
A. Historically, election years are good years. 2008 mitigates against this, of course. There’s a hint in the data that Presidents push policies that will allow them to take a financial hit earlier in their term and then recover from it. Right now, the parties are reacting differently to the economy. Democrats are more confident, especially after the convention, and it affects their behavior. They may purchase more while Republicans are stocking up on Spam. There is no doubt that the debt ceiling debate scared investors, and our current political climate has been shown to be a risk to our economic strength.
Q. Why would a National League team, particularly one with financial constraints, not consider the following approach?
Sign no high priced starting pitchers. Fill your pitching roster with middle reliever/reliever pitchers with different attributes — junk ballers, power pitchers, sidearmers — with a lefty/righty balance. Balance your roster with righty and lefty position players.
In the course of a game, substitute (except for blowouts) for the pitcher when his place comes up in the order, and do so with a righty or lefty batter depending on the pitcher.
Advantages:
- You bring an American League offense to the National League.
- In key situations, you have a disproportionate number of righty/lefty pitching matchups.
- In key situations, you have a disproportionate number of righty/lefty fielding matchups.
- Rather than seeing the same pitcher four or five times in a row, an opposing batter may face batters of different styles and handedness (sic?) in a game.
- Keep payroll down by avoiding high priced starting pitchers.
- Arguable, but may have an injury benefit by avoiding 100 plus pitch outings to pitchers.
- Keeps the entire roster active
Disadvantages/Challenges:
- Will have trouble attracting pitchers focused on traditional counting stats and Ws.
- Will pose a pitching management challenge in rest and IP.
Naturally, in a blow out situation, you could simply have a young pitcher stretch out for 7 or 8 innings and save everyone else’s arm. And it is important to note that now, when there is an upside blowout with a team’s ace, those usual innings are pitched by him in search of the almighty W.
I ran this by Bill James on his website. His comment was that the only downside he saw was the part about attracting top starting pitchers. He also said that, in reality, baseball has been moving this way, albeit glacially, with the growth of middle relievers and closers and emphasis on righty/lefty situational matchups. Thoughts? –RGJ
A. Sure, but that is a huge organization-wide change. Now, you’re looking for a different kind of minor league pitcher. Do you trade prospects? Draft differently? It requires an across the board shift. We’re so caught up in performance metrics that we also forget baseball is basically an entertainment. What the fans like may not make statistical sense. But, you’d get an extra at bat for a pinch hitter, and you see that the same pitcher can have a lower ERA as a reliever than as a starter because he is working in shorter spurts. The payoff could be big, but the barriers to this kind of change are high.
Q. What’s your take on the public equity markets–are they efficient or nearly-efficient? – Dutch
A. This is the most discussed topic in the recent social sciences. In the book, we view markets as not totally efficient. Otherwise, there would be no incentive to make trades.
Q. It appears to me that prediction markets such as Intrade are continuing to grow, so I’m assuming that the insight that one can get from them is changing as more people use them. What’s your opinion of the usefulness of prediction markets and how has that changed over the past few years? How do you feel they will change moving forward? –Lenny
A. We like to see markets with larger capitalization because it is useful. You should be cautious of anyone who says they have a system to beat a market. On the other hand, there are times when Intrade’s model has been 10% off the other markets. So, there are perhaps some issues in the model. Are they efficient? Intrade versus a pundit like Dick Morris is no contest. I hope 538 is competitive with or better than Intrade. I’m a fan of information aggregators and financial motivation because it’s more likely to make you go back and do your own accuracy checks. Intrade has some very smart users and some hacks. The hacks will lose. It’s the opposite of television where you are not incentivized to accuracy but rather to entertainment.
Q. How about some about a fantasy baseball question? In a draft league, do you think it is best to use a good prediction model (Pecota) for example and let it stand so to speak. I mean of course, just line them up top to bottom and only use minor manipulations ( something like inside info on an injury). The question implies the use of intuition compared to the model. Daniel Kahneman refers to this in his book Thinking, Fast and Slow
A. I have not played as much fantasy baseball lately. But, if you limit yourself to 10-5 adjustments in a pool of 700 players, it could work. Focus on injuries and outliers. Average midcareer players will be treated right by the system. Be sparing with your exceptions. Use Pecota if you like it, but hedge with other systems. Get in an auction, as opposed to a draft. It takes more skill, but it’s worth it if you are statistically oriented.
Q. What industries do you see as next to revolutionized by data? I keep hearing that PR is particularly vulnerable, in the emperor-has-no-clothes sense. –Jeff Bladt
A. Public government. The Bloomberg administration is doing some very interesting data gathering. They have a giant database on urban life, and I hope it will result in more efficiencies. In the private sector, advertising is getting much more informed about audiences and product, so it is easier to micro target.
Comments