What A Night! Interpreting the New Hampshire Primary

A few observations about last night’s surprise result in the New Hampshire primary. Let me draw on the thoughts outlined in my column for the Wall Street Journal.

First, red faces all around for the political prognosticators:

Judging by the pre-vote polls and prediction markets, the Democratic primary in New Hampshire created one of the most surprising upsets in U.S. political history. Illinois Sen. Barack Obama was favored in the final pre-election poll of all 12 pollsters who surveyed voters since his surprise victory in Iowa, and was the unanimous favorite among television pundits. The only real question to be resolved appeared to be the size of Mr. Obama’s majority. His loss to New York Sen. Hillary Clinton was equally embarrassing for prediction markets.

How surprising is this result?

Historical comparisons are already being drawn between the New Hampshire primary and the famous 1948 presidential race in which President Harry S. Truman beat Republican challenger Thomas Dewey … Yet the magnitude of the Clinton surprise is arguably even greater. Indeed, historical research by Professors Paul Rhode of the University of Arizona and Koleman Strumpf of Kansas University has shown that in the Truman-Dewey race, prediction markets had seen hope for President Truman despite his dreadful polling numbers, and he was rated an 11% chance of winning the election by election-eve. Thus, Sen. Clinton’s victory on Tuesday was more surprising than President Truman’s in 1948.

And finally, this is a chance to remind ourselves just what a prediction market price means:

While Sen. Clinton’s unexpected victory has yielded red faces among the punditocracy, this also provides a useful opportunity for emphasizing just what a prediction market forecast says. That the price of a contract paying $1 if Sen. Clinton won in New Hampshire was selling for seven cents doesn’t suggest that she was a sure loser. Rather, these prices suggest a probabilistic statement that the ultimate outcome was about a 7% chance. And as any horseplayer can tell you, sometimes the long shots do win.

And let me emphasize, this isn’t a “get-out-of-jail-free” card for my own prognostications. One of my current research projects (with Andrew Leigh and Eric Zitzewitz) documents evidence of a favorite-longshot bias in political prediction markets:

We were led to this research by an age-old racetrack puzzle economists call the “favorite-longshot bias”: horse bettors historically have overbet long shots, and they win less often than their odds suggest. Our research suggests that similar biases hold in political prediction markets.

Against this baseline, it is even more surprising that Ms. Clinton was the winner.

Read the full column here. And here’s Paul Krugman:

There’s no hint that the market saw either Iowa or New Hampshire coming, or knew anything beyond the bloviations of the talking heads.

Perhaps the clearest explanation lies in the title of Krugman’s post: “Nobody knows anything.” If true, it wouldn’t be surprising that any information aggregation mechanism did poorly.

It is also interesting to see the hand-wringing among the pollsters. Gary Langer, a former Freakonomics guest blogger (and ABC’s Poobah of Polling) has a very honest, straightforward piece that I liked. And it is worth revisiting this earlier post by Dubner (quoting Langer) about inaccurate polling and minority candidates.

Finally, a fun aside. I didn’t bet a dime on last night’s race, but at this fantasy prediction market I put my whole portfolio into Clinton to win New Hampshire. If you click through to the leaderboard this morning you’ll see I’m finally winning something! (But click soon — my lead won’t last!)

What are the lessons for pundits, polls, prediction markets, and other prognosticators?