What A Night! Interpreting the New Hampshire Primary

A few observations about last night’s surprise result in the New Hampshire primary. Let me draw on the thoughts outlined in my column for the Wall Street Journal.

First, red faces all around for the political prognosticators:

Judging by the pre-vote polls and prediction markets, the Democratic primary in New Hampshire created one of the most surprising upsets in U.S. political history. Illinois Sen. Barack Obama was favored in the final pre-election poll of all 12 pollsters who surveyed voters since his surprise victory in Iowa, and was the unanimous favorite among television pundits. The only real question to be resolved appeared to be the size of Mr. Obama’s majority. His loss to New York Sen. Hillary Clinton was equally embarrassing for prediction markets.

How surprising is this result?

Historical comparisons are already being drawn between the New Hampshire primary and the famous 1948 presidential race in which President Harry S. Truman beat Republican challenger Thomas Dewey … Yet the magnitude of the Clinton surprise is arguably even greater. Indeed, historical research by Professors Paul Rhode of the University of Arizona and Koleman Strumpf of Kansas University has shown that in the Truman-Dewey race, prediction markets had seen hope for President Truman despite his dreadful polling numbers, and he was rated an 11% chance of winning the election by election-eve. Thus, Sen. Clinton’s victory on Tuesday was more surprising than President Truman’s in 1948.

And finally, this is a chance to remind ourselves just what a prediction market price means:

While Sen. Clinton’s unexpected victory has yielded red faces among the punditocracy, this also provides a useful opportunity for emphasizing just what a prediction market forecast says. That the price of a contract paying $1 if Sen. Clinton won in New Hampshire was selling for seven cents doesn’t suggest that she was a sure loser. Rather, these prices suggest a probabilistic statement that the ultimate outcome was about a 7% chance. And as any horseplayer can tell you, sometimes the long shots do win.

And let me emphasize, this isn’t a “get-out-of-jail-free” card for my own prognostications. One of my current research projects (with Andrew Leigh and Eric Zitzewitz) documents evidence of a favorite-longshot bias in political prediction markets:

We were led to this research by an age-old racetrack puzzle economists call the “favorite-longshot bias”: horse bettors historically have overbet long shots, and they win less often than their odds suggest. Our research suggests that similar biases hold in political prediction markets.

Against this baseline, it is even more surprising that Ms. Clinton was the winner.

Read the full column here. And here’s Paul Krugman:

There’s no hint that the market saw either Iowa or New Hampshire coming, or knew anything beyond the bloviations of the talking heads.

Perhaps the clearest explanation lies in the title of Krugman’s post: “Nobody knows anything.” If true, it wouldn’t be surprising that any information aggregation mechanism did poorly.

It is also interesting to see the hand-wringing among the pollsters. Gary Langer, a former Freakonomics guest blogger (and ABC’s Poobah of Polling) has a very honest, straightforward piece that I liked. And it is worth revisiting this earlier post by Dubner (quoting Langer) about inaccurate polling and minority candidates.

Finally, a fun aside. I didn’t bet a dime on last night’s race, but at this fantasy prediction market I put my whole portfolio into Clinton to win New Hampshire. If you click through to the leaderboard this morning you’ll see I’m finally winning something! (But click soon — my lead won’t last!)

What are the lessons for pundits, polls, prediction markets, and other prognosticators?


FRANCIS HARTIGAN

I wonder if pollsters asked who was more likely to win in the general election. If not, this may be a factor in why they failed to accurately predict the primary results.

Jim

Seems that the prediction market was driven by polling data indicating a significant lead for Obama. A reasonable person would assume that the (supposed) double digit lead in the polls was outside of the margin of error and dump their stock. I can't see this market as having its own predictive power, since its driven so strongly by a small stream of information that proved to be unreliable.

PEZ

I'm just an ignorant Swede. But isn't this the first time a Woman runs for president in the United States? If so I think that some women, who formally haven't felt enough reason to vote, might now actually bother. Since women represent about half the population my guess is that this effect might be quite powerful and may very well make the difference.

discordian

@ 1

Hillary is not running for president at this time.
She is running for her party's nomination.
Women have run for party nominations in past primaries (Shirley Chisolm and Margaret Chase Smith to name 2).

Eric Goebelbecker

Oh my gawd! We still have to actually hold the elections in order to determine the outcome! What a shocker!

How does a contract that pays $1 if the pundits and pollsters get February 5 wrong cost?

If it's anything less than $1, take it!

Nick M.

Obama is the first African-American to have a fighting chance, so I would expect that to bring out more African-Americans the same way Clinton might bring out more women.
Though I think there are more white women in NH than African-Americans, so I guess that would favor Clinton.

Melissa

@6
I think one of the reasons that Romney's campaign is perceived as going south is the amount of money the campaign spent on ads targeted to Iowa and New Hampshire and then didn't win either state.

Marie

Rich and PEZ, you are not correct.

The first woman to run for President was actually Victoria Woodhull, in 1872. The woman presidential candidate to garner the most votes as Lenora Fulani (217,219 votes) in 1988. If Hillary wins the nomination, she'll be the 24th woman to run for president, but will be the first to head a major party.

LL

Gosh, I hope that Iran is as acquiescent to a crying woman as New Hampshire is.

H Tran

A line from 30 Rock's 'third wave feminist' went something like this:

"I'm going to tell everyone I'm voting for Barak Obama but secretly I'm voting for John McCain"

Marcia

Hillary was always meant to win in NH. She should carry New England - that's always been her turf.

David

Didn't these early primary/caucus states get stripped of all (DNC) respectively half (RNC) of their delegates, as punishment for holding their elections earlier than permitted? I remember reading this and never heard that those penalties were lifted, yet the "scorecards" count them with their full number of delegates.

apathetic

Don't people simply skip voting if they think their candidate is going to win? So being massively favored sets you up to lose big.

MJ

A couple of young men stood out side one of the venues Mrs. Clinton spoke at and yelled, "Iron my shirt, iron my shirt." I think that had much more effect then the media knows. It reminded women we are all second class, and every report on this reminds women of the glass ceiling. the men are Mr. Obama, Mr. Edwards, Mr. Romney, and the lone woman candidate is Hillary. She is belittled from all sides. And a bunch of pissed women stood up and said no, we will not iron your shirts.

PEZ

Marie, thanks for the information. But maybe it is safe to say that Hillary Clinton is the first woman to stand a good chance? The figures I have seen show that the polls where quite correct on all but the Clinton votes. My point is that lots of rational people do not see a good reason to bother voting. And now maybe some of these rational females suddenly through that rationality overboard for the prospect of seeing a woman on the post? And this might be something that you decide quite late and/or something you don't tell the pollsters.

S. Heaton

Yeah, I couldn't believe how pathetically the prediction markets performed. But some great comments by PEZ @1 and few others here really nail it. This is an election year with candidates like no other. Combine that with the increasingly difficulty to poll, and there is simply no easy way to predict what will happen with any reliability.

Prediction markets basically hold all of our collective beliefs weighted by our bets. All this is saying is that we have no way of predicting how undecided people will vote will decide at the last second without a very sophisticated psychological model.

Mike

How soon we forget. It's not the Clinton surprise, but the Obama surprise. Clinton was leading all along, it's only been a week since Obama became a real contender.

In Iowa Obama picked up many second choices, Clinton hardly any. That is why Obama won in Iowa the way he did, and that is why the pollsters and probably the pollees in NH got little crazy. It's all down to the way the Dem. caucus works in Iowa.

ShirKi

Hukamoose @12, this phenomena is actually referred to in the Freakonomics book. Black candidates tend to do better in polls than in reality when the vote is private.
I think this relates to what Mike @24 is saying, since Obama wasn't expected to take Iowa and was a "long-shot" candidate until not too long ago.
In general this is one of the most interesting primaries I've ever seen, and although I am not an American citizen I am following closely. I am not even sure who I would prefer to win, and which one (Clinton or Obama) truly would have the better chance winning the presidential elections.
Definitely an interesting election year.

Jonathan

If the favored outcome in prediction markets always came true, it would be strong evidence against the efficiency of the prediction markets. A 93% favorite should only win 93% of the time. Hillary Clinton had about a 1/13 chance of winning the NH primary. There is no reason for the prediction markets to be embarassed that a 1/13 event happened. Anybody who's experienced a 'bad beat' on the river knows that 1/13 events happen often.

Rich Wilson

PEZ- you are correct. The closest we've had was Geraldine Ferraro who ran for VP with Walter Mondale, and was defeated by Reagan/Bush in '84.