What’s the Best Way to Predict American Idol?

I had a sobering moment a couple of years ago when I noticed that MSNBC’s recommendation engine predicted that I would enjoy stories about American Idol. It’s a guilty pleasure, but AI is one of the more normal things that I do with my 11-year-old daughter, Anna.

So my household is quite happy that (fellow Kansas City area native) David Cook won the other night. But Anna has been impatient each week to learn the results and has been logging into dialidol.com to get prior information on who is likely to be voted off the show.

For fans, DialIdol offers “free and safe software you can use to speed dial votes for your favorite contestants.” But what’s particularly cool is that while it is making these calls, it is also keeping track of the number of busy signals: Less popular contestants are less likely to have as many busy signals.

DialIdol predicts who will be voted off each week. They also let you know what the margin of error is in their prediction and give you nice graphics — with green, yellow, and red lights that indicate when a contest is too close to call — although it would be slightly cooler (as I’ve written with regard to political polls) if they gave actual probabilities of winning/losing.

Levitt wrote earlier about making money by betting on contestants using DialIdol. And their predictions have continued to be surprisingly accurate. This season their predictions have been right 97 percent of the time.

What’s more, DialIdol waived off Simon Cowell‘s assessment that David Archuleta was going to win the whole shebang.

After hearing Archuleta on the final night of performances, Simon said: “What we have witnessed is a knockout.” The morning before the winner was announced, Anna logged on and saw that DialIdol was predicting that David Cook was going to win by a large margin. Here’s what their prediction looked like:

Prediction, American Idol

And in fact, Cook did win by a large margin — 56 to 44 percent out of a whopping 97 million votes.

But there are many other ways to predict who would win. TiVo has experimented with using data from 20,000 random anonymous subscribers to find out when viewers fast-forward or re-watch particular contestants. Neat idea, but so far they have had a much lower success rate than DialIdol.

Instead of looking at phone and television behavior, you can also try to make predictions from Internet behavior. Apple might look at iTunes downloads of the contestant songs. Or you could use Google Trends to see which contestant name has been searched more often. David Cook rules by this method as well:


But somewhat surprisingly, Idol does not seem to be a context where the prediction markets have been very successful. It doesn’t seem like Intrade.com covered the show (why?). But realitymarkets.com showed David Archuleta to have a commanding lead even after DialIdol’s Cook prediction came in:

American Idol Prediction

The prediction markets have access to the public information and should tend to do at least as well as DialIdol. But there may still be room for people to make money like Levitt did a couple of years back, simply by betting on the DialIdol sure things.

Given its track record of success, it’s bizarre to me that the vast majority of newspaper articles continue to ignore these predictions as they go about assessing who is likely to win.

Brad Stewart

The problem with RealityMarket's prediction was the lack of trading volume in the contracts. This is also one of the biggest reasons as to why prediction markets fail in general. Regardless, I think there is still important information that can be learned from this market. While David Cook was in the lead for nearly the entire series, David Archuletta started making steady gains against him around the 5th of May. This graph shows it a little better: http://realitymarkets.com/index/144/American-Idol-7-Winner

Additionally it's important to remember prediction markets aren't guaranteeing someone will win if they have the highest price, merely stating a probability. The day before the final show David Cook had dropped down to 80% which means there was still a 20% chance he would lose.

Combine that with the fact that Cook's lead was continuously decreasing the last couple weeks and I think you can make an argument that there is still value in the information the market provides.

-Brad Stewart, Founder RealityMarkets.com



I would be interested to see a breakdown in the gender of people who actually vote for a winner. I would suspect 85% of voters are women, maybe higher. Does anyone disagree? I mean, teenage boys aren't going to vote. Middle aged men aren't going to vote. In fact, I up my prediction to 95% of voters are women.

Jim Bob

Um...Melinda DooLittle the best? How many albums has she sold to date? How many awards (MTV, Music, ETC.) has she won? What was her name again?


Don't forget Simon went on Jay Leno the night before the vote and told America that he was a David Cook fan. Then he praises Archuleta on the final sing-off. I think it was rigged. He wanted Cook fans to get mad enough and motivated enough to get out their cell phones and vote for the 'underdog'. As such Mr. Archuleta was cheated out of his win.


I just assumed the taller guy would win. Cook was significantly taller and I predicted he would win without even seeing him sing.

50-50 chance of being right or statistically proven bias?


I have to agree with #6 that Simon's comment most likely caused an increase in vote for Cook, i.e. when Cook fans heard it, they had more determination to send in repeat votes, while the Archuletta fans already felt an illusion of success and were less likely to vote repeatedly as many times.

I frequently wonder how different the results would be if people were only allowed to vote once. I also wonder if the judges have prior weeks voting data in mind when they make some of their comments. The AI votes would be a wonderful data set to get your hands on to do some analysis of what comments, appearances, other factors correlate with increased votes.

And Ian, Guilty Pleasure? No need to be ashamed, its just entertainment.

Snot Rag Dave

Garry S #8: I have to disagree with your last comment... "Plus, given the costs involved with using cell phones and text messaging, young teeny boppers might have had less opportunity to vote compared to older viewers."

The reverse is more likely... older viewers would be more conscious of the costs and would limit their texting, while the teens and 'tweens who may not pay for their cells (parents do) would text much more, not concerned about whether they are exceeding their monthly allotment. Another consideration: teens and 'tweens text much more than their parents.

I'd like to see a breakdown of the votes received via texting compared to phone-ins.


Facebook has a trend examiner as well, Lexicon, that looks at frequency of terms on wall posts over time. It shows the same trends for Cook vs. Archuleta



Consider that Idol manipulated the outcome by having the judges overpraise Archuletta, while Simon gave a negative opinion of David Cook to end the show. Mr Cowell had previously said that he liked David Cook more , and there were some indications toward the last weeks of the produces manipulating for Cook. Is it in the realm of possibility that Simon's comments were to pump up the sympathy/outrage vote for Cook? Quite possibly.


Perhaps one reason why Intrade did not carry this event is that some people (however few) would have access to non-public information about the winners & losers--such as producers of the show and the vote tabulation company. Would this fall under SEC rules against insider trading?

On the other hand, some people believe there is manipulation of the vote outcome by show producers. If DialIdol is correct 97% of the time, I would think there is not a high probability of such tinkering.


It can't be surprising that DialIdol is more accurate than anything else.

It's actually "sampling" the procedure that is used to pick the contestants. It's almost like if one person ran an exit poll on election day, and another person got to look at the ballots.

It's surprising how confident Simon was about Archuleta. Cook had been beating him in the voting for weeks. Does Simon have this information? Was there a compelling reason to think that Syesha voters were more likely to switch to DA instead of DC?

From the following link, you can use the drop-down menu to see previous weeks. It's interesting to guess where one contestants voters went the week after "their" guy got the boot.



I'm most interested by the failure of the prediction markets to get the right answer. My guess is that the prediction market participants over-thought the results - rather than betting on their gut feel, they tried to time the market. It's a strong datapoint for the fallibility of predictions markets; I can't think of a less important market where there were only two possible answers, and the prediction market was way off. How much worse would they be in a situation where the variables are much more complex and the stakes much higher?


The Tivo method is sure to be less accurate because it shows what viewers are interested in watching not the voting. The voting is skewed by people more likely to vote and especially obsessively vote. That means teenagers and probably even more so teenage girls.

So Melinda Doolittle, a good singer but without rock star charisma isn't going to get votes like David Cook. Daughtrey is another example who has real world rock star chops, as evidenced by sales but didn't draw in votes because the people who listen to hard rock simple aren't the same people who are going to sit and dial votes in over and over.

Snot Rag Dave

AaronS is on the right track. Doolittle was an exceptional talent... but may have been too shy to win enough votes in the end.

Another important aspect to watch is the 'cute' factor. I think Jason Castro got to the final 4 largely due to the 'tween girl vote... while David Archuleta most certainly gained a few grandmas for the same reason. Kristy Lee Cook certainly was favored in large part by her looks, as was Syesha Mercado.

Then again... Taylor Hicks' victory may blow my theory to chunks...


Idol is a popularity contest. People aren't voting for the best singer. They are voting for the person they like best, or more precidely, the person they want to win.

One reason Cook might have won is that Simon loved Archuleta. It was the peoples' way of "Sticking it to the Man."

David Damore

Interesting question.

Here is another method to measure popularity.
Twitter is a short message blog service [ http://twitter.com/home ]. There is a specialized search engine that will search all posts [ http://twitter.summize.com/ ] . By using browser tabs I was able to load a search for "David Cook" and "David Archuleta". Both pages were reloaded at the same time. As time progressed Summize indicated how many new results were posted since refresh. At one point Cook updates were being posted at nearly a 10 - 1 rate. Twitter users were remarking about him at a much higher rate.

This is not scientific by any means. Some errors surely occurred because Archuleta is easily misspelled. At the same time posters may have used initials such as "David C" and "David A" in their posts to Twitter.

What other ways can publicly available data be mined to make actionable and accurate predictions?

Garry S

I too was surprised when I checked dialidol.com the day of the finale and found that Cook was predicted to win by a large margin. Another thing that caused me to believe "Archie" would win is that whenever Idol showed clips of the two finalists' home towns, Archie seemed to have many more supporters than did Cook. Here is a theory: Archie appeals mainly to young teens (and girls), whereas Cook has a broader appeal. Perhaps it was a simple case of Cook winning more of the cohorts. Plus, given the costs involved with using cell phones and text messaging, young teeny boppers might have had less opportunity to vote compared to older viewers.


The Hollywood Stock Exchange also successfully predicted David Cook's win, with a share price higher that than of Archuleta's. It can be a pretty reliable prediction website.

Matt K

Intrade did have an American Idol market, however it was listed on its sister website tradesports.com.
You'll notice that after Simon's assessment that Archuleta would win, Cook's price fell to about 25, but by early morning as the market slowly incorpoarted the dialidol information cook's price rose from 25 to almost 75.


Admittedly, this is not scientific, but in my experience, it is to pick the VERY BEST SINGER...give them a 75% chance of getting to the Top Three...and after that, give them only a 33% chance of winning.

Typically, the best singers get to the Top 6 or so...and the preponderance of the best singers get to the Finals.

But after that, forget it.

Last year, Melinda Doolittle, the best singer EVER on American Idol--not a single bad performance--only made the Top Three.