zbicyclist

$1,000,000 price
743 teams
Expected value: $1346 per team assuming a winner.

Bragging right, though, are worth a lot.

Serene

It's a good idea.

freakophysics

It's an even better deal than you think. Netflix is offering a $10,000 progress prize for the best improvement after one year. Progress was quick initially, but the top teams have stalled at about a 5% improvement. See http://www.dicarlolaw.com/hist_20061105.png for a graph someone made charting the progress.

My guess is that the current crop of leaders are all using the same basic machine learning algorithm. They have optimized it, and there is not much room for improvement left. This approach won't get to a 10% improvement.

This suggests the interesting question: what is the ultimate limit in predicting human behavior. Clearly humans are fickle, and at some level random. If you rated Mission Impossible 3 as a "4" today, what would you rate it next month?

HerrKevin

Freakophysics: The progress prize is actually $50,000.

This competition isn't just great for Netflix. For many computer science students out there like myself who are interested in applications of AI, such as machine learning and data mining, this offers an interesting challenge (and of course a chance at money). Netflix offered lots of great data that students like myself are playing with. I doubt I'll win, but by the end of this competition I'll have learned quite a bit about data mining.

sophistry

They really should change the prediction sample data at periodic intervals or have more of them. At some point people are just data mining on the prediction set, albeit very slowly since they need to wait for Netflix to give them back the score.

ivol

This is a complete waste of time for the data analysts trying to win the prize. I doubt that there is enough structure in the Netflix dataset to achieve that accuracy required for the $1 million prize. The contestants are essentially trying to perform cluster analysis in a high-dimensional space with extremely sparse data and no geometric structure. The currently person in 1st place is an assistant professor named Yi Zhang (founder of WXYZConsulting.com) whose whole life's research work is centered around the problem of "Collaborative Filtering" (which is precisely the problem that Netflix is attempting to solve with this $1 million prize). Yi Zhang, with a PhD thesis in predicting movie ratings for websites and access to the massive research supercomputers at her university, is only able to attain a 5.77% improvement over Netflix. 5.77% is a far cry from the 10% needed to win the $1 million.

It is highly unlikely that anyone will ever reach the 10%, because there is a hard physical limit to the amount of structure in the dataset. This means that it is unlikely that Netflix will ever pay out $1 million to anyone (and I'm sure they knew that from day one). What is far more likely is that someone like Yi Zhang will recieve $50,000 in exchange for writing a performance optimized algorithm that will improve Netflix's system by somewhere between 6 and 8%. $50,000 is a far cry from $1 million, and is hardly worth anyone's time (except for some academic who happaned to write a PhD thesis in this exact same research area and has a lot of 'research' time on his/her hands).

To all the people out there who took data mining / machine learning courses as undergraduates and now see a chance to win the $1 million: don't waste your time and energy, unless you have absolutely nothing else to do.

Read more...

Andrew

Don't be so sure ivol, the leaders have an 8.7% improvement at the moment, and there are several teams close by

http://www.netflixprize.com/leaderboard

zbicyclist

$1,000,000 price
743 teams
Expected value: $1346 per team assuming a winner.

Bragging right, though, are worth a lot.

Serene

It's a good idea.

freakophysics

It's an even better deal than you think. Netflix is offering a $10,000 progress prize for the best improvement after one year. Progress was quick initially, but the top teams have stalled at about a 5% improvement. See http://www.dicarlolaw.com/hist_20061105.png for a graph someone made charting the progress.

My guess is that the current crop of leaders are all using the same basic machine learning algorithm. They have optimized it, and there is not much room for improvement left. This approach won't get to a 10% improvement.

This suggests the interesting question: what is the ultimate limit in predicting human behavior. Clearly humans are fickle, and at some level random. If you rated Mission Impossible 3 as a "4" today, what would you rate it next month?

HerrKevin

Freakophysics: The progress prize is actually $50,000.

This competition isn't just great for Netflix. For many computer science students out there like myself who are interested in applications of AI, such as machine learning and data mining, this offers an interesting challenge (and of course a chance at money). Netflix offered lots of great data that students like myself are playing with. I doubt I'll win, but by the end of this competition I'll have learned quite a bit about data mining.

sophistry

They really should change the prediction sample data at periodic intervals or have more of them. At some point people are just data mining on the prediction set, albeit very slowly since they need to wait for Netflix to give them back the score.

ivol

This is a complete waste of time for the data analysts trying to win the prize. I doubt that there is enough structure in the Netflix dataset to achieve that accuracy required for the $1 million prize. The contestants are essentially trying to perform cluster analysis in a high-dimensional space with extremely sparse data and no geometric structure. The currently person in 1st place is an assistant professor named Yi Zhang (founder of WXYZConsulting.com) whose whole life's research work is centered around the problem of "Collaborative Filtering" (which is precisely the problem that Netflix is attempting to solve with this $1 million prize). Yi Zhang, with a PhD thesis in predicting movie ratings for websites and access to the massive research supercomputers at her university, is only able to attain a 5.77% improvement over Netflix. 5.77% is a far cry from the 10% needed to win the $1 million.

It is highly unlikely that anyone will ever reach the 10%, because there is a hard physical limit to the amount of structure in the dataset. This means that it is unlikely that Netflix will ever pay out $1 million to anyone (and I'm sure they knew that from day one). What is far more likely is that someone like Yi Zhang will recieve $50,000 in exchange for writing a performance optimized algorithm that will improve Netflix's system by somewhere between 6 and 8%. $50,000 is a far cry from $1 million, and is hardly worth anyone's time (except for some academic who happaned to write a PhD thesis in this exact same research area and has a lot of 'research' time on his/her hands).

To all the people out there who took data mining / machine learning courses as undergraduates and now see a chance to win the $1 million: don't waste your time and energy, unless you have absolutely nothing else to do.

Read more...

Andrew

Don't be so sure ivol, the leaders have an 8.7% improvement at the moment, and there are several teams close by

http://www.netflixprize.com/leaderboard