Netflix $1 million prize

Netflix is offering a $1 million prize. This sounds like something that a Freakonomics blog reader should try to win:

Netflix is all about connecting people to the movies they love. To help customers find those movies, we’ve developed our world-class movie recommendation system: CinematchSM. Its job is to predict whether someone will enjoy a movie based on how much they liked or disliked other movies. We use those predictions to make personal movie recommendations based on each customer’s unique tastes. And while Cinematch is doing pretty well, it can always be made better.

Now there are a lot of interesting alternative approaches to how Cinematch works that we haven’t tried. Some are described in the literature, some aren’t. We’re curious whether any of these can beat Cinematch by making better predictions. Because, frankly, if there is a much better approach it could make a big difference to our customers and our business.

So, we thought we’d make a contest out of finding the answer. It’s “easy” really. We provide you with a lot of anonymous rating data, and a prediction accuracy bar that is 10% better than what Cinematch can do on the same training data set. (Accuracy is a measurement of how closely predicted ratings of movies match subsequent actual ratings.) If you develop a system that we judge most beats that bar on the qualifying test set we provide, you get serious money and the bragging rights. But (and you knew there would be a catch, right?) only if you share your method with us and describe to the world how you did it and why it works.

Serious money demands a serious bar. We suspect the 10% improvement is pretty tough, but we also think there is a good chance it can be achieved. It may take months; it might take years. So to keep things interesting, in addition to the Grand Prize, we’re also offering a $50,000 Progress Prize each year the contest runs. It goes to the team whose system we judge shows the most improvement over the previous year’s best accuracy bar on the same qualifying test set. No improvement, no prize. And like the Grand Prize, to win you’ll need to share your method with us and describe it for the world.

I love the Netflix approach to the problem. They could easily spend $1 million internally hiring some programmers or Ph.D’s to try to improve their algorithm, with uncertain results. Instead, by making it a contest and offering up data to outsiders, they will probably succeed in having 100 times as many person-hours devoted to the problem for the same price — or cheaper because they only pay out the million if someone really improves on what they are doing now. In addition they gets lots of free publicity. Truly a brilliant strategy.

TAGS:

Leave A Comment

Comments are moderated and generally will be posted if they are on-topic and not abusive.

 

COMMENTS: 43

View All Comments »
  1. Crosbie says:

    Watch out for the ’42′ syndrome though.

    I note they aren’t also challenging programmers to demonstrate the correctness or wisdom of the problem’s specification.

    They may well get a solution to the problem they’ve stated, but they may not realise that they’ve failed to state the problem they actually have.

    After they’ve paid out $1m, they may discover they need to propose a far trickier challenge, e.g. “$10m to whoever tells us how our challenge should have best been specified given our actual problems and requirements”.
    ;-)

    Thumb up 0 Thumb down 0
  2. Crosbie says:

    Watch out for the ’42′ syndrome though.

    I note they aren’t also challenging programmers to demonstrate the correctness or wisdom of the problem’s specification.

    They may well get a solution to the problem they’ve stated, but they may not realise that they’ve failed to state the problem they actually have.

    After they’ve paid out $1m, they may discover they need to propose a far trickier challenge, e.g. “$10m to whoever tells us how our challenge should have best been specified given our actual problems and requirements”.
    ;-)

    Thumb up 0 Thumb down 0
  3. Craig says:

    Too bad netflix is a scam company and they’re going out of business soon.

    Thumb up 0 Thumb down 0
  4. Craig says:

    Too bad netflix is a scam company and they’re going out of business soon.

    Thumb up 0 Thumb down 0
  5. 711buddha says:

    The idea is brilliant. While they will undoubtedly get more hours of work this way, much of it will be redundant. The real question is, how much original work will they get.

    Thumb up 0 Thumb down 0
  6. 711buddha says:

    The idea is brilliant. While they will undoubtedly get more hours of work this way, much of it will be redundant. The real question is, how much original work will they get.

    Thumb up 0 Thumb down 0
  7. benmiller says:

    1 percentage point up; nine to go.
    http://www.netflixprize.com/leaderboard
    Not bad for a week.

    Thumb up 0 Thumb down 0
  8. benmiller says:

    1 percentage point up; nine to go.
    http://www.netflixprize.com/leaderboard
    Not bad for a week.

    Thumb up 0 Thumb down 0