Microsoft has now responded, with a blog post and a letter, to my post about an experimental study that I coauthored with Yale Law School students Emad Atiq, Sheng Li, Michelle Lu, Christine Tsang, and Tom Maher. Our paper calls into question the validity of claims that people prefer Bing nearly two to one.
In response to several commenters: I do not work for and do not have any consulting relationship with Google.
Microsoft claims that our study is flawed because it relied on their own blind comparison website. They now say that "Bing It On" is meant to be a "lightweight way to challenge people’s assumptions about which search engine actually provides the best results." To be sure, companies often use fantastical or humorous scenarios for free advertising. However, Microsoft’s television commercials present the site as a credible way that people can learn whether they prefer Google or Bing. These commercials show people who discover that they really prefer Bing to Google. The challenge site that they created is either sufficient to provide insights into consumer preferences or it isn’t. The advertisements give the impression that the challenge site is a useful tool. Microsoft can’t have it both ways. If it is a sufficient tool to “challenge people’s assumptions,” then it is sufficient to provide some evidence about whether the assumed preference for Google is accurate.