When you take a sip of Cabernet, what are you tasting? The grape? The tannins? The oak barrel? Or the price?
Believe it or not, the most dominant flavor may be the dollars. Thanks to the work of some intrepid and wine-obsessed economists (yes, there is an American Association of Wine Economists), we are starting to gain a new understanding of the relationship between wine, critics and consumers.
One of these researchers is Robin Goldstein, whose paper detailing more than 6,000 blind tastings reaches the conclusion that “individuals who are unaware of the price do not derive more enjoyment from more expensive wine.”
So why do we pay so much attention to critics and connoisseurs who tell us otherwise?
That’s the question we set out to answer in this podcast. Along the way, you’ll hear details about Goldstein’s research as well as the story of how his “restaurant” in Milan, Osteria L’Intrepido, won an Award of Excellence from Wine Spectator magazine. (Not how you think!)
Also featured: Steve Levitt, who admits his palate is “underdeveloped,” describing a wine-tasting stunt he pulled on his elders at Harvard’s Society of Fellows.
Also, you’ll hear from wine broker Brian DiMarco (featured in the forthcoming documentary Escaping Robert Parker) who pulled a stunt of his own on his very wine-savvy employees. DiMarco also walks us through the mechanics of the wine-purchase business, and describes how price is often a far-too-powerful signal to our taste buds.
A couple of very interesting interviews didn’t make the podcast but are worth a mention here. One was with the noted Princeton economist (and wine buff) Orley Ashenfelter*, who spoke about our general overreliance on experts, whether they’re in the wine field or far beyond:
I mean, S&P, Moody’s, Fitch, these people all rated securities that apparently completely tanked. So there’s obviously something in the demand for expertise, the imprimatur, which is not really about the fact that they do a good job. By the way, those organizations are not transparent either, just as the Wine Spectator isn’t. So there’s some similarity here that I think probably gives us a little insight into things that are much broader than wine and food.
The other interview was with George Taber, author of the fascinating book Judgment of Paris: California vs. France and the Historic 1976 Paris Tasting That Revolutionized Wine. He recalled the moment he realized that even the most sophisticated wine experts can have feet of clay:
And there was just one classic moment when one of the French judges by the name of Raymond Oliver, who was the owner of the Le Grand Vefour restaurant, he had a television show on food in France, he was a big thing in French wine and food circles. He had a white wine in front of him. He looked at the white wine, then he held it up to a light to look at the color very closely. Then he took a sip of it. Then he held it up again. Then he said in French, ‘Ah, back to France.’ And I looked down at my scorecard and he’d just tasted the 1972 Freemark Abbey Chardonnay.
Wishing you the happiest of holiday seasons, and urging you to spend $15 instead of $50 on your next bottle of wine. Go ahead, take the money you save and blow it on the lottery.
* You can hear Ashenfelter in a related Marketplace piece that aired recently.
Stephen J. DUBNER: So Levitt, um, a college friend of yours once told me that your favorite meal during college was a dill pickle, beef jerky and grape soda. Is that true?
Steven D. LEVITT: I did indeed have that for breakfast, but to tell you the truth, it sounded better before I ate it than after.
DUBNER: Steve Levitt, my Freakonomics friend and co-author, an esteemed economist at the University of Chicago, has an extremely refined palate.
DUBNER: All right, so you’ve got beef jerky, you’ve got dill pickle, what are your favorite foods? Like, what are your favorite places...like if you could drive across America and pick any place to stop and eat, what’s it going to be?
LEVITT: You know, I love the Billy Goat tavern. It’s the cheap place that was made famous in the 19-probably-70s on Saturday Night Live, with the ‘cheeseburger- cheeseburger, no Pepsi, Coke.’ Anyway, they have an incredible rib-eye steak sandwich. Pretty much the cheaper the food the better, there’s almost no fast food that I don’t adore.
DUBNER: So KFC?
LEVITT: Yeah, I like KFC. Burgers. Chipotle. I’d kill for Chipotle.
DUBNER: So how would you describe your palate?
LEVITT: Probably underdeveloped. You know, but it’s good. The thing is, it’s a wonderful, wonderful gift to like cheap food. I mean, some people just happen to like expensive food and then they are unhappy most of the time, or else, they spend all their money on food. But if you just, by chance are born loving cheap food, then you can eat everything that you love.
DUBNER: Now, how much do you like wine?
LEVITT: Wine I do not like at all.
ANNOUNCER: From American Public Media and WNYC, this is Freakonomics Radio. Today, why wine experts should just put a cork in it, and why having an untrained palate can save you lots of dough. Here’s your host, Stephen Dubner.
DUBNER: Good wine, we’re told, is the province of smart, superior people. They taste things on a whole ‘nother level than people like you and me.
[WINE TASTING] I taste a lot of minerality...Freshly cut apple...George Clinton...This funky sort of...
DUBNER: But I have a question for you: is that superiority deserved? The wines these people love -- the ones that bring out the natural beauty of the grape but aren’t too funky -- they cost a lot more. But are expensive wines really that much better than cheap wines? Or, is it possible that developing your palate just mucks things up? Complicates things? Maybe we’d all be better off if we had taste buds like Steve Levitt’s.
After graduate school, Levitt was invited to join an elite club at Harvard called the Society of Fellows. A junior fellow like him was paid a modest salary to work on his own research, with few obligations -- other than a formal Monday night dinner with the senior fellows, who were some of the most remarkable scholars alive.
LEVITT: People like Amartya Sen and, Nobel Prize winning physicists and what not, and you sit around a table. I believe the table was Oliver Wendell Holmes table, initially, and he gifted it to the Society of Fellows.
DUBNER: Over dinner, they engaged in witty, learned conversation; they ate venison and other fine foods; and they drank expensive wine -- bottle after bottle. To this budding young economist, with the beef jerky taste and the grape-soda budget, all that pricey wine wasn’t doing him any good. He had a thought.
LEVITT: Innocently, I was young, I didn’t understand how the world worked, I thought like an economist. I suggested that perhaps we should have two tracks at the Society of Fellows; there would be the drinking track and the abstinence track. And for those of us who chose the abstinence track, because the cost of the wine was perhaps $60 per meal, that over the course of 50 weeks of the year, that would work out to be about $3,000 and they could add $3,000 to the paycheck of those of us on the abstinence path.
DUBNER: And did you have any other people in your abstinence camp, or was this just Levitt?
LEVITT: Well, you know it didn’t really get that far because the reaction was quite negative.
DUBNER: Levitt is the kind of person who likes to use data, not a personal agenda, to make his arguments. So he set out to get some data. Wine data. The wine they were drinking at these dinners cost five or 10 times what a cheap bottle of wine cost. Was it really five or 10 times better? He hatched a plan. The Society of Fellows held wine tastings from time to time. He suggested that the next one be his to organize.
LEVITT: So I worked with the wine steward to select two excellent bottles of wine, expensive bottles of wine, you know, probably close to $100 bottles of wine. And then I went to the liquor store that was down the street and I said, can I have the cheapest bottle of wine you have that was the same grape, I don’t remember which grape it was.
DUBNER: Levitt used four decanters. Into the first decanter, he poured one of the expensive bottles of wine. The other expensive wine went into decanter number 2. In the third decanter, he poured the cheap wine, which cost around $8. In the fourth decanter, he repeated one of the expensive wines.
LEVITT: So as far as the people knew, there were four different wines and these were all wines that were coming out of the wine cellar of the Society of Fellows.
DUBNER: So you are tricking them from the outset. You’re leading them to believe the fourth, the cheap wine, is also from the wine cellar.
LEVITT: I don’t....you know. Uh, yes.
DUBNER: They swirled them a bit … sniffed them … and sipped. They wrote down their ratings. As he looked at the numbers, Levitt’s cold economist heart warmed.
LEVITT: The data could not have cooperated more completely with my hypothesis. So for starters the four wines received almost identical ratings on average. Although there were a wide spread among individuals, on average, tallied up, people did not prefer the expensive wines to the cheap wine. On top of that, and this was the thing I was hoping for and dreaming of but didn’t believe would actually come true: It turned out among individuals if you compared how differently they rated any two of the wines that they had, it turned out that by a small margin, people actually rated the same wine from the same bottle but presented in a different decanter as being the most different among the two wines (so the two wines that were absolutely identical, when you looked at the gap between the ratings that the individual gave to those wines, the gap was bigger than they did between the other wines, which actually were different.
DUBNER: A few minutes after the tasting was over, Steve Levitt shared the results with the senior fellows.
LEVITT: The jovial mood in the room suddenly went dark. People realized they had been tricked. That there had been this cheap wine the same wine was in twice. And they really realized that the nature of the game had been somewhat different than what they thought. And when they heard the results, that collectively they had no ability to identify wines, they were not happy. And particularly there was one senior fellow, so one of the professors at Harvard, who was quite outspoken about his knowledge of wine. And he loudly announced that he had a cold. Otherwise he clearly could have made the distinctions and he stormed from the room and left the party prematurely.
DUBNER: What was his discipline?
LEVITT: I think for the sake of anonymity, I should not reveal that particular piece of information. He was a humanist.
DUBNER: Not an economist, in other words.
LEVITT: No. No.
DUBNER: The opposite of an economist.
DUBNER: So what does Levitt’s evil little experiment teach us about wine? Maybe not all that much; it wasn’t a very scientific tasting, really. And perhaps the Society of Fellows was having an off night -- the humanist had a cold, right? Or maybe this was just a group of people who didn’t know nearly as much about wine as they thought they knew. You’d never be able to pull this kind of stunt on wine experts, would you?
Brian DIMARCO: On my team we have a master sommelier, two master of wine candidates, four people who have been in the trade for many years. These are sophisticated wine professionals.
DUBNER: That’s Brian DiMarco, talking about the people he gathered for a blind tasting of his own. Before we get to that, let’s hear a bit more about Brian.
DIMARCO: I specialize in helping customers, consumers, private collectors, and retailers, and restaurants decide what wines to put on their list, what wines to collect, what wines to sell. And I have a small import business and wholesale company. And we distribute wine that we find all over Europe and South America in New York City.
DUBNER: Cool, so you’re almost a wine agent then, yeah, in a way? More than just an importer?
DIMARCO: In many ways, yes.
DUBNER: Brian DiMarco is one of the people who decide what we drink. He goes to France, Italy, California, tastes wine fresh out of the tank. So he determines what’s good -- or bad -- without any critics whispering sweet ratings into his ear. Then he puts his money where his taste buds are -- he writes a check. Now, according to DiMarco, he makes as much money selling a $15 bottle of wine to a restaurant or shop as he does selling a $50 bottle. If that’s true, DiMarco is an honest broker. His job is simply to find wine that you or I would want to drink -- because, think about it, there are thousands upon thousands of bottles to choose from. Just picture the rows of bottles lining the shelves at every wine shop. You kinda, sorta, maybe think you want to buy a merlot. Do you pick the one with the pretty flowers on the label? Do you go with the one that Robert Parker, the high priest of wine ratings, awarded a lot of his Parker points to? Or, like a lot of people, do you let price be your guide? If you believe even a little bit in the free market, you’d have to think that expensive wines cost more because they taste better, right? Brian DiMarco wanted to know how much people were tasting the dollars when they drank an expensive wine. So, like Steve Levitt, he conducted a little experiment, with some of the people who work for him. As DiMarco said, these were no amateurs.
DIMARCO: We did a tasting, brown bag. And we had the exact same wine in both bags. And we told them that one bottle was a $50 bottle to write their reviews, and we told them the other was a $10 bottle, to write the reviews. Of course they were both $20 bottles according to what they would sell at any retail store. And then we reversed it and said now the $10 bottle is really the $50. And everyone liked the $50 bottle better in both circumstances, because they perceived that the price there was either something they were missing, when really these wines are so similar. Yeah but they’re different. A lot of people, two or three people said, is this the same wine? And we said that’s for you to determine, and the more they thought about it, the more they intellectualized it, the more they decided there were differences to the wine.
DUBNER: So this wasn’t a wildly scientific experiment either, but to Brian DiMarco, the message couldn’t have been clearer: When people know a wine is more expensive -- or even think it is -- it tastes better. Now, obviously this idea doesn’t apply just to wine. A house that costs $500,000 ought to be five times better on some level than a $100,000 house -- the size, the construction, the schools and neighborhood. So is a wine that costs $50 five times better than a $10 bottle? Or is it even better at all?
Coming up, We go looking for hardcore, empirical evidence that expensive wines actually better than cheap wines. And we look beyond price to prestige -- to wine ratings, the awards, the whole shebang. Because if your wine -- or your whole wine list -- wins an award from a magazine like Wine Spectator, it’s got to be good, right? Right?
ANNOUNCER: Swirl, sip and spit. We’re back with Freakonomics Radio from American Public Media and WNYC. Here’s your host, Stephen Dubner.
DUBNER: So tell me your name kind of what you do, and how you describe yourself.
Robin GOLDSTEIN: My name is Robin Goldstein. I write about wine and food. Basically my book “The Wine Trials” has been my principal outlet for writing about wine, but I’ve also been publishing academic papers on topics of taste from a cognitive perspective, and an economic perspective, usually coauthored with a colleague from different academic fields. So I’ve been exploring the neuroscience side of it a bit. I’ve been exploring the behavioral side of it. And in particular I’m interested in price signals and how people’s knowledge of price affects their experience of wine on the most basic sensory level.
DUBNER: Now, first let me just ask you, I’ve heard good things about this lovely little restaurant in Milan called Osteria L’Intrepido, You ever been there?
GOLDSTEIN: I’ve been to the restaurant, but it’s actually located in my friend Giuliano’s former apartment in Milan. The restaurant, I wouldn’t say it’s great. Mostly they serve left over pizza, and their wine cellar consists mostly of some leftover bottles of Montepulciano Bruta from three weeks ago.
DUBNER: Meaning it’s not really a restaurant, is it?
GOLDSTEIN: It’s an apartment. So it’s just an address really.
DUBNER: Osteria L’Intrepido doesn’t exist; it’s a fake restaurant that Robin Goldstein made up. Why? Well, it’s a strange story. Goldstein’s research and writing on wine made him skeptical about critics and awards. He believed that so-called experts were, at best, subjective, and that they carried too much influence. He wondered about the awards that magazines like Wine Spectator gave to restaurants for their wine lists. Did an award like that really mean the wines at that restaurant were excellent? So he invented Osteria L’Intrepido, or “Fearless Restaurant.” He created a fake menu, a fake website, and a fake voicemail message, saying the restaurant was closed for vacation…
[IN ITALIAN: OUTGOING VOICE MESSAGE]
DUBNER: As for the Osteria L’Intrepido wine list -- Goldstein made that up too. He included several expensive wines that Wine Spectator itself had given bad reviews in the past. One of them was a 1982 Brunello di Montalcino, which the magazine had given 67 points, or a D+ rating, calling it "barnyardy” and “decayed." He listed another vintage that Wine Spectator had reviewed as “Unacceptable … sweet and cloying … [and] smells like bug spray.” Then off his application went, with the fake wine list and a real money order.
GOLDSTEIN: My hypothesis was that the $250 fee was really the functional part of the application. In other words, the entire awards program was really just an advertising scheme, and that it was being fraudulently misrepresented as an exercise of expert judgment by Wine Spectator.
DUBNER: I see, was a little piece of you expecting that when you applied for this that they would send someone around to drink some of your wine or eat some of your food?
GOLDSTEIN: Well of course that’s the experiment, I didn’t know. I wasn’t sure going into this that I would win an award. There were two questions being tested here. One was, do you have to have a good wine list to win a Wine Spectator award of excellence? And the second was, do you have to exist to win a Wine Spectator award of excellence? So I thought that it was quite possible that my experiment would fail.
DUBNER: He didn’t fail.
[WINE SPECTATOR VOICE MAIL] Hello. I am calling from the Wine Spectator magazine in New York. Calling to congratulate you and let you know that your restaurant was selected as a Wine Spectator restaurant award winner. Hopefully you did receive some information from us. But I’m calling to follow up and see if you might have an interest in publicizing your award with an ad in the upcoming issue which comes out in August as we do have a lot of international business travelers who may be coming to Milan. Feel free to call me back...
DUBNER: And what was the name of the award that you won for your fictional restaurant?
GOLDSTEIN: The Wine Spectator Award of Excellence.
DUBNER: So congratulations, that’s awesome that you’re a winner. I too could be a winner presumably?
GOLSTEIN: Yeah, if your wines are bad enough.
[WINE SPECTATOR VOICE MAIL] Thank you so much. Have a great day. Bye, bye. (click)
DUBNER: In August of 2008, a new organization called the American Association of Wine Economists held its annual conference in Portland, Oregon. That’s where Goldstein revealed his fake Award of Excellence. The press drank it up. From the New York Post: “Wine Mag Humbled by Hoax.” According to the L.A. Times, Wine Spectator was now “Drinking a Hearty Glass of Blush.” Wine Spectator vigorously defended its award system. The executive editor said the magazine never claimed to visit every restaurant, and that it did its due diligence on Osteria L’Intrepido -- looking over its website and calling the restaurant -- but that it kept reaching an answering machine.
Robin Goldstein, for his part, was convinced: The wine system was fundamentally flawed. If a fake restaurant with a wine list that included bad, expensive wines could win an Award of Excellence from one of the most prestigious wine magazines in the world, who are we supposed to trust?
GOLDSTEIN: My takeaway is that expert sources in the media are trusted too much, and that they’re prone to abusing their positions of power as a way of making money. So the phenomenon of what’s really an ad is posing as real expert judgment is very problematic for consumers, because consumers really put trust in these magazines. We put trust in experts. There are so many fields out there where we don’t know as much as the experts do. And so we use experts as an information intermediary, as a proxy for good judgment, in an area where we don’t know as much as the experts are supposed to. When we trust experts too much, and they sell their awards to entities that are really their customers that’s quite problematic.
DUBNER: Robin Goldstein also had an academic paper to present at that conference of wine economists. The paper was called “Do More Expensive Wines Taste Better?” If the Osteria L’Intrepido stunt was just a stunt, and if the Steve Levitt and Brian DiMarco blind tastings we heard about earlier were just unscientific tricks -- well, Goldstein’s paper was the opposite of that. It gathered up and analyzed data from 17 blind tastings that Goldstein himself organized. The data included more than 6,000 observations from more than 500 people, from amateur wine drinkers to sommeliers and wine makers. He tested red wines, whites, roses. The prices ranged from $1.65 a bottle to $150 a bottle. It was as rigorous as you could get.
And what did Goldstein learn? That, overall, people liked expensive wines...less than cheap wines. When you don’t know what a bottle of wine costs, apparently you don’t know how good it’s supposed to taste. Even the most expert tasters could barely tell the difference between expensive wines and cheap ones.
It’s unsettling, isn’t it? Buying a bottle of wine shouldn’t be as complicated as buying a house. But thanks to the layers of experts between us and the grapes, we’ve got performance anxiety. Wine isn’t supposed to be a drag. It’s a celebration, a beloved recipe, civilization in a bottle. You’re drinking the hand-crafted fruit of some farmer’s vines that may go back hundreds of years, grown under the same sun that’s been shining for billions of years. Wouldn’t it be nice to drop the pretense -- to set aside the ratings and price -- and just drink? Brian DiMarco, the wine importer, that’s what he’s really after.
DIMARCO: Who are we to tell the Connecticut housewife that that oaky chardonnay that she’s been slamming down for the last ten years from California that she’s not deriving pleasure from that? She surely is. But there are so many other things. It’s like limiting your musical notes to only playing f and c notes the rest of your life. It’s only eating cheeseburgers and hamburgers, you know, the rest of your life, if you want to try other foods and other cultures and other things. If opening this bottle of rioja somehow takes you to Spain and makes you feel like you want tapas, then it’s done its job. And whether it’s a $15 bottle or a $30 bottle is irrelevant at that point. Then you’re getting into nuances of good and great. But I think the bottle of wine is the ability to either change your day from an alcoholic standpoint, getting to a certain point where you’re like OK this is -- or for people like myself who are in the trade -- it’s something you have with food. Growing up in an Italian household we weren’t drinking great wine, but to me wine wasn’t alcohol. Wine was Carlo Rossi in a jug that was poured into juice glasses for my grandfather and that was what you had with dinner. Drinking to me was Budweiser and Jack Daniels. You know, sneaking out on Friday night to go drink.
DUBNER: Let just ask you, we’re sitting here in a radio studio in downtown New York with kind of mucus-colored sound padding, and artificial light. There’s basically nothing good about this space, there’s no air, there’s nothing. But--
DIMARCO: It’s soulless.
DUBNER: But transport me for a minute. So tell me Brian, the room that you would like to be sitting in right now, and the food that you’d like to be eating right now, and most of all what the wine is with that food, what that wine tastes like to you.
DIMARCO: I think a crisp sancerre would be perfect right now with some sort of croque monsieur, and maybe a salad with a little vinaigrette would be kind of perfect. Not too decadent, but definitely has its place. And maybe have a backup bottle on ice.
DUBNER: Has your table got room for another guy there?
DIMARCO: You’re sitting with me right now. This pressed, the wood table, I think it can hold an ice bucket, I’m pretty confident.
ANNOUNCER: Freakonomics Radio is produced by WNYC, American Public Media and Dubner Productions. Our producers include Andrew Gartrell, whose taste guided this podcast, and Suzie Lechtenberg, who views life as a blind tasting. This podcast is mixed by David Herman, who never touches the stuff. Our executive producer is Collin Campbell. Subscribe to this podcast on iTunes and you’ll get the next episode in your sleep. You can find more audio at FreakonomicsRadio.com. And, as always, if you want to read more about the hidden side of everything, go to Freakonomics.com.