Do More Expensive Wines Taste Better?

DUBNER: So Levitt, a college friend of yours once told me that your favorite meal during college was a dill pickle, beef jerky and grape soda. Is that true?

LEVITT: I did indeed have that for breakfast, but to tell you the truth, it sounded better before I ate it than after.

Steve Levitt, my Freakonomics friend and co-author, an esteemed economist at the University of Chicago, has an extremely refined palate.

DUBNER: All right, so you’ve got beef jerky, you’ve got dill pickle, what are your favorite foods? Like, what are your favorite places…if you could drive across America and pick anyplace to stop and eat, what’s it going to be?

LEVITT: You know, I love the Billy Goat tavern. It’s the cheap place that was made famous in the 19-probably-70s on Saturday Night Live, with the ‘cheeseburger- cheeseburger, no Pepsi, Coke.’ Anyway, they have an incredible rib-eye steak sandwich. Pretty much the cheaper the food the better, there’s almost no fast food that I don’t adore.

DUBNER: So KFC?

LEVITT: Yeah, I like KFC. Burgers. Chipotle. I’d kill for Chipotle.

DUBNER: So how would you describe your palate?

LEVITT: Probably underdeveloped. You know,  but it’s good. The thing is, it’s a wonderful, wonderful gift to like cheap food. Some people just happen to like expensive food and then they are unhappy most of the time, or else, they spend all their money on food. But if you just, by chance are born loving cheap food, then you can eat everything that you love.

DUBNER: Now, how much do you like wine?

LEVITT: Wine I do not like at all.

Good wine, we’re told, is the province of smart, superior people.  They taste things on a whole ‘nother level than people like you and me …

[WINE TASTING] I taste a lot of minerality…Dill, leather, weeds, baby vomit.

But I have a question for you: is that superiority deserved? The wines these people love — the ones that bring out the natural beauty of the grape but aren’t too funky — they cost a lot more. But are expensive wines really that much better than cheap wines?  Or, is it possible that developing your palate just mucks things up?  Complicates things? Maybe we’d all be better off if we had taste buds like Steve Levitt’s.

After graduate school, he was invited to join an elite club at Harvard called the Society of  Fellows. A junior fellow like Levitt was paid a modest salary to work on his own research, with few obligations — other than a formal Monday night dinner with the senior fellows, who were some of the most remarkable scholars alive.

LEVITT: People like Amartya Sen, and Nobel prize winning physicists and what not, and you sit around a table. I believe the table was Oliver Wendell Holmes table initially and he gifted it to the Society of Fellows.  

Over dinner, they engaged in witty, learned conversation; they ate venison and other fine foods; and they drank expensive wine — bottle after bottle. To this budding young economist, with the beef jerky taste and the grape-soda budget, all that pricey wine wasn’t doing him any good. He had a thought.

LEVITT: Innocently, I was young, I didn’t understand how the world worked, I thought like an economist, I suggested that perhaps we should have two tracks at the Society of Fellows, there would be the drinking track and the abstinence track. And for those of us who chose the abstinence track, because the cost of the wine was perhaps $60 per meal, that over the course of fifty weeks of the year, that would work out to be about  $3,000 and they could add $3,000 to the paycheck of those of us on the abstinence path.

DUBNER: And did you have any other people in your abstinence camp, or was this just Levitt?

LEVITT: Well, you know it didn’t really get that far because the reaction was quite negative.

Levitt is the kind of person who likes to use data, not a personal agenda, to make his arguments. So he set out to get some data. Wine data. The wine they were drinking at these dinners cost 5 or 10 times what a cheap bottle of wine cost. Was it really 5 or 10 times better? He hatched a plan. The Society of Fellows held wine tastings from time to time. He suggested that the next one be his to organize.

LEVITT: “So I worked with the wine steward to select two excellent bottles of wine, expensive bottles of wine, you know, probably close to $100 bottles of wine. And then I went to the liquor store that was down the street and I said, can I have the cheapest bottle of wine you have that was the same grape, I don’t remember which grape it was.”

Levitt used four decanters. He poured an expensive bottle of wine into decanter number one. The other expensive wine went into decanter number two. In the third decanter, he poured the cheap wine, which cost around 8 dollars. In the fourth decanter, he repeated one of the expensive wines.

LEVITT: So as far as the people knew, there were four different wines and these were all wines that were coming out of the wine cellar of the Society of Fellows.

DUBNER: So you are tricking them from the outset you’re leading them to believe the cheap wine is also from the wine cellar.

LEVITT: I don’t….you know….uh…. hold on…… Yes.

They swirled them a bit … sniffed them … and sipped. They wrote down their ratings. As he looked at the numbers, Levitt’s cold economist heart warmed.

LEVITT: “The data could not have cooperated more completely with my hypothesis. So for starters the four wines received almost identical ratings on average. Although there were a wide spread among individuals, on average, tallied up, people did not prefer the expensive wines to the cheap wine. On top of that, and this was the thing I was hoping for and dreaming of but didn’t believe would actually come true.  It turned out among individuals if you compared how differently they rated any two of the wines that they had, it turned out that by a small margin, people actually rated the same wine from the same bottle but presented in a different decanter as being the most different of the two wines (so the two wines that were absolutely  identical, when you looked at the gap between the ratings that the individual gave to those wines, the gap was bigger than they did between the other wines, which actually were different.”

A few minutes after the tasting was over, Steve Levitt shared the results with the senior fellows.

LEVITT: The jovial mood in the room suddenly went dark. People realized they had been tricked. That there had been this cheap wine the same wine was in twice. And they really realized that the the nature of the game had been somewhat different than what they thought. And when they heard the results, that collectively they had no ability to identify wines, they were not happy. And particularly there was one senior fellow, so one of the professors at Harvard, who was quite outspoken about his knowledge of wine. And he loudly announced that he had a cold. Otherwise he clearly could have made the distinctions and he stormed from the room and left the party prematurely.

DUBNER: What was his discipline?

LEVITT: I think for the sake of anonymity, I should not reveal that particular piece of information. He was a humanist.

DUBNER: Not an economist, in other words.

LEVITT: No. No.

DUBNER: The opposite of an economist.

LEVITT: Exactly.

So what does Levitt’s evil little experiment tell us about wine? Maybe not all that much. It wasn’t a very scientific tasting, really. And perhaps the Society of Fellows was having an off night — the humanist had a cold, right? Or maybe this was just a group of people who didn’t know nearly as much about wine as they thought they knew. You’d never be able to pull this kind of stunt on wine experts, would you?

DIMARCO: On my team we have a master sommelier, two master of wine candidates, four people who have been in the trade for many years. These are sophisticated wine professionals.  

That’s Brian DiMarco, talking about the people he gathered for a blind tasting of his own. Before we get to that, let’s hear a bit more about Brian.

DIMARCO: I specialize in helping customers, consumer, private collectors, and retailers, and restaurants decide what wines to put on their list, what wines to collect, what wines to sell.  And I have a small import business and wholesale company.  And we distribute wine that we find all over Europe and South America in New York City.

DUBNER: Cool, so you’re almost a wine agent in a way, yeah? More than just an importer?

DIMARCO: In many ways, yes.

Brian DiMarco is one of the people who decide what we drink. He goes to France, Italy, California, tastes wine fresh out of the tank. So he decides what’s good — or bad — without any critics whispering sweet ratings into his ear. Then he puts his money where his taste buds are – he writes a check. According to DiMarco, he makes as much money selling a $15 bottle to a restaurant or shop as he does selling a $50 bottle. If that’s true, DiMarco is an honest broker. His job is to find wine that you or I would want to drink — because, think about it, there are thousands upon thousands of bottles to choose from. Just picture the rows and rows of bottles lining the shelves at every wine shop. You kinda, sorta, maybe think you want to buy a merlot. Do you pick the one with the pretty flowers on the label? Do you go with the one that Robert Parker, the high priest of wine ratings, awarded a lot of his points? Or, like a lot of people, do you let price be your guide? If you believe even a little bit in the free market, you’d have to think that expensive wines cost more because they taste better, right? Brian DiMarco wanted to know how much people were tasting the dollars when they drank an expensive wine. So, like Steve Levitt, he conducted a little experiment, with some of the people who work for him. As DiMarco said, these were no amateurs …

DIMARCO: We did a tasting, brown bag. And we had the exact same wine in both bags. And we told them that one bottle was a fifty-dollar bottle to write their reviews, and we told them the other was a ten-dollar bottle, to write the reviews.  Of course they were both twenty-dollar bottles of wine according to what they would sell at at any retail store. Then we reversed it and said now the ten-dollar bottle is really the fifty. And everyone liked the fifty-dollar bottle better in both circumstances, because they perceived that at the price there was either something they were missing, where in real life they were like the wines are so similar.  Yeah but they’re different.  A lot of people, two or three people said, is this the same wine?  And we said that’s for you to determine, and the more they thought about it, the more they intellectualized it, the more they decided there were differences to the wine.  

So this wasn’t a wildly scientific experiment either, but to Brian DiMarco, the message couldn’t have been clearer: when people know a wine is more expensive — or even think it is — it tastes better. Now, obviously this idea doesn’t apply just to wine. A house that costs $500,000 ought to be five times better on some level than a $100,000 house — the size, the construction, the schools and neighborhood. So is a wine that costs $50 five times better than a $10 bottle? Or is it even better at all?

Coming up … We go looking for hardcore, empirical evidence that expensive wines actually better than cheap wines. And we look beyond price to prestige — to wine ratings, the awards, the whole shebang. Because if your wine — or your whole wine list — wins an award from a magazine like Wine Spectator, it’s got to be good, right? Right?

GOLDSTEIN: My name is Robin Goldstein. I write about wine and food.  Basically my book “The Wine Trials” has been my principal outlet for writing about wine, but I’ve also been publishing academic papers on topics of taste from a cognitive perspective, and an economic perspective, usually coauthored with a colleague from different academic fields. So I’ve been exploring the neuroscience side of it a bit.  I’ve been exploring the behavioral side of it.  And in particular I’m interested in price signals and how people’s knowledge of price affects their experience of wine on the most basic sensory level.

DUBNER: First let me just ask you, I’ve heard good things about this lovely little restaurant in Milan called Osteria L’Intrepido, You ever been there?

GOLDSTEIN: I’ve been to the restaurant, but It’s actually located in my friend Giuliano’s former apartment in Milan.  The restaurant, I wouldn’t say it’s great.  Mostly they serve left over pizza, and their wine cellar consists mostly of left over bottles of Montepulciano Bruta from three weeks ago.

DUBNER: Meaning it’s not really a restaurant, is it?

GOLDSTEIN: It’s an apartment. So it’s just an address really.

*      *      *

Osteria L’Intrepido doesn’t exist; it’s a fake restaurant that Robin Goldstein made up. Why? Well, it’s a strange story. Goldstein’s research and writing on wine made him skeptical about critics and awards. He believed that so-called experts were, at best, subjective, and that they carried too much influence. He wondered about the awards that magazines like Wine Spectator gave to restaurants for their wine lists. Did an award like that really mean the wines were very excellent? So he made up Osteria L’Intrepido, or “Fearless Restaurant.” He created a fake menu, a fake website, and a fake voicemail message, saying the restaurant  was closed for vacation …

As for the Osteria L’Intrepido wine list — Goldstein made that up too. He included several expensive wines that Wine Spectator itself had given bad reviews in the past. One of them was a 1982 Brunello di Montalcino (broo-NELL-oh dee mon-tahl-CHEE-noe) which the magazine had given 67 points, or a D-plus rating, calling it “barnyardy” and “decayed.” He listed another vintage that Wine Spectator had reviewed as “Unacceptable … sweet and cloying … [and] smells like bug spray.” Then off his application went, with the fake wine list and a real money order.

GOLDSTEIN: My hypothesis was that the 250-dollar fee was really the functional part of the application.  In other words, the entire awards program was really just an advertising scheme, and that it was being fraudulently  misrepresented as an exercise of expert judgment by Wine Spectator.

DUBNER: Was a little piece of you expecting that when you applied for this that they would send someone around to drink some of your wine or eat some of your food?

GOLDSTEIN: Well of course that’s the experiment, I didn’t know.  I wasn’t sure going into this that I would win an award. There were two questions being tested here.  One was, do you have to have a good wine list to win a Wine Spectator award of excellence? And the second was, do you have to exist to win a Wine Spectator award of excellence? So I thought that it was quite possible that my experiment would fail [CUT THIS: on at least one of those two counts and I wouldn’t get an award.]

He didn’t fail.

WINE SPECTATOR VOICE MAIL: Hello. I am calling from the Wine Spectator magazine in New York. Calling to congratulate you and let you know that your restaurant was selected as a Wine  Spectator restaurant award winner. Hopefully you did receive some information from us. But I’m calling to follow up and see if you might have an interest in publicizing your award with an ad in the upcoming issue which comes out in August as we do have a lot of international business travelers who may be coming to Milan. Feel free to call me back in New York at 212-xxx-xxxx. Thank you so much. Have a great day. Bye, bye. (click)

DUBNER: And what was the name of the award that you won from your fictional restaurant?

GOLDSTEIN: The Wine Spectator Award of Excellence.

DUBNER: So congratulations, that’s awesome that you’re a winner.  I too could be a winner presumably?

GOLDSTEIN: Yeah, if your wines are bad enough.

In August 2008, a new organization called the American Association of Wine Economists held its annual conference in Portland, Oregon. That’s where Goldstein (“STINE”) revealed his fake Award of Excellence. The press drank it up. From the New York Post: “Wine Mag Humbled by Hoax.” According to the L.A. Times, Wine Spectator was “Drinking a Hearty Glass of Blush.” Wine Spectator vigorously defended its award system. The executive editor said the magazine never claimed to visit every restaurant, and that it did its due diligence on Osteria L’Intrepido — looking over its website and calling the restaurant — but that it kept reaching an answering machine.

Robin Goldstein, for his part, was convinced: the wine system was fundamentally flawed. If a fake restaurant with a wine list that included bad, expensive wines could win an Award of Excellence from one of the most prestigious wine magazines in the world, who are we  supposed to trust?

GOLDSTEIN: My take away is that expert sources in the media are trusted too much, and that they’re prone to abusing their positions of power as a way of making money.  So the phenomenon of what’s really an ad is posing as real expert judgment is very problematic for consumers, because consumers really put trust in these magazines. We put trust in experts.  There are so many fields out there where we don’t know as much as the experts do.  We use experts as an information intermediary, as a proxy for good judgment in an area where we don’t know as much as the experts are supposed to. When we trust experts too much, and they sell their awards to entities that are really their customers that’s quite problematic.

Robin Goldstein also had an academic paper to present at that conference of wine economists. The paper was called “Do More Expensive Wines Taste Better?” If the Osteria L’Intrepido stunt was just a stunt, if the Steve Levitt and Brian DiMarco blind tastings were just unscientific tricks — well, Goldstein’s paper was the opposite of that. It gathered up and analyzed data from 17 blind tastings that Goldstein himself organized. The data included more than 6,000 observations from more than 500 people, from amateur wine drinkers to sommeliers and wine makers. He tested red wines, whites, roses. The prices ranged from $1.65 a bottle to $150. It was as rigorous as you could get.

And what did Goldstein learn? That, overall, people liked expensive wines … less than cheap wines. When you don’t know what a bottle of wine costs, apparently you don’t know how good it’s supposed to taste. Even the most expert tasters could barely tell the difference between expensive wines and cheap ones.

It’s unsettling, isn’t it? Buying a bottle of wine shouldn’t be as complicated as buying a house. But thanks to the layers of experts between us and the grapes, we have performance anxiety. Wine isn’t supposed to be a drag. It’s a celebration, a beloved recipe, civilization in a bottle. You’re drinking the hand-crafted fruit of some farmer’s vines that may go back hundreds of years, grown under the same sun that’s been shining for billions of years. Wouldn’t it be nice to drop the pretense — to set aside the ratings and price — and just drink? Brian DiMarco, the wine importer, that’s what he’s really after …

DIMARCO: Who are we to tell the Connecticut housewife that the oak-y chardonnay that’s she’s been slamming down for the last ten years from California that she’s not deriving pleasure from that.  She surely is.  There are so many other things.  It’s like limiting your musical notes to only playing f and c notes the rest of your life.  It’s only eating cheeseburgers and hamburgers, you know, the rest of your life, if you want to try other foods and other cultures and things.  If opening this bottle of rioja somehow takes you to Spain and makes you feel like you want tapas then it’s done it’s job.  And whether it’s a fifteen-dollar bottle or a thirty-dollar bottle is irrelevant at that point.  Then you’re getting into nuances of good and great.  But I think the bottle of wine is the ability to either change your day from an alcoholic standpoint, getting to a certain point where you’re like ok this is…OR for people like myself who are in the trade it’s something you have with food. Growing up in an Italian household we weren’t drinking great wine, but to me wine wasn’t alcohol.  Wine was Carlo Rossi in a jug that was poured into juice glasses for my grandfather and that was what you had with dinner.  Drinking to me was Budweiser and Jack Daniels.  You know, sneaking out on Friday night to go drink.  

DUBNER: Let me just ask you, we’re sitting here in a radio studio in downtown New York with kind of mucus colored sound padding, and artificial light.  There’s basically nothing good about this space, there’s no air, there’s nothing.

DIMARCO: It’s soulless.

DUBNER: Yeah, it’s soulless to the max.  But transport me for a minute.  So tell me Brian, the room that you would like to be sitting in right now, and the food that you’d like to be eating right now, and most of all what the wine is with that food, what that wine tastes like to you.

DIMARCO: I think a crisp Sancerre would be perfect right now with some sort of croque monsieur, and maybe a salad with a little vinaigrette would be perfect.  Not too decadent, but definitely has its place.  And maybe have a back up bottle on ice.

DUBNER: Has your table got room for another guy there?

DIMARCO: You’re sitting with me right now. This pressed wood table, I think it can hold an ice bucket, I’m pretty confident.