Experience vs. Information, Part 2

I recently blogged about whether we form our opinions more from information than experience. The starting point was a passage in David McCullough‘s book The Great Bridge, and he was comparing modern Americans with our 19th-century counterparts.

I was interested not just as a historical comparison but because the information/experience question is compelling in its own right. Consider a doctor, for instance: as Jerome Groopman writes very well in How Doctors Think, the act of diagnosing a patient is a tricky blend of science (relying primarily on information) and art (relying primarily on experience).

Or pretend for a minute that you’re Manny Ramirez facing Mariano Rivera with an 0-2 count with two outs in the bottom of the ninth inning in the seventh game of the A.L. Championship Series, and you know that Rivera throws a breaking pitch away in 80 percent of such instances — but you also know that he struck you out with a high fastball last night with the same count. (What with Manny being Manny, this may not be the best example.)

But rather than my speculation, let’s hear from someone who’s done some actual research on this question.

Greg Barron, an assistant professor in negotiation, organizations, and markets at the Harvard Business School, e-mailed in response to the earlier post to share with us “what we know so far” about “my favorite topic, decisions from experience.”

Here are Barron’s summaries of two papers he has co-authored. The first, published in Psychological Science, is called “Decisions From Experience and the Effect of Rare Events in Risky Choice.”

1. Decisions made from personal experience and those made from information (what we term “description”) are qualitatively different — in particular regarding rare, but important, events that are overweighted in information-based decisions (see Prospect Theory) but underweighted in decisions from experience. For example, consider G. Gigerenzer‘s observation that more people were killed in the increase in traffic accidents after 9/11 than were killed in the attacks themselves. Risk on the road is learned from experience, and is underweighted.

2. When people have both sources of information, description and personal experience, we find behavioral inertia for whatever information was encountered first. Thus, in the 80’s people who had been practicing unsafe sex for 30 years were not as impressed by the AIDS epidemic as today’s teenagers who are scared silly (despite the rate of HIV transmission with an actual AIDS partner being as low as 1/1000). More neat applications (ex. Vioxx) are in the paper.


An individual

Years ago I had a baby die in utero 3 weeks before the 40 week due date. I was healthy, under 35, we'd had a normal amnio, and the risk of stillbirth was less than 1 in 10,000. But it still happened, to no cause attributable! Years later I can only conclude that statistics are a composite observation and have nothing to do with an individual's experience. Yet educated doctors rely on them.

Ben

But where does the information come from? Isn't is just other people's experiences? So the question is really My Experience versus Other People's Experiences.

John, Boston

Michael F. Martin: You point above "I'm skeptical about the empirical result that events are overweighted by information-based inferences. If that were true, then why do we keep having financial bubbles? Isn't the whole point of the “Black Swan” literature that we are NOT overweighting the possibility of catastrophes?"

You are making an invalid comparison. As individuals we can adjust our actions quickly based on information and experience. As a mob (which economics deals with) we do not have either faculty. This is why nations as a whole are unable to learn from history, but individuals can. For this reason, we will continue to have bubble after bubble until the next learning horizon comes about.

OldBob

Education is the process of reading the fine print. Experience is the result of not reading the fine print.

I opt for education.

WillH

The plural of anecdote is not data. However, there is still anecdotal evidence.

My guess is that some people are simply more trusting of one or the other form of information. It may be a function of experience or genetics or both.

Some people seem to really value their own experiences and/or the experiences of others (e.g. word of mouth). The possibility that their experiences are atypical is not on their radar. As is obvious, both forms of information are important and credible. The key is in finding that balance that enables us to make the 'right' decision most often. My guess is that most of us are far from that mark.

David Braden

Seems a vitally important dimension has been overlooked here, namely, WHO IS THE DECISION-MAKER? In the context of medical decisions, patients often choose (often under extreme stress, and not infrequently with language or cultural barriers) to forfeit the bulk of the decision-making to the medical staff, or to family members; and of course, there are occasions where the patient is incapable of speaking for herself. This problem is compounded by who has to bear which risks associated with myriad sets of decisions.

That MDs mess up in combining information with experience is ancient news (Ken Arrow published a paper decades ago documenting this), but who of us whips out a calculator to do some Bayesian updating after test results come in? I have personally run into this problem with life or death decisions that can be framed fairly simply, such as helping my wife decide whether or not to get an amniocentesis. (And yes, I did the Bayesian updating a priori).

Nevertheless, before getting into the probability/information/experience thing, I make sure I understand as best I can what is at stake, who the stakeholders are, and that all are clear on who the decision-makers are, and in what capacity.

Oh, and David @24 got it right, assuming many occasions of intercourse to come: the average number of occasions before getting aids under the clarified assumptions is 1000. Dan @26 describes the median, and David Pearson @27 calculates it correctly, but that is not the same thing as the average.

Read more...

Dave B

How does one know that they have all the right information without experience to make decisions.

Nick DiGiacomo

You are simply rebranding two well-known concepts -deductive and inductive reasoning. Please give (historical) credit where historical credit is due.

malnicore

It is our conception of who we think we are and who we think we want to be that has the greatest influence. Subjectivity, in other words is the predominant influence. Moreover, subjectivity is easily manipulated by dominant discourses, so who we think we are and who we think we want to be is often to a great extent determined by others. The only means of protection against this "colonization of subjectivity" is to conceptualize oneself beyond the constraints of the subject-object dichotomy that is the fundament of the Western Episteme.

Geoffrey

Jessie:

You misread the statistic: its not a 1 in 1000 chance of contracting AIDS through sex...its a 1 in 1000 chance of contracting AIDS WHEN YOU ALREADY know that your partner is HIV+.

The reason that AIDS was so rampant in the early stages of the disease had to do with the much higher risk levels of those succeptible to AIDS. The people that contracted AIDS early on often had severely compromised immune systems due to prior exposure to syphilis or hepatitis, which were rampant in the gay and IV drug communities in the 1970's; gay men knew the risk of contracting syphilis was high but that it was 100% curable from a pennicillin injection.

Johnny E

The trouble is experience may give false associations. If you did a rain dance once and it happened to rain shortly after that is there a cause and effect?

Then there are stubborn people who think if something is supposed to work a certain way they'll keep trying to do it that way even if it doesn't work.

Or they assume previous experience applies to the new problem, ie. we try to fight the new war like the previous one.

Then there are others who completely ignore the experience because of ideology.

KayTi

Have you looked at the new book by Amanda Ripley titled "The Unthinkable: Who Survives When Disaster Strikes and Why"?
Link to an article reviewing the book:
http://women.timesonline.co.uk/tol/life_and_style/women/the_way_we_live/article4149935.ece

I haven't read the book, but the author's comments in several interviews make me think you Freakonomics folks would be interested in her analysis and conclusions.

Gulley Jimson

Where's George Will? Baseball and minutiae fits him to a tee. I'm sure the destruction of Yankee stadium is but a mirror of western civilization's downfall and the stench was but a small part of the big machine's detritus. Success smells, non?

Bill R

Without experience, information is overwhelming. Without information, experience cannot be efficiently leveraged.

Using experience typically does not stand in contrast to using information, although we all know people to rely too heavily on one or the other.

I think the best question is how can we maximize the use of both information and experience to understand what's going on around us and make advantageous decisions.

As Chase and Simon pointed out long ago, the expert chess player knows how to combine the information on the board with experience to succeed.

The Manny-Mario example misses the mark, because it presents only experience data. The information is in the actual pitch--and you've got to react pretty fast here...

osisbs

"People never learn from the mistakes of others and only rarely learn from their own mistakes."

Given that this quote is true, I'd have to side with the people who think humans run their lives according to misinformation, paranoia, superstition, religious dogma, and emotion.

Jesse Davis

I'm 29, and one of those scared silly of HIV transmission. It's hard to believe that's an irrational fear -- risk of contracting HIV must be much higher than 1/1000 in many circumstances, otherwise we wouldn't have had the epidemic of the 90s. (After all, how long does it take for one HIV+ person to have sex with 1000 people? And after all the work, to have an expected transmission total of 1?) Besides, even if it is only 1/1000, what a high price to pay for bad luck.

Marvin

This reminds me of Robert Axelrod's The Evolution of Cooperation. What is also instructive, a la the Manny-Mario example cited above is the source of information. Manny and Mario are not trying to help each other, whereas your doctor (we assume) is.
http://blackpoliticalanalysis.blogspot.com

steve

familiarity breeds contempt?

Richard

The aids reference is horrific and moronic. I somehow lived through West Hollywood in the 80's --- don't tell me we were not scared. We were trying every way we could to learn how to save each other. I have no idea how old you are, but you obviously were not there with me, and you surely do not have children in high school now. Can anyone say anything in the NYTimes these days?

Michael F. Martin

Maybe I'm missing the subtlety, but aren't we just saying that both information (i.e., statistical inference) and experience (i.e., anecdotal inference) are flawed because they rely on historical data? What other data can we rely on?

I'm skeptical about the empirical result that events are overweighted by information-based inferences. If that were true, then why do we keep having financial bubbles? Isn't the whole point of the "Black Swan" literature that we are NOT overweighting the possibility of catastrophes?

But the idea that there is some path-dependence to decisionmaking that results from exposure to either information or experience first is interesting. Maybe it could be explained also by cultural differences in the approach to data? Physical scientists are often inherently skeptical of intuitions that do not have a discernible mechanism. Sociologists are inherently skeptical of any mechanistic explanation that doesn't explain all of the data.

Read more...