The Dangers of N=1
From a reader named Mike Friedman:
I realized this morning that my daily behavior has been modified by a data point of one. I thought it would be interesting to you. Oh, and apologies in advance for the pun:
For the most part, my morning routine is the same every day. I shower, towel off, comb my hair, yada yada yada. A few weeks ago, while toweling off, I noticed a rather large spider on my towel. Being a little squeamish around spiders, I quickly threw down the towel and disposed of the intruding arachnid. I forgot about the “insect-dent” and went about my day.
Cut to this morning. After getting out of the shower and grabbing my towel, I experienced a moment of self-awareness and realized I was subconsciously checking my towel for spiders before applying it to my face. I realized in that moment of self-awareness that my behavior had been influenced by a single data point.
Consider the numbers: At 35 years old, I figure I have conservatively showered well over 12,500 times in my life, and I have only found one spider in my towel. And yet, for the last three weeks or so, I have been checking my towel every time I shower.
It got me thinking: How often do we allow our behavior to be influenced by single data points. Are there any positive examples?
Mike’s e-mail appealed to me because it touches on a lot of things that have been discussed here over the years, including recency bias, black swans, and the strange case of Baby Emily, whom we wrote about here:
In the early 1980’s, a group of psychologists and linguists banded together to write Narratives From the Crib, a study of how children acquire linguistic skills. Narratives was built around the speech patterns of one child, a 2-year-old girl. Her parents had noticed that she often talked to herself in the crib after they said good night and left her room. They were curious to know what she was saying, so they began to record her chatter. They turned on the tape recorder while they were tucking her in and then left it running.
Eventually they gave the tapes to a psychologist friend, who shared it with her colleagues. The big surprise to these experts was that the girl’s speech was far more sophisticated when she was alone than when she was speaking with her parents. This finding, as Malcolm Gladwell would later write in The Tipping Point, “was critical in changing the views of many child experts.”
The 2-year-old girl in question was referred to as Baby Emily. Her full name? Emily Oster. In retrospect, it would appear that Narratives From the Crib suffers what researchers call an “n of 1” problem, with “n” representing the size of the sample set — a problem that is gravely exacerbated when the one subject turns out to be … well, a good bit brighter than average. Studying how children learn to talk by observing Baby Emily may be a bit like studying how children learn to play golf by studying Tiger Woods.