“AI: Adventures in Ideas”: Getting to “Yes (I’ll Participate in Your Survey)”

Readers may know that I’m not the most qualified person to talk about using surveys. My first attempt — asking street gang members “How does it feel to be black and poor? Very bad, bad, good, …” — was met with laughter, disbelief and, scorn. (I suppose it was all uphill from that point!)

A basic question social scientists confront is: Why would you want to participate in our survey? Interviews can be long and boring; who wants to sit on the phone or stand on a streetcorner answering questions? A few bucks may not be worth the time. In fact, you have likely already perfected methods of avoiding telemarketers and sidewalk interviewers. From a data standpoint, your skilled avoidance is our problem: the views of respondents can differ from non-participants. From political races to consumer habits to opinion polls … we love numbers, and we need participation to get an accurate reading.

In a recent article, “Calling for Participation,” sociologist Jeremy Freese considers effective strategies to ensure respondents give us time for a survey. Where economists emphasize incentives — often monetary ones — Freese finds that interviewers can change their “style” of talk to increase participation. Interestingly, aggressive interviewers who head straight into the questions, presuming your participation, will be rebuffed. They are more likely to hear “No thanks” from people.

(Photo: Micky Aldridge)

Freese also finds that 63 percent of the time individuals pre-emptively say “No thank you”— even before the interviewer has a chance to ask them for a few minutes. To avoid these “blocking moves,” interviewers can first introduce themselves (and their organization and their question) before saying “I’d like you to participate in a survey.”

Somewhat more troubling perhaps, I have found that minor changes in wording can yield entirely different answers. For example, when I asked for an “interview,” individual responses were guided by an overwhelming need to please — people became nervous and told me what they thought I wanted to hear, rather than what they thought to be true. But when I asked to have a “conversation,” they responded by saying, “Thanks for letting me share my story, I’ve never really had a chance to talk about that…” The information they gave was more likely to be true and original.

If this sounds like tales-from-the-world-of-the-obvious, well… that’s a core part of sociology: deploying analytics to study everyday (often common-sense) behavior to locate underlying patterns.

A more profitable way of thinking might be rooted in the ideas of my former colleague (and now Research Scientist at Yahoo!) Duncan Watts. Perhaps we no longer need to survey a large mass of individuals. Instead, let’s just ask those who matter — and then interview this much smaller subset. In a provocative paper on how we influence one another’s political views, Watts writes that not only are we more likely to associate with like-minded people, but, more powerfully, we don’t really challenge our peers to think differently. Like begets like.

So perhaps the traditional social survey is faulty and can be discarded once and for all. Maybe “individual” opinions are misleading, and so we should identify “networked” opinions—highly interconnected belief systems and opinion clusters that may (or may not) have a huge impact on wider public perception. We should be interviewing social networks. But does this even make sense?

In a paper on “influencers,” Watts dispels our notions about who matters:

Under most conditions, cascades do not succeed because of a few highly influential individuals influencing everyone else, but rather on account of a critical mass of easily influenced individuals influencing other easy-to-influence people… Just because the outcome is striking, however, does not on its own imply that there is anything correspondingly special about the characteristics of the individuals involved.

We’re back to the basic premise but with a difference: identify clusters of shared opinions, but don’t necessarily believe that any individual was responsible for creating them. From the standpoint of someone who once asked “How does it feel to be black and poor?”, the prospect of never administering another survey questionnaire is a huge relief.