“AI: Adventures in Ideas”: Getting to “Yes (I’ll Participate in Your Survey)”

Readers may know that I’m not the most qualified person to talk about using surveys. My first attempt — asking street gang members “How does it feel to be black and poor? Very bad, bad, good, …” — was met with laughter, disbelief and, scorn. (I suppose it was all uphill from that point!)

A basic question social scientists confront is: Why would you want to participate in our survey? Interviews can be long and boring; who wants to sit on the phone or stand on a streetcorner answering questions? A few bucks may not be worth the time. In fact, you have likely already perfected methods of avoiding telemarketers and sidewalk interviewers. From a data standpoint, your skilled avoidance is our problem: the views of respondents can differ from non-participants. From political races to consumer habits to opinion polls … we love numbers, and we need participation to get an accurate reading.

In a recent article, “Calling for Participation,” sociologist Jeremy Freese considers effective strategies to ensure respondents give us time for a survey. Where economists emphasize incentives — often monetary ones — Freese finds that interviewers can change their “style” of talk to increase participation. Interestingly, aggressive interviewers who head straight into the questions, presuming your participation, will be rebuffed. They are more likely to hear “No thanks” from people.

(Photo: Micky Aldridge)

Freese also finds that 63 percent of the time individuals pre-emptively say “No thank you”— even before the interviewer has a chance to ask them for a few minutes. To avoid these “blocking moves,” interviewers can first introduce themselves (and their organization and their question) before saying “I’d like you to participate in a survey.” 

Somewhat more troubling perhaps, I have found that minor changes in wording can yield entirely different answers. For example, when I asked for an “interview,” individual responses were guided by an overwhelming need to please — people became nervous and told me what they thought I wanted to hear, rather than what they thought to be true. But when I asked to have a “conversation,” they responded by saying, “Thanks for letting me share my story, I’ve never really had a chance to talk about that…” The information they gave was more likely to be true and original.

If this sounds like tales-from-the-world-of-the-obvious, well… that’s a core part of sociology: deploying analytics to study everyday (often common-sense) behavior to locate underlying patterns. 

A more profitable way of thinking might be rooted in the ideas of my former colleague (and now Research Scientist at Yahoo!) Duncan Watts. Perhaps we no longer need to survey a large mass of individuals. Instead, let’s just ask those who matter — and then interview this much smaller subset. In a provocative paper on how we influence one another’s political views, Watts writes that not only are we more likely to associate with like-minded people, but, more powerfully, we don’t really challenge our peers to think differently. Like begets like.

So perhaps the traditional social survey is faulty and can be discarded once and for all. Maybe “individual” opinions are misleading, and so we should identify “networked” opinions—highly interconnected belief systems and opinion clusters that may (or may not) have a huge impact on wider public perception. We should be interviewing social networks. But does this even make sense?

In a paper on “influencers,” Watts dispels our notions about who matters:

Under most conditions, cascades do not succeed because of a few highly influential individuals influencing everyone else, but rather on account of a critical mass of easily influenced individuals influencing other easy-to-influence people… Just because the outcome is striking, however, does not on its own imply that there is anything correspondingly special about the characteristics of the individuals involved.

We’re back to the basic premise but with a difference: identify clusters of shared opinions, but don’t necessarily believe that any individual was responsible for creating them. From the standpoint of someone who once asked “How does it feel to be black and poor?”, the prospect of never administering another survey questionnaire is a huge relief.

Leave A Comment

Comments are moderated and generally will be posted if they are on-topic and not abusive.

 

COMMENTS: 7

View All Comments »
  1. Steve says:

    Speaking as one of those who say, “No thanks” (and hangs up), before the survey taker is even finished talking, I’ll say I’m more likely to participate in online surveys. With an online survey, I can answer at my leisure, pause and restart at will, and stop halfway without finishing (if I feel the survey is actually a push poll).

    Alternately, if the survey takers lead with the very specific number of questions in the survey (do you have a moment to answer FIVE questions?) or the specific amount of time (do you have FORTY FIVE SECONDS to…) so that I know I’m not committing myself to a trip down a rat hole, I might be more inclined to participate.

    Someone needs to do a study on the differences between people who bother answering surveys vs. those who don’t.

    Thumb up 3 Thumb down 0
  2. Frederik Anseel says:

    Hi Steve

    Well, I actually conducted such a study. A while ago, I invested an absurd amount of time in coding response-enhancing techniques in existing studies to see the effects on response rates. I analyzed over 2000 published surveys. It seems some techniques work better for some respondents than others (e.g., incentives are most effective in students and unemployed respondents). the study eventually ended up in journal of business and psychology, you can find it here. I included a short table with recommendations for researhers, see http://ow.ly/9unjY

    Well-loved. Like or Dislike: Thumb up 7 Thumb down 0
  3. Frost says:

    Recently I conducted a research project which involved interviewing top executives at a certain subset of agencies. Based on prior findings, we used tactics to agressively target a “small” group of executives rather than simply going after as many as possible. To do this we gathered as much information as possible on them, sent personalized letters in advance to explain what we were doing, and then called their office to schedule an appointment to speak with them. People are more receptive if the study appears professional and you are polite about it rather than if they feel like they are just another call from some database.

    And it payed off – we were able to get a 60% response rate from the CEOs of these agencies! We had decided that a high response rate was much more important than the actual number of people that we interviewed, since this would arguably be more representative of the sample.

    Thumb up 4 Thumb down 0
  4. mhenner says:

    I once asked a survey taker how much they were being paid for the time to ask the questions, and then asked if the sponsoring organization would pay me the same amount for answering.
    He was shocked.

    Thumb up 1 Thumb down 1
  5. Shane L says:

    I worked for a short time doing market research surveys over the phone and one problem I found was that many people seemed to immediately assume I wanted to sell them something. They would hang up with words like “don’t want to buy anything, thanks” before I had explained that I wasn’t a salesperson.

    It was a stressful job! Rejection rates were very high and some people were enraged that I was phoning them at all. While this was frustrating for me, I couldn’t help but sympathise with them. I remember one angry respondent explaining that she had just put the baby to bed and the phone call had woken the infant. It can be quite intrusive.

    …But since then I’ve tried to give researchers my time, because I know how stressful it can be to face rejection after rejection.

    Thumb up 2 Thumb down 0
  6. Enter your name... says:

    On a related note:

    I’ve seen a lot about marketing and privacy issues recently. Organizations generally know what I buy, when I buy it, and how I buy it. Amazon.com could probably do my shopping for me at this point, right down to telling me which credit card each purchase belonged to.

    All of which has made me wonder why I still get phone calls asking me for donations to charities and political organizations.

    I’ve had the same phone number for more than 15 years. I have never—and I do mean never—given a single dime to any organization in response to a telephone call. Individual organizations that I do support have stopped calling in response to direct requests (and I will permanently cut off organizations that fail to heed those requests), but I still get phone calls from all sorts of outfits that I know nothing about and don’t care about.

    What I want to know is why they’re calling. I can guarantee that it’s a complete waste of their time and money. I absolutely never make a donation to any outfit that phones. I never have, and I never will. I do respond (very occasionally) to snail mail requests. So why haven’t the marketing folks noticed this? Why am I getting less paper mail and more phone calls?

    (Last year, I had to tell an organization that this was my second request for them to stop the phone calls, and that if they couldn’t figure out how to stop calling me, then the money that I was going to send them would be going to the Electronic Frontier Foundation, which has figured out how to stop phoning me. It produced immediate and sustained results. I suspect that naming one of their “rivals” as the planned beneficiary of their incompetence was motivating to them.)

    Thumb up 0 Thumb down 1
  7. Arturia Locke says:

    I just wanted to say: wasn’t asking the question “How does it feel to be black and poor?” the start of a very fascinating research project, so to speak?

    Social networks can never substitute in sociological research for the importance of individual truthful anecdotes. But research thankfully is no zero sum game.

    The importance of establishing trust in field research with individuals when probing sensitive questions that may be obvious is unquantifiable.

    Thumb up 0 Thumb down 0