A Twitter Experiment

I’m a long-time Twitter skeptic.  It’s difficult for an economist to see a 140 char lmt as a ftr.  My journalist friends tell me I’m dead wrong.  And a recent long and boozy evening with co-founders Evan Williams and Jason Goldman convinced me to give it a try.  Is Twitter worth the hype?  Let’s find out.

Today I’m beginning my Twitter Experiment. I’m now tweeting @justinwolfers. I’m going to keep this up for a couple of weeks as a “burn in” period—basically so that I can learn the ecosystem before my experiment begins. Then on the morning of August 1, I’m going to wake up, and flip a coin. Heads, I’ll open Twitter; tails I won’t.  And I’ll do the same on August 2, and then every day for three months. If the coin comes up heads, it doesn’t necessarily mean that I’ll tweet, just that it will be a Twitter-aware day; I’ll consume the stream, and tweet away if I feel the need. Tails, and I’ll simply tweet “Tails, goodbye,” close the stream (unless I need it for research) and then resist the urge to tweet for the rest of the day.

So far, designing the experiment is easy. But what are the right outcome variables to track? I can think of three things that Twitter does:
1.      Distracts me, sucking valuable time from serious research.

2.      Broadcasts me, perhaps increasing my influence.

3.      Informs me, which may make me a better economist.

Here’s my question for you: How can I measure the influence of my Twitter Experiment on each of these outcomes? I think measuring distraction is easiest—I’ll just evaluate how productive I was each day, using a subjective 0-10 scale. I have some ideas about measuring influence, but none are particularly compelling. And I’m baffled by how I might measure the extent to which Twitter is informing me, making me a better economist.

Crowdsourcing can surely improve my experiment. Do you have any ideas for how better to measure distraction? Or ideas for useful indicators of influence or impact that I can track?  And how best to measure how informed I am? And does anyone else want to join me in this experiment? More data = more insight.

Please add your ideas in the comments.  Or if your idea is ez 2 expln, just tweet me.  #twitexpt.

Leave A Comment

Comments are moderated and generally will be posted if they are on-topic and not abusive.



View All Comments »
  1. Lee says:

    “Tails, and I’ll simple tweet “Tails, goodbye,” close the stream (unless I need it for research) and then resist the urge to tweet for the rest of the day.”

    Unless you need it for research. I think you just demonstrated some value right there.

    Well-loved. Like or Dislike: Thumb up 15 Thumb down 1
    • BSK says:

      The experiment is already undermined if you are going on it on the days you are not supposed to. If you’re off Twitter, you should be entirely off Twitter. Otherwise, the whole thing is bunk.

      Well-loved. Like or Dislike: Thumb up 14 Thumb down 0
  2. KevinH says:

    for 1, there are actually timers available for the computer that will help you track and objective number of hours spend on twitter. I think the subjective scale would also be useful, but it would also be interesting to both the distribution of time spent with twitter and if the time spend on a daily basis correlated with any other factors.

    for 2, can you track web hits to blog posts by you? It wouldn’t measure professional influence, but it would be a decent variable to track.

    Likewise for 3, you could count the number of articles you read each day with a little tally sheet. I’d say you could do 2 tallies, one for academic articles and one for non-academic articles.

    Thumb up 2 Thumb down 0
  3. Michael Vukovich says:

    Influence: how many people follow you and retweet what you say…a good measure for how much you are spreading your word to the masses. Also check out #, if you’re hashmarked often it means people are tagging you in a lot of posts.

    Informing: possibly keep track of how often you ‘use’ information you discover on twitter? Subjective I know however you’re already using a subjective scale for productivity

    Thumb up 0 Thumb down 0
  4. raulbajob says:

    Justin, I cannot see why you are going to flip a coin?¿?
    If you just want to measure distraction, use a stopwatch! And a a possible way to measure your influence is, for example, 1 point for each new follower in a day, 2 points for each metion, and 3 points for each retweet. At the end of each day, just sum the points up.

    Thumb up 6 Thumb down 2
  5. Sol says:

    I think your experiment is very interesting, I am also skeptic about twitter.
    I’d measure the influence considering the number of followers. I’d study how your tweets affect the number of followers, whether your tweets increase or decrease it. I’d even intentionally tweet both “interesting” and “stupid” comments, that is to know if being boring and uninteresting makes people stop following me.
    And of course I would also consider how many replies you get for every tweet. If you can make someone reply you about anything, then I guess you can assume you had some influence on him (which is just what has happened to me, replying at this moment).
    Hope this helps!

    Thumb up 1 Thumb down 0
  6. AZ says:

    I see two problems with recording productivity. As already mentioned, it’s highly subjective (and potentially affected by your knowledge of which kind of day it was). But I would also think that productivity is so variable, and so subject to other influences, that it would be unlikely that an experiment with a relatively small N (even 92) would reveal much.

    Thumb up 4 Thumb down 0
  7. Kathleen Lisson says:

    ‘Distract’ and ‘Inform’ variables both depend on which twitter feeds you are following.

    It may be interesting to measure the level of feedback in an hour that you get on a question posted both on this blog and on twitter. With twitter, your question could be retweeted, giving you potentially increased feedback from a wider community not usually exposed to this blog.

    Thumb up 1 Thumb down 1
  8. Kaia says:

    Do you have any ideas for how better to measure distraction?
    : measure how many times twitter makes you to think about it during the day or limited time such as time for experiment or research. For example, you could think you’d like to check twitter three times for two hours research time. This means that you were bothered by twitter three times. (This sounds too subjective, tho. but I think ‘distraction’ is very sujective concept) As for me, twitter is bothering me almost all day long; I think I’m addicted to it :(

    Or ideas for useful indicators of influence or impact that I can track?
    : Use retweet function. People retweet other’s mentions when they are useful, educational, or just fun. And you are easily able to track not only which one was followed but also how many times the mention was followed. 

    Thumb up 0 Thumb down 0