A Great Example of Bias Within Academia

It is amazing how good we are — even the smartest, most rational people among us — at not recognizing our own biases. (Danny Kahneman memorably calls this being “blind to our blindness.”)

We recently put out a podcast called “The Truth Is Out There … Isn’t It?” about how people decide what to believe about everything from global warming and nuclear risk to UFO’s. It was inspired by the research of Dan Kahan and his colleagues at the Cultural Cognition Project; they have found that we systematically filter our beliefs through our personal and political ideologies. In other words, we allow our biases to influence what we think about theoretically non-ideological issues, but we aren’t aware of that influence.

We’re also working on an upcoming podcast about media bias, which will feature Tim Groseclose (author of Left Turn) and a cast of thousands. Once again, we bump up against the issue of people making seemingly objective judgments that are based, in some large part, on their subjectivity.

If you are at all interested in these kind of bias stories, and especially if you care about the realm of academic economics, you’ll definitely want to look at a new paper by Christis Tombazos and Matthew Dobra, who looked for bias within their own field. The paper (PDF here) is called “Using a Voting Mechanism to Evaluate the Quality of Research in Economics: Lessons from the Australian National Research Assessment” (emphasis added):

As part of the Australian National Research Assessment, the nation’s 133 most senior academic economists participated in a voting process that assigned quality ratings to almost a thousand journals of economics. The ratings were applied on the nation’s 975 academic economists’ publications retroactively by a number of institutions for a variety of purposes. The government used them to rank Universities and to distribute research funds. And Universities used them in hiring decisions, and the determination of salaries and publication bonuses. This study investigates the determinants of voting decisions. We find that voters are influenced by objective measures of journal quality. However, we also find strong evidence that, other things equal, voters assign the highest possible quality rating to journals in which they have published. They also overstate the quality of journals to which they have special access while understating the quality of journals that fall primarily in the fields of expertise of their 842 non-voting colleagues, or in which these non-voting colleagues have published.

Leave A Comment

Comments are moderated and generally will be posted if they are on-topic and not abusive.

 

COMMENTS: 18

View All Comments »
  1. scott sabol says:

    Great post. I also wrote an article on the biases when it comes to weather and weather forecasting in the eyes of the viewing public.

    The Cognitive Dissonance of weather forecasting is very difficult to overcome. Its a battle meteorologists can’t win. The human condition is too powerful.

    http://sabolscience.blogspot.com

    Well-loved. Like or Dislike: Thumb up 14 Thumb down 1
    • Mike B says:

      Wasn’t most of the experimental evidence confirming the existence of Cognitive Dissonance determined to have just been implementations of the Monty Hall Problem?

      Thumb up 1 Thumb down 2
  2. Mike B says:

    Why are you giving Tim Groseclose another bite at the apple? His last Q&A was rife with faulty reasoning and personal assertions posing as fact with little if any substantive research behind them. It is not being fair and balanced to cancel out one bias with another bias. Try fighting bias with the truth.

    Thumb up 5 Thumb down 7
  3. Eric M. Jones. says:

    Great post. And no surprise. You get prizes and awards from those with whom you agree….even when you are all full of beans.

    Scott: Cognitive Dissonance of weather forecasting….

    Just a note: Pilots bet their lives and the lives of their passengers on weather forecasts. They rarely, if ever, criticize the forecasters.

    Well-loved. Like or Dislike: Thumb up 14 Thumb down 3
    • scott sabol says:

      Eric. Very true. I know several pilots whose livelihood depends on weather forecast :) However, its the general television public who I was referring to.

      Thumb up 3 Thumb down 1
  4. Ben says:

    How do you know Tim Groseclose isn’t biased about the media? After all, you have your right wing (economically anyway) ideological filter.

    Thumb up 7 Thumb down 10
    • James says:

      I can’t help but wonder how we can possibly tell the difference between having a right-wing ideological filter, and not having a left-wing one? That is to say that a person with a left-wing bias will perceive a perfectly neutral person to have a right-wing bias. Or vice versa, of course.

      I get this frequently on another forum, where the right-wingers accuse me of being liberal even though I consider myself more of a conservative libertarian than anything.

      Well-loved. Like or Dislike: Thumb up 15 Thumb down 3
    • Joe J says:

      He explained his methodology in his papers and book. If you have comments or complaints about his methodology, please state them. Others have done similar studies with differing methods and come up with similar results.

      Thumb up 5 Thumb down 2
  5. James says:

    “…almost a thousand journals of economics.”

    Well, that certainly smacked one of my cognitive biases. I’d have guessed maybe a couple of dozen journals, and would have expected the total number of economists to be somewhere in the low thousands.

    Well-loved. Like or Dislike: Thumb up 12 Thumb down 1
  6. Travis says:

    Hidden due to low comment rating. Click here to see.

    Disliked! Like or Dislike: Thumb up 9 Thumb down 29
    • twobeef says:

      This is also the paradox of the political donor. Do large businesses donate to a politician in order to sway their behavior, that politicians don’t have set views until someone gives them money for it? Or do they donate to that politician because the politician already wanted to vote a certain way and they just support the candidate that agrees with them? I’d be willing to bet it’s a bit of C: All of the Above, but it’s also hard to separate the two.

      Thumb up 0 Thumb down 3
      • Joe J says:

        Most big buisnesses donate to both candidates, so B is most likely not the answer. It is often seen as more buying future probably needed time, rather than a for or against a issue.
        As in the case of a Senator, I don’t know of any specific bills coming up over the next 6 years will have anything to do with my company, but a million donation to each candidate now means if such a vote comes up , whoever is Senator will favorably listen to me.

        Thumb up 1 Thumb down 1
    • method1220 says:

      Hidden due to low comment rating. Click here to see.

      Disliked! Like or Dislike: Thumb up 0 Thumb down 29
  7. ryan says:

    Hidden due to low comment rating. Click here to see.

    Disliked! Like or Dislike: Thumb up 4 Thumb down 16
  8. Allie says:

    I agree that we allow our biases to influence what we think about non-ideological issues, but we aren’t aware of that influence. We also let the media and our friends influence our bias. This sometimes makes it hard to see the clear picture because our biases get in the way.

    Well-loved. Like or Dislike: Thumb up 12 Thumb down 1