A Great Example of Bias Within Academia

It is amazing how good we are — even the smartest, most rational people among us — at not recognizing our own biases. (Danny Kahneman memorably calls this being “blind to our blindness.”)

We recently put out a podcast called “The Truth Is Out There … Isn’t It?” about how people decide what to believe about everything from global warming and nuclear risk to UFO’s. It was inspired by the research of Dan Kahan and his colleagues at the Cultural Cognition Project; they have found that we systematically filter our beliefs through our personal and political ideologies. In other words, we allow our biases to influence what we think about theoretically non-ideological issues, but we aren’t aware of that influence.

We’re also working on an upcoming podcast about media bias, which will feature Tim Groseclose (author of Left Turn) and a cast of thousands. Once again, we bump up against the issue of people making seemingly objective judgments that are based, in some large part, on their subjectivity.

If you are at all interested in these kind of bias stories, and especially if you care about the realm of academic economics, you’ll definitely want to look at a new paper by Christis Tombazos and Matthew Dobra, who looked for bias within their own field. The paper (PDF here) is called “Using a Voting Mechanism to Evaluate the Quality of Research in Economics: Lessons from the Australian National Research Assessment” (emphasis added):

As part of the Australian National Research Assessment, the nation’s 133 most senior academic economists participated in a voting process that assigned quality ratings to almost a thousand journals of economics. The ratings were applied on the nation’s 975 academic economists’ publications retroactively by a number of institutions for a variety of purposes. The government used them to rank Universities and to distribute research funds. And Universities used them in hiring decisions, and the determination of salaries and publication bonuses. This study investigates the determinants of voting decisions. We find that voters are influenced by objective measures of journal quality. However, we also find strong evidence that, other things equal, voters assign the highest possible quality rating to journals in which they have published. They also overstate the quality of journals to which they have special access while understating the quality of journals that fall primarily in the fields of expertise of their 842 non-voting colleagues, or in which these non-voting colleagues have published.

scott sabol

Great post. I also wrote an article on the biases when it comes to weather and weather forecasting in the eyes of the viewing public.

The Cognitive Dissonance of weather forecasting is very difficult to overcome. Its a battle meteorologists can't win. The human condition is too powerful.


Mike B

Wasn't most of the experimental evidence confirming the existence of Cognitive Dissonance determined to have just been implementations of the Monty Hall Problem?

Mike B

Why are you giving Tim Groseclose another bite at the apple? His last Q&A was rife with faulty reasoning and personal assertions posing as fact with little if any substantive research behind them. It is not being fair and balanced to cancel out one bias with another bias. Try fighting bias with the truth.

Eric M. Jones.

Great post. And no surprise. You get prizes and awards from those with whom you agree....even when you are all full of beans.

Scott: Cognitive Dissonance of weather forecasting....

Just a note: Pilots bet their lives and the lives of their passengers on weather forecasts. They rarely, if ever, criticize the forecasters.

scott sabol

Eric. Very true. I know several pilots whose livelihood depends on weather forecast :) However, its the general television public who I was referring to.


How do you know Tim Groseclose isn't biased about the media? After all, you have your right wing (economically anyway) ideological filter.


I can't help but wonder how we can possibly tell the difference between having a right-wing ideological filter, and not having a left-wing one? That is to say that a person with a left-wing bias will perceive a perfectly neutral person to have a right-wing bias. Or vice versa, of course.

I get this frequently on another forum, where the right-wingers accuse me of being liberal even though I consider myself more of a conservative libertarian than anything.


"...almost a thousand journals of economics."

Well, that certainly smacked one of my cognitive biases. I'd have guessed maybe a couple of dozen journals, and would have expected the total number of economists to be somewhere in the low thousands.


Is this really a bias, or a surprising one? After all, aren't academics more likely to spend effort reading / publishing in journals which they think are the highest quality?


This is also the paradox of the political donor. Do large businesses donate to a politician in order to sway their behavior, that politicians don't have set views until someone gives them money for it? Or do they donate to that politician because the politician already wanted to vote a certain way and they just support the candidate that agrees with them? I'd be willing to bet it's a bit of C: All of the Above, but it's also hard to separate the two.

Joe J

Most big buisnesses donate to both candidates, so B is most likely not the answer. It is often seen as more buying future probably needed time, rather than a for or against a issue.
As in the case of a Senator, I don't know of any specific bills coming up over the next 6 years will have anything to do with my company, but a million donation to each candidate now means if such a vote comes up , whoever is Senator will favorably listen to me.


Again- why do economists think they are the only ones who investigate phenomena? Why not just look up cognitive dissonance?


I agree that we allow our biases to influence what we think about non-ideological issues, but we aren’t aware of that influence. We also let the media and our friends influence our bias. This sometimes makes it hard to see the clear picture because our biases get in the way.


yes, UFO. We have a picture of one now...or do we....just because they...if they are in the picture that we took outside of Pluto...they may not even come....so if they dont come....that surely means that they dont exist....gotta go to church now.....


Well, we're getting a lttlie off-topic here, but if you think people don't actually believe the gambler's fallacy, I think there's good evidence to the contrary. Kahnemann and Tversky (Judgment Under Uncertainty, 1982) make it a subset of the local representativeness belief (at page 7). See also Lindman and Edwards (1961), Journal of Experimental Psychology. Ten minutes in a casino will lead to the same conclusion . Watch people who write down long strings of roulette results and watch what they do. Note that, contrary to Carl's hypothesis, people who write down these strings, before they start gambling, would have no reason to bet for or against red on their first bet if there had been a long streak of reds. In fact, I've watched them, and their first bet under these circumstances is almost invariably black, which suggests that, if anything, the gambler's fallacy is somewhat stronger than the hot hand fallacy. I have no doubt that people who want to gamble will create rationalizations for their behavior, but those rationalizations, so long as they are predicated on the notion that the odds can be turned in their favor in a purely mechanical game are pretty close to the definition of bias.

[WORDPRESS HASHCASH] The poster sent us '0 which is not a hashcash value.