How Biased Is Your Media?: A New Freakonomics Radio Podcast

Listen now:

(Photo: Jon S)

When it comes to politics and media, the left argues that the right is more biased than the left while the right argues that the left is more biased than the right. Who’s right?

That’s what we try to answer in our latest podcast, “How Biased Is Your Media?” (You can download/subscribe at iTunes, get the RSS feed, listen live via the media player above, or read the transcript below.) In a way, this episode is a follow-up to a podcast we put out a few months ago called “The Truth Is Out There, Isn’t It?,” which examined how we choose to believe what we believe about a variety of important issues. In this episode, we apply that same idea in a small-bore fashion, going after media bias.

You’ll hear from a variety of media practitioners and academic scholars who’ve been brave (foolhardy?) enough to wade into the media-bias debate. Among the practitioners: Glenn Beck (who’s been on Freakonomics Radio before), Ann Coulter, Juan Williams, and Andrew Rosenthal, the editorial page editor at The New York Times.

Everyone of course has his opinion about media bias, but we were trying to get beyond opinion. As Steve Levitt points out, this is no simple matter:

LEVITT: Measuring media bias is a really difficult endeavor because unlike what economists usually study, which are numbers and quantities, media bias is all expressed in words.

So we look at some of the recent empirical work on media bias, in which research scholars use words as data to better understand whether a) media bias exists; b) if so, to what degree, and in what directions; and c) what purpose/s it serves. In a 2004 paperTim Groseclose and Jeff Milyo took a stab at media bias; that paper became the launching point of Groseclose’s book Left Turn: How Liberal Media Bias Distorts the American Mind. You’ll hear from Gloseclose about his methodologies and findings, and you can read an earlier Q&A with him hereHere’s how Levitt has described the Groseclose-Milyo analysis:

LEVITT: Tim Groseclose and Jeff Milyo estimate how left-wing or right-wing media outlets are based on what research by think tanks they mention in their stories. They then compare that to the think-tank research that elected officials cite when they talk on the House or Senate floor, to calibrate where the media fits relative to the Congress. They find some interesting answers: most of the media does have a liberal bias (throwing out the editorial page, the Wall Street Journal is the most liberal of all, even beating the New York Times!). Fox News is one of the few outlets that is right of center.

Here’s how 20 major media outlets rank on Groseclose and Milyo’s slant scale, with 100 representing the most liberal and zero the most conservative:

  ABC Good Morning America 56.1  
  ABC World News Tonight 61.0  
  CBS Early Show 66.6  
  CBS Evening News 73.7  
  CNN NewsNight with Aaron Brown 56.0  
  Drudge Report 60.4  
  Fox News Spec. Rept. w/ Brit Hume 39.7  
  Los Angeles Times 70.0  
  NBC Nightly News 61.6  
  NBC Today Show 64.0  
  New York Times 73.7  
  Newshour with Jim Lehrer 55.8  
  Newsweek 66.3  
  NPR Morning Edition 66.3  
  Time Magazine 65.4  
  U.S. News and World Report 65.8  
  USA Today 63.4  
  Wall Street Journal 85.1  
  Washington Post 66.6  
  Washington Times 35.4  

And here’s a chart of some well-known Congressmembers’ “Political Quotients,” based on their voting records. Again, 100 represents the most liberal, zero is the most conservative. (In the episode, you’ll hear where Levitt, Groseclose, and I rank on this “PQ” scale. You can take a quiz to find out your own PQ here.)

University of Chicago economists Matthew Gentzkow and Jesse Shapiro have also done some interesting research on media bias. In a 2010 study, they used text as data to look at common Democratic and Republic phrases in Congress to help determine which way newspapers lean — and, most important, why. Here are some common Democratic phrases:

And some of the phrases favored by Congressional Republicans:

In this episode, you’ll also hear from Danny Kahneman, author of Thinking, Fast and Slow, on why bias is hard for each of us to see.


Tony King

I love reading Freakonomics columns but this is one of your worst. I have studied media bias at length, and you fall right into the trap in this one. The problem with this approach is with a conservative congress in 2004, anything remotely critical becomes a left wing media bias. The same could be done to show a right wing bias with a democratic congress.

The fundamental problem with bias studies is to find a bias, one must first know reality. Too often, today, we take 2 talking heads with an agenda, put them together, and let them spin and call it analyzing the story. They aren't analysing anything. They are trying to spin for political gain and all facts went out the window after the 10 second story snippit. So if the truth is left or right of the ideology of the nation, then the truth begins to be seen as "biased" even if it is true. In such a case, proclaiming the world is round and the govt should recognize it becomes a left wing bias; proclaiming the world is round and the private sector will prove it becomes a right wing bia. But both focus on the bias of approach instead of the error... The world is round, not flat.

The question is how do you get at the truth of the story to find good bias data? I have yet to find a good, sound methodology for finding bias, which is why we still create measures that the average person doesn't understand, but allow both sides to shout "BIAS!". Maybe the answer isn't in a simple bias matrix but rather in understanding the profit strategies of today's media. After all, they aren't loss leaders of the 70s anymore, they are big business of the 21st century, the information age. (Hint, hint)

Read more...

J

I'd argue with some of your points in the last paragraph, but instead let's agree a media source that tells us they're unbiased has already lied to us once.

Zach Brannan

Not to be a stickler, but your graphic cites Barney Frank and JFK representing my home state of Mississippi. I think you meant "MA".

164

Is that a mistake, the WSJ at 85.1more liberal than the New York Times?

Simonsez

Very disappointing episode. Judging "slant" by the mere presence of phrases, with absolutely no regard for the context in which those phrases are used, is absurd. According to this methodology, if a pundit were to state, "Poor people don't deserve any tax breaks. Increase taxes on the middle class if you want to see economic growth. Cut funding on those living in poverty and cut health care if you want to fix the budget deficit", this methodology would evaluate that pundit as a far-leaning liberal.

I would have expected a little more intellectual skepticism from you guys when a guest comes on spouting such complete nonsense.

Allen Hughes

I only have to look at one issue to know the media is incredibly biased toward liberal agenda. When gas prices were at their high points during Bush years, every day we heard nothing but the steady drumbeat of "what is Bush going to do about this regressive gas price" problem. The poor are hit especially hard by it yadayadayada. It is reported today on a rare occasion, but never with any emotional display and utter disdain for government causing it

James

What you see here is not really a left-right bias, but a variant on the boiling frog effect. What attracts media attention is not stasis, but change: the higher the rate of change, the more attention it attracts. So when gas prices are increasing rapidly, that attracts media attention. When prices remain static (or decrease slowly, as they have been until recent weeks*), that's not news, regardless of whether the base price is low or high.

*Indeed, if the media were notably biased to the left, one would think we'd have seen weekly stories along the lines of "Obama policy success, gas prices drop again this week! (0.1 cent, but who's counting?)"

Wilson

What I never see mentioned in these discussions is something I call the "Cubs fan" effect. Daniel Kahneman probably has a better name for it. It's like this: it's an article of faith among Cubs fans that sportscasters on national TV networks are biased against the Cubs. So whenever a game is broadcast on Fox or NBC, you can guarantee you will hear critical comments about the Cubs' players and management.

I believed this too when I lived in Chicago, so when I moved to St. Louis I was surprised to find that Cardinals fans feel the same way: network sportscasters are biased against their home team too! It turns out it's a universal phenomenon.

The way I see it is, we don't think too much about comments that accord with our own beliefs. We let those pass by, thinking "well of course!" But when we hear something that clashes with our beliefs, we really notice it.

lutraphobe

Here's another way to put it:

If 80% of a report agrees with my view and 20% doesn't, I'd view that report as 100% biased. If 100% or a report agrees with my view, I'd call that 0% biased.

edel

ABC, CBS, USA Today, Washington Post, wow... virtually all media are "Liberal"!
That should pause us for a second to see that something is not right here; literally!

I don't doubt in how the study was conducted, but that definitely something else may be at play; Likely there is a disconnect of what conservative politicians talk about and what the society (press) care about, even conservatives.

Is a bit like the catholic church talking against anti-conceptive methods while catholics used them as much as any one else. Should we come to the conclusion so that 98% of catholics aren't catholics? No, just that their leaders are totally disconnected with their followers on their main topics of discussion, unlike other religious groups.

Sorry, but most conservatives I know could care less about "embryos", "stem cells" and "pass the bill" while they do about "credit card", "middle class" and "budget deficit" as any one else are.

Read more...

life is biased

I think this is a great article and points out some of the issues of trying to identify bias in several different venues. I think you see this even manifest in the comments section. While reading some of these, it occurred to me that a rather open look at a study conducted on potential bias, comes across to some as being “very disappointing episode.”

Then when you read the reasoning behind your comment, you see that it is based clearly on underlying biases for what answer they were hoping the study would find. There isn't really a sound argument against the article for how the studies were conducted, and yet the whole article is prefaced by the statement "Measuring media bias is a really difficult endeavor because unlike what economists usually study, which is numbers and quantities, media bias is all expressed in words."

While the article is talking about analyzing data to find markers which point to sentiment behind a statement, comments reply with rather grandiose and immeasurable statements like “The fundamental problem with bias studies is to find a bias, one must first know reality.”

It seems that the sentiment behind these comments is the core of the issue that the article is trying to illustrate. When you are looking at data that consists of points which are more complex than say a number, then results are difficult to ascertain. While the article clear points this out, it offers several studies to illustrate certain trends which are present in speech that MAY have some correlation to sentiment.

To most people media bias is somewhere along the lines of Justice Potter stating that “I’ll know it when I see it.” Yet, of the studies there are some surprises which I did not see coming – the left leaning WSJ and the left leaning Drudgereport?

It is an interesting problem to consider measuring sentiment in an unbiased way; I appreciate the article in considering the complexities in such a short space. While measuring bias may be difficult, it seems more difficult to remove your own filter and allow the data to present its own story.

Read more...

Seb

I guess it depends on where you put the center. Seing Ron Paul on the left makes me seriously doubt about either the methodology or where the center is. From Michelle Bachman perspective, everything is on left and from a real communist everything is on the right.

I'm canadian and from my perspective, the bias seems heavily on the right.

Also, I guess there should be a disctinction about social issues and economic issues. I always find odd that people on the right believe in free market and don't want the government to intervene, except when it's issues like abortion, gay marriages and military actions.

Personnally, I'm mostly on the right on both, live and let live.

UmYeah

I find it very humorous that the even the scoring scale is "biased." Why not Conservative=100 and Liberal=0?

In most applications a higher score is better such as; Grades, Sports, Return on investment, Viewership, Voters, Credit score, etc... In fact, I can only think of two counter situations; Golf and time based speed contests like running or swimming.

With a little thought they could have made it non-biased... Ideas from the readers?

Darren

I find these bias estimates to be incredible. They seem to rely on one fundamental assumption: Democrats and Republicans cite to equal amounts of think-tank research when they talk on the House and Senate floor. This is the only realistic way that the media could be expected to cite to an equal number of studies in their reports that were used by Democrats and Republicans. If, in fact, Democrats cite to more research in House and Senate debates, the sources they cite are more likely to be referred to in news reports than the smaller number of sources cited by Republicans. This would show up in this study as a "left-wing" bias. Do these bias rankings merely indicate that Democrats rhetorically rely on studies, while Republicans rely on other techniques? Or that Republicans rely on studies, but don't cite to them explicitly?

ben

You didn't ask the NY Times Editorial guy (Andrew Rosenthal) about his Political Quotient.

Bias?

Awesome episonde BTW. Thanks

Marc

While correlation is a wonderful tool, I think several of the phrases chosen as Liberal or Conservative do not pass the "sniff test". Simple noun phrases like "American people", "Republican senators", "oil companies" , "budget deficit" , "federal spending", "border security", etc. while they may be avoided by strong partisans in Congress, are simply required to discuss issues of the day. If there is a news outlet which does *not* use these phrases, they are simply not covering the news.

Martin

Why use Ann Coulter as the conservative spokesman? She is an entertainer - interested in creating controversy and promoting herself. Why not get someone serious? She has described herself as a polemicist. She is an embarrassment to conservatives.

Travis

I was with most of the talk until I heard that the average voter's PQ was assumed to be around 30, which is more conservative than many of the politicians measured. For some reason, that brought the whole methodology into question for me, perhaps because it's fairly hard to believe.

Also, discussing "bais" without discussing "accuracy" feels like a fools errand. If you have two think tanks, one traditionally "conservative" and one viewed as "liberal," it seems the citation or reliance on one or the other could be, in part, due to the accuracy of the think tank and not solely its political leanings. You end up with a situation where you have a very accurate presentation of something, but is viewed as "biased" because the source is viewed as biased.

Eric M. Jones.

I have three word suggestion for anyone who want to get pretty-much unbiased news:

BBC

Ben

Wall Street Journal more liberal than NYT? This doesn't pass the laugh test.