> 0 How Biased Is Your Media? (Ep. 62) - Freakonomics Freakonomics

How Biased Is Your Media? (Ep. 62)

Listen now:

(Photo: Jon S)

When it comes to politics and media, the left argues that the right is more biased than the left while the right argues that the left is more biased than the right. Who’s right?

That’s what we try to answer in our latest podcast, “How Biased Is Your Media?” (You can download/subscribe at iTunes, get the RSS feed, listen live via the media player above, or read the transcript.) In a way, this episode is a follow-up to a podcast we put out a few months ago called “The Truth Is Out There, Isn’t It?,” which examined how we choose to believe what we believe about a variety of important issues. In this episode, we apply that same idea in a small-bore fashion, going after media bias.

You’ll hear from a variety of media practitioners and academic scholars who’ve been brave (foolhardy?) enough to wade into the media-bias debate. Among the practitioners: Glenn Beck (who’s been on Freakonomics Radio before), Ann Coulter, Juan Williams, and Andrew Rosenthal, the editorial page editor at The New York Times.

Everyone of course has his opinion about media bias, but we were trying to get beyond opinion. As Steve Levitt points out, this is no simple matter:

LEVITT: Measuring media bias is a really difficult endeavor because unlike what economists usually study, which are numbers and quantities, media bias is all expressed in words.

So we look at some of the recent empirical work on media bias, in which research scholars use words as data to better understand whether a) media bias exists; b) if so, to what degree, and in what directions; and c) what purpose/s it serves. In a 2004 paperTim Groseclose and Jeff Milyo took a stab at media bias; that paper became the launching point of Groseclose’s book Left Turn: How Liberal Media Bias Distorts the American Mind. You’ll hear from Gloseclose about his methodologies and findings, and you can read an earlier Q&A with him hereHere’s how Levitt has described the Groseclose-Milyo analysis:

LEVITT: Tim Groseclose and Jeff Milyo estimate how left-wing or right-wing media outlets are based on what research by think tanks they mention in their stories. They then compare that to the think-tank research that elected officials cite when they talk on the House or Senate floor, to calibrate where the media fits relative to the Congress. They find some interesting answers: most of the media does have a liberal bias (throwing out the editorial page, the Wall Street Journal is the most liberal of all, even beating the New York Times!). Fox News is one of the few outlets that is right of center.

Here’s how 20 major media outlets rank on Groseclose and Milyo’s slant scale, with 100 representing the most liberal and zero the most conservative:

ABC Good Morning America 56.1
ABC World News Tonight 61.0
CBS Early Show 66.6
CBS Evening News 73.7
CNN NewsNight with Aaron Brown 56.0
Drudge Report 60.4
Fox News Spec. Rept. w/ Brit Hume 39.7
Los Angeles Times 70.0
NBC Nightly News 61.6
NBC Today Show 64.0
New York Times 73.7
Newshour with Jim Lehrer 55.8
Newsweek 66.3
NPR Morning Edition 66.3
Time Magazine 65.4
U.S. News and World Report 65.8
USA Today 63.4
Wall Street Journal 85.1
Washington Post 66.6
Washington Times 35.4

And here’s a chart of some well-known Congressmembers’ “Political Quotients,” based on their voting records. Again, 100 represents the most liberal, zero is the most conservative. (In the episode, you’ll hear where Levitt, Groseclose, and I rank on this “PQ” scale. You can take a quiz to find out your own PQ here.)

University of Chicago economists Matthew Gentzkow and Jesse Shapiro have also done some interesting research on media bias. In a 2010 study, they used text as data to look at common Democratic and Republic phrases in Congress to help determine which way newspapers lean — and, most important, why. Here are some common Democratic phrases:

And some of the phrases favored by Congressional Republicans:

In this episode, you’ll also hear from Danny Kahneman, author of Thinking, Fast and Slow, on why bias is hard for each of us to see.


The PQ test is very partisan. It mostly includes questions to bills that Democrats favored, it tells you exactly how each party voted (not a blind test), and the way is written favors the Republican point of view.

When he confirms that most Americans are conservative, did he choose a random pool of people? Did he have control groups?

This PQ test reminds me of the stupid (non-scientific) poll of Fox News.


The media is biased more towards ratings than anything else. Historically, the mainstream news networks came off as a bit liberal because they were trying to sensationalize stories to make them more interesting to reel in viewers. Fox News came along to fill the demand for sensational rightwing stories. Entertainment and money: that is their agenda.

Also, I think you can readily see the bias by which stories the respective news agencies focus their attention on. Why don't you measure how much time they spend on stories that are damaging to a particular political group. That is a more objective way to measure than counting phrases and words out of context.


I find the "Political Quotient" scale to be absolutely absurd, far beyond the point where any conclusions drawn from it could be remotely considered useful. Tim Groseclose arbitrarily picks two politicians, Michelle Bachmann and Nancy Pelosi, as "most conservative" and "most liberal", when they'd best be labeled as "most Republican" and "most Democratic", scores them based on their votes on ideas that have had enough approval from the political process in Washington to become bills for them to vote on, and takes a MEDIAN score between their two results and declares that "centrist"! And then applies that ideal to the opinions of the country at large! Ridiculous.

Bachmann is anti-birth control, a concept that has a 98% approval rating among the American public. I don't know of any position Nancy Pelosi takes that has only 2% of the country agreeing with her. However, she's not the farthest "left" in congress- in case Mr. Glosecose forgot, we have an honest to god Socialist, Bernie Sanders.

As for the second portion of the show, I would've found the comparisons much more interesting with intelligently chosen terms instead of computer generated ones- for example, a Democrat's "estate tax" is a Republican's "death tax". A Republican's "Voter fraud law" is a Democrat's "Voter suppression law". A Democrat's "investment" is a Republican's "waste".



There seems to be a contradiction in two of the points made. Groseclose claims that most Americans are more Conservative than the news sources, but later it is stated that news sources are bias towards their constituents political positions. If that is the case, most news sources would be conservative.

Also, simplifying agendas by liberal and conservative is misleading. For example, almost all news sources have an agenda to promote consumerism- spurred largely by there advertising partners. What about the agenda to be politically correct, not so much because it is the right thing to do, but to not marginalize the media's customers. None of these fall into the conservative/liberal debate.


Congratulations! You have been perfectly played by the right-wingers, who have totally learned how to game the system. They scream bias and that everyone is a 20 (conservative), and you just parrot their claims.
How 'bout you study actual truth? E.g., how consumers of which news know the most actual facts? Nah, that wouldn't be "fair and balanced"!

Etienne Low-Decarie

Subject: Correlation vs causation

Dear Prof. Levitt and Dubner,

I love the closing statement of the podcast that accompanies this post. It is fascinating how, in US politics, opinions on two unrelated topics can so heavily be tied together, even fused beyond distinction, by the need to place opinions either far to the right or far to the left.
I have however a larger concern about this post/podcast and the research it present. In most of your blogs/podcasts, you highlight cases in which correlation from observational studies can not support interpretation of causation and often present fascinating experiments meant to tease causation away from correlation.
However, in this post/podcast, this pitfall of observational studies is not highlighted. From the same data, a dramatically different interpretation can arise. In this case, I would suggest the hypothesis that as whole, media is not biased and is presenting current facts and that the correlation between speeches of representatives and media reflects the amount of facts used by representatives to present their cases. On many issues in US politics, as you have often highlighted, facts are trivialized and seen as a hindrance. I would be fascinated by research that would investigate such an interpretation.
Please continue your efforts to distinguish facts from fiction, causes from possibly spurious correlations.



I think a major problem is that the PQ of politicians is determined in a very binary fashion across very different eras. The Republicans have become more conservative over the past decades, so that something that Nixon would have voted for in 1950 might actually count as being liberal now. Things that Sara Palin or Michelle Bachman say might be at a -100 on this chart if it had been created in 1975 instead of 2012. Also, this only shows their votes, but doesn't take into account a politicians stated ideas. So lets say a Senator wanted to let everyone into the US, no matter what country they were from or what skills they had, that Senator would be extremely liberal, but that bill would never even be up for a vote.

Dustin Richard

I’ll join the chorus – I usually love Freakonomics but giving this Groseclose guy any press for such a terrible “study” was a pretty poor choice.

The first problem is that of scale: I’m reminded of “Bias in Quantifying Judgements” by E.C. Poulton (thanks google for making most of it available here: http://books.google.com/books?hl=en&lr=&id=uC8ZidQVs48C&oi=fnd&pg=PR21&dq=poulton+1989&ots=U7ymHsnHgi&sig=NsrzEg2Cait43EOy0mhXMxzgVco#v=onepage&q=poulton%201989&f=false)

A good lesson to be taken from it is that if you weight the scale you can get any result you want. For example: Asking you the question “How often have you been dumped?” and creating a scale starting at 0 and ending at 2 or more is much more likely to depress you and classify you as unwanted than a scale that starts at 0 and ends at 10 or more.

Using a partisan think-tank from America limits the scale similarly in that it mirrors the very false dichotomies in thinking that Dubner himself outlined at the end of the podcast as being nonsensical. “Left” and “Right” are vestiges of the French revolution – and by the standards of those spunky old revolutionaries American politics score more accurately as “Right” and “Center-Right”. An examination of the history of American politics shifting right from the get-go and staying right would’ve grounded the discussion in reality a bit more.

These guys give a better “outside the box” take on politics that I’d normally expect from Freakonomics: www.politicalcompass.org/ . I’d imagine they’d do a much better job at coming up with a method for measuring slant than Groseclose.

Finally, though it’s been said by others I have to repeat it: using a catch-phrase is an unreliable method for determining slant. Many critics of the right use the language of the right when quoting and vice-versa with critics of the left. A great extreme example is Stephen Colbert – I’m sure an analysis would score him in the single digits of Groseclose’s scale and yet I don’t think he’s winning over hearts and minds into voting Republican.



The program started out pretty well, but then it took a left turn.

How much of the time did you give the NY Times guy to bash everybody who is not the Times? How much time did you give anybody who is on the right of the Times -- oh, right, just a hair above zero.

But you pretty well proved Groseclose's point.