How to Spot Advocacy Science: John Taylor Edition

Photo: iStockphoto

Sometimes you see the perfect piece of evidence. The scatter plot that is just so. The data line up perfectly. And then you realize, perhaps they’re just too perfect. What you are seeing is advocacy, dressed up as science. Here’s an example, provided by John Taylor (via Greg Mankiw):

 

 

Taylor’s conclusion:

The data on spending shares show that the most effective way to reduce unemployment is to raise investment as a share of GDP.

But why begin the scatter plot in 1990? There’s no good reason. In fact, most folks typically download the entire history of available macro data. So let’s see what happens if we extend it back to, say, 1970:

 

Hmm… What conclusions should we draw about this relationship? And now why do you think Taylor began his sample in 1990?

Actually, we should use all the available data. The chart below goes back to 1948, when these series—in their current form—began:

 

Now what’s your conclusion?

Here’s Mankiw’s assessment of Taylor’s claim:

There’s no doubt that the strength of the correlation is impressive.


But when you look beyond the cherry-picked sample, the correlation is a decidedly unimpressive -0.14.

Here’s my conclusion: On balance, times in which the investment share is higher, are slightly more likely to be good times. But I’m not sure why. Is it—as Taylor asserts—that high investment shares create good times? Or is it that good times encourage investment? Or is it a third factor—perhaps in good times the government doesn’t need to prime the fiscal pump, and so the investment share is higher? Or is it something else?

Be wary of economists wielding short samples.

Leave A Comment

Comments are moderated and generally will be posted if they are on-topic and not abusive.

 

COMMENTS: 37

View All Comments »
  1. Anna says:

    The NYT publishes advocacy, junk science polls and studies all the time, unfortunately. Everybody in the media has an agenda to push, to the detriment of our country’s future well-being.

    Thumb up 0 Thumb down 1
    • Joshua Northey says:

      Why single out the NYT (which I am not huge fan of personally).? Many journals publish “advocacy science” (I am not sure that is a useful category). It is the nature of the beast. If you don’t include narrative urgency into your articles how on earth are you going to drum up more funding for your next round of research? If journals don’t publish articles with narrative urgency how on earth are they going to sell copies?

      So much science that is published takes the form of:

      wild speculation based on conclusion A
      evidence A
      argument/science
      valid conclusion A
      wild speculation based on conclusion A

      You find out that polar bear populations are decreasing. You use science to discover this is due to the ice receding. Then you couch the whole thing in a narrative about how climate change is going to destroy the world’s habitability. Which is a complete nonsequitur because it is not at all clear that this disappearance of polar bears is even going to noticeably harm the productivity of arctic ecosystem, much less make the world uninhabitable.

      Most science I encounter is great, but the conclusions and speculations that accompany that science are as biased as the ones you get from any random person.

      Thumb up 0 Thumb down 1
  2. Michael Fisk says:

    Could part of the difference be a function of differences in monetary policy and money demand from various eras? There is the possibility that, due to changes in capital markets, a correlation could exist now that didn’t exist before. But yes, the correlation is almost suspicious, even more so when there’s no firm conclusions to be drawn from it.

    Thumb up 0 Thumb down 0
  3. Clancy says:

    Wow, I just read Krugman’s equally thorough take-down of Taylor’s graph. http://krugman.blogs.nytimes.com/2011/03/30/whats-behind-low-investment/

    But Krugman takes a different angle and points out the reason for the drop in investment is mostly the real-estate bust.
    Guess today was a bad day to post misleading charts.

    Thumb up 0 Thumb down 0
  4. Wendy says:

    I am not an economist so I don’t really know anything about spending shares but as a scientist I would say that something really interesting happened in the 70′s and 80′s. The rest of the data basically fits Taylor’s pattern so a more interesting question would be: why was there such high unemployment in the late 1970′s and 1980′s despite the fact that the investment to GDP ratio was relatively high? This isn’t cherry picking it is just figuring out when a pattern holds and when it doesn’t.

    Thumb up 0 Thumb down 0
  5. Joshua says:

    Taylor might well be right if his statement is appropriately restricted. It looks like there is very little, if any, correlation prior to 1990, followed by phenomenal correlation. Given that stark contrast it might well be that something changed around 1990 that led to high correlation between GDP and investment.

    Of course, even under this interpretation Taylor would still be at fault for presenting the data as though they were representative.

    Thumb up 0 Thumb down 0
  6. Wendy says:

    Alternatively you could ask what the heck happened in 1990, because before that it looks like the pattern might have been the opposite of what Taylor describes. Simple linear regression isn’t always the solution, sometimes you have to break the data into chunks to figure out what happened when.

    Thumb up 0 Thumb down 0
  7. jonathan says:

    Putting aside Taylor’s conclusion, which is silly because he ignores housing investment’s cliff dive, I’d love to see this kind of data mapped against traditional correlations for investment, notably interest rates and inflation. What I see in Taylor’s bit and then your graphs is another example of the oddity of the current situation, with zero rates not creating investment – despite the theoretical arguments of Austrians et al – and thus this weird kind of wiggling around by people unwilling to accept that the data says their models are wrong. Thus, we aren’t seeing investment so therefore rather than accept our models are wrong, we blame something inchoate floating in the air. That isn’t economics. We can translate that kind of thing into economics through surveys that measure confidence and those say that demand worries (sales) are 3x normal while government-related issues (regulation, taxation) are roughly the same as normal. But rather than accept that data, the choice is to pick that inchoate thing for which there is only hand-picked anecdotal evidence.

    Thumb up 0 Thumb down 0
  8. serge d'agostino says:

    these stuffs show that economics are historical science: which is (maybe) true from 1990 to 2010 (or which is a good correlation), can be false (or bad correlation) between 1948 and 1990, etc.

    Thumb up 0 Thumb down 0