We Had Better Get Our Next Book Out: John DiNardo Is Getting Bored

As mentioned on MarginalRevolution, the economist John DiNardo has been quite busy over the last few years criticizing Freakonomics. He has written no fewer than three papers on the subject.

It’s too bad that he didn’t offer the standard academic courtesy of sending his criticisms directly to me before writing them up; if he had, I could have helped clarify a few things.

For instance, his recent American Law and Economics Review article, “Freakonomics: Scholarship in the Service of Storytelling,” devotes three or four journal pages to an alleged set of sins committed in a paper I wrote with Dan Kessler back in 1999.

These criticisms are based on an embarrassingly bad piece of research by Cheryl Marie Webster, Anthony Doob, and Franklin Zimring. (I apologize: the only version I could find of the paper is gated.) If DiNardo had bothered to read my reply to that piece, I suspect he would not have written what he wrote.

And if he ever does bother to read it, perhaps we will consider publishing an erratum. Although I guess that would get in the way of his storytelling.

DiNardo also spends two pages asserting that we misrepresented a paper by Christian Pop-Eleches on the impact of banning abortion in Romania. DiNardo writes that Pop-Eleches finds “virtually the opposite” of what we report in Freakonomics. A quick comparison of the two will make clear just how shamelessly we distorted Pop-Eleches’ arguments. In Freakonomics, we describe Pop-Eleches’ work as follows:

The cohort born after the abortion ban would do worse in every measurable way: they would test lower in school, they would have less success in the labor market, and they would prove much more likely to become criminals.

Here is the abstract of the version of the Pop-Eleches paper that we cited:

…controlling for composition using observable background variables, children born after the ban on abortions had worse educational and labor market achievements as adults. Additionally, I provide evidence of crowding in the school system and some suggestive evidence that cohorts born after the introduction of the abortion ban had higher infant mortality and increased criminal behavior later in life.

The introduction of the Pop-Eleches paper says:

This finding is consistent with the view that children who were unwanted during pregnancy had worse socio-economic outcomes once they became adults.

“Virtually the opposite?” Sounds to me like Freakonomics and Pop-Eleches are saying the same thing. While it’s true that we cited a yet-to-be-published version of Pop-Eleches’s paper (which was the most current version available at the time we were writing Freakonomics), the conclusions of the published version (which appeared in the Journal of Political Economy, and which I edited) were unchanged.

C’mon, John, you’re a top economist, and our book is 300 pages long. You must have better criticisms than that! When we publish our next book, give me a call beforehand, and I’ll tell you what’s really wrong with it.


j

I had a landlord named Dinardo once. I bet they were related. He also lacked much of the same courtesy (though maybe not in the academic sense). :-)

Ajay

Prof Levgitt,

I totally concurr with #1. Some remarks aren't worth dignifying. The work speaks for itself.

Alex

I aguree with #8.
No need for political correctness, it's your blog. Write whatever you feel like. And if soneone publishes trash, you should call on it, doesn't matter if he is a top economist or the janitor.

elle

publicity, perhaps?

Tony

I am a huge fan of this blog. That said, I agree with others who take issue with the tone of this post. It reads like you are more interested in stroking your ego than responding to criticism.

Jason

The criticism was spot on.

Levitt should not have reduced the Pop-Eleches paper's result to that single paragraph because it clearly led readers to think that the kids born after the abortion ban had really miserable lives on average.

Finding out later that the raw averages actually go the other way, and only after running controls do the results revert, is simply not satisfying.

Pete

For what it's worth. I'm with #8. He should have had the courtesy to show you the paper before it was published.

Tony

To Mike (#23): what do you mean "ignorantly"? Have you even read DiNardo's critiques? You are blindly siding with Levitt based on what? His blog entry? The fact that a lot of readers are acting like groupies is really demeaning for this blog.

mgroves

While Dubner and Levitt are good, I wouldn't put them in the same league as Plato, Jesus, and Shakespeare. However, I would certainly concur that with great popularity comes great criticism.

B K Ray

I fall into queue with the first poster. While I do understand that you will want to scratch some hemmoroids harder than others (haha hemmoroids) Leavitt, Dubner and a few others have been placed, whether willingly or not, into a place above the fray. And when someone, who is above the fray engages in the fray, you kind of want them to be witty and maybe a little self-deprecating, just because it reaffirms our hope that it is not neccessary to stoop to someone else's level to squash them like a bug.

EmilyAnabel

The tone doesn't bother me- it is a standard courtesy to circulate drafts of related/derivative/challenging/critical work, and ignoring that standard professional courtesy speaks volumes about the personality involved.

Otto

Hold on...fair criticism is just that, but if this guy is demonstrably besmirching their good name with disingenuous research, then the authors are obligated to defend themselves.

AaronS

Levitt,

Look, you are not only an academic hero of many people, but are an all-around likeable fellow, as well.

Please be careful to not allow too strong of a sense of academic honor to cause you to play the "self-defense" game. Your reputation is secure. And besides, even such notables as Plato, Jesus, and Shakespeare have critics--do not expect that any notable work will be either perfect or beyond fair or unfair criticism.

I say rewrite this thread, taking out the more aggressive sort of statements that are so unlike your usual writing. Here's my proposal:

"Here is a criticism of Freakonomics by a top economist...here's my reply."

That's all. Let your noble work speak for itself...no need to do the Trump vs. Rosie thing--you're much better than that.

F

I fully cocurr with #1. Please take his suggestion and rewrite the post.

John

I am suprised Steven Levitt allows himself to be upset by criticisms of a fellow academic... No need to come down so hard on DiNardo... You enjoy a tremendous advantage over him: 1) he cannot reject your papers at any one of the major outlets of the profession, while you can make sure he never sees his name will never appear in the Journal of Polotical Economy, co-edited by you;
2) you have a larger and wider audience than he has (NY Times vs his less well known blog).
So, what is the point of coming down so hard on him?

Max

Give Levitt a break. His reaction wasn't harsh at all; in fact, I found it rather playful.

And regardless of the tone, it seems like DiNardo had a rebuttal coming.

Gabor

To Andy:
Pop-Eleches indeed found that children born after the ban on abortion do better BUT this is because educated women had had more abortions beforehand. Therefore the ban led to a higher share of children with educated parents who are expected to perform better. However, when Pop-Eleches controlled for this change in the composition of children, he found that these kids did worse. (It's all in the introduction of his paper)
DiNardo must have cited the simple description of the data, while Levitt referred to the proper (ceteris paribus) effect of the ban.

Mike

Most of you seem to be OK with someone walking all over you and trying to embarrass you without fact. Levitt reply wasn't even that aggressive, but has every right to defend himself if someone tries to embarrass him. If another economist is going to pull that crap, he/she better be prepared to look like a fool. You have to make an example of these people or they will continue to ignorantly attack others.

James

I know it may seem as though DiNardo has taken to attacking Freakonomics without cause, but a little digging will show that the 3 seperate reviews were actually 1 that was split up for the sake of publishing. I won't comment on whether either side is right or wrong, but will say that DiNardo is a very intelligent man (for what it's worth) and would suggest that this "attack" is merely an attempt to be thorough from someone who is very well versed in statistical methods.

Stephen M (Ethesis)

It looks as if he was led astray by the abstract. If he had communicated the review (or if they had followed normal procedure by letting someone see the review), he would have gotten a heads up.

Ah well. Live and learn.