“Creative” use of data by death penalty proponents

The academic debate on the death penalty is heating up again. Three decades ago, Isaac Ehrlich went head to head with a dozen critics. In the end, most unbiased observers concluded that the results were too sensitive to minor changes in specifications to draw any strong conclusions.

More recently, a host of authors have once again been arguing that the death penalty is an effective homicide deterrent. For instance, in recent testimony in front of the U.S. Senate, economist Paul Rubin began his comments by saying, “Recent research on the relationship between capital punishment and homicide has created a consensus among most economists who have studied the issue that capital punishment deters murder.”

That struck me as a strange thing to argue given what I know about the situation.

I have a paper with Larry Katz and Ellen Shustorovich in 2003. We conclude that “…there is little systematic evidence that the execution rate influences crime ratese…” Of 20 different specifications we examine, in only three cases do we find a statistically significant reduction in homicide from the death penalty. In one case, we find a statistically significant increase. (In contrast to the findings on executions, we find that bad prison conditions do seem to reduce crime.)

Here is how Rubin describes my paper’s results in his testimony:

“Another recent paper by Lawrence Katz, Steven D. Levitt, and Ellen Shustorovich uses state-level panel data covering the period 1950 to 1990 to measure the relationship between prison conditions, capital punishment, and crime rates….In several estimations, both the prison death rate and the execution rate are found to have significant, negative relationships with murder rates…”

What about the other 17 specifications?

More damaging to the case in favor of a death penalty is an exhaustive new study by John Donohue (my co-author on the abortion-crime stuff) and Justin Wolfers (an economist at Wharton). In the paper, Donohue and Wolfers provide a devastating critique of the existing studies, including the ones cited by Rubin in his testimony.

Rubin’s testimony on the Donohue and Wolfers paper is as follows:

“A recent paper in the Stanford Law Review questions some of these studies. This paper purports to show that the estimates of a deterrent effect are “fragile” and can be changed by statistical manipulation. The results of this paper have not been evaluated by competent scholars; the Stanford Law Review, like all law reviews, is edited by students who have no particular competence in econometrics. Moreover, Professors Wolfers and Donohue chose not to make their paper available online through a service such as SSRN or the BE Press, so that the scholarly community did not have access to their analysis before it was published.”

While in the abstract I agree that it is preferable to publish work in peer reviewed journals, it is worth remembering that peer-review is nothing like a guarantee of quality. At best, one hopes that peer review culls out the truly bad research. But there are enough journals that all but the worst work can eventually be published in a peer reviewed journal. And when two top scholars like Donohue and Wolfers publish something, peer reviewed or not, it is worth taking note.

But, this is the kicker. If you have the patience to download a paper from the Social Science Research Network, click here and download this paper. If you compare Rubin’s testimony, you will find that in most parts it follows word for word what is in this paper (published, I should add, in a non peer reviewed journal).

Which wouldn’t be a big deal, except the paper that exactly matches Rubin’s testimony isn’t even written by Paul Rubin, it is written by his sometimes co-author Joanna Shepherd.