“Creative” use of data by death penalty proponents

The academic debate on the death penalty is heating up again. Three decades ago, Isaac Ehrlich went head to head with a dozen critics. In the end, most unbiased observers concluded that the results were too sensitive to minor changes in specifications to draw any strong conclusions.

More recently, a host of authors have once again been arguing that the death penalty is an effective homicide deterrent. For instance, in recent testimony in front of the U.S. Senate, economist Paul Rubin began his comments by saying, “Recent research on the relationship between capital punishment and homicide has created a consensus among most economists who have studied the issue that capital punishment deters murder.”

That struck me as a strange thing to argue given what I know about the situation.

I have a paper with Larry Katz and Ellen Shustorovich in 2003. We conclude that “…there is little systematic evidence that the execution rate influences crime ratese…” Of 20 different specifications we examine, in only three cases do we find a statistically significant reduction in homicide from the death penalty. In one case, we find a statistically significant increase. (In contrast to the findings on executions, we find that bad prison conditions do seem to reduce crime.)

Here is how Rubin describes my paper’s results in his testimony:

“Another recent paper by Lawrence Katz, Steven D. Levitt, and Ellen Shustorovich uses state-level panel data covering the period 1950 to 1990 to measure the relationship between prison conditions, capital punishment, and crime rates….In several estimations, both the prison death rate and the execution rate are found to have significant, negative relationships with murder rates…”

What about the other 17 specifications?

More damaging to the case in favor of a death penalty is an exhaustive new study by John Donohue (my co-author on the abortion-crime stuff) and Justin Wolfers (an economist at Wharton). In the paper, Donohue and Wolfers provide a devastating critique of the existing studies, including the ones cited by Rubin in his testimony.

Rubin’s testimony on the Donohue and Wolfers paper is as follows:

“A recent paper in the Stanford Law Review questions some of these studies. This paper purports to show that the estimates of a deterrent effect are “fragile” and can be changed by statistical manipulation. The results of this paper have not been evaluated by competent scholars; the Stanford Law Review, like all law reviews, is edited by students who have no particular competence in econometrics. Moreover, Professors Wolfers and Donohue chose not to make their paper available online through a service such as SSRN or the BE Press, so that the scholarly community did not have access to their analysis before it was published.”

While in the abstract I agree that it is preferable to publish work in peer reviewed journals, it is worth remembering that peer-review is nothing like a guarantee of quality. At best, one hopes that peer review culls out the truly bad research. But there are enough journals that all but the worst work can eventually be published in a peer reviewed journal. And when two top scholars like Donohue and Wolfers publish something, peer reviewed or not, it is worth taking note.

But, this is the kicker. If you have the patience to download a paper from the Social Science Research Network, click here and download this paper. If you compare Rubin’s testimony, you will find that in most parts it follows word for word what is in this paper (published, I should add, in a non peer reviewed journal).

Which wouldn’t be a big deal, except the paper that exactly matches Rubin’s testimony isn’t even written by Paul Rubin, it is written by his sometimes co-author Joanna Shepherd.


Ben Golub

That *is* pretty disturbing -- especially considering the slight reordering here and there.

Rubin's testimony: Cross-sectional studies also suffer serious problems. Most importantly, they preclude any consideration of what happens to crime, law enforcement, and judicial processes over time. Cross-section data also prevent researchers from controlling for jurisdiction-specific characteristics that could be related to murder, such as greater urban density in some states.

Shepherd's paper: Cross-sectional studies also suffer serious problems. Most importantly, they prevent researchers from using so-called fixed-effects estimation to control for jurisdiction-specific characteristics that could be related to murder. For example, with cross-sectional data, a researcher cannot control for fundamental but immeasurable variables, such as a violent culture in southern states. Cross-section data also preclude any consideration of what happens to crime, law enforcement, and judicial processes over time.

Read more...

smili

Sadly, political decisions are made first, and then they go out to find the necessary facts/"science" to justify the decision.

Science seems less likely to inform nowadays, and seems more likely to be perverted for reasons of justification of a decision that's already been made. If each side has their "science" experts and only listens to who they agree with without acknowledgement of concensus, what are we left with?

Kirit

Coincidentaly I put something up on my web site about US death penalties (from an outsider's perspective). My argument (or should that be hope?) was that it seemed to be dieing it's own death through repeated eight amendment appeals - probably no news to those of you who have spent more time looking at this than I have.

The article is at http://www.kirit.com/Death%20penalties%20in%20the%20US

(I'll be adding in the references from here once I've read them myself)

Deltoid

Death Penalty

When I wrote about David Frum's voodoo criminology in support of the death penalty, I didn't mention anyone of the recent research that purports to find a deterrent effect for the death penalty because Frum didn't cite it. That research...

X-Tra Rant » Statistics & The Death Penalty

[...] A very interesting post by Steven Levitt at the Freakonomics blog about the “creative” use of statistics by death penalty supporters. [...]

JGUNS

Here is the thing I wonder about these studies on the deterrent factor: Has anyone ever looked at Singapore? They have the death penalty for simple drug possession and there is almost no drugs there. According to those that believe that the Death Penalty is not a deterrent, how would they explain this?

Secondly, those against capital punishment seem always to focus on the "detterent factor" argument. Almost no one that I know that is for the death penalty is for it because they believe it is a deterrent. They are for it because they think that there are some crimes and some criminals that forfeit their right to live due to the heinousness of their crimes. Therefore, these studies are unlikely to sway most people that are for capital punishment.

econopete

JGUNS:

You raise some interesting points. However, regarding Singapore, a correlation does not indicate a cause and effect. There's a high correlation between churches and crime, but does that mean that the churches are causing the crime? No, it just so happens that there is greater crime where there are more churches, which is in more densely populated areas (cities).

I agree with your second point. Anyone I've heard who favors the death penalty thinks that certain individuals should be put down like dogs.

Often the people I hear against the death penalty argue that the wrong people are convicted of the crime, thus performing a disservice to all of society.

Blandy

How do you prove (or at least show supporting evidence) that the execution rate influences the crime rate?

Wouldn't a state with a high crime rate be more likeley to have more executions than one with a low crime rate? In this case wouldn't the high crime rate be causing more executions, not the other way round?

loikll

Do I really have to point out the obvious? Of COURSE the death penalty deters murders. At a very minimum, it prevents the known killer whose eyeballs you just set on fire from ever killing again. Duh.

Considering he's the one guy in the room proven to be capable of committing murder, that's a pretty solid accomplishment.

Blandy

I don't disagree with the logic, but is there a way to prove that the execution rate (not the fact that a state has the death penalty) actually CAUSES the low crime rate? It's one thing to say that it's linked, but the causality is a different matter.

heigham

We are talking about incentives here. Murder is an activity that includes a high frequency of acts on impulse. That is the type of behaviour that incentives are least likely to condition effectively.

Drugs are an industry where decisions are determined by economic incentives (Read Freakonomics). A risk of 100% loss of everything - death - is to be expected to inluence drug industry behaviour in Singapore and elsewhere.

However, if the aim is to deter behaviour which leads to the loss of innocent lives, the most effective area to deploy the death penalty is probably in deterring dangerous driving. Can anyone think of a way to falsify this statistically?

Of course, since the system is run by humans, innocent drivers would sometimes be executed - as innocent people unlucky enough to be found in the same place as drugs probably are, on occasion, in Singapore.

Crash

I wonder what the impact is on the rate of recidivism.

CJ

I agreed with the Prof's research however isn't their a simpler deterence link everyone is missing? If your on death row and are put to death, isn't it a gurentee that your not going to kill anyone again, therefore making the deterent pretty effective? And this also helps with the thinking on a prison break, a leniat judge on appeal setting someone free, or even a Governor letting them go? If their dead by means of the death penalty in that case its 100% effective, issue over.

Jon VanderPlas

JGUNS seems to argue that death penalty advocates have a need for revenge that only an execution can fulfill. A better way to make this point is that justice requires some criminals to be put to death. Execution, not life in prison, is the just punishment for the worst and most evil of criminal acts, regardless of its deterrent on other violent crime.

Steve

All the above posts that say that death penalty deters 'at the very least' the murderer being executed - what happens if the guy being executed ISN'T the murderer?

When you execute innocent people - and it's clear we've executed HUNDREDS of innocent people - any positive deterrent effect on crime is outweighed by the resulting cynicism, disrespect, and hostility towards law enforcement. This is especially true in the minority communities from whom many of the executed people who were innocent come from.

Melody

loikll: That would be a fabulous argument, if it were true, DUH. A few nights ago, I spent a loooong time going through the data on the Texas Department of Criminal Justice webpage. They have a page listing all the offenders no longer on death row. One after another you see the reason listed as: Conviction reversed, conviction overturned, etc. You can even see for yourself: http://www.tdcj.state.tx.us/stat/permanentout.htm

So it would be nothing short of idiotic to try to say that ever person we've put to death was fully guilty of their crime. I'm sure the economists here could whirl up the likelihood that the state got ALL the innocents off death row prior to their executions - I suspect it's not very likely. Moreover, the death penalty is supposed to be used for only the most heinous of crimes, but particularly here in Texas, we see that provision flagrantly violated. Tonight the state will murder a man, Kenneth Foster, for not murdering a man, nor having prior knowledge that his friend was going to murder someone (this fact was even proven in court testimony). Now that's the most heinous crime? I think not (the man who pulled the trigger was executed last year). Texas is just overzealous in murdering its citizens, particularly if they're a minority. The whole system is constitutional at best, state-sanctioned murder at worst.

Read more...

Melody

Um, yeah...I meant to say UNconstitutional at best. Big faux pas.

Mark

The death penalty is not a deterrent ... anywhere. Someone bent on committing a crime "worthy" of death is not going to be stopped by anything. Nothing.

And if we are going to fool ourselves into believing the death penalty is actually a deterrent then why isn't it televised ... on prime time?

The death penalty is nothing short of revenge, which is an inability to overcome rage - to forgive. As such, the crime owns the revenger, not the perpetrator.

If you don't want killing, well, don't kill.

Alex

No one seems to be pointing out the fact that the death penalty is also expensive. It is much cheaper to just send someone to jail for life than to sentence them to death and pay for all the trials and appeals that the convict will inevitably go through.

oldguard

While working at San Quentin in 1992 & 1993, I was a member of the Execution Team. The two men whose executions I was involved in brought so much pain and death to their victims. One spent 14 years on Death Row, exhausting his appeals. They were executed so much more humanely than their victims. They were allowed to be with their families, almost right up to the end unlike their victims. They were allowed visits with a spiritual advisor, unlike their victims. They died knowing why, unlike their victims. Is the death penalty a deterrent? I only know these two men will never kill again. If I could change anything it would be the number of appeals a convicted murderer can file. I'd give them two.