How the Supreme Court Misread My Research: Empirics and the Death Penalty

Cass Sunstein and I have an Op-Ed in today’s Washington Post, discussing the mis-reading of available empirical evidence in recent death penalty jurisprudence.

Some background: A recent Supreme Court ruling (Baze v. Rees) reaffirmed the constitutionality of the death penalty, and along the way, the justices revisited the empirical literature on whether the death penalty has been shown to be an effective deterrent.

Justice Stevens cited research by John Donohue and myself in concluding that “there remains no reliable statistical evidence that capital punishment in fact deters potential offenders.” Countering this, Justice Scalia cited a paper by Cass Sunstein and Adrian Vermeule, in arguing that the data do in fact point to deterrence.

As two of the supposed flag bearers for the competing views cited by the court, Sunstein and I thought it worth poring over the data to see what we agree on. It turns out there’s a lot of agreement between us:

“In short, the best reading of the accumulated data is that they do not establish a deterrent effect of the death penalty.”

Our close reading of the court’s decision leads us to conclude that:

“Unfortunately, they [the Supreme Court] misread the evidence.”

To Justice Scalia, we recommend an updated reading list. Justice Scalia

relies on the suggestion by Sunstein and Vermeule that some evidence suggests a possible deterrent effect. But that suggestion actually catalyzed Donohue and Wolfers’s study of available empirical evidence. Existing studies contain significant statistical errors and slightly different approaches yield widely varying findings, a problem exacerbated by researchers’ tendency to report only those results supporting their conclusions. This led Sunstein and Vermeule to acknowledge: “We do not know whether deterrence has been shown … Nor do we conclude that the evidence of deterrence has reached some threshold of reliability that permits or requires government action.

To Justice Stevens, who suggests that “In the absence of such evidence, deterrence cannot serve as a sufficient penological justification for this uniquely severe and irrevocable punishment,” we provide a note of caution:

“the absence of evidence of deterrence should not be confused with evidence of absence.”

The morality of the death penalty is a tough question, to be sure. But it should not be further complicated by a misreading of the evidence.

The full Op-Ed is available here (or else ungated, here). A readable introduction to this literature is available here.

For those interested in the empirical observations in our Op-Ed, I have put together a longer document discussing that evidence, available here; there are plenty of pretty pictures and pointers to background material.

Don’t want to take my word on the evidence? I have collected the available data and posted it here — feel free to generate your own findings. And read Levitt’s $0.02, here, here, and here.


Ryan

A supreme court justice is to determine the constitutionality not the effectiveness.

Unbelievable that both sides of the argument forgot what their one job is. I wonder if we jut took 9 idiots and told them, "you have one thing to do, guard the constitution." I bet they would do a better job because of their simple mindedness.

There's only one question, is capital punishment 'cruel and unusual'?

ids

" . . . who suggests that . . . [t]he morality of the death penalty is a tough question, to be sure." Provide a note of caution:

happiness in us can trump morality, so the question is moot. The death penalty makes many people happy, even SC Justices. Anyhow, good to bring light to it.

Coleman

#17:

Mencken's thoughts run completely counter to the spirit of "no cruel and unusual punishment." By his argument, we should make people suffer just as they've made their victims suffer. If they've tortured, they should be tortured; if they've raped, they should be raped. Isn't this the only way to really have katharsis - for people to get what they've given? Revenge by any other name is still revenge.

Sean

The death penalty is bad for one very important reason... if you put the wrong person away and you kill them, you can't let them out later when you realize you screwed up. Remember why Illinois imposed a moratorium? 52% of the people on death row were found to be wrongly imprisoned. I think life without parole should be used a lot more than it is, but am 100% against the death penalty unless you can prove we never put the wrong person away. Which, being human, we alway will in some cases. Google "Ronald Cotton" and see what comes up.

Ida B. Wells

I saw a poll the other day indicating that the majority of people across all demographic groups support capital punishment. Christians, who champion a book one might read to speak against the practice, call for it as strongly as anyone. If their savior didn't convince them, I'm not sure we should hold out hope for stats.

Probably better to just keep our heads down, lest the advocates of the penalty begin to suspect we few opponents are a danger to society as well.

Dudley Sharp

I think Prof. Wolfers left something out.

There are some problems with:

A Death Penalty Puzzle: The Murky Evidence for and Against Deterrence (Washington Post, By Cass R. Sunstein and Justin Wolfers, June 30, 2008; Page A11)http://www.washingtonpost.com/wp-dyn/content/article/2008/06/29/AR2008062901476.html

The article states: "But that suggestion (of the many recent studies finding for deterrence) actually catalyzed Donohue and Wolfers's study of available empirical evidence. Existing studies contain significant statistical errors, and slightly different approaches yield widely varying findings, a problem exacerbated by researchers' tendency to report only those results supporting their conclusions."

REPLY: ". . . researchers' tendency to report only those results supporting their conclusions."! My goodness, this seems much like the pot calling the kettle black. Please see the four replies to Donahue and Wolfers, at bottom - replies which, collectively, appear to lay waste to Donahue and Wolfers criticism. It should also be pointed out that Donahue and Wolfers chose not to publish their criticism in a peer reviewed publication.

The article continues: "In short, the best reading of the accumulated data is that they do not establish a deterrent effect of the death penalty."

REPLY: I think it is arguable that is an unfair reading of the data.

My take is that it would be more accurate to state that the social science of determining the measurable effect of general deterrence is challenging, at best. The strength of the recent econometric studies is suggestive of a deterrent effect of the death penalty and the criticism of those studies seems much less robust than the studies, themselves.

In general terms, the weight of the evidence is that all prospects for a negative outcome deter some. There appears to be few, if any, exceptions to that rule.

The article continues: "But as executions resume, the debates over the death penalty should not be distorted by a misunderstanding of what the evidence actually shows."

REPLY: Again, it appears that misunderstanding is, precisely, what this newest article was distributing, however unintended.

The article continues: "A prominent line of reasoning, endorsed by several justices, holds that if capital punishment fails to deter crime, it serves no useful purpose and hence is cruel and unusual, violating the Eighth Amendment." "While some favor the death penalty on retributive grounds, many others (including President Bush) argue that the only sound reason for capital punishment is to deter murder."

REPLY: The lack of balance or understanding by S&W appears quite robust. There are many well defined reasons for criminal sanction, outside of deterrence. Retribution, justice, upholding the social contract, incapacitation, among others. Obviously, capital punishment does not violate the Eight Amendment, as both the 5th and 14th Amendments assert the constitutionality of the death penalty. These are well known and important facts, not in S&W's article.

Both the current Pres. Bush and former VP Al Gore stated they support the death penalty because of deterrence. However, it is important to clarify. Does anyone believe that either Gore of Bush would support the execution of someone if it wasn't deserved? I suggest the answer is no. It's a fair question. Ask them. Then ask them, if the death penalty did not deter like men as Bib Laden or Hitler, would you still support their executions? Does anyone not know their answer? Of course not.

The article continues: "We concur with Scalia that if a strong deterrent effect could be demonstrated, a plausible argument could be made on behalf of executions. But what if the evidence is inconclusive?"

REPLY: Inconclusive? The weight of the evidence is that some are deterred by any prospect of a negative outcome. But, let's project an imaginary world where the evidence is completely neutral on the effects of negative prospects, where there is no evidence of what incentives mean to behavior.

We have two equally balanced prospects. The death penalty/executions deter and the death penalty/executions don't deter.

This prospect is neither inconclusive or equally balanced, because you have a prospect between sparing innocent life, via death penalty/execution deterrence or a prospect of death penalty/execution, with no deterrence, but enhanced incapacitation.

If deterrence is inconclusive, the prospect of saving innocent lives is not.

In addition, the deterrence calculus becomes- we execute and save additional innocent lives (via deterrence) or we execute and don't save additional innocent lives (via deterrence). The weight falls upon saving additional lives, if we have inconclusive data, because the weight is greater for saving additional innocent lives than on not saving additional innocent lives.

With this type of calculus, the only argument for not executing is if the evidence is clear that the death penalty/execution is sacrificing more innocents, via brutalization - the prospect that by executing, you are inspiring potential murderers to become real murderers. Brutalization is discounted because the evidence for deterrence is much more robust and the practical evidence that crimes will increase because of raised sanction doesn't exist.

As noted, there are a number of good reasons to argue in favor of the death penalty, other than deterrence. Even if the death penalty is a very marginal deterrent, it is still a robust argument in favor of the sanction.

Let's say that every four executions deters one potential murderer and every twenty death sentences deters one potential murderer. That would mean that around 650 innocents had been spared since 1973. Even marginal deterrence equals major benefit.

This does not take into account the two other ways that the death penalty saves additional innocent lives. Enhanced incapacitation and enhanced due process.

Sincerely, Dudley Sharp

-------------------------

four replies to Donahue and Wolfers

Hashem Dezhbakhsh & Paul H. Rubin

From the 'Econometrics of Capital Punishment' to the 'Capital Punishment' of Econometrics: On the Use and Abuse of Sensitivity Analysis (September 2007). Emory Law and Economics Research Paper No. 07-18, http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1018533

Abstract: The academic debate over the deterrent effect of capital punishment has intensified again with a major policy outcome at stake. About two dozen empirical studies have recently emerged that explore the issue. Donohue and Wolfers (2005) claim to have examined the recent studies and shown the evidence is not robust to specification changes. We argue that the narrow scope of their study does not warrant this claim.

---------------

(2006) "This analysis shows that ((Donohue and Wolfers' "D&W's")attempts to make the deterrence effect disappear are ineffective." (p 16)

--- Existence of the death penalty, in law, has a statistically significant impact on reducing murders. (p 23)

--- Execution rates show significant impact in reducing murders. (p 13 & 23)

--- Death row commutations, and other removals, increase murders. (p13 & 23)

--- The criticism of our studies is flawed and does not effect the strength of the measured deterrent effect.

"The Impact of Incentives On Human Behavior: Can we Make It Disappear? The Case of the Death Penalty", Naci H. Mocan, R. Kaj Grittings, NBER Working Paper, 10/06, www(dot)nber.org/papers/w12631

--------------------

(2006) " . . . (Donohue and Wolfers' "D&W") criticisms of Zimmerman's analysis are misrepresentative, moot or unsupportable in terms of the analyses they perform." "It is shown that Zimmerman's published empirical results, or the conclusions drawn from them, are not in any way refuted by D&W's critique." (pg 3) "This later estimate suggests that each execution deters 14 murders on average . . .". (pg 7) "It is shown that D&W made a number of serious misinterpretations in their review of Zimmerman's study and that none of the analyses put forward by D&W (which ostensibly refute Zimmerman's original results and conclusions) hold up under scrutiny. (pg8) " . . . D&W do not even report Zimmerman's "preferred" results correctly, and then proceed by carrying on this error throughout the remainder of their critique."(pg8) "Of course, (D&W's) omission tends to create a strong impression that Zimmerman's analysis 'purports to find reliable relationships between executions and homicides', when his actual conclusions regarding the deterrent effect of capital punishment are far more agnostic." (pg10) " . . . D&W's method of interpreting their results is not consistent with that proscribed by the received econometric literature on randomized testing . . .". "As such, D&W's interpretation of their randomized test in itself does not (and cannot) reasonably lead one to conclude that Zimmerman's estimates suggesting a deterrent effect of capital punishment are spurious." (pg12) " . . . D&W do not appear to have interpreted their randomization test in any meaningful fashion." (pg14) " . . . the state clustering correction employed by D&W may not be producing statistically meaningful results." (pg16) "And while D&W once lamented that recent econometric studies purporting to demonstrate a deterrent effect of capital punishment yield 'heat rather than light', as shown herein, their criticisms of Zimmerman (2004) tend to yield 'smoke rather than fire'."(pg26)

Zimmerman, Paul R., "On the Uses and 'Abuses' of Empirical Evidence in

the Death Penalty Debate" (November 2006). ssrn(dot)com/abstract=948424

-----------

(2007) "Had (D&W's) paper been subjected to the normal blind peer review process in an authoritative economic journal it is highly unlikely that it would have survived intact , if at all. "

"(D&W's) Quibbling over numerous and sometimes meaningless statistical issues obscures the picture painted by the cumulative effect of the nearly dozen studies published since the turn of the 21st century."

"Using differing methodologies and data sets at least five groups of scholars each working independently (and often without knowledge of the others) have arrived at the same conclusion—there is significant and robust evidence that executions deter some homicides. While there may be merit in some of (D&W's) specific criticisms, none addresses the totality of the collection of studies. The probability that chance alone explains the coincidence of these virtually simultaneous conclusions is negligible."

"DW’s unsupported claim that the appropriate variable in studies of deterrence using these borrowed tools from portfolio analysis is the amount or level of homicides in the respective jurisdictions. This claim is without theoretical basis or empirical precedent. "

"With regard to DW’s specific comments on our two papers (Cloninger & Marchesini, 2001 & 2006) we find very little requiring defense. Implicit in their critique, and explicitly stated in private communications, DW were able to replicate our results based on data we furnished, at their request, as well as data they acquired independently. "

"Reflections on a Critique", Dale O Cloninger and Roberto Marchesini, forthcoming Applied Economic Letters

Read more...

PaulK

James2: "you will clearly see that the founding fathers believed capital punishment to be neither cruel nor unusual." - what history is that? I see that the founding fathers were of many views on this other than for treason. The strongest held belief seems to have been for the military to use capital punishment for some murders and for treason. Other than that, it does not seem to have become fashionable until the early 1800s where some States were using it more and more, including big public spectacles. As you can well imagine, these were not held to any standards (e.g. mass executions, executing 10 year olds, torture, etc) and the Supreme Court did eventually step in to determine that quartering (for example) was indeed cruel and unusual, etc.

Andrew

#8 So you kill 23,000 people who won't murder again (1150 of whom are innocent) to save 280 people. Also, you further legitamise killing as a morally justified punishment, and so increase the number of revenge killings in society. Very clever.

Silas

Comment #1:

When deciding whether something qualifies as "cruel and unusual punishment," morality determines legality...under the Constitution.

HL Mencken

Their fundamental error consists in assuming that the whole aim of punishing criminals is to deter other (potential) criminals--that we hang or electrocute A simply in order to so alarm B that he will not kill C. This, I believe, is an assumption which confuses a part with the whole. Deterrence, obviously, is one of the aims of punishment, but it is surely not the only one. On the contrary, there are at least half a dozen, and some are probably quite as important. At least one of them, practically considered, is more important. Commonly, it is described as revenge, but revenge is really not the word for it. I borrow a better term from the late Aristotle: katharsis. Katharsis, so used, means a salubrious discharge of emotions, a healthy letting off of steam. A school-boy, disliking his teacher, deposits a tack upon the pedagogical chair; the teacher jumps and the boy laughs. This is katharsis. What I contend is that one of the prime objects of all judicial punishments is to afford the same grateful relief (a) to the immediate victims of the criminal punished, and (b) to the general body of moral and timorous men.

From "The Penalty of Death", H. L. Mencken, 1926

Read more...

Tom

“the absence of evidence of deterrence should not be confused with evidence of absence.”

While that's true, there are limits to it. If I think that, for example, there are weapons of mass destruction in a particular country, I probably shouldn't invade it for that reason without some evidence. There isn't evidence of absence; but if I'm going to take a drastic and irreversible step, I ought to be pretty sure of my grounds.

Similarly, it's alright to try out some new social program to prevent crime on spec, without much evidence either way about its effectiveness, but it's quite another to deliberately take someone's life for that reason.

JSN

On average it costs $1 million per year to incarcerate 39 prisoners but it costs much more to incarcerate a prisoner serving an LWOP sentence for 39 years because geriatric incarceration costs are so large. On the other hand the litigation costs of a DP sentence are so large some counties will not ask for a DP sentence because they cannot afford to pay the legal costs. Is this unequal justice? I don't know if the courts have been asked to rule on that.

In any case DP and LWOP sentences incapacitate the prisoner so they cannot re-offend. Some people call incapacitation specific deterrence because it applies to the specific offender where general deterrence or self-deterrence apply to all persons (if fact only those who know the penalty is DP or LWOP). Investigations of the influence of general deterrence have had ambiguous outcomes. I don't think that should be a surprise because most people don't spend any time thinking about committing a first degree homicide.

Just retribution should be proportional or equivalent in severity to the original offense and should also protect public safety and rehabilitate the offender. In the US we tend to treat all persons sentenced to a DP or LWOP as dangerous habitual offenders when in fact some are unlikely to commit a new offense of any kind. It is uncommon for a Governor to commute a DP sentence to LWOP sentence or a LWOP sentence to a parole eligible sentence because of the adverse political consequences.

Read more...

Durbrow

Related to the economic costs of life imprisonment v. execution... California has announced that its execution system is bankrupted... A statewide commission suggested that California can save several hundred millions of dollars by dismantling it.

http://www.kcra.com/news/16752354/detail.html?rss=sac&psp=news

Brian

The word "myself" in the third paragraph would probably best be replaced by "me."

There was a WSJ editorial on the subject of lazy misuse of the word "myself" a few months ago.

steve

Perhaps it's more a characteristic of the world we live in. The Death Penalty might be a huge deterent if it was actually seen and used. In Freakonomics there was a section that described how someone was more likely to die as a crack dealer in chicago than on death row.

Surely the death penalty is therefore no deterent for any crack dealer in chicago since it will actually make them safer if they were caught for murdering someone. Come on, they get 2-3 meals a day for several years before they face death, which might actually be a better life for the dealer than selling crack.

Jbal

@#7, I remember reading a few years back that all the appeals associated with a death penalty charge actually cost taxpayers more than incarcerating a person for life.

Chris

Is it the Supreme Court's responsibility to determine Morality or Legality?

B. Riley

@ #3

I agree with your statement about the Constitution not mentioning deterrence as a qualifier for punishment of a crime.

Let's assume we did have strong evidence that capital punishment is a deterrent.

Are we seriously talking about telling another human being that their life no longer has any intrinsic value, and then killing them, for the purpose of deterring crime? That's amazing to me.

James2

If you read your history, you will clearly see that the founding fathers believed capital punishment to be neither cruel nor unusual. If the the majority in a state beleives capital is immoral or poor policy, the state is free make it illegal within their boundaries. At the federal level you can argue that the state's process is flawed and prevents "equal protection" or "due process", but cannot make a intellectually honest arguement that captital punishment, as a concept or option, is unconstitutional

Robin

What about the 98.8% of released murders that didn't kill, or at least weren't caught, a second time. How would we determine which criminals are in the 1.2%? The problem with capital punishment is that it is irreversible so mistakes can't be corrected.