Jury Poker: In Criminal Trials, the Odds Aren’t Good

A landmark study has been published by Northwestern statistics professor Bruce Spencer offering statistical and empirical data on the accuracy of U.S. jury verdicts. His method involved comparing the decision of a jury with the decision of the judge hearing the case, accomplished by having the judge fill out a questionnaire during jury deliberations. The data pool consisted of 290 criminal cases in Los Angeles; Washington D.C.; Maricopa County, Ariz.; and the Bronx, from 2000 to 2001.

Spencer’s analysis suggests that juries render incorrect verdicts an estimated 15% of the time (with an estimated standard of error of +/- 4 points), while judges are wrong in 12% of verdicts (including both convictions and acquittals). For those trials involving incorrect convictions, the chances of mistake are even higher: 25% of innocent defendants put on trial were handed a conviction by juries, and 37% were convicted by judges. Guilty defendants, meanwhile, were or would have been set free 10% of the time by a jury and 13% of the time by a judge.

While Spencer cautioned that more data is needed before reaching broad conclusions about the accuracy of criminal trials, his findings certainly highlight the vagaries of our current judicial system (which we’ve touched on before). Not to mention provide a reason to look again toward Steve Landsburg‘s theory that juries should be paid for their time and penalized for incorrect verdicts.

(Hat tip: Jian Li.)


Mack

It doesn't look as if jury trials are much more incorrect overall than bench trials, which underscores a point made in comments to the Landsburg post: a trial is only as good as the information that gets heard. It's hard to see how penalizing the judge or jury can affect that.

Plus, I can't make the numbers cited here add up. If judges are wrong on guilty defendants 37% of the time, and on innocent defendants 13% of the time, how can they do better overall than juries who are wrong on guilty defendants 10% and wrong on innocent ones 25%?

Can someone explain how that could translate to an overall rate of 10% for judges and 12% for juries?

jonathank

Any discussion of criminal verdicts needs to be put in the context that 90%+ of the cases brought in a major city plead out. (In my experience, the statistics showed about 94% cop a plea of some sort.) This means the trials are either cases where the defendant insists on trial, where the prosecution has a strong case and the defendant may as well take a shot at trial (e.g., where the penalty is life w/o parole) or where the issue is cloudy. My experience is most of the 6 or so percent that go to trial are the last group: cases where there may be conflicting testimony or when an alibi of some form is on offer, where there may be some mitigating circumstance (e.g., sympathy or the defendant is mentally unstable or retarded), where the prosecution has a problem (e.g., a blown search & seizure), etc.

In these cases, the ones that go to trial, it's not easy to get conviction - maybe 50% - but that's what one would expect. Perhaps more germane to the issue, guilt and innocence in these cases tends to be more in the eye of the beholder. I'm not sure the methodology adequately controls for that.

Read more...

Mack

Clearly I got the guilties and innocents confused there, but the ratios still don't make sense to me.

schadenfreude

How does he know what the correct verdict is? Maybe we could save a lot of money by doing away with trials and simply have Bruce Spencer determine guilt or innocence?

schadenfreude

From the article, it looks like the False conviction/False acquittal error rates for juries are 25%/14% and for judges 37%/2%.

Nathaniel

I'm with mack, there is a very nonsensical set of numbers there.

edwinlee

Our legal process is an example of the operation of cascaded Bayesian filters. A Bayesian filter is an imperfect test that, when screening a pool of items, like people who may or may not have cancer, substantially improves our knowledge of who has cancer and who doesn't, but always has some rate of false positives and false negatives. Turns out that no limited filtering system (i.e. finite resources) can operate without some degree of uncertainty... which results in both false positives (in the legal system it means innocent people judged guilty) and false negatives (guilty people judge innocent). One can use skew a system's resources to minimize either false positives or false negatives, but not both. To reduce both requires exponentially more resources.

There are several Bayesian filters in the legal system. They include the police and prosecution teams (skewed to false positives), the judge (supposedly skewed to a balance), the legal assumption of innocence until proved guilty (skewed to false negatives) and the jury.... which are fought over by prosecution and defense to be staffed by people who will skew the results one way or another. Plea bargaining is another Bayesian filter.

Our legal system may still be flawed, but it's better than a system of presumed guilt... which minimizes false negatives but substantially increases false positives. (As in the poor devils held at Guantanamo without the benefit of due process because "we can't afford to let any terrorist go free.")

Turns out our brains are full of cascaded Bayesian filters... its the basis of all our decision making.

Read more...

elfy

_If judges are wrong on guilty defendants 13% of the time, and on innocent defendants 37% of the time, how can they do better overall than juries who are wrong on guilty defendants 10% and wrong on innocent ones 25%?_

It might just be a typo in the post, but it is theoretically possible anyway if, say, innocent people overwhelmingly opt for jury trial. If a judge sees 150 guilty and 50 innocent people, then they'd get 38 wrong overall. If a jury sees 50 guilty and 150 innocent people, they'd get 43 wrong overall.

lermit

Actually, this intriguing debate rings a specific bell, which is going to be a bit unwinding with no playing either poker, judges, juries, or lawyers about why there was a big problem in the political realms (before and) after the big oil crisis of 1973...

And this is, of course, the fall of the gold standard on... about 1973. Actualley the replacement of the gold standard for the infamous Fed 'fiat maney supervision' to put it with words that are never enogh is, not, indeed, the monetarist haven.

This search for the gold standard that got loss in the seventie, provided for a scare worse than the NYC blackout, as displayed in Watergate-Nixon affair. (The 18 minutes of tape for the Watergate situation are yet to be found...).
To spare yew all biblioh details, I know because I got cable and I saw it on the Hitory Chanel.

Anyway, it seems that the political crisis just right before the US withrowal from Vietnam went together with a terrible surge in Latin American dictatorships end coups detats, besides the US Watergate scandal...

Anyhow, so this scare because of the replacement of the gold standard with alternative possibilities (no camels or sheep, btw), gave grounds, as it mightv been said, to a terrible terrible geopolitical scare. It got replaced by not actually a single one alternative means of currency backing, such as, well, Fed responsibility in, to put it blantly, each of its manifestations and spheres of influence.

But after the fall/replacement of the gold standard with so-called ' fiat currencies ', no culprit other than the alchemy substance was to be found for historically related relevant and substantially so political situations that were seeing the ligth of day on thus a moment.

Black pijamas and grey trunks, as a'd say, do not match, to honowr you Freaconomists!!

So, this was seen in the European realms with the incorporation of a currency basket which would in turn turn Francs, Florins, Crowns, Pesetahs, et al. into Euros. This was also part of a wider context in the geopolitical realm, to put it in words that might have a sound which may be from harsh to crude.

Back to the point. Barings Brothers as a disaster would see the first flirts with leverage and futures much after, in 1973 too, the inauguration of the new Chicago Futures and Options Exchange.

Thus, to have a context for a jury and not taking accountability for what their decision might have been, it's just terribly terribly ridiculous and outrageous. Kind of tickery key and terribly mey. Preposterous!

It's like the gold standard has its standards and the fiat currency's got it's uses. Right...??

To sum it up, with all its takes and gives and give-and-takes, it might seem that the field of economics does have a certain... influx of reality in the impact of the law...

Thanks...!!

.lerrmit (kinda cool eh'en' incnspikews cenzrn' bpeewfbz!!~!!!!1... ;) ;)

Read more...

lermit

ha ha

http://www.youtube.com/watch?v=jU5NkuCiOUA

Mack

lermit, dude -- gotta start taking the meds earlier in the day...

frankenduf

what would interest me is has the percentage of convictions been constant over time, or if it has changed- breakdown by race might be telling as well

dilbert69

I would question anyone's ability to determine the "correct" result of a criminal trial if we define "correct" as "a guilty verdict if the person did the crime and a not-guilty verdict if he/she did not." Only the perpetrator, the victim if any and if alive, and an omniscient supreme being if any would really know whether the person did the crime with 100% certainty (and even victims sometimes identify the wrong person). If by "correct" we mean a guilty verdict when the prosecution proved its case beyond a reasonable doubt and a not-guilty verdict when it failed to, this would be a lot easier for a trained legal expert such as a judge to determine. For example, I believe OJ did the crime, but I also believe that the prosecution failed to prove this beyond a reasonable doubt, so the verdict in that case, however unjust, was legally "correct." Of course, I'm not a legal expert, but I suspect many legal experts would agree with my assessment.

Read more...

egretman

lermitt, how do you do that?

Your rant that is. Do you have a babelfish filter? Maybe put in some speech of Ron Paul's on the gold standard and out pops a stream of consciousness that only those characters in On Flew Over the Cuckoos Nest would appreciate for what it is, just f*cking "a" brilliant to the point of sarcastic bombastic over-the-top greatness?

I wish I could do whatever you do.

lermit

Sorry I was not drunk. Just getting creative, Mack and egretman.

Thanks

.lermit

Mack

It doesn't look as if jury trials are much more incorrect overall than bench trials, which underscores a point made in comments to the Landsburg post: a trial is only as good as the information that gets heard. It's hard to see how penalizing the judge or jury can affect that.

Plus, I can't make the numbers cited here add up. If judges are wrong on guilty defendants 37% of the time, and on innocent defendants 13% of the time, how can they do better overall than juries who are wrong on guilty defendants 10% and wrong on innocent ones 25%?

Can someone explain how that could translate to an overall rate of 10% for judges and 12% for juries?

jonathank

Any discussion of criminal verdicts needs to be put in the context that 90%+ of the cases brought in a major city plead out. (In my experience, the statistics showed about 94% cop a plea of some sort.) This means the trials are either cases where the defendant insists on trial, where the prosecution has a strong case and the defendant may as well take a shot at trial (e.g., where the penalty is life w/o parole) or where the issue is cloudy. My experience is most of the 6 or so percent that go to trial are the last group: cases where there may be conflicting testimony or when an alibi of some form is on offer, where there may be some mitigating circumstance (e.g., sympathy or the defendant is mentally unstable or retarded), where the prosecution has a problem (e.g., a blown search & seizure), etc.

In these cases, the ones that go to trial, it's not easy to get conviction - maybe 50% - but that's what one would expect. Perhaps more germane to the issue, guilt and innocence in these cases tends to be more in the eye of the beholder. I'm not sure the methodology adequately controls for that.

Read more...

Mack

Clearly I got the guilties and innocents confused there, but the ratios still don't make sense to me.

schadenfreude

How does he know what the correct verdict is? Maybe we could save a lot of money by doing away with trials and simply have Bruce Spencer determine guilt or innocence?

schadenfreude

From the article, it looks like the False conviction/False acquittal error rates for juries are 25%/14% and for judges 37%/2%.