criminal-law jury comparative-law civil-legal-system bench-trial
Any solid research on error rate of criminal conviction system: jury vs judges?
<p>To my knowledge most countries use one of two systems for determining the guilt of the accused in criminal cases: “jury of your peers” (who don’t have to justify their decision) vs “judg(es), possibly with lay jurors” (who have to justify their decision). Also some countries allow the striking of deals pre-trial / instead of trial.</p>

<p>Is there solid research on which system performs better (as in how many erroneous convictions, how many erroneous dismissals) in which circumstances?</p>

<p>It is hard to measure accuracy in absolute terms because figuring out the truth is the problem that trials seek to solve in the first place.</p>
<p>This is especially true of cases that go to trial. Cases that are almost sure to be resolved one way or the other by any tier of fact generally plea bargain (in criminal cases) or settle prior to trial (in civil cases). The plea bargaining rate is 90%-95% in criminal cases and less than 5% of civil cases go to trial.</p>
<p><a href="https://www.wisbar.org/NewsPublications/InsideTrack/Pages/Article.aspx?Volume=6&amp;Issue=6&amp;ArticleID=11435" rel="nofollow noreferrer">For better or for worse</a>, civil jury trial have become much less common in the last half century, while criminal jury trial rates have been comparatively much more stable (criminal jury trials have <a href="https://judicialstudies.duke.edu/wp-content/uploads/2018/01/JUDICATURE101.4-vanishing.pdf" rel="nofollow noreferrer">become increasingly less common in federal trial courts</a>, however, where the rate at which criminal cases go to trial is about half that of state courts).</p>
<p>The good news is that this suggests that the accuracy of the entire process acting in the shadow of trial in most cases, is much greater than the accuracy of trials themselves, which tend to involve the hardest and closest cases. The bad news is that to the extent that trial outcomes are unpredictable, even those who are innocent or not at fault in a civil case, will sometimes reach an unfair agreed plea bargain or settlement.</p>
<p>As <a href="https://scholarship.law.cornell.edu/cgi/viewcontent.cgi?referer=https://www.google.com/&amp;httpsredir=1&amp;article=3566&amp;context=clr" rel="nofollow noreferrer">a 1992 law review article</a> comparing outcomes in civil cases between judges and juries notes, &quot;The empirical results prove resistant to simple explanations.&quot;</p>
<p>Some key studies summarized <a href="http://criminal-justice.iresearchnet.com/forensic-psychology/jury-decisions-vs-judges-decisions/" rel="nofollow noreferrer">here</a> concluded that:</p>
<blockquote>
<p>The judge and jury in the Kalven-Zeisel survey of 3,500 criminal cases
agreed in 78% of the cases on whether or not to convict. When they
disagreed, the judge would have convicted when the jury acquitted in
19% of the cases, and the jury convicted when the judge would have
acquitted in 3% of the cases—a net leniency rate of 16%. Disagreement
rates were no higher when the judge characterized the evidence as
difficult than when the judge characterized it as easy, suggesting
that the disagreements were not produced by the jury’s inability to
understand the evidence. Disagreement rates did rise when the judge
characterized the evidence as close rather than clear, indicating that
disagreement cases were, at least in the judge’s view, more likely to
be those cases that were susceptible to more than one defensible
verdict. Primary explanations offered for the overall differences were
differences in judgments about the credibility of witnesses and a
different threshold of reasonable doubt. Two smaller, more recent
studies using the Kalven-Zeisel method have shown remarkably similar
patterns in criminal cases, obtaining 74% to 75% agreement, with a
greater leniency of 13% to 20% from the jury. Studies outside the
United States have shown similarly high levels of agreement between
professionals and juries or lay judges in criminal cases.</p>
<p>For the 4,000 civil trials in their judicial survey, Kalven and Zeisel
obtained the same agreement rate of 78% on liability, but disagreement
was almost equally divided, so that in 12% of the cases, the jury
found for the plaintiff, while the judge favored the defense and in
10% of the cases, the jury found for the defense, while the judge
would have made an award. Awards by juries were 20% higher on average
than awards by judges. Several smaller, recent studies of civil jury
cases in several locations have indicated agreement rates on liability
between 63% and 77%, but it is unclear whether any overall change has
occurred over time because no national study comparable with the
Kalven and Zeisel study has been conducted. Because punitive damages
are awarded so rarely (in roughly 3% of contract and tort cases),
researchers conducting case-specific judicial surveys have not been
able to compare judge and jury decisions on punitive damages.</p>
</blockquote>
<p>While the discrepancy between judges and juries in criminal cases (one in six cases) is quite large, this suggests that no more than 1-1.5% of criminal defendants charged with felonies in the system as a whole are inappropriately acquitted by juries, and that no more than about 0.16% of criminal defendants charged with felonies are wrongfully convicted by judges (since only about 10% of criminal trials in the U.S. are bench trials). Moreover, as discussed below, the &quot;inappropriate acquittal&quot; by juries rate as a share of system-wide felony cases is probably lower because there are good reasons to think that the higher conviction rate in bench trials than in jury trials has more to do with the biases of judges than with more accurate fact finding by judges.</p>
<p>A <a href="http://washparkprophet.blogspot.com/2010/09/impact-of-race-on-jury-verdicts.html" rel="nofollow noreferrer">2010 empirical study</a> looking at the impact of the racial composition of juries on the likelihood of convictions in felony cases in a Florida county found that statistically, a similar number of felony cases (82%) that go to trial are &quot;easy&quot; cases in which any finder of fact would reach the same result, but that the acquittal rate in other cases varied dramatically based upon the racial composition of the jury. This study provides further support for the hypothesis that close cases are more strongly influenced by the implicit biases of the fact finders than by their capacity to assess the facts accurately.</p>
<p>Incidentally, this higher conviction rate is not particular to the United States, <a href="https://www.cairn.info/revue-internationale-de-droit-penal-2001-1-page-215.htm" rel="nofollow noreferrer">it has also been true historically in Japan</a> during periods when it had a jury trial system, even though its legal codes use German legal codes as a starting point.</p>
<p>I have personally had discussions with several South Korean and Japanese judges (while I was in law school and they were pursuing graduate law degrees in the U.S. while on sabbatical from their judicial positions) who say that they have a policy of alway believing the prosecutors in criminal cases and assuming as a starting point, at least, that any defense testimony that contradicts the prosecution's case is a result of lies by the defense witnesses, which doesn't speak highly for the accuracy of bench trials in criminal cases in those countries.</p>
<p>An experienced judge <a href="https://chicagounbound.uchicago.edu/cgi/viewcontent.cgi?article=1069&amp;context=uclf" rel="nofollow noreferrer">argued in a law review article</a> in the late 1980s based upon his lengthy career presiding over jury trials, that juries were more accurate. Probably the strongest argument he makes is that judges are prone to bias, in part, though what many people would now call the process of <a href="https://en.wikipedia.org/wiki/Regulatory_capture" rel="nofollow noreferrer">regulatory capture</a>. <a href="http://via.library.depaul.edu/cgi/viewcontent.cgi?article=3927&amp;context=law-review" rel="nofollow noreferrer">Other experienced trial judges</a> echo this positive assessment of juries as noted in a 2012 article, for example, noting that juries are often better grounded in the reality of everyday life and in community standards than judges.</p>
<p>A <a href="https://www.innocenceproject.org/study-juries-often-get-it-wrong/" rel="nofollow noreferrer">2007 study</a> (full text <a href="https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.1740-1461.2007.00090.x" rel="nofollow noreferrer">here</a>) reached a conclusion on the accuracy of jury trials in criminal cases based upon the a priori assumption that judges are at least as accurate as juries, which isn't helpful at resolving your question because it assumes the answer rather than proving it, but does shed some light on the minimum extent to which one or the other triers of fact would have reached the wrong conclusion:</p>
<blockquote>
<p>A new Northwestern University study shows that juries in criminal
cases are reaching incorrect verdicts. The study, which looked at 271
cases in four areas of Illinois, found that as many as one in eight
juries is making the wrong decision – by convicting an innocent person
or acquitting a guilty one.</p>
<p>In each case, while the jury deliberated, the judge filled out a
questionnaire detailing what his or her verdict would have been had it
been a bench trial. The verdicts only matched in 77 percent of cases.
The study assumed that judges are at least as likely as a jury to make
a correct verdict, leading to the conclusion that juries are only
correct 87 percent of the time or less.</p>
<p>The study was conducted by Bruce Spencer, a Northwestern statistics
professor, and will be published in the July issue of the Journal of
Empirical Legal Studies. Spencer said in a statement that it would
take a much larger study to truly predict the accuracy of jury
verdicts nationwide in all cases.</p>
</blockquote>
<p>The same 2007 study is also critically analyzed <a href="http://jurylaw.typepad.com/deliberations/2007/07/right-and-wrong.html" rel="nofollow noreferrer">here</a>, and notes that judges again are more likely to convict than juries in cases where defendants opt for jury trials (which they do about 90% of the time in criminal cases in state courts).</p>
<p>A <a href="https://scholarlycommons.law.northwestern.edu/cgi/viewcontent.cgi?article=7269&amp;context=jclc" rel="nofollow noreferrer">different 2007 study</a> used evidence mostly derived from DNA based exonerations in capital rape-murder cases from the 1980s to conclude that the wrongful conviction rate in those cases following jury trials is 3.3% to 5%. Given that judges are much less likely to acquit than juries, this strongly suggests that the wrongful conviction rate for bench trials in serious criminal cases would be much higher than it is in jury trials. The methodology of the study doesn't make it possible to estimate the wrongful acquittal rate in jury trials which is further muddied by the question of the extent to which <a href="https://en.wikipedia.org/wiki/Jury_nullification" rel="nofollow noreferrer">jury nullification</a>, in the form of an acquittal of someone who is actually guilty beyond a reasonable doubt, is a legitimate exercise of mercy or is problematic - something also influenced by perceptions of where jury nullification is most common (e.g. this may disproportionately benefit police tried for crimes allegedly committed while carrying out their duties).</p>
<p>It is worth noting, however, that juries in capital cases (i.e. those where the death penalty is a potential sentence) are systemically less pro-defendant than other juries because they are &quot;death qualified&quot; which is to say that people opposed to the death penalty are systemically removed for cause from the jury pool, which makes juries in capital cases more likely to convict than juries in other cases. This means that it is also probably likely that wrongful conviction rates are higher in capital cases than in other cases. Indeed, one reason that prosecutors choose to bring death penalty charges in a case is to increase the odds that a defendant will plea bargain to avoid the risk of the death penalty, and to increase the odds that the defendant will be convicted at trial, in a criminal prosecution where the evidence is factually weak compared to a typical felony case that prosecutors would bring.</p>
<p>Yet <a href="https://www.researchgate.net/profile/James_Frank4/publication/249718672_Wrongful_ConvictionPerceptions_of_Criminal_Justice_Professionals_Regarding_the_Frequency_of_Wrongful_Conviction_and_the_Extent_of_System_Errors/links/55ca102408aea2d9bdcbf557.pdf" rel="nofollow noreferrer">another 2007 study</a> concluded that:</p>
<blockquote>
<p>Drawing on a sample of 798 Ohio criminal justice professionals
(police, prosecutors, defense attorneys, judges), the authors examine
respondents’ perceptions regarding the frequency of system errors
(i.e., professional error and misconduct suggested by previous
research to be associated with wrongful conviction), and wrongful
felony conviction. Results indicate that respondents perceive system
errors to occur more than infrequently but less than moderately
frequent. Respondents also perceive that wrongful felony conviction
occurs in their own jurisdictions in .5% to 1% of all felony cases,
and in the United States in 1% to 3% of all felony cases. Respondents,
however, specify an acceptable rate of wrongful conviction to be less
than .5%.</p>
</blockquote>
<p>A replication of this study with a survey of comparable Michigan officials <a href="https://www.tandfonline.com/doi/abs/10.1080/07418820801954563" rel="nofollow noreferrer">published in 2008</a> gave rise to almost identical results.</p>
<p>The review of literature in the 2007 paper surveying Ohio officials also notes the notable fact that 7% of convicted felons whose convictions are reversed on appeal and who are then retried before a new jury are acquitted (I don't have data breaking out the retrial results for capital convictions that are reversed on appeal and retried easily at hand right now), which provides another order of magnitude estimate of the reliability of jury trials (although, of course, cases reversed on appeal are by definition cases whose verdicts are not reliable for some specified reason, relative to cases in which convictions are not appealed or are affirmed on appeal).</p>
<p>About <a href="https://scholarship.law.cornell.edu/cgi/viewcontent.cgi?referer=https://www.google.com/&amp;httpsredir=1&amp;article=1366&amp;context=facpub" rel="nofollow noreferrer">40% of criminal convictions following a trial are appealed</a> (the appeal rate is very nearly 100% in death penalty cases). This is much higher than the rate of appeals of civil jury verdicts, in part, because convicted criminal defendants who cannot afford a lawyer are entitled to a direct appeal at state expense (although it can be pointless to do so if you already have a felony criminal record, you are likely to be released before the appeal is ruled upon after considering your sentence and time served, and your prospects of success on appeal are poor, which is the case in many minor felonies). About <a href="https://www.bjs.gov/content/pub/pdf/casc.pdf" rel="nofollow noreferrer">8% of cases appealed in criminal cases</a> result in a reversal of at least part of a trial court decision (although the figure is about 19% in death penalty cases). So, the odds of a criminal conviction following a trial being appealed, reversed on appeal, and then retried resulting in a conviction is roughly 0.2% (one in five hundred).</p>
<p>There is also <a href="https://heinonline.org/HOL/LandingPage?handle=hein.journals/cjust18&amp;div=7&amp;id=&amp;page=" rel="nofollow noreferrer">considerable evidence to support</a> the hypothesis that certain facts patterns are much more prone to producing wrongful convictions than others. For example, excessive faith in eye witness identification of defendants (especially in cases of white witnesses identifying black defendants who are strangers to the witness) and scientifically unsound forms of expert testimony are particularly common causes of wrongful convictions.</p>
<p>A <a href="https://scholarlycommons.law.northwestern.edu/cgi/viewcontent.cgi?referer=https://scholar.google.com/&amp;httpsredir=1&amp;article=7364&amp;context=jclc" rel="nofollow noreferrer">2010 review article</a> examining 100 years of research on the question of wrongful conviction notes that &quot;traditional sources of error (eyewitness misidentification, false confessions, perjured testimony, forensic error, tunnel vision, prosecutorial misconduct, ineffective assistance of counsel, etc.) are contributing sources, not exclusive causes, of wrongful convictions.&quot;</p>
<p>A <a href="https://heinonline.org/HOL/LandingPage?handle=hein.journals/hjlpp22&amp;div=47&amp;id=&amp;page=" rel="nofollow noreferrer">1998 law review article</a> looks at a small sample of convictions allegedly obtained with false confessions and also examines the tricky question of what &quot;innocence&quot; and a &quot;wrongful conviction&quot; really mean. For example, suppose that someone falsely confesses to facts that would make him guilty of first degree murder when in fact he was actually an accessory to manslaughter. He isn't truly &quot;innocent&quot; of any crime related to the death in question, but is likely to be convicted of a crime much more serious than the crime of which he is actually guilty.</p>
<p>A <a href="https://digitalcommons.law.villanova.edu/cgi/viewcontent.cgi?article=1177&amp;context=vlr" rel="nofollow noreferrer">2006 law review article</a> examining a single case in detail in the context of broader research finings comes to similar conclusions about high risk fact patterns, but throws in another - clusters of mass wrongful convictions driven by one or many deeply corrupt police officers in a single department who routinely lie and falsify evidence to obtain large numbers of wrongful convictions.</p>
<p>A <a href="https://blogs.wsj.com/numbers/how-accurate-are-juries-147/" rel="nofollow noreferrer">Wall Street Journal blog post</a> from July 16, 2017 cites a Journal of Empirical Legal Research article in support of the proposition that juries get about one in six criminal cases wrong and that judges do only slightly better.</p>
<p><a href="http://litigationinsights.com/case-strategies/judges-versus-jurors-decide-differently/" rel="nofollow noreferrer">Some studies</a> have looked at indications that judges are not better than juries at accurately reaching correct outcomes that don't look at the actual outcomes themselves:</p>
<blockquote>
<p>In study after study, researchers have found more similarities than
differences (see Robbennolt, 2006; Rowland, et al., 2010). For
example:</p>
<p>Retired judges were no better than college sophomores, teachers and
other novices at spotting witness mendacity.</p>
<p>Retired judges were no better than mock jurors at ignoring
inadmissible evidence.</p>
<p>Retired judges evince no greater understanding of the scientific
principles underlying Daubert standards than do mock jurors.</p>
<p>Moreover, social science research makes it very clear that many
decisions by trial judges, federal and state, are affected by
extra-legal political influences.  For example:</p>
<p>The decisions of Federal district judges, usually thought to be better
trained and less susceptible to extra-legal influences than are their
state counterparts, can for many case types (e.g., abortion) be
predicted at a high rate of accuracy simply by knowing the judge’s
appointing president.</p>
<p>Elected state judges’ sentencing decisions change in tandem with the
proximity of elections</p>
</blockquote>
<p>A comprehensive review of prior studies in <a href="https://ir.law.fsu.edu/cgi/viewcontent.cgi?article=1261&amp;context=lr" rel="nofollow noreferrer">a 2005 article</a> reiterates almost all of the conclusions from various kinds of data reached above - i.e. juries are more lenient but their skills at assessing facts are similar to those of judges.</p>
<p><strong>Arguments For And Against The Jury System Unrelated To Their Accuracy</strong></p>
<p>A <a href="https://illinoislawreview.org/wp-content/ilr-content/articles/2011/3/Gershowitz.pdf" rel="nofollow noreferrer">2011 law review article</a> reiterates that judges are far more likely to convict that juries, on average, which it argues is good even if it comes at the expense of lower accuracy. This article does not squarely address the real money question of whether judges or juries are more accurate in their decisions to convict or acquit defendants. The author argues that certainty of punishment is more important to deterring bad conduct than sentence length or the accuracy of the process.</p>
<p>A <a href="https://law.yale.edu/system/files/documents/pdf/Faculty/Langbein_Mixed_Court_and_Jury_Court.pdf" rel="nofollow noreferrer">1981 article by John Langbein</a>, one of the leading scholars in the field of comparing the European legal system (where serious criminal trials often involve &quot;mixed courts&quot; with a judge and lay judges serving together in a capacity that isn't quite a true American style jury) to the American legal system concluded that:</p>
<blockquote>
<p>The conclusion is that the mixed court serves the jury policies well,
though not fully; and that it is a superior alternative to the
indigenous nontrial devices-plea bargaining and bench trial-that have
displaced the jury from routine American practice.</p>
</blockquote>
<p>It is also worth noting that one of the important justifications for using jury trials rather than bench trials or mixed trials, is that it enhances the institutional legitimacy of the judiciary, regardless of its accuracy, in a system where both triers of fact sometimes make mistakes.</p>
<p>This is because in a jury trial system, an outcome that is perceived as incorrect can be blamed on the diffuse and effectively anonymous jury who cannot be held accountable, deflecting blame from judges who could be held accountable or whose bad decisions would reflect poorly on the judiciary, or the judge in question at least. It is easier to support the legitimacy of the judicial system if judges are rarely blamed for reaching the hard verdict in high stakes cases.</p>
<p>Also, since juries deliver only a verdict, and not a detailed justification for their ruling as a judge in a bench trial must, the jury verdict, <a href="http://www.abc.net.au/news/2013-11-28/barns-simon-gittany-and-the-case-for-judge-only-trials/5121624" rel="nofollow noreferrer">while less transparent</a>, does allow people who might potentially agree with the result even if they wouldn't have agreed with the means by which that particular jury actually reached its decision, to give the jury the benefit of the doubt, again increasing popular acceptance of the result.</p>
