Rate of Wrongful Convictions – You Can’t Improve What You Don’t Measure – Six Sigma and the US Justice System

(Editor’s note:  some readers may find this post a little “dry”, but I believe it’s none the less relevant.)

Just what is the rate of wrongful convictions in the US?  Nobody knows for sure.

Estimates of the rate of wrongful convictions in the US span a range from 0.5% to 5.0%, based upon several recent studies done by law schools, and cited on this blog.  SCOTUS Justice Antonin Scalia happens to believe that it’s 0.027%, but he is clearly not connected with reality.  Anything in the 0.5% to 5.0% range is a staggering number, and would cry out for remedy.

What if we were confident that the justice system produced the correct result 99.99966% of the time, and we had the data to prove it?  Another way to say this is that the justice system would get the verdict ‘wrong’ only 3.4 times per million cases, or a 0.00034% wrongful conviction rate.  That would be a wonderful thing.

This is exactly the kind of performance we have come to expect from the companies that manufacture all the products we buy and use very day.  You might be old enough to remember (unfortunately, I am) the days when it was not that uncommon to buy a new product, get it home, take it out of the box, and have it not work, or be missing a part, or have cosmetic blemishes.  This unacceptable level of product defects was costing industry a huge amount of money, because they would have to make good on warranties, and they were also losing customers.  However, it took them quite a while to finally realize this.

A perfect example of how an intolerable situation with process output defects was turned around is Japan in the early 50’s.  At that time, anything with ‘Made in Japan’ on it was globally, universally synonymous with “junk”.  W. Edwards Deming started working with Japanese industry in the early 50’s, teaching them the concepts of statistical quality management.  The Japanese embraced this, because they saw and understood its value, and by the 80’s, the Japanese were universally recognized as producing flawless products.  Their reputation for quality became known around the world.

In the early 80’s, Motorola recognized that they were losing billions of dollars (that’s “billion” with a “B”) on product quality issues.  In response to that situation, they invented a thing called Six Sigma.  Six Sigma started with Deming’s principles of statistical quality control and wove them into a structured methodology for improving the output defect rate of any process.  And by the way, the “justice system” is a process – it has inputs and outputs, the ultimate output being a verdict in a criminal case.

The nominal goal of a “six sigma” effort is to improve a process to the point where the number of process output defects statistically falls within the sixth standard deviation from the mean – there’s lots of statistics here, but that basically means 3.4 defects per million; and, it doesn’t have to stop there.  The statistical symbol for “standard deviation” is the lower case greek letter sigma, σ.  Hence, the name “six sigma”.  This methodology has been widely adopted by industry; the result being that when you bring a new product home today, you absolutely expect it to work right out of the box.

Why not apply six sigma to the justice sytem?!  Imagine a justice system with a wrongful conviction rate of 0.00034%.  By applying six sigma principles, this kind of result, and even better, is realistically achievable.  Well …. there are some practical reasons why this can’t be done in general – not the least of which is that the key components of the justice system are comprised of politically elected positions – legislatures, judges, prosecutors.  And every state has its own unique version of a “justice system”.  But there are six sigma concepts that can be applied in “pieces”.  To start, the most basic underlying principle of six sigma is MEASUREMENT.  If you can’t (or don’t) measure it, you can’t improve it.  You have to have the DATA to be able to determine what is broken, how badly it’s broken, and where; before you undertake to fix it.

Referring back to the opening paragraph – we can only estimate what the wrongful conviction rate in the US is, and only then within a broad range.  That’s because the data doesn’t exist to do anything better.  We really don’t even understand the real size of the problem today.

Implementing six sigma also requires some means of taking the resulting output data, determining the root cause of problems, and feeding it back into the process design to effect the desired improvement. In the case of the justice system, the “process design” function is accomplished by state legislatures (or Congress) – probably the least effective and least efficient mechanism to get anything done I can think of.  As an example – I would suggest that one of the root causes of wrongful convictions is having politically elected prosecutors.  If it were determined by actual data that this is true, then the feedback into the process design would result in a law to make ‘prosecutor’ a non-partisan appointed or hired position.

In fact, undertaking any kind of six sigma related changes to the justice system would require passage of a law to even get the process started.  It would have to be funded, and somebody would have to be “in charge”; all with the sanction of the legislature, and we know what a Herculean task that is.  However, Ohio’s SB77 is an outstanding example of what is possible.  This law has become a nationwide model for justice system reform, and it’s passage was justified with the limited data that we do have today (and an incredible amount of hard work by a group of highly dedicated people).  Imagine the case that could be presented to a legislature if we had comprehensive, hard data.

The new National Registry of Exonerations is a huge step in the right direction.  It’s the best source of hard data on wrongful convictions I have seen to date.  The Innocence Project also maintains a data base on exonerations based upon DNA.  But this is just a start.  We need much more comprehensive data collection at the state level, because this is where the laws get made.  We have no idea how many wrongful convictions and innocent incarcerated people disappear into the shadows, and nobody ever knows about them.  If we just had the data to know what to fix or change, and to prove the case, we could start bringing some light to the shadows.  Maybe  ….  maybe  …  we could even start approaching six sigma quality in our justice system.

You can’t fix what you don’t measure.

10 responses to “Rate of Wrongful Convictions – You Can’t Improve What You Don’t Measure – Six Sigma and the US Justice System

  1. I looked at the Ohio Senate Bills numbered 77 from 1997 to the present, and couldn’t find anything like what you described. Can you tell us what year, or the Ohio Revised Code section where it can be found, please?

  2. morrisonbonpasse

    Thank you for bringing commonplace private sector standards to bear on the legal system. Why not? The Six Sigma goal would be 34 wrongful convictions per ten million criminal trials per year. Even that number seems intolerable. Nonetheless, how many criminal trials are concluded annually in the U.S.? With that number in hand, a realistic goal can be set.
    Morrison Bonpasse
    Chad Evans Wrongly Convicted Committee

  3. Docile Jim Brady – Columbus OH 43209

    Good write and sorry I missed it when first posted.

  4. Pingback: Why I Think the US Justice System is Broken – and Why It’s Not Getting Fixed | Wrongful Convictions Blog

  5. Phil, You’ve been way ahead of the pack. Sorry I missed your July 2, 2012, article. Thank you for posting it in Martin Yant’s article! Both of you have hit at the core of identifying the crisis and solutions to tackle the broken criminal justice system, to restore fair administration of justice across the board.

  6. Update: 1,347 exonerations! The National Exoneration Registry http://www.law.umich.edu/special/exoneration/Pages/about.aspx

  7. Pingback: Justice System Reform – Why We Can’t Get it Right. It’s All About Root Cause. | Wrongful Convictions Blog

  8. Pingback: A Major Cause of Wrongful Convictions …….. POLITICS ?! | Wrongful Convictions Blog

  9. Pingback: Comment on the Nature and State of the (US) Justice System | Wrongful Convictions Blog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s