Quality Standards Prevent Mistakes

Written By John E. Grant  |  Legal Project Management, Metrics & Measurement, Productivity  |  1 Comments

Share 0
Tweet 0
Share 0

Quality standards prevent mistakes.

As a standalone sentiment it seems like a no-brainier. Lawyers strive for quality: how often have you seen lawyer marketing with claims like “We provide our clients with the highest quality legal work,” or “We do quality work at an outstanding value”?

Of course we strive for quality. It’s why people hire professionals like us, and it’s what we’re trained to do (especially when it is drilled into us by our superiors).

Why, then, do lawyers keep messing up?

Because we’re human, of course, but there’s more to it than that.

I think it is because our concept of quality, as a standalone notion, is awfully subjective. Most lawyers’ notion of quality control for much of their work is “I’m an expert, so I know quality when I see it.”

It’s not that we don’t have objective standards: The Bluebook (The Bluebook: A Uniform System of Citation (Columbia Law Review Ass’n et al. eds., 20th ed. 2015)) is a series of quality standards. Entities that require Bluebook citation are essentially saying that any reference to another work does not meet the requisite definition of quality unless the cite is Bluebook-formatted. The Rules of Procedure are standards. Laws are meant to create standards of behavior. These are the writ-large quality control standards that govern the way we practice our craft.

But dive down to the actual practice of law as executed in many law offices, and that “I know it when I see it” version of quality control starts taking over.

Q: “Is this a good indemnification clause?”

A: “I don’t know, but Yoona is the expert on that, let’s have her review it.”

Q: “Is the MSJ in the Radley case ready to file?”

A: “I think so, but let me read through it one more time.”

Q: “Are we going to hit our revenue targets this quarter?”

A: “Seems like we’ve been a little light recently so I’m not sure.”

We’re shooting for quality answers to all of those questions, but, absent a defined standard, we’re going on feel.

The Checklist Manifesto Isn’t About To-Do Lists

If you’ve been paying any attention at all to law practice management theories over the last 5+ years, you’ve probably come across someone talking about Atul Gawande’s The Checklist Manifesto. Maybe you’ve read it yourself (or the bite-sized New Yorker article based on it). However you’ve encountered it, you probably were drawn to the title—humans love checklists, especially to-do lists.

But have you ever paid attention to the full title of the book?

In case your images aren’t loading, it reads “The Checklist Manifesto: How To Get Things Right.”

Not “How To Get Things Done.” Not “How To Do More With Less.”

How To Get Things Right.

This is a book about quality control.

Take the pre-flight checklist that pilots use. At first blush you might think of it as a to-do list containing all the things you need accomplish before the plane leaves the ground. If we were still allowed to see into the cockpit, you might even witness the pilot and co-pilot running down the list, verbally “check”ing each item down the page.

The point Gawande makes, however, is that the checklist itself doesn’t tell the pilots how to get the plane from LA to Alberquerque—that requires training and experience and expertise.

At the same time, these highly-trained experts would never strap into the pilot’s seat, take a look around, and say, “this plane is ready for takeoff because I know ready when I see it.” They run through the pre-flight checklist. Every. Single. Time. Even when it contains items that seem painfully obvious, or may have never not been “checked” in the life experience of that pilot. Why?

Because the pre-flight checklist is not a to-do list, it is a quality control standard. It doesn’t tell the pilot how to get from point A to point B through the air, it establishes a set of conditions that must be true in order for that journey to be safe. The pilots know that checking each and every item on the list gives them the best possible chance of a quality outcome to the flight.

Put another way, they know that if any one of the items on that list is not true, the occupants of the plane (pilots included) risk a decidedly low-quality experience.

Manafort’s Redaction Mistake

I was inspired to write this post from a news event in early 2019. Here are a couple of typical headlines:

Botched redactions expose new details of Mueller’s Manafort case (MSNBC), and

Manafort attorney’s redaction error reveals he provided Trump polling data to Russian operative (Washington Times).

Pretty much all political ideologies agree this was a big oops.

This news story immediately sparked discussion among the legal Twitterati about (1) how stupid Manafort’s lawyers must be to make this mistake and (2) how they are clearly not using the right tools. I snarked about it too.


I’m not going to debate #1: This kind of mistake, especially in a high-profile case, is clearly pretty bad (unless you believe it was done intentionally). But the second part of the discussion, the one about tools, totally misses the point.

Some people couldn’t believe that Manafort’s lawyers weren’t using / paying for Acrobat DC:

Others suggested going old-school:

Turns out they both work (as do many other methods), but why?

Because they perform against a simple, yet mostly undocumented, quality standard for removing sensitive information from a public document. I think we can boil that standard down to three elements:

?  Sensitive information is not visible to the naked eye.
?  Sensitive information is not present in document metadata.
?  Protections cannot be removed or reversed.

If these three conditions are true, then you can release your document (formerly) containing sensitive information into the wild and maintain a quality outcome. If any of the three conditions are false… well, hello world.

Not only does adherence to the standard help ensure quality, it can prevent unnecessary work (or worry). Take this response to the redaction debate for example:

Not to pick on Keren, but I wouldn’t want to pay my lawyer to execute two viable redaction methods when one will suffice (i.e. meet the standard). Lack of confidence in your standard (or that your method meets it) can result in waste.

And, as Ivy Grey pointed out, there are lots of tools for meeting this standard, but you need to take the time to (1) ensure the tool you are using executes all three steps and (2) you know how to use it.

Learn to use the tools you have properly. That’s the key.

I’d venture that just about every lawyer in America has the tools necessary to execute the standard: That old school “sharpie and copier” method does, in fact, work. If you don’t need to redact things very often, this is a perfectly acceptable tool set. Sure it is inefficient, but why invest in fancy tech to solve an infrequent problem?

If you redact more often, or if you need PDF software for other things (like Bates-stamping or e-filing), then investing in a tool makes more sense. But you need to gauge the tool’s performance against your quality standard—for redaction and for anything else you use it to do. If you don’t have a standard, you’re going on hearsay (X told me this is the best tool, and he’s the expert, so I’m going with it).

The importance of quality standards grows with the importance of the work you’re doing. Driving to work in the morning? A rough standard may do. Trying to fly a multi-million dollar spaceship to Pluto (and beyond)? You’ll probably need to ensure that a few more stringent standards are met.

How to Use Quality Control Standards

This is a lengthy topic on its own, but three thoughts on how to use QC standards effectively:

(1) Make them clear. Remember, this is not a to-do list. Especially at first, it makes no sense to dive into too much detail. “Sensitive Information is not visible to the naked eye” is a standard. “Select the desired text, choose redact from the menu, select the redaction color to black, etc.” is not.

(2) Write them down. Standards that aren’t in writing have a way of morphing to fit the situation. Or going through a game of “telephone” as they get communicated to your team. A standard should be consistent, and the best way to do that is to put it in writing.

(3) Measure work against the standard. Before I went to law school I spent nearly a decade in the tech industry where quality control / quality assurance is a separate and necessary phase to every project. There is an entire profession dedicated to performing this function. In legal, we too often leave it as an afterthought. Or, because we’re highly-trained and experienced professionals we assume we’ll get it right. This is folly.

One of the key takeaways of The Checklist Manifesto is how the outcome of work performed by highly trained professionals (the primary story in the book is about surgeons) improved dramatically when their teams defined a simple quality standard and then empowered every member of the team to measure work against that standard. If it works for surgeons, it will work for you.

Have questions about standards and how to apply them in your practice? Join one of my online office hours sessions and I’ll answer any legal ops questions you may have. Just .

Post 2019-3

 Previous
Next
    • {"email":"Email address invalid","url":"Website address invalid","required":"Required field missing"}
    >