Client Report: A. Peter Catinella, MD, MPH
APeter.Catinella@bannerhealth.com
John Wauters
(Statlab student)
11/6/2015
Executive summary:
You met with the Statlab (Dr. Billheimer and student J. Wauters) about a paper you are writing
to explain/introduce QI (quality improvement statistics and methods) to family physicians. Your
concern was when to use QI methods, and when not to.
Background:
QI (quality improvement) is a system of techniques for monitoring the stability and
predictability of a process, originally developed for use in industry. The roots of QI can be traced to
methods invented by Dr. Walter Shewhart of Bell Labs in the 1920's. In brief, QI involves a "toolbox"
of techniques for analyzing time-series data for signs of non-random variation, with the intent of
identifying and minimizing as much of the variability as possible and maximizing consistency.
You have a significant background and experience with QI methods (which you first learned
while teaching in Utah), but you've noticed many physicians don't use them, and many others don't use
them correctly. This is a problem not only because of the explosive growth of the use of QI charts and
tools in published papers (a phenomenon which ironically is itself a subject of published papers), but
also because physicians now must implement a quality improvement program to maintain their medical
licenses. You have written a paper (or possibly the first of two papers) introducing family physicians to
the basic concepts and tools of QI, and had some concern if you had the statistics correct.
Problems:
There are several obstacles to widespread correct implementation of QI methods in a family
practice setting, but they are not overwhelming obstacles. QI uses a relatively small set of relatively
simple tools, but they are not obvious and require family doctors (or anyone else for that matter) to take
the time to learn and implement. Implementing QI can also require a change in mindset, from thinking
of each patient as having a unique set of issues and responses to thinking about a process that must be
consistent across all patients. QI uses statistical concepts, but uses a different vocabulary than
mathematical statistics: "voice of the customer" is not a term most med students learn in their
epidemiology classes. As you point out in your paper, QI takes a very different approach than statistics
doctors were probably exposed to in med school: where statistical research typically attempts to show a
difference between a treatment and control group at a specific time by showing the assumption of no
difference makes the observed variation highly unlikely, QI methods are looking for a lack of
difference between a process and a desired mean outcome by observing that the patterns in the response
over an extended period are highly likely. Also, there is the issue that even if the tools of QI are
procedurally understood, the issue of when to apply them remains. As you point out in your paper,
there are some issues that should always be investigated, regardless of trends or runs in data. The
application of QI assumes a consistent procedure is in place, which may or may not be the case. Poor
or inconsistent data collection will undermine the process. Control charts are not designed to
distinguish between better or worse interventions. The right chart must be used for the right task. Even
the right tool may not detect a change quickly.
Page 1 of 2
Client Report: A. Peter Catinella, MD, MPH
APeter.Catinella@bannerhealth.com
Results:
The paper you have written is readable, informative, uses informative figures, and is (as best I
can determine) factually correct.
Conclusion:
Your paper is a good introduction to the concepts of QI, and highlights many of the issues with
correctly implementing a QI program.
Page 2 of 2