Why Process Improvement?
Process Improvement in
Radiation Oncology
Sasa Mutic
A strategy, philosophy, process and leadership
approach for operating in a superior way. Results
include:
Quality
Patient safety
Patient and Employee
Satisfaction
Washington University School of Medicine
Mallinckrodt Institute of Radiology
St. Louis, Missouri
Background
Errors in Radiation Oncology
• Staff and public exposures
• Misadministrations
– Underdose
– Overdose
– Anatomical misses
• Magnitude
– From few percent to lethal
doses
– From couple of millimeters
to complete misses
• Regulatory
– Nuclear Regulatory
Commission
– Errors that do not
necessarily affect patients
but have regulatory/legal
consequences
Patient flow
Efficiency
Background
Our Environment
• Sources
– Staff
– Software
– Hardware
• Random
– Affect one to few patients
• Systematic
– Affect hundreds of patients
– Potentially in a short period
• Diverse patient population
• Diverse technology
• Diverse procedures
– Multiple and drastically different ways to treat a
single cancer site
• Enabling culture – Very few limits
– Number of patients
– Procedures
– Complexity
– etc.
Background
System Performance
Global Problem
a) The demands on
our operations
continually change
“…it calls into question the
integrity of hospital systems
and their ability to pick up
errors and the capability to
make sustainable changes.”
– Patient numbers
– Available staff
– Available machines
Sir Liam Donaldson, Chief Medical
Officer, Department of Health
Towards Safer Radiotherapy. London: The
Royal College of Radiologists, 2008
.
Radiotherapy Risk Profile, Geneva:
World Health Organization, 2009
Stable System
• To achieve stable performance, a change
in one system parameter must be
compensated for by changes in other
parameters in a timely fashion
–This change has to be proportional and
correctly directed
–Must remain in a state where changes
are expected
–Bad things may not happen for a long
time
.
b) Well designed
systems maintain
constant
performance
c) Poorly designed
systems cannot
cope with these
changes
a)
b)
c)
Process Improvement Tools
Process Improvement
A systematic approach to improve the outcomes to meet
the customer requirements.
Identify the Process requirements (Define)
Review the outcomes (Measure)
Identify the opportunities for Improvement (Analyze)
Improve the process
Control the process: Sustain the Improvement
Everything that we do is a “Process”
Define
Analyze
• Process flow chart
• Pareto Chart
• FMEA
Measure
• Trend Chart
• Statistical Process
• Control charts
• Fishbone Diagram
• Histogram
• 5-Why analysis
Improve
• Brainstorming
• Error proofing
Control
• Control Plan
• 5S
• FMEA
Lean, Six Sigma, Theory of Constraints
Petri Nets
DEFINE
We can do this very well
• Petri net is a graphical tool for describing
and studying systems that are concurrent,
parallel, distributed and stochastic
• A Petri Net consists of four components
– Place, drawn as a circle, denoting event
– Transition, drawn as a thick bar, denoting event
transfer with a period of delay time
– Arcs, drawn as arrow, connecting places and
transitions
– Token, drawn as a dot, contained in places denoting
the data
Modeling and error analysis of clinical processes in
Radiation Therapy – Oddiraju- WE-D-211A-2
IDEF0
• Developed on an initiative by the US Air Force
• Method to model activities of an organization
or a system
Define
~200 patients/day
~150 faculty, staff,
trainees
–
–
–
–
Inputs
Outputs
Controls
Mechanisms
• Model starts at the highest level and can
describe activities and processes down to the
smallest steps
WU Rad Onc
Define - Radiation Oncology Process
MEASURE
Still learning how to do this
Quantifying Event Rates
Quantify - Historical Data
• We have developed a voluntary
electronic reporting and analysis
system
• Potential and actual events (near hits
and errors) are tracked
• Track explicit and random errors
• Use the data to feed the process
improvement for our department and
potentially for other institutions
Jul-06
Jan-07
Oct-06
Apr-06
Jul-05
Jan-06
Oct-05
Apr-05
Jul-04
Jan-05
Oct-04
Apr-04
Jul-03
Jan-04
Oct-03
Apr-03
Jul-02
Jan-03
Oct-02
Apr-02
Jul-01
Jan-02
Oct-01
Apr-01
Jul-00
Jan-01
Oct-00
Apr-00
Reported Events
Quaterly Event Reports
80
70
60
50
40
30
20
10
0
Quarter
TH-D-BRD-3 – Mutic – Experience with error reporting and tracking
database tool for process improvement in radiation oncology
Our Current Data
Small to Sentinel Events
Events Recorded
250
200
No. of events recorded
“We know that single events are rare, but we do
not know how small events can become chained
together so that they result in a disastrous
outcome. In the absence of this understanding,
people must wait until some crisis actually
occurs before they can diagnose a problem,
rather than be in a position to detect a potential
problem before it emerges. To anticipate and
forestall disasters is to understand regulations in
the ways small events can combine to have
disproportionally large effects.”
150
100
50
K.E. Weick, “The vulnerable system: an analysis of the Tenerife air
disaster” in P.J. Forst et al Reframing Organizational Culture
0
Jul-07 Aug- Sep07
07
Oct07
Nov- Dec07
07
Jan08
Feb- Mar08
08
Apr08
May08
Month
Jun- Jul-08 Aug- Sep08
08
08
Oct08
Nov- Dec08
08
Jan09
Feb- Mar09
09
Dosimetry
Therapy
Therapy
IMPAC Entries
Dosimetry
Nursing
18
45
IMPAC Prob
Tx plan Incorrect
IMPAC Prob
16
40
Port-film Prob
Initial Chart to Phy late
Customer/Patient Satisfaction
Delivery Prob
% Event Free Patients
Calc Prob
35
12
30
10
25
F req uency
Frequency
Customer/Patient Satisfaction
14
8
20
6
15
4
10
5
2
0
Jul-07 Aug-07 Sep-07 Oct-07 Nov-07 Dec-07 Jan-08 Feb-08 Mar-08 Apr-08 May-08 Jun-08
0
Jul07
Dosimetry Board
Aug- Sep- Oct- Nov- Dec- Jan- Feb- Mar- Apr- May- Jun08
08
08
08
08
08
07
07
07
07
07
Jul08
Aug- Sep- Oct- Nov- Dec- Jan- Feb- Mar09
09
09
08
08
08
08
08
Tim e
T ime
Simulation
Physics
Physics
Simulator
35
35
IMPAC Prob
IMPAC Prob
Scheduling Prob
Calc Prob
25
25
20
20
Frequency
Frequency
30
Customer/Patient Satisfaction
Customer/Patient Satis fac tion
15
15
10
10
5
5
Clinic - Overall
Sim-film Prob
QA incorrect
30
0
Jul07
Jul-08 Aug-08 Sep-08 Oct-08 Nov-08 Dec-08 Jan-09 Feb-09 Mar-09
-5
-2
Aug- Sep- Oct- Nov- Dec- Jan- Feb- Mar08
08
08
07
07
07
07
07
Apr- May- Jun08
08
08
Jul08
Aug- Sep- Oct- Nov- Dec- Jan- Feb- Mar09
09
09
08
08
08
08
08
0
Jul07
Aug- Sep- Oct- Nov- Dec- Jan- Feb- Mar- Apr- May- Jun08
08
08
08
08
08
07
07
07
07
07
Jul- Aug- Sep- Oct- Nov- Dec- Jan- Feb- Mar08
08
09
09
09
08
08
08
08
-5
-5
Time
Time
Why Fluctuations?
Impac Problems
45
ANALYZE
Dosimetry
40
35
Physics
Still learning how to do this
Frequency
30
25
20
15
10
5
0
Jul- Aug- Sep- Oct- Nov- Dec- Jan- Feb- Mar- Apr- May- Jun- Jul- Aug- Sep- Oct- Nov- Dec- Jan- Feb- Mar07
07
07
07
07
07
08
08
08
08
08
08
08
08
08
08
08
08
09
09
09
Time
Why Fluctuations?
Numer of CT Simulations/Week (Includes HDR scans)
17%
Increase
65
60
50
5/4/2009
5/18/2009
4/6/2009
4/20/2009
3/9/2009
3/23/2009
2/9/2009
1/26/2009
1/12/2009
12/29/2008
12/1/2008
12/15/2008
11/3/2008
Average Performed
CT-Simulations/Week
55.4
43.9
11/17/2008
10/6/2008
9/8/2008
9/22/2008
8/25/2008
7/28/2008
7/14/2008
30
Time Span
Jan-Feb 09
Jul-Dec 08
8/11/2008
35
10/20/2008
40
2/23/2009
45
6/30/2008
Number of Scans
55
Week of
Number of Simulations
Average Jan, Feb, March
WU Rad Onc
Overall average
Our Data
Correlation Data
Clinical
Clinical
Dosimetry
Nursing
Physics
Dosimetry
Nursing
Physics
Therapy
0.0765 0.39 0.31 0.32
0.37 0.66 0.31
0.47 0.80
0.51
Therapy
Dosimetry
Front Desk
Simulation
Facilitates Critical
Point Checklists
Physics
Factor
0.1-0.3
0.3-0.6
0.6-1.0
Correlation
Small
Medium
Large
Front Desk
Simulation
0.26
0.32
0.72
0.49
0.83
0.00
0.62
0.40
0.48
0.55
0.38
Process Improvement Board
Note: The table below reflects the actual number of each event reported by month. It is not in percentage.
18
16
Aug-07
Oct-07
Dec-07
Feb-08
Apr-08
Jun-08
Aug-08
Oct-08
Jan-09
14
No. of Events
IMPROVE
12
10
8
6
Becoming much better at this
4
2
MU Calculations
Incorrect/missing
data
Incorrect Depth
Communication
Problems
Initial chart to
physics late
Tx plan incorrect
Failed to place
Rx approval on
Shift-sheet
missing/incorrect
Set-up fields
incorrect/missing
Pre-port fields not
ready
Tx calender
missing/incorrect
Entry
incorrect/missing
0
Dosimetry
Explicit Events Dosimetry
The downside to the electronic world
As implemented today
IMPROVE
R&V Related Events
• Record and Verify (R&V) system was
originally designed to operate as an
independent system (Big Brother)
• Today these systems are integral part of
the delivery process and the independent
verification process is missing
• If data in the R&V system is wrong there is
much less opportunity and chance that the
error may be discovered
Possible Solution
EQS
• Electronic QA system (EQS)
• Independent system which compares TPS
data with the data in the R&V system
• Greatly improves ability to compare initial
data transfer and consistency of data in the
R&V
Siochi et al, JCMP, 10, 2009
EQS
• Parameters that can be checked
electronically
–Plan quality and merits
–Plan parameters
–Data transfer
–Ongoing data integrity
–Weekly chart checks
• Still need manual process as the final
check
EQS
• An independent system
• Meant to complement the electronic
process and enhance patient safety
• Not intended as the only check
• The greatest benefit in pointing out
potential problems
Issue of Reporting Fatigue
Events Recorded
250
Sustain
No. of events recorded
200
Great culture, need better
feedback process
150
100
50
0
Jul-07 Aug- Sep07
07
Oct07
Nov- Dec07
07
Jan08
Feb08
Mar08
Apr08
May08
Jun- Jul-08 Aug08
08
Sep08
Oct08
Nov- Dec08
08
Month
Software Distribution and Data Collection
Conclusions
Each clinic with its own largely independent database
Hospital
Hospital
Hospital
Hospital
Hospital
Hospital
Centralized Database
Manufacturers
Regulatory Agencies
Professional Societies
• Ability to define radiation oncology
processes is very good
• Sustainable data collection possible
• Need to collect broader parameters to
determine failure triggers
• Electronic processes could facilitate
standardization among institutions and
benchmarking
Jan09
Feb- Mar09
09