Why Process Improvement? Process Improvement in Radiation Oncology Sasa Mutic A strategy, philosophy, process and leadership approach for operating in a superior way. Results include: Quality Patient safety Patient and Employee Satisfaction Washington University School of Medicine Mallinckrodt Institute of Radiology St. Louis, Missouri Background Errors in Radiation Oncology • Staff and public exposures • Misadministrations – Underdose – Overdose – Anatomical misses • Magnitude – From few percent to lethal doses – From couple of millimeters to complete misses • Regulatory – Nuclear Regulatory Commission – Errors that do not necessarily affect patients but have regulatory/legal consequences Patient flow Efficiency Background Our Environment • Sources – Staff – Software – Hardware • Random – Affect one to few patients • Systematic – Affect hundreds of patients – Potentially in a short period • Diverse patient population • Diverse technology • Diverse procedures – Multiple and drastically different ways to treat a single cancer site • Enabling culture – Very few limits – Number of patients – Procedures – Complexity – etc. Background System Performance Global Problem a) The demands on our operations continually change “…it calls into question the integrity of hospital systems and their ability to pick up errors and the capability to make sustainable changes.” – Patient numbers – Available staff – Available machines Sir Liam Donaldson, Chief Medical Officer, Department of Health Towards Safer Radiotherapy. London: The Royal College of Radiologists, 2008 . Radiotherapy Risk Profile, Geneva: World Health Organization, 2009 Stable System • To achieve stable performance, a change in one system parameter must be compensated for by changes in other parameters in a timely fashion –This change has to be proportional and correctly directed –Must remain in a state where changes are expected –Bad things may not happen for a long time . b) Well designed systems maintain constant performance c) Poorly designed systems cannot cope with these changes a) b) c) Process Improvement Tools Process Improvement A systematic approach to improve the outcomes to meet the customer requirements. Identify the Process requirements (Define) Review the outcomes (Measure) Identify the opportunities for Improvement (Analyze) Improve the process Control the process: Sustain the Improvement Everything that we do is a “Process” Define Analyze • Process flow chart • Pareto Chart • FMEA Measure • Trend Chart • Statistical Process • Control charts • Fishbone Diagram • Histogram • 5-Why analysis Improve • Brainstorming • Error proofing Control • Control Plan • 5S • FMEA Lean, Six Sigma, Theory of Constraints Petri Nets DEFINE We can do this very well • Petri net is a graphical tool for describing and studying systems that are concurrent, parallel, distributed and stochastic • A Petri Net consists of four components – Place, drawn as a circle, denoting event – Transition, drawn as a thick bar, denoting event transfer with a period of delay time – Arcs, drawn as arrow, connecting places and transitions – Token, drawn as a dot, contained in places denoting the data Modeling and error analysis of clinical processes in Radiation Therapy – Oddiraju- WE-D-211A-2 IDEF0 • Developed on an initiative by the US Air Force • Method to model activities of an organization or a system Define ~200 patients/day ~150 faculty, staff, trainees – – – – Inputs Outputs Controls Mechanisms • Model starts at the highest level and can describe activities and processes down to the smallest steps WU Rad Onc Define - Radiation Oncology Process MEASURE Still learning how to do this Quantifying Event Rates Quantify - Historical Data • We have developed a voluntary electronic reporting and analysis system • Potential and actual events (near hits and errors) are tracked • Track explicit and random errors • Use the data to feed the process improvement for our department and potentially for other institutions Jul-06 Jan-07 Oct-06 Apr-06 Jul-05 Jan-06 Oct-05 Apr-05 Jul-04 Jan-05 Oct-04 Apr-04 Jul-03 Jan-04 Oct-03 Apr-03 Jul-02 Jan-03 Oct-02 Apr-02 Jul-01 Jan-02 Oct-01 Apr-01 Jul-00 Jan-01 Oct-00 Apr-00 Reported Events Quaterly Event Reports 80 70 60 50 40 30 20 10 0 Quarter TH-D-BRD-3 – Mutic – Experience with error reporting and tracking database tool for process improvement in radiation oncology Our Current Data Small to Sentinel Events Events Recorded 250 200 No. of events recorded “We know that single events are rare, but we do not know how small events can become chained together so that they result in a disastrous outcome. In the absence of this understanding, people must wait until some crisis actually occurs before they can diagnose a problem, rather than be in a position to detect a potential problem before it emerges. To anticipate and forestall disasters is to understand regulations in the ways small events can combine to have disproportionally large effects.” 150 100 50 K.E. Weick, “The vulnerable system: an analysis of the Tenerife air disaster” in P.J. Forst et al Reframing Organizational Culture 0 Jul-07 Aug- Sep07 07 Oct07 Nov- Dec07 07 Jan08 Feb- Mar08 08 Apr08 May08 Month Jun- Jul-08 Aug- Sep08 08 08 Oct08 Nov- Dec08 08 Jan09 Feb- Mar09 09 Dosimetry Therapy Therapy IMPAC Entries Dosimetry Nursing 18 45 IMPAC Prob Tx plan Incorrect IMPAC Prob 16 40 Port-film Prob Initial Chart to Phy late Customer/Patient Satisfaction Delivery Prob % Event Free Patients Calc Prob 35 12 30 10 25 F req uency Frequency Customer/Patient Satisfaction 14 8 20 6 15 4 10 5 2 0 Jul-07 Aug-07 Sep-07 Oct-07 Nov-07 Dec-07 Jan-08 Feb-08 Mar-08 Apr-08 May-08 Jun-08 0 Jul07 Dosimetry Board Aug- Sep- Oct- Nov- Dec- Jan- Feb- Mar- Apr- May- Jun08 08 08 08 08 08 07 07 07 07 07 Jul08 Aug- Sep- Oct- Nov- Dec- Jan- Feb- Mar09 09 09 08 08 08 08 08 Tim e T ime Simulation Physics Physics Simulator 35 35 IMPAC Prob IMPAC Prob Scheduling Prob Calc Prob 25 25 20 20 Frequency Frequency 30 Customer/Patient Satisfaction Customer/Patient Satis fac tion 15 15 10 10 5 5 Clinic - Overall Sim-film Prob QA incorrect 30 0 Jul07 Jul-08 Aug-08 Sep-08 Oct-08 Nov-08 Dec-08 Jan-09 Feb-09 Mar-09 -5 -2 Aug- Sep- Oct- Nov- Dec- Jan- Feb- Mar08 08 08 07 07 07 07 07 Apr- May- Jun08 08 08 Jul08 Aug- Sep- Oct- Nov- Dec- Jan- Feb- Mar09 09 09 08 08 08 08 08 0 Jul07 Aug- Sep- Oct- Nov- Dec- Jan- Feb- Mar- Apr- May- Jun08 08 08 08 08 08 07 07 07 07 07 Jul- Aug- Sep- Oct- Nov- Dec- Jan- Feb- Mar08 08 09 09 09 08 08 08 08 -5 -5 Time Time Why Fluctuations? Impac Problems 45 ANALYZE Dosimetry 40 35 Physics Still learning how to do this Frequency 30 25 20 15 10 5 0 Jul- Aug- Sep- Oct- Nov- Dec- Jan- Feb- Mar- Apr- May- Jun- Jul- Aug- Sep- Oct- Nov- Dec- Jan- Feb- Mar07 07 07 07 07 07 08 08 08 08 08 08 08 08 08 08 08 08 09 09 09 Time Why Fluctuations? Numer of CT Simulations/Week (Includes HDR scans) 17% Increase 65 60 50 5/4/2009 5/18/2009 4/6/2009 4/20/2009 3/9/2009 3/23/2009 2/9/2009 1/26/2009 1/12/2009 12/29/2008 12/1/2008 12/15/2008 11/3/2008 Average Performed CT-Simulations/Week 55.4 43.9 11/17/2008 10/6/2008 9/8/2008 9/22/2008 8/25/2008 7/28/2008 7/14/2008 30 Time Span Jan-Feb 09 Jul-Dec 08 8/11/2008 35 10/20/2008 40 2/23/2009 45 6/30/2008 Number of Scans 55 Week of Number of Simulations Average Jan, Feb, March WU Rad Onc Overall average Our Data Correlation Data Clinical Clinical Dosimetry Nursing Physics Dosimetry Nursing Physics Therapy 0.0765 0.39 0.31 0.32 0.37 0.66 0.31 0.47 0.80 0.51 Therapy Dosimetry Front Desk Simulation Facilitates Critical Point Checklists Physics Factor 0.1-0.3 0.3-0.6 0.6-1.0 Correlation Small Medium Large Front Desk Simulation 0.26 0.32 0.72 0.49 0.83 0.00 0.62 0.40 0.48 0.55 0.38 Process Improvement Board Note: The table below reflects the actual number of each event reported by month. It is not in percentage. 18 16 Aug-07 Oct-07 Dec-07 Feb-08 Apr-08 Jun-08 Aug-08 Oct-08 Jan-09 14 No. of Events IMPROVE 12 10 8 6 Becoming much better at this 4 2 MU Calculations Incorrect/missing data Incorrect Depth Communication Problems Initial chart to physics late Tx plan incorrect Failed to place Rx approval on Shift-sheet missing/incorrect Set-up fields incorrect/missing Pre-port fields not ready Tx calender missing/incorrect Entry incorrect/missing 0 Dosimetry Explicit Events Dosimetry The downside to the electronic world As implemented today IMPROVE R&V Related Events • Record and Verify (R&V) system was originally designed to operate as an independent system (Big Brother) • Today these systems are integral part of the delivery process and the independent verification process is missing • If data in the R&V system is wrong there is much less opportunity and chance that the error may be discovered Possible Solution EQS • Electronic QA system (EQS) • Independent system which compares TPS data with the data in the R&V system • Greatly improves ability to compare initial data transfer and consistency of data in the R&V Siochi et al, JCMP, 10, 2009 EQS • Parameters that can be checked electronically –Plan quality and merits –Plan parameters –Data transfer –Ongoing data integrity –Weekly chart checks • Still need manual process as the final check EQS • An independent system • Meant to complement the electronic process and enhance patient safety • Not intended as the only check • The greatest benefit in pointing out potential problems Issue of Reporting Fatigue Events Recorded 250 Sustain No. of events recorded 200 Great culture, need better feedback process 150 100 50 0 Jul-07 Aug- Sep07 07 Oct07 Nov- Dec07 07 Jan08 Feb08 Mar08 Apr08 May08 Jun- Jul-08 Aug08 08 Sep08 Oct08 Nov- Dec08 08 Month Software Distribution and Data Collection Conclusions Each clinic with its own largely independent database Hospital Hospital Hospital Hospital Hospital Hospital Centralized Database Manufacturers Regulatory Agencies Professional Societies • Ability to define radiation oncology processes is very good • Sustainable data collection possible • Need to collect broader parameters to determine failure triggers • Electronic processes could facilitate standardization among institutions and benchmarking Jan09 Feb- Mar09 09