8/2/2012
PQI – Control Charts, Event
Reporting, and FMEA
Todd Pawlicki
Elements of PQI Projects
•
•
•
•
•
Relevance to patient care
Relevance to diplomate's practice
Identifiable metrics and/or measurable endpoints
Practice guidelines and technical standards
An action plan to address areas for improvement
– Subsequent remeasurement to assess progress and/or
improvement
http://www.theabr.org/moc-ro-comp4
Error Management
• Reactive, Proactive, Prospective
– Three approaches to error management
• Incident learning systems
– Reactive & Proactive
• Failure Modes & Effects Analysis
– Prospective
1
8/2/2012
Basis for Understanding Statistical
Process Control
Tolerance Limits
Action
Limits
Action
Limits
Accept
Target
XmR
chart
Individual
values
Control Charts: Individual Values
mR
chart
Moving
Range
Sample number or Time
Sample number or Time
XmR Chart
mR
1.128
x=
1
N
∑x
Individual
values
UNPL = x + 3 ⋅
Sample number or Time
LNPL = x − 3 ⋅
mR
1.128
(n = 1, and use d2 for n = 2)
2
8/2/2012
Two Example
Control Charts
• Clinical specifications
– Set process
requirements
• Control chart limits
– Quantify process
performance
Pawlicki, Yoo, Court et al. Radiother Oncol 2008
Event Reporting System
http://www.ihe.ca/publications/library/archived/a-reference-guide-for-learning-fromincidents-in-radiation-treatment
Investigation
• All incidents are investigated
• Depth and priority of investigation depends on
– Severity of incident
– Frequency of occurrence
• Assessment
– Impact and process domain(s)
• Report
– Causal analysis, corrective actions, and follow-up
3
8/2/2012
Choosing A Project From Events
• By type
– Clinical, occupational, operational, environmental, security/other
• By impact
– Critical, major, serious, minor
– Near miss
• By domain
– Where in the Radiation Treatment process did the incident
occur?
Corrective Action
• Actions to address causes
– Target to improve system performance
• Integrate with other business processes
– Capital budgeting
– Change management
– Training
• Assign to individuals
• Follow up reports / data
Learning
• Lessons learned are distilled and communicated
• Supervisor responsible for communication
• Quality Assurance Committee responsible for
overall review of incident patterns
• Communication requirements depending on
incident severity
– Stop the press vs. Department email
4
8/2/2012
Example: Forgetting bolus
Example: Forgetting bolus
Control Chart
FMEA
• Failure Modes and Effects Analysis
– Provides a structured way of prioritizing risk reduction
strategies.
– Helps to focus efforts aimed at minimizing adverse
outcomes.
5
8/2/2012
FMEA – Background
• History
– Developed by the Aerospace industry (~1960s)
• In the electromechanical age
– Widely applied in automotive and airline industries
• Use
– Most effective when applied before a design is
constructed
– Primarily a prospective tool
FMEA – Vocabulary
• Failure Mode: How a part or process can fail to
meet specifications.
• Cause: A deficiency that results in a failure
mode; sources of variation.
• Effect: Impact on customer if the failure mode is
not prevented or corrected.
Risk Priority Number (RPN)
Risk Priority Number =
Probability of Occurrence
X
Severity
X
Probability of NOT being
detected
6
8/2/2012
FMEA – Metrics
• Occurrence (O)
– Probability that the failure mode occurs
• Severity (S)
– Severity of the effect on the final outcome resulting from the
failure mode if it is not detected
• Lack of Detectability (D)
– Probability that the failure will NOT be detected
No input/control Responsible for operation
9) Sources Ordered
10) Sources inventoried
11) Sources delivered
to Radiation Oncology
12) Sources inventoried
into Rad Onc
1) MD consult
2) H&P
3) Database entry
4) Prescription
dictated
Initial
Patient Consult
16) Plaque insert
17) Patient survey
18) Room survey
Source Acquisition
Implant
Processes leading to
LDR Implant
Slide courtesy of
Dan Scanderbeg
Successful LDR
Implant
Treatment Plan
Plaque Preparation
5) Source type
selected
6) Hand
calculation
7) Treatment plan
8) Source activity
selected
13) Calibration check
14) Assembly
15) Sterilization
Process Step
Potential
Failure Mode
Effect of
Failure Mode
O rank
S Rank
D rank
5) Source type
selected
Wrong source
type selected
Wrong dose
delivered
7
9
9
567
6) Hand calc
Wrong depth or
duration
Wrong dose
delivered
8
9
7
504
7) Tx Plan
Wrong depth or
duration
Wrong dose
delivered
9
9
7
567
8) Source
activity selected
Wrong source
activity selected
Wrong dose
delivered
7
9
6
378
9) Order placed
Wrong activity
ordered
Wrong dose
delivered
5
8
6
240
14) Assembly
Improper
equipment used
Wrong dose
delivered/geogra
phic miss
7
9
9
567
14) Assembly
Improper
construction
Seeds migrate
6
9
4
216
15) Sterilization
Improper
handling
Seeds migrate
5
9
4
180
RPN
Slide courtesy of
Dan Scanderbeg
7
8/2/2012
List sorted in order of RPN (high to low)
RPN ~ 550 used as cutoff
Process Step
Potential
Failure Mode
Effect of Failure
Mode
5) Source type
selected
Wrong source type
selected
Wrong dose
delivered
7
9
9
567
7) Tx Plan
Wrong depth or
duration
Wrong dose
delivered
9
9
7
567
14) Assembly
Improper equipment
used
Wrong dose
delivered/geographi
c miss
7
9
9
567
6) Hand calc
Wrong depth or
duration
Wrong dose
delivered
8
9
7
504
8) Source activity
selected
Wrong source
activity selected
Wrong dose
delivered
7
9
6
378
9) Order placed
Wrong activity
ordered
Wrong dose
delivered
5
8
6
240
14) Assembly
Improper
construction
Seeds migrate
6
9
4
216
15) Sterilization
Improper handling
Seeds migrate
5
9
4
180
O rank
S Rank
D rank
RPN
Slide courtesy of
Dan Scanderbeg
Slide courtesy of Dan Scanderbeg
Case Identifier
Type of Case
Physician
Scheduled Time of Case
Scheduled Duration of Case
Physics Start Time for Case
Physics Stop Time for Case
Paperwork & Notes
Example of data
tracking
Use web-based
form to gather
data into Exceltype form for
analysis.
Slide courtesy of Dan Scanderbeg
Example of Analysis
• Over 3 weeks – physics brachy schedule was
logged using Google Documents
• Results
– 20 of 26 (77%) of cases finished later than scheduled
– Cases finished later than scheduled time
• Max = 78 min
• Ave = 31.5 min
– 8 occurrences of cases booked back-to-back
– 4 occurrences of cases doubled booked
8
8/2/2012
Next Steps
• Create intervention to improve processes
• Document results
Summary
• Control charts for analysis and deciding
when to act
• Event Recording System to identify issues
• FMEA to prioritize effort
9