The New EP23 ‒ Laboratory
Quality Control Based on
Risk Management,
Approved Guideline
Luann Ochs, MS
Senior Vice President – Operations
Clinical and Laboratory Standards Institute
Objectives
• Describe the various types of control processes.
• Identify CLSI document EP23-A as a resource for
developing a laboratory quality control (QC) plan based on
risk management.
• Review key aspects of risk management.
2
Today’s Quality Control
• Advantages
– QC monitors the end product (result) of the entire test system.
– QC has target values: if assay recovers target, then everything is
assumed stable (instrument, reagent, operator, sample).
• Disadvantages
– When a problem is detected, one must go back and reanalyze
patients since last “good” QC.
– If results are released, then results may need to be corrected.
– For Point of Care devices, does traditional QC work for every
test?
• Need to get to fully automated analyzers that eliminate
errors up front
– Until that time, need a robust QC plan (QCP)
3
Types of Quality Control
• “On-Board” or Analyzer QC – built-in device
controls or system checks
• Internal QC – laboratory-analyzed surrogate
sample controls
• External QC – blind proficiency survey
• Other types of QC – control processes either
engineered by a manufacturer or enacted by a
laboratory to ensure result reliability
4
5
Quality Control Limitations
• No single QC procedure can cover all devices,
because the devices may differ.
• QC practices developed over the years have provided
laboratories with some degree of assurance that
results are valid.
• Newer devices have built-in electronic controls, and
“on-board” chemical and biological controls.
• QC information from the manufacturer increases the
user’s understanding of device’s overall quality
assurance requirements.
6
The Quality Control Toolbox
• Every QC tool has its strengths and weaknesses (there is
no perfect QC tool).
• Implement a combination of tools in order to properly
control a test.
• EP23 explains the strengths and weaknesses of the
different QC processes, and helps the laboratory
determine the right combination of tools: The Right QC
7
CLSI Document EP23
• Laboratory Quality Control Based on Risk Management;
Approved Guideline
• James H. Nichols, PhD, DABCC, FACB, Chairholder of
the document development committee
• EP23 describes good laboratory practice for developing a
QCP based on the manufacturer’s risk mitigation
information, applicable regulatory and accreditation
requirements, and the individual health care and
laboratory setting.
8
The Scenario
• CLSI document EP23 provides guidance on
developing an appropriate QCP that will:
–
–
–
–
Save time and money.
Use electronic and/or integrated QC features.
Use other sources of QC information.
Conform to one’s laboratory and clinical use of the test.
9
Developing a QCP
MEASURING SYSTEM INFORMATION
Medical
Requirements for
the Test Results
Regulatory and
Accreditation
Requirements
Test System Information
- Provided by the Manufacturer
- Obtained by the Laboratory
Information About
Health Care and
Test Site Setting
PROCESS
Risk Assessment
OUTPUT
Quality Control Plan
PROCESS
Post-implementation Monitoring
10
Measuring System Information
• Gather information from several sources:
– Medical requirements for the test results
• Allowable performance specifications via physicians
– Regulatory and accreditation requirements
• Clinical Laboratory Improvement Amendments
– Test/test system information
• User’s manual, reagent package insert, literature
• Prior laboratory data
– Health care and testing site settings
• Temperature conditions, operator training programs
11
Developing a Process Map
• Break down all phases of the test/test system into
steps, so that weak points can be identified.
• Each step can be analyzed to find potential failure
modes that could present significant risk to
patients.
• Process can then be further analyzed to see if
controls can be put into place to avoid the failures.
12
Process Map
13
Developing a Process Map (cont’d)
• Compile this information.
• A process map or a fishbone diagram is one example.
1
Samples
Sample Integrity
- Lipemia
- Hemolysis
- Interfering subtances
- Clotting
- Incorrect tube
2
Operator
Atmospheric Environment
- Dust
- Temperature
- Humidity
Operator Capacity
- Training
- Competency
Sample Presentation
- Bubbles
- Inadequate volume
4
Laboratory Environment
Operator staffing
- Short staffing
- Correct staffing
Utility Environment
- Electrical
- Water quality
- Pressure
Identify Potential Hazards
Incorrect Test Result
Reagent Degradation
- Shipping
- Storage
- Used past expiration
- Preparation
Quality Control Material Degradation
- Shipping
- Storage
- Used past expiration
- Preparation
3
Reagents
Calibrator Degradation
- Shipping
- Storage
- Used past expiration
- Preparation
Instrument Failure
- Software failure
- Optics drift
- Electronic instability
Inadequate Instrument Maintenance
- Dirty optics
- Contamination
- Scratches
5
Measuring
System
14
The Risk Assessment
• Once the process map is created, examine
it for places where errors could occur.
• Five major areas:
–
–
–
–
–
Samples
Operator
Reagents
Laboratory environment
Measuring system
15
Key Process Steps
PREPARATION
Pre-analytical
Pre-examination
TESTING
Analytical
Examination
RESULT REPORTING
Post-analytical
Post-examination
Think about what steps can be taken to
reduce errors “unrelated” to the actual testing
of the sample.
16
Perform the Risk Assessment
• Identify the potential failures and their causes.
– Review the process map, fishbone diagram, manufacturer’s
instructions, etc.
• Assess each potential failure.
• Where a failure could occur, add an element to
the QCP that will reduce the possibility of that
failure, making residual risk acceptable.
– For some types of failures, the manufacturer’s information
may already have a quality check in place.
17
Perform the Risk Assessment
• “Risk Assessment” and “Risk Management” are
formal terms for what you’re already doing.
18
Risk Definition
• Risk – the chance of suffering or encountering
harm or loss (Webster's Dictionary and Thesaurus.
Ashland, OH: Landall, Inc.; 1993).
• Risk, can be estimated through a combination of
the probability of occurrence of harm and the
severity of that harm (ISO/IEC Guide 51).
• Risk, essentially is the potential for an error to
occur that could lead to patient/staff harm.
19
Life is a Continual Risk Assessment
You assess risk
every day,
all the time, usually
without even
thinking about it.
20
In the Laboratory
You do this every day!
21
Assess the Risk of Harm Due to Failures
Defective IVD:
Hazard
Incorrect or
delayed
test result:
Hazard
Incorrect or
delayed
medical
treatment:
Hazardous
Situation
Injury or death:
Harm
22
Perform the Risk Assessment (cont’d)
• Construct a table; see which types of errors
are detected and which ones are not.
– If not detected, it must be included in the QCP.
• For each possible failure, assess the
likelihood of that failure occurring, and the
severity of consequences if it occurs.
– Do this for each identified failure.
– Use all of the information gathered in order to make
these assessments.
23
Risk Assessment Worksheet
24
Frequency (1 – 5) Example
(Sometimes called “Probability”)
Common
Terms
Frequent
Rating
Probable
4
Occasional
3
Remote
2
Improbable
1
5
Example
(ISO 14971)
≥ 1/1,000
< 1/1,000 and
≥1/10,000
< 1/10,000 and
≥1/100,000
< 1/100,000 and
≥1/1,000,000
< 1/1,000,000 and
≥10,000,000
Practical
Example
More than
1x/week
Once every
few months
Once a year
Once every
few years
Unlikely to
ever happen
The laboratory must decide – and it may be different for different tests!
25
Severity (1 – 5) Example
Common
Terms
Catastrophic
Critical
Rating
Serious
3
Minor
2
Negligible
1
5
4
Possible Description
(ISO 14971)
Results in patient death
Results in permanent injury of lifethreatening injury
Results in injury or impairment
requiring professional medical
intervention
Results in temporary injury or
impairment not requiring professional
medical intervention
Inconvenience or temporary discomfort
26
Detectability (1 – 5)
What is the likelihood that the control process
detects or prevents the failure?
Common
Terms
Low
Rating
Example
5
4
Control is ineffective
Control less likely to detect the
failure
Control may or may not detect the
failure
Control almost always detects the
failure
Control can detect the failure
3
2
High
1
27
Risk Acceptability Matrix
Severity of Harm
Probability of
Harm
Frequent
Negligible
Minor
Serious
Critical
Catastrophic
Unacceptable Unacceptable Unacceptable Unacceptable Unacceptable
Probable
Acceptable
Occasional
Acceptable
Acceptable
Remote
Acceptable
Acceptable
Acceptable
Improbable
Acceptable
Acceptable
Acceptable
Unacceptable Unacceptable Unacceptable Unacceptable
Unacceptable Unacceptable Unacceptable
Unacceptable Unacceptable
Acceptable
Unacceptable
28
Criticality
• Multiply Frequency x Severity x Detectability
Example: Probable (4) x Catastrophic (5) x
High likelihood to detect failure (1) = 20
Criticality
Result
Low
<10
Mid
10 – 20
High
>20
Higher criticality numbers must have quality control actions in place.
29
Assemble the Quality Control Plan
• Use the information gathered earlier to assess all
of the identified risks and their control measures.
• Construct the QCP.
• Include each of the identified QCP actions in the
QCP.
30
Monitor QCP for Effectiveness
• Verify that the QCP that is put in place actually
works
• Continue to monitor errors and control failures.
• If an error occurs:
– Take the appropriate corrective action.
– Investigate the cause of the error.
– Once the cause is understood, evaluate whether any
changes need to be made in the QCP.
31
Monitor QCP for Effectiveness (cont’d)
• Review any complaints that the laboratory
receives from health care providers.
– These complaints may include pointing out another
source of QC “failure” that must be addressed.
• For patient safety, the QCP should be reviewed
and monitored on an ongoing basis to ensure that
the QCP is optimal.
32
Summary
• A QCP is necessary for result quality, and each
QCP is unique.
• A QCP is the industry standard. It depends upon
the extent to which the device’s features achieve
their intended purpose in union with the
laboratory’s expectation for ensuring quality
results.
• Once implemented, the QCP is monitored for
effectiveness and modified as needed to maintain
risk at a clinically acceptable level.
33
EP23 Companion Products
Plus – More fully worked examples coming soon
www.clsi.org
34