CLSI EP23™—Laboratory
Quality Control Based on Risk
Management
James H. Nichols, PhD, DABCC, FACB
Chairholder
EP23 Document Development Committee
Professor of Clinical Pathology, Microbiology and Immunology
Vanderbilt University School of Medicine
Medical Director, Clinical Chemistry
Nashville, Tennessee, USA
Objectives
• Review key aspects of risk management.
• Describe the various types of control
processes.
• Identify CLSI document EP23 as a resource for
developing a laboratory quality control (QC)
plan based on risk management.
• Use CLSI document EP23 to develop a quality
control plan (QCP) based on risk management
for a simple, moderate complexity device.
2
Risk
Would you walk underneath this piano?
3
Risk Management
• Risk management is not a new concept;
laboratories:
– Evaluate the performance of new devices.
– Troubleshoot instrument problems.
– Respond to physician complaints.
– Estimate harm to a patient from incorrect results.
– Take actions to prevent errors.
• Risk management is a formal term for what
clinical laboratories are already doing every
day.
4
Risk Management Definition
• Systematic application of management
policies, procedures, and practices to the tasks
of analyzing, evaluating, controlling, and
monitoring risk (ISO 14971)
5
Risk Definition
• Risk – the chance of suffering or encountering
harm or loss (Webster's Dictionary and Thesaurus.
Ashland, OH: Landall, Inc.; 1993).
• Risk can be estimated through a combination
of the probability of occurrence of harm and
the severity of that harm (ISO/IEC Guide 51).
• Risk, essentially, is the potential for an error to
occur that could lead to patient/staff harm.
6
What could go wrong?
7
Sources of Laboratory Error
•
•
•
•
Environmental:
– Temperature
– Humidity
– Light intensity
– Altitude
Operator:
– Improper specimen preparation, handling
– Incorrect test interpretation
– Failure to follow test system instructions
Specimen:
– Bubbles
– Clots
– Incorrect tube additive
Analysis:
– Calibration factor incorrect
– Mechanical failure
8
Managing Risk With
a Quality Control Process
9
Quality Control
• Advantages
– QC monitors the end product (result) of the entire test system.
– QC has target values: if assay recovers the target, then everything is
assumed stable (ie, instrument, reagent, operator, sample).
• Disadvantages
– When a problem is detected, one must go back and reanalyze patients
since the last “good” QC.
– If results are released, then results may need to be corrected.
• Need to get to fully automated analyzers that eliminate errors up
front
– Until that time, need a robust QC plan (QCP)
10
Types of Quality Control
• “On-Board” or Analyzer QC – built-in device
controls or system checks
• Internal QC – laboratory-analyzed surrogate
sample controls
• External QC – blind proficiency survey
• Other types of QC – control processes either
engineered by a manufacturer or enacted by a
laboratory to ensure result reliability
11
12
Quality Control Limitations
• No single QC procedure can cover all devices, because the
devices may differ.
• QC practices developed over the years have provided
laboratories with some degree of assurance that results are
valid.
• Newer devices have built-in electronic controls, and “onboard” chemical and biological controls.
• QC information from the manufacturer increases the user’s
understanding of device’s overall quality assurance
requirements.
ISO. Clinical laboratory medicine – In vitro diagnostic medical devices –
Validation of user quality control procedures by the manufacturer. ISO
15198. Geneva, Switzerland: International Organization for
Standardization; 2004.
13
LaboratoryManufacturer Partnership
• Developing a quality plan surrounding a laboratory device
requires a partnership between the manufacturer and the
laboratory.
• Some sources of error may be detected automatically by the
device and prevented, while others may require the laboratory
to take action, such as analyzing surrogate sample QC on
receipt of new lots of reagents.
• Clear communication of potential sources of error and
delineation of laboratory and manufacturer roles for how to
detect and prevent those risks is necessary.
14
Quality Control
15
CLSI Document EP23
• Laboratory Quality Control Based on Risk
Management; Approved Guideline (EP23-A™)
• James H. Nichols, PhD, DABCC, FACB, Chairholder of
the document development committee
• EP23 describes good laboratory practice for developing
a QCP based on the manufacturer’s risk mitigation
information, applicable regulatory and accreditation
requirements, and the individual health care and
laboratory setting.
16
The Scenario
• CLSI document EP23 provides guidance on
developing an appropriate QCP that will:
– Save time and money.
– Use electronic and/or integrated QC features.
– Use other sources of QC information.
– Conform to one’s laboratory and clinical use of the
test.
17
Developing a QCP
18
The Quality Control Toolbox
• Every QC tool has its strengths and weaknesses
(there is no perfect QC tool).
• Implement a combination of tools in order to
properly control a test.
• EP23 explains the strengths and weaknesses of
the different QC processes.
19
Examples of
Quality Control Tools
•
•
•
•
•
•
•
•
•
•
•
Intralaboratory QC
Interlaboratory QC
Integrated (built-in) QC
Measuring system function checks
Electronic system checks
Calibration checks
Repeat testing of patient samples
Monitoring aggregated patient results
Implausible values
Delta checks
Correlation of multiple analytes in same sample
20
What Could Go Wrong?
21
Gather Information
for the Risk Assessment
• Gather information from several sources:
– Regulatory and accreditation requirements
• Clinical Laboratory Improvement Amendments Test/test
system information
• User’s manual, reagent package insert, literature
– Health care and testing site settings
• Temperature conditions, operator training programs
– Medical requirements for the test results
• Allowable performance specifications via physicians
22
Developing a Process Map
• Break down all phases of the test or test
system into steps, so that weak points can be
identified.
• Each step can be analyzed to find potential
failure modes that could present significant
risk to patients.
• Process can then be further analyzed to see if
controls can be put into place to avoid the
failures.
23
Process Map
Process Map
High-Level
Measurement Process
3
Startup/maintenance/
calibration
Measuring system
Examination ready
Reagents/calibrators/
parts procurement and
storage
Start
2
Operator training and
compentency
4
Laboratory
environment
1
Samples received at
measuring system
Sample acceptability
is evaluated
Samples are loaded and
tested (retested)
5
Measuring system
error message or
malfunction?
Yes
Yes
No
Troubleshooting
performed and
corrective action taken
No
Results are
evaluated
Performance
verified?
Results are reported
No
Yes
Repeat
examination
required?
Samples are
unloaded
and stored
End
24
Key Process Steps
• View the preexamination (preanalytical),
examination (analytical), and
postexamination (postanalytical) areas of the
laboratory.
• Think about what steps can be taken to
reduce potential errors “unrelated” to the
actual testing of the sample.
25
Where Is the
Risk in the Process?
What could possibly
go wrong?
26
Identify the Risks
1
Samples
Sample Integrity
- Lipemia
- Hemolysis
- Interfering subtances
- Clotting
- Incorrect tube
2
Operator
Atmospheric Environment
- Dust
- Temperature
- Humidity
Operator Capacity
- Training
- Competency
Sample Presentation
- Bubbles
- Inadequate volume
4
Laboratory Environment
Operator staffing
- Short staffing
- Correct staffing
Utility Environment
- Electrical
- Water quality
- Pressure
Identify Potential Hazards
Incorrect Test Result
Reagent Degradation
- Shipping
- Storage
- Used past expiration
- Preparation
Quality Control Material Degradation
- Shipping
- Storage
- Used past expiration
- Preparation
3
Reagents
Calibrator Degradation
- Shipping
- Storage
- Used past expiration
- Preparation
Instrument Failure
- Software failure
- Optics drift
- Electronic instability
Inadequate Instrument Maintenance
- Dirty optics
- Contamination
- Scratches
5
Measuring
System
27
Hazard or Risk Identification
• Some areas to consider for weaknesses in the
process:
– Testing personnel training
and competency
– Reagent/calibrator/parts
procurement and storage
– Patient sample
acceptability
– System startup
– System calibration
– Loading and testing of
patient samples
– Proper device function
– Test result review
28
Risk Assessment
29
Perform the
Risk Assessment
• Identify the potential failures and their causes.
– Review the process map, fishbone diagram,
manufacturer’s instructions, etc.
• Assess each potential failure.
• Where a failure could occur, add an element to
the QCP that will reduce the possibility of that
failure, making residual risk acceptable.
– For some types of failures, the manufacturer’s
information may already have a quality check in place.
30
Perform the
Risk Assessment
(cont’d)
• Construct a table; see which types of errors are
detected and which ones are not.
– If not detected, it must be included in the QCP.
• For each possible failure, assess the possibility
of that failure.
– Do this for each identified failure.
– Use all of the information gathered in order to make
these assessments.
31
Assemble the
Quality Control Plan
• Use the information gathered earlier to
assess all of the identified risks and their
control measures.
• Construct the QCP.
• Include each of the identified QCP actions in
the QCP.
32
What could go wrong?
33
Monitor Quality Control
Plan for Effectiveness
• Verify that the QCP that is put in place actually
works.
• Continue to monitor errors and control failures.
• If an error occurs:
– Take the appropriate corrective action.
– Investigate the cause of the error.
– Once the cause is understood, evaluate whether any
changes need to be made in the QCP.
34
Monitor Quality Control
Plan for Effectiveness (cont’d)
• Review any complaints that the laboratory
receives from health care providers.
– These complaints may include pointing out
another source of QC “failure” that must be
addressed.
• For patient safety, the QCP should be
reviewed and monitored on an ongoing basis
to ensure that the QCP is optimal.
35
Don’t Be Discouraged—Risk
Management Is Documenting Much of
What We Already Do!
36
Polling Question #1
What is a QCP?
1) The frequency of liquid controls for a test
2) A form that defines required specifications of
materials from suppliers
3) A document that describes the practices,
processes, and sequences of specified activities
to control the quality of a particular measuring
system
4) None of the above
37
The Scenario
• A laboratory director wants to develop a QCP
– Incorporates the right QC processes for the specific
test
– Uses adequate QC to control for their potential
error sources
– Follows manufacturer’s instructions
• A unit-use blood gas analyzer will be used as
the example.
38
Developing a
Quality Control Plan
39
Polling Question #2
Raise your hand if you are responsible for
blood gas or electrolyte testing in your
facility, or if you are a manufacturer or
distributor of blood gas/electrolyte test
systems.
40
Gather the Information
41
Blood Gas and Electrolytes
• Generic unit-use blood gas/lytes
analyzer in a same-day surgical
center
• Low volume: 0 - 5 tests/day
• Need for daily liquid QC uses two
kits ($10 each) and adds to
turnaround time (TAT).
• Adoption of nontraditional QC
through EP23 would improve
cost, test, and labor efficiency.
42
Blood Gas and Electrolytes
• Portable clinical analyzer
for in vitro quantification
of various analytes in
whole blood
• Analyzers and cartridges
should be used by
healthcare professionals
trained to use the system
according to the facility’s
policies and procedures.
43
Blood Gas and Electrolytes
• System consists of:
– Portable clinical analyzer
– Test cartridges sealed in
foil pouch for protection
during storage
– Quality assurance
materials
• Control solutions
• Calibration verification set
Device
Device
Device
Device
– Data Management System
• Server class computer
• Data management software
• Wireless connectivity and
LIS/HIS interfaces
44
Blood Gas and Electrolytes
• Unit use cartridge contains all components to
perform the testing
– Calibrating solution
– Reagents
– Sample handling system
– Sensors
• Analyzer automatically controls all steps of the
testing process:
– Fluid movement
– Calibration
– Reagent mixing
– Thermal control
45
Cartridge Operations
• Cartridges are standardized to plasma core lab
methods using multi-point calibration curves
stored in the device memory that are stable over
many lots
• Upon insertion, a calibrant solution in the
cartridge is passed across the sensors.
• Signals produced by the sensor’s responses to
calibrant are measured – A one-point calibration
adjusts the offset of stored multi-point
calibration curve.
• Analyzer then moves sample over sensors, and
the signal of the sensor responses to the sample
are measured off the adjusted calibration curve
46
Polling Question #3
What types of quality control processes can help
laboratories manage their risk of errors for a blood
gas/electrolyte test system?
1) Liquid quality control samples
2) Manufacturer checks and simulated internal
electronic controls
3) Staff training and competency
4) All of the above
47
Internal Control Processes
• Simulated internal QC - diagnostic check of the edge
connector, internal electronics, and analyte circuitry.
• Internal QC simulates electronic signals produced by
the sensors during a cartridge test.
• An isolated region of the internal circuit board sends a
range of simulated sensor signals through the
cartridge measurement channels
• Range of signals encompasses entire linear range
expected from blood analytes
• Next, conductivity out of the connector pins is
measured, insuring no contamination is present in the
edge connector which would interfere with the test.
• Signal measurements must fall within strict
predetermined thresholds to pass.
48
Quality Control Recommendations
• Internal Simulated QC:
– Automatically performed by device every 8 hrs
– If significant change in analyzer temp (cold to hot)
– Whenever performance of device in question
• Liquid QC:
–
–
–
–
Each shipment of cartridges
New lots of cartridges
If cartridges experience temperature shift >8°C (15°F)
Periodically as required by facility policies
• Temperature Verification
– Monitored continuously during each patient test, but
verification cartridge available and recommended
annually, or as required by facility policy
49
Create a Process Map
Identify Weak Steps for
Hazards or Risk of Error
50
Polling Question #4
What are some common sources of
error for a blood gas/electrolyte test
system?
1) Shipping and storage temperature of
cartridges
2) Operator technique
3) Device failure
4) All of the above
51
Finding the Failure Points
Hazard Identification
1
Samples
Sample Integrity
- Lipemia
- Hemolysis
- Interfering subtances
- Clotting
- Incorrect tube
2
Operator
Atmospheric Environment
- Dust
- Temperature
- Humidity
Operator Capacity
- Training
- Competency
Sample Presentation
- Bubbles
- Inadequate volume
4
Laboratory Environment
Operator staffing
- Short staffing
- Correct staffing
Utility Environment
- Electrical
- Water quality
- Pressure
Identify Potential Hazards
Incorrect Test Result
Reagent Degradation
- Shipping
- Storage
- Used past expiration
- Preparation
Quality Control Material Degradation
- Shipping
- Storage
- Used past expiration
- Preparation
3
Reagents
Calibrator Degradation
- Shipping
- Storage
- Used past expiration
- Preparation
Instrument Failure
- Software failure
- Optics drift
- Electronic instability
Inadequate Instrument Maintenance
- Dirty optics
- Contamination
- Scratches
5
Measuring
System
52
Process Map:
Finding the Failure Points
• Work from the current package insert
• Test order – electronic or hardcopy
• Test collection
–
–
–
–
–
Incorrect collection – bubbles, sample exposure to air
Wrong tube type – calcium titrated, heparinized BG tubes
Indirect phlebotomy – line draw contamination
Undermixing/overmixing – sample clots, hemolysis
Analytic delay – glucose, BG, pH, iCa, etc.
–
–
–
–
–
Wrong sample volume loaded onto cartridge
Incorrect procedure, timing, result interpretation
Expired reagent
Reagent exposure during shipment
Degradation during storage
• Analysis
• Infection Control
• Result reporting errors
53
Conduct a Risk Assessment
Identify Control Processes for
Each Hazard That Maintain
Risk at a Clinically Acceptable
Level
54
Blood Gas and Electrolytes
Risk Assessment
• Refer to Appendix A, CLSI document EP18 for
a more comprehensive list of error sources.
• Work from the manufacturer’s current
package insert.
• Samples
– Physician order – POCT possible w/o order, need written
or electronic physician order before commencing test.
– Wrong tube type – train to use BG syringes/tubes
– Line contamination – train on preferred collection and
techniques if catheter collection is only option
– Sample mixing – analyzer has clot detection, but will not
detect hemolysis - train on proper mixing technique
– Analytic delay – YES – train to analyze immediately, no
longer than 15-30 mins of collection
55
Blood Gas and Electrolytes
Risk Assessment (cont’d)
• Operator
– Operator lock-out – prevents use of analyzer by
untrained operators
– Sample volume detection – Analyzer detects inadequate
sample volume and prevents overloading.
– Incorrect procedure, timing, result interpretation –
analysis and result interpretation fully automated,
clotted sample or bubbles will be detected by analyzer.
– Expired reagents – cartridges are bar-coded with lot
number and expiration date, analyzer prevents use past
expiration
– Wireless Connectivity – data management automates
reporting of result provided patient properly identified,
train on proper patient ID, use barcoded wristbands
– Infection control – train to clean and disinfect after each
use
56
Blood Gas and Electrolytes
Risk Assessment
• Reagents
– Test exposure outside specifications (eg, temperature,
humidity) during shipment – analyze liquid QC with each
shipment
– Lot-to-lot variability – analyze liquid QC with each lot
– Liquid QC degradation – monitor refrigerator (2 to 8°C),
bring to room temperature at least 30 minutes before use,
discard within 30 days of opening bottle
– Degradation during storage – monitor storage conditions,
(stable @ room temp for 2 years). If refrigerated, bring to
room temperature at least 30 minutes before use, (analyze
liquid QC due to temp change). What about other QC
frequency?
57
Polling Question #5
How should laboratories determine the
optimal frequency of liquid quality controls?
1) Refer to the manufacturer’s package insert
recommendations
2) Identify local and regional regulatory
requirements
3) Conduct a risk assessment
4) All of the above
58
Liquid Quality Control Frequency
• Minimum – follow manufacturer recommendations and
regulatory requirements (CLIA for BG analysis – one QC
sample q 8 hr, two levels q 24 hrs, one QC w/ each pt
sample unless calibration every 30 mins)
• Manufacturer recommends liquid QC with each
– Shipment
– New lot
– Significant change in cartridge temperature (>8°C)
– Whenever question of test system performance
• Options for determining liquid QC frequency
– Peer publications – verify what others are already doing
– Develop QC rules based on six-sigma of test system
– Verify in your facility:
– analyze 2 levels each day for several weeks, then reduce to every few
days, weekly or monthly after more experience with test system
• QC-lockout assists with compliance
59
Blood Gas and Electrolytes
Risk Assessment (cont’d)
• Environment
– Incorrect collection – train staff anaerobic phlebotomy
– Compliance with documentation – risk bases on prior
issues with other testing noted at this location
(refrigerator monitoring, QC documentation, etc.)
• Clinical Application
– Immediate medical decisions – test results used to
manage critical patients, higher risk since only one
chance to get right result!
– Sample not stable – analyze immediately, presents
higher risk since can’t be repeated!
60
Summarize the
Quality Control Plan
61
Blood Gas and Electrolytes
Quality Control Plan
• Analyze liquid QC.
–
–
–
–
–
–
Each new shipment*
Start of a new lot*
After significant change in cartridge temperature (>8°C) *
Whenever uncertainty about analyzer performance*
Monthly (based on facility verification and experience with test)
Note: Simulated multi-level quality control automatic every 8
hours and internal calibration with each test cartridge **
• Use checklist to document training/competency.
–
–
–
–
–
–
–
–
Test only when electronic or written physician order
Proper patient identification
Use BG syringes/tubes for specimen collection
Arterial BG collection (preferred), or line draws as required
Use anaerobic technique
Mix specimens appropriately and analyze immediately
Monitor refrigerator and room temperatures
Clean and disinfect analyzers after each use *
(* Manufacturer recommendations)
(** Mandated by accreditation regulations)
62
Polling Question #6
What should a laboratory do once a QCP is
drafted?
1) Have a party; the laboratory has now proved
the quality of its test.
2) Nothing; once developed, the QCP is the end
of the risk management process.
3) Monitor its QCP for effectiveness, and modify
the plan as needed.
4) None of the above.
63
Implement the
Quality Control Plan
Monitor for Failure/Errors
and Modify Quality Control
Plan as Needed
64
Quality Control Plan
Implementation: BG/Electrolyte Monitors
• Frequency of analyzer error codes
• Liquid QC failure rates
• Frequency of specimen issues – hemolysis,
clots, or other problems
• Number of physician complaints: results
that don’t match clinical situation
• Any other unexpected error
65
Risk Management will help you learn
about your processes and weaknesses
66
Summary
• Risk management is something laboratories are
already doing. EP23 simply formalizes this.
• A QCP is necessary for result quality, and each
QCP is unique.
• A QCP is the industry standard. It depends upon
the extent to which the device’s features achieve
their intended purpose in union with the
laboratory’s expectation for ensuring quality
results.
• Once implemented, the QCP is monitored for
effectiveness and modified as needed to
maintain risk at a clinically acceptable level.
67
Now that you’ve survived this…
68