GUIDANCE FOR
ROOT-CAUSE ANALYSIS
OF NON-SATISFACTORY
EXTERNAL QUALITY
ASSESSMENT RESULTS
1st Edition
2017
B-PTS Programme
European Directorate
for the Quality
of Medicines
& HealthCare
Direction européenne
de la qualité
du médicament
& soins de santé
This guide has been elaborated by
the Council of Europe Blood
Proficiency Testing Scheme (B-PTS)
Advisory Group.
For more information, please visit
https://go.edqm.eu/BPTS
All rights conferred by virtue of the
International Copyright Convention
are specifically reserved to the Council
of Europe and any reproduction
or translation requires the written
consent of the Publisher.
Director of the Publication: Dr S. Keitel
Page layout and cover: EDQM
Photo: © syaheir – Fotolia.com
Illustration: © kilroy79 – Fotolia.com
European Directorate for the Quality
of Medicines & HealthCare (EDQM)
Council of Europe
7, allée Kastner
CS 30026
F-67081 STRASBOURG
FRANCE
Internet: www.edqm.eu
© Council of Europe, 2017
2
CONTENTS
Introduction4
1. To start with: Terms and Definitions
2. Managing non-satisfactory PTS results
7
10
3. Root-cause analysis of non-satisfactory PTS results 13
4. Concluding remarks
20
References21
Acronyms22
Authors23
3
INTRODUCTION
Participation in External Quality Assessment (EQA) programmes such as Proficiency Testing Scheme (PTS) studies,
has been recognised as a mandatory
component of a Quality Management
System (QMS) as laid down in the
European Union (EU) Commission
Directive 2005/62/EC[1] implementing
Directive 2002/98/EC[2] of the European
Parliament and of the Council as regards
Community standards and specifications
relating to a quality system for blood establishments, and the Good Practice Guidelines (the GPG)[3], and as prescribed in
the Council of Europe (CoE) Guide to the
preparation, use and quality assurance of
blood components[4].
European legislation and technical instruments governing
the participation in proficiency testing schemes:
- Directive 2005/62/EC
- Good Practice Guidelines
- Guide to the preparation, use and quality assurance of
blood components
4
As laid down in the International Organization for Standardization (ISO)/International Electrotechnical Commission
(IEC) 17043:2010 standard on Conformity Assessment - General Requirements for
Proficiency Testing[5], proficiency testing
is a method for the measurement of the
performance of laboratories, based on
inter-laboratory comparisons. Participation in PTS provides laboratories with an
objective means of assessing and demonstrating the reliability of the data they are
producing. Thus for PTS studies the participants are asked to use their routine
methods, rather than being prescribed
the use of a common method, as is done
in certain collaborative studies. Proficiency testing covers the overall performance of a laboratory. This includes the
whole process from receipt and storage of
the samples, to the experimental work in
the laboratory, the interpretation and the
transcription of the data and conclusions
onto the reporting sheet. Failure at any
stage of this process affects the proficiency
of the laboratory.
DID YOU KNOW?
Proficiency Testing
covers the whole testing
process –this includes
the pre-analytical, the
analytical and post-analytical phases.
5
As part of a QMS, any unsatisfactory
result in a PTS study should be treated as
a non-conformity (NC) and, thus must
be carefully investigated for causative
factors and followed by the implementation of corrective and preventive actions
to prevent reoccurrence[1][3][4].
DID YOU KNOW?
The Good Practice Guidelines (the GPG) has become a
mandatory standard to be implemented in Blood Establishments of the European Economic Area (EEA) Member States.
Element 9.4.6 of the GPG states that ‘an appropriate level of
root-cause analysis work should be applied during the investigation of deviations. In cases where the true root cause(s) cannot
be determined, consideration should be given to identifying the
most likely root cause(s) and to addressing those.’
Non-Conformity at a
glance
Non-conformity is a
wide-ranging concept/
broad term embracing
a number of concepts:
non-conformance,
non-compliance, deviations, complaints and recall.
The term non-conformity
is here understood in its
broad sense.
Although the present document provides information on the
general management of a NC, it primarily aims at guiding laboratories of blood establishments on root-cause analysis of an
unsatisfactory result and/or performance that may occur in a
PTS study.
This guidance document has been prepared by the Council of
Europe Blood Proficiency Testing Scheme Advisory Group
(B-PTS AG) and EDQM responsible officer.
6
1. To start with: Terms and Definitions
For the purposes of this document, the
definitions given in ISO 9000[6], ISO
17043[5] Standards, the Guide to the preparation, use and quality assurance of blood
components[4] (hereinafter referred to as
the Blood Guide) and the EU blood legislation are used.
Blood establishment
• Any structure or body that is responsible for
any aspect of the collection and testing of
human blood or blood components, whatever
their intended purpose, and their processing,
storage and distribution if intended for transfusion. This does not include hospital blood
banks [Blood Guide, 19th Edition] [Directive
2002/98/EC];
Proficiency testing
• Evaluation of participant performance
against pre-established criteria by means
of inter-laboratory comparisons [ISO
17043:2010];
Quality
• Degree to which a set of inherent characteristics of a product, system or process fulfil
requirements [ISO 9000:2015];
• Totality of characteristics of an entity that
bear on its ability to satisfy stated and
implied needs. Consistent and reliable
7
performance of services or products in
conformity with specified standards [Blood
Guide, 19th Edition];
Quality management (QM)
• Management (coordinated activities to direct
and control an organisation) with regard to
quality [ISO 9000:2015];
• The coordinated activities to direct and
control an organisation with regard to quality
at all levels within the blood establishment
[Blood Guide, 19th Edition];
Quality system (QS)
• The organisational structure, responsibilities, procedures, processes, and resources for
implementing quality management [Blood
Guide, 19th Edition];
Quality Management System (QMS)
• Part of a management system (set of interrelated or interacting elements of an organisation to establish policies and objectives and
processes to achieve those objectives) with
regard to quality [ISO 9000:2015];
Non-conformity
• Non-fulfillment of a requirement [ISO
9000:2015];
Note: This term may also be commonly referred to as nonconformance, non-compliance, deviations, complaints
and recall in various regulatory documents and guidance.
8
Correction
• Action to eliminate a detected non-conformity [ISO 9000:2015];
Corrective action
• Action to eliminate the cause of a non-conformity [ISO 9000:2015];
Preventive action
• Action taken to eliminate the cause of a
potential non-conformity or other potential
undesirable situation [ISO 9000:2005].
9
2. Managing non-satisfactory PTS results
Before an investigation and root-cause
analysis is initiated on a non-satisfactory
PTS result and/or performance, laboratories of blood establishments must ensure
that an adequate procedure on the management of non-conformities (NCs) has
been elaborated and implemented. The
procedure must be made available to all
personnel. The flowchart below provides
guidance on how non-satisfactory PTS
results and/or performance are typically
managed.
This flowchart has been elaborated
drawing on the work done within the
EDQM Blood Quality Management
Programme.
More on non-conformity management:
Read section 9. Non-conformance and recall of the Good
Practice Guidelines
10
Figure 1 - Flowchart management of non-satisfactory PTS result(s)
PTS non-satisfactory
results/performance
identified
Immediate correction(s)
Perform immediate risk assessment
RCA STEP
Document NC & report it
to Quality Manager (QM)
& personnel concerned
No
3.1
Has the NC been correctly
documented?
Yes
NC report reviewed &
approved by QM
Investigate & document
the investigation
No
3.2
Has the
investigation been correctly
documented?
Yes
Investigation reviewed and
approved
To be documented
11
3.3
ROOT CAUSE IDENTIFICATION
3.3
ROOT CAUSE IDENTIFICATION
Root cause(s) identified &
documented
Corrective Action Preventive Action (CAPA) plan
Approval by QM
Implementation
Effectiveness Monitoring
Have the action been
effective?
Yes
Close NC
12
No
3.4
Dissemination of CAPA
plan to personnel involved
including due dates
3. Root-cause analysis of non-satisfactory
PTS results
Root-Cause Analysis (RCA) is a systematic approach used for problem solving
aiming at identifying the root causes of a
NC.
The following provides guidance on how
to perform root-cause analysis of non-satisfactory PTS results.
RCA is a process of four (4) major steps
involving the following:
• Documentation of the non-conformant PTS result (3.1),
• Investigation (3.2),
• Root Cause Identification/Analysis
(3.3) and,
• Follow-up (3.4)
These phases are reflected in the previous
flowchart.
3.1. Description and immediate data collection
Description and proper documentation
of a non-satisfactory result is a key step
before initiating investigation and performing root-cause analysis.
When identified, the non-compliant PTS
result should immediately be properly and
13
factually described. Its description should
comprise at minimum:
• The name/code of the sample involved;
• The non-conformant measurand (e.g.
HBsAg);
• The name, version, batch number
of the assay (s) used; name of the
equipment used;
• The name of the computerised
system(s) used if applicable;
• The name of the operator and/or any
laboratory staff member involved in
the testing process;
• The date and time of the testing;
This data should be carefully documented
and recorded in a NC registry.
3.2. Investigation
In a second step, with a view to facilitating
root-cause identification, and wherever
needed a deeper investigation should be
performed.
This might include for example:
• The review of all kind of documentation such as:
■■ prints/logs/records of the assays/instruments used;
■■ prints/log/records of the results
obtained on the same run and/or day;
■■ prints/log/records of the results
obtained with the same batch of
reagent, on the same instruments;
14
■■ prints/log/records of results of the
internal and external quality control
obtained on the same run/day;
■■ work plans;
■■ computer logs/records.
• Preliminary verification such as:
■■ verification of data transmission or
transcription;
■■ verification of the quality controls;
■■ verification of the calibration and
maintenance of the instrument;
■■ verification of the qualification of the
instrument and validation of the assay
and reagents;
■■ verification of the alarms of the instrument;
■■ verification of the expiry date and
integrity of the reagents;
■■ verification of the storage conditions of
samples and reagents.
Additional evidence may also be collected.
3.3. Investigation of causal factors/root-cause identification
Following data collection, investigation
and preliminary verifications, root-cause
identification may be initiated.
Proper investigation of causative factors
should whenever possible be done in a
multidisciplinary team and should also
involve the quality manager, or his/her
substitute.
15
The root-cause identification may be
performed using various thinking
methods. Examples of thinking methods
are provided in the box below.
WHICH OF THE FOLLOWING
THINKING METHODS DO YOU KNOW?
•
•
•
•
•
Interviews;
Brainstorming;
Mind-mapping;
5Ws (Who, What, Where, When, Why);
5Ms/7Ms (Method, Machine, Material, HuMan
Power, Measurement/Medium; + Management,
Financial Means)
• Cause/effect diagram or causal factor charting;
• Flowcharting;
• Checklist.
The choice of the thinking method usually
depends on the type and complexity of
NC and its associated risks. The following
table provides a non-exhaustive list of
possible causative factors for a non-conformant B-PTS result.
16
Process
Possible Causative factors/Root cause
Analytical Phase
- Malfunction of the instrument (sampler
not aligned/obstruction/clotting);
- Incorrect incubation time;
- Invalid result due to presence of inhibitors;
- Cross-contamination;
- Carry-over.
Testing
Pre-analytical Phase
- Non appropriate management/handling
of the samples during the pre-analytical
phase:
• Storage conditions not respected;
• Thawing procedure not respected;
• Sample mixing not correctly done;
• Sample incorrectly transferred into
a new tube;
• Contamination during handling.
- Mistake during labelling of secondary
tubes;
- Samples not adequately placed in the
run/plate; inversion of samples on the
plate.
Post-analytical Phase
- Algorithm incorrectly applied;
- Failure in transmission of the results;
- Misreading of the results;
- Mistake or inversion of results during
transcription;
- Misinterpretation of raw data;
- Incorrect unitage.
17
Management of material &
equipment
- Material/reagents expired;
- Material/reagents released despite non-conformities (e.g. defect);
- Storage conditions of material/reagents not respected;
- Instrument/equipment not (properly) validated qualified/calibrated/
maintained;
- Assay not validated to the degree needed;
- Computerised system not validated;
- Computerised system not adequate for PTS samples /for manual entry.
- Assay failure (please see next box)
Human
resources
- Staff not correctly applying the testing procedure and any other
procedures that may affect the testing of the samples (e.g. Hygiene
and safety procedure);
- Staff not appropriately trained.
Procurement
supplier
selection
Possible Causative factors/Root cause
- Assay does not meet user requirement specifications (URS) (e.g. sensitivity);
- Acceptable limits for quality controls.
Procedure
Process
- Procedure on teh management of PTS samples absent or not adequate;
- Procedure not up-to-date according to manufacturer’s instructions;
- Manufacturer’s instructions not respected.
18
DID YOU KNOW?
Testing assays and
reagents used in blood
establishments are
considered as Medical
Devices (MD) in the
European Union. They
must be CE marked.
➢ ASSAY FAILURE
Whenever there is a good reason to believe
that a non-satisfactory result may be
related to a failure of the assay the blood
establishment should take the necessary
measures to inform the manufacturer of
the assay and the National Competent
Authority. The national procedure on
medical device vigilance system is to be
followed.
3.4. Follow-up
Following the identification of the rootcause(s), appropriate corrective actions
and preventive actions (CAPA) are to be
taken. CAPA should treat the cause and
prevent the NC reoccurring.
The effectiveness of the CAPA plan is to be
monitored.
Should the same, non-conformant PTS
result be recurrent, it is recommended to
perform a retrospective evaluation and
repeat the root-cause analysis.
19
4. Concluding remarks
DOCUMENTATION
All of the above-described steps of the
management of a non-conformant PTS
result are to be documented and reviewed
by the NC owner and Quality Manager or
his/her substitute.
PTS SAMPLES RETESTING
Should the retesting of the PTS samples
be required as part of the investigation
or root-cause identification, following
completion of the PTS study and upon
request, the EDQM will make available
additional samples to laboratories that are
not satisfied with their results and want to
monitor the effectiveness of their CAPA.
Requests should be sent to EDQM_B_PTS@edqm.eu.
20
REFERENCES
[1]
uropean Union. Commission Directive 2005/62/EC of
E
30 September 2005 implementing Directive 2002/98/EC
of the European Parliament and of the Council as regards
Community standards and specifications relating to a
quality system for blood establishments. 2005.
[2]
uropean Union. Directive 2002/98/EC of the European
E
Parliament and of the Council of 27 January 2003 setting
standards of quality and safety for the collection, testing,
processing, storage and distribution of human blood and
blood components and amending Directive 2001/83/E.
2003.
[3]
uropean Directorate for Quality of Medicines and
E
HealthCare (EDQM)/Council of Europe. Good Practice
Guidelines for blood establishments and hospital blood
banks, 2017.
[4]
uropean Directorate for Quality of Medicines and
E
HealthCare (EDQM)/Council of Europe. Guide to the
preparation, use and quality assurance of blood components, 19th ed. 2017.
[5]
I nternational Organization for Standardization. ISO/IEC
17043:2010 – Conformity assessment – General requirements for proficiency testing, 2010.
[6]
International Organization for Standardization. ISO
9000: 2015 - Quality management systems -- Fundamentals and vocabulary, 2015.
21
ACRONYMS
CAPACorrective Action Preventive
Action
CoE
Council of Europe
EU
European Union
EQAExternal Quality Assessment
GPGGood Practice Guidelines
IECInternational Electrotechnical
Commission (IEC)
ISOInternational Organization for
Standardisation
NC
Non-Conformity
PTSProficiency Testing Scheme
QMQuality Management
QMSQuality Management System
RCARoot Cause Analysis
22
AUTHORS
Marie-Laure HecquetEuropean Directorate for
the Quality of Medicines
and Healthcare, France
Simonetta Pupella
National Blood Center, Italy
Danièle Sondag-Thull
Belgium Red Cross, Belgium
Maja TomicicNational Institute of Transfusion Medicine, Croatia
Michael Chudy
Paul Ehrlich Institute, Germany
Giulio PisaniIstituto Superiore di Sanità,
Italy
23
The Council of Europe is the continent’s leading human rights organisation. It
comprises 47 member states, 28 of which are members of the European Union.
The European Directorate for the Quality of Medicines & HealthCare (EDQM) is a
directorate of the Council of Europe. Its ­mission is to contribute to the basic human
right of access to good quality medicines and healthcare and to promote and protect
public health.
ENG
www.edqm.eu
B-PTS Programme
European Directorate
for the Quality
of Medicines
& HealthCare
Direction européenne
de la qualité
du médicament
& soins de santé