BELAC 2-106 Rev 1-2011
PROFICIENCY TESTING:
GUIDELINES ON THE LEVEL OF PARTICIPATION AND
EVALUATION OF PERFORMANCES IN PROFICIENCY
TESTING ACTIVITIES IN THE CONTEXT OF
ACCREDITATION ASSESSEMENTS
The only valid versions of the documents of the BELAC management system are those available from the
internet website.
English translation for information only
French and Dutch versions remain the authoritative documents
Applicable from: 01.02.2011
BELAC 2-106 Rev 1 – 2011 - 1/ 11
BELAC 2-106 Rev 1 – 2011 - 2/ 11
DOCUMENT HISTORY
Revised and date of approval
Reason for revision
Impact of revision
0
First document
Not applicable
Replacement of reference to
Guide ISO 43 by the standard
NBN EN ISO/IEC 17043:2010
Full document
Extension of application area to
all types of laboratories and
inspection bodies performing
tests.
Section 1
Revision of guidelines concerning
frequency of participation.
Section 4.3.2
CC 19.12.2006
1
CC written procedure with end
date 17.01.2011
Addition
of
the
concept
« measurement audit » as part of
the accreditation assessment .
Section 4.4.1
BELAC 2-106 Rev 1 – 2011 - 3/ 11
TABLE OF CONTENTS
1.
OBJECTIVES AND REFERENCES TO NORMATIVE DOCUMENTS ............... 5
2.
RECIPIENTS ....................................................................................................... 5
3.
DEFINITIONS...................................................................................................... 6
3.1.
Proficiency Testing ................................................................................................................................. 6
3.2.
Interlaboratory comparisons ................................................................................................................. 6
3.3.
Measurement Audit /Bilateral proficiency testing ............................................................................... 6
4. GENERAL POLICY AND PRACTICAL ASPECTS REGARDING
PARTICIPATION IN PROFICIENCY TESTING. ........................................................ 7
4.1.
General .................................................................................................................................................... 7
4.2.
Terms for participation in proficiency testing ..................................................................................... 7
4.2.1. Proficiency testing selection: general provisions ................................................................................ 7
4.2.2. Proficiency testing selection: additional provisions ............................................................................. 8
4.3.
Responsibilities of the laboratory concerning the participation in proficiency testing. ................... 9
4.3.1. Documentation ..................................................................................................................................... 9
4.3.2. Frequency of participations .................................................................................................................. 9
4.3.3. Statistical evaluation of the results ..................................................................................................... 10
4.3.4. Corrective actions .............................................................................................................................. 10
4.4.
BELAC responsibilities in the evaluation by BELAC of the results of participation in proficiency
testing. .................................................................................................................................................... 10
4.4.1. Evaluation during accreditation-assessments ..................................................................................... 10
4.4.2. Communication of results of proficiency testing to BELAC. ............................................................ 11
4.4.3. Taking into account the proficiency testing results in a certain technical sector ............................... 11
4.5.
Sanctions within the context of result evaluation of proficiency testing .......................................... 11
BELAC 2-106 Rev 1 – 2011 - 4/ 11
PROFICIENCY TESTING:
GUIDELINES ON THE LEVEL OF PARTICIPATION AND
EVALUATION OF PERFORMANCES IN PROFICIENCY
TESTING ACTIVITIES IN THE CONTEXT OF
ACCREDITATION ASSESSMENTS
1.
OBJECTIVES
DOCUMENTS
AND
REFERENCES
TO
NORMATIVE
•
The standards NBN EN ISO/IEC 17025 and NBN EN ISO 15189 require that laboratories
seek confirmation for confidence in their results and organize quality control procedures
for monitoring the validity of performed tests. This monitoring includes the participation
in interlaboratory comparisons or proficiency testing programs. In this document, the term
laboratory covers testing laboratories, calibration laboratories and medical laboratories.
•
The purpose of this document is to identify the BELAC requirements concerning the
participation of laboratories in proficiency testing. The document further determines the
criteria for evaluation of performance in the context of an accreditation process.
•
Principles and requirements described within this document also apply to inspection
bodies performing themselves critical tests and calibrations activities (cfr. BELAC 2-201
clause 1.4).
•
This document refers to and is in compliance with:
- standard NBN EN ISO/IEC 17011;
- standard NBN EN ISO/IEC 17025;
- standard NBN EN ISO/IEC 17020;
- standard NBN EN ISO/IEC 17043;
- relevant EA en ILAC guidelines (cfr www.european-accreditation.org and
www.ilac.org)
2.
RECIPIENTS
With follow up of modifications:
Members of the Accreditation Board
Members of the Coordination Commission
The accreditation secretariat
Assessors / experts from the sector laboratories and inspection
Accredited laboratories and inspection bodies
Without follow up of modifications:
Laboratories and inspection bodies having submitted a request to accreditation
Any external request
BELAC 2-106 Rev 1 – 2011 - 5/ 11
3.
DEFINITIONS
Note: Direct quotations from standards and other normative references are written in italic.
3.1.
Proficiency Testing
Evaluation of participant performance against pre-established criteria by means of
interlaboratory comparisons.
NOTE 1 For the purposes of this International Standard, the term “proficiency testing” is
taken in its widest sense and includes, but is not limited to:
a) quantitative scheme — where the objective is to quantify one or more measurands of the
proficiency test item;
b) qualitative scheme — where the objective is to identify or describe one or more
characteristics of the proficiency test item;
c) sequential scheme — where one or more proficiency test items are distributed sequentially
for testing or measurement and returned to the proficiency testing provider at intervals;
d) simultaneous scheme — where proficiency test items are distributed for concurrent testing
or measurement within a defined time period;
e) single occasion exercise — where proficiency test items are provided on a single occasion;
f) continuous scheme — where proficiency test items are provided at regular intervals;
g) sampling — where samples are taken for subsequent analysis; and
h) data transformation and interpretation — where sets of data or other information are
furnished and the information is processed to provide an interpretation (or other outcome).
NOTE 2 Some providers of proficiency testing in the medical area use the term “External
Quality Assessment (EQA)” for their proficiency testing schemes, or for their broader
programmes, or both.
(NBN EN ISO/IEC 17043 clause 3.7)
3.2.
Interlaboratory comparisons
Organization, performance and evaluation of measurements or tests on the same or similar
items by two or more laboratories in accordance with predetermined conditions
(NBN EN ISO/IEC 17043 clause 3.4)
3.3.
Measurement Audit /Bilateral proficiency testing
Laboratories receive a test item with accurately determined characteristics, which is to be
tested in the context of an accreditation process. The test item is given either by the assessor
or provided by a third party.
BELAC 2-106 Rev 1 – 2011 - 6/ 11
4.
4.1.
-
GENERAL POLICY AND PRACTICAL ASPECTS REGARDING
PARTICIPATION IN PROFICIENCY TESTING.
General
Proficiency testing is a means by which laboratories can prove their technical competence
and the level of operation of their management system. Proficiency testing is in most
cases a part of the validation procedure.
Accreditation bodies use the evaluation of the performance level of laboratories during
assessments as an essential element that in the context of the mutual recognitions, can be
used for the acceptance of results on national and international level. Additionally, it can
provide an indication for effectiveness and efficiency of assessors and in general, confirm
the value of the granted accreditation.
-
Analyses of the performance in proficiency testing shall be used as a tool to improve the
level of operation of the laboratory management system. It is not intended to be used as a
tool to confirm or not the accreditation status .
However, repeated or major failures during the performance of proficiency testing should
result in sanctions and this to ensure the credibility of the accreditation concept.
-
Improvement and enforcement of laboratory quality can also be accomplished by regular
participation in interlaboratory comparisons used for different purposes (e.g. method
validation or characterization of reference materials). These interlaboratory comparisons
should be adapted to be able to service the purpose of proficiency testing. The results
obtained by interlaboratory comparisons should be interpreted with great caution.
-
In specific sectors where no proficiency testing is available, the laboratory should use
alternative resources to prove its technical competence. These include
o The regular use of (certified) reference materials;
o Comparison of results obtained by independent techniques;
o Comparison of the results of 2 or more laboratories on their own initiative.
The general requirements concerning the policy on participation in proficiency testing
also apply to laboratories using alternative means.
4.2.
Terms for participation in proficiency testing
4.2.1. Proficiency testing selection: general provisions
-
A high level of technical competence is required from the organizer of proficiency testing.
This competence is necessary for the planning, preparation, performance, interpretation
and documentation of the results of these tests.
BELAC 2-106 Rev 1 – 2011 - 7/ 11
-
The proficiency testing organizer can be selected by the laboratories. This organizer is
considered to be a service supplier (ISO 17025 clause 4.6). The selection should be based
on the evaluation of the suitability of proposed services regarding the needs. Previous
experience of the laboratory with the organizer of proficiency testing can serve as the
basis for the selection.
In general, the evaluation should include the following elements:
• Competence of the proficiency testing organizer: this can be proven by accreditation
according to the standard NBN EN ISO/IEC 17043. If the laboratory relies on an
organizer that is not accredited, the laboratory is required to perform and document an
evaluation.
• Material/matrix type: should, as far as possible, correspond with the materials/matrixes
examined in the laboratory;
• Parameters to be measured: should include as many parameters as possible as actually
determined by the laboratory;
• Range of measurement: should be as close as possible to the actual situation in the
laboratory;
• Frequency: should meet the laboratories needs to ensure that participation in
proficiency testing is an efficient means to evaluate quality management. Other means
can also be used, by the laboratory, to ensure the quality of their results;
• Statistical result analysis: the used protocol should be clearly defined.
-
In many cases it will be difficult – if not impossible – to select a proficiency testing that
covers all of the above mentioned criteria, and each test/calibration or group of
test/calibrations covered by the accreditation. A compromise will be necessary and it is
the laboratory’s task to justify its choices while ensuring adequate coverage of activities
covered by accreditation (price / efficiency ratio).
-
BELAC recommends laboratories to consult the EPTIS database (European PT
Information System – www.eptis.bam.de) to, whenever necessary, identify the
appropriate proficiency testing for their activities.
4.2.2. Proficiency testing selection: additional provisions
-
BELAC can require laboratories to participate in specific proficiency testing, on their
own costs, in case elements indicate that there is need to do so; ; in this case BELAC will
preferentially select an organizer whose management conforms to the requirements of
the standard NBN EN ISO/IEC 17043.
-
Arguments that might lead to such initiatives are: the absence of proficiency testing in a
certain sector or special problems discovered during assessment of a laboratory in a
specific sector.
-
In some sectors, participation in specific proficiency testing is mandatory in the context
of the contract with the customer, and in particular in the case accreditation is a
prerequisite for agreement by a regulatory authority.
-
In the context of the international mutual recognitions by EA and ILAC, international
proficiency testing programs are organized. The purpose is to ensure mutual trust on an
international level. The laboratories invited by BELAC to participate in these tests are
BELAC 2-106 Rev 1 – 2011 - 8/ 11
obliged to participate, but they cannot demand any compensation for the provided
services.
4.3.
Responsibilities of the laboratory concerning the participation in proficiency
testing.
4.3.1. Documentation
-
The policy of the laboratory concerning the participation in proficiency testing shall be
documented in the quality manual or in other documents pertaining to the management
system of the laboratory. Will be especially considered: planning, performance, evaluation
of the results, corrective actions and archiving.
For selection of proficiency testing organizers: see 4.2.1.
-
The laboratories shall moreover maintain an updated synthesis of the results of
proficiency testing in which they participated, or in case of lack of proficiency testing, of
other initiatives taken to ensure the validity of the test results. This shall be the case for
each test/calibration or group of tests/calibrations covered by accreditation. This
document shall be available to BELAC in preparation of each assessment.
4.3.2. Frequency of participations
-
An accredited laboratory shall take part in proficiency testing on a regular testing and
define its frequency of participation based on a risk assessment. This should take into
account the following:
Number of performed tests, calibrations or measurements;
The possibility to use the results of a specific proficiency testing for
assessment of performance of a group of tests based on similar measuring
techniques and/or concerning a group of products (equal level of technical
competence);
Number of involved operators and frequency of turnover in technical staff;
The education level and general experience of the personnel;
The availability or not of source of traceability (national standards, certified
reference material);
The complexity of the used measurement technique;
The level of criticality of the result, taking into account its final use;
The use of alternative techniques for the evaluation of the results in addition to
the participation in proficiency testing.
A re-evaluation of the choice of participation frequency shall be performed in case of
important changes in laboratory organization.
-
BELAC can, for a specific sector, impose a higher participation frequency or specific
implementation provisions.
-
An accreditation for one or several tests/calibrations can only be provided if the
performance level of the laboratory can be proven by its results of proficiency testing or,
in the absence of these proficiency testing, by other sources as mentioned above.
-
If proficiency testing is available, the laboratory should justify why it did not participate.
BELAC 2-106 Rev 1 – 2011 - 9/ 11
4.3.3. Statistical evaluation of the results
-
The statistical result evaluation is a key element to estimate the laboratories performance.
Investigation concerning consistency between performance (in terms of trueness and
precision) and the uncertainty reported by the laboratory is of great importance to decide
if corrective actions have to be taken and to determine the characteristics of these actions.
4.3.4. Corrective actions
-
In the case of unsatisfactory results the laboratory should investigate the causes and
possible consequences and if necessary take appropriate actions. These actions should
involve feedback to analysis results or calibration certificates that could be influenced and
which have already been send to customers. The actions should also comprise some
technical corrective and preventive measures that guaranty that the quality of the future
test results will satisfy the costumers demands and expectations.
-
As long as the laboratory cannot prove its test/calibration competence, either via
proficiency testing or any other source available to the laboratory, it is not allowed to
issue results covered by accreditation. If this situation continues, the laboratory is obliged
to inform BELAC of this situation.
4.4.
BELAC responsibilities in the evaluation by BELAC of the results of
participation in proficiency testing.
4.4.1. Evaluation during accreditation assessments
-
During each assessment the BELAC assessors shall check the performance in proficiency
testing. They have to evaluate efficiency of corrective actions and preventive measures.
The assessor specifically focus on following items:
-
-
the choice of proficiency testing and participation frequency;
the quality of the proficiency testing organizer;
the examination of raw data;
the performance conditions should match with those used for the routine-analyses;
the analysis of the result by the laboratory;
the consistency with the uncertainty of measurements normally reported;
the extent of the observed deviation in relation to the customers’ expectations;
the corrective actions and measures undertaken concerning analysis reports that were
already sent to customers when proficiency testing outcome resulted in modifications
of the test procedure;
the complexity of the test.
-
The findings shall be mentioned in the assessment report.
-
Where appropriate and in particular in calibration activities, the assessor can ask for a
bilateral test/calibration (measurement audit) being performed as part of the assessment
activities. .
BELAC 2-106 Rev 1 – 2011 - 10/ 11
4.4.2. Communication of results of proficiency testing to BELAC.
The Accreditation secretariat investigates the results of the proficiency testing in which
BELAC accredited laboratories participated in case they are presented to BELAC for
example by competent authorities or international organizations. In case of unsatisfactory
performance, BELAC contacts the concerned laboratory and the competent technical
assessors. An additional assessment will be organized if necessary.
4.4.3. Taking into account the proficiency testing results in a certain technical sector
Repeated and collective unsatisfactory performance of BELAC accredited laboratories in a
certain sector should result in a thorough discussion in the Accreditation Bureau with respect
to the need for general corrective measures; this can include additional training of BELAC
assessors active in this specified sector.
4.5.
Sanctions within the context of result evaluation of proficiency testing
An unsatisfactory performance without effective corrective actions or repeated (consecutive
or not) bad performances in proficiency testing, will result in a sanction. This will normally
be a temporary suspension of the concerned test/calibration from the accreditation scope.
When, for a particular laboratory, problems arise in multiple sectors the complete
accreditations status can be questioned.
An inappropriate management of the participation in proficiency testing is indeed a proof of
an ineffective quality management system, in particular as far as the control of test results is
concerned , which is the main aim of accreditation.
Before deciding to reduce the scope or to withdraw the accreditation status, the Accreditation
Bureau, as decision-making body, will contact the involved assessor and the concerned
laboratory will be given the opportunity to present its arguments. . Focus will be on aspects
described in 4.4.1.
___________________________________________________________________________
BELAC 2-106 Rev 1 – 2011 - 11/ 11