Article in press - uncorrected proof
Clin Chem Lab Med 2006;44(8):1039–1043 2006 by Walter de Gruyter • Berlin • New York. DOI 10.1515/CCLM.2006.174
2006/92
Guidelines for setting up an External Quality Assessment
Scheme for blood smear interpretation. Part II: survey
preparation, statistical evaluation and reporting
Hematology Working Group of the European
External Committee for External Quality
Assurance Programmes in Laboratory
Medicine (EQALM)
Keywords: blood smears; external quality assessment; guidelines; reporting; statistical evaluation.
Introduction
Juan-Lluis Vives Corrons1, Marjan Van Blerk2,
Stephanie Albarède3, Gabriela Gutierrez4, Silke
Heller5, Gunnar Nordin6, Milan Skitek7, Andre
Deom8, Katalin Horváth9, Barbara De la Salle10
and Jean-Claude Libeer2,*
1
CDB-IDIBAPS, Hospital Clinic i Provincial,
Barcelona, Spain
2
Department of Clinical Biology, Scientific Institute
of Public Health, Brussels, Belgium
3
AEHH EQAS, CDB, Hospital Clinic i Provincial,
Barcelona, Spain
4
Unité Contrôle National de Qualité, DEDIMAFSSAPS, Paris, France
5
INSTAND, Berlin, Germany
6
EQUALIS, Uppsala, Sweden
7
Clinical Institute for Clinical Chemistry and
Biochemistry, U.M.C. Ljubljana, Ljubljana, Slovenia
8
CSCQ, Chêne Bourg, Switzerland
9
QualiCont, Szeged, Hungary
10
UK NEQAS for Haematology, London, UK
Abstract
Blood smear analysis is a well-known technique in
medical laboratories. The clinical relevance of this
analysis and its interpretation are very important.
Consequently, monitoring of laboratory performance
by an external quality assessment scheme is strongly
recommended. This article represents a very practical
guidance document for External Quality Assessment
Scheme (EQAS) organizers for setting up blood smear
schemes. In the first part of the guidelines, the Hematology Working Group of the European External Committee for External Quality Assurance Programmes in
Laboratory Medicine (EQALM) published practical
information for the preparation of blood smears for
use in an EQAS. Part II focuses on aspects such as
survey preparation, statistical evaluation and reporting, and describes particular details for organizing
blood morphology EQA surveys by means of virtual
microscopy.
Clin Chem Lab Med 2006;44:1039–43.
*Corresponding author: Jean-Claude Libeer, Clinical
Biology Department. Scientific Institute of Public Health,
Juliette Wytsmanstraat 14, 1050 Brussels, Belgium
Phone: q32-2-6425527, Fax: q32-2-6425645,
E-mail: jean-claude.libeer@iph.fgov.be
Examination of a blood smear (or film) is the first
approach in the diagnosis of hematological disorders
and is commonly requested by physicians as a
response to perceived clinical features or to an abnormality shown in a previous complete blood count.
Blood smear findings include a full quantitative differential leukocyte count, discovery of the presence
of unusual cells, and the morphology of red cells,
platelets and leukocytes. ‘‘Reading’’ a blood smear
requires expertise and only laboratory-trained staff
members are able to undertake this pivotal task. An
External Quality Assessment Scheme (EQAS) for
blood smear interpretation is a method for assessing
the professional skills of laboratory staff to recognize
and interpret morphological features. A recent overview on diagnosis based on blood smears was published by Bain (1). Only a few reports are available in
the literature on blood film evaluations as a quality
control activity (2–4). It was the intention of the Working Group to produce a practical guideline for EQAS
organizers. In the first part of the guidelines, the
Hematology Working Group of the European External
Committee for External Quality Assurance Programmes in Laboratory Medicine (EQALM) published
practical information for the preparation of blood
smears for use in an EQAS (5). Part II focuses on
aspects such as survey preparation, statistical evaluation and reporting.
Depending on the scheme design, an EQAS for
blood smear interpretation will focus on three different types of evaluations:
1. Evaluation of the quality of the smears and the
staining. For this type of evaluation, the participant is requested to send some routine stained
samples to the EQAS organizer.
2. Evaluation of blood cell morphology (BCM) and
the leukocyte differential count (LDC). For this
type of evaluation, the EQAS organizer sends one
or more blood smears to the participants. The participants perform the BCM and LDC using their
routine procedures and send the results back to
the organizer according to a standard reply
format, depending on the level of skill of the laboratory for this type of analysis. For basic laboratories, the ability to differentiate a normal from
an abnormal smear can be sufficient if, under routine conditions, abnormal smears are referred to
a more specialized laboratory. Other laboratories
Brought to you by | Marquette University Raynor Memorial Libraries
Authenticated | 10.248.254.158
Download Date | 8/17/14 4:21 PM
Article in press - uncorrected proof
1040
Vives Corrons et al.: Guidelines for the preparation, statistical evaluation and reporting of EQAS for blood smears
must be able to recognize the most relevant
abnormalities in the three cell lines and recognize
abnormal cells that can be found in the blood (for
instance, tumor cells and endothelial cells).
3. Evaluation of the information provided to the clinician and linked to the BCM examination. This
type of evaluation focuses on more specialized
laboratories. Additional information, such as clinical data and results of other determinations,
allows the possibility of suggesting a diagnosis.
In this document, we focus only on the evaluation
of BCM and LDC. Accordingly, the following aspects
are considered:
• Survey preparation and mailing of control
material;
• Statistical evaluation of the results;
• Reporting the evaluation data and relevant clinical
information.
As computers decrease in cost and increase in power, pathologists are becoming more comfortable with
their use. For this reason, in the next few years, virtual
microscopy will become an important tool for EQA in
both clinical pathology and hematology (bone marrow, cervix cytology, histology, sperm morphology,
urine sediment analysis, microbial stains). In this
case, EQAS organizers can use digitized blood smears
as described by Lundin et al. (6). Particular aspects for
organizing blood morphology EQA surveys by means
of virtual microscopy are given in a separate chapter.
Survey preparation and mailing of control
material
The traditional method for the design of a hematology
EQAS is to send out stained glass smears. Recommendations for their preparation are given in part I of
this document (5). Blood smears packed into plastic
containers or aircraft envelopes can be sent by ordinary mail as ‘‘UN 3733’’ diagnostic specimens. Recommendations on the mailing of such samples are
given elsewhere (7).
Number of surveys per year
A minimum of three surveys with at least two control
samples is recommended.
Information provided to the participants
1. Short clinical background (age and gender of the
patient, reason for admission, relevant symptoms
and other information if useful for morphology
evaluation).
2. Complete blood count.
3. Additional information, if necessary (i.e., results
obtained by flow cytometry, bone marrow analysis, «).
Evaluation data required in the reply form
1. Status of the laboratory (basic, routine hospital
laboratory, specialized hematology laboratory,
«).
2. Appreciation of the quality of the blood smear
(good/bad, acceptable or not acceptable for evaluation). New slides must be available if a participant estimates that his slide is not acceptable.
3. Appreciation of the BCM (normal/abnormal).
4. Description of the main abnormalities observed in
red blood cells, white blood cells and platelets
(from a list provided by the EQAS organizer).
5. LDC performed according to the routine procedure of each laboratory.
6. A diagnostic suggestion (from a list provided by
the EQAS organizer). This list should be based on
the WHO classification of diseases (8) and of
hematological tumors (9).
7. Suggestion for further tests (free text).
Statistical evaluation of the results
BCM
Statements on the presence or absence of certain significant red blood cells and leukocyte abnormalities
and on the sufficiency, morphology, excess or lack of
platelets in the sample are qualitative results. Therefore, participant results are compared to a ‘‘correct’’
result as defined by a steering committee, which
should consist of at least three expert laboratories.
Use of a consensus of participant results is not recommended in order to avoid bias if a majority of routine laboratories have made errors in the BCM. The
data analysis and report should include the number
or percentage of participants for each abnormal element observed.
Global performance assessment
The ‘‘correct’’
answers are determined by the steering committee
and the laboratory from which the material originated, who should provide a report of the definitive or
likely diagnosis, as well as any additional information,
if necessary.
The number of participants considering the smear
as normal or abnormal and those who have suggested a diagnosis from the list should be scored and
printed in the report. If analysis of the data reveals
any common faults affecting results, this information
can be presented as learning points in a more detailed
practical exercise.
Individual performance assessment
Individual
results are considered accurate if all the significant
features are reported correctly, and acceptable when
minor features are missed or added. The EQAS
organizer should use predefined criteria for ‘‘significant features’’ and ‘‘minor features’’. The result is
considered unsatisfactory when significant morphological features (as defined by the consensus of the
Brought to you by | Marquette University Raynor Memorial Libraries
Authenticated | 10.248.254.158
Download Date | 8/17/14 4:21 PM
Article in press - uncorrected proof
Vives Corrons et al.: Guidelines for the preparation, statistical evaluation and reporting of EQAS for blood smears 1041
expert group) have been missed or when a clinically
misleading result is reported. The steering committee
establishes the principles necessary for identifying
poor or unsatisfactory performance. Acceptable performance will vary with the difficulty of the sample
and with the expected results, depending on the skill
of individual laboratories.
LDC
From the results provided by the participants, the
EQAS organizer calculates the response rate, the
mean (or median) and the standard deviation (SD) for
each cell type. The initial arithmetic mean and SD are
determined and then recalculated after exclusion of
outliers. More information on the use of statistics in
EQA schemes can be found in ISO/FDIS 13528 (10). In
a non-parametric statistical approach, no presumption is made on the distribution and the median is
used as the central value. In the approach of Tukey
(11), the SD is derived from the equation SDs
(P75–P25/1.349), where P25 and P75 are the 25th and
75th percentiles, respectively. Using this procedure,
outliers are automatically excluded in the calculation.
If the sum of the different cell types falls outside
99–101% for a given laboratory, the results must be
excluded from the statistical analysis.
The numerical results provided by each laboratory
are compared with the consensus mean or the median of the results reported, or are compared to the
results from at least three expert laboratories. The
range of acceptable results may be calculated using
the consensus value "2 (or 3) SD. An alternative
method based on the statistical studies of Rümke (12)
can be used to delineate the appropriate confidence
bands. The deviation allowed from the consensus value may also be determined according to clinical
requirements or based on biology goals (13). The reference value and the range of acceptable results
may be determined by a group of expert laboratories
(14, 15). The choice between these different approaches should be the responsibility of the steering
committee.
EQAS reports
A preliminary report, including the target values of
the results calculated using an appropriate procedure,
should be provided to the participating laboratories
as soon as possible after the closing date and always
before the next survey. This report should include
information concerning the distribution of results
from all the participants or participant groups, together with an indication of the individual performance as
described in the IFCC guidelines for the requirements
for the competence of EQAP organizers in medical
laboratories (16). All the reports must be clear and
comprehensive.
Individual performance assessment report
This report should include the evaluation results for
BCM and LDC, and the suggested diagnosis.
For BCM the evaluation report should include:
1. The results of the individual participant, the consensus result provided by the expert panel or the
steering committee (expected or correct diagnosis
and acceptable alternative diagnoses) and the
number of responses for each diagnosis given.
2. The number (or percentage) of participants reporting each of the most frequently observed abnormal elements.
3. When possible, and in particular if a web site is
available, photographs of the main abnormalities
with a short description of the elements observed.
For LDC the evaluation report should include:
1. A statistical analysis (mean, median, SD, percentile, and coefficient of variation) of the global
results for each leukocyte subpopulation (neutrophils, bands, monocytes, lymphocytes, eosinophils, basophils and abnormal cells). Results can
also be presented as a scatter diagram or histogram with the overall distribution.
2. The assessment of individual (participant) results
calculated using a scoring system (e.g., z-score,
Rümke method).
For the suggested diagnosis the evaluation report
should include:
1. The result of the individual participant and the
correct diagnosis selected from a list of diagnostic
possibilities. To avoid misunderstanding given by
the use of different terminology for the same disease, EQAS organizers should provide a list of
diagnostic possibilities (8, 9).
2. The number (or percentage) of participants reporting each of the diagnostic possibilities.
Global performance assessment report
The participants should also receive a report of the
global results obtained from all the participants without individual performance identification. This report
can be used as an educational tool by including additional information when necessary. The EQAS organizer should hold the copyright of these reports to
assure that the data have been received and used
properly, as described in the IFCC guidelines for the
requirements for the competence of EQAP organizers
in medical laboratories (16). The report should include
the evaluation results for BCM and LDC, and the suggested diagnosis.
For BCM the evaluation report should include:
1. A summary of the results for all participants and
the consensus results provided by the expert
panel.
2. The number (or percentage) of participants reporting each of the most frequently observed abnormal elements.
3. When possible, and in particular if a web site is
available, photographs of the main abnormalities.
Brought to you by | Marquette University Raynor Memorial Libraries
Authenticated | 10.248.254.158
Download Date | 8/17/14 4:21 PM
Article in press - uncorrected proof
1042
Vives Corrons et al.: Guidelines for the preparation, statistical evaluation and reporting of EQAS for blood smears
For LDC the evaluation report should include a statistical analysis of the global results for each leukocyte subpopulation (neutrophils, bands, monocytes,
lymphocytes, eosinophils, basophils and abnormal
cells). Results can also be presented as a scatter diagram or histogram with the overall distribution.
For the suggested diagnosis, the evaluation report
should include the consensus of the expert panel
given as expected or correct diagnosis or an acceptable diagnostic alternative as well as the number of
responses for each diagnosis provided. Additional
information should be included for educational
purposes:
1. Photographs of the abnormal morphological features, either in the report or provided on a web
site.
2. A complete description of the case, including a full
clinical background, with results of other necessary investigations and treatment.
3. Information about differential diagnoses (diagnoses with pathologies giving similar results or a
false diagnosis given by a majority of participants). In this case, photographs could also be
worthwhile.
4. An article written by an expert, containing comments on the results, prognosis for the patient,
and latest medical developments for the pathology concerned.
The reports can be distributed as a printed version
and sent by ordinary mail or by fax, or using electronic media or the Internet. The report may be sent
directly to an e-mail address or may be accessible on
a web site using safe security systems to guarantee
the confidentiality of individual performance assessments. Since the global report does not contain
details of individual performances, the issue of confidentiality is less important than for the individual
reports. The global report can also be distributed on
DVD/CD-ROM as an alternative for participants without rapid Internet access.
EQA for blood smears by means of virtual
microscopy
Virtual microscopy is very likely to mature in the next
few years to become the method of choice for EQA
schemes in pathology. In addition, applications in
hematology, especially for bone marrow, as well as
peripheral blood smear schemes, will become common EQA practice. Overviews of the applications in
general (17) and in hematology in particular (18) have
been published.
Instead of receiving real glass smear samples, participants receive digitized blood smears directly by
e-mail or on the Internet, or sent in digital format (CD
or DVD). Image capture technology is now able to
mimic a microscope nearly completely; continuous
scanning of a whole slide prevents the small shifts
observed between tiles in the tile scanning technique,
which requires tiles to be stitched together afterwards. Zoom facilities are an alternative for magnifi-
cations; microtuning is now included in the newer
software as a ‘‘z-stack’’ in different layers.
EQA organizations can have access to image capture technology in large pathology departments and
increasing numbers of commercial companies are
also offering this service. Both EQA organizations and
experts can prepare a library of digitized slides. This
library could be managed somewhere on a central
server, together with the software applications, and
could be shared by several EQA organizations. Each
organizer could then choose images from the library
and set up their own scheme design. The organization
of such surveys requires participants with access to a
recent PC and a screen with acceptable resolution.
New technology could have a built-in system control,
checking the local computer settings and blocking
user access if minimum requirements are not met.
In order to familiarize participants with this new
medium, it is recommended that the first EQA is sent
as a glass smear and a digitized smear of the same
preparation. The possibilities provided by digital
imaging combined with Internet applications open
totally new opportunities in EQAS for medical
laboratories.
Every element or field on the digitized slide can
now be unequivocally defined by x-y coordinates,
offering possibilities never available before in EQA.
For EQA on peripheral blood smears, the following
new possibilities for virtual microscopy have been
identified:
1. Advanced tracking, which allows checking of
which elements have been identified by each participant. This allows the EQA organizer to verify if
a participant has correctly identified the observed
leukocyte elements.
2. Reports can now include references to particular
fields of interest in the whole smear.
3. Reporting in real time.
4. Improving the educational role of EQA: the EQA
organizer can pre-annotate a number of leukocytes and ask participants to categorize each cell.
With this feature, the interpretation of each preannotated cell can be compared to the consensus
evaluation.
5. Familiarization of participants with other applications, such as EQA of bone marrow and cervix
cytology.
References
1. Bain B. Diagnosis from the blood smear. N Engl J Med
2005;353:498–507.
2. Lewis SM. Blood film evaluations as quality control activity. Clin Lab Haematol 1990;12(Suppl 1):119–27.
3. Rajamäki A. Interlaboratory variation of leukocyte differential count: results from the Finnish proficiency testing
programme in haematological morphology, 1974–1977.
Scand J Clin Lab Invest 1979,39:613–7.
4. Rajamäki A. External quality control in haematological
morphology: a method to assess the performance of an
individual laboratory and changes in it. Scand J Clin Lab
Invest 1980,40:79–84.
Brought to you by | Marquette University Raynor Memorial Libraries
Authenticated | 10.248.254.158
Download Date | 8/17/14 4:21 PM
Article in press - uncorrected proof
Vives Corrons et al.: Guidelines for the preparation, statistical evaluation and reporting of EQAS for blood smears 1043
5. Vives Corrons JL, Albarède S, Flandrin G, Heller S, Horvath K, Houwen B, et al. Guidelines for blood smear
preparation and staining procedure for setting up an
external quality assessment scheme for blood smear
interpretation. Part I: control material. Clin Chem Lab
Med 2004;42:922–6.
6. Lundin M, Lundin J, Isola J. Virtual microscopy. Applications in digital pathology. J Clin Pathol 2004;57:
1250–1.
7. http: // www.iata.org / NR / ContentConnector/CS2000/SiteInterface / pdf / cargo / dg / Consignment_diagnostic_specimens_2003.pdf.
8. WHO. International classification of diseases. http://
www.who.int/classifications/icd/en/.
9. Jaffe ES, Harris NL, Stein H, Vardiman JW, editors. WHO
classification of tumours of haematopoietic and lymphoid tissues. Lyon: IARC Press, 2001.
10. ISO/FDIS 13528. Statistical methods for use in proficiency testing by interlaboratory comparisons, 2005.
11. Tukey JW. Exploratory data analysis. Reading, MA:
Addison-Wesley, 1977.
12. Rümke CL. De nauwkeurigheid van percentages: tabellen
met betrouwbaarheidsintervallen. Ned Tijdschr Geneesk 1976;120:2052–8.
13. Skitek M. Acceptability limits based on biology goals in
haematology EQAS. Accred Qual Assur 2005;10:112–5.
14. Heller S. External quality assessment scheme for haematology in Germany. Ann Ist Super Sanit 1995;31:
87–93.
15. Vives Corrons JL, Gutierrez G, Jou JM, Reverter JC, Martı́nez Brotons F, Domingo A, et al. Characteristics and
performance of the External Quality Assessment
Scheme (EQAS) for haematology in Spain. 10 years of
experience. Ann Ist Super Sanita 1995;31:95–101.
16. Maziotta D, Harel D, Schumann G, Libeer JC. Guidelines
for the requirement for the competence of EQAP organizers in medical laboratories. IFCC/EMD/C-AQ 2003 http://
www.eqalm.org/education/IFCC%20document_PDF.pdf.
17. Libeer JC. Virtual microscopy: revolutionary means for
EQAS in the near future. Accred Qual Assur 2006, DOI/
10.1007/s00769-006-0146-4. In press.
18. Lee SH. Virtual microscopy: applications to hematology.
Lab Hematol 2005;11:38–45.
Received February 27, 2006, accepted April 27, 2006
Brought to you by | Marquette University Raynor Memorial Libraries
Authenticated | 10.248.254.158
Download Date | 8/17/14 4:21 PM