Article in press - uncorrected proof Clin Chem Lab Med 2006;44(8):1039–1043 2006 by Walter de Gruyter • Berlin • New York. DOI 10.1515/CCLM.2006.174 2006/92 Guidelines for setting up an External Quality Assessment Scheme for blood smear interpretation. Part II: survey preparation, statistical evaluation and reporting Hematology Working Group of the European External Committee for External Quality Assurance Programmes in Laboratory Medicine (EQALM) Keywords: blood smears; external quality assessment; guidelines; reporting; statistical evaluation. Introduction Juan-Lluis Vives Corrons1, Marjan Van Blerk2, Stephanie Albarède3, Gabriela Gutierrez4, Silke Heller5, Gunnar Nordin6, Milan Skitek7, Andre Deom8, Katalin Horváth9, Barbara De la Salle10 and Jean-Claude Libeer2,* 1 CDB-IDIBAPS, Hospital Clinic i Provincial, Barcelona, Spain 2 Department of Clinical Biology, Scientific Institute of Public Health, Brussels, Belgium 3 AEHH EQAS, CDB, Hospital Clinic i Provincial, Barcelona, Spain 4 Unité Contrôle National de Qualité, DEDIMAFSSAPS, Paris, France 5 INSTAND, Berlin, Germany 6 EQUALIS, Uppsala, Sweden 7 Clinical Institute for Clinical Chemistry and Biochemistry, U.M.C. Ljubljana, Ljubljana, Slovenia 8 CSCQ, Chêne Bourg, Switzerland 9 QualiCont, Szeged, Hungary 10 UK NEQAS for Haematology, London, UK Abstract Blood smear analysis is a well-known technique in medical laboratories. The clinical relevance of this analysis and its interpretation are very important. Consequently, monitoring of laboratory performance by an external quality assessment scheme is strongly recommended. This article represents a very practical guidance document for External Quality Assessment Scheme (EQAS) organizers for setting up blood smear schemes. In the first part of the guidelines, the Hematology Working Group of the European External Committee for External Quality Assurance Programmes in Laboratory Medicine (EQALM) published practical information for the preparation of blood smears for use in an EQAS. Part II focuses on aspects such as survey preparation, statistical evaluation and reporting, and describes particular details for organizing blood morphology EQA surveys by means of virtual microscopy. Clin Chem Lab Med 2006;44:1039–43. *Corresponding author: Jean-Claude Libeer, Clinical Biology Department. Scientific Institute of Public Health, Juliette Wytsmanstraat 14, 1050 Brussels, Belgium Phone: q32-2-6425527, Fax: q32-2-6425645, E-mail: jean-claude.libeer@iph.fgov.be Examination of a blood smear (or film) is the first approach in the diagnosis of hematological disorders and is commonly requested by physicians as a response to perceived clinical features or to an abnormality shown in a previous complete blood count. Blood smear findings include a full quantitative differential leukocyte count, discovery of the presence of unusual cells, and the morphology of red cells, platelets and leukocytes. ‘‘Reading’’ a blood smear requires expertise and only laboratory-trained staff members are able to undertake this pivotal task. An External Quality Assessment Scheme (EQAS) for blood smear interpretation is a method for assessing the professional skills of laboratory staff to recognize and interpret morphological features. A recent overview on diagnosis based on blood smears was published by Bain (1). Only a few reports are available in the literature on blood film evaluations as a quality control activity (2–4). It was the intention of the Working Group to produce a practical guideline for EQAS organizers. In the first part of the guidelines, the Hematology Working Group of the European External Committee for External Quality Assurance Programmes in Laboratory Medicine (EQALM) published practical information for the preparation of blood smears for use in an EQAS (5). Part II focuses on aspects such as survey preparation, statistical evaluation and reporting. Depending on the scheme design, an EQAS for blood smear interpretation will focus on three different types of evaluations: 1. Evaluation of the quality of the smears and the staining. For this type of evaluation, the participant is requested to send some routine stained samples to the EQAS organizer. 2. Evaluation of blood cell morphology (BCM) and the leukocyte differential count (LDC). For this type of evaluation, the EQAS organizer sends one or more blood smears to the participants. The participants perform the BCM and LDC using their routine procedures and send the results back to the organizer according to a standard reply format, depending on the level of skill of the laboratory for this type of analysis. For basic laboratories, the ability to differentiate a normal from an abnormal smear can be sufficient if, under routine conditions, abnormal smears are referred to a more specialized laboratory. Other laboratories Brought to you by | Marquette University Raynor Memorial Libraries Authenticated | 10.248.254.158 Download Date | 8/17/14 4:21 PM Article in press - uncorrected proof 1040 Vives Corrons et al.: Guidelines for the preparation, statistical evaluation and reporting of EQAS for blood smears must be able to recognize the most relevant abnormalities in the three cell lines and recognize abnormal cells that can be found in the blood (for instance, tumor cells and endothelial cells). 3. Evaluation of the information provided to the clinician and linked to the BCM examination. This type of evaluation focuses on more specialized laboratories. Additional information, such as clinical data and results of other determinations, allows the possibility of suggesting a diagnosis. In this document, we focus only on the evaluation of BCM and LDC. Accordingly, the following aspects are considered: • Survey preparation and mailing of control material; • Statistical evaluation of the results; • Reporting the evaluation data and relevant clinical information. As computers decrease in cost and increase in power, pathologists are becoming more comfortable with their use. For this reason, in the next few years, virtual microscopy will become an important tool for EQA in both clinical pathology and hematology (bone marrow, cervix cytology, histology, sperm morphology, urine sediment analysis, microbial stains). In this case, EQAS organizers can use digitized blood smears as described by Lundin et al. (6). Particular aspects for organizing blood morphology EQA surveys by means of virtual microscopy are given in a separate chapter. Survey preparation and mailing of control material The traditional method for the design of a hematology EQAS is to send out stained glass smears. Recommendations for their preparation are given in part I of this document (5). Blood smears packed into plastic containers or aircraft envelopes can be sent by ordinary mail as ‘‘UN 3733’’ diagnostic specimens. Recommendations on the mailing of such samples are given elsewhere (7). Number of surveys per year A minimum of three surveys with at least two control samples is recommended. Information provided to the participants 1. Short clinical background (age and gender of the patient, reason for admission, relevant symptoms and other information if useful for morphology evaluation). 2. Complete blood count. 3. Additional information, if necessary (i.e., results obtained by flow cytometry, bone marrow analysis, «). Evaluation data required in the reply form 1. Status of the laboratory (basic, routine hospital laboratory, specialized hematology laboratory, «). 2. Appreciation of the quality of the blood smear (good/bad, acceptable or not acceptable for evaluation). New slides must be available if a participant estimates that his slide is not acceptable. 3. Appreciation of the BCM (normal/abnormal). 4. Description of the main abnormalities observed in red blood cells, white blood cells and platelets (from a list provided by the EQAS organizer). 5. LDC performed according to the routine procedure of each laboratory. 6. A diagnostic suggestion (from a list provided by the EQAS organizer). This list should be based on the WHO classification of diseases (8) and of hematological tumors (9). 7. Suggestion for further tests (free text). Statistical evaluation of the results BCM Statements on the presence or absence of certain significant red blood cells and leukocyte abnormalities and on the sufficiency, morphology, excess or lack of platelets in the sample are qualitative results. Therefore, participant results are compared to a ‘‘correct’’ result as defined by a steering committee, which should consist of at least three expert laboratories. Use of a consensus of participant results is not recommended in order to avoid bias if a majority of routine laboratories have made errors in the BCM. The data analysis and report should include the number or percentage of participants for each abnormal element observed. Global performance assessment The ‘‘correct’’ answers are determined by the steering committee and the laboratory from which the material originated, who should provide a report of the definitive or likely diagnosis, as well as any additional information, if necessary. The number of participants considering the smear as normal or abnormal and those who have suggested a diagnosis from the list should be scored and printed in the report. If analysis of the data reveals any common faults affecting results, this information can be presented as learning points in a more detailed practical exercise. Individual performance assessment Individual results are considered accurate if all the significant features are reported correctly, and acceptable when minor features are missed or added. The EQAS organizer should use predefined criteria for ‘‘significant features’’ and ‘‘minor features’’. The result is considered unsatisfactory when significant morphological features (as defined by the consensus of the Brought to you by | Marquette University Raynor Memorial Libraries Authenticated | 10.248.254.158 Download Date | 8/17/14 4:21 PM Article in press - uncorrected proof Vives Corrons et al.: Guidelines for the preparation, statistical evaluation and reporting of EQAS for blood smears 1041 expert group) have been missed or when a clinically misleading result is reported. The steering committee establishes the principles necessary for identifying poor or unsatisfactory performance. Acceptable performance will vary with the difficulty of the sample and with the expected results, depending on the skill of individual laboratories. LDC From the results provided by the participants, the EQAS organizer calculates the response rate, the mean (or median) and the standard deviation (SD) for each cell type. The initial arithmetic mean and SD are determined and then recalculated after exclusion of outliers. More information on the use of statistics in EQA schemes can be found in ISO/FDIS 13528 (10). In a non-parametric statistical approach, no presumption is made on the distribution and the median is used as the central value. In the approach of Tukey (11), the SD is derived from the equation SDs (P75–P25/1.349), where P25 and P75 are the 25th and 75th percentiles, respectively. Using this procedure, outliers are automatically excluded in the calculation. If the sum of the different cell types falls outside 99–101% for a given laboratory, the results must be excluded from the statistical analysis. The numerical results provided by each laboratory are compared with the consensus mean or the median of the results reported, or are compared to the results from at least three expert laboratories. The range of acceptable results may be calculated using the consensus value "2 (or 3) SD. An alternative method based on the statistical studies of Rümke (12) can be used to delineate the appropriate confidence bands. The deviation allowed from the consensus value may also be determined according to clinical requirements or based on biology goals (13). The reference value and the range of acceptable results may be determined by a group of expert laboratories (14, 15). The choice between these different approaches should be the responsibility of the steering committee. EQAS reports A preliminary report, including the target values of the results calculated using an appropriate procedure, should be provided to the participating laboratories as soon as possible after the closing date and always before the next survey. This report should include information concerning the distribution of results from all the participants or participant groups, together with an indication of the individual performance as described in the IFCC guidelines for the requirements for the competence of EQAP organizers in medical laboratories (16). All the reports must be clear and comprehensive. Individual performance assessment report This report should include the evaluation results for BCM and LDC, and the suggested diagnosis. For BCM the evaluation report should include: 1. The results of the individual participant, the consensus result provided by the expert panel or the steering committee (expected or correct diagnosis and acceptable alternative diagnoses) and the number of responses for each diagnosis given. 2. The number (or percentage) of participants reporting each of the most frequently observed abnormal elements. 3. When possible, and in particular if a web site is available, photographs of the main abnormalities with a short description of the elements observed. For LDC the evaluation report should include: 1. A statistical analysis (mean, median, SD, percentile, and coefficient of variation) of the global results for each leukocyte subpopulation (neutrophils, bands, monocytes, lymphocytes, eosinophils, basophils and abnormal cells). Results can also be presented as a scatter diagram or histogram with the overall distribution. 2. The assessment of individual (participant) results calculated using a scoring system (e.g., z-score, Rümke method). For the suggested diagnosis the evaluation report should include: 1. The result of the individual participant and the correct diagnosis selected from a list of diagnostic possibilities. To avoid misunderstanding given by the use of different terminology for the same disease, EQAS organizers should provide a list of diagnostic possibilities (8, 9). 2. The number (or percentage) of participants reporting each of the diagnostic possibilities. Global performance assessment report The participants should also receive a report of the global results obtained from all the participants without individual performance identification. This report can be used as an educational tool by including additional information when necessary. The EQAS organizer should hold the copyright of these reports to assure that the data have been received and used properly, as described in the IFCC guidelines for the requirements for the competence of EQAP organizers in medical laboratories (16). The report should include the evaluation results for BCM and LDC, and the suggested diagnosis. For BCM the evaluation report should include: 1. A summary of the results for all participants and the consensus results provided by the expert panel. 2. The number (or percentage) of participants reporting each of the most frequently observed abnormal elements. 3. When possible, and in particular if a web site is available, photographs of the main abnormalities. Brought to you by | Marquette University Raynor Memorial Libraries Authenticated | 10.248.254.158 Download Date | 8/17/14 4:21 PM Article in press - uncorrected proof 1042 Vives Corrons et al.: Guidelines for the preparation, statistical evaluation and reporting of EQAS for blood smears For LDC the evaluation report should include a statistical analysis of the global results for each leukocyte subpopulation (neutrophils, bands, monocytes, lymphocytes, eosinophils, basophils and abnormal cells). Results can also be presented as a scatter diagram or histogram with the overall distribution. For the suggested diagnosis, the evaluation report should include the consensus of the expert panel given as expected or correct diagnosis or an acceptable diagnostic alternative as well as the number of responses for each diagnosis provided. Additional information should be included for educational purposes: 1. Photographs of the abnormal morphological features, either in the report or provided on a web site. 2. A complete description of the case, including a full clinical background, with results of other necessary investigations and treatment. 3. Information about differential diagnoses (diagnoses with pathologies giving similar results or a false diagnosis given by a majority of participants). In this case, photographs could also be worthwhile. 4. An article written by an expert, containing comments on the results, prognosis for the patient, and latest medical developments for the pathology concerned. The reports can be distributed as a printed version and sent by ordinary mail or by fax, or using electronic media or the Internet. The report may be sent directly to an e-mail address or may be accessible on a web site using safe security systems to guarantee the confidentiality of individual performance assessments. Since the global report does not contain details of individual performances, the issue of confidentiality is less important than for the individual reports. The global report can also be distributed on DVD/CD-ROM as an alternative for participants without rapid Internet access. EQA for blood smears by means of virtual microscopy Virtual microscopy is very likely to mature in the next few years to become the method of choice for EQA schemes in pathology. In addition, applications in hematology, especially for bone marrow, as well as peripheral blood smear schemes, will become common EQA practice. Overviews of the applications in general (17) and in hematology in particular (18) have been published. Instead of receiving real glass smear samples, participants receive digitized blood smears directly by e-mail or on the Internet, or sent in digital format (CD or DVD). Image capture technology is now able to mimic a microscope nearly completely; continuous scanning of a whole slide prevents the small shifts observed between tiles in the tile scanning technique, which requires tiles to be stitched together afterwards. Zoom facilities are an alternative for magnifi- cations; microtuning is now included in the newer software as a ‘‘z-stack’’ in different layers. EQA organizations can have access to image capture technology in large pathology departments and increasing numbers of commercial companies are also offering this service. Both EQA organizations and experts can prepare a library of digitized slides. This library could be managed somewhere on a central server, together with the software applications, and could be shared by several EQA organizations. Each organizer could then choose images from the library and set up their own scheme design. The organization of such surveys requires participants with access to a recent PC and a screen with acceptable resolution. New technology could have a built-in system control, checking the local computer settings and blocking user access if minimum requirements are not met. In order to familiarize participants with this new medium, it is recommended that the first EQA is sent as a glass smear and a digitized smear of the same preparation. The possibilities provided by digital imaging combined with Internet applications open totally new opportunities in EQAS for medical laboratories. Every element or field on the digitized slide can now be unequivocally defined by x-y coordinates, offering possibilities never available before in EQA. For EQA on peripheral blood smears, the following new possibilities for virtual microscopy have been identified: 1. Advanced tracking, which allows checking of which elements have been identified by each participant. This allows the EQA organizer to verify if a participant has correctly identified the observed leukocyte elements. 2. Reports can now include references to particular fields of interest in the whole smear. 3. Reporting in real time. 4. Improving the educational role of EQA: the EQA organizer can pre-annotate a number of leukocytes and ask participants to categorize each cell. With this feature, the interpretation of each preannotated cell can be compared to the consensus evaluation. 5. Familiarization of participants with other applications, such as EQA of bone marrow and cervix cytology. References 1. Bain B. Diagnosis from the blood smear. N Engl J Med 2005;353:498–507. 2. Lewis SM. Blood film evaluations as quality control activity. Clin Lab Haematol 1990;12(Suppl 1):119–27. 3. Rajamäki A. Interlaboratory variation of leukocyte differential count: results from the Finnish proficiency testing programme in haematological morphology, 1974–1977. Scand J Clin Lab Invest 1979,39:613–7. 4. Rajamäki A. External quality control in haematological morphology: a method to assess the performance of an individual laboratory and changes in it. Scand J Clin Lab Invest 1980,40:79–84. Brought to you by | Marquette University Raynor Memorial Libraries Authenticated | 10.248.254.158 Download Date | 8/17/14 4:21 PM Article in press - uncorrected proof Vives Corrons et al.: Guidelines for the preparation, statistical evaluation and reporting of EQAS for blood smears 1043 5. Vives Corrons JL, Albarède S, Flandrin G, Heller S, Horvath K, Houwen B, et al. Guidelines for blood smear preparation and staining procedure for setting up an external quality assessment scheme for blood smear interpretation. Part I: control material. Clin Chem Lab Med 2004;42:922–6. 6. Lundin M, Lundin J, Isola J. Virtual microscopy. Applications in digital pathology. J Clin Pathol 2004;57: 1250–1. 7. http: // www.iata.org / NR / ContentConnector/CS2000/SiteInterface / pdf / cargo / dg / Consignment_diagnostic_specimens_2003.pdf. 8. WHO. International classification of diseases. http:// www.who.int/classifications/icd/en/. 9. Jaffe ES, Harris NL, Stein H, Vardiman JW, editors. WHO classification of tumours of haematopoietic and lymphoid tissues. Lyon: IARC Press, 2001. 10. ISO/FDIS 13528. Statistical methods for use in proficiency testing by interlaboratory comparisons, 2005. 11. Tukey JW. Exploratory data analysis. Reading, MA: Addison-Wesley, 1977. 12. Rümke CL. De nauwkeurigheid van percentages: tabellen met betrouwbaarheidsintervallen. Ned Tijdschr Geneesk 1976;120:2052–8. 13. Skitek M. Acceptability limits based on biology goals in haematology EQAS. Accred Qual Assur 2005;10:112–5. 14. Heller S. External quality assessment scheme for haematology in Germany. Ann Ist Super Sanit 1995;31: 87–93. 15. Vives Corrons JL, Gutierrez G, Jou JM, Reverter JC, Martı́nez Brotons F, Domingo A, et al. Characteristics and performance of the External Quality Assessment Scheme (EQAS) for haematology in Spain. 10 years of experience. Ann Ist Super Sanita 1995;31:95–101. 16. Maziotta D, Harel D, Schumann G, Libeer JC. Guidelines for the requirement for the competence of EQAP organizers in medical laboratories. IFCC/EMD/C-AQ 2003 http:// www.eqalm.org/education/IFCC%20document_PDF.pdf. 17. Libeer JC. Virtual microscopy: revolutionary means for EQAS in the near future. Accred Qual Assur 2006, DOI/ 10.1007/s00769-006-0146-4. In press. 18. Lee SH. Virtual microscopy: applications to hematology. Lab Hematol 2005;11:38–45. Received February 27, 2006, accepted April 27, 2006 Brought to you by | Marquette University Raynor Memorial Libraries Authenticated | 10.248.254.158 Download Date | 8/17/14 4:21 PM