Anatomic Pathology/ Cytotechnologist On-Site Adequacy Evaluation of Thyroid FNA Cytotechnologist-Attended On-Site Adequacy Evaluation of Thyroid Fine-Needle Aspiration Comparison With Cytopathologists and Correlation With the Final Interpretation Matthew T. Olson, MD,1 Armanda D. Tatsas, MD,1 and Syed Z. Ali, MD1,2 Key Words: On-site evaluation for adequacy; Thyroid FNA; Cytotechnologist; Cytopathologist DOI: 10.1309/AJCP84AXSRABZCTZ Abstract Recent increases in the number of thyroid fineneedle aspiration (FNAs) biopsies and the popularity of on-site evaluation for adequacy (OSEA) have led many practices, including ours, to rely on cytotechnologists for performing OSEA. We retrospectively analyzed the accuracy of a cytotechnologist against that of a cytopathologist in performing OSEA and making the final diagnosis. Of 2,261 thyroid FNA specimens evaluated over a 33-month period under ultrasound guidance with OSEA, the cytotechnologist attended 64.7% (1,462/2,261) of the procedures whereas the cytopathologist attended 35.3% (799/2,261). There was no difference in the adequacy downgrade rate for cytotechnologists compared with that for cytopathologists during this study period (4.1% vs 5.0% downgrade rate, P = .33). Regardless of who rendered the OSEA, subadequate specimens had a higher rate of indeterminate diagnosis (25.2%) than those specimens deemed adequate at the time of OSEA (11.9%, P = .00001). These results indicate that the accuracy of cytotechnologists is comparable with that of cytopathologists in conducting OSEA of the thyroid. 90 90 Am J Clin Pathol 2012;138:90-95 DOI: 10.1309/AJCP84AXSRABZCTZ Fine-needle aspiration (FNA) biopsy of the thyroid is a valuable and time-proven diagnostic modality for the assessment of thyroid nodules.1-3 Despite its widespread use, the procedure has the potential for yielding material that is inadequate or suboptimal for rendering a diagnosis. A shortage of cellularity is a common factor leading to the inability to make a definitively malignant or benign diagnosis. For this reason, adequacy evaluation is a key component of cytopathologic evaluation of thyroid FNA and is included as a component of The Bethesda System for Reporting Thyroid Cytopathology (TBSRTC).4-7 Pathologists and clinical investigators have tested multiple strategies to minimize the frequency of inadequate thyroid FNA specimens. These have included gross examination of the direct smear,8-10 universal increase in the number of passes and slides,11,12 standardization of techniques for noncytologist specimen handling,13 and microscopic on-site evaluation of adequacy (OSEA) of a direct smear. Of these, most of the attention in the literature has focused on OSEA by cytopathologists. The investigations to date that have focused on thyroid OSEA have had mixed results, including a decrease in the inadequacy rate,14,15 no change in the inadequacy rate,16 or a small change in the inadequacy rate that does not justify the increased procedure time and expense.12 Even if the decrease in the inadequacy rate is modest, the benefits of OSEA are clear when they are viewed on a large institutional scale.17 An on-site cytologist brings other advantages such as a preliminary impression of the disease process and the ability to process material appropriately for special studies. For these reasons, the demand for OSEA is expected © American Society for Clinical Pathology Anatomic Pathology / Original Article to increase, and this will lead to increased demand for cytologists. Cytotechnologists appear ideally placed to address the rising demand caused by OSEA, so there is a need to evaluate their performance in this evolving role. However, the literature to date on this subject has been limited. One article demonstrated excellent accuracy in OSEA when cytotechnologists attended the OSEA instead of cytopathologists.18 Another study demonstrated agreement between the cytopathologists and cytotechnologists in a paired analysis.19 The number of thyroid FNAs in both of these series is relatively small because of the lack of these investigators’ experience with cytotechnologist-attended OSEA. By contrast, our institution has several years of experience with cytotechnologist-attended OSEA. Our current analysis focuses on our institution’s experience with a large number of cytotechnologist-attended OSEA. Because cytopathologists also performed OSEA in the same period, we also compared the performance of cytotechnologists and cytopathologists in this role. Materials and Methods Patients and Specimens All cases of thyroid cytopathology were retrieved from the computerized record system between January 2009 and September 2011. Of all cases, 99.4% were performed with ultrasound guidance and OSEA (2,261/2,275). These were chosen for analysis. The FNA procedures were performed by a combination of endocrinologists and interventional radiologists. Smears evaluated on site were prepared in duplicate and included at least 1 air-dried slide for Romanowsky staining and on-site evaluation and an alcohol-fixed slide for Papanicolaou staining for each pass. Each procedure was given a final adequacy evaluation of “adequate,” “less than optimal,” or “inadequate.” Preliminary diagnostic impressions are not routinely given at our institution. After the procedure, both the Romanowsky- and Papanicolaou-stained slides were previewed by a cytotechnologist and signed out by a boardcertified cytopathologist by strict application of TBSRTC guidelines. If the OSEA was attended by a cytopathologist, the same cytopathologist rendered the final diagnosis and adequacy evaluation. As required under TBSRTC, the final diagnosis included an evaluation of adequacy with the same 3 choices available during on-site evaluation. Data Analysis and Statistics The retrieved data were queried for the cytologist performing the OSEA, the OSEA verdict, the final adequacy, and the final TBSRTC diagnosis. A Perl script automated this process, and each case underwent a rapid visual inspection with a Filemaker (Santa Clara, CA)–based graphical user interface that allowed for the corrections of any errors, such as typographical irregularities, made by the automated analysis. Nodule sizes were obtained from the pathology report when they were recorded. When they were not recorded in the pathology report, the size was obtained from the ultrasound guidance procedure note or radiologic report. After this process, it was confirmed that 24 aspirates (1.1%) were collected in the absence of any identifiable nodule. The adequacies and diagnoses from these aspirates were counted for statistical purposes but excluded from calculation of the metrics regarding the nodule size. Half of the diagnostic material was unavailable to the cytologist at the time of OSEA, therefore accuracy was defined as either complete agreement or an upgrade of adequacy in the final diagnosis. Inaccuracy was defined as any downgrade in adequacy. While the total number of needle passes is not tracked explicitly in the pathology database, the number of smears is documented and was used to approximate the number of needle passes (needle passes = integer [(smears + 1)/2]). The differences of sex distribution for patients with cytopathologists vs cytotechnologists attending the OSEA were assessed with the Fisher exact test. Differences in age, nodule size, and number of passes for these patient groups were evaluated with the Student t test. Differences involving accuracy and adequacy were evaluated with the χ2 test. All statistical tests, calculations, and graphs were performed using R scripts executed in the R console (http://cran.r-project.org). Results Over the past 10 years, the number of thyroid FNAs at our institution underwent a median annual growth rate of 2.0% in the number of patients and 2.5% in the number of thyroid FNA procedures, from 422 FNAs (357 patients) in 2002 to 821 FNAs (647 patients) in 2011. Our analysis began in 2009 for several reasons. First, the universal implementation of TBSRTC began at this time in our institution, and the standardized accuracy reporting that accompanied this change allowed for the data consistency required for such analyses. Second, the standardized diagnostic categories allow for a simultaneous analysis of the effect of specimen adequacy on the rates of the indeterminate categories. Third, although on-site evaluation has been practiced for longer than this period, it was not universal until this time. During the study period, only 17 thyroid FNAs—comprising 0.6% of all thyroid FNAs at our institution—were excluded from the study for lack of OSEA. Monthly counts of cytotechnologist- and cytopathologist-attended OSEA are shown in ❚Figure 1❚. Overall, cytotechnologists attended 64.7% of the thyroid FNAs during © American Society for Clinical Pathology Am J Clin Pathol 2012;138:90-95 91 DOI: 10.1309/AJCP84AXSRABZCTZ 91 91 Olson et al / Cytotechnologist On-Site Adequacy Evaluation of Thyroid FNA 100 T T FNA OSEA Count T T 80 T T 60 T C T C C C 40 20 C TT T M M M C T C M T M MM M C M C C M C M M M C C M MM M CM MC M M C T C C CCC Gender (M/F) Age (y) Nodule cize (cm) T C M C M M M M M 11/2009 10/2010 M 09/2011 ❚Figure 1❚ On-site evaluation of adequacy (OSEA) statistics for cytotechnologists (C) and cytopathologists (M) over the study period. The total number of fine-needle aspirates for the month is shown (T). the study period, with a range of 34.0% to 97.7%. The monthly FNA counts varied from 43 to 101, with most of the increased volume being met by increased allocation of cytotechnologist time to attending thyroid FNA. The dramatic shift at the end of the study period reflects a change in our clinical practice; cytotechnologists are now sent to almost all thyroid OSEA. ❚Table 1❚ shows a demographic comparison of patients who had a cytotechnologist- vs cytopathologistattended OSEA. The demographics of these 2 patient groups did not differ significantly. The on-site adequacy determinations are compared with the final adequacy determinations for both cytotechnologists ❚Table 2❚ and cytopathologists ❚Table 3❚, and the individual breakdowns for the 3 most active cytopathologists and 5 most active cytopathologists are shown in ❚Figure 2❚. Cytopathologists and cytotechnologists had similar accuracy rates of 96% and 95%, respectively (P = .33). The on-site adequacy rate was also comparable for cytotechnologists and cytopathologists (61.6% vs 65.6%; P = .0693); our study found no correlation between the experience of the cytotechnologist and the accuracy rate (P = .11). Interestingly, the final adequacy was higher than the on-site adequacy in more cytotechnologist-attended cases (26%) than in cytopathologist-attended cases (17%), and this difference is significant (P = .000001). In addition, the percentage of cases that were ultimately signed out as adequate was also significantly higher for the cases in which a cytotechnologist was present for the OSEA (81.0% vs 77.2%; P = .038). The adequacy underestimation is possibly related to the finding that cytotechnologists had access to fewer passes than cytopathologists at the time of OSEA. The median number of passes in a cytotechnologist-attended OSEA is 3 whereas cytopathologist-attended OSEAs have a median of 5 passes (P < . 000001). Am J Clin Pathol 2012;138:90-95 DOI: 10.1309/AJCP84AXSRABZCTZ Cytopathologist OSEA P 1/3.7 54.3 ± 14.9 2.2 ± 1.2 1/3.6 54.7 ± 14.8 2.1 ± 1.2 .51 .55 .53 M Month 92 92 Cytotechnologist OSEA OSEA, on-site evaluation of adequacy. * Data are given as mean ± SD. M 0 01/2009 T C T CC T T MCC CM M CT T T T T T T T T C C T T T ❚Table 1❚ Comparison of Patients With Cytotechnologist OSEA vs Cytopathologist OSEA* T C T ❚Table 2❚ On-Site vs Final Adequacy for Cytotechnologist-Attended OSEA* Final Adequacy OSEA (N = 1,462) Inadequate (n = 161) Less than optimal (n = 400) Adequate (n = 901) Inadequate (n = 79) Less Than Optimal (n = 199) Adequate (n = 1,184) 59 (36.7) 15 (3.75) 57 (35.4) 103 (25.8) 45 (28.0) 282 (70.5) 5 (0.55) 39 (4.33) 857 (95.1) OSEA, on-site evaluation of adequacy. * The numbers as a percentage of the OSEA determination are given in parentheses. Boldface data indicate inaccurate OSEA determinations; all others indicate accuracy. ❚Table 3❚ On-Site vs Final Adequacy for Cytopathologist-Attended OSEA* Final Adequacy OSEA (N = 799) Inadequate (n = 93) Less than optimal (n = 182) Adequate (n = 524) Inadequate (n = 68) Less Than Adequate Optimal (n = 114) (n = 617) 52 (55.9) 15 (8.24) 19 (20.4) 71 (39.0) 22 (23.7) 96 (52.8) 1 (0.19) 24 (4.58) 499 (95.2) OSEA, on-site evaluation of adequacy. * The numbers as a percentage of the OSEA determination are given in parentheses. Boldface data indicate inaccurate OSEA determinations; all others indicate accuracy. The relationships between the on-site and final adequacy determinations and TBSRTC diagnoses are shown in ❚Figure 3❚. Figure 3A demonstrates that when cases are upgraded from the inadequate category, they have a higher proportion of the indeterminate categories than cases that are deemed to be adequate on site (25.2% vs 11.9%; P = .00001). This finding holds when the cases deemed less than optimal and inadequate on site are included, even though the indeterminate rate is slightly lower (19.9% vs 11.9%; P = .000002). The final indeterminate diagnosis rate was comparable regardless of whether a cytotechnologist (20.7%) or cytopathologist (17.8%) rendered the OSEA (P = .43). Regardless of the OSEA, it is clear from Figure 3B that the final adequacy is © American Society for Clinical Pathology 72.5% (129/178) 25.3% (45/178) 2.2% (4/178) 79.2% (99/125) 16.0% (20/125) 4.8% (6/125) 82.4% (89/108) 15.7% (17/108) 1.9% (2/108) 90.4% (47/52) 1.9% (1/52) 7.7% (4/52) 9.5% (23/243) 8.6% (21/243) 100 1.5% (12/793) 200 76.1% (140/184) 16.8% (31/184) 7.1% (13/184) 300 74.6% (334/448) 400 33.5% (266/793) FNA OSEA Count 500 81.9% (199/243) Complete agreement with OSEA OSEA upgraded OSEA downgraded 18.5% (83/448) 6.9% (31/448) 600 64.9% (515/793) Anatomic Pathology / Original Article MD2 MD3 MD4 MD5 0 CT1 CT2 CT3 MD1 ❚Figure 2❚ Follow-up of on-site evaluations of adequacy (OSEA) for cytotechnologists (CT) and cytopathologists (MD). associated with indeterminate TBSRTC diagnoses. Specimens with a final less-than-optimal adequacy account for only 13.8% of all thyroid FNAs but contribute 30.0% of all the indeterminate diagnoses. The indeterminate diagnosis rate for cases with a final adequacy of less-than-optimal specimens is 29.4% compared with 11.9% for cases deemed adequate at sign out (P < .0000001). Discussion The results of our study demonstrate an acceptable interchangeability of cytotechnologists and cytopathologists for OSEA of the thyroid. For the analysis here, accuracy is defined as both complete agreement and adequacy upgrade, and inaccuracy is defined as an adequacy downgrade. Others have defined accuracy in this context as the percentage of cases that are called adequate on site and are subsequently found to be adequate and unequivocally benign or malignant.18 Although this definition may make sense for other types of cytology specimens, it is too stringent for thyroid cytology. The rates of indeterminate diagnoses are widely known and outlined by TBSRTC, so it is possible for a thyroid FNA specimen to be both adequate and indeterminate. In addition, because half of the material is unstained at the time of OSEA, adequacy upgrades should not be considered an inaccuracy. For the purposes of this article as well as our practice, the only unacceptable discordance between on-site and final adequacy determinations is an adequacy downgrade. While the adequacy downgrade rate is the same for both cytology providers, adequacy evaluations by a cytotechnologist are more likely to be upgraded than those made by a cytopathologist. This is of no concern given that patients receiving cytotechnologist-attended OSEA do not undergo more passes. However, it is a feature of the data that warrants some discussion because it may reflect a limitation of this retrospective study. At our institution, cytotechnologists had less exposure to inexperienced operators given the days and clinics they served during the study period. Based on other reports,20 one would expect this to explain both the lower number of passes and higher final adequacy rates for patients who had a cytotechnologist at their on-site evaluation. Records are not available about which or how many passes are performed by trainees, so it is impossible to fully quantify the contribution of operator experience to the adequacy upgrade rates. However, if the increased adequacy upgrade rate were purely because of a different combination of aspirators and trainees, one would expect that the on-site adequacy rate would also be higher. This is not the case; the on-site adequacy rate is comparable for both cytotechnologists and cytopathologists. There appears to be a tendency for cytotechnologists to underestimate adequacy on site. This behavior is probably unavoidable given that cytotechnologists do not sign out their own aspiration cases and cytopathologists do. © American Society for Clinical Pathology Am J Clin Pathol 2012;138:90-95 93 DOI: 10.1309/AJCP84AXSRABZCTZ 93 93 6.5% (93/1,425) 0.9% (5/582) 0.8% (2/254) 1.0% (14/1,425) 1.2% (3/254) 0.7% (4/582) 4.2% (60/1,425) 3.8% (22/582) 0.8% (2/254) 13.1% (76/582) 5.2% (30/582) 20 0.42% (6/1,425) 40 6.7% (95/1,425) 60 40.9% (104/254) 43.7% (111/254) % Cases 80 Inadequate Less than optimal Adequate 11.8% (30/254) 100 75.9% (442/582) A 81.0% (1,154/1,425) Olson et al / Cytotechnologist On-Site Adequacy Evaluation of Thyroid FNA 0 SFM 0 Inadequate Benign AUS SFN/SHCN SFM 5.5% (100/1,802) 0.8% (15/1,802) 20 1.9% (6/313) 40 4.3% (78/1,802) 60 Malignant Inadequate Less than optimal Adequate 25.6% (80/313) % Cases 80 SFN/SHCN 1.9% (6/313) 100 6.8% (122/1,802) 70.6% (221/313) B AUS 0% (0/313) Benign 82.5% (1,487/1,802) Inadequate Malignant ❚Figure 3❚ A, Relationship between on-site adequacy evaluation and sign-out diagnosis. B, Relationship between sign-out adequacy and diagnosis. AUS, atypia of undetermined significance; SFM, suspicious for malignancy; SFN, suspicious for follicular neoplasm; SHCN, suspicious for Hürthle cell neoplasm. The ultimate goal of adequacy evaluations is to reduce the need for repeat procedures and ambiguous final diagnoses. By this measure, cytotechnologists and cytopathologists perform similarly. As expected, specimens identified as suboptimal at the time of OSEA have a significantly higher rate of indeterminate classifications, and this is true regardless of the type of cytologist performing the OSEA. This finding suggests that both cytopathologists and cytotechnologists are able 94 94 Am J Clin Pathol 2012;138:90-95 DOI: 10.1309/AJCP84AXSRABZCTZ to recognize quantitatively compromised aspirates. Because our institution does not provide on-site preliminary diagnoses, the data here cannot be used to determine if cytotechnologists and cytopathologists are equally capable at identifying qualitatively compromised specimens on site at our institution. This will be an area for future inquiry at our institution and elsewhere because parity on this skill is necessary in the setting of new molecular tests such as Afirma21 (Veracyte, © American Society for Clinical Pathology Anatomic Pathology / Original Article South San Francisco, CA), which requires a dedicated pass and storage in a specific medium. There is currently no mechanism for laboratories to bill for cytotechnologist services rendered during OSEA.18,22 In the absence of any revenue generation, the financial justifications for cytotechnologist-attended OSEA currently involve savings from a decreased rebiopsy rate on an institutional level or savings vis-à-vis cytopathologist time.19 The economics of cytotechnologist-attended OSEA for our institution are beyond the scope of this article. However, there is clearly a need for billing codes that will allow for all laboratories to recuperate the personnel costs involved in cytopathologist OSEA. More studies such as ours are needed to demonstrate the suitability of cytopathologists to perform OSEA and to formulate a fee that is commensurate with the value of this service. The data presented here demonstrate that cytotechnologists can perform OSEA for thyroid FNA with a level of accuracy similar to that of cytopathologists. At our institution, we noted a slight tendency for underestimating adequacy, but the number of passes was not greater. Specimens deemed to be suboptimal on site were more likely than adequate ones to be placed in the indeterminate categories on final diagnosis. This was true regardless of the type of cytologist who performed the OSEA. From the 1Departments of Pathology and 2Radiology, The Johns Hopkins Hospital, Baltimore, MD. Address reprint requests to Dr Ali: Department of Pathology, The Johns Hopkins Hospital, Path 406, 600 N Wolfe St, Baltimore, MD 21287; sali@jhmi.edu. References 1. Wong CK, Wheeler MH. Thyroid nodules: rational management. World J Surg. 2000;24:934-941. 2. Hamberger B, Gharib H, Melton LJ, et al. Fine-needle aspiration biopsy of thyroid nodules: impact on thyroid practice and cost of care. Am J Med. 1982;73:381-384. 3. Silverman JF, West RL, Finley JL, et al. Fine-needle aspiration versus large-needle biopsy or cutting biopsy in evaluation of thyroid nodules. Diagn Cytopathol. 1986;2:25-30. 4. Cibas ES, Ali SZ, Conference NTFSotS. The Bethesda System for Reporting Thyroid Cytopathology. Am J Clin Pathol. 2009;132:658-665. 5. Cibas ES, Ali SZ. The Bethesda System for Reporting Thyroid Cytopathology. Thyroid. 2009;19:1159-1165. 6. Cibas ES, Sanchez MA. The National Cancer Institute thyroid fine-needle aspiration state-of-the-science conference: inspiration for a uniform terminology linked to management guidelines. Cancer. 2008;114:71-73. 7. Baloch ZW, Cibas ES, Clark DP, et al. The National Cancer Institute thyroid fine needle aspiration state of the science conference: a summation. Cytojournal. 2008;5:6. 8. Mayall F, Cormack A, Slater S, et al. The utility of assessing the gross appearances of FNA specimens. Cytopathology. 2010;21:395-397. 9. Prayaga AK, Vijaya K. Role of unstained smears in determining sample adequacy. Acta Cytol. 2004;48:321-324. 10. Nguyen YP, Maple JT, Zhang Q, et al. Reliability of gross visual assessment of specimen adequacy during EUS-guided FNA of pancreatic masses. Gastrointest Endosc. 2009;69:12641270. 11. Rossi ED, Morassi F, Santeusanio G, et al. Thyroid fine needle aspiration cytology processed by ThinPrep: an additional slide decreased the number of inadequate results. Cytopathology. 2010;21:97-102. 12. Eedes CR, Wang HH. Cost-effectiveness of immediate specimen adequacy assessment of thyroid fine-needle aspirations. Am J Clin Pathol. 2004;121:64-69. 13. Sidiropoulos N, Dumont LJ, Golding AC, et al. Quality improvement by standardization of procurement and processing of thyroid fine-needle aspirates in the absence of on-site cytological evaluation. Thyroid. 2009;19:1049-1052. 14. Zhu W, Michael CW. How important is on-site adequacy assessment for thyroid FNA? An evaluation of 883 cases. Diagn Cytopathol. 2007;35:183-186. 15. Redman R, Zalaznick H, Mazzaferri EL, et al. The impact of assessing specimen adequacy and number of needle passes for fine-needle aspiration biopsy of thyroid nodules. Thyroid. 2006;16:55-60. 16. O’Malley ME, Weir MM, Hahn PF, et al. US-guided fineneedle aspiration biopsy of thyroid nodules: adequacy of cytologic material and procedure time with and without immediate cytologic analysis. Radiology. 2002;222:383-387. 17. Nasuti JF, Gupta PK, Baloch ZW. Diagnostic value and cost-effectiveness of on-site evaluation of fine-needle aspiration specimens: review of 5,688 cases. Diagn Cytopathol. 2002;27:1-4. 18. Burlingame OO, Kessé KO, Silverman SG, et al. On-site adequacy evaluations performed by cytotechnologists: correlation with final interpretations of 5241 image-guided fine needle aspiration biopsies [published online ahead of print August 21, 2011]. Cancer Cytopathol. 2011. 19. Wotruba AL, Stewart J, Scheberl T, et al. Added value, decreased cost: The evolving role of the cytotechnologist for preliminary screening and triage of thyroid aspirates. Diagn Cytopathol. 2011;39:896-899. 20. Ghofrani M, Beckman D, Rimm DL. The value of onsite adequacy assessment of thyroid fine-needle aspirations is a function of operator experience. Cancer. 2006;108:110-113. 21. Chudova D, Wilde JI, Wang ET, et al. Molecular classification of thyroid nodules using high-dimensionality genomic data. J Clin Endocrinol Metab. 2010;95:5296-5304. 22. Dhillon I, Pitman MB, Demay RM, et al. Compensation crisis related to the onsite adequacy evaluation during FNA procedures: urgent proactive input from cytopathology community is critical to establish appropriate reimbursement for CPT code 88172 (or its new counterpart if introduced in the future). Cytojournal. 2010;7:23. © American Society for Clinical Pathology Am J Clin Pathol 2012;138:90-95 95 DOI: 10.1309/AJCP84AXSRABZCTZ 95 95