Raouf E. Nakhleh, MD
Mayo Clinic Florida

Raouf E Nakhleh, MD
• I have no financial relationships to disclose

• Discuss evidence based guideline development and their impact on the future from a QA perspective
• Discuss issues related to immunohistochemistry validation
• Discuss ongoing assessment of practicing pathologists

• Not new
• Subject to regulation (USA)
• Formalization of issues not previously addressed

• How does one judge good medical practice?
• Much easier in retrospect, particularly when there is a negative outcome.
• Literature is vast with conflicting knowledge
• In the past, informal evolution of practice
• Emphasis on local standards
• Today national standards

• In particular situation:
• Guidelines provide a road map on how to proceed.
• Guidelines help us judge appropriateness of care
• From a practitioner’s perspective, easier to know if you are meeting your obligations
• From a quality assurance perspective, easier to create measures for comparison

• In pathology
• Cancer protocols and checklists
• Standard set by physicians
• Reporting standards
• Accreditation by Commission on Cancer
• Clinical Laboratory Improvement Act
• Standard set by legal decree
• Procedural and quality standards

• Cancer reporting
• 1980 report
• Breast, right, mastectomy:
• Infiltrating ductal carcinoma
• 1 of 20 lymph nodes positive
• Margins free

•
• INVASIVE BREAST CANCER
• Macroscopic
• 1. Specimen (partial breast, total breast including nipple and skin, etc)
• 2. Procedure (excision without wire mastectomy, etc)
‐ guided localization, total
• 3. Lymph Node Sampling (no lymph nodes present, sentinel lymph nodes, etc)
• 4. Specimen Integrity (single intact specimen, multiple designated specimens, etc)
• 5. Specimen Size (greatest dimension in cm)
• 6. Specimen Laterality (right, left, not specified)
• Microscopic
• 7. Tumor Size (size of largest invasive cancer, may also be based on macroscopic)
• 8. Tumor Focality (single focus, multiple foci, etc)
• 9. Skin extent (not present, not involved, invades dermis, etc)*
• 10. Nipple extent (not present, not involved, DICS, Paget’s disease)
• 11. Skeletal muscle extent (not present, present, etc)
• 12. Ductal Carcinoma In Situ (DCIS not present, present, etc)
• 13. Lobular Carcinoma In Situ (LCIS not present, present, etc)
• 14. Histologic Type of Invasive Carcinoma (ductal NOS, mucinous, papillary, etc)
• 15. Nottingham Score Overall Grade (1, 2, 3, not graded)
• 16. Tubular Differentiation (1, 2, 3, not graded)
• 17. Nuclear Pleomorphism (1, 2, 3, not graded)
• 18. Mitotic Count (1, 2, 3, not graded)
• 19. Margins (involved by invasive cancer, involved by DCIS, etc)
• 20. Lymph Nodes ‐ total number examined*
• 21. Lymph Nodes ‐ number with micrometastases*
• 22. Lymph Nodes ‐ number with macrometastases*
• 23. Lymph Nodes 0 size of largest metastatic deposit*
• 24. Tumor pT Stage (pTX, pT0, etc)
• 25. Regional Lymph Nodes pN Stage (pNX, pN0, pN1a, etc)
• 26. Distant Metastasis M Stage (not applicable, pM1, etc)
• Ancillary Studies
• 27. Estrogen Receptor Disposition (performed, not performed, etc)
• 28. Progesterone Receptor Disposition (performed, not performed, etc)
• 29. HER2 Disposition (performed, not performed, etc)
• * If not applicable to the specimen, code as if the element was present in the report

• Physician Quality Reporting System
• Tax Relief and Health Care Act of 2006
• Incentive pay (up to 2% of eligible Part B) for physician who report quality measures (payfor-reporting)
• Voluntary program
• 2015 becomes mandatory

• Colon Cancer grade and stage (2008)
• Breast Cancer grade and stage (2008)
• Barrett’s esophagus (2012)
• Prostate grade and stage (2012)
• HER2 determination in breast cancer (2012)

• Payment of ACO’s rewards quality and efficiency rather than volume
• Monitor practice patterns and use performance data to improve the quality of care
• CMS establishes performance measures
• If met, then eligible for shared savings
• Over time - higher standards
• Use evidence-based guidelines

Overview of Potential Quality System
Clinical studies
National or local QA review
Evidence based medicine
Practice Guidelines
Inspection, QA review and
PT of an institutions
Improvement
Accreditation
Addition of standards

• What’s in our present and future?

College of American Pathologists
Pathology and Laboratory Quality Center
• The Center develops evidence-based guidelines and consensus statements related to the practice of pathology and laboratory medicine. Through them, we continually improve the quality of diagnostic medicine and patient outcomes.

College of American Pathologists
Pathology and Laboratory Quality Center
• HER2, ER and PR before official Center creation
• CAP/ADASP Consensus Statements on Effective
Communication of Urgent Diagnoses and Significant
Unexpected Diagnoses in Surgical Pathology and
Cytopathology (published)
• Validating Whole Slide Imaging Systems for Diagnostic
Purposes in Digital Pathology
(complete, not published)
• CAP/ASCCP Lower Anogenital Squamous Terminology (LAST)
Standardization Project for HPV-associated Lesions
(complete, not published)
• CAP/IASLC/AMP Molecular Testing Guidelines for Selection of
Lung Cancer Patients for EGFR and ALK Tyrosine Kinase
Inhibitors
(near completion)

• Immunohistochemistry (IHC) Assay Analytic Validation
Principles
• CAP/ASH Algorithm for Initial Work-up of Acute Leukemia
• ASCP/CAP/AMP/ASCO Molecular Markers for the Evaluation of
Colorectal Cancer
• CAP-ADASP Interpretive Diagnostic Error Reduction in
Surgical Pathology and Cytopathology
• Bone Marrow Synoptic Reporting for Hematologic Neoplasms
• CAP-NSH Uniform Labeling Requirements for Blocks and
Slides in Surgical Pathology
• ASCO/CAP Guideline Recommendations for HER2 Testing in
Breast Cancer – Update to 2007 Edition
• ASCO/CAP Guideline Recommendations for HER2 Testing in
Gastric Cancer

1.
Patient risk, patient safety and quality; issues that affect patient care and quality outcomes
2. Performance characteristics of assay and ability to reach consensus. Do problems exist including high false positive/false negative rates and lack of uniformity in practice with significant ability to reach consensus?
3. Amount of available evidence to predict a clinical response to an agent or change a patient outcome
4. Guidelines are lacking but regulatory bodies or other professional organizations are trying to preempt specialty development of guidelines or approval/disapproval or show unusual interest
5. Adoption momentum of a particular assay. Consider the number of pathologists affected.
18

6. Feasibility of significantly changing practice of pathology or medicine (e.g., using Ultra
Sound Guided FNA to change cytology practice).
7. Ability to collaborate with the right partner(s) in development of a Center product to facilitate development and acceptance, and improve overall probability of success.
8. The public's perception of the issue negatively affects the image of pathologists and the pathologists' role in medicine.
9. Evaluation of existing methods is obsolete or does not bring value in patient care
19

Submit and Select
Ideas
Maintain
Determine Scope and Form
Workgroup
Publish and
Implement
Research and
Review
Review and
Approve
Complete
Recommendations
Solicit Public
Comment

• Antibodies are medical devices
• Classification base on patient safety
• Class I
• Adjunctive diagnostic information
• Class II
• Prognostic or predictive data
• Separately reported
• More stringent quality assurance
• Class III
• Require FDA premarket approval

• Provide the appropriate HER2 assay validation procedure
• Validation with FISH or with another IHC lab
• Prescribe appropriate fixation time and documentation: 6 – 48 Hours
• Modifies scoring criteria to enhance specificity
• Ongoing competency assessment for pathologists
• Enroll in HER2 proficiency testing service

• Guideline introduced in late 2006
• Mandated in 2008
• LAP checklist standards related to HER2 effective in late 2008
• CAP sponsored survey to determine how well labs are able to comply (late 2008)
• Leads to insight of effectiveness of guideline
(2010)

• Properly validate HER2 assays
• Ensuring appropriate fixation
• use the ASCO/CAP scoring criteria
• Enforce HER2 proficiency testing

• Arch Pathol Lab Med 2010;134: 728-734
• HER2 assay validation
• Concordance with FISH
• 81% of Labs achieved 95% concordance for(-)
• 73% of Labs achieved 95% concordance for(+)
• Concordance with another IHC LAB
• 72% of Labs achieved 95% concordance for(-)
• 68% of Labs achieved 95% concordance for(+)

• Fixation time and documentation
• 86% made changes to address fixation
• 60% process specimens on the weekend
• Documentation is addressed in reports in 70% of laboratories

• ASCO/CAP scoring criteria
• 84% using criteria
• Ongoing pathologists competency assessment
• 91% have a program in place

• Most labs are meeting HER2 guideline
• Gaps still exist, particularly validation
• Deficiencies hopefully corrected over time with inspection cycle
• Follow up survey conducted at the end of 2011, currently under analysis

• What about other antibodies?
• Gap in understanding of Validation in AP
• Few documents address IHC antibody assay validation
• Definition
• Confirmation through a defined process that a method performs as intended

31
• Use manufacturer’s instructions
• Use high-quality test materials with known target values
• Test materials should be of similar type as patient samples
• Complete documentation, including procedure and results
• Lab director review/approve results

• For cell markers, (e.g. keratin, actin, Melan A etc.) how many cases should be performed in validation study?
• Should you validate with another lab or another technique?
• What level of concordance is acceptable?
• Are there situations where validation is not necessary?
• What about cytology material?

• New antibody introduction – Prognostic
• Example: MIB-1 (Ki-67)
• Proliferation marker (estimate of mitotic count)
• Antibody needs to work at spectrum of findings (0 –
100%)
• Include tissues with various levels of proliferation
• Carcinoid tumors of lung (0-10%)
• Typical carcinoid (0-3%)
• Atypical carcinoid (2-10%)
• Small cell carcinoma (25-90%)
• Validation cases should cover the spectrum of findings

Immunohistochemistry Validation
Procedures and Practice: A College of
American Pathologists Survey of 727
Laboratories
Lindsay Hardy, MD (Boston, MA)
Raouf Nakhleh, MD (Jacksonville, FL) Jeffrey Goldsmith, MD (Boston, MA)
Patrick Fitzgibbons, MD (Fullerton, CA) Richard Eisen, MD (Greenwich, CT)
Mary Beth Beasley, MD (New York, NY) Rhona Souers, MS (Northfield, IL)

• September 2010: Questionnaire
• 1064 surveys distributed with CAP proficiency test mailing (HER-2)
• 754 returned (Oct. 2010)
• 27 excluded
• 727 included
• 2 major sections
• Non-FDA approved, non-predictive
• Non-FDA approved, predictive other than
HER-2

Validation Procedures
Laboratory has a procedure for validation of new antibodies?
Procedure specifies the number of cases to be used?
Percent
68
54

• 68% had a written procedure
• 86% validated the most recently introduced non-predictive marker (despite lack of procedures for some)

Validation Procedures
# Cases used in most recent validation:
Positive
Negative
2
1
7
5
20
15

Most recent validation
Weakly or focally positive cases were included
Results confirmed by running parallel at another lab (same tissue)
Cases were tested on multiple days (between-run precision)
Percent
40
18
53

Validation Procedures
Non-
Predictive
Percent Percent
54
Predictive Markers: o ER o PR o ER/PR o CD117/C-KIT o KI-67 o P53 o P63 o PMS2 o EGFR o Other

• 46% had a written procedure
• 75% validated the most recently introduced predictive marker other than HER-2

Validation Procedures
10 th
Pctl
25 th
Pctl
Median
75 th
Pctl
Minimum # cases to validate new
Ab:
Positive 3 5 11 20
Negative
Total Cases
Acceptable concordance (%):
Positive
Negative
1
5
90
50
5
10
90
90
10
25
95
95
15
35
95
95
# Cases used in most recent validation:
Positive
Negative
2
1
5
3
10
5
20
13
90 th
Pctl
25
25
50
99
99
30
20

Validation Procedures
Non-
Predictive
Percent Percent
37

• IHC validation: 727 labs
• Validation was performed at higher rates than the availability of written procedures
• Labs are generally following procedures
• Most meet designated minimum number of cases
• Trend to slightly fewer
• Guideline is needed to assist in Predictive and non-predictive IHC
• Uncertainty as to the # of cases needed
• Uncertainty as to when to revalidate vs. verification
• Uncertainty RE validation in cytology

• Why now?

• Time limited certification (10 yrs)
• The four part MOC process requires diplomates to submit documentation in the following areas:
• Part I – Professional Standing
• Part II – Lifelong Learning and Self-Assessment
• Part III – Cognitive Expertise
• Part IV – Practice Performance Assessment

• Ms.08.01.01 Monitoring Performance; The organized medical staff defines the circumstances requiring monitoring and evaluation of a practitioner's performance (FPPE)
• Ms.08.01.03 Use of Monitoring Information: Ongoing professional practice evaluation information is factored into the decision to maintain existing privileges, to revise existing privileges, or to revoke an existing privilege prior to or at the time of renewal

• Use of objective performance data in the granting and/or maintenance of practice privileges
Ongoing Professional Practice Evaluation – OPPE
• On an ongoing basis for privileged practitioners
Focused Professional Practice Evaluation – FPPE
• When a practitioner is first privileged
• When new privileges are first granted to an already privileged practitioner
• When insufficient activity or “performance issues” are identified for a privileged practitioner

• Professional competency assessment
• Is part of the peer review process
(protected)
• Is not publicly reported or stored in credentialing files
• Is a tool for evaluation, not an adverse action
• Produces some data points for assessment
• Thresholds are minimum bars, not goals

• Patient care
• Medical/Clinical knowledge
• Practice-based learning and improvement
• Interpersonal and communication skills
• Professionalism
• Systems-based practice

REQUIREMENTS
• Evaluation must be ongoing – More frequently than annually
• Process must be clearly defined
• What data (“metrics”) will be monitored
• Who will be responsible for review of data
• How often data will be reviewed
• How performance data will be used for decision making
• How data will be archived and made available

POSSIBLE OUTCOMES
• Privilege being evaluated may be continued, limited, or revoked
POSSIBLE ACTIONS
• Continue privilege – no action required
• Suspend or revoke privilege because it’s no longer performed or required
• Initiate FPPE – Due to insufficient activity or identified
“performance issues” with privilege

• WHAT METRICS ARE APPROPRIATE FOR
PATHOLOGISTS?
• Metrics for core vs. delineated privileges
• Core privileges – Privileges broadly applicable to the majority of practitioners in a discipline
• e.g. anatomic pathologists, clinical pathologists, etc.
• Delineated privileges – Privileges specific to a subset of practitioners within a discipline
• e.g. surgical pathologists, cytopathologists, hematopathologists, microbiologists, etc,

• EXAMPLES OF METRICS FOR CORE PRIVILEGES IN AP OR
CP
• Patient care – Turnaround time for key reports
• Medical knowledge – General CME credits, successful MOC
• Practice-based learning and improvement – Participation in
SAMs
• Interpersonal and communication skills – 360° evaluation provided by clinicians, lab staff, etc.
• Professionalism – Attendance at medical staff meetings
• System-based practice – Membership on hospital or other organizational committees

• EXAMPLES OF METRICS FOR DELINIATED PRIVILEGES IN
SPECIFIC AREAS
• Patient care
• Surgical pathology – Turnaround time for biopsy reports
Number or % amended reports
IOC vs. permanent dx discrepancies
• Cytopathology – FNA success rate
• Clinical pathology subspecialties – timeliness of administrative and clinical reports (transfusion reaction)
• All subspecialties – Peer review of new method validations in relevant lab section(s)
• Medical knowledge
• Each subspecialty – CME credits relevant to subspecialty

• DIFFERENCES FROM OPPE
• More restricted in scope – “focused”
• Episodic rather than continuous, with finite endpoint
• SIMILARITIES TO OPPE
• Basic process must be pre-defined and consistent
• Similar monitoring methodologies (review of records, direct observation, monitoring practice techniques, discussion with others involved in care)

• CIRCUMSTANCES WHEN NEEDED
1. When a practitioner is first privileged
2. When new privileges are first granted to an already privileged practitioner
3.
When insufficient activity or “performance issues” are identified for a privileged practitioner