What's New in Anatomic Pathology Quality Assurance

advertisement

What’s new in Anatomic Pathology

Quality Assurance?

Raouf E. Nakhleh, MD

Mayo Clinic Florida

Disclosure Information

Raouf E Nakhleh, MD

• I have no financial relationships to disclose

Objectives

• Discuss evidence based guideline development and their impact on the future from a QA perspective

• Discuss issues related to immunohistochemistry validation

• Discuss ongoing assessment of practicing pathologists

Why these topics

• Not new

• Subject to regulation (USA)

• Formalization of issues not previously addressed

Evidence Based Guidelines

• How does one judge good medical practice?

• Much easier in retrospect, particularly when there is a negative outcome.

• Literature is vast with conflicting knowledge

• In the past, informal evolution of practice

• Emphasis on local standards

• Today national standards

Evidence Based Guidelines

• In particular situation:

• Guidelines provide a road map on how to proceed.

• Guidelines help us judge appropriateness of care

• From a practitioner’s perspective, easier to know if you are meeting your obligations

• From a quality assurance perspective, easier to create measures for comparison

Evidence Based Guidelines – Examples

• In pathology

• Cancer protocols and checklists

• Standard set by physicians

• Reporting standards

• Accreditation by Commission on Cancer

• Clinical Laboratory Improvement Act

• Standard set by legal decree

• Procedural and quality standards

Evidence Based Guidelines

• Cancer reporting

• 1980 report

• Breast, right, mastectomy:

• Infiltrating ductal carcinoma

• 1 of 20 lymph nodes positive

• Margins free

Evidence Based Guidelines

• INVASIVE BREAST CANCER

• Macroscopic

• 1. Specimen (partial breast, total breast including nipple and skin, etc)

• 2. Procedure (excision without wire mastectomy, etc)

‐ guided localization, total

• 3. Lymph Node Sampling (no lymph nodes present, sentinel lymph nodes, etc)

• 4. Specimen Integrity (single intact specimen, multiple designated specimens, etc)

• 5. Specimen Size (greatest dimension in cm)

• 6. Specimen Laterality (right, left, not specified)

• Microscopic

• 7. Tumor Size (size of largest invasive cancer, may also be based on macroscopic)

• 8. Tumor Focality (single focus, multiple foci, etc)

• 9. Skin extent (not present, not involved, invades dermis, etc)*

• 10. Nipple extent (not present, not involved, DICS, Paget’s disease)

• 11. Skeletal muscle extent (not present, present, etc)

• 12. Ductal Carcinoma In Situ (DCIS not present, present, etc)

• 13. Lobular Carcinoma In Situ (LCIS not present, present, etc)

• 14. Histologic Type of Invasive Carcinoma (ductal NOS, mucinous, papillary, etc)

• 15. Nottingham Score Overall Grade (1, 2, 3, not graded)

• 16. Tubular Differentiation (1, 2, 3, not graded)

• 17. Nuclear Pleomorphism (1, 2, 3, not graded)

• 18. Mitotic Count (1, 2, 3, not graded)

• 19. Margins (involved by invasive cancer, involved by DCIS, etc)

• 20. Lymph Nodes ‐ total number examined*

• 21. Lymph Nodes ‐ number with micrometastases*

• 22. Lymph Nodes ‐ number with macrometastases*

• 23. Lymph Nodes 0 size of largest metastatic deposit*

• 24. Tumor pT Stage (pTX, pT0, etc)

• 25. Regional Lymph Nodes pN Stage (pNX, pN0, pN1a, etc)

• 26. Distant Metastasis M Stage (not applicable, pM1, etc)

• Ancillary Studies

• 27. Estrogen Receptor Disposition (performed, not performed, etc)

• 28. Progesterone Receptor Disposition (performed, not performed, etc)

• 29. HER2 Disposition (performed, not performed, etc)

• * If not applicable to the specimen, code as if the element was present in the report

Evidence Based Guidelines – Laws

• Physician Quality Reporting System

• Tax Relief and Health Care Act of 2006

• Incentive pay (up to 2% of eligible Part B) for physician who report quality measures (payfor-reporting)

• Voluntary program

• 2015 becomes mandatory

Physician Quality Reporting System

• Colon Cancer grade and stage (2008)

• Breast Cancer grade and stage (2008)

• Barrett’s esophagus (2012)

• Prostate grade and stage (2012)

• HER2 determination in breast cancer (2012)

Accountable Care Organization (ACO)

• Payment of ACO’s rewards quality and efficiency rather than volume

• Monitor practice patterns and use performance data to improve the quality of care

• CMS establishes performance measures

• If met, then eligible for shared savings

• Over time - higher standards

• Use evidence-based guidelines

Overview of Potential Quality System

Clinical studies

National or local QA review

Evidence based medicine

Practice Guidelines

Inspection, QA review and

PT of an institutions

Improvement

Accreditation

Addition of standards

Evidence Based Guidelines

• What’s in our present and future?

College of American Pathologists

Pathology and Laboratory Quality Center

• The Center develops evidence-based guidelines and consensus statements related to the practice of pathology and laboratory medicine. Through them, we continually improve the quality of diagnostic medicine and patient outcomes.

College of American Pathologists

Pathology and Laboratory Quality Center

• HER2, ER and PR before official Center creation

• CAP/ADASP Consensus Statements on Effective

Communication of Urgent Diagnoses and Significant

Unexpected Diagnoses in Surgical Pathology and

Cytopathology (published)

• Validating Whole Slide Imaging Systems for Diagnostic

Purposes in Digital Pathology

(complete, not published)

• CAP/ASCCP Lower Anogenital Squamous Terminology (LAST)

Standardization Project for HPV-associated Lesions

(complete, not published)

• CAP/IASLC/AMP Molecular Testing Guidelines for Selection of

Lung Cancer Patients for EGFR and ALK Tyrosine Kinase

Inhibitors

(near completion)

Guidelines in the works

• Immunohistochemistry (IHC) Assay Analytic Validation

Principles

• CAP/ASH Algorithm for Initial Work-up of Acute Leukemia

• ASCP/CAP/AMP/ASCO Molecular Markers for the Evaluation of

Colorectal Cancer

• CAP-ADASP Interpretive Diagnostic Error Reduction in

Surgical Pathology and Cytopathology

• Bone Marrow Synoptic Reporting for Hematologic Neoplasms

• CAP-NSH Uniform Labeling Requirements for Blocks and

Slides in Surgical Pathology

• ASCO/CAP Guideline Recommendations for HER2 Testing in

Breast Cancer – Update to 2007 Edition

• ASCO/CAP Guideline Recommendations for HER2 Testing in

Gastric Cancer

Selection Criteria for Center Topics

1.

Patient risk, patient safety and quality; issues that affect patient care and quality outcomes

2. Performance characteristics of assay and ability to reach consensus. Do problems exist including high false positive/false negative rates and lack of uniformity in practice with significant ability to reach consensus?

3. Amount of available evidence to predict a clinical response to an agent or change a patient outcome

4. Guidelines are lacking but regulatory bodies or other professional organizations are trying to preempt specialty development of guidelines or approval/disapproval or show unusual interest

5. Adoption momentum of a particular assay. Consider the number of pathologists affected.

18

Selection Criteria for Center Topics

6. Feasibility of significantly changing practice of pathology or medicine (e.g., using Ultra

Sound Guided FNA to change cytology practice).

7. Ability to collaborate with the right partner(s) in development of a Center product to facilitate development and acceptance, and improve overall probability of success.

8. The public's perception of the issue negatively affects the image of pathologists and the pathologists' role in medicine.

9. Evaluation of existing methods is obsolete or does not bring value in patient care

19

The Center Process for Developing

Guidelines and Consensus Statements

Submit and Select

Ideas

Maintain

Determine Scope and Form

Workgroup

Publish and

Implement

Research and

Review

Review and

Approve

Complete

Recommendations

Solicit Public

Comment

IHC Antibodies and FDA Classification

• Antibodies are medical devices

• Classification base on patient safety

• Class I

• Adjunctive diagnostic information

• Class II

• Prognostic or predictive data

• Separately reported

• More stringent quality assurance

• Class III

• Require FDA premarket approval

HER2 Guideline

• Provide the appropriate HER2 assay validation procedure

• Validation with FISH or with another IHC lab

• Prescribe appropriate fixation time and documentation: 6 – 48 Hours

• Modifies scoring criteria to enhance specificity

• Ongoing competency assessment for pathologists

• Enroll in HER2 proficiency testing service

HER2

• Guideline introduced in late 2006

• Mandated in 2008

• LAP checklist standards related to HER2 effective in late 2008

• CAP sponsored survey to determine how well labs are able to comply (late 2008)

• Leads to insight of effectiveness of guideline

(2010)

LAP HER2 Standards

• Properly validate HER2 assays

• Ensuring appropriate fixation

• use the ASCO/CAP scoring criteria

• Enforce HER2 proficiency testing

2008 HER2 Survey

• Arch Pathol Lab Med 2010;134: 728-734

• HER2 assay validation

• Concordance with FISH

• 81% of Labs achieved 95% concordance for(-)

• 73% of Labs achieved 95% concordance for(+)

• Concordance with another IHC LAB

• 72% of Labs achieved 95% concordance for(-)

• 68% of Labs achieved 95% concordance for(+)

2008 HER2 Survey

• Fixation time and documentation

• 86% made changes to address fixation

• 60% process specimens on the weekend

• Documentation is addressed in reports in 70% of laboratories

2008 HER2 Survey

• ASCO/CAP scoring criteria

• 84% using criteria

• Ongoing pathologists competency assessment

• 91% have a program in place

2008 HER2 Survey – Summary

• Most labs are meeting HER2 guideline

• Gaps still exist, particularly validation

• Deficiencies hopefully corrected over time with inspection cycle

• Follow up survey conducted at the end of 2011, currently under analysis

Validation

• What about other antibodies?

• Gap in understanding of Validation in AP

• Few documents address IHC antibody assay validation

• Definition

• Confirmation through a defined process that a method performs as intended

31

Principles of Test Validation

• Use manufacturer’s instructions

• Use high-quality test materials with known target values

• Test materials should be of similar type as patient samples

• Complete documentation, including procedure and results

• Lab director review/approve results

IHC Validation

• For cell markers, (e.g. keratin, actin, Melan A etc.) how many cases should be performed in validation study?

• Should you validate with another lab or another technique?

• What level of concordance is acceptable?

• Are there situations where validation is not necessary?

• What about cytology material?

IHC Validation – Example

• New antibody introduction – Prognostic

• Example: MIB-1 (Ki-67)

• Proliferation marker (estimate of mitotic count)

• Antibody needs to work at spectrum of findings (0 –

100%)

• Include tissues with various levels of proliferation

• Carcinoid tumors of lung (0-10%)

• Typical carcinoid (0-3%)

• Atypical carcinoid (2-10%)

• Small cell carcinoma (25-90%)

• Validation cases should cover the spectrum of findings

Immunohistochemistry Validation

Procedures and Practice: A College of

American Pathologists Survey of 727

Laboratories

Lindsay Hardy, MD (Boston, MA)

Raouf Nakhleh, MD (Jacksonville, FL) Jeffrey Goldsmith, MD (Boston, MA)

Patrick Fitzgibbons, MD (Fullerton, CA) Richard Eisen, MD (Greenwich, CT)

Mary Beth Beasley, MD (New York, NY) Rhona Souers, MS (Northfield, IL)

Methods

• September 2010: Questionnaire

• 1064 surveys distributed with CAP proficiency test mailing (HER-2)

• 754 returned (Oct. 2010)

• 27 excluded

• 727 included

• 2 major sections

• Non-FDA approved, non-predictive

• Non-FDA approved, predictive other than

HER-2

Results: Non-predictive

Validation Procedures

Laboratory has a procedure for validation of new antibodies?

Procedure specifies the number of cases to be used?

Percent

68

54

Results: Non-predictive

• 68% had a written procedure

• 86% validated the most recently introduced non-predictive marker (despite lack of procedures for some)

Results: Non-predictive

Validation Procedures

# Cases used in most recent validation:

Positive

Negative

2

1

7

5

20

15

Results: Non-predictive

Most recent validation

Weakly or focally positive cases were included

Results confirmed by running parallel at another lab (same tissue)

Cases were tested on multiple days (between-run precision)

Percent

40

18

53

Results: Predictive, excluding HER2

Validation Procedures

Non-

Predictive

Percent Percent

54

Predictive Markers: o ER o PR o ER/PR o CD117/C-KIT o KI-67 o P53 o P63 o PMS2 o EGFR o Other

Results: Predictive, excluding HER2

• 46% had a written procedure

• 75% validated the most recently introduced predictive marker other than HER-2

Results: Predictive, excluding HER2

Validation Procedures

10 th

Pctl

25 th

Pctl

Median

75 th

Pctl

Minimum # cases to validate new

Ab:

Positive 3 5 11 20

Negative

Total Cases

Acceptable concordance (%):

Positive

Negative

1

5

90

50

5

10

90

90

10

25

95

95

15

35

95

95

# Cases used in most recent validation:

Positive

Negative

2

1

5

3

10

5

20

13

90 th

Pctl

25

25

50

99

99

30

20

Revalidation

Validation Procedures

Non-

Predictive

Percent Percent

37

Summary

• IHC validation: 727 labs

• Validation was performed at higher rates than the availability of written procedures

• Labs are generally following procedures

• Most meet designated minimum number of cases

• Trend to slightly fewer

• Guideline is needed to assist in Predictive and non-predictive IHC

• Uncertainty as to the # of cases needed

• Uncertainty as to when to revalidate vs. verification

• Uncertainty RE validation in cytology

Ongoing assessment of practicing pathologists

• Why now?

American Board of Pathologists

• Time limited certification (10 yrs)

• The four part MOC process requires diplomates to submit documentation in the following areas:

• Part I – Professional Standing

• Part II – Lifelong Learning and Self-Assessment

• Part III – Cognitive Expertise

• Part IV – Practice Performance Assessment

Joint Commission Standards

• Ms.08.01.01 Monitoring Performance; The organized medical staff defines the circumstances requiring monitoring and evaluation of a practitioner's performance (FPPE)

• Ms.08.01.03 Use of Monitoring Information: Ongoing professional practice evaluation information is factored into the decision to maintain existing privileges, to revise existing privileges, or to revoke an existing privilege prior to or at the time of renewal

Professional Practice Evaluation

• Use of objective performance data in the granting and/or maintenance of practice privileges

Ongoing Professional Practice Evaluation – OPPE

• On an ongoing basis for privileged practitioners

Focused Professional Practice Evaluation – FPPE

• When a practitioner is first privileged

• When new privileges are first granted to an already privileged practitioner

• When insufficient activity or “performance issues” are identified for a privileged practitioner

Concepts

• Professional competency assessment

• Is part of the peer review process

(protected)

• Is not publicly reported or stored in credentialing files

• Is a tool for evaluation, not an adverse action

• Produces some data points for assessment

• Thresholds are minimum bars, not goals

OPPE – Six Core Competencies

• Patient care

• Medical/Clinical knowledge

• Practice-based learning and improvement

• Interpersonal and communication skills

• Professionalism

• Systems-based practice

OPPE - Ongoing Professional Practice

Evaluation

REQUIREMENTS

• Evaluation must be ongoing – More frequently than annually

• Process must be clearly defined

• What data (“metrics”) will be monitored

• Who will be responsible for review of data

• How often data will be reviewed

• How performance data will be used for decision making

• How data will be archived and made available

OPPE - Ongoing Professional Practice

Evaluation

POSSIBLE OUTCOMES

• Privilege being evaluated may be continued, limited, or revoked

POSSIBLE ACTIONS

• Continue privilege – no action required

• Suspend or revoke privilege because it’s no longer performed or required

• Initiate FPPE – Due to insufficient activity or identified

“performance issues” with privilege

OPPE - Ongoing Professional Practice

Evaluation

• WHAT METRICS ARE APPROPRIATE FOR

PATHOLOGISTS?

• Metrics for core vs. delineated privileges

• Core privileges – Privileges broadly applicable to the majority of practitioners in a discipline

• e.g. anatomic pathologists, clinical pathologists, etc.

• Delineated privileges – Privileges specific to a subset of practitioners within a discipline

• e.g. surgical pathologists, cytopathologists, hematopathologists, microbiologists, etc,

OPPE - Ongoing Professional Practice

Evaluation

• EXAMPLES OF METRICS FOR CORE PRIVILEGES IN AP OR

CP

• Patient care – Turnaround time for key reports

• Medical knowledge – General CME credits, successful MOC

• Practice-based learning and improvement – Participation in

SAMs

• Interpersonal and communication skills – 360° evaluation provided by clinicians, lab staff, etc.

• Professionalism – Attendance at medical staff meetings

• System-based practice – Membership on hospital or other organizational committees

OPPE - Ongoing Professional Practice

Evaluation

• EXAMPLES OF METRICS FOR DELINIATED PRIVILEGES IN

SPECIFIC AREAS

• Patient care

• Surgical pathology – Turnaround time for biopsy reports

Number or % amended reports

IOC vs. permanent dx discrepancies

• Cytopathology – FNA success rate

• Clinical pathology subspecialties – timeliness of administrative and clinical reports (transfusion reaction)

• All subspecialties – Peer review of new method validations in relevant lab section(s)

• Medical knowledge

• Each subspecialty – CME credits relevant to subspecialty

FPPE - Focused Professional Practice

Evaluation

• DIFFERENCES FROM OPPE

• More restricted in scope – “focused”

• Episodic rather than continuous, with finite endpoint

• SIMILARITIES TO OPPE

• Basic process must be pre-defined and consistent

• Similar monitoring methodologies (review of records, direct observation, monitoring practice techniques, discussion with others involved in care)

FPPE - Focused Professional Practice

Evaluation

• CIRCUMSTANCES WHEN NEEDED

1. When a practitioner is first privileged

2. When new privileges are first granted to an already privileged practitioner

3.

When insufficient activity or “performance issues” are identified for a privileged practitioner

Thank You!

Download