I NTERVENTIONS TO REDUCE
INAPPROPRIATE TEST UTILIZATION
Diagnostic Error in Medicine
12 November 2012
Paul L Epner

T RENDS SUGGEST INCREASED DIAGNOSTIC ERRORS

Aging population means more diagnoses

Increasing chronic comorbidities mean increased diagnostic complexity

Decreasing number of primary care physicians combined with emphasis on “cost effectiveness” means less time with patients

Anecdotal evidence of reduced skills in taking history and conducting physical

Diagnosis is an evolving term
2

D EFINITION OF DIAGNOSIS IS EXPANDING



The cause of symptoms (traditional)
The condition’s subtype (for best treatment)


Antimicrobial susceptibility testing
Tumor typing
The body’s likely response to treatments

The stratification of risk
3

T HE ROLE OF LABORATORY TESTING IN DIAGNOSIS IS
LIMITED BUT IMPORTANT AND LIKELY INCREASING



In a study of 248 hospitalized patients, 246 had definitive diagnosis within 3 months of hospitalization.
The primary determinant of diagnosis for 215 with
“exact” in-hospital diagnosis was:



History and Physical – 48.4%
Radiologic exam – 33.5%
Blood test or culture – 9.8%
Study limitations


 did not examine diagnostic error did not examine time to diagnosis did not examine appropriate use of diagnostic tools
4
Source: Wahner-Roedler, D. L.et al. (2007). Who makes the diagnosis? The role of clinical skills and diagnostic test results. Journal of evaluation in clinical practice , 13 (3)

O LDER STUDIES YIELD COMPARABLE RESULTS




80 prospective outpatient cases
Final diagnosis made



Following history - 61 (76%)
Following physical – 10 (12%)
Following laboratory – 9 (11%)
Confidence in diagnosis rose with more information

Following history – 7.1 (scale of 1 to 10)


Following physical – 8.2
Following laboratory – 9.3
Some evidence that skill in conducting history and physical is decreasing while reliance on data is increasing
5
Source: M.C. Peterson, J.H. Holbrook, D. Von Hales, N.L. Smith, and L.V. Staker, “Contributions of the history, physical examination, and laboratory investigation in making medical diagnoses.,” The Western journal of medicine, vol. 156, Feb. 1992.

T HE ROLE OF TESTING IN DIAGNOSTIC E RRORS IS
SIGNIFICANT
N= 583 Cases 6
G. D. Schiff et al., “Diagnostic error in medicine: analysis of 583 physician-reported errors.,” Archives of internal medicine , vol. 169, no. 20, pp. 1881-7, Nov. 2009.

U.S. MALPRACTICE CASES CONFIRM SIGNIFICANCE
Of 307 closed cases (ambulatory) studied because they alleged missed or delayed diagnosis, 181 did involve diagnostic errors that harmed patients
Source: T. GANDHI, A. KACHALIA, E.J. Thomas, A.L. Puopolo, C. Yoon, T. Brennan, and D. Studdert, “Missed and delayed diagnoses in the ambulatory setting: A study of closed malpractice claims.,” Annals of internal medicine, vol. 145, 2006.
7

T RADITIONAL LABORATORY QUALITY MEASURES ARE NOT
SPECIFIC FOR PATIENT HARM OR DIAGNOSTIC ERRORS




Prolonged turn-around time
Error logs

Missing ID, Hemolysis, Short fills, Interface error logs,
Incomplete requisitions, uncollected samples, order entry errors, lost specimens, contaminated specimens
Incident reports
Corrected result reports
8

A FRAMEWORK FOR LABORATORY RELATED DIAGNOSTIC
ERRORS HAS BEEN DEFINED *


Inappropriate test is ordered
Appropriate test is not ordered



Appropriate test result utilization is delayed
Appropriate test result is not properly utilized




Knowledge deficit
Failure of synthesis
Misleading result
Systematic failure
Appropriate test result is wrong
9
*Adapted from P Epner and M Astion, “Focusing on Test Ordering Practices to Cut
Diagnostic Errors,” Clinical Laboratory News, vol. 38, no. 7, July 2012

T HE FRAMEWORK GUIDES INTERVENTIONS

Inappropriate test ordered or appropriate test not ordered






CPOE design and monitoring
Algorithms, clinical pathways, guidelines
Reflex testing
Data mining
Inter-physician variance analysis
Resource utilization committee
10

T HE FRAMEWORK GUIDES INTERVENTIONS



Test result not utilized properly or fully



Interpretive comments
EMR interface
Real-time triggers
Test result delayed or not retrieved

Process monitor

Discharge monitor
Appropriate test result is wrong

Delta checks



Controls/Calibrations
Autoverification
Second read (AP)
11

R EQUISITION DESIGN



Design changes focused on medical necessity, reduction in panels, test groupings linked to specialty, etc.
Reduction in tests per visit occurred
No assessment of impact on Dx errors was made
Source: J.F. Emerson and S.S. Emerson, “The impact of requisition design on laboratory utilization,” American Journal of Clinical Pathology , vol. 116, Dec. 2001.
12

C LINICAL DECISION SUPPORT / BEST PRACTICE ALERTS
Source: Jones, Jay, “Lab Enterprise Analytics,” Executive War College 2009
13

D IAGNOSTIC ALGORITHMS
Clinical variables drive six distinct but potentially overlapping algorithms for prolonged PTT






Evaluation preoperatively of an asymptomatic prolonged PTT
Evaluation of a persistently prolonged PTT with bleeding
Evaluation of a persistently prolonged PTT without bleeding
Evaluation of an elderly patient without bleeding history accompanied by sudden development of soft tissue hematomas and/or persistent and significant gastrointestinal or genitourinary hemorrhage
Evaluation of hospitalized newborn with prolonged PTT
Evaluation of a unexplained prolonged PTT following multiple, appropriate workups; searching for rare diagnoses
14
Source: Tcherniantchouk, O., Laposata, M., & Marques, M. B. (2012). The isolated prolonged PTT. American journal of hematology .

Developed by the Centers for Disease Control with the support of the
Algorithm Subgroup of CLIHC™
15

16

17

R EFLEX AND REFLECTIVE TESTING



Creating protocols for the sequential addition of tests based on earlier results reduces diagnostic delays and patient inconvenience while reducing test volume
Reflex testing can improve diagnostic accuracy
The improvement in diagnostic accuracy is linked to the threshold criteria and varies with the clinical scenario
Source: R. Srivastava, W. a Bartlett, I.M. Kennedy, A. Hiney, C. Fletcher, and M.J.
Murphy, “Reflex and reflective testing: efficiency and effectiveness of adding on laboratory tests.,” Annals of clinical biochemistry , vol. 47, May. 2010.
18

D ATA MINING





Data mining is the process of nontrivial extraction of implicit, previously unknown and potentially useful information from data stored in repositories.
1
Strategies can be driven by published guidelines
Retrospective study 2 of more than 450,000 HPV tests against new guideline published in 2004


HPV testing is contraindicated in women under age 21
HPV testing is contraindicated without positive cytology.
Study showed multi-year improvements in compliance
Data mining is a tool that identifies opportunities for education or other interventions
1 Lee, S.J. and Siau,K., “A review of data mining techniques,”
Systems, Vol. 101, January 2001.
Industrial Management & Data
2 B.H. Shirts and B.R. Jackson, “Informatics methods for laboratory evaluation of HPV ordering patterns with an example from a nationwide sample in the United States, 2003-2009.,” Journal of pathology informatics , vol. 1, Jan. 2010.
19

P HYSICIAN LEVEL PERFORMANCE FEEDBACK



When physicians are given feedback on their test ordering patterns compared to colleagues or guidelines, test ordering behavior changes.
In one study 1 , clinicians were educated about the laboratory tests needed to monitor patients on antihypertensive medication. Additionally, they were given feedback on their testing patterns. Appropriate testing improved.
In another study 2 , quarterly feedback of practice requesting rates for nine laboratory tests, enhanced with educational messages were provided to primary care physicians which proved to be an effective strategy for reducing inappropriate testing
1 Lafata, J.E. et al, “Academic detailing to improve laboratory testing among outpatient medication
2 users.,” Medical care , vol. 45, Oct. 2007.
Thomas, R.E. et al, “Effect of enhanced feedback and brief educational reminder messages on laboratory test requesting in primary care: a cluster randomised trial.,” Lancet , vol. 367, Jun. 2006.
20

R ESOURCE UTILIZATION COMMITTEE


Typically involves locally driven consensus
One study is noteworthy for assessment of patient impact.*
*Neilson, E. G., Johnson, K. B., Rosenbloom, S. T., Dupont, W. D., Talbert, D., Giuse, D. A., Kaiser, A., et al. (2004).
The impact of peer management on test-ordering behavior. Annals of internal medicine, 141(3), 196–204.
21

F OCUS ON SYSTEMATIC ERROR REDUCTION


Many laboratory professionals routinely drive initiatives to reduce systematic errors.


Tools in use




Lean
6 Sigma
Root Cause Analysis
Failure Mode & Effect Analysis
Bias in problem selection may exist


Within the laboratory walls
Within the control or shared control of the laboratory
Evidence for the use of these tools to eliminate
is difficult to find
22

I NTERPRETIVE COMMENTS


Criteria for providing interpretive comments have been described *

 a decision on treatment is indicated by the results in combination with the clinical details provided a result is unexpected

 a specific question has been posed but it is not obvious whether the results provide the answer a clinician has requested a test with which he/she is not likely to be familiar
Areas where Interpretive reports are most relevant
* E. Piva and M. Plebani, “Interpretative reports and critical values.,” international journal of clinical chemistry , vol. 404, 2009.
Clinica chimica acta;
23

D IAGNOSTIC MANAGEMENT TEAMS AT V ANDERBILT
ENSURE APPROPRIATE CONSULTATIVE SERVICES
24

P ENDING LAB RESULTS : PROCESS MONITORING

Shifts the focus from catching failures e.g., clinical event monitors to workflow process control

Some efforts are ongoing: MSTART (Multi-Step Task
Alerting, Reminding, and Tracking)
*Tarkan, S., Plaisant, C., Shneiderman, B., & Hettinger, A. (2010). Improving Timely Clinical
Lab Test Result Management: A Generative XML Process Model to Support Medical Care.
25

P ENDING LAB RESULTS : DISCHARGE MONITOR




Several attempts to create automated tools have been tried with limited success
Positive results were obtained with a system of email notifications 1
A computer-based antimicrobial monitoring (CBAM) system has been used to ensure positive microbiology cultures receive attention with improved outcomes 2
Discharge systems need to alert both hospital-based and primary care physician
1 Dalal, A. K., Schnipper, J. L., Poon, E. G., Williams, D. H., Rossi-Roh, K., Macleay, A., Liang, C. L., et al.
(2012). Design and implementation of an automated email notification system for results of tests pending at discharge. Journal of the American Medical Informatics Association : JAMIA , 19 (4), 523–8.
2 Wilson, J. W., Marshall, W. F., & Estes, L. L. (2011). Detecting delayed microbiology results after hospital discharge: improving patient safety through an automated medical informatics tool. Mayo Clinic proceedings.
Mayo Clinic, 86(12), 1181–5. doi:10.4065/mcp.2011.0415
26

T OOLS EXIST ; PROVING VALUE IS MORE DIFFICULT

Robust research on the role of laboratory services does not exist

Research on the effectiveness of available tools is limited
27

I MPROVEMENTS IN TEST SELECTION AND RESULTS
INTERPRETATION (ITSRI) – A R ESEARCH A GENDA
Strategic Intent


Establish empirically the optimum role for laboratory medicine’s physicians and scientists to maximize positive patient outcomes


Appropriate testing
Appropriate interpretation
Identify evidence-based interventions that support the optimum role
28

ITSRI S TATUS




Diagnostic Process Variation

Chief complaint specific

Diagnosis specific

Test domain specific
Intervention effectiveness





NorthShore University HealthSystem
Virginia Commonwealth University
Kaiser Permanente
29

O THER EFFORTS ONGOING


Diagnostic errors and the clinical laboratory

AHRQ ACTION II

CLIHC™
Significant challenges remain


Lack of funding and resources
Shifting the focus from laboratory costs
30

AHRQ FUNDED
RESEARCH


Awarded to RTI in
August, 2011; 18 month effort
Developing risk assessment tools which will be tested in three sites:



Vanderbilt
Emory
Seattle Children’s
31

R EFERRAL L ABORATORY R ISK A SSESSMENT
32

I DENTIFICATION AND P RIORITIZATION OF R ISK
33

C LINICAL L ABORATORY INTEGRATION INTO
HEALTHCARE COLLABORATIVE – CLIHC™



CDC sponsored
Seeking to break down the barriers between care providers and laboratory professionals
Key initiatives are moving forward






A survey of medical schools to understand curricular changes since 1992 involving laboratory medicine
A survey of pathology residency programs quantifying time spent teaching consultation
A survey of primary care clinicians to quantify the barriers to appropriate laboratory utilization
An initiative to define nomenclature issues and investigate technology strategies for addressing them
An initiative that will develop and publish algorithms to guide clinicians in the use of complex tests (with iPhone app)
An initiative that seeks to experimentally determine the effectiveness of laboratory interventions on diagnostic error reduction (ITSRI)
34

K EY MESSAGES




Diagnostic error is a major patient safety problem
The total testing process is a significant source of diagnostic errors
Laboratory-directed interventions are available and can be effective in reducing errors
Laboratory physicians and scientists will realize other benefits from leading collaborative efforts

Improve patient outcomes



Strengthen relationships with clinicians
Reduce the level of risk in the health system
Become indispensable stewards of clinical data
35

F
T
:
HE CLINICAL LAB
S MISSION SHOULD NOT BE
A LTHOUGH NECESSARY , IT IS NOT SUFFICIENT
HE CLINICAL LAB
S MISSION SHOULD BE
* Epner, Paul, “Impact of Laboratory Services on Diagnostic Errors,” ThinkLab ‘11
36