RTI International
Health IT Safety Webinar Series
How Can We Improve Diagnosis and
Safety Using Health IT?
May 8, 2015
1:00-2:30pm EDT
RTI International is a trade name of Research Triangle Institute.
1
www.rti.org
RTI International
Housekeeping





2
For any technical questions, please type your question into the
Questions panel at lower right.
All telephone lines are muted. Due to the number of attendees,
please use the Questions panel to ask any questions during the
webinar.
Closed captioning for today’s session is available at
http://www.captionedtext.com/ Event #: 2618527
Q&A will take place at the conclusion of each presentation. Slides
and a copy of the recording of this session will be posted at
www.healthitsafety.org
For general questions about the webinar series, please contact
healthitsafety@rti.org
RTI International
Health IT Safety Webinar Series
This series of 10 webinars focused on health IT and patient
safety issues will occur monthly through September 2015.
These webinars are funded by the Office of the National
Coordinator for Health Information Technology (ONC) and
are being conducted by RTI International, a non-profit
research organization, as part of a year-long project to
develop a road map for a Health IT Safety Center for ONC
(contract HHSP23320095651WC).
Additional information is available at:
www.healthitsafety.org
3
RTI International
Today’s Presentations
Jason Maude, Founder and CEO, Isabel Healthcare
Hardeep Singh, MD, MPH, Chief, Health Policy, Quality &
Informatics, Michael E. DeBakey VA Medical Center and
Baylor College of Medicine, Houston TX
Michael Kanter, MD, Regional Medical Director of Quality and
Clinical Analysis at Southern California Permanente Medical
Group
Mark Graber, MD, RTI International (Moderator)
4
RTI International
Today’s Moderator

5
Mark Graber, a senior fellow in the Health Care Quality and
Outcomes Program, joined RTI International in 2011, bringing more
than 30 years of experience in academic medicine and health care
research with a focus on quality of care and patient safety. Dr.
Graber is an internationally recognized authority on diagnostic error
in medicine and has published extensively in this area. He founded
and chaired the first two Diagnostic Error in Medicine conferences in
2008 and 2009 and founded the Society to Improve Diagnosis in
Medicine in 2011.
The EHR - Enabling Diagnosis
 Collects & organizes all the relevant information
 Clear, legible record of notes, diagnostic tests and
consults; Help organize thinking, planning, &
follow-up
 Enables communication & care coordination
 Enables telehealth, research, predictive analytics
 Enables patient engagement (Open Records)
6
 Using health information technology to reduce
diagnostic errors. El Kareh, R; Hasan, O; Schiff, GD.
BMJ Quality Safety 2013
doi:10.1136/bmjqs-2013-001884
 Can electronic clinical documentation help prevent
diagnostic errors? Gordon D. Schiff; David W.
Bates; NEJM 2010; 362: 1066-9
7
How Health IT Improves
Diagnostic Quality and Safety
 Decision support for DDX (DXplain, Isabel, VisualDX)
Jason Maude, Founder of Isabel
 Trigger Tools -- Helps identify patients at risk for
diagnostic error (revisits, admissions after a clinic
appointment)
Hardeep Singh – Baylor and Houston VAMC
 Safety Nets - Catching errors before there is harm
Michael Kanter - Kaiser Permanente
8
RTI International
Today’s Speakers

Jason Maude serves as Chief Executive Officer and Founder of
Isabel Healthcare Ltd, a company founded 10 years ago to
develop tools to help physicians and patients improve diagnosis
decision making after his daughter was badly misdiagnosed. Prior
to founding Isabel Healthcare Ltd, Maude spent 12 years working
in the finance industry in London at a number of top-ranked
investment banks.
Presentation:
“Health IT to Enable Differential Diagnosis”
9
Isabel Healthcare
“The right diagnosis and
treatment-quicker”
May 2015
Jason Maude
CEO & Founder Isabel Healthcare
10
How it began - a nearly fatal misdiagnosis in 1999
Isabel Maude badly
misdiagnosed as a
toddler
Isabel application
inspired by this critical
need to get the
diagnosis right the
first time.
11
Isabel Maude survived ~ and thrives today
12
Who is Isabel Healthcare?
 Isabel Medical Charity established 1999
 Isabel Healthcare Ltd established 2004
 Isabel Healthcare Ltd owned by charity (37%)
and external investors (2 angels and 1
corporate)
 Established leader in diagnosis decision
support systems for clinicians and patients
 Used by large health systems, hospitals,
practices, individual clinicians and patients
13
Recognizing disease in time
DX
before
=
DX
after
=
14
Where do DX support tools fit?
Even more specialists
More specialists
10,000 to 12,000
diseases
Specialists
I just have
these
symptoms
Emergency
care
PCP
200-300 diseases
DX tools enable patients to be better informed about their
symptoms and diagnosis and doctors to manage and refer more
appropriately.
15
Brief history of DX support tools
 1st generation started appearing in 1970’s
 ‘Artificial Intelligence’ (AI) or ‘Expert Systems’: if, then = DXplain,
QMR, Internist
 New technology enabling 2nd generation tools from 2000 - Isabel
 Statistical Natural Language Processing (SNLP) = pattern matching
 Natural Language Processing (NLP) = what does it mean
 3rd generation tools use NLP+SNLP = Isabel Active Intelligence/IBM
Watson
16
Isabel Performs 2 Essential Functions
1.
Hypothesis generation
2.
Fast gateway (with no additional logins) to Clinical Reference
Resources
•
Evidence Based content from Dynamed/ BMJ’s Best
Practice and various external web based resources
•
Point of Care information on key symptoms, 1st and 2nd line
tests and treatment guidelines.
•
Draws clinicians attention to knowledge when needed
17
When do
Isabel?
Why wait, clinicians
useyou
Isabel
for…
What we do
 Patients with whom there is a
diagnostic doubt
 Referral for diagnosis
 Appropriate tests to order
 Latest treatment guidelines
 Training/Education
18
Long validation process- over 30 published articles
• Study showed 22% increase in diagnostic skills in 4th Year
Medical Students
22% Skills
Improvement
29% of Doctors
Reduced
Referrals
36% Redirected
Referrals
• Study in UK primary care showed that after using Isabel,
in 29% of cases GP did not need to refer and 36% of cases
helped refer more appropriately
• Recent systematic review by NIHR at Univ of Manchester
highlighted Isabel as most accurate tool and concluded
that “DDX tools are potentially an efficacious way of
reducing or preventing diagnostic errors”
19
Demonstration-one portal for all questions
20
Demo: Get Checklist
21
Demo: Dynamed
22
3rd generation dx support within Cerner
Differential Diagnoses within ‘Smart Zone’
Implementation…….the hard bit!
1. Must be part of institutional strategy
2. Clinical Leadership
3. A passionate champion
4. Made part of routine care
5. Continual awareness
DX support is the means not the end
25
Every month 80% of population report symptoms
26
Google
•
100 billion searches made
through Google per month
•
5% or 5 billion health
searches made per month
27
For patient, Isabel performs 3 essential functions
1.
Enables them to build own differential diagnosis and become
informed - suggest that doctor thinks about a disease
2.
Links to relevant and credible patient information
3.
Helps them decide where to present
28
Hospital branded symptom checker
29
Hospital specific triage and venues
30
Engaging patient before visit
Providing information previsit can improve efficiency
and manage expectations
•
•
•
Data collected from symptom
checker
• Gets patient to think about
their symptoms prior to visit
• Allows patient to add other
comments
Customized questions to collect
additional info that can assist during
the visit
Info can be sent to EMR, emailed to
practice/physician pre-visit
31
Myth vs Our Experience
 What most think
 Consumers armed with material found on the internet
are more difficult to deal with
 Most searches are done with one symptom
 Consumers get spooked when presented with conditions
that are “scary”
 What we know
 Only 12-13% of searches on our public site consist of
single symptoms
 Not a single complaint from organizations using the tool
for engagement about difficult patients
 Almost 100% positive feedback from consumers!
32
Recognising disease in time - using DX support!
DX
before
=
DX
after
=
33
RTI International
Today’s Speakers

Dr. Singh is Chief of the Health Policy, Quality & Informatics research
program at the Houston VA Center for Innovations in Quality,
Effectiveness and Safety, and Associate Professor of Medicine at
Baylor College of Medicine. He conducts multidisciplinary research
on diagnostic errors, and on using health information technology to
deliver safe and effective patient care. He co-developed the "ONC
SAFER Guides" for safe and effective electronic health record use
under a contract with Office of the National Coordinator for Health IT.
He is a Fellow of the American College of Medical Informatics
(ACMI) and received the 2012 Presidential Early Career Award for
Scientists and Engineers from The White House.
Presentation:
“Measuring Missed Opportunities in Diagnosis in Health IT-Enabled
Health Systems.”
34
MEASURING MISSED OPPORTUNITIES
IN DIAGNOSIS IN HEALTH IT-ENABLED
HEALTH SYSTEMS
HARDEEP SINGH, MD, MPH
HOUSTON VETERANS AFFAIRS CENTER FOR INNOVATIONS IN QUALITY,
EFFECTIVENESS & SAFETY
MICHAEL E. DEBAKEY VA MEDICAL CENTER
BAYLOR COLLEGE OF MEDICINE
35
Objectives
36

In context of health IT-enabled
health systems:


Discuss lessons learned in measurement
of missed opportunities in diagnosis
Discuss how we can make progress in
reducing them
Key to Reducing Misdiagnosis
37
We cannot improve what we cannot measure
And
We cannot measure what we cannot define!
Early Work
38


Evaluated evidence of ‘errors’ in an integrated
system
Detailed review of comprehensive EHR to
evaluate diagnostic process in the patient’s
journey across the continuum of care

Data available from primary care,
specialty (secondary) care, ER, hospital,
diagnostics (lab/imaging/pathology),
procedures
39
39
High Level Findings
40



Common conditions missed in outpatient
settings despite clear red-flags
(1 in 20 US adults/year)
Abnormal test results missed about
8-10% of the time
Better understanding of where to focus
Singh et al BMJQS 2014; Singh et al Am J Med 2010 & Singh et al Archives of Int Med 2009
Defining Preventable Diagnostic Harm
41
MISSED
OPPORTUNITIES
A
Missed
opportunities in
diagnosis due to
system and/or
cognitive factors
HARM
(from delayed or
wrong treatment/test)
B
Preventable
diagnostic harm
C
NO MISSED
OPPORTUNITIES
D
Delayed/wrong
diagnosis
associated with
patient harm but no
clear evidence of
missed
opportunities
Delayed/wrong
diagnosis but no
clear evidence of
missed
opportunities
Adapted from Singh Jt Comm J Qual Patient Saf 2014
Intersection of Health IT & Diagnostic Safety
42


Our goals are to use health IT to
measure and reduce diagnostic errors
and harm, but ..
Current Reality: Trying to ensure health
IT itself is being used ‘safely’
Adapted from Sittig & Singh N Engl J Med. 2012
43
Diagnostic Safety in the EHR Era
44

EHRs must support provider cognition and
decision-making

What else besides innovation getting in the way?


Use of templates and documentation issues

Reduced physician efficiency

Data display issues leading to ambiguity

“iPatient” and effect on critical thinking skills?
E-Communication breakdowns persist
Sittig et al JAMIA 2015
45
And More Digital Data Is on the Way
46


Smartphone

Wearables
“Patients can now continuously monitor their
data real-time and send it to their docs”
Lessons from Research
47

EHR-based systems better than paper

Not achieving full potential

Need sociotechnical approaches to
improve diagnostic safety
Sittig and Singh JGIM 2012; Sittig and Singh QSHC 2010
8-dimensional Socio-Technical Model of
Safe & Effective Health IT Use
48
Workflow & Communication
External Rules & Regulations
Organizational Policies,
Procedures, & Culture
Content
Personnel
Hardware &
Software
Measurement
& Monitoring
Sittig and Singh QSHC 2010
To Enable Rigorous Measurement
49

Missed opportunity measurement must
reflect real-world practice

more than just what’s in “the doctors head”

systems, team members, and patients, all
inevitably influence clinicians’ thought
processes
Singh BMJQS 2013
50
Safer Dx Framework for
Measurement & Reduction
Singh & Sittig BMJQS 2015
50
‘Triggers’ Facilitate Measurement &
Reduction
51
Patient-Primary Care
Provider Encounter
Diagnostic Tests
Trigger
More Patient-Provider
Encounters
Trigger
Consultations to
Sub-specialists
Trigger
Correct Diagnosis
Certain Diagnosis
Uncertain Diagnosis
Singh & Thomas AHRQ Special Report 2009
Why Triggers Are a First Step?
52

Algorithms to select high-risk patient
records for further reviews to look for
missed opportunities


Picking up ‘needles in a haystack’ by
making the haystack smaller
Application retrospective or prospective
surveillance
Singh et al JAMA IM 2013
Electronic health record-based triggers to detect potential
delays in cancer diagnosis
Daniel R Murphy,1,2 Archana Laxmisan,1,2 Brian A Reis,1,2 Eric J Thomas,3 Adol Esquivel,4 Samuel N Forjuoh,5 Rohan
Parikh,6 Myrna M Khan,1,2 Hardeep Singh1,2
53
Time for Surveillance & Safety Net?
54

Creating ‘intelligence’ related to diagnostic
safety needs resources and time investment




Institutions/practices have many competing priorities
Will it give bang for the buck outside of research?
Contacting docs
Unintended consequences need to be monitored

More (or unnecessary) testing/treatment could occur
Zwann and Singh Diagnosis 2015
55
The Checklist is
structured as a quick
way to enter and print
your self-assessment.
Your selections on the
checklist will
automatically update
the related section
of the corresponding
recommended practice
worksheet.
56
SAFER Guides ; Sittig, Ash, Singh. Am J Manag Care. 2014
The Worksheet provides
guidance on
implementing the
Practice.
Take Home Messages
57


Health IT needs to be leveraged to start
exploring, understanding and possibly
reducing missed opportunities in diagnosis
Plenty of tools and strategies available to
start addressing diagnostic safety related
to missed follow-up issues
Thank you…
58
Funding Agencies





Department of Veterans Affairs
Agency for Health Care Research & Quality
National Institute of Health
Office of National Coordinator (SAFER Guides)
Multidisciplinary team at Houston-based
VA Health Services Research Center of Innovation
Contact Information…
Hardeep Singh, MD, MPH
hardeeps@bcm.edu
@HardeepSinghMD
RTI International
Today’s Speakers

Dr. Kanter provides oversight of quality activities to improve care for
the 3.5 million members in the Southern California Kaiser
Permanente Region, including HEDIS results, JCAHO core
measures, Leapfrog survey results, peer review, and hospital quality
and performance improvement. His interests include patient safety,
risk management, population care management, technology
assessment and deployment of new technology, creation and
implementation of evidence-based guidelines, laboratory utilization
management, and analysis of outpatient access and patient
satisfaction data.
Presentation:
“Using Health IT as a Safety Net for Diagnostic Errors.”
59
USING HEALTH IT AS A SAFETY
NET FOR DIAGNOSTIC ERRORS
MICHAEL H KANTER MD
MEDICAL DIRECTOR FOR QUALITY AND CLINICAL ANALYSIS
SOUTHERN CALIFORNIA PERMANENTE MEDICAL GROUP
60
Types of Diagnostic Errors
Availability
heuristic
Anchoring
heuristic
Failure to
follow up
abnormal
tests
Information
overload
Visual
diagnostic
errors
Diagnostic
Errors
Framing
effects
Blind
obedience
Premature
closure
61
Kaiser Permanente SCAL SureNet
A regional program that systematically identifies
patients who have inadvertent lapses in care
Using a small, centralized team with limited
clinical scope capacity to intervene before harm
reaches the patient
As well as several automated electronic tools,
consistently used by accountable frontline staff,
to track certain abnormal results for all members
62
Current Portfolio
Diagnosis Detection/ Follow Up
• PSA Electronic Safety Net
• +FIT Electronic Safety Net
• Abnormal Pap Electronic Safety Net
• Kidney Disease (Repeat Creatinine)
• High Risk CKD Follow-up
• Colon Cancer (Iron Deficiency Anemia/Rectal
Bleeding + No colonoscopy)
• Maternal Hepatitis B
• Abdominal Aortic Aneurysm Tracking
• Post Splenectomy Immunizations
• Positive Chlamydia Follow up
• Down Syndrome Care Coordination
• Sickle Cell Care Coordination
• Hepatitis C (+Antibody + No confirmatory test )
• Newborn Hearing Screening
• Lung Nodules
• Unintended Pregnancy Follow up
Medication Safety
• Annual Lab Monitoring: Digoxin (K+, level and
SCr), Diuretics (K+ and SCr)
• Amiodarone (Preventive monitoring plan)
• Acetaminophen Overuse
• Elderly Care Drug-Disease (Falls)
• Elderly Care Drug-Disease (Dementia)
• Elderly Care High Dose Digoxin Conversion
• Interacting Statin Combinations (Gemfibrozil
and/or Amiodarone)
Counseling
/
• Diuretic MedicationPost-TAB
Induced
Hyponatremia
Birth Control
• Medication Induced Hyperkalemia
• NSAIDs in CKD 4-5, Dialysis, Kidney Transplant
• INH ALT monitoring
• Monitoring Plaquenil Eye Monitoring
• Metformin B12 monitoring
• Ethambutol eye monitoring
63
PSA Safety Net
• Identify cases of elevated PSA
• Remove those with a diagnosis of prostate cancer
• Remove those with a follow up appointment in urology, oncology, or radiation
therapy
• Remove those in hospice or palliative care
• Remaining cases require chart review using existing prostate cancer case
managers
64
PSA SureNet: Results
Three Year Period
8,076 patients for case
management
3,833 patients received
Urology appointments
2,204 patients underwent
prostate biopsy
745 Prostate Cancers
diagnosed
Zero claims related to
missed abnormal PSA’s
•
65
CKD Diagnosis Detection:
Unrepeated Creatinine
12,396 lab orders placed for patients
with an abnormal creatinine not
repeated after 90 days
6,981 total labs repeated within 90
days (56.3%)
3,668 (52%) New CKDs identified
•
66
Diagnosis Detection and Follow Up :
Rectal Bleeding w/o Colonoscopy
Chart review revealed delays in
diagnosis of colorectal cancer due to
lack of follow up on patients with a
history of rectal bleeding with
endoscopy
•
•
•
•
M/W 55-75 (not terminally ill)
No colonoscopy in past 10 years
GFR > 60
History of rectal bleeding
• Outpatient encounter with ICD9 diagnosis code of 569.3x
or 455.xx
67
Diagnosis Detection and Follow Up :
Rectal Bleeding w/o Colonoscopy
168 patients with a history of rectal bleeding and no
colonoscopy identified
26 completed colonoscopy
6 had normal results
1 patient had an adenocarcinoma with spread to
local lymph node, 1 had a carcinoid tumor
7 had one or more tubular adenomas, 11 had one
or more hyperplastic polyps, hemorrhoids or colitis
68
Why are abnormal results not followed up?
• Huge volume of testing requires > 6 sigma level of reliability
• Formatting of reporting results may not be conducive to seeing abnormalities
• Lack of systems to manage abnormal results
• The patient is a critical part of the system
• EMR does not solve the problem
• Handoffs between physicians and/or providers
• Problem requires future study.
69
IT Issues to Consider
• Use discrete data whenever possible while avoiding free text comments
• Understand what data you have access to
•
•
•
•
•
•
•
•
Claims
Laboratory
Radiology
Pharmacy
Office visits
Hospitalizations
Diagnosis/diagnostic codes
Procedure codes
• Understand where you can get help – physician office staff, case managers,
etc.
70
Process for New Electronic Clinical Surveillance
Programs – Kaiser Permanente SureNet
In addition to Subject Matter Experts, SMES
panel always includes:
• Leadership
• Clinical guidelines
•
71
Criteria for Evaluating Electronic Clinical
Surveillance Programs
• CLINICAL IMPACT—Will it improve patient safety or quality?
• If yes,
• How many people will be affected?
• What is the severity of the potential safety gap?
• IDENTIFICATION—Is the potential safety gap readily identifiable using
existing electronic health data? That is:
• Is the information required discretely coded?
• If the information required is not discretely coded, is it noted consistently in a way
that makes natural language processing a reasonable option?
• If the potential safety problem is not readily identifiable, is it important enough
that revisions to existing documentation should be considered?
•
72
Important Cultural Issues for
Diagnostic Errors
• Blame-free culture
• Feedback to physicians needs to be such that the physician will stay
engaged with closing the loop on the diagnostic error (inform the
patient and provide appropriate follow up care)
• Diagnostic error continues until the correct diagnosis
is made
• Encourage reporting errors and near misses by rapidly building
new safety nets based on these reports
• Transparency is required
• Initial launch of new program may identify larger prevalence of cases
of errors even if the annual incidence is not so great
73
Important Practical Issues for SureNet
Program Success
• SureNet programs should have small impact on the physicians practice
• Need to balance the false positive notification of a possible error against the
sensitivity of the net but for the most part favor specificity over sensitivity
• Do as much as possible for the physicians, which may include notifying the patient
to get repeat labs or other studies
• Get buy in to new programs prior to implementation
• Use physician champions to help promote programs
74
Justifying the Cost
of a SureNet Program
• Create some that will decrease malpractice litigation
• Minimize manual review of records which is costly
• If possible measure errors identified
• allows for assessing if program is achieving anything
• allows for improvement over time and removal of waste
•
75
Problems Likely to Occur in a
SureNet Program
• Angry Patients
• May discover errors many years old in HCV and creatinine safety nets
• May discover delay in diagnosis of cancer
• Angry Physicians
• May be upset if they are notified of a potential error and, in fact, no such error occurred
• May be upset if the patient gets upset and blames the physician
• Upset staff managing the program
• Conversations with upset patients or physicians are difficult
• Ethical issues may arise on when to notify patients of bad outcomes vs. risk of getting sued
•
76
Summary
• Harms from failure to follow up on abnormal results can be
mitigated using Health IT
• Every delivery system will have a somewhat different approach
• It is not necessary to know the baseline rate of errors
• Cultural issues need to be addressed
77
RTI International
Questions and Wrap Up
Speaker Contact
Information
Next Webinar – planned for

Jason Maude
Jason.Maude@isabelhealthcare.com
Hardeep Singh
hardeeps@bcm.edu
Michael Kanter
Michael.H.Kanter@kp.org
78
EHR Usability and Health IT
Safety
June 18, 2015
1-2:30p EDT

Please visit:
www.healthitsafety.org or
contact healthitsafety@rti.org for
more information on the entire
webinar series