Mayo Clinic – Rochester, MN
Time to Reap: Improving Quality
by Harvesting Data from the EMR
Marie J. Eidem,
Eidem, BS
Aaron A. Kurtzhals, BS
James M. Naessens,
Naessens, MPH
Mayo Integrated Clinical Systems
(MICS)
Measuring Quality
• External
• JCAHO
• CMS
• Leapfrog
• National Quality Forum
• Blue Cross Blue Shield
• Minnesota Community Measurement
• Institute for Healthcare Improvement 100K Lives . . .
• Internal
• Continuous Improvement teams
• Mayo Board of Governors
• Clinical Practice Quality Oversight Committee
• Departmental Quality committees . . .
Study Question
Can a computer algorithm match the
accuracy of manual chart review for
determining whether Pneumococcal
vaccination was given to hospitalized
pneumonia patients (as reported for
ORYX Core Measures)?
Methods
• Population:
• Inpatients from July 20042004-March 2005
• Discharge diagnosis of community acquired
pneumonia
• Age>=65
• N=224
• Compare measure manually abstracted (RN) vs.
result electronically calculated (EMR)
• Second nurse reviewed sample of patients with
discrepancies to determine “true”
true” (T)
immunization status
First Step: Reliable Algorithm
Pneumococcal Vaccination
Measure for Pneumonia
• The very first attempt was not
correct
• Previously collected data, the
EMR’
EMR’s audit trail, and the data
generated by the algorithm helped
to find errors
• Developing the algorithm was an
iterative process
Pneumococcal Vaccination
Measure for Pneumonia
Truth
+
-
Truth
+
-
E
+ 151 0
M
48 25
R -
197
3
R +
N 2 22
PPV=151/151(100%)
PPV=197/200(98.5%)
NPV=25/73(34.2%)
NPV=22/24(91.6%)
+
E
+ 151 0
M
48 25
R -
•
•
•
•
from EMR.
Manually review “negatives”
negatives” until NPV becomes
acceptable.
Reminders/education on use of Immunization
Module.
Repeat process with other common quality
measures.
Develop method to ensure accuracy of EMREMRbased measurement.
Truth
+
197
3
R +
N 2 22
PPV=151/151(100%)
PPV=197/200(98.5%)
NPV=25/73(34.2%)
NPV=22/24(91.6%)
Mayo Integrated Clinical Systems
(MICS)
• Chart+
• Clinical Document Management Reports
• LastWord
• LW Navigation (Home Screens & Chart
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Next Steps
• Proceed with prepre-filling “positive”
positive” immunizations
Truth
-
(CDM)
Clinical Notes
• Hospital Summary
Digital Dictation
Documents Browser
History Location System, Mass (HLSM)
Image Capture Environment (ICE)
Master Sheet
Mayo Single Logon (MSL)
Orders97
Patient Appointment Guide (PAG)
Patient Provided Information (PPI)
QREADS
Scheduling (GPAS, MSS)
Shorthand
Tabs)
Allergies
Chart Summary View
Episodes
Flowsheet Charting
• Immunizations
• InBox
• Medication Management
• Orders:
•Cumulative Orders97 Summary
•
•
•
•
(COS)
•Inpatient
•Plan of Care
Outside Film & Media
Patient CheckCheck-In Locator (PCIL)
Problem List/Service Recognition (PLSR)
Results and Reports (Viewers/Labs,
Documents Display, Printing, Patient List
Manager)
Lessons Learned
• Initial algorithm will contain flaws–
flaws– plan to revise.
• Computer can easily determine measure when
•
•
service is performed and documentation
standards followed.
Verification of EMREMR-based measures extremely
valuable.
• Large sample to establish accuracy
• BuyBuy-in from stakeholders to use EMR
EMR has wealth of capabilities for quality
measurement.