Time to Reap: Improving Quality
by Harvesting Data from the EMR
Marie J. Eidem, BS
Aaron A. Kurtzhals, BS
James M. Naessens, MPH
Mayo Clinic – Rochester, MN
Mayo Integrated Clinical Systems
(MICS)
Measuring Quality
• External
• JCAHO
• CMS
• Leapfrog
• National Quality Forum
• Blue Cross Blue Shield
• Minnesota Community Measurement
• Institute for Healthcare Improvement 100K Lives . . .
• Internal
• Continuous Improvement teams
• Mayo Board of Governors
• Clinical Practice Quality Oversight Committee
• Departmental Quality committees . . .
Study Question
Can a computer algorithm match the
accuracy of manual chart review for
determining whether Pneumococcal
vaccination was given to hospitalized
pneumonia patients (as reported for
ORYX Core Measures)?
Methods
• Population:
• Inpatients from July 2004-March 2005
• Discharge diagnosis of community acquired
•
•
pneumonia
Age>=65
N=224
• Compare measure manually abstracted (RN) vs.
result electronically calculated (EMR)
• Second nurse reviewed sample of patients with
discrepancies to determine “true” (T)
immunization status
First Step: Reliable Algorithm
• The very first attempt was not
correct
• Previously collected data, the
EMR’s audit trail, and the data
generated by the algorithm helped
to find errors
• Developing the algorithm was an
iterative process
Pneumococcal Vaccination
Measure for Pneumonia
Truth
+
E
+ 151 0
M
48 25
R -
Truth
+
R +
N -
197
3
2
22
PPV=151/151(100%)
PPV=197/200(98.5%)
NPV=25/73(34.2%)
NPV=22/24(91.6%)
Pneumococcal Vaccination
Measure for Pneumonia
Truth
+
E
+ 151 0
M
48 25
R -
Truth
+
R +
N -
197
3
2
22
PPV=151/151(100%)
PPV=197/200(98.5%)
NPV=25/73(34.2%)
NPV=22/24(91.6%)
Mayo Integrated Clinical Systems
(MICS)
• Chart+
• Clinical Document Management Reports
• LastWord
• LW Navigation (Home Screens & Chart
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
(CDM)
Clinical Notes
• Hospital Summary
Digital Dictation
Documents Browser
History Location System, Mass (HLSM)
Image Capture Environment (ICE)
Master Sheet
Mayo Single Logon (MSL)
Orders97
Patient Appointment Guide (PAG)
Patient Provided Information (PPI)
QREADS
Scheduling (GPAS, MSS)
Shorthand
Tabs)
Allergies
Chart Summary View
Episodes
Flowsheet Charting
• Immunizations
• InBox
• Medication Management
• Orders:
•Cumulative Orders97 Summary
•
•
•
•
(COS)
•Inpatient
•Plan of Care
Outside Film & Media
Patient Check-In Locator (PCIL)
Problem List/Service Recognition (PLSR)
Results and Reports (Viewers/Labs,
Documents Display, Printing, Patient List
Manager)
Next Steps
• Proceed with pre-filling “positive” immunizations
•
•
•
•
from EMR.
Manually review “negatives” until NPV becomes
acceptable.
Reminders/education on use of Immunization
Module.
Repeat process with other common quality
measures.
Develop method to ensure accuracy of EMRbased measurement.
Lessons Learned
• Initial algorithm will contain flaws– plan to revise.
• Computer can easily determine measure when
•
•
service is performed and documentation
standards followed.
Verification of EMR-based measures extremely
valuable.
• Large sample to establish accuracy
• Buy-in from stakeholders to use EMR
EMR has wealth of capabilities for quality
measurement.