Using Electronic Medical Records
Data for Health Services Research
Case Study: Development and Use
of Ambulatory Adverse Event
Trigger Tools
Hillary Mull
VA Boston Healthcare System
Boston University School of Public Health
June 27, 2010
AcademyHealth ARM 2010
1
Outline

Description of trigger study

Domains of data merge challenges



Data conversions
Data availability
Data de-identification

Examples of merged data quality problems

Conclusions
AcademyHealth ARM 2010
2
Objectives

Developed and tested a set of triggers to identify
outpatient adverse events (AEs)

Triggers are algorithms that use electronic patient data to
identify patterns consistent with a possible AE

E.g. the combination of a lab value threshold and an
active prescription

Requires high quality electronic patient data to
1.
Develop the surveillance rule
2.
Evaluate whether an AE occurred
AcademyHealth ARM 2010
3
Develop Triggers
Merge Data
Program triggers
and perform
chart
classification
Analyze trigger
performance
•
•
•
•
Clinicians review literature to develop potential outpatient AE triggers
Review these triggers with focus groups
Hold Delphi process to refine trigger logic, develop new triggers
Establish list of outpatient AE triggers to test
• Obtain de-identified data extracts from
each site
• Merge variables from each site into TIDS
database
• Iterative process of checking data quality
and obtaining new data extracts
Site 1
• Program trigger algorithms to run on
data in TIDS database and flag cases
• Iterative process of running trigger and
checking data quality
• Build mock EMR (TIDS viewer) to
interface with TIDS database
• Perform chart classification
Site 2
List of
triggers
Site
3
TIDS database:
Inpatient, outpatient,
vitals, demographics,
lab, pharmacy, notes
Program
trigger
algorithms
Trigger-flagged
cases
TIDS
Viewer
Chart
Review
• Calculate positive predictive value and
95% confidence intervals for each trigger
AcademyHealth ARM 2010
4
Data Merge Overview

We developed a list of data elements for each site’s
data programmer

EMRs were not designed for HSR; site-based
programmers were not familiar with data extractions
for HSR

Discovered numerous inconsistencies with data


Six revisions to guidelines over 8 months
A problem at one site often required another data pull at a
different site to ensure consistency
AcademyHealth ARM 2010
5
Data Challenges: Conversions

Same information, different coding by institutions

Patient sex: M/F vs 1/2/3
Units of measure: metric vs US vs missing
ICD-9-CM codes stored with or without periods

Pharmacy dosages: 1 vial vs 50 ml

Lab titles and results inconsistent across settings



Lack of documentation on coding practices

Numeric results within text data
AcademyHealth ARM 2010
6
Data Challenges: Availability

Missing pharmacy dosage/fill data




Missing National Drug Codes (NDCs) in pharmacy data
Free text vs. standardized daily dosage information
ICD-9-CM procedure codes were unavailable for
some procedures
Lack of units in lab data
AcademyHealth ARM 2010
7
Example of Pharmacy Sig

TAKE ONE-HALF TABLET BY MOUTH EVERY DAY FOR
2 WEEKS, THEN TAKE ONE-HALF TABLET TWO (2)
TIMES A DAY FOR 2 WEEKS, THEN TAKE ONE
TABLET TWO (2) TIMES A DAY FOR 2 WEEKS, THEN
TAKE TWO TABLETS TWO (2) TIMES A DAY FOR 2
WEEKS, THEN TAKE THREE TABLETS TWO (2) TIMES
A DAY FOR 2 WEEKS, THEN TAKE FOUR TABLETS
TWO (2) TIMES A DAY INCREASE
DOSE GRADUALLY. WHEN GOING FROM 25 TO 50
MG START WITH INCREASING THE AM DOSE FOR
2WEEKS, THEN THE AM AND PM DOSE. DO THIS
WHEN INCREASING FROM 50 TO 75 AND 75 TO
100. IF QUESTIONS PLEASE CALL.
AcademyHealth ARM 2010
8
Data Challenges:
De-identification

Obtaining EMR data from each site required
development and site-based validation of a deidentification algorithm

Produced gaps in the data



Fuzzy pattern and word matching removed some key
clinical terms from clinical notes
Removal of dates resulted in loss of information about
clinical order
De-identification made notes difficult to read
AcademyHealth ARM 2010
9
Example of De-Identification
from Patient Notes
AcademyHealth ARM 2010
10
Review of Merged Data #1
AcademyHealth ARM 2010
11
Review of Merged Data #2
300,000
250,000
200,000
150,000
100,000
50,000
0
1st Data Pull 39 Note
Types
2nd Data Pull 145 Note
Types
AcademyHealth ARM 2010
12
Conclusions

What patterns in the data did we NOT look
into?

How can we be sure the electronic data is
reliable?

How can we better predict the
time/complexity of merging electronic clinical
data from multiple sites?
AcademyHealth ARM 2010
13