Measuring EHR Usage and Patient-Provider
Communication During Outpatient Visits
Dr. Zia Agha, Dr. Alan Calvitti
VA San Diego HSR&D
DISCLAIMER: The views and opinions expressed in this presentation are those of the author and do not necessarily represent official policy or position of HIMSS.
Conflict of Interest Disclosure
Zia Agha, MD
Alan Calvitti, PhD
Have no real or apparent
conflicts of interest to report.
Research funded by Veterans Administration
HSR&D Project IIR-07-196-2
© 2013 HIMSS
Learning Objectives
• Describe EHR usage and task efficiency leveraging
time-domain process measures
• Recognize challenges posed by current technology to
providers who multitask between EHR workflow and
addressing patient needs
• Identify and rank-order the most commonly used EHR
components
• Discuss how to Correlate EHR usage with patientprovider communication patterns
• Recognize how to Leverage analysis results for EHR
design improvements
Health IT Usability
Science & Engineering
• NIST, ISO define usability as “effectiveness,
efficiency and satisfaction with which users can
perform intended tasks”
– Measuring usability (a non-functional, lifecycle
property) is a science.
– Improving usability is achieved through engineering
(functional requirements)
PACE Study Motivation
• Little quantitative data from real clinical settings
(vs. lab experiments with predetermined tasks)
• Clinical settings enable sampling realistic:
variability across patients, multitasking (patientprovider-EHR interaction)
Profile EHR usage and task efficiency
Leveraging time-domain measures
• PACE study:
– Outpatient visits with established patients VA Medical
Center (San Diego)
• Time-stamped visit process data:
– EHR activity (VA’s CPRS/VistA); Non-Verbal provider
workflow; Verbal patient-provider communication
(dialogue)
• Additional data:
– patient & provider satisfaction; agreement; interviews
Types of Visit Process Data by Source
Room Video
Usability Software
NonVerbal & Clinical Workflow
Mouse Activity
Vocalization, Verbal
EHR Screen Recording
Multi-Level EHR Activity Data
Usability software logs time-stamped mouse
activity + manual review of screen video:
– Screens: Notes, Labs, Medications, Reports, Clinical
Reminders
– Tasks: View v. Order Entry; Current v. Previous Note
– Modes: Text Box, v. Radio Button v. Scroll Bar
– Order Entry: Meds, Labs, Consults, Imaging
– Alerts & Reminders
Rank-order EHR component usage
• Hierarchical Task Analysis
– Rank-Order: screens, tasks by visit, by provider…
• Sequential Task Analysis
– Most frequent transitions between screens, tasks…
Temporal analysis of EHR + NonVerbal
• Time-at-task profiling excluding physical exams,
interruptions
• Conditional profiling of EHR activity while
sharing screen
• Verbal analysis (patient provider dialogue) 3minute windows of highest & lowest EHR activity
excluding interruptions
Synchronizing Visit Process Data
Sync’d 94% of visits (EHR+Room
Video)
physicians shared EHR In only 25% of visits Synchronizing EHR Activity + NonVerbal
2 sample visits
Top Chromogram=EHR Mouse Clicks
Bottom Chromogram=NonVerbal
& Clinical Workflow
Categories of User-EHR Interactions
User-EHR interactions:
– Information Retrieval: browsing, searching, decisionsupport
– Documentation: progress notes, reports
– Order Entry: meds, labs, consults, imaging
– Coordination: reminders, care team communication
Physicians Multitask Throughout Visit
Example ~60 min visit EHR Notes
(Documentation + Info Retrieval)
Physical Exam
EHR Labs
(Info Retrieval)
EHR Orders
(Order Entry)
Orders
EHR Screens
NonVerbal
Vocalizations
VOC: Black = Silence
Toward Quantifying Multitasking
Variability of EHR Activity Across Visits
Low EHR Activity
High EHR Activity
Distribution EHR Screens (Mouse Clicks)
Notes 40%
Consults 3%
Reports 4%
Labs 10%
Orders 27%
Meds 10%
Distribution of View Tasks while in Notes
By mouse clicks
View Current Notes 83%
Create New Note 3%
View Notes by Other Providers 7%
View Previous Notes by PCP 7%
Distribution Non Verbal Activity (Time)
Gaze to EHR 40%
Gaze to Paper/Meds 6%
Physical Exam 16%
Gaze to Patient 35%
Longer Visits, More Gaze to EHR
In longer visits, physicians look at the computer more
Visit Length vs. EHR Activity
x‐axis = mouse activity
each point = 1 visit, color coded to physician ID
trend: higher EHR activity in longer visits
x‐axis = visit length
Combined Distribution EHR + NonVerbal
Transitions Between EHR Screens
Reports
Topology: Notes is central “hub” of activity
Cover Sheet
Labs
Other
Notes
Orders
Meds
Consults
Patient
Selection
© 2013 HIMSS
Review/Sign
Weights: fraction of ~3k mouse clicks between distinct screens (navigation)
Transitions Between EHR Tasks
Cover
Meds
Reminders
(Notes)
Cover/Vita
ls
Orders
Views
Consults
Previous
Note
Reports
Notes
Other Provider Notes
Reminders
Meds
Other
Review/
Sign
Recent
Labs
Consults
© 2013 HIMSS
What Screens Do Physicians Share?
Labs 45%
Cover Sheet 5%
Meds 25%
Orders 8%
Notes 14%
EHR Activity: Mouse vs Time-at-Task
5 sample visits
Left bars: distribution by mouse clicks Right bars: distribution by time‐at‐
task
Order Entry Analysis (Meds)
Each row = 1 visit
This one has no Meds Orders
165 mouse clicks to order 6 meds
Order Entry Analysis (Consults)
We also profile Labs, Imaging, Reminders…
82 mouse clicks to order 2 Consults
Order Entry Analysis Summary
Leverage Results  Drive EHR redesign
• Documentation & Order Entry
– Notes: on separate monitor; unstructured ASCII file 
Structure data searchable tags, links, autocomplete
– Minimize navigation burden & pull-down lists
• Information Retrieval & Visualization:
– Better Formatting, help Share
– Labs, Meds, clinical events on multiple screens 
leverage visualization like ZUI timeline browsers
Thank You!
Zia: zia.agha@va.gov
Alan: acalvitti@ucsd.edu
…and the PACE Team:
Kristin Bell – UCSD & VA San Diego
Yunan Chen – UCI
Neil Farber – UCSD
Mark Gabuzda – VA San Diego
Barbara Gray – VA HSRD
Lin Liu – UCSD & VA HSRD
Richard Street – Texas A&M U
Danielle Zuest – VA HSRD