Estimating Task Execution Time in EHRs Using the Keystroke
Estimating Task Execution Time in EHRs Using the Keystroke-Level Model
Louis Lee, Yuanyuan Li, MD,MS, Krisanne Graves, MSN, RN, Amy Franklin PhD, Muhammad F. Walji, PhD, Jiajie Zhang, PhD
The University of Texas Health Science Center at Houston – School of Biomedical Informatics
Introduction
• A major barrier to EHR adoption is that clinicians find these systems inefficient and difficult to use[1]
• There is a lack of objective and reliable measures of user performance for common clinical tasks in EHRs
• Keystroke Level Models (KLM) can be used to analyze work flows and identify factors for work flow optimization, and have been demonstrated to accurately predict skilled user performance time [2, 3]
Research Goals
• Measure skilled user performance time for tasks within NIST meaningful use cases (MUCs) [4]
• Establish benchmarks for MUC time
• Compare and analyze work flows for MUCs across EHRs
Method
• CogTool [5], a KLM tool, was used to analyze 6 MUCs (e.g., ePrescribing, smoking status) across 3 EHRs
• Task execution times were predicted by CogTool, and resulting times were compared across EHRs
• Time for each physical operator (see Figure 2) was analyzed for variability
• Recommendations for improving task efficiency were generated
For additional information, please contact
SHARPC@uth.tmc.edu
Results
EHR3
EHR2
EHR1 ePrescribing
Smoking Status
Problem List
BMI
Demographics
Vital signs
0 20 40 60 80 100 120 140
Time (s)
Figure 1
. Predicted task execution time for 6
MU objectives across 3 EHRs
Type
Move Mouse
Move and Hover
Left Triple-Click
Left Double-Click
Left Click
EHR3
EHR2
EHR1
Home Keyboard
Home Mouse
0 5 10 15 20 25 30 35 40
Time (s)
Figure 2.
Time for physical operators in
Recording Problem List across 3 EHRs
Portion of work flow for recording problem list for EHR1
Patient
Dashboard: click “medical records”
“Problem List”
Tab: Click “Add
Diagnosis”
Medical Records Webpage:
Maximize the screen, click
“Problem List”
“Problem List”
Tab: Click “Add
Diagnosis”
“Patient Diagnosis” pop-out window:
Left Click entry field; Type “tia”
Left Click “search”
Fig.5. Problem List Recording Structure for EHR1
Left Click the correct diagnosis from pop-up search result box
Left Click the “Acute” status from the pull down list
Left Click “Save”
Continues
Summary of Conclusions
Although KLM results were different across MUCs, there was surprising consistency in task time across the 3 EHR products
• Problem List had the longest execution time and greatest variability across EHRs
Next Steps
• Compare and analyze clinical task work flows in different EHRs
• Optimize the work flow according KLM to reduce task execution time
• Collecting data across larger samples will allow creation of benchmarks for NIST MUCs
References
1. Smelcer JB, Miller-Jacobs H, Kantrovich L. Usability of Electronic
Medical Records. Journal of Usability Studies.2009;4:70-84.
2. John BE, Suzuki S. Toward cognitive modeling for predicting usability.
To appear in HCI international 2009.
3. John BE. Using predictive human performance models to inspire and support UI design recommendations. Proceeding of ACM CHI’11 session on predicting & modeling human behaviors 2011.
4.
http://www.nist.gov/index.html
(accessed Oct 14th, 2011).
5.
http://cogtool.hcii.cs.cmu.edu/
(accessed Feb 15th, 2011).
Acknowledgement
This project was supported by Grant No. 10510592 for Patient-
Centered Cognitive Support under the Strategic Health IT
Advanced Research Projects (SHARP) from the Office of the
National Coordinator for Health Information Technology.
