Health Information Technology –
Research Challenges
Michael E. Wiklund
General Manager – Human Factors Engineering
UL–Wiklund (formerly Wiklund Research & Design)
UL and the UL logo are trademarks of UL LLC © 2013
The Research Project
NIST retained Wiklund R&D (now part of UL) to develop
guidelines for EHR usability and to develop a framework for
EHR usability assessment.
Project funded by the American Recovery and Reinvestment
Act of 2009.
Key tasks:
• Survey, observe, and interview EHR users
• Conduct HFE inspection of sample EHRs
• Conduct usability test of sample EHRs
• Develop EHR user interface design guidelines
• Develop recommended process for EHR usability
assessment
• Document results in a report
2
Progress
• Conducted large scale survey of EHR users
• 559 respondents: physicians, nurses, administrators
•
Vanderbilt University Medical Center facilities
•
3 month process
•
18% response rate
•
Facilitated high response rate by visiting departments to invited responses
•
Evaluated user experiences with StarPanel and multiple “off-the-shelf” EHRs that are
integrated into Vanderbilt’s EHR
•
Many new insights pertaining to topics such as: information saturation, value of
defaults, attention to updating baseline information, information integration
•
Future reports will share distilled findings
3
Topic might not be riveting…
Photo credit: http://etc.usf.edu/clippix/pix/airplane-rivets-2_medium.jpg
…but it is pertinent!
4
UL-Wiklund’s Partners
• Vanderbilt University
• University of Wisconsin (initially)
• Design Science
5
Project Status
We have made slow progress.
There are government rules pertaining to:
• Paperwork reduction
• Protecting human subjects
• EHR vendors having equal opportunity to
submit their EHRs for assessment
• Contract performance period
• Legal consent by the sponsoring agency
Photo credit: http://3.bp.blogspot.com/MwaoVCK5aKA/TtpmP0o0dsI/AAAAAAAABms/WyJh53uuBP4/s1600/T+
and+H.jpg
6
Paperwork Reduction Act (44 U.S.C. 3501 et seq.)
Federal law enacted in 1980.
Excerpts:
•
Minimize the paperwork burden for individuals, small
businesses, educational and nonprofit institutions, Federal
contractors, State, local and tribal governments, and other
persons resulting from the collection of information by or for
the Federal Government.
•
Ensure the greatest possible public benefit from and
maximize the utility of information created, collected,
maintained, used, shared and disseminated by or for the
Federal Government.
•
Minimize the cost to the Federal Government of the creation,
collection, maintenance, use, dissemination, and disposition of
information.
7
Paperwork Reduction Act (44 U.S.C. 3501 et seq.)
Our research plan called for using the same questions (e.g., survey questions) to
collect data from more than 10 people.
Example: We have already collected data from hundreds of EHR users (e.g.,
physicians, nurses, technicians) working for Vanderbilt University Medical Center.
Such requests:
•
Describe the data (i.e., information) that you are collecting.
•
Explain why the data is needed.
•
Estimates the time and cost (for the public) to respond to the request.
Project impact:
NIST had to place an information collection request with Office of Management and
Budget (OMB) to collect EHR-related data.
Approval process was lengthy.
8
Federal Wide Assurance (FWA)
An institution commits to Health and Human Services (HHS) that
it will comply with the requirements in the HHS Protection of
Human Subjects regulations at 45 CFR part 46.
HHS human subject protection regulations…were first issued in
1974.
In 1978, the National Commission or the Protection of Human
Subjects of Biomedical and Behavioral Research published
“Ethical Principles and Guidelines for the Protection of Human
Subjects of Research,” also known as the Belmont Report, named
after the Belmont Conference Center where the Commission met
when drafting the report.
Source: http://www.hhs.gov/ohrp/humansubjects/index.html
9
Federal Wide Assurance (FWA)
Project impact:
We submitted a formal, written, and binding commitment to Office for Human
Research Protections (OHRP) stating that UL-Wiklund would fully comply with
human subjects protection guidelines.
Our submission included:
•Company’s identifying information, including the identify of human protections
administrator.
•List of the ethical principles that we would follow; we cited Belmont Report.
•Designation of IRBs that would review our research (activities covered under the
FWA).
•Binding letter of agreement with external IRB to properly oversee our research.
Approval process was lengthy.
10
Institutional Review Boards
NIST required that its own IRB review and approve all research plans. This may
be the case with other federal agencies.
Research plans also had to be approved by research sites, such as Vanderbilt
University, to conduct research in their hospitals.
Medical institutions are unlikely to abdicate responsibility to another
organization – even a federal agency – for human subjects projection
associated with research on their premises.
Approval process was lengthy.
11
Equal opportunity for EHR assessment
Our plan has been to assess a diverse set of EHRs by means of HFE
inspection and usability testing.
Simple path would be to select several EHRs for assessment based on
appropriate selection criteria (e.g., use environment, user interface features).
This path was problematic because it could give (or be perceived as giving)
certain EHRs vendors an advantage or disadvantage.
Solution was to advertise in the Federal Register the opportunity for developers
to submit their EHRs for assessment.
Important: We will evaluate the EHRs only for the purpose of identifying good
design practices. Our published results will not cite specific EHRs.
This step was time consuming.
12
Announcement on 2-14-12
13
Announcement on 2-14-12
NIST is soliciting interest in supplying electronic
health record (EHR) systems for use by NIST in
research to develop a framework for assessing the
usability of health information technology (HIT)
systems, EHRs in particular, and performanceoriented user interface design guidelines for EHRs.
14
Legal Review
NIST’s legal team had to review all research plans before they could be
submitted to the agency’s own IRB for further review.
15
Many hurdles…
Photo credit:
http://msnbcmedia2.msn.com/j/MSNBC/Components/Slideshows/_production/_archive/2012%20LONDON-OLYMPICS/ss-120727-emotional-moments-july27-aug4/ss-120804-oly-emotionalmoments/ss-120804-emotional-moments-21.ss_full.jpg
…but necessary.
16
Lessons Learned
• Recognize that government-sponsored research introduces multiple
responsibilities that might not otherwise come into play when
conducting privately-funded research.
• Add considerable time into the research schedule to obtain FWA and
PRA approvals.
• Plan to obtain multiple IRB approvals when conducting multi-site
research, particularly when conducting it at healthcare institutions.
• Perfect your research plans early because changing them
(presumably for the better) would require many re-approvals.
17
Questions?
UL and the UL logo are trademarks of UL LLC © 2013