Evaluating the Outcomes of New York
Community Home Health Care
Interoperability Program(NYCHHIP):
Understanding the Challenges of
Evaluating Technological Innovations in
Home Care
Peri Rosenfeld
Rosenfeld, PhD
Caroline Kim, MPH
Robert J. Rosati,, PhD
AcademyHealth, June 27, 2010
Visiting
s t g Nurse
u se Se
Service
ce o
of New
e York
o
 Nation
Nation’s
s
largest not
not-for-profit
for profit home care
agency with an average daily census of
28,000 patients
 Over 100,000 referrals made to the Adult
Acute Care Program in 2009
 65% from hospitals
 18% from physician offices
Information Technology
gy Capacity
p
y

VNSNY has dedicated Information Technology
gy
(IT) Department consisting of approximately 190
staff equally divided between applications
development and infrastructure support
 3500 pen tablets (point of service computing)
used by clinicians in the field
 Electronic health records known as the Patient Care
Record System (PCRS), are generated when a case
is assigned to clinicians
 EHR includes Plan of Care, OASIS, Clinical
Documentation, Medication and other other data
New York Community Home Health
Interoperability Program (NYCHHIP)
 Two
year (2006
(2006-2008)
2008) initiative funded
through New York State Healthcare
Efficiency & Affordability Law (NY
HEAL)
 Obtained no-cost extension to June ‘09
 Formal evaluation completed FebruaryMarch 2010
Interoperability
te ope ab ty
 The
ability of different information
technology systems, software applications
and networks to communicate, to
exchange
g data accurately,
y effectively
y and
consistently, and to use the information
that has been exchanged.
exchanged
(National Alliance for Health Information Technology, 2005)
Specific
p
Milestones

Install appropriate software
at participating physician
practices

Design and develop
electronic data exchange
between the physician
offices and VNSNY
(e.g.POC, non-urgent
messaging)

Enable electronic referrals
by
yp
physician
y
p
practices to
VNSNY

Create electronic data
exchange to allow
VNSNY and physician
practices to receive lab
results electronically

Create portal to allow
patients access to their
own data

Evaluate the extent to
which NYCHHIP
achieved its desired g
goals
Desired
es ed Outco
Outcomes
es
 Improvements in the completeness and accuracy of
information exchanged
 Efficiencies in documentation and transactions
between MD and RN
 Consistent provider and patient use and satisfaction
with system
 Positive impact on service coordination
 Improvements in patient outcomes
Implementation
p e e tat o

VNSNY IT responsible for:
 Development and project management of technical specifications including
consultation with nurses and other clinical and operational staff
 Implementing technical parameters to allow the data exchange between
VNSNY, laboratories and MD practices.

Software vendors were to tailor existing tools to meet the needs of
VNSNY and the MD practices

MD Practices were to be pilot sites and partners in developing data
exchange

Evaluators from the Center for Home Care Policy & Research
 Identified process and outcomes measures to assess impact on clinical and
other
th indicators
i di t
 Determined appropriate methods to collect data and conduct evaluation
research
Methodology
et odo ogy

The evaluation adopted a descriptive
descriptive, longitudinal
approach, using qualitative and quantitative methods to
monitor progress and outcomes from baseline to
completion
l ti off th
the ttwo year study
t d period.
i d
 Data sources included:
 Surveys of physician comfort levels and utilization of
electronic communications;
 Monitoring
g administrative data on the number and
quality of electronic communications over time;
 Surveys of satisfaction with electronic innovation;
 Utilization data on patient portal interface,
 In depth interviews with key stakeholders
Two Years Later: Status of Milestones
 Install appropriate software at participating physician
practices: Accomplished
 Design
D i and
dd
develop
l electronic
l t i d
data
t exchange
h
b
between
t
physician practices and VNSNY: Partially Implemented



Difficulty identifying practices with enough homecare patients in
concentrated
d geographic
hi area to serve as pilot
il sites
i
Technical requirements of project were a challenge for practices
Difficulties with support and follow-up from software vendors
 Enable electronic referrals by physician practices to
VNSNY: Partially Implemented
 Obstacles were similar to those encountered above
 Too few e-referrals to measure any impact on physician satisfaction
or improvements in efficiencies
 Additional progress made since February
Two Years Later: Status of Milestones

Create electronic data exchange to allow VNSNY and
physician practices to lab results electronically : Partially
Accomplished
 VNSNY and some practices receive lab results electronically
 E-labs eliminated use of fax and reduced administrative errors
through internal checks from other data sources at VNSNY
 Time spent resolving lab results was unchanged because RNs
continued to review all lab results, not just those out of range

Create portal to allow patients access to their own data:
Accomplished
 Patient Portal was created
 Difficulty recruiting patients to test the portal due to lack of
access to computers and lack of interest

Evaluate the extent to which NYCHHIP achieved its desired
goals Partially Accomplished
 Scope of original evaluation plan was overly ambitious
 Combination of operational deadlines and technical challenges
overrode original evaluation design
MDs Have Limited Involvement in
H
Home
Care
C
Processes
P
Percent o
of MD s Indiicating
A ny Involvemeent
Figure 1. Percent of Physician with Any Involvement in
Selected Home Care Admission Processes
50%
40%
43.4%
38.6%
33 7%
33.7%
30%
20%
10%
0%
Referrals
Plan of Care (POC)
Home Care Admission Processes
Lab Results
Selected Outcomes of Lab Portal
Figure 2. Comparison of Actions Required to Resolve Lab
Results at Baseline vs. Post Implementation
100%
Perccent
80%
76.7% 79.6%
60%
40%
24.1%
14.6%
20%
1.5% 1.5%
10.0% 12.2%
0%
Contact Clinician
Contact MD
Contact Patient
Actions Taken to Resolve Lab Results
Baseline
Post Implementation
Re-Fax Result to MD
Office
Nurse Satisfaction with Lab Portal
Per cent
Figure 3.Percent of End Users Indicating Very Satisfied or
Satisfied by Satisfaction Measures
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
66.7%
66.7%
55.6%
55.6%
55.6%
55.6%
44.4%
44.4%
Accessing
A
i th
the Navigating
N i ti IInfo
f
Completeness
C
l t
A
Accuracy
off
Vi l
Visual
Ad
Adequacy
off
A il bilit off
Availability
O
Overall
ll
New E-Lab
w /in Lab Portal of Information
Informantion Presentation of Training to Use
Technical
Satisfaction
Portal
Available in Lab Available in Lab Information in New Lab Portal
Support
w ith Electronic
Portal
Portal
Lab Portal
Services w hen
Lab Portal
Needed
Satsifaction Measure
Lessons Learned from Evaluation
and
dK
Key IInformants
f
t

Articulate realistic timeframes and assessments of
efforts needed to effect change
 Funder mandated deliverables and timetables may contribute to
unrealistic goals and objectives
 Distinction between establishing “proof of concept” and evaluating
change and outcomes needs to be clarified.


See Torda, O, et al 2010; Fernandopulle, R, etal 2010; DesRouches & Jha, 2009
Fragmented service delivery system between home and
hospital care (as compared to integrated systems) poses
significant challenges for development of cross setting
EHR
 Issue of volume and scalability – small number of transactions
requires longer periods of time to evaluate outcome and achieve
ROI
 Lack of standardization/uniformity across MD practices impeded
ability to demonstrate cost effectiveness
Lessons
esso s Learned
ea ed (co
(cont’d)
t d)

Establish clear expectations and timelines for software
vendors
d


Dedicated project manager needs to oversee all components
and work closely with evaluators


Fernandopulle, et al, 2010; Miller, et al, 2003; Felt-Lisk, et al, 2009
Torda, et al, 2010
Establish clear expectations and eligibility criteria for
physician practices to include innovators vs
vs. late adopters
adopters.
 Funding timeline too short to recruit practices to serve as pilot
sites


Fernandopulle R,
Fernandopulle,
R etal 2010; Miller et al,
al 2003; Mostashari
Mostashari, et al,
al 2009
Adhere to schedule of routine communication with internal
and external stakeholders

Freidman,, et al,, 2009
Policy
o cy Implications
p cat o s
 State
or national level efforts (e.g. RHIOs)
may be more effective and scalable than
individual organizational efforts at cross
crosssetting data exchange
 Given rapid changes in software and other
technological
g
innovation,, sustainabilityy is a
challenge
 Difficult to demonstrate Return on
Investment
References
e e e ces
1
1.
2.
3.
4.
5.
6
6.
7.
Torda, P
Torda
P., Han
Han, E
E. S
S., & Scholle
Scholle, S
S. H
H. (2010)
(2010). Easing the adoption and use of
electronic health records in small practices. Health Aff (Millwood), 29(4), 668-675.
Fernandopulle, R., & Patel, N. (2010). How the electronic health record did not
measure up to the demands of our medical home practice. Health Aff (Millwood),
29(4) 622-628
29(4),
622-628.
DesRoches, C. M., & Jha, A. K. (2009). Health Information Technology in the
United States: On the Cusp of Change, 2009 o. Document Number)
Miller, R. H., Sima, I., Newman, J. (2003). Electronic Medical Records: Lessons
from Small Physician Practices
Practices. Oakland
Oakland, California: California HealthCare
Foundation.
Mostashari, F., Tripathi, M., & Kendall, M. (2009). A tale of two large community
electronic health record extension projects. Health Aff (Millwood), 28(2), 345-356.
Felt Lisk S
Felt-Lisk,
S., Johnson
Johnson, L
L., Fleming
Fleming, C.,
C Shapiro,
Shapiro R.,
R & Natzke,
Natzke B
B. (2009)
(2009). Toward
Understanding EHR Use in Small Physician Practices. Health Care Financing
Review, 31, 11-22.
Friedman, M. A., Schueth, A., & Bell, D. S. (2009). Interoperable electronic
prescribing in the United States: a progress report
report. Health Aff (Millwood),
(Millwood) 28(2)
28(2),
393-403.