Laura A. Giordano, RN, MBA1
Lessons Learned
& the Dry Run of the
HCAHPS survey
Mary Anne Hope, MS1
William G. Lehrman, PhD2
Elizabeth Goldstein, PhD2
~
~
AcademyHealth Research Meeting
1Health
Orlando, Florida
June 4, 2007
1
2.
3.
4.
Services Advisory Group (HSAG)
for Medicare & Medicaid Services (CMS)
2
1: Premises of HCAHPS
Overview of Presentation
1.
2Centers
Premises of HCAHPS
Challenges of HCAHPS
Purpose and method of Dry Run
Lessons learned and findings
from Dry Run
1. Compatibility: HCAHPS must be
compatible with patient surveys
currently conducted by hospitals
2. Comparability: HCAHPS must
produce publicly reported scores
that permit fair and valid
comparisons among all participating
hospitals
3
4
2: Challenges of HCAHPS, cont.
2: Challenges of HCAHPS
Answer:
How can valid results be achieved when
HCAHPS permits diversity in:
– Standardization
– Participation by spectrum of hospitals
and survey vendors
– Centralization
– Four modes of survey administration
– Training and technical assistance
– Both stand-alone and integrated formats
– Oversight
– Common protocols
… As demonstrated through, and
– Two methods of data submission
5
refined by, the HCAHPS Dry Run
6
1
Participation in the Dry Run
3: Purpose of the Dry Run
• Opportunity for hospitals and survey vendors
to gain experience in HCAHPS administration,
data collection and data submission
• 2,826 short term acute care hospitals
participated in the Spring 2006 Dry Run
(April, May and/or June)
– 1.8% self-administered the survey
• Allow CMS to evaluate and refine:
– Implementation protocols and policy
– Guidelines for survey administration
– Data submission processes
– 1.1% administered for self and other hospitals
– 97.1% used a survey vendor
• 576,744 surveys submitted: 84% Mail;
12% Telephone; 0.5% Mixed; 3% Active IVR; 0.5% unknown
7
Standardization in HCAHPS:
Standardization in HCAHPS:
Survey implementation
Training and Technical Assistance
Survey vendors and self-administering hospitals
follow standardized procedures for:
All participating survey vendors and selfadministering hospitals were required to:
– Survey modes (four)
– Attend introductory and update training
– Generating frame of eligible discharges
– Apply and receive approval to participate
– Survey administration (timing, etc.)
– Follow Quality Assurance Guidelines and
policy updates posted on
www.hcahpsonline.org
– Data collection (protocols, etc.)
9
Standardization in HCAHPS:
– Data submission (XML or On-line tool)
10
Centralization in HCAHPS
Oversight
All participating survey vendors and selfadministering hospitals will be required to:
To further assure credibility:
• Centralized data storage and analysis
– Develop & implement Quality Assurance Plans
• Centralized official public reporting of
hospital results
– Participate in on-site visits, as scheduled
• Random or targeted
• Centralized assistance and guidance
– Successfully submit HCAHPS survey data to
warehouse by due date
• Data then analyzed for errors, anomalies, outliers
8
• Centralized oversight
11
12
2
4: Lessons Learned from Dry Run
Principal Findings of Dry Run
• Update Quality Assurance Guidelines
• Problematic issues:
• Schedule additional required training
– Wide range of technical knowledge and resources
among survey vendors & self-administering hospitals
• Re-evaluate data specifications
– Utilizing appropriate sampling types
• Make adjustments for common errors
– Understanding data coding specifications
• Clarify and standardize content of
Quality Assurance Plans
– Unsuccessful and late data submissions
– XML file formats
13
Conclusions
– Attention to submission and feedback reports
14
Comments?
HCAHPS Dry Run:
LGiordano@azqio.sdps.org
• Provided invaluable ‘actual’ experience
• Led to improvements in data coding,
collection and submission
william.lehrman@cms.hhs.gov
• Led to increased emphasis on data checks
and analysis
elizabeth.goldstein@cms.hhs.gov
• Validated efficacy of standardization
and centralization
15
16
3