Assessing Cultural Competency from the
Patient’s Perspective:
The CAHPS Cultural Competency (CC) Item Set
Robert Weech-Maldonado
Department of Health Services
Administration
University of Alabama at Birmingham
1
Collaborators

Adam Carle
Carle, University of Cincinnati
School of Medicine
 Beverly Weidmer
Weidmer, RAND
 Margarita Hurtado, AIR
 Quyen Ngo-Metzger, University of
California, Irvine
 Ron D. Hays, UCLA
2
Disparities
p
in Care

IOM (2003) report “Unequal
Unequal Treatment”
Treatment
documented disparities in care




Racial/ethnic disparities in access to health care
Minorities suffer from poor outcomes
Factors associated with health disparities
p
Cultural competency one of the strategies to
address health disparities
3
Cultural Competency and
Health Disparities
C
Cultural
Diverse
Populations + Competency
Appropriate
Services for
Minority
Group
Improved
Outcomes
For Minority
Group
Members
Members
Reduction of
Health
Disparities
Brach and Fraser (2000)
4
What Is Cultural Competence?



Multiple
p definitions and frameworks for cultural
competency
Care that is responsive to diversity and cultural
factors such as language, beliefs, attitudes and
behaviors that affect health and health care
The ongoing capacity of healthcare systems,
organizations and professionals to provide for
diverse patient populations high quality care
that is family- and patient-centered and
equitable (NQF
(NQF, 2009)
5
Assessments of Cultural
Competency


Importance
p
Levels

Organizational


Cultural Competency Assessment Tool for Hospitals
(CCATH): operationalizes the CLAS standards in the
hospital context
Patient

Patient Experiences with Care


Patient Assessments of Cultural Competency (PACC)


Consumer Assessments of Healthcare Providers and Systems
(CAHPS ®)
CAHPS Cultural Competency Item Set
Provider
6
Cultural Competence and CAHPS

Consumer Assessments of Healthcare Providers and
Systems (CAHPS ®)





CAHPS surveys examine quality and performance based on
consumer experiences
Funded by the Agency for Healthcare Research and Quality
CAHPS I- Health plan survey included patient-doctor
communication and research on racial/ethnic and
language differences
CAHPS II- research on cultural competence and initial
development and testing of a cultural competency item
set
CAHPS III- inclusion of Cultural Competency (CC)
Item Set into CAHPS family of instruments
7
Development
p
of CAHPS CC
Development
p
of
Conceptual
Model
Item
Development
Translation
Into
Spanish
Cognitive
Testing
Field
Test
8
Measuring
g Culturally
y Competent
p
Care
Patient Factors
Patient Provider
Communication
Respect for patient
preferences/shared
decision making
g
Experiences leading to
trust or distrust
Experiences with
discrimination
Health literacy strategies
Language
L
services
i
Provider Factors
Health Care System Factors
9
CAHPS and Cultural Competence
(Gaps in Current Measures)


Patient-Provider Communication

Preventive care

Use of complementary and alternative medicine
Shared Decision-Making
g



Linguistic Competency

A
Access
tto language
l
services
i

Health literacy aspects
Experiences Leading to Trust/Distrust


Respect for patient preferences
Level of trust, caring, truth-telling
Experiences of Discrimination

Due to race/ethnicity, insurance, language
10
Item Development
p

Literature review of existing measures
 Adapted or modified measures in the
public domain
 Wrote new items for domains/subd
domains
i ffor which
hi h we were unable
bl tto
identify existing measures
11
Overview of Draft Item Set


Supplemental item set for the CAHPS
Clinician and Group Survey
Included 6 composites and 47 items







Patient-Provider
Patient
Provider Communication (5 items)
Alternative Medicine (6 items)
Shared Decision
Decision-making
making (7 items)
Experiences of Discrimination (12 items)
Trust ((7 items))
Language Access (10 items)
Development and testing of health literacy
items funded by a separate project
12
Translation into Spanish
p
Used modified “translation by
y committee
approach”
 Conducted 2 forward translations using ATA
certified,
tifi d professional
f
i
l ttranslators
l t


Provided translators background information
(purpose, characteristics of target audience,
mode of data collection)
Reviewed and reconciled translation
diff
differences
b
by committee
i
off translators
l
and
d
bilingual members of CAHPS Cultural
Comparability team
13
Cognitive
g
Testing
g




Assess whether p
patients understand key
y
concepts as intended
Identify terms, items, response options that
are problematic
Assess appropriateness of Spanish language
translation
Findings used to revise and refine survey
items
14
Cognitive
g
Testing
g



Semi-structured interview with scripted probes
Used concurrent, think aloud method to
interview
18 interviews conducted




9 in Spanish and 9 in English
L A
Los
Angeles,
l
B
Boston,
t
Ch
Chapell Hill (NC)
Mix of respondents in terms of age,
race/ethnicity,
y, gender,
g
, and level of education
Set targets for Hispanic subgroups
15
Findings
g from Cognitive
g
Interviews





Respondents generally understood survey
items and provided meaningful responses
Item set covered issues and experiences that
were relevant and important to the
respondents
S
Several
l respondents
d t h
had
d problems
bl
ffollowing
ll i
the skips (particularly Spanish speakers)
Some translation iss
issues
es identified
Some items were confusing or difficult to
understand
16
Revisions to Surveyy

Shortened some items to make them
easier to understand
 Modified
M difi d translation
t
l ti off some items
it
 Dropped items that
Were redundant
 Did not provide meaningful data

17
Field Test

Sample
Stratified random sample by race/ethnicity
and language
 6,000 Medicaid managed care enrollees
from two health plans (CA and NY)


Survey

Mixed mode

Two-stage mail phase
 Two-stage phone phase
26% response rate

18
Field Test

Analytic sample limited to respondents who
had



A personal doctor
Visited their personal doctor at least once during the
l t 12 months
last
th
Racial/ethnic composition of final sample
(
(N=991)
)






White- 15%
Black- 15%
Hispanic 34%
HispanicAsian- 17%
Other- 18%
Missing- 1%
19
Data Analysis
y

Psychometric analysis
Exploratory factor analysis
 Confirmatory factor analysis (CFA)
 Multitrait scaling analysis
 Internal consistency (Cronbach alphas)
 Multi-group CFA

20
Data Analysis
y

Regression analysis
Assess convergent validity
 Overall doctor rating (0-10)=
(0 10)= f (CAHPS CC
composite, gender, age, education, and
perceived health status)
 CAHPS CC composites

Items
e s co
converted
e ed to
o0
0-100
00 scale
sca e
 Average of item scores within composite

21
Results

Exploratory
p
y factor analysis
y
((eigenvalues
g
> 1)) and
confirmatory factor analysis (CFI= 0.91; TLI= 0.99;
RMSEA= 0.04) provided support for a seven-factor
structure







Doctor Communication-Positive Behaviors (5 items)
Doctor Communication-Negative Behaviors (4 items)
Doctor Communication-Health Promotion ((4 items))
Doctor Communication-Alternative Medicine (2 items)
Shared Decision Making (2 items)
Equitable Treatment (2 items)
Trust (5 items)
22
Results

Multitrait scaling


Item-scale correlations above 0.30 for all items
Item discrimination


Items correlated more with their hypothesized scale than
with other scales
Internal consistencyy


Ranged from 0.58 for Doctor CommunicationAlternative Medicine to 0.92 for Doctor CommunicationPositive Behaviors
Exceeded 0.70 for four of the seven composites
23
Results

Multi-group
Multi
group CFA

General support for measurement
equivalence for Whites
Whites, Blacks and
Hispanics for:




Doctor Communication-Positive Behaviors (5 items)
D t Communication-Negative
Doctor
C
i ti N
ti B
Behaviors
h i
(4 it
items))
Doctor Communication-Health Promotion (4 items)
Trust (5 items)
24
Results

Psychometric analysis among non-English
non English
speakers provided support for one additional
domain


Access to Interpreter Services (5 items)
Regression results showed that all CAHPS CC
composites were positively and significantly
associated with overall doctor rating
25
Conclusions

The CAHPS CC item set





Assesses culturally competent care from the
patient’s perspective
Demonstrates adequate measurement properties
Addresses aspects of care that are important to
patients’ ratings of care
Health care organizations wanting to improve
their CAHPS ratings can implement quality
improvement to address CAHPS CC domains
R
Recommend
d th
the it
item sett as a supplemental
l
t l
module for the CAHPS health plan and
clinician and g
group
p survey
y instruments
26
Acknowledgements
g

Commonwealth Fund- field test and
dissemination


Project Officer- Melinda K. Abrams
AHRQ- development and cognitive testing


Project Officer- Chuck Darby
Project Staff- Ana Caponiti
Anne Beal- former Sr. Project Officer at
Commonwealth Fund
 Karen Bogen- member of CAHPS II Cultural
Comparability Team
 Other CAHPS II and CAHPS III teams

27