Overview of Presentation
Partnering with
Community in Survey
Design
• Context
– Problem of mistrust
– Role of community partnerships in solving this
problem
• Example: Disparities in Minnesota Health
Care Programs
– Principal Findings: Challenges and Rewards of
Partnership
Kathleen Thiede Call, Ph.D.
• Lessons learned and challenge to field
AcademyHealth, Seattle WA
June 27, 2006
Supported by a grant from the Minnesota Department of Human Services
2
Solution: Community “Involvement”
in Research
Problem: Mistrust
• Community mistrust of
“Community-based participatory research in
public health is a collaborative approach to
research that equitably involves…
community members, organizational
representatives, and researchers in all
aspects of the research process.” (Israel et
Medical/Health care system
Medical research
al., 2000)
Survey research
Academic
Research
Community
Based
3
4
Principal Findings: Challenges and
Rewards of Partnership
From Theory to Practice
• Project: Disparities and barriers to
• Constraints to participatory research
• Survey development
• Conceptual equivalence and interviewer
utilization among Minnesota health care
program enrollees
• Task: Creation and implementation of
statewide survey of program enrollees (e.g.,
Medicaid, MinnesotaCare) stratified by
race/ethnicity
quality
• Survey administration and response rates
5
6
1
Constraints: Project Timeline and
Milestones
Background research
Jan-Feb 03
Focus groups
Feb 03
Sample drawn
Feb 03
Instrument development
Feb-June 03
Survey translation
June 03
Data collection
July-Sep 03
Data cleaning, weighting
Sep 03
Data analysis/reporting
Sep-Nov 03
Final report submitted
Dec 03
Participatory Components of
Disparities Project
• Administrative structure
– Project Management Team
– Subcommittees
• Communication
• Resource distribution
7
8
Organizational Structure
of Research Team
Project Administration
• Project Management Team (PMT) comprised of
key academic/institutional and community
researchers charged with project oversight
Project
Management
Team
–
–
–
–
Stratis Health – Fiscal Agent (EQRO, federal match)
University of Minnesota – PI
The Urban Coalition
Community Researchers (Hmong, African American,
Somali, Hispanic, American Indian)
– DHS project director
Focus
Group
• Project subcommittees responsible for specific
Instrument
Development
Survey
Administration
Data
Analysis
Dissemination
project tasks
9
10
Communication
Resource Distribution
• Although limited, budget allowed for support of
• Continuous intentional communication between
academic/institutional and community researchers
PMT and subcommittee members
• Distribution flow:
– Monthly meetings of PMT, weekly meetings of
subcommittees
– Communication via telephone, e-mail
– Communication facilitated by overlapping
subcommittee membership
Stratis Health
University of Minnesota
• Focus on process as well as tasks
• Decision-making through consensus where
The Urban Coalition
(facilitated community contracting)
possible, given time constraints
11
12
2
Conceptual Equivalence and Interviewer
Quality
Survey Creation
• Focus groups to inform survey content
• Multi-lingual members participation in design of English
survey ensured conceptual equivalence in translation.
Two major problems in all 3 languages:
– Understand community definitions of prevention,
service seeking and barriers
– Translations too literal
• Mixed mode survey administration:
• Spanish: “Indian Health Center” word for “Indian” that could be confused
with “from India” instead of American Indian
• Hmong: “tus kws kho mob” (the one who cures diseases) instead of “doctor”
which is term used
– Created English mail version
– Translated for telephone follow-up: English, Spanish,
Somali, Hmong
– Face page included instructions in all 4 languages and
phone numbers for those wishing to complete phone
survey
– Errors
• Spanish back translation yielded “It could be said...” instead of “Would you
say…”[response options]. For example “In general, how would you rate your
overall health? Would you say it is excellent, very good, good, fair or poor?
• Hmong language-- multiple ways of saying “Yes/No” and the response code
selected must correspond to the question asked. Sonetimes instead of using
“Yes/No” the equivalent of “I believe/do not believe,” or “can do/cannot do,”
(Pab, Txhawj, Ua Tau, Mus, etc) is more appropriate.
• Somali translation yielded inappropriately “loaded” phrasing of questions or
simply did not capture meaning of question.
• Translation and pre-testing overseen by multilingual, multi-cultural PMT members
13
14
PMT Process and Focus Group Influence
on Survey
Expansion of Survey: Confidence in
Providers
22
• Expansions to survey
P le a s e i n d i c a t e t h e e x t e n t t o w h i c h y o u a g r e e o r
d i s a g r e e w i t h e a c h o f t h e f o ll o w in g s t a t e m e n t s :
– Sources of preventive care (turn to help keep from
getting sick)
• Expanded response categories beyond doctor or clinic
– Spiritual or traditional healer, shaman
– Chiropractor
– Acupuncturist or herbalist
• Beliefs: “To what extent do you agree or disagree with the following
statement: There is little doctors can do to keep you from getting
sick”
– Discrimination
• Worry about going to the doctor or clinic for a check-up because you
might get bad news
SOM EW HAT
A GREE
SOM EW HAT
D IS A G R E E
STRONGLY
D IS A G R E E
a
I a m a f r a id t h a t m y p r o v id e r
m ig h t n o t d o e n o u g h t o f in d
o u t w h a t is r e a lly m a k in g
m e s ic k
b
I a m a f r a id t h a t t h e h e a lt h
c a r e I r e c e iv e m ig h t a c t u a lly
m a k e m e w o rs e
c
I a m a f r a id t h a t m y p r o v id e r
m ig h t t e ll m e t h a t I h a v e a n
illn e s s t h a t I d o n ’t r e a lly
have
d
I a m a f r a id t h a t m y p r o v id e r
m ig h t f a il t o f in d a n illn e s s
th a t I d o h a v e
• Economic and class based-- Being enrolled in a Minnesota Health
Care Program such as Medicaid, Medical Assistance or
MinnesotaCare
– Barriers
STRONGLY
AGREE
15
16
Response and Cooperation Rates
Survey Administration
(overall response rate 54%)
• Hiring and interviewer quality control
overseen by multi-lingual, multi-cultural
PMT members
• Sample drawn February; Fielded April –
July 2003
• Initial mailing to 9,350 enrollees
Stratum
SRS
American
Indian
African
American
Hispanic/
Latino
Hmong
Somali
– De-duplication at household level
– All ages, with parent responding for sampled
child
Target
Mail
Completes
Mail +
Phone Phone
1856
528
Lower
response
rate
63.0%
42.9%
Lower
Coop.
rate
84.6%
80.7%
1400
600
1379
338
477
190
600
354
227
581
46.7%
78.0%
600
324
339
663
54.5%
85.0%
600
600
585
304
112
324
697
628
56.5%
50.4%
70.3%
75.0%
Total of 4,953 surveys completed
17
18
3
Summary of Value Added Through
Community Partnership
Lessons Learned in Two Areas
• Participatory process
• Mail survey in English only (Cost savings)
• Instrument design
– Lose a little….gain a lot!
– Mistakes are the most important part
– Conflict happens! Intentional communication allows us
to build relationships, partnerships and most
importantly TRUST
– Sharing power challenges everyone
– Content of survey somewhat expanded
– Accessible wording
• Quality of translations
• Forward momentum can help counter personal politics
– Involvement of multi-lingual, multi-cultural
researchers in survey design increased
conceptual equivalence
– Process is as important as product!
• Survey research community
– No need to compromise methodological rigor
– Improved quality and serves larger purpose—
ownership and application of results vs subjects of
research
• Quality of interviewing
– Hiring and training of interviewers
19
20
Collaborators
•
•
•
•
•
•
•
•
•
•
For More Information
Tim Beebe, University of Minnesota
Heather Britt, The Urban Coalition
Valeng Cha, Cha Consulting
Charity Kreider, University of Minnesota
Jennifer Lundblad, Stratis Health
Donna McAlpine, University of Minnesota
Jim McRae, Minnesota Department of Human Services
Betty Moore, Independent Consultant (formerly with the
Indian Health Board)
Sirad Osman, New Americans Community Services
Walter Suarez, Midwest Center for HIPAA Education
• Main page:
http://www.dhs.state.mn.us/HealthCare/pmqi/defa
ult.htm#Research-Evaluation
• Full report:
http://www.dhs.state.mn.us/HealthCare/pmqipdfs/Disparities-final-report-Dec2003.pdf
• Focus group report:
http://edocs.dhs.state.mn.us/lfserver/Legacy/DM0171-ENG
21
22
Potential Strengths of Participatory
Research
•
•
•
•
Builds or rebuilds trust with communities
Improves quality of research
Enhances research capacity
Empowering for all partners
– (e.g., decision-making, resources, use of results)
• Benefit from diverse perspectives/skills
• Relevance and applicability of research
23
24
4
Potential Weaknesses/Challenges of
Participatory Research
•
•
•
•
Threats to traditional scientific rigor
Lack of standardization
Time consuming endeavor
Research questions and methodology often
specified after project initiation
• Not always rewarded in an academic
environment
• Appropriate funding mechanisms difficult
25
5