Partnering with
Community in Survey
Design
Kathleen Thiede Call, Ph.D.
AcademyHealth, Seattle WA
June 27, 2006
Supported by a grant from the Minnesota Department of Human Services
Overview of Presentation
• Context
– Problem of mistrust
– Role of community partnerships in solving this
problem
• Example: Disparities in Minnesota Health
Care Programs
– Principal Findings: Challenges and Rewards of
Partnership
• Lessons learned and challenge to field
2
Problem: Mistrust
• Community mistrust of
Medical/Health care system
Medical research
Survey research
3
Solution: Community “Involvement”
in Research
“Community-based participatory research in
public health is a collaborative approach to
research that equitably involves…
community members, organizational
representatives, and researchers in all
aspects of the research process.” (Israel et
al., 2000)
Academic
Research
Community
Based
4
From Theory to Practice
• Project: Disparities and barriers to
utilization among Minnesota health care
program enrollees
• Task: Creation and implementation of
statewide survey of program enrollees (e.g.,
Medicaid, MinnesotaCare) stratified by
race/ethnicity
5
Principal Findings: Challenges and
Rewards of Partnership
• Constraints to participatory research
• Survey development
• Conceptual equivalence and interviewer
quality
• Survey administration and response rates
6
Constraints: Project Timeline and
Milestones
Background research
Jan-Feb 03
Focus groups
Feb 03
Sample drawn
Feb 03
Instrument development
Feb-June 03
Survey translation
June 03
Data collection
July-Sep 03
Data cleaning, weighting
Sep 03
Data analysis/reporting
Sep-Nov 03
Final report submitted
Dec 03
7
Participatory Components of
Disparities Project
• Administrative structure
– Project Management Team
– Subcommittees
• Communication
• Resource distribution
8
Project Administration
• Project Management Team (PMT) comprised of
key academic/institutional and community
researchers charged with project oversight
–
–
–
–
Stratis Health – Fiscal Agent (EQRO, federal match)
University of Minnesota – PI
The Urban Coalition
Community Researchers (Hmong, African American,
Somali, Hispanic, American Indian)
– DHS project director
• Project subcommittees responsible for specific
project tasks
9
Organizational Structure
of Research Team
Project
Management
Team
Focus
Group
Instrument
Development
Survey
Administration
Data
Analysis
Dissemination
10
Communication
• Continuous intentional communication between
PMT and subcommittee members
– Monthly meetings of PMT, weekly meetings of
subcommittees
– Communication via telephone, e-mail
– Communication facilitated by overlapping
subcommittee membership
• Focus on process as well as tasks
• Decision-making through consensus where
possible, given time constraints
11
Resource Distribution
• Although limited, budget allowed for support of
•
academic/institutional and community researchers
Distribution flow:
Stratis Health
University of Minnesota
The Urban Coalition
(facilitated community contracting)
12
Survey Creation
• Focus groups to inform survey content
– Understand community definitions of prevention,
service seeking and barriers
• Mixed mode survey administration:
– Created English mail version
– Translated for telephone follow-up: English, Spanish,
Somali, Hmong
– Face page included instructions in all 4 languages and
phone numbers for those wishing to complete phone
survey
• Translation and pre-testing overseen by multilingual, multi-cultural PMT members
13
Conceptual Equivalence and Interviewer
Quality
• Multi-lingual members participation in design of English
survey ensured conceptual equivalence in translation.
Two major problems in all 3 languages:
– Translations too literal
• Spanish: “Indian Health Center” word for “Indian” that could be confused
with “from India” instead of American Indian
• Hmong: “tus kws kho mob” (the one who cures diseases) instead of “doctor”
which is term used
– Errors
• Spanish back translation yielded “It could be said...” instead of “Would you
say…”[response options]. For example “In general, how would you rate your
overall health? Would you say it is excellent, very good, good, fair or poor?
• Hmong language-- multiple ways of saying “Yes/No” and the response code
selected must correspond to the question asked. Sonetimes instead of using
“Yes/No” the equivalent of “I believe/do not believe,” or “can do/cannot do,”
(Pab, Txhawj, Ua Tau, Mus, etc) is more appropriate.
• Somali translation yielded inappropriately “loaded” phrasing of questions or
simply did not capture meaning of question.
14
PMT Process and Focus Group Influence
on Survey
• Expansions to survey
– Sources of preventive care (turn to help keep from
getting sick)
• Expanded response categories beyond doctor or clinic
– Spiritual or traditional healer, shaman
– Chiropractor
– Acupuncturist or herbalist
• Beliefs: “To what extent do you agree or disagree with the following
statement: There is little doctors can do to keep you from getting
sick”
– Discrimination
• Economic and class based-- Being enrolled in a Minnesota Health
Care Program such as Medicaid, Medical Assistance or
MinnesotaCare
– Barriers
• Worry about going to the doctor or clinic for a check-up because you
might get bad news
15
Expansion of Survey: Confidence in
Providers
22 Please indicate the extent to which you agree or
disagree with each of the following statements:
STRONGLY
AGREE
SOMEWHAT
AGREE
SOMEWHAT
DISAGREE
STRONGLY
DISAGREE
a I am afraid that my provider
might not do enough to find
out what is really making
me sick




b I am afraid that the health
care I receive might actually
make me worse




I am afraid that my provider
might tell me that I have an
illness that I don’t really
have




d I am afraid that my provider
might fail to find an illness
that I do have




c
16
Survey Administration
• Hiring and interviewer quality control
overseen by multi-lingual, multi-cultural
PMT members
• Sample drawn February; Fielded April –
July 2003
• Initial mailing to 9,350 enrollees
– De-duplication at household level
– All ages, with parent responding for sampled
child
17
Response and Cooperation Rates
(overall response rate 54%)
Stratum
SRS
American
Indian
African
American
Hispanic/
Latino
Hmong
Somali
Completes
Mail +
Mail Phone Phone
1400
600
1379
338
477
190
1856
528
Lower
response
rate
63.0%
42.9%
600
354
227
581
46.7%
78.0%
600
324
339
663
54.5%
85.0%
600
600
585
304
112
324
697
628
56.5%
50.4%
70.3%
75.0%
Target
Lower
Coop.
rate
84.6%
80.7%
Total of 4,953 surveys completed
18
Summary of Value Added Through
Community Partnership
• Mail survey in English only (Cost savings)
• Instrument design
– Content of survey somewhat expanded
– Accessible wording
• Quality of translations
– Involvement of multi-lingual, multi-cultural
researchers in survey design increased
conceptual equivalence
• Quality of interviewing
– Hiring and training of interviewers
19
Lessons Learned in Two Areas
• Participatory process
– Lose a little….gain a lot!
– Mistakes are the most important part
– Conflict happens! Intentional communication allows us
to build relationships, partnerships and most
importantly TRUST
– Sharing power challenges everyone
• Forward momentum can help counter personal politics
– Process is as important as product!
• Survey research community
– No need to compromise methodological rigor
– Improved quality and serves larger purpose—
ownership and application of results vs subjects of
research
20
Collaborators
•
•
•
•
•
•
•
•
•
•
Tim Beebe, University of Minnesota
Heather Britt, The Urban Coalition
Valeng Cha, Cha Consulting
Charity Kreider, University of Minnesota
Jennifer Lundblad, Stratis Health
Donna McAlpine, University of Minnesota
Jim McRae, Minnesota Department of Human Services
Betty Moore, Independent Consultant (formerly with the
Indian Health Board)
Sirad Osman, New Americans Community Services
Walter Suarez, Midwest Center for HIPAA Education
21
For More Information
• Main page:
http://www.dhs.state.mn.us/HealthCare/pmqi/defa
ult.htm#Research-Evaluation
• Full report:
http://www.dhs.state.mn.us/HealthCare/pmqipdfs/Disparities-final-report-Dec2003.pdf
• Focus group report:
http://edocs.dhs.state.mn.us/lfserver/Legacy/DM0171-ENG
22
23
Potential Strengths of Participatory
Research
•
•
•
•
Builds or rebuilds trust with communities
Improves quality of research
Enhances research capacity
Empowering for all partners
– (e.g., decision-making, resources, use of results)
• Benefit from diverse perspectives/skills
• Relevance and applicability of research
24
Potential Weaknesses/Challenges of
Participatory Research
•
•
•
•
Threats to traditional scientific rigor
Lack of standardization
Time consuming endeavor
Research questions and methodology often
specified after project initiation
• Not always rewarded in an academic
environment
• Appropriate funding mechanisms difficult
25