How to Get the Most Valid Data
from Hard-to-Reach Populations
Miriam Boeri, David Gibson, Craig Rafuse, Ben Tyndall
Kennesaw State University
Suburban Conference 2010
Funding for this research was provided by the National Institute on Drug Abuse
1R21DA025298-01A1
University of Pennsylvania Arts and Sciences. 2009. “Next Door But Invisible: The World of Homelessness and
Drug Addiction.” YouTube Web site. Retrieved April 23, 2010. (http://www.youtube.com/v/lT1PjDvL6cs).
Study
Maturing Out Theory
Study on Older Drug
Users
© 2009 Focus 12 “Old Junkies” ad campaign
http://www.focus12.co.uk/oldjunkies
Methods
Ethnographic fieldwork
Face to face interviews
Quality control
Keith Richards
Sample
100 older adults 45
and older in 2010
Sampling
– Convenience
– Snowball
– Theoretical
IRB approved
Certificate of
Confidentiality
Consent
2-5 hour interview
Craig Rafuse
© Bill Waterson
Miriam Boeri, PhD
David Gibson
Ben Tyndall
Recruiting Hard to Reach
and Hidden Populations
Primary Recruitment - Researcher Initiated
Secondary Recruitment - Referral
Tertiary Recruitment - Passive
Networks
Family
Friends
Coworkers
Former Users
Gatekeepers
Individuals who grant access to a larger
network of users
William S. Burroughs
1914-1997
Streets, Clubs, and Pubs
Gain acceptance of manager,
owner, or promoter
TALK TO EVERYONE
Fliers
Must be IRB
approved
Approval of the
property owner
or manager
Community Outreach, Ministries, and Harm
Reduction Centers
Methadone Clinics
Atlanta Harm Reduction Center – “The Bluff”
Must Ministries – Marietta
Homeless Shelters
Needle Exchanges
Screening
How old are you?
How did you hear about the survey?
What is/was your drug of choice?
When was the last time you used a substance?
What year were you born?
“Fluff questions” or misdirecting questions
Gaining the Confidence of
Respondents
Professional/academic association
Certificate of confidentiality
Secure storage of data
Street cred
Dress down
Alfonso Ribeiro as Carlton Banks. The Fresh Prince of Bel-Air.
© NBC Productions.
The Interview Process: Juggling
Time, Energy and Research Goals
ar
e
Qui ckTi m
e™
a nd
a
d eco mp r
e sso r
ne e de d
to
see
thi s
p i cture.
What We Juggle
High Energy
^
^
Time > > > > > > > > > Research goals
^
^
Low Energy
© R. Crumb
Setting the Stage
Tuning In
Cordial Atmosphere
Respectful Attitude and
Tone of Voice
Soliciting Their Consent
Putting Them in the
Driver’s Seat
The Qualitative Discussion
Turning Points Matrix
Letting It Happen or
Strict Guidance?
Use Confrontation
Carefully
Not An Interrogation
The Quantitative Survey
What, Why, Where,
How and When
Cloning Years
Keeping Them
Focused
Collaboration
Gift Exchange
QuickTime™ and a
decompressor
are needed to see this picture.
Management and
Quality Control of the Data
Data management
– Survey creation and revision
– Technology
Quality control
– Survey data
– Interview recordings and transcripts
Original Survey
Long-form survey covered every year
– Every year or missing data?
– Cumbersome and time consuming
Shorter survey still needs to capture entire
life
Born
Started
injecting
heroin
Started
smoking
cigarettes
First child
Married
12 Step
Residential
treatment
Relapse
Divorce
Clean
Yearly Survey
Advantages:
New survey for every year
Skip forward and backward
Speed of survey tends to
outweigh disadvantages
Survey Development
Mandatory responses
Question revisions
Survey has some fluidity
Technology
Netbooks
Recorders
Encrypted USB drives
Quality Control - Survey
“Create Notes”
Some problems
Quality control
Review interviews
Quality Control - Survey
Export data to SPSS
Visualize drug/life history
Check for mistakes/inconsistencies
Quality Control - Survey
Data triangulation
INTERVIEWER
NOTES
SURVEY
DATA
RECORDING
Quality Control - Transcription
Transcribers have
a tough job
Interviewer listens
to recording and
follows along with
transcript
Final Thoughts on Quality Control
Relive/re-experience interviews
Rigorous triangulation
Strengthens team skills
Protects our investment of time, energy
and money in each interview
Conclusion
Targeted recruitment for different types of
users
Finding a balance between quantitative
and qualitative
Time, space, and emotions
Validity and reliability
Keeping it organized
QUESTIONS?