Gathering Credible Data
Seattle Indian Health Board
Urban Indian Health Institute
Shayla R. Compton, MPH
University of Washington School of Public Health
Objectives of Session
• Describe how data collection fits into program evaluation
• Discuss general principles of data collection
• Discuss considerations for selecting the right data
collection methods for you
• Overview of qualitative and quantitative data collection
methods
Urban Indian Health Institute, a division of the Seattle Indian Health Board
Evaluation
Program
Improvement
A systematic way of
collecting information
about the
characteristics,
activities, products,
and outcomes of a
program.
Requirements
Accountability
Evaluation
Public
Relations
Fundraising
Decision
Making
Patton, M.Q. (2008) Utilization-focused evaluation (4th edition).
Thousand Oaks, CA: Sage.
Urban Indian Health Institute, a division of the Seattle Indian Health Board
Evaluation Framework
Engage
stakeholders
Ensure use
and share
lessons
learned
Describe the
program
Justify
conclusions
Focus the
evaluation
design
Gather
credible
evidence
http://www.cdc.gov/eval/framework/index.htm
Urban Indian Health Institute, a division of the Seattle Indian Health Board
Data Collection Principles
•
•
•
•
•
Respect for clients
Clear and honest communication
Ensure the evaluation is culturally responsive
Address all questions or concerns
Share the data with participants and stakeholders
Urban Indian Health Institute, a division of the Seattle Indian Health Board
Data Collection Considerations
What information is
needed?
Where is the
information?
How much time is
needed/available?
What resources are
available?
Taylor-Powell, E. Planning a Program Evaluation (1996)
Urban Indian Health Institute, a division of the Seattle Indian Health Board
Qualitative vs. Quantitative
Qualitative Data:
• Clean and tidy
• Bright lights
Quantitative Data:
• Number of staff
• Proportion of patients
that are elders
Urban Indian Health Institute, a division of the Seattle Indian Health Board
Qualitative Methods
Interviews
Creative Expression & Storytelling
Focus Groups
Participant Observation
Urban Indian Health Institute, a division of the Seattle Indian Health Board
Quantitative Methods
Observation Checklists
Tests
Document Review
Biomedical Markers
Urban Indian Health Institute, a division of the Seattle Indian Health Board
Surveys
Mixed Method Approach
Qualitative
Method
Focus
Groups
Credible
Data
Gathered
Quantitative
Method
Interviews
Document
Review
Urban Indian Health Institute, a division of the Seattle Indian Health Board
Surveys
Mixed Method Approach:
Triangulation
Quantitative
Data &
Results
Interpretation
Urban Indian Health Institute, a division of the Seattle Indian Health Board
Qualitative
Data &
Results
Selecting the Best Method
Evaluation Question
What we ask
Indicators
What we examine
Data Collection Method
How we collect the information to answer our question
Urban Indian Health Institute, a division of the Seattle Indian Health Board
Practice & Review
Process Objective: Recruit 150 youth to
participate in GONA.
Evaluation Question
• How effective were outreach efforts in engaging youth in
GONA?
Indicators
• Attendance
• Where participants heard about the event
• Types of outreach conducted
• Number of flyers posted, listserv postings
Urban Indian Health Institute, a division of the Seattle Indian Health Board
Practice & Review
•
•
•
•
Urban Indian Health Institute, a division of the Seattle Indian Health Board
Attendance logs
Summary reports
Outreach materials
Social media activity
Practice & Review
Outcome Objective: 30 days after discharge, clients
released from residential substance abuse treatment
will report implementing stress management skills
not involving alcohol or substance abuse.
Evaluation Question
• How are clients managing stress post discharge?
Indicators
• Identified coping skills for managing stress
• Perceived strengths and weaknesses of programs offered posttreatment
Urban Indian Health Institute, a division of the Seattle Indian Health Board
Practice & Review
• Multiple
perspectives
• Direct feedback
Urban Indian Health Institute, a division of the Seattle Indian Health Board
Summary
• Respect for clients & community
• Choose the method that best fits your evaluation
question, the source of information, and
resources (financial, expertise, time, etc.).
Urban Indian Health Institute, a division of the Seattle Indian Health Board
Questions
Contact Julie Loughran at juliel@uihi.org or 206.812.3042 to
receive individualized consultation on program evaluation.
Urban Indian Health Institute, a division of the Seattle Indian Health Board
26
Data Collection Resources
American Indian Higher Education Consortium
http://indigeval.aihec.org/Pages/Documents.aspx
Creative Narrations
http://www.creativenarrations.net/who
National Science Foundation
http://www.nsf.gov/pubs/2002/nsf02057/nsf02057_4.pdf
Northwest Center for Public Health Practice
https://www.nwcphp.org/training
University of Wisconsin-Extension
http://learningstore.uwex.edu/assets/pdfs/g3658-4.pdf
Urban Indian Health Institute, a division of the Seattle Indian Health Board
Urban Indian Health Institute
Seattle Indian Health Board
606-12th Avenue South
Seattle, WA 98114
Phone: (206) 812-3030
Fax: (206) 812-3044
Email: info@uihi.org
Website: www.uihi.org