Goals, Outcomes and
Program Evaluation
Community Memorial Foundation
March 5, 2014
Agenda
•
•
•
•
•
Introductions
Logic Models
Goals, Outcomes, Baselines and Results
Implementation and Outcomes Evaluation
Q and A
Session Objectives
• To define terms used in CMF responsive
grants application regarding program
outcomes and evaluation
• To introduce tools and frameworks to
facilitate the learning of program
evaluation
Presenters
• Tom Fuechtmann
Program Officer, Community Memorial Foundation
tfuechtmann@cmfdn.org
• Jon Korfmacher, PhD
Associate Professor, Erikson Institute
jkorfmacher@Erikson.edu
Where Program Evaluation is helpful
• Understand, verify or increase the impact
of services on clients
• Improve delivery mechanisms to be more
efficient and less costly
• Verify that you're doing what you think
you're doing
• Facilitate management really thinking
about what their program is all about,
including goals, how it meets goals and
knowing if it has met its goals or not.
Where Program Evaluation is helpful
• Produce data or verify results that can be
used for public relations and promoting
services in the community.
• Produce valid comparisons between
programs to decide which should be
retained, e.g., in the face of pending
budget cuts.
• Fully examine and describe effective
programs for duplication elsewhere
Logic Models
A way to summarize how your program
works: a “picture” of your program.
Note: There is no one “correct” way to
develop a logic model
* Logic models must be submitted for
requests over $25,000
How is a logic model used?
• Logic Models have multiple functions:
– Program planning
– Program management
– Evaluation
– Communication
• Within your agency
• With people outside your agency
Logic Model Components
Inputs
(What we invest)
Outputs
Activities
(What we do)
Outcomes:
Short-term
Participants
(Who we serve)
(Expect)
Assumptions:
What is necessary for this to happen?
Outcomes: Outcomes:
Medium
Long-term
(Want)
(Hope)
External Factors:
What else may impact outcomes?
Logic Model Components
Inputs
(What we invest)
• Resources dedicated to or
consumed by the project
–
–
–
–
Staff or volunteer resources
Funding
Equipment
Materials or other resources
Logic Model Components
Inputs
(What we invest)
Outputs
Activities
(What we do)
Participants
(Who we serve)
Ex: 10-week
parent training
session for teen
parents
Ex: Teen parents
attend at least
7/10 weeks of
parent training
Logic Model Components
Inputs
(What we invest)
Outputs
Activities
(What we do)
Outcomes:
Short-term
Participants
(Who we serve)
(Expect)
Outcomes: Outcomes:
Medium
Long-term
(Want)
(Hope)
• What changes do we expect to see right away?
• What changes would we want to see after that?
• What changes would we hope to see after that?
Ex: College HIV prevention
Short Term
College students
have knowledge
of how HIV/AIDS
is spread
Intermediate
College students
practice safe sex
Long Term
Lowered
incidences of HIV
cases on college
campuses
Closer in Time
More distant in time
Easier to
Measure
Harder to measure
More attributable
to Project
Less attributable to
project
Congratulations!
You now have a logic model!
(Now what?)
Remember…
1. Logic models are working documents
• Responsive programs need evolving models
• They can and should be open to change
2. The process is as important as the
product
3. Your logic model is only the first step
of an evaluation
CMF Application
Application continued
Application continued
• Goal: What you hope to achieve with this program (ex.
To Reduce Homelessness)
• Outcomes: Changes that will occur as a result of your
work (short term, intermediate, long term). Ex: 20% of
those receiving services will be in permanent housing 6
months after graduating from program.
• Baseline: What is the current level of care? Ex. Only 2%
are in permanent housing after a period of 6 months.
Evaluating Outcomes: What
changes have occurred?
• Outcomes: the changes you expect to
see as a result of your work
• Indicators: the specific way you measure
an outcome. In other words, “How will I
know it?”
Outcome Examples
Short Term
Intermediate
What do we Expect to
Occur?
What do we Want to
occur?
Seniors increase
knowledge of food
contamination risks
Participants learn jobseeking skills
Teen mothers report
negative attitudes
towards smoking while
pregnant
Seniors practice safe
food handling
Long Term
What do we Hope will
occur?
Seniors experience
lowered incidence of
food borne illness
Outcomes: Common types of
Indicators
• New knowledge
• Changed decisions
• Increased skills
• Modified behavior
• Changed attitudes,
opinions or values
• Changed policies
• Changed motivation
or aspirations
• Changed conditions
Ex: HIV/AIDS Prevention
Outcome
Indicator
Intermediate: Students
practice safe sex
Poor: Number of college
students in program
over time.
Better: Percent students
who talk about illness
with friends
Strong: Percent
students reporting
regular condom use
Long Term: Number
HIV/AIDS cases
decrease on college
campus
Evaluating Outcomes:
Indicator Statements
Elements of a Strong Statement:
– How much?
– Who?
– What ?
– When?
Evaluating Outcomes:
Indicator Statements
Example:
“ 75% of program participants find parttime employment within six months
of enrollment.”
But…
Have a rationale for the indicator
(e.g., why 75%?)
Collecting Evaluation Information
Questions to Ask Yourself
– What resources do you have?
– Quantitative or Qualitative?
– Standardized or non-standardized?
What makes the most sense for your program
and the outcome in question?
CMF and Evaluation
Outcome Objectives and Program Review
– What is the impact and how will this move
CMF’s mission forward?
– What are the chances those results will be
achieved?
– Given the opportunities before us, is this the
best use of the Foundation’s assets?
CMF and Evaluation (continued)
• Grantees as Partners
• Trust
• Open Dialogue
• A Learning Foundation
Questions and Answers
Additional Resources
• Innovation Network
www.innonet.org
• Free Management Library
www.managementhelp.org/evaluatn/fnl_eval.htm
• University of Wisconsin- Extension (Logic
Model) http://www1.uwex.edu/ces/lmcourse/.