Beyond logical frameworks to
program impact pathways
CIIFAD M&E Workshop
5 November 2011  135 Emerson Hall
Sunny S. Kim, MPH
PhD candidate
Division of Nutritional Sciences
Role of research in evaluation
• Evaluation usually focuses on an internal situation,
such as collecting data about specific programs, with
no intent to generalize the results to other settings and
situations. Evaluation determines the merit, worth, or
value of things.
• Evaluation asks what should be, which makes reference
to the standards and criteria of evaluations.
• Program-related research usually conducted with to
the intent to generalize the findings from a sample to a
larger population.
• Research asks why, which indicates a theoretical basis.
Basic steps of evaluation
• Identifying program goals and objectives
• Mapping the program
• Selecting an evaluation design
– Sample size, control group, sampling frame
• Selecting indicators
• Determining data collection strategy
• Developing data collection instruments
• Analyzing the data
• Communicating the results
Program theory vs. logic model
• Program theory explains why a program is expected
to work, and a logic model illustrates a program
theory.
• Program logic model is a picture of how your
organization does its work… it links the theoretical
assumptions of the program
and program activities/
processes and outcomes.
Translating program theory…
Program theory has also been referred to as:
•
•
•
•
•
program logic or intervention logic
theory-based or theory-driven evaluation
theory of change
theory-of-action
impact pathway analysis
all ways of developing a causal modal, linking program
inputs and activities to intended or observed outcomes
Translating program theory…
• Program theories are often captured in a series of
“if-then” statements.
• And between the “if” and “then,” there should be
“how” and “why”, i.e. evidence or well-established
connections (established theory, previous results,
published research, feedback from participants).
IF…
THEN…
IF…
THEN…
+ CONTEXT
… into logic models
• a commonly-used tool for illustrating program theory.
• a picture of describing the sequence of activities to
bring about change and how these activities are linked
to the results the program is expected to achieve.
• often presented in a flow chart that illustrates the
linkages between program components and outcomes:
Inputs
Resources, supplies, staff
Activities
Processes, techniques, tools, events, technology, actions
Outputs
Direct results of activities (size and/or scope)
Outcomes Changes in attitudes, behaviors, knowledge, skills, status,
or level of functioning
Impact
Organizational, population, community, and/or system
level changes (improved conditions, increased capacity,
and/or policy changes)
Program Logic Model Template
… into logic models
• “Logic models come in as many sizes and shapes as the
programs they represent.” [W.F. Kellogg Foundation, 2001, p.7]
• No best logic model, only the model that fits your
program best and provides the information you need
in the format that is most helpful.
Why use a logic model?
• Design and planning  helps to explain and illustrate
program concepts and approach for key stakeholders
(advocate and train); finds “gaps” in the theory or logic of
a program and work to resolve them .
• Implementation and management  focuses attention
on the most important connections between actions and
results for monitoring, improved programming, and
reporting.
• Evaluation, communication,
and marketing  presents progress
toward goals; provides a way to involve
and engage stakeholders in the design,
processes, and use of evaluation.
Why use a logic model?
• Donor requirements:
Log frame
(OECD/DAC, USAID, CIDA…)
Results framework
(USAID…)
Goals
Objectives
Outputs
Activities
Resources
Assumptions
Means of
verification
Indicators
Goal: Health status improved
Strategic Objective: Use of key
services and behaviors improved
Intermediate
Result 1:
Access and
availability of
services and
supplies
increased
Intermediate
Result 2:
Quality of
services
improved
Intermediate
Result 3:
Demand for
key services
and behaviors
increased
Intermediate
Result 4:
Social and
policy
environment
enabled
Logic model example
Intervention model of
Programa de Nutrición Infantil (PNI)
COMPONENTS
INTERMEDIATE
OUTCOMES
Maternal and child
health and nutrition
Basic sanitation and
sanitary education
Strategies:
Information
Education
Social
communication
Improved health and
nutrition knowledge and
practices
Increased access to health
services
Strengthening of
civil society
Improved basic sanitation
Small-scale
economic activities
Improved development
plans of CBOs in health
and sanitation
FINAL
OUTCOMES
Improved health
and nutritional
status of children
<3 years
Improved health
and nutritional
status of pregnant
women
Results framework of
Programa de Nutrición Infantil (PNI)
GOAL: Reduced rates of child malnutrition
Strategic Objective: Improved health and nutritional status of
children <3 years and pregnant women
Intermediate
Result 1:
Improved nutrition,
health knowledge
and practices of
those attended in
the target
communities
Intermediate
Result 2:
Increased access to
health services of
the target population
Intermediate
Result 3:
Improved basic
sanitation in the
target communities
Intermediate Result 4:
Community-based
organizations applied
their improved
knowledge on health and
sanitation in the
formulation of their
development plans
From boxes to arrows…
• While “filling in” the logic models with the
program components , we must not neglect
the mechanisms and linkages… don’t forget
about your program theory or impact
pathways.
Description of mothers workshops
“The theme/issue is the training that you give through the
participatory methdology to the mothers. If you just give the
theory, it is the theme. The workshop is the practice. So if
that day, for example, the mothers didn’t bring food to
prepare their workshop, then the [health] promoter only
gives the theory. They discuss, socialize, and receive only the
theories in the small groups… Then the workshop returns
another day to learn about doing. For example, the health
themes, they make their workshop do dramatizations. The
mothers do their dramatizations, it tells the priority, what
happens and everything, and so they are going to learn.”
[Regional level, ex-ADRA adviser]
Impact pathway of mothers workshops
Household food
availability and
access
Husbands
permit/ support
women to
participate
CHAs
convene
pregnant
women and
mothers of
children <3
years to
participate in
workshops
Women see
incentive to
participate
(e.g. food
baskets or
small
economic
activities)
CHAs raise
topic of child
feeding (i.e., BF
and CF;
balanced diets;
feeding sick
children) and
nutrition for
pregnant
women
CHAs raise
topic of health
and care for
children (i.e.,
diarrhea and
infections;
respiratory
illness; early
stimulation) and
reproductive
health and
prenatal care
CHA raise topic
of hygiene and
sanitation
CHAs raise
topic of self
esteem and
family violence
Influencing
factors –
facilitators
CHAs and
mothers
discuss
ideas and
experiences
CHAs and
mothers
demonstrate
recipes
Use of local
language
and readily
accessible
foods
CHAs and
mothers
discuss
ideas and
experiences
CHAs and
mothers
discuss
ideas and
experiences
CHAs and
mothers
discuss
ideas and
experiences
CHAs and
mothers
perform
dramatizations
CHAs and
mothers
demonstrate
practices
CHAs and
mothers
perform
dramatizations
Mothers
understand
nutritional
needs of
children and
pregnant
women and
know feeding
and cooking
practices
Mothers
understand
child health,
reproductive
health and
prenatal care
issues and
know
appropriate
practices
?
Mothers
understand
issues of
hygiene and
sanitation and
know
appropriate
practices
Mothers
understand
issues of self
esteem and
violence and
know how to
address them
Mothers
recall
nutritional
needs and
feeding and
cooking
practices
Repeated reminders
or internal/external
stimulus (e.g. home
visits, concern for
child’s future)
Mothers
recall child
health,
reproductive
health and
prenatal care
issues and
appropriate
practices
Mothers
implement
feeding
practices and
recipes
Quality health
service access
and availability
Mothers
implement
care
practices and
seek health
services
Children consume
adequate and
appropriate foods
Pregnant women
consume adequate
and appropriate
foods
Pregnant women
receive appropriate
care and health
attention in a timely
manner
Children receive
appropriate care
and health
attention in a timely
manner
Household
access to
clean water
Mothers
recall
information
on hygiene
and
sanitation
Mothers
implement
hygiene and
sanitation
practices
Mothers
recall
information
on self
esteem and
violence
Mothers
implement
practices
related to
self esteem
and violence
Children exposed
to safe and clean
home
environments and
less exposure to
contamination
Children exposed
to safe and caring
home
environments
Husbands are
sensitized about
domestic violence
Nutritional
status of
pregnant
women
improved
Child
nutrition
and
growth
improved
Main references
• Kellogg Foundation. (2004). Logic model development guide:
Using logic models to bring together planning, evaluation, and
action. Battle Creek, MI: W.K. Kellogg Foundation.
• Rogers, P.J. (2000). Program theory: Not whether programs
work but how they work. In D.L. Stufflebeam, G.F. Madaus, &
T. Kellaghan (Eds), Evaluation models, 209-232. Boston, MA:
Kluwer Academic Publishers.
• Rogers, P.J. (2008). Using program theory to evaluate
complicated and complex aspects of interventions. Evaluation,
14(1): 29-48.
• Taylor-Powell, E. & Henert, E. (2008). Developing a logic
model: teaching and training guide. Madison, WI: University of
Wisconsin-Extension.