Really Using
(Useful)
Theories of Change
IPDET 2013
John Mayne, Advisor on Public Sector Performance
john.mayne@rogers.com
Using Theories of Change
1. Understanding and agreeing on interventions
2. Addressing equity issues
3. Designing interventions
4. Ex ante evaluation
5. Designing monitoring systems
6. Managing adaptively
7. Reporting performance
8. Designing evaluation questions, methods and tools
9. Making causal claims
10. Generalizing to other locations
A Generic Results Chain
Impacts
Direct
Benefits
External
Influences
Behaviour
changes
Capacity changes in
knowledge, attitudes
skills, opportunities &
incentives
Reach &
Reaction
Timeline
Activities and Outputs
with respect to
beneficiaries
A Generic Results Chain
Impacts
Direct
Benefits
External
Influences
Behaviour
changes
Capacity changes in
knowledge, attitudes
skills, opportunities &
incentives
Reach &
Reaction
Timeline
Activities and Outputs
with respect to
beneficiaries
Note
• No labeling of
outcomes types
• Focus on program
theory
• Timeline
• Built on capacity and
practices changes
• Explicit recognition
of reach
• Feedback loops
Assumptions and Risks
Impacts
Direct
Benefits
External
Influences
Behaviour
changes
Capacity changes in
knowledge, attitudes
skills, opportunities &
incentives
Reach &
Reaction
Timeline
Activities and Outputs
with respect to
beneficiaries
Behaviour Change
Assumptions: How are changes in
knowledge, attitudes, skills,
opportunities and/or incentives
expected to change behaviour? What
events and conditions have to
happen? What factors influence these
processes?
Risks: Risks to the link not occurring.
Other Explanatory Factors: Peer or
trend pressure; other interventions
An assumption is NOT a
description of the causal
link
A Generic Beneficiaries Theory of Change
Impacts
Impact Assumptions
& Risks
Direct
Benefits
Other explanatory factors
Direct Benefits
Assumptions & Risks
External
Influences
Behaviour
changes
Other explanatory factors
Behaviour Change
Assumptions & Risks
Capacity changes in
knowledge, attitudes
skills, opportunities &
incentives
Other explanatory factors
Capacity Change
Assumptions & Risks
Reach &
Reaction
Other explanatory factors
Reach Assumptions &
Risks
Timeline
Activities and Outputs
with respect to
beneficiaries
Results Chain for Enhancing Education Outcomes for Girls
Improved
education
outcomes for girls
Girls learning
improves
Girls more engaged
in and wanting an
education
External Influences
Parents
Peers
Religion
Culture
Education resources
Teachers provide girls
with more empathetic
and supportive
teaching in schools
Capacity changes in
teachers’ knowledge,
attitudes and skills
All relevant
teachers get the
training
timeline
Awareness and skills
training provided to
teachers on girls’
education needs
Intervention to improve
educational outcomes for
girls by providing gender
sensitivity training to
teachers, in a region where
girls education has not been
a priority.
Other Factors for Enhancing Education Outcomes for Girls
Girls learning
improves
Girls more
engaged in and
wanting an
education
Teachers provide
girls with more
empathetic and
supportive teaching
in schools
• Parents and
community support
• Time for studying
• Access to schools
• Accommodation in
schools
Theory of Change for Enhancing Education Outcomes for Girls
Improved
education
outcomes for girls
Girls learning
improves
Unintended effect: Boys’
education suffers
Girls more engaged
in and wanting an
education
External Influences
Parents
Peers
Religion
Culture
Education resources
Teachers provide girls
with more empathetic
and supportive
teaching in schools
Capacity changes in
teachers’ knowledge,
attitudes and skills
All relevant
teachers get the
training
timeline
Awareness and skills
training provided to
teachers on girls’
education needs
Girls’ Direct Benefit Assumptions &
Risks
Girls have time to study
Girls education supported by parents and
community
Focus on girls is not disruptive to boy’s
education
Other explanatory factors
Internet access becomes widely available
Girls’ Behaviour
Assumptions & Risks
Girls have access to schools
Schools can adequately
accommodate girls
Other explanatory factors
Internet access becomes widely
available
Other factors
Seems clear that more than training teachers
may be required, such as:
• Getting girls to school
• Parental and community support
• Schools accommodate girls
• Teachers support girls education
• Empathetic teaching is rewarded
• Focus on and resources for girls’ education are
ongoing (outcomes take time)
Theory of Change for Enhancing Education Outcomes for Girls
Improved
education
outcomes for girls
Impact Assumptions & Risks
Girls learning
improves
Unintended effect: Boys’
education suffers
Girls more engaged
in and wanting an
education
External Influences
Parents
Peers
Religion
Culture
Education resources
Teachers provide girls
with more empathetic
and supportive
teaching in schools
Capacity changes in
teachers’ knowledge,
attitudes and skills
 Resources and will continue for
multi-year involvement
Girls’ Direct Benefit Assumptions &
Risks
 Girls have time to study
 Girls education supported by parents and
community
 Focus on girls is not disruptive to boy’s
education
Other explanatory factors
 Internet access becomes widely available
Girls’ Behaviour Assumptions & Risks
 Girls have access to schools
 Schools can adequately accommodate girls
Other explanatory factors
 Internet access becomes widely available
Teacher Behaviour Change
Assumptions & Risks
 Training relates to the local conditions
 Teachers are assessed in part on their
application of the training
Capacity Change Assumptions and
Risks
All relevant
teachers get the
training
timeline
Awareness and skills
training provided to
teachers on girls’
education needs
• Most teachers want to better education
for girls
Reach Assumptions & Risks
 Only teachers with a predisposition
to the importance of girls’
education attend
Equity: Distribution of impacts
• As set out so far, ‘girls’ is an general group
• But among ‘girls’, there are worst-off girls, who may
need special attention (e.g., access, income,
conservative parents)
• And as already noted, there may be unintended
impacts on boys
• To get at these distributional affects, we can build
nested theories of reach:
– A ToC for a particular reach group within an overall ToC
• And can then tell reach impact stories about the
effects and impacts of different reach groups
Theory of Change for Enhancing Education Outcomes for Girls
Improved
education
outcomes for girls
Worst-Off Girls Theory of Reach
Impact Assumptions & Risks
Girls
Theory of Reach
Girls learning
improves
Unintended effect: Boys’
education suffers
Girls more engaged
in and wanting an
education
External Influences
Parents
Peers
Religion
Culture
Education resources
Teachers provide girls
with more empathetic
and supportive
teaching in schools
Capacity changes in
teachers’ knowledge,
attitudes and skills
 Resources and will continue for
multi-year involvement
Girls’ Direct Benefit Assumptions &
Risks
 Girls have time to study
 Girls education supported by parents and
community
 Focus on girls is not disruptive to boy’s
education
Other explanatory factors
 Internet access becomes widely available
Girls’ Behaviour Assumptions & Risks
 Girls have access to schools
 Schools can adequately accommodate girls
Other explanatory factors
 Internet access becomes widely available
Teacher Behaviour Change
Assumptions & Risks
 Training relates to the local conditions
 Teachers are assessed in part on their
application of the training
Capacity Change Assumptions and
Risks
Teachers
Theory of Reach
timeline
All relevant
teachers get the
training
Awareness and skills
training provided to
teachers on girls’
education needs
• Most teachers want to better education
for girls
Reach Assumptions & Risks
 Only teachers with a predisposition
to the importance of girls’
education attend
Theory of Change for Enhancing Education Outcomes for Girls
Improved
education
outcomes for girls
Impact Assumptions & Risks
Girls learning
improves
Unintended effect: Boys’
education suffers
Girls more engaged
in and wanting an
education
External Influences
Parents
Peers
Religion
Culture
Education resources
Teachers provide girls
with more empathetic
and supportive
teaching in schools
Capacity changes in
teachers’ knowledge,
attitudes and skills
 Resources and will continue for
multi-year involvement
Girls’ Direct Benefit Assumptions &
Risks
 Girls have time to study
 Girls education supported by parents and
community
 Focus on girls is not disruptive to boy’s
education
Other explanatory factors
 Internet access becomes widely available
Girls’ Behaviour Assumptions & Risks
 Girls have access to schools
 Schools can adequately accommodate girls
Other explanatory factors
 Internet access becomes widely available
Teacher Behaviour Change
Assumptions & Risks
 Training relates to the local conditions
 Teachers are assessed in part on their
application of the training
Capacity Change Assumptions and
Risks
All relevant
teachers get the
training
timeline
Awareness and skills
training provided to
teachers on girls’
education needs
• Most teachers want to better education
for girls
Reach Assumptions & Risks
 Only teachers with a predisposition
to the importance of girls’
education attend
Theory of Change for Enhancing Education Outcomes for Girls
Improved
education
outcomes for girls
Impact Assumptions & Risks
Girls learning
improves
Unintended effect: Boys’
education suffers
Girls more engaged
in and wanting an
education
 Resources and will continue for
multi-year involvement
Girls’ Direct Benefit
Assumptions/Risks
 Girls have time to study
 Girls education supported by parents and
community
 Focus on girls is not disruptive to boy’s
education
Other explanatory factors
 Internet access becomes widely available
External Influences
Parents
Peers
Religion
Culture
Education resources
Teachers provide girls
with more empathetic
and supportive
teaching in schools
Capacity changes in
teachers’ knowledge,
attitudes and skills
All relevant
teachers get the
training
 Girls have access to schools
 Schools can adequately accommodate girls
Other explanatory factors
 Internet access becomes widely available
Teacher Behaviour Change
Assumptions & Risks
 Training relates to the local conditions
 Teachers are assessed in part on their
application of the training
Capacity Change Assumptions and
Risks
• Most teachers want to better education
for girls
Reach Assumptions & Risks
timeline
Awareness and skills
training provided to
teachers on girls’
education needs
 Only teachers with a predisposition
to the importance of girls’
education attend
Supporting activities
Girls’ Behaviour Assumptions & Risks
Theory of Change for Enhancing Education Outcomes for Girls
Improved
education
outcomes for girls
Impact Assumptions & Risks
Girls
Theory of Reach
Parents
Theory of Reach
Girls learning
improves
Unintended effect: Boys’
education suffers
Girls more engaged
in and wanting an
education
 Resources and will continue for
multi-year involvement
Girls’ Direct Benefit
Assumptions/Risks
 Girls have time to study
 Girls education supported by parents and
community
 Focus on girls is not disruptive to boy’s
education
Other explanatory factors
 Internet access becomes widely available
External Influences
Parents
Peers
Religion
Culture
Education resources
Teachers
Theory of Reach
Teachers provide girls
with more empathetic
and supportive
teaching in schools
Capacity changes in
teachers’ knowledge,
attitudes and skills
All relevant
teachers get the
training
 Girls have access to schools
 Schools can adequately accommodate girls
Other explanatory factors
 Internet access becomes widely available
Teacher Behaviour Change
Assumptions & Risks
 Training relates to the local conditions
 Teachers are assessed in part on their
application of the training
Capacity Change Assumptions and
Risks
• Most teachers want to better education
for girls
Reach Assumptions & Risks
timeline
Awareness and skills
training provided to
teachers on girls’
education needs
 Only teachers with a predisposition
to the importance of girls’
education attend
Supporting activities
Girls’ Behaviour Assumptions & Risks
Theory of Reach for Enhancing Education
Outcomes for Worst-Off Girls
Improved education
outcomes for worstoff girls
Worst-Off girls
learning improves
Unintended effect: Boys’
education suffers
Worst-Off girls
more engaged in
and wanting an
education
timeline
Teachers provide (all)
girls with more
empathetic and
supportive teaching in
schools
timeline
Impact Assumptions & Risks

Resources and will continue
for multi-year involvement
Worst-Off Girls’
Behaviour & Direct
Benefit Change
Assumptions & Risks
 Worst-off girls have
access to schools
 Worst-off girls education
supported by parents and
community
 Worst-off girls have time
to study
 Schools can adequately
accommodate girls
 Focus on girls is not
disruptive to boy’s
education
Other explanatory factors
 Internet access becomes
widely available
Partner Activities
Worst-off girls
are provided
transportation
to schools
Support
Activities
Engagement
with
governments
Engaged with
parents &
communities
with worst-off
girls
Engagement
with NGOs
intervention
activities
results
Nested
theory of
reach for
access by
worst-off
girls
A Basic Theory of Change for Comprehensive Interventions
Timeline
Impact
Impact Assumptions
& Risks
Direct Benefit
Changes
Other Explanatory
Factors
Behaviour Changes
Other Explanatory
Factors
Behavioural Change
Assumptions & Risks
Capacity Changes
Other Explanatory
Factors
Capacity Change
Assumptions & Risks
Reach &
Reaction
Timeline
Activities and Outputs
Other Explanatory
Factors
Reach Assumptions &
Risks
Reach, Capacity change
Behavioural Change
External
Influences
Supporting Activities and Outputs
Direct Benefits
Assumptions & Risks
A Basic Theory of Change for Comprehensive Interventions
Timeline
Impact
Different
Beneficiary
Nested ToC
Impact Assumptions
& Risks
Direct Benefit
Changes
Other Explanatory
Factors
Behaviour Changes
Other Explanatory
Factors
Behavioural Change
Assumptions & Risks
Different
Intermediary
Nested ToC
Capacity Changes
Other Explanatory
Factors
Capacity Change
Assumptions & Risks
Reach &
Reaction
Timeline
Activities and Outputs
Other Explanatory
Factors
Reach Assumptions &
Risks
Reach, Capacity change
Behavioural Change
External
Influences
Supporting Activities and Outputs
Direct Benefits
Assumptions & Risks
Using Theories of Change
5. Designing monitoring systems
– likely need more than just a focus on results—the
whole ToC, including assumptions and risks
6. Adaptive management
– Identify what to be monitored
– On a regular basis, reflect on what is being
observed and on your ToC
– Revise your intervention activities and your ToC as
you learn
Using Theories of Change
7. Reporting performance
– A verified ToC is your performance story
8. Designing evaluation questions, methods,
tools
– Identifies questions
– Adds power to case studies by confirming the ToC
– Basis for surveys et al
There are potential Evaluation Questions
associated with each box and arrow
Impacts
Assumption Questions
• Did the events/conditions
occur?
• Did risks materialize?
• Are other explanations
plausible?
External
Influences
Causal Question
• To what extent are the
behavioural changes the
result of the capacity
changes?
Direct
Benefits
Behaviour
changes
Capacity changes in
knowledge, attitudes
skills, opportunities &
incentives
Reach &
Reaction
Timeline
Activities and Outputs
with respect to
beneficiaries
Behaviour Change
• What behavioural
(practice) changes are
observed?
Behaviour Change
Assumptions: How are changes in
knowledge, attitudes, skills, opportunities
and/or incentives expected to change
behaviour? What events and conditions
have to happen? What factors influence
these processes?
Risks: Risks to the link not occurring.
Other Explanatory Factors: Peer or trend
pressure; other interventions
Capacity Change
• What capacity) changes
are observed?
Using Theories of Change
9. Making causal claims
– Based on generative causality
– ToC are models of causality
– Approaches such as contribution analysis
demonstrates causality
Theories of Change as
Causal Packages
ToC is a model of the intervention as a
contributing cause
– ToC identifies supporting factors (assumptions)
and confounding factors (risks)
– Intervention activities plus the assumptions are
sufficient to bring about the intended impacts
– Explains how and why intended impacts are
expected to occur
A Basic Theory of Change
Timeline
Impact
Impact Assumptions & Risks
Direct Benefit
Changes
Other Explanatory Factors
Direct Benefits Assumptions &
Risks
External
Influences
Behaviour Changes
Behavioural Change Assumptions
& Risks
Other Explanatory Factors
Capacity Changes
Capacity Change Assumptions &
Risks
Reach &
Reaction
Other Explanatory Factors
Reach Assumptions & Risks
Timeline
Activities and Outputs
Causal
package
Supporting Activities and Outputs
Other Explanatory Factors
Contribution Analysis
Using a generative perspective on causality, CA
shows that an intervention is a contributory cause:
• The expected result occurred
• The ToC (causal package) is sufficient
– supporting factors (assumptions) occurred and any
other supporting factors have been included in the ToC
– plausible rival explanations have been accounted for
• The intervention is necessary for the ToC to be
sufficient
• And can explore the role the intervention played,
such as a trigger
Using Theories of Change
10.Generalizing
– What works where and when?
– ToC explicitly includes the context and the causal
mechanisms
– Confirming a ToC provides significant evidence on
if and how the intervention would work elsewhere
Messages
• ToC are worth the effort to develop
• Not a throw away, but a powerful tool to use
in evaluation and in managing
• Lots of fun … and lots of debate!
References
Mayne, J. (forthcoming). Using Theories of Reach to Enhance Equity Considerations in
Evaluation. In Evaluation and Equity. K. Forss and M. Marra, Eds: Transaction.
Mayne, J. (2013). Making Causal Claims. Connections, June. Available at
http://www.europeanevaluation.org/images/file/files/ees-newsletter-2013-06june.pdf.
Mayne, J. (2012). Contribution Analysis: Coming of Age? Evaluation 18(3), 270-280.
Mayne, J. (2012). Making Causal Claims, ILAC Brief No. 26: The Institutional Learning
and Change Initiative. Available at http://www.cgiarilac.org/files/publications/mayne_making_causal_claims_ilac_brief_26.pdf
Stern, E, Stame, N., Mayne, J., Forss, K, Davies, R. and Befani, B. (2012). Broadening
the Range of Designs and Methods for Impact Evaluation, Working Paper 38,
Department for International Development, London. Available at
http://www.dfid.gov.uk/R4D/Output/189575/Default.aspx
Mayne, J. (2008). Contribution Analysis: An Approach to Exploring Cause and Effect,
ILAC Brief 16. Available at http://www.cgiarilac.org/files/publications/briefs/ILAC_Brief16_Contribution_Analysis.pdf