ILRI
Process and Partnership for
Pro-Poor Policy Change
Methods for Assessing
Policy Impact
Process and Partnerships for
Pro-Poor Policy Change,
Project Initiation Workshop 1
ILRI, 21st February 2005
ILRI
Process and Partnership for
Pro-Poor Policy Change
Workshop Outline
• Introduction to the RAPID Framework and
ILRI/ODI Project
• Case Study Approach
• Episode Study Approach
• Outcome Mapping Approach
• RAPID Outcome Assessment (ROA)
Approach
Lunch
• Practical Sessions
ILRI
Process and Partnership for
Pro-Poor Policy Change
Workshop Purpose & Objectives
Purpose:
To familiarise the participants with the general approach and
specific methods to be used in the SDP case study
Objectives
By the end of the workshop, participants will:
• understand the Process and Partnership for Pro-Poor
Policy Change project’s purpose and general approach
• have the opportunity to contribute their own suggestions to
improve the project;
• understand, and have had the chance to try out the three
key methods which will be used in the project;
• assess the usefulness of the approaches in their own work.
An introduction to the
RAPID Framework and
ILRI/ODI Project
ILRI
Process and Partnership for
Pro-Poor Policy Change
Definitions
• Research: “any systematic effort to increase the
stock of knowledge”
• Policy: a “purposive course of action followed by an
actor or set of actors”
– Agendas / policy horizons
– Official statements documents
– Patterns of spending
– Implementation processes
– Activities on the ground
ILRI
Process and Partnership for
Pro-Poor Policy Change
Policy Processes
- Identify a policy problem
- Commission research
- Assess the results
- Select the best policy
- Establish the policy framework
- Implement the policy
- The problem is solved
ILRI
Process and Partnership for
Pro-Poor Policy Change
Reality…
• “The whole life of policy is a chaos of purposes
and accidents. It is not at all a matter of the rational
implementation of the so-called decisions through
selected strategies 1”
• “Most policy research on African agriculture is
irrelevant to agricultural and overall economic
policy in Africa2”
1
- Clay & Schaffer (1984), Room for Manoeuvre; An Exploration of Public Policy in
Agricultural and Rural Development, Heineman Educational Books, London
2 – Omamo (2003), Policy Research on African Agriculture: Trends, Gaps, and Challenges,
International Service for National Agricultural Research (ISNAR) Research Report No 21
ILRI
Process and Partnership for
Pro-Poor Policy Change
Existing theory
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
Linear model
Percolation model, Weiss
Tipping point model, Gladwell
‘Context, evidence, links’ framework, ODI
Policy narratives, Roe
Systems model (NSI)
External forces, Lindquist
‘Room for manoeuvre’, Clay & Schaffer
‘Street level bureaucrats’, Lipsky
Policy as social experiments, Rondinelli
Policy Streams & Windows, Kingdon
Disjointed incrementalism, Lindquist
The ‘tipping point’, Gladwell
Crisis model, Kuhn
‘Framework of possible thought’,
Chomsky
16. Variables for Credibility, Beach
17. The source is as important as content,
Gladwell
18. Linear model of communication, Shannon
19. Interactive model,
20. Simple and surprising stories,
Communication Theory
21. Provide solutions, Marketing Theory I
22. Find the right packaging, Marketing II
23. Elicit a response, Kottler
24. Translation of technology, Volkow
25. Epistemic communities
26. Policy communities
27. Advocacy coalitions etc, Pross
28. Negotiation through networks, Sebattier
29. Shadow networks, Klickert
30. Chains of accountability, Fine
31. Communication for social change,
Rockefeller
32. Wheels and webs, Chapman & Fisher
www.odi.org.uk/rapid/lessons/theory
ILRI
Process and Partnership for
Pro-Poor Policy Change
Existing theory – a short list
•
•
•
•
•
•
•
•
Policy narratives, Roe
Systems of Innovation Model, (NSI)
‘Room for manoeuvre’, Clay & Schaffer
‘Street level bureaucrats’, Lipsky
Policy as social experiments, Rondene
Policy streams and policy windows, Kingdon
Disjointed Incrementalism, Lindblom
Social Epidemics, Gladwell
• The RAPID Framework
ILRI
Process and Partnership for
Pro-Poor Policy Change
An Analytical Framework
External Influences
Socio-economic and
cultural influences,
donor policies etc
The links between policy
and research communities –
networks, relationships, power,
competing discourses, trust,
knowledge etc.
The political context –
political and economic structures
and processes, culture, institutional
pressures, incremental vs radical
change etc.
The evidence – credibility, the
degree it challenges received
wisdom, research approaches
and methodology, simplicity of
the message, how it is packaged
etc
ILRI
Process and Partnership for
Pro-Poor Policy Change
Case Studies
• Sustainable Livelihoods: The
Evolution of DFID Policy
• The PRSP Initiative: Research in
Multilateral Policy Change
• The adoption of Ethical
Principles in Humanitarian Aid
post Rwanda
• Animal Health Care in Kenya:
Evidence fails to influence Policy
ILRI
Process and Partnership for
Pro-Poor Policy Change
A Practical Framework
External Influences
Politics and
Policymaking
Campaigning,
Lobbying
Scientific
information
exchange &
validation
political context
Media,
Advocacy,
Networking
links
Policy analysis, &
research
Research,
learning &
thinking
evidence
ILRI
Process and Partnership for
Pro-Poor Policy Change
What you need to know
• The external environment: Who are the key actors?
What is their agenda? How do they influence the political
context?
• The political context: Is there political interest in
change? Is there room for manoeuvre? How do they
perceive the problem?
• The evidence: Is it there? Is it relevant? Is it practically
useful? Are the concepts familiar or new? Does it need repackaging?
• Links: Who are the key individuals? Are there existing
networks to use? How best to transfer the information? The
media? Campaigns?
ILRI
Process and Partnership for
Pro-Poor Policy Change
What researchers need to do
What researchers
need to know
What researchers
need to do
Political Context:
• Get to know the policymakers. • Work with them – seek
commissions
• Identify friends and foes.
• Strategic opportunism –
• Prepare for policy
prepare for known events
opportunities.
+ resources for others
• Look out for policy windows.
• Who are the policymakers?
• Is there demand for ideas?
• What is the policy process?
Evidence
• What is the current theory?
• What are the narratives?
• How divergent is it?
Links
• Who are the stakeholders?
• What networks exist?
• Who are the connectors,
mavens and salesmen?
•
•
•
•
•
Establish credibility
Provide practical solutions
Establish legitimacy.
Present clear options
Use familiar narratives.
• Get to know the others
• Work through existing
networks.
• Build coalitions.
• Build new policy networks.
How to do it
• Build a reputation
• Action-research
• Pilot projects to generate
legitimacy
• Good communication
• Build partnerships.
• Identify key networkers,
mavens and salesmen.
• Use informal contacts
ILRI
Process and Partnership for
Pro-Poor Policy Change
Policy entrepreneurs
Storytellers
Engineers
Networkers
Fixers
ILRI
Process and Partnership for
Pro-Poor Policy Change
Practical Tools
Overarching Tools
- The RAPID Framework
- Using the Framework
- The Entrepreneurship
Questionnaire
Communication Tools
- Communications Strategy
- SWOT analysis
- Message Design
- Making use of the media
Policy Influence Tools
- Influence Mapping & Power Mapping
- Lobbying and Advocacy
- Campaigning: A Simple Guide
- Competency self-assessment
Context Assessment Tools
- Stakeholder Analysis
- Forcefield Analysis
- Writeshops
- Policy Mapping
- Political Context Mapping
Research Tools
- Case Studies
- Episode Studies
- Surveys
- Bibliometric Analysis
- Focus Group Discussion
ILRI
Process and Partnership for
Pro-Poor Policy Change
Practical Application
• Within ODI
• Workshops for researchers, policy makers
and activists.
• Advice to a DFID forest/ground water
research project in India:
– Less research
– More communication
– Developing champions in regional and national
government
– Local, Regional & National advocacy campaign
ILRI
Process and Partnership for
Pro-Poor Policy Change
Further Information / Resources
• ODI Working Papers
• Bridging Research and
Policy Book
• Meeting series
Monograph
• Tools for Policy Impact
• RAPID Briefing Paper
• www.odi.org.uk/rapid
ILRI
Process and Partnership for
Pro-Poor Policy Change
Can ILRI do it?
Yes, but:
• It this its role?
• “Global Public Good” Research vs Policy Advocacy
• Probably needs to do both:
How?
• Understand the political context
• Get the evidence & package it well
• Strategic networking / lobbying / campaigning
• Collaboration….
ILRI
ILRI
Process and Partnership for
Pro-Poor Policy Change
International Livestock Research Institute
Process and partnership for
pro-poor policy change
The New DfID funded Project
ILRI
ILRI
Process and Partnership for
Pro-Poor Policy Change
International Livestock Research Institute
• Project Leaders: ODI / ILRI
• Key collaborators: ECAPAPA
• Case study collaborators in Kenya:
– MoLFD / KARI
– Range of NGOs & other SDP partners
ILRI
ILRI
Process and Partnership for
Pro-Poor Policy Change
Why would I be interested?
International Livestock Research Institute
• Not all research is expected or intended to lead
to policy change, but there may be;
– Specific cases where research is expected to;
• provide evidence for policy change
• identify potential policies (or impact of)
• influence the policy making process
(advocacy)
– Cases where speculative research becomes
relevant because of changes in circumstance
ILRI
ILRI
Process and Partnership for
Pro-Poor Policy Change
The project …
International Livestock Research Institute
• Ideas for methods and approaches
• Lessons learnt from earlier activities
• Identification of appropriate communication
tools
ILRI
ILRI
Process and Partnership for
Pro-Poor Policy Change
What will we be doing?
International Livestock Research Institute
•
Three case studies in three DIFFERENT
countries
– A project considered to have influenced
policy change
– A stream of research addressing a particular
policy area
– A clear policy change;
•
•
•
New policy statement
New law
Irrefutable change in way something is done
ILRI
ILRI
Process and Partnership for
Pro-Poor Policy Change
What will we be doing?
International Livestock Research Institute
•
Three case studies
– SDP and impact on changed view of
informal milk trade
– ????
– ????
ILRI
ILRI
Process and Partnership for
Pro-Poor Policy Change
International Livestock Research Institute
• Discussion:
– Can ILRI hope to influence pro-poor policy
through research?
– Any good case studies?
Case Study Approach
ILRI
Process and Partnership for
Pro-Poor Policy Change
What is a Case Study?
Definition:
" A systematic inquiry into an event or a set of
related events which aims to describe and
explain the phenomenon of interest" Bromley
(1990)
ILRI
Process and Partnership for
Pro-Poor Policy Change
Why is it useful?
Goal :
to describe as accurately as possible the
fullest, most complete description of the case.
• An ideal methodology when a holistic, indepth investigation is needed
• Designed to bring out the details from the
viewpoint of the participants by using
multiple sources of data
ILRI
Process and Partnership for
Pro-Poor Policy Change
Types of Case Study
Types of case studies:
• Exploratory,
• Explanatory,
• Descriptive (Yin, 1993)
Stake (1995) included three others:
• Intrinsic - when the researcher has an interest in
the case;
• Instrumental - when the case is used to
understand more than what is obvious to the
observer;
• Collective - when a group of cases is studied.
ILRI
Process and Partnership for
Pro-Poor Policy Change
Issues
• The unit of analysis is a critical factor
• Typically a system of action rather than an
individual or group of individuals
• Tend to be selective, focusing on one or two
issues that are fundamental to understanding the
system being examined
• Case studies are multi-perspectival analyses
• The researcher considers not just the voice and
perspective of the actors, but also of the relevant
groups of actors and the interaction between them
• They give a voice to the powerless and voiceless.
ILRI
Process and Partnership for
Pro-Poor Policy Change
Triangulation
• Data source triangulation, when the researcher looks
for the data to remain the same in different contexts;
• Investigator triangulation, when several investigators
examine the same phenomenon;
• Theory triangulation, when investigators with
different view points interpret the same results; and
• Methodological triangulation, when one approach is
followed by another, to increase confidence in the
interpretation.
ILRI
Process and Partnership for
Pro-Poor Policy Change
Applications
•
•
•
•
To explain complex causal links between
research and policy
To describe the real-life context in which
policy has been influenced by research
To describe the policy influencing process
itself
To explore those situations in which the
policy intervention being evaluated has no
clear set of outcomes.
ILRI
Process and Partnership for
Pro-Poor Policy Change
Process
1. Design the case study protocol:
–
–
determine the required skills
develop and review the protocol
2. Conduct the case study:
–
–
–
prepare for data collection
distribute questionnaire
conduct interviews
3. Analyze case study evidence:
–
analytic strategy
4. Develop conclusions, recommendations, and
implications based on the evidence
Episode Study
Approach
ILRI
Process and Partnership for
Pro-Poor Policy Change
What is an Episode Study
“A study that focuses on a clear policy change
and tracks back to assess what impact
research had among the variety of issues that
led to the policy change”.
ILRI
Process and Partnership for
Pro-Poor Policy Change
What is the purpose?
Tracking backwards from policy change to
any particular research which influence policy
• an excellent way of investigating the
influence of research on policy
• Can focus on a single episode or
comparative episodes.
ILRI
Process and Partnership for
Pro-Poor Policy Change
Advantage
• The process of working backwards in time
gives a more realistic view of the broad
range of factors – other than research – that
influence policy
• Tracking forward probably overemphasizes
the importance of research
ILRI
Process and Partnership for
Pro-Poor Policy Change
Issues
• Policy processes are complex, multi-layered and
change over time
• Often difficult to isolate the impact of research
from other factors
• Actors may ‘re-write history’
• Important to seek the views of a wide range of
informed stakeholders
• The process of preparing an episode study is
iterative
• Key facts and / or inconsistencies need to be
cross-checked with key informants
ILRI
Process and Partnership for
Pro-Poor Policy Change
Process
1. Identify a clear policy change.
2. Identify key Research Questions (draw on
RAPID framework)
3. Explore how and why those policy
decisions and practices took place
4. Assess the relative role of research in that
process by drawing on the framework.
ILRI
Process and Partnership for
Pro-Poor Policy Change
Apply the RAPID Framework
External Influences
Politics and
Policymaking
Campaigning,
Lobbying
Scientific
information
exchange &
validation
political context
Media,
Advocacy,
Networking
links
Policy analysis, &
research
Research,
learning &
thinking
evidence
ILRI
Process and Partnership for
Pro-Poor Policy Change
Key Questions
• The external environment: Who are the key actors?
What is their agenda? How do they influence the political
context?
• The political context: Is there political interest in
change? Is there room for manoeuvre? How do they
perceive the problem?
• The evidence: Is it there? Is it relevant? Is it practically
useful? Are the concepts familiar or new? Does it need repackaging?
• Links: Who are the key individuals? Are there existing
networks to use? How best to transfer the information? The
media? Campaigns?
ILRI
Process and Partnership for
Pro-Poor Policy Change
Methods
Steps 3 and 4 can be done through a variety
of methods:
• review of the literature;
• interviews with key actors;
• capturing the authors’ own experience; and
• discussions at workshops.
Episode Study Examples
ILRI
Process and Partnership for
Pro-Poor Policy Change
Paravets in Kenya
1970s - Professionalisation of Public
Services.Research
International
- Structural Adjustment → Collapse
- Paravet projects emerge.
1980s - ITDG projects
projects.– collaborative research.
- Privatisation
Privatisation.
- ITDG Paravet network
network.and change of DVS.
1990s
- Rapid spread in North. The Hubl Study
Dr Kajume
- KVB letter (January 1998).
- Multistakeholder WSs → new policies.
2000s
- Still not approved / passed!
ILRI
Process and Partnership for
Pro-Poor Policy Change
PRSPs – Political Context
• Widespread awareness of a “problem” with
international development policy in late 90s
• Failure of SAPs (and Asian financial crisis)
• Mounting public pressure for debt relief
• Stagnation of Comprehensive Development
Framework idea
• Diverging agendas (UK – Poverty, US –
Governance)
• WB/IMF Annual General Meeting, Sept 1999
ILRI
Process and Partnership for
Pro-Poor Policy Change
PRSPs – Evidence
• Long-term academic research informing new
focus on poverty, participation, ownership, aid
effectiveness etc
• Applied policy research:
– ESAF reviews
– HIPC review
– SPA Working Groups
– NGO research on debt
• Uganda’s PEAP
ILRI
Process and Partnership for
Pro-Poor Policy Change
PRSPs – Links
• WB, IMF, SPA, Bilaterals, NGOs all involved
• Formal an informal networks
• “None of the players was more than two
handshakes away from any of the others”
Outcome Mapping
ILRI
Process and Partnership for
Pro-Poor Policy Change
What is it?
• an integrated PM&E tool
• a system to think holistically & strategically about
how we intend to achieve result
• an approach that focuses on changes in the
behaviour, relationships or actions of partners (as
outcomes)
• a methodology that characterizes and assesses the
program’s contributions to the achievement of
outcomes
• an approach for designing in relation to the broader
development context but assessing within your
sphere of influence
ILRI
Process and Partnership for
Pro-Poor Policy Change
Focus: On Behavioural Change
The Focus of Outcome Mapping
Inputs
Activities
Outputs
Outcomes
Impacts
Behavioural Changes 25
ILRI
Process and Partnership for
Pro-Poor Policy Change
How can it be used?
• For a program to tell its performance story in
outcome terms by:
– articulating its goals and designing its activities
– designing a monitoring system for assessing
internal performance and outcomes of partners
– setting a use-oriented evaluation plan
ILRI
Process and Partnership for
Pro-Poor Policy Change
Why use it?
• Focussing on changes in partners’ behaviour,
relationships, or actions allows a program to:
– measure results within its sphere of influence
– obtain feedback about its efforts in order to
improve its performance
– take credit for its contributions to the achievement
of outcomes
– show progress towards outcomes
ILRI
Process and Partnership for
Pro-Poor Policy Change
Terminology
• Outcomes: changes in behaviours, relationships,
activities and/or actions of the people, groups and
organisations with whom we work
• Vision: the broad human, social and environmental
betterment we desire
• Mission: how we intend to contribute towards the
achievement of the vision
• Boundary partners: individuals, groups and
organisations with whom we interact directly to
effect change
• Outcome challenges: changes behaviours of the
boundary partners as identified by the vision
ILRI
Process and Partnership for
Pro-Poor Policy Change
The Three Stages
ILRI
Process and Partnership for
Pro-Poor Policy Change
Intentional Study Design
Why ?
Vision Statement
How ?
Mission
Strategy Maps
Organizational
Practices
Who ?
What ?
Boundary Partners
Outcome Challenges
Progress Markers
ILRI
Process and Partnership for
Pro-Poor Policy Change
Performance Monitoring
• Provides a framework for a continuous
monitoring of the initiative as a tool to
achieve its outcomes.
• The program uses progress markers, a set
of graduated indicators of behavioural
change, identified in the intentional design
stage to clarify direction with its primary
partners and to monitor outcomes
ILRI
Process and Partnership for
Pro-Poor Policy Change
Evaluation Planning
• Helps identify the evaluation priorities
assessing the strategy at greater depth than
the performance monitoring stage
ILRI
Process and Partnership for
Pro-Poor Policy Change
Main Elements
RAPID Outcome
Assessment
ILRI
Process and Partnership for
Pro-Poor Policy Change
What is it?
• A Visual Tool
• Combines the outcome mapping concept within a
case study & episode study approach
• Systematic approach to collecting information about
changes in behaviour of key project partners that
contributed to the policy change
• Assessment of the contribution of the project
(programme, strategy, etc.) to observed changes in
behaviour –and ultimately to the policy change
ILRI
Process and Partnership for
Pro-Poor Policy Change
Approach
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
Describe policy environment at end
Describe policy environment at the beginning
Identify the key policy actors
Identify key boundary partners
Describe boundary partner behaviour at end
Describe boundary partner behaviour at beginning
Describe changes in BP behaviour
Describe changes in project (strategic/opportunistic)
Describe external influences
Determine level of impact of changes in project
Determine level of impact of external influences
Check through external interviews
Write report
ILRI
Process and Partnership for
Pro-Poor Policy Change
Sources & Outputs
1. Literature review
- Project background, progress, (published) achievements
2. Participatory workshop with staff (and BP)
–
–
Gather detailed information
Identify issues for further investigation
3. Interviews with key informants to:
–
–
–
Triangulate the result of the workshop,
Fill the gaps of information
Clarify causality
4. Report Writing
–
Visual and Narrative
ILRI
Process and Partnership for
Pro-Poor Policy Change
ROA Terms and Definitions
• Boundary partners: individuals, groups and
organisations with whom we interact directly
to effect change.
• Outcomes: changes in behaviours,
relationships, activities and/or actions of the
people, groups and organisations with whom
we work.
• Behaviours: the way we or our boundary
partners do or think about things.
ILRI
Process and Partnership for
Pro-Poor Policy Change
The key steps of the ROA framework
1. Describe the policy environment at the end
of the project
0
1
2
3
4
5
6
7
8
9
10
Policy Change
Policy Environment
BP
Project
EE
ILRI
Process and Partnership for
Pro-Poor Policy Change
The key steps of the ROA framework
2. Describe the policy environment at the
beginning of the project
0
1
2
3
4
5
6
7
8
9
10
Policy Change
Policy Environment
BP
Project
EE
ILRI
Process and Partnership for
Pro-Poor Policy Change
The key steps of the ROA framework
3/4. Identify key policy actors and boundary
partners (that were influential at end)
Before
BP
0
1
2
3
4
year/month
5
Today
6
7
8
9
10
BP1
BP3
BP4
BP5
BP6
BP7
Project
EE
Policy Change
Policy Environment
BP2
ILRI
Process and Partnership for
Pro-Poor Policy Change
The key steps of the ROA framework
5. Describe the behaviours of the boundary
partners that contributed to the change in
the policy environment or policy
Before
0
1
2
3
4
year/month
5
Today
6
7
8
9
10
BP1
5
BP2
4
BP3
8
BP4
7
BP5
4
BP6
9
BP7
3
Project
EE
Policy Change
Policy Environment
BP
ILRI
Process and Partnership for
Pro-Poor Policy Change
The key steps of the ROA framework
6. Describe the behaviours of the boundary partners
at the beginning of the project
Before
0
1
2
3
4
year/month
5
Today
6
7
8
9
10
BP1
0
5
BP2
0
4
BP3
0
8
BP4
0
7
BP5
0
4
BP6
0
9
BP7
0
3
Project
EE
Policy Change
Policy Environment
BP
ILRI
Process and Partnership for
Pro-Poor Policy Change
The key steps of the ROA framework
7. Map the key changes in behaviour for each
boundary partner from the start of the project
0
1
2
BP1
0
1
2 3,4
BP2
0
1
BP3
0
1
2
BP4
0
1
2
BP5
0
BP6
0
BP7
0
Project
EE
3
4
year/month
5
Today
6
7
8
9
10
5
2,3
1,2
4
3
3
3,4,5
4,5,6
7
8
4
5
6
7
1
2
3
4
6,7,8
9
1, 2
3
Policy Change
Policy Environment
Before
BP
ILRI
Process and Partnership for
Pro-Poor Policy Change
The key steps of the ROA framework
8. Map the key changes in the project including
organisational changes, outputs and changes in
behaviour during the same period.
0
1
2
BP1
0
1
2 3,4
BP2
0
1
BP3
0
1
2
BP4
0
1
2
BP5
0
BP6
0
BP7
0
Project
0
EE
3
4
year/month
5
Today
6
7
8
9
10
5
2,3
1,2
4
3
3
4,5,6
7
4
5
6
7
1
2
3
4
3,4,5
6,7,8
9
1, 2
1 2,3
4,5
8
6
7
8
Policy Change
Policy Environment
Before
BP
9
3
10
ILRI
Process and Partnership for
Pro-Poor Policy Change
The key steps of the ROA framework
9. Map the external influences including the actions f
strategic partners and other exogenous factors
during the same period
0
1
2
3
BP1
0
1
2 3,4
BP2
0
1
BP3
0
1
2
BP4
0
1
2
BP5
0
BP6
0
BP7
0
Project
0
1 2,3
EE
0
1
4
year/month
5
Today
6
7
8
9
10
5
2,3
1,2
4
3
3
4,5,6
7
8
4
5
6
7
1
2
3
4
3,4,5
6,7,8
Policy Change
Policy Environment
Before
BP
9
1, 2
4,5
6
7
8
9
2 3,4
5
6
7
8
3
10
9
10
11
ILRI
Process and Partnership for
Pro-Poor Policy Change
The key steps of the ROA framework
10. Determine the level of impact/influence of the
project on the changes in behaviour of the
boundary partners
0
1
2
3
BP1
0
1
2 3,4
BP2
0
1
BP3
0
1
2
BP4
0
1
2
BP5
0
BP6
0
BP7
0
Project
0
1 2,3
EE
0
1
4
year/month
5
Today
6
7
8
9
10
5
2,3
1,2
4
3
3
4,5,6
7
8
4
5
6
7
1
2
3
4
3,4,5
6,7,8
Policy Change
Policy Environment
Before
BP
9
1, 2
4,5
6
7
8
9
2 3,4
5
6
7
8
3
10
9
10
11
Direct influence
Indirect influence
External influence
ILRI
Process and Partnership for
Pro-Poor Policy Change
The key steps of the ROA framework
11. Determine the level of impact/influence of external
influences on the changes in behaviour of the
boundary partners and the project
0
1
2
3
BP1
0
1
2 3,4
BP2
0
1
BP3
0
1
2
BP4
0
1
2
BP5
0
BP6
0
BP7
0
Project
0
1 2,3
EE
0
1
4
year/month
5
Today
6
7
8
9
10
5
2,3
1,2
4
3
3
4,5,6
7
8
4
5
6
7
1
2
3
4
3,4,5
6,7,8
Policy Change
Policy Environment
Before
BP
9
1, 2
4,5
6
7
8
9
2 3,4
5
6
7
8
3
10
9
10
11
Direct influence
Indirect influence
External influence
ILRI
Process and Partnership for
Pro-Poor Policy Change
The key steps of the ROA framework
12. Refine conclusions with in-depth interviews and
assess the real contribution of the project on the
policy change
0
1
2
3
BP1
0
1
2 3,4
BP2
0
1
BP3
0
1
2
BP4
0
1
2
BP5
0
BP6
0
BP7
0
Project
0
1 2,3
EE
0
1
4
year/month
5
Today
6
7
8
9
10
5
2,3
1,2
4
3
3
4,5,6
7
8
4
5
6
7
1
2
3
4
3,4,5
6,7,8
Policy Change
Policy Environment
Before
BP
9
1, 2
4,5
6
7
8
9
2 3,4
5
6
7
8
9
3
10
11
10
11
Direct influence
Indirect influence
External influence