Evaluation of Research Impact: Options and Challenges
18 October 2012
Simon Batchelor (IDS & Gamos)
IMPACT AND LEARNING
Issues to raise
Boundaries and responsibilities of Researchers
Theory of Change
Policy change?
Farmer/User change?
Methods – my three favourites
–
–
–
–
Actor networks
Qualitative Document Analysis
Delphic
One that I remain unsure about?
If time – two last points
2
Boundaries
Where does the responsibility of researchers end?
Different types of research
Engagement during the research
–
–
“a need to engage with a representative sample of the end
users to ensure that new hybrids or practices fit the ‘real world’
farming systems”.
“there are the actors at the boundary of the research who
might take the research forward. At some point research that
has led to successful product development will need to go to
scale”.
Stakeholder maps and communication strategies
–
3
Helped by a Theory of Change
Theory of Change
Isobel in the room. 
Complexity, Process, linear logic models
–
–
–
–
–
–
–
–
4
It is possible to move away from a linear logic model and
present a ‘Theory of Change’ in a nonlinear way.
This also moves away from the idea of an impact pathway to,
let’s say, an impact landscape.
For instance, A discussion paper by Hivos (Guijt and
Retolaza, 2012) view the Theory of Change as involving
dimensions of change and identify the following 5 dimensions:
1) The actors (individuals or groups) who are trying to bring
about change;
2) The context or situation that influences the actors and
which they are trying to change;
3) The ideas or theories that influence the actors when they
consider how to act in a certain situation;
4) The strategic plan that describes the reasons and provides
a framework for taking particular action; and
5) The reflection and decision-‐making processes that help
actors to develop strategy, review success and failure and
make improvements to ideas and strategies.
HIVOS Dimensions for a Theory of Change
Reproduced from Guijt and Retolaza, (2012)
5
Policy Change
Emphasis of Theories of Change should be on the
Causal mechanisms.
We use Steven Framework
–
Steven, D. (2007) ‘Evaluation and the New Public Diplomacy’,
presentation to the Future of Public Diplomacy, 842nd Wilton
Park Conference, River Path Associates
Changing perceptions and public opinion;
Setting an agenda by reframing the way an issue is
debated and creating pressure for change
Building networks that support delivery of change;
–
6
Role of Champions
Developing capacity within organisations to allow
them to understand and respond to an issue;
Changing institutions, for example influencing
strategy and resource allocations within
organisations (often government, but may be private
sector).
What about widgets?
Different types of research
Farmer extension
Private Sector engagement
Pro-poor research, para- poor research, per- poor
research
Blatant self promotion – Changing the Financial
Landscape of Africa Batchelor S, IDS Bulletin
Volume 43 Number 5 September 2012
7
Methods? My 3 Favourite
Actor networks
–
–
Social network analysis (if time the story of Catia)
Netmapping (Power cube)
Qualitative Document Analysis
–
New media challenges
Delphic consultation
–
–
User surveys
Engage to activate
Not sure yet about Sensemaker?
–
8
Any experience in the room?
Two last points
Learning!!!!!!!!!
Difference between Research communication and
Intermediary Work
9
Example of QDA analysis
10