Fragile states: Perspectives from
evaluations
AFDB Evaluation Week, 5th December 2012.
Presentation by Beate Bull
Evaluation Department, Norwegian Agency for Development
Cooperation, Norad.
Norwegian development aid budget
13.04.20
15
Side/Page 2
Ten largest recipients of aid
13.04.20
15
Side/Page 3
Perspectives from peacebuilding /fragile states
evaluations based on:
•
Evaluations done for and by the Norwegian development cooperation
Agency, and reviewing others
•
A meta-review of evaluations of support to state building
by Gravingholt, J. og
Leininger, J. 2012
•
OECD/DAC Guidance on evaluating peacebuilding activities in settings
of conflict and fragility
13.04.20
15
Side/Page 4
List of recent Norad(supported) evaluations of support to peace-building in
settings of conflict and fragility
- “Evaluation of Norwegian Development Cooperation with Afghanistan 2001-2011”, (2012),
http://www.norad.no/no/s%C3%B8k?q=aiding+the+peace;
-
«Pawns of Peace. Evaluation of Norwegian peace efforts on Sri Lanka 1997-2009 (2011)”,
http://www.norad.no/no/resultater/publikasjoner/evalueringer/publikasjon?key=386346(2011)
-
“Aiding the Peace”: A Multi-donor Evaluation of Support to Conflict Prevention and Peacebuilding Activities in
Southern Sudan 2005-2010. ITAD Ltd., United Kingdom. http://www.norad.no/no/s%C3%B8k?q=aiding+the+peace
-
Evaluation of Norwegian Peace efforts in Haiti 1998-2009 (2009).
(See also one-page summaries in - Norad Evaluation Department’s Annual Reports (2011,2010,2009))
http://www.norad.no/en/evaluation or contact us on post-eval@norad.no
13.04.20
15
Side/Page 5
Findings across the evaluations
1) That there are weaknesses in the analysis of the
situation and of the conflict in both the planning and
implementation phase –
– Implications:
- limit relevance of the intervention/support/
- limit the evaluation’s possibility to say something bout
about relevance (Afghanistan/ South Sudan)
- reduces the likelihood of conflict sensitivity analysis being
conducted
- More…
13.04.20
15
Side/Page 6
Findings from the evaluations cont.
•
2) Not enough resources are set aside to follow up and assess
progress, during implementation (South Sudan, Afhganistan, Haiti
and Sri Lanka).
– At times, the staffing in embassies were far from adequate
(Afganistan- in particular, South Sudan, Haiti)
Implications:
– Not enough resources to quality assure programmes – are they on
the right track (do we do what we say we shall do): (Afghanistan50 % of scools not adapted to girls needs (latrines/protective
walls)).
– Delays
– The danger of aid not being relevant/not adapted to changing
context or maybe contribute to aggraving the conflict(s)
– Not enough resources to monitor whether aid becomes a stake in
the conflict
13.04.20
15
Side/cccPag7
e
Findings from evaluations cont.
• 3) Too much emphasis from donors are put on
harmonisation and coordination at the capital level in
the partner country at the cost of sharing knowledge
about local context, adapting activities to local conditions
an presence in the field.
• Example: South Sudan: Donor Coordination meetings did
not revolve around sharing conflict analyses, and discussing
how to coordinate aid to address local conflicts, but bigger
diplomatic issues: referendum 2010.
• Example: Afghanistan: the Norwegian funding to the ARTF
remained remarkably consistent over the years despite
important changes in the context
13.04.20
15
Side/Page 8
Some key challenges to peacebuilding
evaluations
• The threat of violent conflict
• The reality is often complex,
stakes are high, everything becomes
political,– ‘all voices
to be heard’? How to be
perceived as impartial and
balanced-key to the credibility of
the evaluation?
• Theory-poor field
• Evaluations can do harm
(evaluators leave, others stay behind)
13.04.20
15
Side/Page 9
How to deal with a challenging context…
• How to conduct conflict analysis
• How to conduct conflict sensitive
evaluations (do no harm)
• How to analyse theories
of change and their underlying
assumptions
• Surprises that are expected
and those unexpected
http://www.oecd.org/dac/evaluationofdevelopmentprogrammes/dcdndep/evaluatingconflictpreventionandpeacebuilding.htm
13.04.20
15
Side/Page 10
.
Findings from a (2012) meta review of evaluations of
support to state building in fragile states:
Few evaluations
are concerned
with:
- explicating a
theory of change;
- constructing a
credible
counterfactual;
- and making use
of quantitative
methods where
possible.
13.04.20
15
Side/Page 11
Which leads to:
- An inbuilt tendency
to reproduce the
conventional
wisdom
- instead of testing
implicitly assumed
causality chains
- or exploring what
the alternatives
would have been.
What do the evaluations of peacebuilding-field
look at?
•
Peace building evaluations conducted by many donors and mulitlateral
organisations seem to have focused on understanding and mapping
the terrain and their own internal organisation,
• They are concerned with:
– Coordination between different actors, planning,
– whether a conflict analysis is used or not for programming (most
often it is not),
– types of interventions, (socio-economic, humanitarian, peace
building, governance)
– conflict sensitivity,
– Inputs, activities, and outputs
Rather than
•
results, what works and what does not work (what do we base our
knowledge on/ which assumptions do we base the interventions on?
13.04.20
15
Side/Page 12
ُ
‫ش ْك ًرا‬
13.04.20
15
Side/Page 13
Findings from other evaluations
support to state building
1) A mixed methods repeated survey (2007-2009) in northeastern Afghanistan (2000 respondents in 80 villages )
commissioned by the German MFA found that development aid
had a small positive effect on the populations’s attitudes
towards foreign forces and the Afghan state. But, this effect
dissappeared when the population experienced a deterioration
in the security situation. Moreover, the small positive effect
from aid on attitudes depended on the perception of aid as
useful. The study also found that it is not the amount of
projects that impact the afghan attitudes, but their perceived
usefulness. Böhnke, J. R., J. Koehler, and Ch. Zuercher. 2010. “Assessing the Impact of Development
Cooperation in North East Afghanistan 2005 – 2009. Final Report.” BMZ)
13.04.20
15
Side/Page 14
State building continued
2) Statebuilding with emphasis on capacity building seem to
be more relevant in post conflict situations, rather than in
situations of ongoing conflict. This is based on findings from a
Danish evaluation and a UNDP evaluation (2013) of
statebuilding support to Somalia. The latter evaluation
concludes: if objectives of strengthened governance systems
shall be achieved, a minimum of stability is required.
(“Evaluation of the Danish engagement in and around Somalia 2006-10”, 2011 http://www.netpublikationer.dk/um/11094/
“Evaluation of UNDP support to conflict-affected countries in the context of UN Peace Operations”, Draft final (not to be quoted),
Evaluation Office, UNDP, Forthcoming January 2013.)
13.04.20
15
Side/Page 15