European Commission
Joint Evaluation Unit common to
EuropeAid, Relex and Development
Methodology for Evaluation
of Budget support operations
at Country level
Conceptual and Methodological Note
EXISTING BUDGET SUPPORT EVALUATIONS
2004 Tanzania
2006 Joint Evaluation of 7 countries
from 1994 to 2004 budget support
2007 Ghana
Evaluations addressing mainly THE PROCESS
And not enough the CHAIN OF RESULTS
 IMPACTS AT THE GLOBAL LEVEL
 IMPACTS ON SECTOR POLICIES
1. An inventory of the inputs provided through budget support
during a given period.
2. An assessment of the performance of these inputs in terms of
direct outputs and induced outputs.
3. An identification of the changes which have occurred during the
period in the partner country and the key causal factors driving
those changes related to the budget support.
4. An assessment of the extent to which budget support may have
contributed to the results identified at the outcome and impact
levels.
5. An assessment of the overall relevance of budget support
arrangements in the light of the evolving country and sector
specific context, the aid policy and the related goals.
A Three components approach methodology combining
 An evaluation of the budget support itself
 An assessment of the budget support related outcomes and impacts of
the government strategy
 An exploration of the linkages between the budget support process and
some outcomes and impacts of the government strategy
Focused on the main topics of the budget support
 Each topic is associated with one or several Evaluation Questions,
Judgement Criteria and Indicators
 To ensure harmonisation of the evaluations in various contexts
GBS/SBS COMPREHENSIVE EVALUATION FRAMEWORK
EVALUATION OF THE BUDGET SUPPORT
Overview of the national
context and its evolution
Entry conditions for budget
support
Assessment of the inputs
provided by the budget
support
Volume of funds, policy dialogue and
conditionalities, technical assistance
and capacity building, harmonisation
and alignment
Assessment of the direct
outputs of the budget
support
Effects of the identified inputs
including effects on transaction
costs
Assessment of the
induced outputs and the
corresponding
government actions
Changes that budget support has
induced in comparison to what was
expected
ASSESSMENT OF IMPACTS AND OUTCOMES
Government
• Economic growth
Review of the impacts of
the government strategy:
specifically those related
to the budget support
deliverables
• Incidence of income and non income
poverty
e.g. • Degree of empowerment and social
inclusion
• Evolution of MDG indicators
• Achieving sector development objectives
for SBS
Causal factors
• Changes in business confidence and
competitiveness
Identification of the
factors that have
contributed to the key
changes
e.g.
Identification of the factors
external to the government
strategy which could have had
an impact on the mentioned
changes
• Changes in the use and appreciation by
the beneficiaries of goods and services
provided by the public sector
• Changes in the degree of confidence of
the general public sector
• Regional and international economic
e.g.
• Political environment
• Other unforeseen event
Relationship
Identification of the outputs of
the government strategy which
appear to have been of
greatest importance in
generating change at outcomes
level
• Positive linkages between government
actions and improved outcomes
and
• Failure or inappropriate actions which
can have jeopardised the quality of the
outcome
ASSESSMENT OF THE INFLUENCE OF BUDGET SUPPORT
« INTEGRATION AND CONSISTENCIES BETWEEN
COMPONENTS 1 & 2 »
Comparison of the results of
component 1 and component 2
 Improved understanding of
the influence of budget support
on the institutional and policy
changes
 Increased consideration of
the role of dominant factors in
the determination of outcomes
and impacts
If consistencies: indicate causal
links showing positive
conclusion for budget support
If no consistency: find a
possible explanation
GBS/SBS COMPREHENSIVE EVALUATION FRAMEWORK
Component 1
Component 3
Component 2
HOW TO MANAGE AND
IMPLEMENT THIS EVALUATION?
CHALLENGES AND PROPOSALS
We must take into account:
1.
THE STAKEHOLDERS
2.
THE LEAD OF THE EVALUATION
3.
THE CONSISTENCY, HARMONISATION AND DISSEMINATION ISSUES
4.
THE IMPLEMENTATION
5.
THE CAPACITY BUILDING ISSUE
STAKEHOLDERS
• To whom this evaluation is for?
• Are the donors the only stakeholders?
• Proposal 1
All donors, the government and other main
stakeholders involved in the budget support
evaluated should be part of the evaluation.
The evaluation of budget support operation
in a designed country should be part of a
multi-donor evaluation programme.
EVALUATION LEAD
• Who will take the lead for managing the
evaluation?
• From where will the evaluation be managed?
• Proposal 2
One of the main donors in the country
should be chosen by the other donors to
take the lead and consequently for
contracting with the evaluators. Special
arrangements should be set up to involve
representatives in the country.
CONSISTENCY, HARMONISATION
AND DISSEMINATION ISSUES
• Proposal 3
To ensure consistency, the headquarter
Evaluation Service of the lead donor should
be responsible for the evaluation
methodology and quality assurance. Each
donor can have dedicated persons in his
Evaluation Service to go into the countries
where the evaluation takes place, at crucial
moments.
• Proposal 4
A network of donors should be set up to
ensure harmonisation and dissemination.
IMPLEMENTATION
• Who will do the evaluation?
• Who has currently the competencies to evaluate
budget support and how to manage it?
• Proposal 5
A team of external independent consultants
familiar with the main topics of the evaluation
should be contracted for each evaluation.
CAPACITY BUILDING
• Do we need a specific training on budget support
evaluation?
• Proposal 6
Make sure that few consultants are immediately
operational and have been already trained in existing
trainings:
• Proposal 7
Train the identified people within the Evaluation
services of the Member States, some external
independent consultants and the main governmental
stakeholders of the recipient countries.
1. Issue paper (May 2008)
2. Steering group at OCDE / DAC level (June 2008)
3. 1st meeting of a core group (September 2008)
4. DAC Evaluation Network (18 – 19 November 2008)
5. 2nd meeting of core group (December 2008)
6. Common methodological document and standard terms of
reference (early 2009)
7. Testing methodological approach with partners’
agreement (2009)