Evaluation Office
Evaluating Capacity Development
David Todd
Senior Evaluation Officer
GEF Evaluation Office
1
Evaluation Office
Country Case Studies: Overview



2
GEF portfolios in the Philippines and Vietnam
include considerable capacity development
activities
Results are generally positive and in some
areas significant
Capacity development support has been
relevant, in line with national policy priorities
and with a good level of national ownership
Evaluation Office
Philippines CD Achievements (1)



3
Identified and began to implement
strategies to reduce GHG emissions
Developed quantitative estimates of global
and co-benefits of GHG reducing policies
and technologies
Engaged national stakeholders to
commence adoption of cost-effective
renewable energy strategies
Evaluation Office
Philippines CD Achievements (2)


4
Built analytical, institutional and human
capacity for multi-disciplinary monitoring and
analysis of GHG mitigation and environmental
impacts of alternative strategies
Put in place legislation and policies enabling
increased energy conservation and efficiency
Evaluation Office
Vietnam CD Achievements (1)



5
Positive results in individual level capacity
development activities in several projects
Raised awareness of energy efficiency and
renewable energy
Built skills to gather and analyze GHG data
and to formulate least-cost GHG abatement
strategies
Evaluation Office
Vietnam CD Achievements (2)


6
Changing institutional environment - need for
new skills and approaches recognized improvements to individual capabilities often
accompanied by changes to institutional
structures and procedures that mean these
capabilities can be utilized.
However, in other cases there are questions
over the sustainability and effectiveness of
activities, in the absence of a dynamic
institutional environment.
Evaluation Office
Effectiveness



7
The effectiveness of capacity development activities
has varied
In Vietnam some activities were effective in providing
new skills and institutional capacities that showed
direct and immediate results in the concerned sector;
but in other cases, the activities had less immediate
results,
In the Philippines, institutions have been unable to
provide appropriate incentives for trained staff, and
opportunities to use new skills have proved limited
Evaluation Office
Efficiency


8
Efficiency of capacity development activities
was difficult to assess, since the activities
rarely comprise a defined budget heading
during project implementation or monitoring.
GEF capacity development activities have
usually met their immediate output and
outcome targets, although a few projects have
suffered unusual delays in implementation.
Evaluation Office
Sustainability




9
A number of changes to policies and
strategies have a strong level of commitment
There are doubts about the sustainability of a
number of capacity development outcomes.
Individual skills need continued support
Low sustainability of alternative livelihoods in
rural energy projects
Evaluation Office
Shortcomings





10
Project design often lacked clarity about how the improved
capacity will be used.
Weakness in training programs - tendency to plan and execute
training as a “one-shot” solution.
Ignore importance of progression and repetition in training.
Progression is needed to allow successful trainees, who have
made use of their new knowledge and skills, to undertake more
advanced courses, to reach higher levels of expertise, thereby
further strengthening institutional performance.
Repetition needed to deal with the attrition of trained personnel,
which is a common problem in Government institutions and for
increased needs as the value of improved approaches generates
new demands.
Evaluation Office
Solution


11
Ensure training approaches build on existing
bodies in the country or region: e.g.
Universities, or specialist public or private
sector training institutions.
These can be supported to adapt their existing
programs or create new ones to address the
key environment-related skills identified as
necessary during project preparation.
Evaluation Office
Solution


12
In some cases, it may be effective to develop
new specialist training bodies in a region.
“One-shot” training inputs by international
consultants should be a strategy of last resort,
when it is evident that the required expertise is
not yet available in the region.
Evaluation Office
Further Evaluation Work


13
Further work is needed to analyze capacity
development across the GEF portfolio.
Meta-evaluation of capacity development
findings based on a review of a sample of
Terminal Evaluations and previous EO and IA
reports, to explore the prevalence of the key
issues identified by the country case studies.
Evaluation Office
Local Benefits Study



14
Have started the meta-evaluation with LBS
LBS reviewed climate change 30 projects that
intended to provide local livelihood or other
benefits for the poor
Individual field case studies were carried out
for five of the projects
Evaluation Office
Results from LBS (1)
Policy changes achieved:
 Appreciation of alternative energy and
development of adoption strategies at the
national level
 Changes in tax systems, tariffs, standards &
codes and laws to promote renewables
 National support for awareness raising for
consumers and private sector investors
15
Evaluation Office
Results from LBS (2)



16
Projects contributed to enhanced institutional
capacities for energy management at national,
regional and local levels
Increased the private sector’s awareness of
renewable technologies
Increased capacity of government and private
sector to work together through public-private
partnerships
Evaluation Office
Results from LBS (3)


17
Major weakness of many projects was the
failure to deliver intended livelihood benefits
for rural communities targeted
Take up of new technologies was therefore
not by poor people, restricting scope for
scaling up and of attaining CC objectives
Evaluation Office
Common Findings

18
The evaluation team will also hold discussions
with the GEF-UNDP-UNEP Support Program
for Capacity Development to explore
commonalities between its assessment of
results and those of the evaluation.
Evaluation Office
Objective of the GEF
Evaluation Office’s Work

19
To enable the development of a set of tools,
which will enable forthcoming Annual
Performance Reports, Country Program
Evaluations and OPS4 to evaluate the
achievements of capacity development
activities on a broader scale.