UNDP-GEF Terminal Evaluations Review Form
I.
Background Data
Project Information
Project Title (per project appraisal document
submitted to GEF):
UNDP Project ID:
GEF Project ID:
Region:
Countries:
GEF Operational Program / Strategic Program:
Coastal and Marine Biodiversity Conservation and
Sustainable Use in the Con Dao islands region
2091
1031
Asia
Viet Nam
SP 1: Catalyzing sustainability of protected area
systems
Executing Agency:
Project Partners :
Project and Evaluation Milestones
(Milestones)
CEO endorsement:
Agency approval date:
(Expected Dates)
n/a
n/a
Implementation start:
Midterm Evaluation:
Project Completion:
Terminal Evaluation Completion:
TE & QA Review particulars
Terminal Evaluation author(s):
TE QA Reviewer:
TE QA Review date:
Draft TE QA Submission Date:
Final TE QA Submission Date:
UNDP EO task manager approval date & sign:
II.
a.
b.
(Actual dates)
Apr 2006
May 2006 (basis of official
start)
Jun 2007 (actual
inception)
n/a
Jul 2009
Sophie van der Meeren, Nguyen Huong Thuy Phan
Evelyn Wong
Jan 2011
19-Mar-11 Alan Fox
Project Design & Objectives
List the overall environmental objectives of the project:
The project will catalyze conservation and sustainable management of the globally significant
coastal and marine biodiversity and ecologically processes of the Con Dao islands region.
List the development objectives of the project:
Objective 1: Protection of globally significant coastal and marine biodiversity in Con Dao islands
1
c.
d.
e.
f.
III.
a.
region with increased participation of local communities
Objective 2: Improved enabling environment for management of coastal and marine
biodiversity in Con Dao islands region with links between biodiversity conservation and socioeconomic development and strengthened institutional capacity and legal frameworks.
Objective 3: Sustainable protection and financing of coastal and marine biodiversity in Con Dao
islands region through support from ecotourism development.
Identify the key project components:
1. Strengthening participatory biodiversity conservation management in Con Dao National Park.
2. Strengthened environmental management in Con Dao district.
3. Sustainable Tourism Strategy for Con Dao archipelago to support Biodiversity conservation
4. Adaptive Management & Learning.
Briefly assess the quality of project design, noting in particular the logical connections of
objectives to outcomes and outputs taking into account risks and assumptions, and the proposed
budget.
As set out in the MTE, the project design lacked a discussion on climate change, discussion and
attention on marine threatened species, consideration of the several different types of fisheries
exploiters: external, internal permanent and internal migrant, and did not adequately integrate
terrestrial and marine resource management issues.
Have indicators been established for the achievement of project outputs and outcomes? If so do
they build from GEF guidance, are they SMART?1
The projects Monitoring and Evaluation Framework established baselines, SMART indicators and
a time line for regular progress reporting. Baselines were established through a series of
quantitative and qualitative studies and assessments, including consultation with key
stakeholders.
Were there any delays in the project preparation phase, prior to project Inception? If so, why? Did
the delays result in changes to objectives and planned outcomes?
The project was endorsed by the Global Environment Facility (GEF) in April 2006 and
subsequently approved by GOV and UNDP, which was the basis for official start of the project in
May 2006. An Inception workshop was held in August 2006, however the final Inception report
was only presented in June 2007. This was largely due to a change in the senior technical advisor
(STA) assigned to the project by UNDP during the inception period. Project inception was
however not finalised until a year later, in June 2007. This effectively means that main activities
of the project have been undertaken in two years, rather than the three originally planned. The
first year of the project largely involved procurement activities and some baseline studies.
Achieving project outcomes in two years was very ambitious, particularly given the projects
participatory and consultative approach.
Project Implementation
Indicate whether there were changes in the environmental and development objectives of the
project during implementation, why these changes were made and what was the approval
process. 2
1
The GEF Secretariat has issued guidance on the use of “SMART” indicators (Specific, Measureable, Achievable and
Attributable, Relevant and Realistic, and Time-bound, Timely, Trackable and Targeted). GEF climate change
indicators were developed in 2000, IW in 2002 & 2003, and biodiversity in 2003. For a description, see
www.gefweb.org/MonitoringandEvaluation/MEPoliciesProcedures/MEPIndicators/mepindicators.html
2
b.
Were there delays in project implementation and completion? If so, indicate reasons and
comment on whether the delays affected project outcomes and sustainability.
A number of delays in initiating project activities also impacted on project outcomes. Delays in
starting the Trust Fund work appear to have been largely due to the difficulty of finding a
suitably qualified consultant within the limited budget. There were also delays in initiating the
AIG pilot projects. Target beneficiaries for AIG support were identified as those fishers most
impacted by MPA zoning, preference was to be given to those who had signed the MPA Natural
Resources Conservation Convention. This meant that AIG activities could not start until the
zones had been designated and the Conservation Convention completed. Both processes
involved comprehensive community consultation. The AIG consultation process also itself
involved extensive household and individual surveys and interviews, particularly with fishers. As
outlined above, lack of consistency in the groups consulted complicated decision making on AIG
projects, which was largely realized through the CCG. The end result of these delays is that at
GEF/UNDP project end there were no concrete outcomes in terms of establishing either the
Conservation Trust Fund or alternative income generating opportunities for local Con Dao
fishers. Had AIG and Trust Fund activities started earlier in the project cycle it may have been
possible for the project to achieve outcomes by project end, although it should be noted that
additional outcomes were envisioned to be achieved by the end of the DANIDA funded
components, scheduled for completion 3 months after closure of the UNDP/GEF activities.
Co-financing3
Source of co-financing
Total4
Type (cash or in-kind)
Expected
Actual
Host gov’t contribution
Local communities
GEF Agency (ies)
(GEF Trust Fund)
Other agencies (bilateral / multilateral)
WWF Indochina (from DANIDA)
Private sector
in-kind
In kind
$254,480
$48,100
$254,850
cash
$970,450
$970,450
$299,300
$299,300
cash
2
The GEF guidance indicates the following possible reasons for changes: a) original objectives were not sufficiently
articulated; b) exogenous conditions changed, due to which a change in objectives was needed; c) project was
restructured because original objectives were overambitious; d) project was restructured because of a lack of
progress; e) other (specify).
3
GEF focuses considerable attention on the extent of co-financing across the portfolio, and uses co-financing as a
proxy for country ownership as well as an important indicator for sustainability. Accordingly, this table should be
filled in based on the numbers set out in the ProDoc for expected co-financing and then in the TE and/or Project
Close out note, on actual or realized co-financing. Especially in GEF-2 & 3 projects there may be instances where
no actual co-financing numbers have been tabulated, and/or no cash & in-kind breakouts have been provided.
4
Includes both co-financing during project design stage (PIF/PPG) and Project Implementation (ProDoc)
3
NGOs
WWF additional contribution
Other
Conservation Trust Fund
Total co-financing
a.
b.
c.
d.
Cash
$205,200
$45,400
$45,400
$852,850
Is there sufficient clarity in the reported co-financing to substantiate in-kind and cash
co-financing from all listed sources?
No
Comment:
Neither the mid-term evaluation nor the final evaluation did an adequate job discussing the
budget. However the co-financing figures are well set out in the 2010 APR/PIR, clarifying cans
and in-kind co financing. The ProDoc indicates that the above $45,400 is from a 'Conservation
Trust Fund'. The APR/PIR indicates this is UNDP/TRAC funding. The APR/PIR also mentions an
additional $2000 contribution from an international fund, without specifying which. The
APR/PIR lists the government contribution as cash, however the ProDoc indicates this is an inkind contribution.
If there is a difference in the level of expected and actual co financing, what were the Yes
reasons?
Comment (to what extent, and note whether justifications were given):
Total co-financing indicated in the 2010 APR/PIR is $852,850, which is on par with the amount
set out in the Project Document, discounting the local communities in-kind budget which is in
the ProDoc but not indicated in the APR/PIR , nor the MTE & TE
Were components supported by co-financing well integrated into the overall
No
project?
Comment :
Indications from the TE are that the WWF component ($299,300 cash from DANIDA
implemented through the WWF) was not as well integrated, in particular due the fact that the
WWF were not incorporated into the project management unit in Con Dao, but instead were
based in Hanoi.
Did the extent of materialization of co-financing affect project outcomes and/or
Yes
sustainability?
Comment:
The dual funding arrangements3 and associated dual lead and support roles of WWF and PMU in
implementing inter-dependent project activities caused a number of issues and challenges, although
both agencies contributed strongly to achieving project outcomes. The remoteness and difficulty of
organising travel to Con Dao exacerbated the situation.
Additional comments on co-financing:
Monitoring and Evaluation
a.
Assess the quality of the M&E design at project start up.
MU
Comment:
M&E plan was deemed insufficient in the ProDoc and was to be strengthened during the
4
project inception phase, including impact and outcome indicators and means of verification.
Assess the quality of M&E Plan Implementation.
MS
Comment:
The MTE indicates that the M&E framework was still under development - yet to be finalized,
including quarterly and annual work plans and budgets, plus detailed activity reports. Despite
not having a completed and operable M&E framework, the MTE evaluators though the M&E for
the project was adequate. It indicates that the reporting mechanisms in use were fulfilling the
M&E function adequately.
It is apparent that the MTE and TEs were carried out with small budgets and mimal number of
days. The TE consultant complained that only 20 days in total were allocated for the 2 person
team to carry out the exercise, including 5 days on mission, which was deemed insufficient.
Was sufficient funding provided for M&E in the project budget?
Expectation is that the budget was low. Budget information was not included in the main
evaluation report and annexes were not provided. The TE references difficulties hiring a senior
technical advisor due to budget constraints, and indicates insufficient budget for the terminal
evaluation.
Was a midterm evaluation carried out and were MTE recommendations taken into account in
subsequent project implementation?
A midterm was carried out. The terminal evaluation did not focus on this aspect, except to say that mid
term evaluators strongly recommended that fisherman be incorporated into the CCG but this was
apparently not possible under Vietnamese legislation.
Were PIR self-evaluation ratings consistent with the MTE and TE findings? If not, were these
discrepancies identified by the project steering committee and addressed?
PIR self-evaluation was uniformly satisfactory in all but the first PIR. Evidence suggests that the PIRs for
this project were not rigorously developed as many have incomplete information. There is no indication
that discrepancies between self-reporting and evaluation assessments were taken up by the project
steering committee.
Country Ownership
a.
Are there indications that government agencies in addition to the GEF Focal Point have
participated in the project? If so, how?
Yes, The project was implemented according to the UNDP National Execution Modality (NEX) which aims
to ensure that projects are executed in close partnership with the relevant ministries and local
government departments and in collaboration with local communities and community based
organisations. Con Dao National Park (CDNP) managed the project on behalf of the Ba Ria - Vung Tau
Provincial Peoples‟ Committee (BR-VT PPC) through a Project Management Unit (PMU) housed in CDNP
offices on Con Son island. Technical support was provided by UNDP, through a Senior Technical Advisor
(STA). The World Wide Fund for Nature (WWF) also provided technical support to some UNDP / GEF
project components and was responsible for managing DANIDA funded components. Key community and
public sector stakeholders were involved through a range of consultative processes and through direct
involvement in a number of project activities.
b.
Has the level of country ownership affected project outcomes and sustainability? If so, in what
ways?
The project aligned effectively with national and provincial strategic frameworks and established
good ownership of key project outcomes with Con Dao National Park, who are the main agency
responsible for biodiversity conservation in Con Dao. This positive outcome is strongly associated
with the consultative design process, NEX modality and lead implementation of the project by
CDNP.
5
IA & EA Execution
a.
Assessment of the quality of Implementing Agency Execution:
MU
 Focus on results
 Adequacy of supervision
 Quality of risk management
 Responsiveness to significant implementation problems (if any)
 Quality and timeliness of technical support to the project team
 Candor and realism in supervision reporting
 Suitability of chosen executing agency for project execution
Comment:
UNDP delegated execution to the Provincial People's Committee of Bia Ria-Vung Tau, who took
responsibility for quality assurance and supervision of the project management, carried out by
the Con Dao National Park
b.
The dual funding arrangements for the project and the institutional roles of both PMU and
WWF in managing different but inter-dependent components of the project created an unusual
and somewhat complex implementation modality: This and the fact that WWF are based in
Hanoi whereas the PMU were based at the project site in Con Dao, combined with the logistical
difficulties of getting to Con Dao, caused a number of difficulties for effective project
implementation. Discussions during the final evaluation with PMU, CDNP, UNDP and WWF
indicate that a complex interplay of issues was involved which impacted negatively on the
effectiveness of implementation.
It is difficult at project end to gauge results, when the WWF/DANIDA portion extended several
months beyond the UNDP/GEF project closure.
Assessment of the quality of Executing Agency Execution:
MS
 Focus on results
 Adequacy of supervision inputs and processes
 Quality and timeliness of financial management support to the Project Team
 Quality of risk management
 Candor and realism in supervision reporting
Comment:
Having the project managed by the Government of Vietnam, with execution through the
through the Provincial People's Committee of Bia Ria-Vung Tau, ensured strong country
ownership and alignment within the national context of marine protected area management.
It also tied more closely to the project the relevant district development planning strategies
and processes. Nevertheless, aspects such as risk assessment, financial management support,
and direct supervision of the PMU, appear to be weak.
Overall Quality of Project Outcomes
(UNDP EO rate)
MS
(TE rate)
MS
Comments and justifications:
Overall the project was found to have contributed significantly to:
• Establishing an MPA framework for Con Dao and building the capacity of CDNP for marine protected area
6
management;
• strengthening consideration of environmental sustainability in District and Provincial level planning frameworks;
• Raising the awareness of a wide range of stakeholders on the importance of biodiversity conservation and
sustainable development for the Con Dao islands region, particularly in relation to sustainable tourism
development.
Overall Quality of Project Implementation / Execution
(UNDP EO rate)
U/A
(TE Rate)
MS
Comments and justifications:
IV.
Assessment of Outcomes & Sustainability5
UNDP EO
Criteria6
Terminal Evaluation
Relevance
HS
HS
 Were the project’s outcomes consistent with
the focal areas / operational program strategies
and country priorities?
Comments and justifications:
All project objectives and outcomes were evaluated to be highly relevant to the goal of catalysing
conservation and sustainable management of the globally significant coastal and marine biodiversity
and ecological processes of the Con Dao Islands Region.
The UNDP / GEF project supported Con Dao National Park (CDNP) to establish a marine protected area
management framework. It also supported the Con Dao islands region to incorporate strategic
environmental assessment (SEA) into district planning frameworks, and advocated for sustainable
tourism development. This was highly relevant to the Con Dao islands region due to the threat posed
to globally significant biodiversity by planned unsustainable levels and patterns of economic
development at project start.
MS
Effectiveness
MU
 Are the actual project outcomes commensurate
with the original or modified project objectives?
Comments and justifications:
The project had mixed results in achieving its objectives and outcomes with some strong and other
weaker outcomes.
MS
Efficiency
MS
5
Each of the TEs should use the provided rating scale (set out at the end of this template / guide). This review
should include its own ratings assessments of project relevance, effectiveness, sustainability, etc, with reference
provided on the same ratings provided in the Terminal Evaluation. The reviewer will need to justify a different
rating based on new information or indicating errors made in the TE assessment. In cases where there is
insufficient information in the TE to make a determination, the reviewer should indicated U/A = unable to assess.
6
See Guidelines for GEF Agencies… 7 – 14 for a discussion on the criteria definitions. The bulleted points are taken
from this GEF guidance. The ratings will all be based on the six point scale of satisfactory achievement except for
sustainability – which switches to likelihood of achievement (also considering the likelihood of avoidance of risks to
sustainability).
7


Was the project cost effective?
Was project implementation delayed, and if it
was, did that affect cost effectiveness?
Comments and justifications:
Yes, the considerable delay in project inception means that most core activities were in fact
implemented in two years rather than the three originally planned. The project covered a broad
breadth of areas, which was efficient in enabling the project to influence an integrated cross section
of issues relevant to biodiversity conservation but also meant that the available budget was spread
quite thinly.
Sustainability
UNDP EO
TE
(likelihood)
(likelihood)
ML
a.
Financial resources:
ML
What is the likelihood that financial resources
will be available to continue activities resulting
in continued benefits?
ML
b.
Socio-political:
ML
Is there sufficient public stakeholder awareness
and support for the continuation of activities
providing benefit?
ML
c.
Institutional framework and governance:
ML
Are required systems for accountability and
transparency plus technical know-how in place?
ML
d.
Environmental :
MU
Are there environmental risks that can
undermine the future flow of project
environmental benefits?
Comments and justifications:
 Sustainability of key biodiversity conservation and sustainable use outcomes is strongly
influenced by 'current' resource use management issues and by the continuing threats to
biodiversity conservation of planned 'future' developments. Although the project worked
strategically to influence district planning frameworks, the sustainability of biodiversity
conservation outcomes depends on on-going national and provincial planning processes and
the execution of these at the district level. It remains to be seen whether the sector master
plans which fall under the SEDP will include adequate provision for environmental impact
assessment and mitigation and whether District authorities will give the necessary weight to
consideration of environmental sustainability within developments on Con Dao.
 The project influenced tourism development planning towards more sustainable levels and
patterns of tourism development, however it did not directly capture the threat posed to
biodiversity conservation by expansion of the Ben Dam port. Proposed expansion of the Ben
Dam port has a number of risks for prioritised tourism development and biodiversity
conservation, including the potential impact of increased numbers of offshore fishers,
petroleum vessels and cruise liners on the Con Dao islands resource base.
 There are also still a number of significant future biodiversity conservation and sustainable
use threats and these represent a risk to achievement of project objectives (ongoing
8
anthropogenic and climate impacts).
Monitoring & Evaluation system
(UNDP EO rating)
(TE rating)
S
I.
Is there a sound M&E plan in place to monitor
MS
results and track progress toward achieving
objectives?
II.
Was the M&E plan sufficiently budgeted and
then carried out?
Comments and justifications:
 At project inception a revised monitoring and evaluation framework was developed with
indicators and benchmarks for monitoring project performance. Baselines were established
and stored in a database in CDNP and are to be made available to the public through the
CDNP website.
 Monitoring and evaluation systems were established focused on quantitative data and
consultation with key stakeholders to collect qualitative data. A timeline and budget were
established and the project reporting system was set up to meet UNDP / GEF standards. The
logframe was reasonably well structured and provided a useful tool for project implementers
and managers. The indicators selected to measure the achievement of outcomes and
objectives were clear and measurable although they could have been strengthened to
improve focus on achievement of key outcomes and impacts and to improve monitoring and
evaluation.
 The experience of CDNP in using project monitoring and evaluation systems and SMART
indicators has strengthened their capacity for effective monitoring and evaluation. CDNP staff
also received training in biodiversity monitoring and assessment under the project and a
biodiversity monitoring and evaluation system and database was established.
 Neither the MTE nor TE provided information on the M&E budget, however the TE evaluator
indicated the budget and timing were insufficient to carry out the assignment.
Overall Project Rating
MS
MS
Comments and justifications:
Key indicators were not successfully met. The Goal level indicators assess the extent to which the
project has achieved biodiversity conservation and sustainable use and do not include indicators that
could assess the catalytic role of the project. The focus of the Goal level indicators is firmly on the
ultimate marine and coastal conservation objective of maintaining or enhancing levels of biodiversity
(coral and key species). In its evaluation against these indicators, the May 2009 Performance
Indicators Report assessed that the project cannot be shown to have had a significant impact in
increasing or maintaining biodiversity levels. Although the project has not secured conservation and
sustainable use of the globally significant coastal and marine biodiversity of the Con Dao islands
region, it has acted as a catalyst to improve conditions for biodiversity conservation and sustainable
use in Con Dao in a number of ways.
V.
Impact7
7
All TEs discuss sustainability but this is not synonymous with impact. It is important to look for references to
verifiable pollution reduction and ecological status improvement, and/or whether there are process indictors that
suggest such impacts should occur in the future as a result of project achievements. If the project is a foundation
9
a.
The project has demonstrated verifiable improvement in ecological status.
b.
Comment:
The project has not been successful in reducing actual current resource use pressures and therefore
in improving or maintaining biodiversity levels in the marine and coastal areas of the Con Dao
islands region.
The project has demonstrated verifiable reductions in stress on ecological systems.
No
c.
d.
e.
No
Comment:
See above. There are also still a number of significant future biodiversity conservation and
sustainable use threats and these represent a risk.
The project has demonstrated through specified process indicators that progress is
Yes
being made towards achievement of stress reduction and/or ecological
improvement.
Comment:
The project indicators emphasize capacity building and development planning. The project has
contributed to strengthening conditions for biodiversity conservation and sustainable use in Con
Dao and to reducing the likelihood of future threats linked to unsustainable levels of tourism
development in District planning agendas.
As a result of the project, there have been regulatory and policy changes at regional, No
national and/or local levels.
Comment:
Establishing an MPA system for Vietnam has been a national priority for some time, but final
legislation is pending the results of different pilot models of MPA administration.
As one of the first MPAs to be established in Vietnam, the project's outputs and outcomes have the
potential to guide future MPA establishment in Vietnam
Summarize the achieved intended or unintended impacts of the project:
Overall the project was found to have contributed to:
• Establishing an MPA framework for Con Dao and building the capacity of CDNP for marine
protected area management;
• strengthening consideration of environmental sustainability in District and Provincial level
planning frameworks;
• Raising the awareness of a wide range of stakeholders on the importance of biodiversity
conservation and sustainable development for the Con Dao islands region, particularly in
relation to sustainable tourism development.
These positive outcomes reflect the emphasis of project indicators on capacity building and
development planning. The project also established good ownership of key project outcomes
with CDNP and aligned effectively with national and provincial strategic frameworks.
It should be noted however that the two key indicators for achieving the project's main goal:
stable or increasing live coral populations and key fish species populations maintained or
enhanced, were only partially met. The May 2009 performance indicators showed that while
live coral cover was stable or increasing in most areas, key species populations continued to
decline, including coral reef fish, giant clams and nesting turtles.
setting effort, it is not expected to achieve stress reduction and/or status change impacts in the short to medium
term.
10
VI.
a.
Catalytic Role8
Production of a public goodLowest level of catalytic result, including for instance
development of new technologies and approaches. No significant actions were
taken to build on this achievement, so the catalytic effect is left to ‘market forces’.
Yes
Comment:
Project was designed and carried out in a way that offered new approaches for Vietnam in
fisheries management, through the establishment of the marine protected area. The project
did not break new ground in terms of international innovative practices, but did offer a new
approach for Vietnam, in particular with respect to stakeholder involvement.
Demonstration Steps have been taken to catalyze the public good, for instance
Yes
through the development of demonstration sites, successful information
dissemination and training.
Comment:
Project included and was focused on the Con Dao island region, including three MPA
demonstration sites.
Replication Activities, demonstrations, and/or techniques are repeated within or
Yes
outside the project, nationally or internationally.
b.
c.
Comment:
The Con Dao effort constitutes 3 of 15 demo MPA sites in Viet Nam, with government plans to
now complete the development of national policies and procedures based on lessons learned,
and to replicate across its coastal waters.
Scaling up Approaches developed through the project are taken up on a regional /
No
national scale, becoming widely accepted, and perhaps legally required.
d.
Comment:
Although there was (and still is) no national legislation in place for MPAs, the Projects aims and
objectives are consistent with the principles and strategies outlined in MASPAS and with the
existing fisheries law.

Mainstreaming9
8
The GEF EO Guidelines on TERs notes that at this point there is no specific guidance for assessing catalytic roles
and replication in GEF projects. The expectation is that this will be required in the future, hence a section in the
review effort to consider four levels of catalytic results. No ratings are expected, however the reviewer should
consider the extent to which the project has achieved: a) production of a public good, b) demonstration, c)
replication, and d) scaling up.
9
In addition to UNDP/GEF objectives aimed at global environmental benefit, other UNDP objectives should be
considered in the Review. These include the extent to which mainstreaming of key UNDP objectives on poverty
alleviation, crisis prevention and recovery, and democratic governance have been taken into account.
Increasingly, UNDP is expecting that these objectives are acknowledged in UNDP-GEF projects; however this has
not been a steadfast requirement in the past. It will be useful to note instances where such mainstreaming may
have occurred, in order to emphasize these linkages. For instance, if environmental protection activities have
contributed to improvements in local economic well being, better preparations to cope with natural disasters, and
broader improvements in governance, this should be mentioned. Both UNDP and GEF are focusing greater
11
a.
b.
c.
d.
VII.
a.
Based from the ProDoc and Terminal Evaluation, has the project focused attention on the nexus
of issues connecting environmental protection with other socio-economic priorities of UNDP such
as poverty reduction, crisis prevention and recovery, and democratic governance?
Yes, The project focused very much on sustainable livelihoods, however it did not focus in any
way on climate change nor the other goals of UNDP support through its practice areas.
Is the focus of the project identified as a priority in the UNDP country programme and as a
national/regional priority in a corresponding development plan?
The Con Dao islands region is listed as a highest priority conservation area in Vietnam’s
Biodiversity Action Plan and in the National GEF strategy. Con Dao is among the first three of
fifteen proposed marine protected areas (MPA) which the Vietnamese Government aims to
establish by 2010. Establishing an MPA system for Vietnam has been a national priority for some
time, but final legislation is pending the results of different pilot models of MPA administration. The
project aligned with national Vietnamese development priorities and plans, including the National
Management Strategy for a Protected Area System (MASPAS). There is provision for the
establishment of 15 MPAs by 2010 with Con Dao included as one of three MPA pilot projects.
Although there was (and still is) no national legislation in place for MPAs, the Projects aims and
objectives are consistent with the principles and strategies outlined in MASPAS and with the
existing fisheries law
Have gender issues been taken into account in project design and implementation (i.e. project
team composition, gender-related aspects of pollution impacts, stakeholder outreach to women’s
groups, etc.)? If so, indicate how.
Gender issues were not especially well considered. They were not a factor in the ProDoc, and
not a consideration in PMU hiring. The APR/PIR indicates that there (one) women union
representative was a member of the Condao Consultative group and actively participated in the
awareness raising and planning and implementation of the project activities. Neither the MTE
nor TE discuss gender issues even though it is mentioned as an issue to review in the MTE ToR.
Is it possible to identify and define positive or negative effects of the project on local populations
(e.g. income generation/job creation, improved natural resource management arrangements
with local groups, improvement in policy frameworks for resource allocation and distribution,
regeneration of natural resources for long term sustainability)?
TE notes that fishers involved in the coral planting initiative viewed coral replanting as a potential
income earning opportunity. However no indication is given whether additional income was
generated. The WWF/DANIDA effort was focused very much on alternative income generation. The
results were not impressive. A seaweed farming project involved three households, with the project
abandoned after 3 months due to parasites. A tourism transport pilot was developed, which
involved training 13 fishermen on tourism, and converting their boats for handling tourists. This
was part of the DANIDA funded effort that was still ongoing when the UNDP/GEF project closed. So
both of the AIG projects had demonstrated no success. The project also worked with local
stakeholders on a variety of awareness raising efforts.
Project Lessons & Recommendations
Summarize the key lessons, good practices and approaches mentioned in the TE that may be
applicable to other projects:
 Project demonstrates positive impact of fostering good ownership of project objectives and
attention to ensure that gender issues are taken into account in project formulation and implementation, (see
UNDP Gender Equality Strategy 2008-2011).
12
b.
c.
outcomes within the national agency responsible for biodiversity conservation and with
relevant Government institutions responsible for development planning.
 Project had to overcome resistance and fears from local fisherman, underscoring the need
to include fishermen in project design so that they can contribute to, have an
understanding of, and sense of ownership of MPA project objectives and outcomes
 Viable options for effective alternative income generation in remote locations such as Con
Dao may be limited.
 Stakeholders can sometimes emerge during the life of the project that have a key impact
on the achievement of project outcomes. In the case of the Con Dao project this was an
external stakeholder group, the offshore fishermen. Arrangements need to be made to
bring them into the process if possible.
 Enforcement is a key issue for achieving sustainable use and biodiversity conservation in
Con Dao. The increasing number of offshore boats in the area is beyond the capacity of
current enforcement systems to police. Weakness in MPA enforcement is exacerbated by
the complexity of national legislation relative to patrolling and enforcing marine protected
areas.
 lessons to be learnt on the potential impact of delays in achieving project outcomes by
project end. This relates both to recognition of the difficulty of recruiting staff to remote
locations and to the sequencing of activities within a project.
 The involvement of two implementing agencies with different financial reporting
requirements to different donors also had an impact on project implementation. Although
both WWF and PMU within CDNP contributed significantly to the project, there are lessons
to be learnt on the benefit of establishing simple and streamlined implementation
modalities within projects.
 Consideration of the likely impacts of increased climatic events associated with Climate
Change is a pressing issue for Con Dao development planning, yet the project included no
consideration of climate change.
Summarize the main recommendations set out in the TE:
 it would be useful for GEF / UNDP to undertake a post project review of impact in one
to two years time. This would enable more effective evaluation of the actual impact of
the project in supporting improved biodiversity conservation and sustainable use in the
Con Dao islands region.
 When using more than on implementing agency, GEF / UNDP should seek to ensure
implementation modalities are streamlined, management responsibilities are clear and
well co-ordinated and donor inputs are harmonized.
 UNDP / GEF projects should try to ensure that co-financing is more measurable and can
be quantified. Monitoring of in-kind contributions should be undertaken during project
implementation to support evaluation
 projects need to consider how fishermen can be represented within MPA project
design and implementation.
 In order to achieve results with alternative income generation:
•
broadly assess livelihood support opportunities as part of design
•
ensure that appropriate timeframes and budget are allocated to maximise the
potential for effective AIG outcomes.
•
consider other ways to improve target beneficiary groups‟ existing livelihoods,
if AIG opportunities are limited, towards increasing their overall wellbeing
Are there lessons and recommendations stemming from the project outcomes and
No
impacts that were not mentioned in the TE?
13
Comment (suggest lessons and recommendations):
VIII.
a.
b.
c.
Quality of Terminal Evaluation Report
Assessment Criteria10
Rating
Does the terminal evaluation report (TE) present an assessment of all relevant
S
outcomes and achievement of project objectives in the context of the focal area
program indicators, if applicable?
 Project outputs and outcomes are assessed against ProDoc/LFA plans,
including verifiable indicators.
 Variances between planned and actual results have been identified and
assessed, including adaptive management strategies used.
 TE has addressed project contributions towards GEF objectives (global
environmental benefit) as well as national development goals and
strategies.
Comment:
The TE provides an assessment of the achievement of relevant objectives and associated
outcomes against the ProDoc/LFA, identifies variances and discusses at length adaptive
management strategies used between planned and actual results. The TE addresses project
contributions towards GEF objectives and national development goals. Each objective was
rated against relevance, effectiveness and efficiency.
Is the TE consistent, the evidence presented complete and convincing, and the
MS
ratings well substantiated?
 Assessment of project results is well organized, clearly written and
unambiguous.
Comment:
The Evaluators provided ratings and reasons for the achievement of each objective and
indicator. These are clearly explained and well organized within the TE, with supportive
information. The TE was unable to provide useful information on the financial aspects of the
project, indicating that this information was unavailable.
Does the TE present a sound assessment of the sustainability of project outcomes?
S
 Project design measures and/ or strategies for sustaining project results are
assessed.
 Socio-political risks to sustainability are considered, including stakeholder
ownership.
 Institutional frameworks and governance risks to sustainability are
considered, including requisite systems for accountability and transparency
and required technical know-how.
 Financial and technical resources required for sustaining project results are
10
The first six of these criteria are taken directly from the Guidelines for GEF Agencies in Conducting Terminal
Evaluations (pg. 16); the rest are additional questions to get a better fix on TE quality. The bullets under each initial
statement have been added as additional points to aid in considering TE quality.
14
considered.
d.
e.
Comment:
The TE assesses the major components of sustainability and replicability issues and uses the
above mentioned categories and provided a rating for each outcome. The Evaluator provided a
number of observations and reasons for assessments.
Are the lessons and recommendations listed in the TE supported by the evidence
MS
presented and relevant to the GEF portfolio and future projects?
 The lessons learned draw upon specific observations or data compiled
during the evaluation.
 The recommendations provide specific advice for the project exit strategy or
post-project sustainability.
 The recommendations provide specific advice for future projects or
programming, or similar projects.
 The recommendations are sufficiently practical and in the realm of
feasibility for potential implementation.
 The lessons learned and recommendations are concise, clearly written and
understandable.
Comment:
The TE provided important lessons and recommendations for Vietnam in developing marine
protected areas, as well as for UNDP and GEF project management.
Does the TE provide the actual project costs (totals, per activity, and per source) and MU
actual co-financing used?
 Project cost and funding data are presented, including actual co-financing
from each source.
 Variances between planned and actual expenditures are assessed and
explained.
 Observations from financial audits completed for the project are considered.
 Issues related to financing and co-financing commitments and performance
are discussed and explained.
Comment:
TE explains that at the time of the final evaluation mission complete information was not
available on actual expenditure. A final project audit and completion report was then supposed
to be prepared by the project, (however it is not included yet in the PIMS 5 months later). The
evaluators decided not to 'preempt' the audit results and therefore made no effort to analyze
project expenditures, claiming the data was not available. Furthermore, the PMU indicated it
did not have information on DANIDA funded activities which were still ongoing. It was
indicated that the inter-relation between GEF and DANIDA funded components, lack of
complete financial data sets, and packaging of activities within existing PMU data sets, meant
that it was difficult to draw conclusions as to how expenditure impacted on achievement of
outcomes. The TE also indicated it was not possible to quantify the extent to which in kind cofinancing was effectively or incrementally provided to the project as provision of staff time and
amenities were not quantified by the project and it was therefore not possible to measure or
quantify these at project end. The evaluators indicate they nevertheless provided some
15
f.
g.
h.
discussion on budgets and financial reporting and the affect on activities and outcomes based
on currently available data, however it is very limited.
The TE reviewer does not find this explanation compelling. There was clearly a breakdown in
communication and information flow between the PMU and the TE evaluation team. It should
have been possible to utilize ATLAS budget information, and also the APR/PIRs, to at least have
some discussion on project finances, the initial budget that was set out, how much co-financing
was actually received. The TE makes frequent reference in the text about the limitations on
objectives based on the 'available budget' but then doesn't indicate anywhere what that
available budget was.
Does the TE include an assessment of the quality of the M&E plan at entry, the
MS
operation of the M&E system used during implementation, and the extent M&E was
sufficiently budgeted for during preparation and properly funded during
implementation?
 The existence and quality of the M&E plan are assessed, including baseline
conditions, methodology and roles and responsibilities.
 The extent to which M&E were sufficiently budgeted and funded during
project preparation and implementation is assessed.
 The effectiveness of monitoring indicators from the project document for
measuring progress and performance is assessed.
 Compliance with the progress and financial reporting requirements/
schedule is assessed, including quality and timeliness of reports.
 The value and effectiveness of the monitoring and evaluation reports and
process was discussed with participants and assessed.
 The follow-up actions, or adaptive management, taken to respond to
monitoring and evaluation reports is assessed.
Comment:
The TE includes an assessment of the existence and quality of the M&E plan set out in the
project inception report. The TE provides a review of the quality and implementation of the
M&E plan and the effectiveness of monitoring indicators during the duration of the project as
for longer-term environmental monitoring. The Evaluators does not present assessments of the
extent to which M&E was budgeted and funded. Also, the TE indicates that the M&E plan was
developed and set out in the Inception Report, yet the MTE evaluator indicates that it had not
been finalized and implemented by the time of the midterm.
Has the TE identified the processes and factors that affected the attainment of
Yes
results?
 Including (inter alia): project design; country ownership; stakeholder
involvement; financial planning; project implementation; Executing Agency
supervision; co-financing.
Comment: The TE explained these factors in detail and commented on their quality (except for
financial planning and co-financing as explained in section ©).
Does the TE provide an assessment of the project assumptions and risks as set out in Yes
the Project Document and LFA?
 The TE provides an assessment of the stated assumptions and risks, whether
they are logical and robust, and have helped to determine activities and
planned outputs.
16

i.
j.
k.
l.
Externalities (i.e. effects of climate change, global economic crisis, etc) were
considered.
Comment:
The TE discusses continuing and emerging issues and pressures, and assesses the project
assumptions and risks set out in the ProDoc. Externalities were taken into consideration.
Were key stakeholders or stakeholder groups, or a representative sample thereof,
No
consulted during the TE fact finding process?
1. The TE includes a listing of persons interviewed.
2. A questionnaire was distributed and an analysis of responses is provided.
3. Members of civil society, NGOs and private sector were given an opportunity
to provide their views.
4. As relevant, local ministry officials were interviewed.
5. Pertinent national ministry officials in addition to the environmental focal
point were interviewed.
Comment:
The TE provides a list of people interviewed and the stakeholders consulted in Annexes –
however the Annexes were not attached to the TE electronic version and not included in the
PIMS database. The TE contains a schedule of the evaluation mission.
Does the TE Terms of Reference comply with UNDP & GEF guidance, including
No
11
concerning conflicts of interest ?
Comment (gaps, conflicts, inconsistencies):
Difficult to ascertain as the TOR for the MTE and TE were not made available and not included in the
PIMS database.
Does the TE demonstrate good evaluation practice?
Yes
Comment (TE strengths and weaknesses):
TE describes the evaluator’s methodology of evaluation as well as limitations and constraints of
the mission.
Strengths:
Clear and coherent; good development (environmental, socio-economic and institutional)
context for project background information; very logical and organized so information was easy
to find and straight to the point; good analysis of the logframe, quantitative data, activities and
outputs
Weaknesses:
No analysis of gender perspective; lack of financial analysis (although TE provided explanation
for this shortcoming and provided some analysis based on currently available data); Annexes
were not attached to the TE (i.e. list of key stakeholders, planned and actual expenditure table,
PMU expenditure table, list of people consulted, list of documents consulted, TOR, etc.)
Are there any evaluation findings involving significant administrative, financial
No
and/or human resources issues that require UNDP follow up, (e.g. corruption,
mismanagement, etc.)?
(Note issues and UNDP management response)
11
UNEG, GEF & UNDP guidelines stipulate that except in rare and unusual circumstances, Evaluators engaged by a
UN agency shall not have had any responsibility for the design, implementation or supervision of any of the
projects, programs or policies that they are evaluating.
17
Overall Rating for the Terminal Evaluation12
MS
Justification:
The overall TE was well organized, logical, easy to read and in-depth. The TE
substantiated its conclusions and findings and offered well considered and relevant
lessons learned and recommendations. The lack of financial information is a
significant weakness.
IX.
a.
Management responses, Subsequent Actions and Sustainability
A Management response13 to the evaluation was submitted.
Yes
Date: Nov 2009
14
b.
List key proposed follow-up actions :

Indicate actions that have been taken subsequently 15:
c.
X.
Additional Comments 16
XI.
Information Sources for TE QA preparation
Prodoc
Proposal
APR-PIR (2007, 2008, 2009)
MTE (2008)
TE (2009)
XII.
Information Gaps
12
Use GEF numerical ratings 1-6, whereby 6 = highly satisfactory, etc. Only the first six of the criteria set out above
(a-f) are used in the GEF TE quality rating, UNDP additions have a yes/no and comment section. To enable
consistency with the GEF EO expectations, use the GEF calculation for assessing the overall quality of the TE report
= o.3*(a+b) + 0.1* (c+d+e+f). This weights the overall rating towards relevance and achievement plus report
consistency and substantiation.
13
The responsible UNDP country office / regional bureau should provide a response to the TE including actions
that will be taken as a result of the findings. This gets posted to the Evaluation Resource Center.
14
Of particular interest are the activities planned to scale up and replicate project outputs.
15
Consideration of subsequent actions by the responsible office will require a voice or written contact.
16
TE QA Reviewers should note any additional issues of interest / concern including ‘lessons’ of relevance to
subsequent evaluations and TE QA efforts.
18
XIII.
Ratings Scale
General ratings17:
 Highly Satisfactory (HS): no shortcomings
 Satisfactory (S): minor
 Moderately Satisfactory (MS):moderate
 Moderately Unsatisfactory (MU):
significant
 Unsatisfactory (U): major
 Highly Unsatisfactory (HU): severe
 Not applicable (N/A)
 Unable to assess (U/A)
Project sustainability ratings:
 Likely (L): negligible risks to sustainability
 Moderately Likely (ML) : moderate risks
 Moderately Unlikely (MU): significant
risks
 Unlikely (U): severe risks
 Highly Unlikely (HU)
 Not Applicable (N/A)
 Unable to Assess (U/A)
17
The ratings are supposed to be included in each of the TEs, based from GEF guidance. Shortcomings relate to
achievement of objectives in terms of relevance, effectiveness or efficiency, as well as the quality of M&E systems.
This same scale should be used to rate shortcomings in the terminal evaluation report.
19