UNDP-GEF Terminal Evaluations Review Form
I.
Background Data
Project Information
Project Title (per project appraisal document
submitted to GEF)
UNDP Project ID:
GEF Project ID:
Region:
Countries:
GEF Operational Program / Strategic Program:
Executing Agency:
Project Partners :
Project and Evaluation Milestones
(Milestones)
CEO endorsement:
Agency approval date:
Participatory Community-based Conservation in the
Anjozorobe Forest Corridor
1290
1929
Africa
Madagascar
OP 4: Biodiversity
FANAMBY and World Wide Fund for Nature (WWF)
Ministry of Environment, Water and Forests, National
Parks of Madagascar (ANGAP), Ministry of Agriculture,
University of Antananarivo, PSDR, Fonds
d'Intervention pour le Développement (FID), local and
regional authorities
(Expected Dates)
No information available
No information available
(Actual dates)
May 21, 2003
February 26, 2004
*ProDoc Signature Date
Implementation start:
Midterm Evaluation:
Project Completion:
Terminal Evaluation Completion:
TE & QA Review particulars
Terminal Evaluation author(s):
TE QA Reviewer:
TE QA Review date:
Draft TE QA Submission Date:
Final TE QA Submission Date:
UNDP EO task manager approval date & sign:
II.
a.
January 2004
September 2006
December 2007
At end of the project
March 2004
September 2006
March 31, 2008
April 2008
Dr. Dominique Roby
Alain Lafontaine
November 6, 2010
November 19, 2010
20, December 2010
Alan Fox, 22, December 2010
Project Design & Objectives
List the overall environmental objectives of the project:
Biodiversity and habitat are conserved in the globally significant highland forest corridor of Anjozorobe,
Madagascar
1
b.
List the development objectives of the project:
No development objective is specifically listed nor identified in the Pro Doc
c.
Identify the key project components:
There are six key project components also called project outcomes and objectives:
 Component 1: Reliable and updated socio-economic and biological database of the Corridor’s
ecosystems*
 Component 2: Creation of the first regional forest reserve (development of local and regional
conservation policies, development of management plan and replicable forest reserve system
 Component 3: Development and implementation of a Participatory Resource Management Plan, using
the three-tiered model
 Component 4: Development of the strategic plan for land tenure security and control of itinerant
agriculture practices.
 Component 5: Establishment of an innovative tax system for financial sustainability of the RFR
 Component 6: Development and testing of sustainable harvesting techniques, alternative income
generation activities, and intensive sustainable agriculture
*The ProDoc outlines six project components as outlined above. While the logical frame matrix outlines the
same six components, the evaluators found inconsistencies in language used between documents (Proposal,
ProDoc, Logical frame Matrix, PIRs).
d.
Briefly assess the quality of project design, noting in particular the logical connections of
objectives to outcomes and outputs taking into account risks and assumptions, and the proposed
budget.
The project design does not use clear and coherent Results-Based Management (RBM) principles and there
is an absence of clear outputs which logically connect to project outcomes and which logically contribute to
the overall objective/goal of the project. Outcome statements are primarily presented as outputs (i.e.
development and testing of strategic plan for land tenure security). There are however clear linkages from
activities to output statements. In terms of risks and assumptions, the project documents clearly outline key
assumptions as related to each component area, however risks are not taken into account, nor are
mitigation measures planned for in such cases.
e.
Have indicators been established for the achievement of project outputs and outcomes? If so do
they build from GEF guidance, are they SMART?1
Indicators have been developed in the logical frame matrix, however indicators are not considered SMART
as per GEF guidance and in fact are phrased as output and outcome result statements as opposed as
statements of measurement for stated expected results. Interestingly, a separate monitoring plan has been
established for ecological and socio-economic monitoring to "better orient actions targeting sustainable
natural resource management and biodiversity conservation" using properly formulated indicators (i.e.
Number of endemic species protected), but no where are these indicators linked to the overall results
framework of the project.
f.
Were there any delays in the project preparation phase, prior to project Inception? If so, why? Did
the delays result in changes to objectives and planned outcomes?
N/A
III.
a.
Project Implementation
Indicate whether there were changes in the environmental and development objectives of the
1
The GEF Secretariat has issued guidance on the use of “SMART” indicators (Specific, Measureable, Achievable and
Attributable, Relevant and Realistic, and Time-bound, Timely, Trackable and Targeted). GEF climate change
indicators were developed in 2000, IW in 2002 & 2003, and biodiversity in 2003. For a description, see
www.gefweb.org/MonitoringandEvaluation/MEPoliciesProcedures/MEPIndicators/mepindicators.html
2
project during implementation, why these changes were made and what was the approval
process 2
The project objective as stated in the ProDoc: Biodiversity and habitat are conserved in the globally
significant highland forest corridor of Anjozorobe, Madagascar, was rearticulated as The biodiversity and
habitat in the Forest Corridor of Anjozorobe are conserved and used in a sustainable manner by the MidTerm Evaluation and finally to Conserve and develop the habitats and biodiversity in the Forest Corridor of
Anjozorobe–Angavo in partnership with, and to the benefits of, women and men living there. These
changes are not stated in any PIRs nor are there explanations to changes. While minimal, consistency in
project objectives in all documents is required.
b.
Were there delays in project implementation and completion? If so, indicate reasons and
comment on whether the delays affected project outcomes and sustainability.
The project completion was delayed by three months (March 2008 as opposed to December 2007) as
project commenced six months late.
Co-financing3
Source of co-financing
Total4
Type (cash or in-kind)
Expected
100,000
Actual
254,664
host gov’t contribution (Malagasy)
In-Kind
GEF Agency (ies)
GEF UNDP
Cash
975,000
936,166
UNDP
Cash
100,000
139,850
Other agencies (bilateral / multilateral)
Private sector (Boogie Pilgrim)
In-Kind
40,000
37,500
NGOs
FID
Cash
30,000
20,000
FANAMBY
In-Kind
60,000
63,000
WWF
Cash
130,000
108,427
Tanya Meva
Cash
70,000
21,322
Other
85,709
Local Communities
In-kind
40,000
Total co-financing
1,545,000
1,666,638
a.
Is there sufficient clarity in the reported co-financing to substantiate in-kind and cash (Yes/no)
co-financing from all listed sources?
Yes
2
The GEF guidance indicates the following possible reasons for changes: a) original objectives were not sufficiently
articulated; b) exogenous conditions changed, due to which a change in objectives was needed; c) project was
restructured because original objectives were overambitious; d) project was restructured because of a lack of
progress; e) other (specify).
3
GEF focuses considerable attention on the extent of co-financing across the portfolio, and uses co-financing as a
proxy for country ownership as well as an important indicator for sustainability. Accordingly, this table should be
filled in based on the numbers set out in the ProDoc for expected co-financing and then in the TE and/or Project
Close out note, on actual or realized co-financing. Especially in GEF-2 & 3 projects there may be instances where
no actual co-financing numbers have been tabulated, and/or no cash & in-kind breakouts have been provided.
4
Includes both co-financing during project design stage (PIF/PPG) and Project Implementation (ProDoc)
3
Comment:
Financial reporting and analysis is clear.
b.
If there is a difference in the level of expected and actual co financing, what were the
reasons?
Comment (to what extent, and note whether justifications were given)
(Yes/no)
Yes
There were considerable variances in the expected and actual co financing and overall payments made as of
March 31, 2008 were $121,638 more than planned. Contribution from Tany Meva was suspended for the
last phase of the project due to a change in its operation policy and WWF contributions were $21,573 lower
than expected due to modifications in budget availability. Increase in community in-kind contribution by
$45,709 was also noted due to increased participation in interventions and Government contributions
represent twice the contribution planned in the ProDoc through direct contribution and tax exemption.
c.
Were components supported by co-financing well integrated into the overall
project?
(Yes/no)
Yes
Comment
All co-financing amounts are directly linked to the project components
d.
Did the extent of materialization of co-financing affect project outcomes and/or
sustainability
Comment
(Yes/no)
Yes
The project was able to leverage additional resources from six agencies/organizations (Partnership Program
for Eastern and Southern Africa, PSDR, ADER, WTO, Ministry of Civil Engineering and Ministry for
Decentralization and Land Planning, Alaotra) for in-kind and cash totalling approximately $269,347.
Additional comments on co-financing:
The project has been able to leverage additional resources and despite changes in planned contribution, there is
evidence that there is a suitable level of co-financing for all component interventions.
Monitoring and Evaluation
a.
Assess the quality of the M&E design at project start up
Comment:
(rate) U
The project has developed a clear M&E plan during the design phase with specific objectives: i) to analyse
project progress, impacts and achievement; ii) to assess the relationship between activities planned in the
project document and those implemented in the field, using performance indicators; iii) To re-orient the
project as needed (adaptive management); iv) To draw recommendations for future natural resources
management transfer of activities to other areas; and v) To allow inter-project evaluations and systematic
exchange (with other GEF projects). At the same time, the project design does not use clear and coherent
Results-Based Management (RBM) principles and there is an absence of demonstrable outcomes which
logically contribute to the overall objective/goal of the project. SMART indicators are additionally absent
and are stated as output and outcome statements. While the overall M&E design does include ecological
and socio-economic indicators, these would have best been used as indicators for the results framework
and not as a separate plan and are disjointed for coherent monitoring. The terminal evaluation found
similar issues with the M&E design.
Assess the quality of M&E Plan Implementation
(rate) U
Comment:
In the ProDoc, it is stated that the University of Antananarivo was responsible for the establishment of a
monitoring and evaluation system, yet there is no indication of such a system being used. The M&E plan
also stated that, "the project will be monitored and evaluated in close collaboration with WWF and the
Administration, and will follow the guidelines established by UNDP-GEF...[and] will include monitoring of
project progress, ecological monitoring and socio-economic monitoring". PIR documents do not include
progress on these indicators and reporting on results is limited to stating if outputs have been completed.
4
For example, PIR 2007 indicated that monitoring on agriculture indicators (although absent from the logical
framework) is being conducted "on a normal basis", yet actual results are missing. The M&E plan also
suggested the monitoring of impact indicators related to income of households affected by the
intervention as well as ecological monitoring (illegal activities, fires, water, lemurs, surface of the natural
forest), however no reporting on this occurred. The terminal evaluation equally found that "there is some
ad hoc monitoring and evaluation, but no overall strategy and/or no regular collection of results with some
participatory ecological monitoring which has just started [therefore] results cannot be used yet". Some
suggestions however were implemented from the mid-term such as monitoring the flow rate of water
streams. Three pilot sites were used, however data from two sites was not considered reliable.
Was sufficient funding provided for M&E in the project budget?
A total of MGA 257,246,098 ($129,922 approx) was allocated for ecological monitoring, database
development and overall monitoring and evaluation. For M&E alone, this represents 3.71% of the overall
budget as outlined in Table 2 of the Terminal Evaluation. From the perspective of the independent
evaluator, this is not considered sufficient despite the TE's opinion that it is a suitable amount.
Was a midterm evaluation carried out and were MTE recommendations taken into account in
subsequent project implementation?
A mid-term evaluation was carried out (actual document not provided). The final evaluation did not specify
these and the degree to which they were taken into account into subsequent implementation. PIR 2008 did
note whether recommendations were carried out since MTE, including the following:
 Build communities capacities to negotiate with the private sector  information dissemination on
market information and exchange visits conducted
 Resume the production of the quarterly paper  quarterly information bulletins developed in July
and October 2007
 Intensify awareness raising activities among communities on the mid and long term benefits related to
forest conservation and sustainable management of resources on their land  training sessions
carried out on demand and quarterly basis information and communication meetings held on
regular basis
 Raise awareness of communities on the potentially harmful effects of developing tourism activities
within their living environment  boundaries for eco-tourism defined and established in November
2006
 Take into account environmental risks in the making of paths and setting up of any infrastructure 
committee established in May 2007. No indication of results of committee.
 Identify and establish the baseline status for ecological and biodiversity impact indicators  forest fire
monitoring began. No indication of baseline status being established
 Keep up activities for monitoring agricultural production  The transfer to the community of
information system is currently under way.
 Involve the Fishery Department in the monitoring of freshwater crayfish harvests trainings
conducted in May 2007
Were PIR self-evaluation ratings consistent with the MTE and TE findings? If not, were these
discrepancies identified by the project steering committee and addressed?
Ratings from PIR self-evaluation and TE findings were consistent with the majority of result areas rated as
Satisfactory.
Country Ownership
a.
Are there indications that government agencies in addition to the GEF Focal Point have
participated in the project? if so, how?

A MPAS Commission was established under the MEEFT authority to develop new tools for
management of forest protection
Involvement of Territorial Authorities includes the following:
 Participation of fokontany presidents to introductory information campaigns and to the delimitation of
the protected area, and to the organization and coordination of control and surveillance rounds with
5








b.
the Quartiers mobiles in the eastern part of the corridor;
Terminal Evaluation of the Anjozorobe-Angavo Forest Corridor Project 12
Fokontany presidents assuming more responsibility in the conception and organization of events in
favour of environment such as the World Environment Day and regional fairs on this theme.
Creation of an “Environment” section in all communes, as a result both from encouragement by the
Presidency as well as promotion, by FID, of “green” projects that contribute to environment
preservation;
Creation of an Environment Commission within every POIC;
Involvement of mayors and POIC in the development of the taxation system and introduction of a
system for the distribution of tax revenues related to forest products, with a view to fund recurrent
costs of forest patrols and controls;
Identification and mobilization of revenue sources by POIC to cover expenses related to the
enforcement of legal measures about natural resource management (pro-environmental events, forest
restoration, forest surveillance, and dispute settlement);
Involvement of mayors and POIC, fokontany presidents and administrative officers in charge of
Environment, Water and Forests in the organization of forest controls and in the transfer and follow-up
of the offense reports to the appropriate authorities;
Participation in Project Steering Committee
Has the level of country ownership affected project outcomes and sustainability? If so, in what
ways?
Increasing the surface of protected areas is one of the major priorities of the "Madagascar Action Plan" and
national ownership of the project has been high. A MPAS Commission was established under the MEEFT
authority to develop new tools for management of forest protection. There was a halt on forest
exploitation and mining licensing in the priority conservation areas in 2004 for a 2 year period and renewed
again in 2006 which according to the TE, "indicating authorities’ significant support to the establishment of
protected areas in the country". In addition, the higher than expected level of co-financing from national
stakeholders mentioned earlier was instrumental in filling the gap left by reduced co-financing from other
sources.
IA & EA Execution
a.
Assessment of the quality of Implementing Agency Execution:
 Focus on results
 Adequacy of supervision
 Quality of risk management
 Responsiveness to significant implementation problems (if any)
 Quality and timeliness of technical support to the project team
 Candor and realism in supervision reporting
 Suitability of chosen executing agency for project execution
Comment:
(rate)
S
Review of PIRs from 2005-2008 reveals that approximately 6 visits were made by UNDP and a tripartite
meeting was held in 2005. The majority of the time from the UNDP Programme officer was spent on
delivering financial and technical outputs, overall monitoring and backstopping. Significant time has been
provided for financial analysis and administrative support from the IA. There is evidence that Fanamby was
suitably chosen through assessment of their capacity to implement the new cash management system
HACT. In terms of risk management, the reporting in ATLAS (2007) does identify project risks and adequate
management response.
b.
Assessment of the quality of Executing Agency Execution:
 Focus on results
(rate)
S
6
 Adequacy of supervision inputs and processes
 Quality and timeliness of financial management support to the Project Team
 Quality of risk management
 Candor and realism in supervision reporting
Comment:
Overall Quality of Project Outcomes
(UNDP EO rate)
S
(TE rate)
S
Comments and justifications:
While the overall quality of project outcomes rated satisfactory, notable was ratings of HS by the TE for income
generation projects (ecotourism) that protect forests and which are implemented by communities and analysis of
data for development planning. A high level of participation from community stakeholders and the installment of a
three-tier management system also successful however focus on ensuring awareness of the benefits of alternative
incomes for communities is needed in the extended phase.
Overall Quality of Project Implementation / Execution
(UNDP EO rate)
S
(TE Rate)
S
Comments and justifications:
The TE found strong communication and trust building during implementation and execution of the project with
key partners. Decisions to equip field offices with equipment facilitated closer field presence. Overall execution by
a national level NGO was advantageous in community-based planning and alternative incomes in protected areas
and established coherence in key messages around forest protection.
IV.
Assessment of Outcomes & Sustainability5
Criteria6
Relevance
 Were the project’s outcomes consistent with
the focal areas / operational program strategies
and country priorities?
Comments and justifications:
UNDP EO
(rate)
S
Terminal Evaluation
(rate)
TE didn’t assign a rating
for Relevance
Project outcomes were consistent with the focal area of biodiversity conservation and in line with the country
priorities as outlined in the MSAP. Strategies around land tenure consistent however no outcomes achieved on this
front and only a fraction of land plots in each fokontany supported that may possibly lead to land tenure.
Strategies on income generation of alternative products, including ecotourism, and the links between benefits
attributed to development of these products and the conservation of biodiversity or the management of the
5
Each of the TEs should use the provided rating scale (set out at the end of this template / guide). This review
should include its own ratings assessments of project relevance, effectiveness, sustainability, etc, with reference
provided on the same ratings provided in the Terminal Evaluation. The reviewer will need to justify a different
rating based on new information or indicating errors made in the TE assessment. In cases where there is
insufficient information in the TE to make a determination, the reviewer should indicated U/A = unable to assess.
6
See Guidelines for GEF Agencies… 7 – 14 for a discussion on the criteria definitions. The bulleted points are
taken from this GEF guidance. The ratings will all be based on the 6 point scale of satisfactory achievement except
for sustainability – which switches to likelihood of achievement (also considering the likelihood of avoidance of
risks to sustainability)
7
protected area has not always been clearly established.
Effectiveness
No rating available
MS
 Are the actual project outcomes commensurate
with the original or modified project objectives?
Comments and justifications:
TE didn’t assign a rating
for Effectiveness
In looking at the six objectives of the project in the ProDoc and PIRs for each year of the project, a heavy focus on
the implementation of a participatory resource management model for a conversation site was effectively
delivered. At the same time, it is unclear based on reporting if the objectives which focused on testing Forestry
Brigade Model, program for land tenure security and innovative tax system and better adoption of intensive
agriculture technique are commensurate with outcomes as progress reported is activity focused and outcomes of
this nature require more detailed assessment.
TE didn’t assign an
Efficiency
No rating available
official rating for
MS
 Was the project cost effective?
Efficiency

Was project implementation delayed, and if it
was, did that affect cost effectiveness?
Comments and justifications:
The TE did not provide an official rating, however has stated that "overall, the project achievement rate (91%) and
rates vary little between results and co-funding sources, showing that planned activities were carried out
according to the planning". At the same time the TE stated that it was not possible to determine the cost-efficiency
of the project activities, except for protected area management.
Sustainability
a.
b.
c.
d.
Financial resources:
What is the likelihood that financial resources
will be available to continue activities resulting
in continued benefits?
Socio-political:
Is there sufficient public stakeholder awareness
and support for the continuation of activities
providing benefit?
Institutional framework and governance:
Are required systems for accountability and
transparency plus technical know-how in place?
Environmental :
Are there environmental risks that can
undermine the future flow of project
environmental benefits?
Comments and justifications:
UNDP EO
(likelihood)
No rating available
L
TE
(likelihood)
TE didn’t assign an
official rating for
Sustainability
No rating available
L
TE didn’t assign an
official rating for
Sustainability
No rating available
ML
TE didn’t assign an
official rating for
Sustainability
No rating available
ML
TE didn’t assign an
official rating for
Sustainability
a) The project has been able to leverage resources from additional stakeholders (both in-kind and cash) focused
specifically on land tenure issues, safeguard sub-projects to compensate restricted resource access for populations
affected by the creation or extension of protected areas in the framework of the EP III, development of ecotourism
products in the fokontany of Antashabe, and general equipment costs.
b) The project has ensured stakeholder participation and TE observed high level of ownership at all levels including
from local communities, local authorities, mayors, and technical services.
8
c) Project strategies around land tenure offices still in establishment phase with POIC and PNF and sustainability is
contingent on commune contribution to land tenure office operating costs and functionality of communal agents
which are relatively new. Other systems such as taxation and capacity enhancement has been positive, but a
further phase of accompaniment is necessary to attain autonomy.
d) impacts on neighbouring forests could undermine future flow of project's targeted environmental benefits
through pressure transfer from one forest to another. Environmental impact assessment recommended to
minimize such risks.
Monitoring & Evaluation system
 Is there a sound M&E plan in place to monitor
results and track progress toward achieving
objectives?
 Was the M&E plan sufficiently budgeted and
then carried out?
Comments and justifications:
UNDP EO rating
MU
(TE rating)
TE didn’t assign an
official rating
A sound M&E plan was designed although implementation of the plan is unclear. Monitoring of progress also
unclear and absence of baseline data and information does not facilitate demonstrable change. Budget for
monitoring insufficient as outlined in aforementioned sections. Overall, the TE equally evaluated M&E activities as
insufficient.
Overall Project Rating
No rating available
S
(TE rating)
S
Comments and justifications:
Based on information available in TE, the project has had relevant success in achieving its objectives overall. Issues
around sustainability of protected areas and ordinances which will keep areas protected is still unknown. However
a high level of national and community level participation. Extended phase design is already remedying issues
which have been identified in the TE specifically around monitoring, sustainability and increasing awareness
around protection of areas against exploitation.
V.
a.
Impact7
The project has demonstrated verifiable improvement in ecological status
(Yes/no)
Essentially
yes, but
verifiable
improvements
within time
span not
possible
Comment:
According to TE, the establishment and legalization of a system including several protection levels,
comprising strict preservation areas, controlled use zones and sustainable resource use zones is the major
impact of the project. This includes legal status for 52,200 ha territory comprising of 28,000 ha of natural
which has had provisional protection since December 2005 (order 20.023 / MINENVEF – 2005) and
prorogated for a 12-month period (order 380 / 2007 / MINENVEF). At the same time, at the time of the
7
All TEs discuss sustainability but this is not synonymous with impact. It is important to look for references to
verifiable pollution reduction and ecological status improvement, and/or whether there are process indictors that
suggest such impacts should occur in the future as a result of project achievements. If the project is a foundation
setting effort, it is not expected to achieve stress reduction and/or status change impacts in the short to medium
term.
9
evaluation, the MPAS commission was drawing up an order which will extend the management delegation
contracts for various protected areas which have a provisional status and which temporary protection
period will expire once protected areas will be gazetted. In terms of improving social degradation and
deforestation, low water flows can be restored by forestation however cannot be determined within life of
the project. Where surface erosion is severe, forestation may reduce erosion and sedimentation within 10
to 20 years. Finally, Species inventories established previously to the project do not provide information on
the abundance and distribution of target species.
b.
The project has demonstrated verifiable reductions in stress on ecological systems
(Yes/no)
Yes
Comment:
As above. In addition, the evaluation demonstrated increased efforts for the sustainable development of
forest products and forest restoration and notable according to the TE is that "pressures on the forest
corridor due to fires, illegal logging and clearing are reduced by comparison with forests outside the
protected area". The project also demonstrated that fire occurrence on edge of fire blocks inside protected
areas is lower than on edge of blocks located in north and south of protected area.
c.
The project has demonstrated through specified process indicators that progress is
being made towards achievement of stress reduction and/or ecological
improvement.
Comment
(Yes/no)
No
Minimal use of process indicators to demonstrate stress reduction. Systems however established to produce
data for biological and ecological information to inform processes at commune levels and for planning of
delimitated zones and boundary creation. Reporting on protected hectares - 52, 200 and clear protection of
five main threatened species
d.
As a result of the project, there have been regulatory and policy changes at regional,
national and/or local levels.
Comment
(Yes/no)
Yes
There has been several areas of progress in the regulatory and policy frameworks. First, orders signed in
2004 for a 2-year period and renewed in 2006 for forest protection and mining licensing in priority
conservation areas. The decree amended in December 2005 to integrate new protected area categories (III,
V and VI) planned by the MPAS and to allow resorting to other types of governance. Challenges however in
scaling up this protection into larger planning framework. Second, The POIC introduced a system to allocate
tax revenues from forest products to fund recurrent monitoring costs for forest patrols and control.
Strengthening of enforcement of existing laws pertaining to forest fires and prohibition of clearing.
e.
Summarize the achieved intended or unintended impacts of the project:



Long term impacts of the project are still unknown particularly if there have been demonstrable
reduction on the pressure of forests and protected biodiversity. Impact indicators corresponded to
operational indicators for activities rather than assessing reduction in pressure of forests and
conservation. At the same time, the project has seen legal status for 52,200 ha territory comprising of
28,000 ha of natural forest which has had provisional protection since December 2005.
The TE evaluation demonstrated increased efforts for the sustainable development of forest products
and forest restoration and notable according to the TE is that "pressures on the forest corridor due to
fires, illegal logging and clearing are reduced by comparison with forests outside the protected area".
The project also demonstrated that fire occurrence on edge of fire blocks inside protected areas is
lower than on edge of blocks located in north and south of protected area.
Impacts at the community level unknown and unable to assess although the TE determined that there
seemed to be a shift in focus from biodiversity and ecosystem conservation to the development of
sustainable alternative income generating activities.
10
VI.
a.
Catalytic Role8
Production of a public good
Lowest level of catalytic result, including for instance development of new
technologies and approaches. No significant actions were taken to build on this
achievement, so the catalytic effect is left to ‘market forces’
Comment
b.
Demonstration
Steps have been taken to catalyze the public good, for instance through the
development of demonstration sites, successful information dissemination and
training
Comment
(Yes/no)
No
(Yes/no)
Yes
Demonstration sites focused on intensification of agriculture/production so community members can
participate in private sector. Total of 13 sites established with over 360 households. The project had a
strong focus on communication and information dissemination to a variety of stakeholders and training on
forest legislation, monitoring and use of data for development planning, local tax system.
c.
Replication
(Yes/no)
Activities, demonstrations, and/or techniques are repeated within or outside the
Yes
project, nationally or internationally.
Comment
 Replication of farming techniques and extended the use of improved seeds among households and

d.
fokontany, of taxation systems among fokontany, communes and POIC, of approaches to secure land
tenure among fokontany and communes, of first level operational management structures among
fokontany and of second level among communes and POIC. Experience being adapted beyond project
target areas initiated by two other Fanamby intervention sites, with MBG and UNESCO/World
Heritage, and through the coordination of the MPAS commission (provision protection document to
establish new protected areas).
The POIC introduced a system to allocate tax revenues from forest products to fund recurrent
monitoring costs for forest patrols and control.
Scaling up
Approaches developed through the project are taken up on a regional / national
scale, becoming widely accepted, and perhaps legally required.
Comment


(Yes/no)
Yes
Approach for issuing temporary protection order and the management delegation contract were
transferred as models to all actors who are setting up protected areas in the Madagascar Protected
Area System (MPAS) framework. The MPAS also plans to draw lessons from the experience acquired
through the approach adopted by the project to set up new protected areas. Being considered
innovative as first experience which serves to improve elaboration of legal framework for MPAS under
which new protected areas will be established regionally.
The POIC introduced a system to allocate tax revenues from forest products to fund recurrent
monitoring costs for forest patrols and control.
8
The GEF EO Guidelines on TERs notes that at this point there is no specific guidance for assessing catalytic roles
and replication in GEF projects. The expectation is that this will be required in the future, hence a section in the
review effort to consider four levels of catalytic results. No ratings are expected, however the reviewer should
consider the extent to which the project has achieved: a) production of a public good, b) demonstration, c)
replication, and d)scaling up.
11
I.
a.
Mainstreaming9
Based from the ProDoc and Terminal Evaluation, has the project focused attention on the nexus
of issues connecting environmental protection with other socio-economic priorities of UNDP such
as poverty reduction, crisis prevention and recovery, and democratic governance?
The project's objective of income generation of populations inside and near protected areas through
alternative income (low intensive agriculture and ecotourism) is aligned with both issues concerning
environmental protection and socio-economic priorities. In addition, a strong focus on democratic
governance through community-based planning and conservation in tandem with local authorities and
though co-management. The ProDoc specifically outlines that the project "addresses priorities established
in the United Nations Framework Plan for Development Assistance (UNDAF) in Madagascar for the years
2005-2008 in the arenas of Governance, food security and the Environment (cross cutting objectives) although food security was not an outcome or impact from the project.
b.
Is the focus of the project identified as a priority in the UNDP country programme and as a
national/regional priority in a corresponding development plan?
No information in ProDoc
c.
Have gender issues been taken into account in project design and implementation, (i.e. project
team composition, gender-related aspects of pollution impacts, stakeholder outreach to women’s
groups, etc)? If so, indicate how.
There is no indication that gender equality issues have been taken into account into the design or
implementation aside from stating that they are beneficiaries of the intervention and promotion of tourism
industry, typically female-focused area. Equally there is no indication that any gender analysis was
conducted during the design phase to address gender equality issues related to conservation issues or
alternative income sources. Reporting in the PIR is also void of gender results or issues and data for
community participants are not sex-disaggregated. Minimal efforts to gauge community opinion, including
those of women, was conducted during field missions and while the TE included discussions with womenonly groups, overall findings from these discussions do not address how gender equality was considered or
how women have been impacted.
d.
Is it possible to identify and define positive or negative effects of the project on local populations
(e.g. income generation/job creation, improved natural resource management arrangements
with local groups, improvement in policy frameworks for resource allocation and distribution,
regeneration of natural resources for long term sustainability)?
The project is community-based and focused on multi-dimension of local populations, specifically income
generation through alternative income (ecotourism), improved natural resource management
arrangements with local authorities and the establishment of a three-tier forest management system and
overall protection of forests from clearing and mining for long term sustainability. Aside from increased
community participation in the aforementioned areas, degree to which local populations (348 HH) are
affected is unrecorded and monitoring on increased incomes unknown. Tourism activities generated
9
In addition to UNDP/GEF objectives aimed at global environmental benefit, other UNDP objectives should be
considered in the Review. These include the extent to which mainstreaming of key UNDP objectives on poverty
alleviation, crisis prevention and recovery, and democratic governance have been taken into account.
Increasingly, UNDP is expecting that these objectives are acknowledged in UNDP-GEF projects; however this has
not been a steadfast requirement in the past. It will be useful to note instances where such mainstreaming may
have occurred, in order to emphasize these linkages. For instance, if environmental protection activities have
contributed to improvements in local economic well being, better preparations to cope with natural disasters, and
broader improvements in governance, this should be mentioned. Both UNDP and GEF are focusing greater
attention to ensure that gender issues are taken into account in project formulation and implementation, (see
UNDP Gender Equality Strategy 2008-2011).
12
minimal amounts ($100) 281,000 MGA.
VII.
a.
Project Lessons & Recommendations
Summarize the key lessons, good practices and approaches mentioned in the TE that may be
applicable to other projects:
There are three main recommendations for facilitating execution of similar projects:
 Integration of the protected area in planning on a wider scale: over 3-4 years, MEEFT should evaluate
integration of protected area into larger scale land-use planning, and implementation of such planning
at the regional level to support the protected area and its impacts. In addition, national and local
authorities must be brought to a better understanding of the approach that underlies the
establishment of a protected area and its potential benefits in order to develop their ownership and
support the integration of the protected area’s objectives into the wider scale development planning.
 Ensure active involvement of local populations in management of protected area: this includes
increase in investment in community capacity and of local authorities to ensure full integration in the
protected area's management structure.
 Creation of new protected areas: prior to establishing protected areas, socio-economic impact studies
should be conducted; ecological and socioeconomic indicators should be established and baseline data
collected, particularly around household revenues; stream water monitoring (quantity and quality)
should be included in creation of protected areas as well as monitoring of surfaces and condition
(intact, deforested, regenerating, etc.); involvement of representatives from peripheral communes in
delimiting new protected areas; and environmental impact assessments should include assessment of
potential impacts of establishment of protected area on neighbouring forests, specifically on risks to
transfer pressure from one area to another in order to identify additional mitigation measures.
b.
Summarize the main recommendations set out in the TE
As above
c.
Are there lessons and recommendations stemming from the project outcomes and
impacts that were not mentioned in the TE?
Comment (suggest lessons and recommendations);
(Yes/no)
Yes
A clear M&E plan linked directly to the results framework with SMART indicators focused on socioeconomic and ecological monitoring would strengthen demonstration of project results.
VIII.
a.
Quality of Terminal Evaluation Report
Assessment Criteria10
Does the terminal evaluation report (TE) present an assessment of all relevant
outcomes and achievement of project objectives in the context of the focal area
program indicators, if applicable?
 Project outputs and outcomes are assessed against ProDoc/LFA plans,
including verifiable indicators.
 Variances between planned and actual results have been identified and
Rating
(rate)
S
10
The first 6 of these criteria are taken directly from the Guidelines for GEF Agencies in Conducting Terminal
Evaluations (pg 16); the rest are additional questions to get a better fix on TE quality. The bullets under each initial
statement have been added as additional points to aid in considering TE quality.
13
assessed, including adaptive management strategies used.
 TE has addressed project contributions towards GEF objectives (global
environmental benefit) as well as national development goals and strategies
Comment:
There is provision of all relevant outcomes since the midterm evaluation and general comparison since this
time period and all are assessed again the logical framework with use of indicators as outlined in the
ProDoc. Variances between planned and actual results not clearly articulated, but addressed throughout
the TE.
b.
Is the TE consistent, the evidence presented complete and convincing, and the
ratings well substantiated?
 Assessment of project results is well organized, clearly written and
unambiguous.
Comment:
(rate)
MS
The TE lacks clear ratings in all areas except project outcomes and does not follow standardized structure
as outlined in the ToR and evaluation guidelines. The TE was missing components as dictated by the ToR
and assessment of effectiveness, relevance and efficiency unclear.
c.
Does the TE presents a sound assessment of the sustainability of project outcomes?
 Project design measures and/ or strategies for sustaining project results are
assessed.
 Socio-political risks to sustainability are considered, including stakeholder
ownership
 Institutional frameworks and governance risks to sustainability are
considered, including requisite systems for accountability and transparency
and required technical know-how
 Financial and technical resources required for sustaining project results are
considered
Comment:
(rate)
S
All aspects related to sustainability well covered. Minimal discussion on risks to sustainability, however
contingent on regulatory framework being implemented and extended protection areas. No mention of
financial or technical resources required to sustain project results, however suggestions to sustain results
presented well.
d.
Are the lessons and recommendations listed in the TE supported by the evidence
presented and relevant to the GEF portfolio and future projects?
 The lessons learned draw upon specific observations or data compiled
during the evaluation.
 The recommendations provide specific advice for the project exit strategy or
post-project sustainability.
 The recommendations provide specific advice for future projects or
programming, or similar projects.
 The recommendations are sufficiently practical and in the realm of
feasibility for potential implementation.
 The lessons learned and recommendations are concise, clearly written and
understandable.
Comment:
(rate)
S
14
Overall lessons and recommendations are relevant, specific and clearly written.
e.
Does the TE provides the actual project costs (totals, per activity, and per source)
and actual co-financing used?
 Project cost and funding data are presented, including actual co-financing
from each source.
 Variances between planned and actual expenditures are assessed and
explained.
 Observations from financial audits completed for the project are considered.
 Issues related to financing and co-financing commitments and performance
are discussed and explained
Comment:
(rate)
HS
Financial analysis, including variances on planned and expenditures clear and well presented. Variance
explanations very clear. No mention of financial audits consulted, but issues related to financing and cofinancing well described.
f.
Does the TE include an assessment of the quality of the M&E plan at entry, the
operation of the M&E system used during implementation, and the extent M&E was
sufficiently budgeted for during preparation and properly funded during
implementation?
 The existence and quality of the M&E plan are assessed, including baseline
conditions, methodology and roles and responsibilities.
 The extent to which M&E were sufficiently budgeted and funded during
project preparation and implementation is assessed.
 The effectiveness of monitoring indicators from the project document for
measuring progress and performance is assessed.
 Compliance with the progress and financial reporting requirements/
schedule is assessed, including quality and timeliness of reports.
 The value and effectiveness of the monitoring and evaluation reports and
process was discussed with participants and assessed.
 The follow-up actions, or adaptive management, taken to respond to
monitoring and evaluation reports is assessed.
Comment:
(rate)
MU
The TE provides an overall assessment of the M&E which focuses on indicators as outlined in the M&E plan.
There is a good analysis of potential uses of economic and ecological indicators which could have been
used and is posited as a recommendation for future projects. Further detail regarding lack of baseline data,
consistency in use of indicators and results statements and suggested improvements for RBM principles
would have enhanced the TE in this area. There was no assessment whether budget for monitoring and
evaluation was sufficient, nor if data was used to inform adaptive management practices.
g.
Has the TE identified the processes and factors that affected the attainment of
results?
 Including (inter alia): project design; country ownership; stakeholder
involvement; financial planning; project implementation; Executing Agency
supervision; co-financing)
Comment:
(Yes/no)
Yes
Specifically focused on extension of protected areas based on regulatory framework - not an issue with
execution of project by EA
15
h.
Does the TE provide an assessment of the project assumptions and risks as set out in (Yes/no)
the Project Document and LFA?
No
 The TE provides an assessment of the stated assumptions and risks, whether
they are logical and robust, and have helped to determine activities and
planned outputs.
 Externalities, (i.e. effects of climate change, global economic crisi, etc), were
been considered
Comment: The TE does not provide an extensive assessment and minimal recommendation regarding
impact assessments to determine risks from transfer to neighbouring forests. The TE does mention briefly
that assumption of community involvement for sustainability has been successful.
i.
Were key stakeholders or stakeholder groups, or a representative sample thereof,
consulted during the TE fact finding process?
 The TE includes a listing of persons interviewed
 A questionnaire was distributed and an analysis of responses is provided
 Members of civil society, NGOs and private sector were given an opportunity
to provide their views
 As relevant, local ministry officials were interviewed
 Pertinent national ministry officials in addition to the environmental focal
point were interviewed
Comment:
(Yes/no)
Yes
The TE includes list of persons interviewed which involves a diversity of stakeholders including community
members, representatives from the Fokontany, government representatives from MEEFT, NGOs, producer
associations and project partners. Interview schedules for stakeholders not included as annex.
Questionnaires were not utilized as a methodology.
j.
Does the TE Terms of Reference comply with UNDP & GEF guidance, including
concerning conflicts of interest11?
(Yes/no)
Yes
Comment (gaps, conflicts, inconsistencies):
The TE Terms of Reference complied with UNDP & GEF guidance however there is no explicit statement
regarding conflict of interest.
k.
Does the TE demonstrate good evaluation practice?
(Yes/no)
Yes
Comment (TE strengths and weaknesses):
Strengths: TE is clearly written and has a suitable structure. Replication and Financial analysis is detailed.
Clear methodology and clear recommendations which are specific for replication efforts.
Weaknesses: TE does not provide required ratings as per the ToR. Sections for sustainability are not
included and where addressed does not provide in-depth analysis. Raw data and interview schedules not
available for triangulation.
l.
Are there any evaluation findings involving significant administrative, financial
and/or human resources issues that require UNDP follow up, (e.g. corruption,
mismanagement, etc)?
(Note issues and UNDP management response)
(Yes/no)
No
11
UNEG, GEF & UNDP guidelines stipulate that except in rare and unusual circumstances, Evaluators engaged by a
UN agency shall not have had any responsibility for the design, implementation or supervision of any of the
projects, programs or policies that they are evaluating.
16
Overall Rating for the Terminal Evaluation12
Justification:
Overall the TE is clearly written, although minor omission of prerequisites as outlined
(rate)
S
in the ToR and specific rations for all TE requirements could have improved the evaluation.
Clear analysis in terms of financial variances, national and stakeholder involvements and
analysis of conservation strategies. Focus on evaluation components such as efficiency,
relevance, effectiveness and sustainability could have been clearer and strengthened.
IX.
a.
Management responses, Subsequent Actions and Sustainability
A Management response13 to the evaluation was submitted
(Yes/no): Yes
Date: October 2008

b.
14
List key proposed follow-up actions
New PA project in the UNDP/GEF pipeline has been proposed. It builds largely on the Anjozerobé model
tested through the evaluated project and integrates recommendations to integrate protected area on a
wider scale through i) evaluation by MEEFT to integrate protected area into larger scale land-use planning;
ii) implement these plans at the regional scale to support the protected area and its impacts; iii) increase
understanding of national and local authorities of approach that underlies the establishment of a protected
area and integrate into wider scale development planning; iv) Perform socioeconomic impact studies prior
to the establishment of the protected area; v) Determine the ecological and socioeconomic impact
indicators and acquire baseline data, particularly concerning household revenues, before starting project
Intervention; vi) include stream water monitoring; vii) involve representatives from peripheral communities
when delimiting new protected areas; and viii) Environmental impact to include assessment of potential
impacts of the establishment of a protected area on neighbouring forests, in particular the
assessment of the risks to transfer the pressure from one forest to another.

c.
Indicate actions that have been taken subsequently 15
New PA project in the UNDP/GEF pipeline has been proposed.
X.
Additional Comments 16
XI.
Information Sources for TE QA preparation
1290 PIR (2005, 2006, 2007, 2008)
1290 Project Document
1290 Project Document Budget
12
Use GEF numerical ratings 1-6, whereby 6= highly satisfactory, etc. Only the first 6 of the criteria set out above
(a-f) are used in the GEF TE quality rating, UNDP additions have a yes/no and comment section. To enable
consistency with the GEF EO expectations, use the GEF calculation for assessing the overall quality of the TE report
= o.3*(a+b) + 0.1* (c+d+e+f). This weights the overall rating towards relevance and achievement plus report
consistency and substantiation.
13
The responsible UNDP country office / regional bureau should provide a response to the TE including actions
that will be taken as a result of the findings. This gets posted to the Evaluation Resource Center
14
Of particular interest are the activities planned to scale up and replicate project outputs.
15
Consideration of subsequent actions by the responsible office will require a voice or written contact.
16
TE QA Reviewers should note any additional issues of interest / concern including ‘lessons’ of relevance to
subsequent evaluations and TE QA efforts.
17
1290 Project Proposal
1290 Terminal Evaluation
XII.
Information Gaps
XIII.
Ratings Scale
General ratings17:
 Highly Satisfactory (HS): no shortcomings
 Satisfactory (S): minor
 Moderately Satisfactory (MS):moderate
 Moderately Unsatisfactory (MU):
significant
 Unsatisfactory (U): major
 Highly Unsatisfactory (HU): severe
 Not applicable (N/A)
 Unable to assess (U/A)
Project sustainability ratings:
 Likely (L): negligible risks to sustainability
 Moderately Likely (ML) : moderate risks
 Moderately Unlikely (MU): significant
risks
 Unlikely (U): severe risks
 Highly Unlikely (HU)
 Not Applicable (N/A)
 Unable to Assess (U/A)
17
The ratings are supposed to be included in each of the TEs, based from GEF guidance. Shortcomings relate to
achievement of objectives in terms of relevance, effectiveness or efficiency, as well as the quality of M&E systems.
This same scale should be used to rate shortcomings in the terminal evaluation report.
18