United Nations Environment Programme
Terminal Evaluation of the UNEP Project
Disaster Reduction through Awareness, Preparedness and
Prevention Mechanisms in Coastal Settlements in Asia:
Demonstration in Tourism Destinations
CP/4020-06-05 = CPL 5068 3977 2643
Lorne Kriwoken
Evaluation Office
August 2009
Disclaimer
The views expressed in this report are those of the evaluator alone and do not
necessarily reflect the views or policies of UNEP or of any individual or
organisation consulted.
Page ii of 58
Table of Contents
Acronyms and Abbreviations .......................................................................................... iv
Executive Summary .......................................................................................................... 5
1. Project Background and Overview .............................................................................. 6
1.1 Project Rationale ..................................................................................................... 6
1.2 Executing Arrangements ......................................................................................... 7
1.3 Legislative Mandate.................................................................................................. 8
1.4 Project Activities and Budget................................................................................... 8
2. Objective, Scope and Methods of Evaluation ............................................................ 9
2.1. Objective and Scope of the Evaluation..................................................................... 9
2.2 Methodology ............................................................................................................ 9
2.3 Key Evaluation Principles..................................................................................... 11
3.1 Attainment of Objectives and Planned Results .................................................... 12
3.1.2 Relevance ........................................................................................................ 14
3.1.3 Efficiency ....................................................................................................... 15
3.2 Assessment of Sustainability of Project Outcomes ............................................... 16
3.2.1 Institutional Frameworks and Governance ..................................................... 16
3.2.2 Financial Resources ........................................................................................ 17
3.2.3 Socio-political Factors .................................................................................... 17
3.2.4 Environmental Sustainability .......................................................................... 18
3.3 Achievement of Outputs and Activities................................................................... 18
3.3.1 India................................................................................................................. 19
3.3.2 ANSAO, Thailand ........................................................................................... 21
3.3.3 PMP, Thailand................................................................................................. 23
3.4 Catalytic Role ........................................................................................................ 24
3.5 Assessment of Monitoring and Evaluation Systems ................................................ 26
3.6 Preparation and Readiness ...................................................................................... 27
3.7 Country Ownership................................................................................................. 27
3.8 Stakeholder Participation and Public Awareness..................................................... 28
3.9 Financial Planning ................................................................................................. 29
3.10 Implementation Approach ................................................................................... 30
4.
Conclusions and Ratings of Project Implementation Success ................................. 33
5. Lessons Learned ....................................................................................................... 36
Annex 1 : Terms of Reference ....................................................................................... 38
Page iii of 58
Acronyms and Abbreviations
ANSAO-PP
Ao Nang Sub-District Organization Administration, Phi Phi
Island Authority of Krabi, Thailand
APELL
Awareness and Preparedness for Emergencies at the Local
Level
CG
Coordinating Groups
DDPM
Department of Disaster Prevention and Mitigation, Thailand
DTIE
UNEP Division of Technology, Industry and Economics
EOU
UNEP Evaluation and Oversight Unit
IH&RA
International Hotel and Restaurant Association
NSCI
National Safety Council of India
PM
Project Manager
PMP
Patong Municipality of Phuket, Thailand
SRSA
Swedish Rescue Services Agency
TOR
Terms of Reference
UNEP
United Nations Environment Programme
Page iv of 58
Executive Summary
The project was entitled “Disaster Reduction through Awareness, Preparedness and
Prevention Mechanisms in Coastal Settlements in Asia – Demonstration in Tourism
Destinations.” It aimed to increase the disaster management capacity of disaster and
tourism stakeholders in three tsunami-affected tourism destinations of Kanniyakumari
(India), Patong and Ao Nang in Phi Phi Island (Thailand)
United Nations Environment Programme Division of Technology, Industry and Economics
(UNEP DTIE) was the Project Coordinator and worked with the Swedish Rescue Services
Agency (SRSA), the International Hotels & Restaurants Association (IH&RA), the
National Safety Council of India (NSCI), the Thailand Department of Disaster Prevention
and Mitigation (DDPM), Patong Municipality of Phuket (PMP) and the Ao Nang SubDistrict Organization Administration, Phi Phi Island Authority of Krabi (ANSAO).
The Awareness and Preparedness for Emergencies at the Local Level (APELL) process
was adopted as a framework for the project. The ten step APELL process supported
integrated emergency response for local communities, governments and emergency
agencies. It promoted risk awareness, risk reduction and mitigation, and the development
of preparedness between industry, local authorities and the local population.
Disaster management operational capacity of local project partners and beneficiaries was
improved through training on disaster prevention and preparedness, risk assessment,
emergency planning and emergency drills. Local awareness on disaster preparedness
planning and response was promulgated with the establishment of local Coordinating
Groups (CGs), APELL training and workshops on capacity building. The dissemination of
material included brochures, signs, CDs and toolkits. Guidelines for the development and
replication of a crisis communication centre were completed. Integrated disaster
management plans were developed and vulnerability mapping and shelter assessment
undertaken.
In February 2009 a terminal evaluation sought to determine the extent that the project’s
aims had been attained and to assess any positive and negative aspects with the project.
The methodology involved field visits and interviews at the NSCI Headquarters in Mumbai
(India) the DDPM Headquarters in Bangkok and the Patong Municipality of Phuket
(Thailand). Telephone conversations and e-mail correspondence were conducted with
UNEP representatives in Paris (France) and Nairobi (Kenya). A desk review of relevant
documents was undertaken.
Overall the project was successful with the attainment of project objectives and results.
There was evidence that the project increased disaster management operational capacity of
local partners and beneficiaries and increased local awareness of disaster preparedness,
planning and response. There was an associated increase in the relevance of environmental
management in disaster reduction and enhanced planning and preparedness for emergency
services. Greater cooperation was supported between the tourism sector and local and
national authorities.
Page 5 of 58
Recommendations
Significant time is required in the initial preparation of the project. This includes putting in
place appropriate administrative frameworks, reporting mechanisms and ensuring that each
partner has a common understanding of the project’s overall aims and objectives. UNEP
should factor in additional time at the commencement of project activities as part of the
overall time for project implementation.
The preparation and implementation of an M&E plan should be formally supported at the
outset of the project design. This would allow systematic development of baselines for
existing conditions and verifiable indicators of success. Such a plan would also support
specific procedures for data collection and importantly, monitoring of project outcomes
against project objectives.
In order that the project outcomes are sustainable beyond the short to medium terms there
needs to be increased engagement with relevant local government authorities responsible
for disaster planning and mitigation, the private sector including hotels and tourism
operators and the local community. UNEP DTIE now has proven expertise in the delivery
of disaster response and mitigation projects. This expertise should be further developed
with other capacity building and implementation activities undertaken by UNEP in the
region.
1. Project Background and Overview
1.1 Project Rationale
On 26 December 2004 an undersea Sumatra-Andaman earthquake with a magnitude of 9.1
on the Richter scale triggered the Indian Ocean tsunami. This tsunami devastated coastal
communities bordering the Indian Ocean in India, Sri Lanka, Thailand and Indonesia.
Significant humanitarian, economic and environment impact was felt throughout the
region. UNEP immediately began working with governments in the region and established
a Task Force to respond to requests from affected countries.
The goal of this project was to minimise casualties, property and environmental damage
from natural and human-made disasters of three tsunami-hit tourism destinations. This was
done by improving the ability of local authorities and the private sector to manage natural
and human-made disasters and by educating residents and tourists on how to properly react
to disasters. Two of the three tourism destinations were located in Thailand (Patong,
Phuket and Phi Phi Island, Krabi) and one was located in the Tamil Nadu State, India
(Kanniyakumari).
The project was planned and implemented using UNEP’s Awareness and Preparedness for
Emergencies at the Local Level (APELL) process at the tourism destinations. The APELL
process has been used extensively for other activities or sectors including ports and the
mining industry. APELL is a process designed to create public awareness of hazards and to
ensure that communities and emergency services are adequately trained, coordinated and
prepared to cope with disaster. Project activities included assessment of local hazards,
APELL demonstrations, and risk communication. Target groups in India and Thailand
Page 6 of 58
were: local authorities including emergency services in the tourism sector and local
residents and tourists.
The project aimed specifically at:
a) improving disaster preparedness capabilities of the local authorities, local tourism
industry and residents of disaster-prone coastal tourism destinations for effective
emergency response through the adaptation of the existing APELL process in
tourism destinations, taking into account the challenges (e.g. cultural, linguistic,
logistic) inherent to tourism destinations;
b) raising awareness of local authorities, local residents and tourists, and of the local
tourism industry about the importance of natural and human-made disaster
prevention and preparedness (through awareness raising events and the production
and distribution of brochures) and about the importance of environmental
management and conservation, and its relationship with disaster reduction (through
specific training sessions to local residents);
c) improving local environmental management and coastal planning capabilities of
local authorities through the identification of local hazards, assessment of local
vulnerabilities, suggestion of preventive measures, and production of vulnerability
maps with high risk zones that will allow proper risk communication, and land use
planning that will in turn promote disaster reduction and environmental
conservation;
d) improving risk communication between the local authorities, the private sector
(tourism and other private business) and the exposed community (tourists and local
population) on local disaster issues to allow better and effective response in crisis
situations leading to reduction of a loss of life and of environmental damage; and
e) increasing the trust of tourists in the tourism destinations by disseminating the
project results and by promoting the application of proposed project approach to
other disaster-exposed tourism destinations in Asia.
1.2 Executing Arrangements
The UNEP Division of Technology, Industry and Economics (UNEP-DTIE) coordinated
the overall project implementation and was responsible in reporting and liaised with the
European Commission, supervised project partners, and worked closely with the National
Safety Council of India (NSCI), Swedish Rescue Services Agency (SRSA), International
Hotel and Restaurant Association (IH&RA), Patong Municipality of Phuket (PMP) and the
Phi Phi Island Authority of Krabi (ANSAO) to ensure that the activities were consistent in
the three tourism destinations. This Division also provided technical expertise on the
APELL process, supported and ran joint training workshops with SRSA. Further, it
assisted with selection of government authorities that participated in project events and
signed agreements with project partners.
SRSA provided the expertise in local capacity building for the APELL process and on the
theme areas that supported disaster prevention and preparedness. This included risk
assessment, vulnerability mapping, emergency planning, emergency drills, evacuation and
Page 7 of 58
shelter. SRSA supported the project implementation in Thailand through the PMP and
ANSAO local project managers.
NSCI received technical support from UNEP-DTIE and SRSA and guaranteed local
project implementation in Kanniyakumari. NSCI has the Tamil Nadu sub-centre (i.e. the
National APELL sub-centre) in Tuticorin, about 100 km from Kanniyakumari. This subcentre has experts in the local knowledge (language, culture and legislative framework)
that are needed to support the Kanniyakumari project. The Project Manager (PM) based in
Tuticorin was supported by a project facilitator locally identified by NSCI.
The IH&RA provided technical expertise to the project and took the lead in preparing
material, conducted training workshops for the tourism sector and played a key role in the
replication of the approach to other tourism destinations. It also guaranteed that expertise
from the sector and other regions would be transferred to the affected region and ensured
that the tourism sector views were incorporated in the development and delivery of all
materials developed in the project.
The Patong Municipality of Phuket and the Phi Phi Island Authority of Krabi were partners
responsible for local project management logistics and the transfer of knowledge to their
local partners in the project. At the initial phase of the project municipalities identified
local project managers that would undertake implementation at both locations. UNEPDTIE worked through the Department of Disaster Prevention and Mitigation (DDPM)
which was not officially a project partner but was co-opted and latter became an active
player. All communications in Thailand were channelled through DDPM.
1.3 Legislative Mandate
The project had the following legislative mandates:
a) GC-23/7 - Strengthening environmental emergency response and developing
disaster prevention, preparedness, mitigation and early-warning systems in the
aftermath of the Indian Ocean tsunami disaster;
b) GC-22/8 - Further improvement of environmental emergency prevention,
preparedness, assessment, response and mitigation; and
c) GC-22/7 - Engaging Business and Industry.
1.4 Project Activities and Budget
The project duration was planned for 24 months starting 1 October 2006 and ending 30
December 2008. The project activities were organised into five phases:
a) Phase A: Assessment Phase and Setting-up the Project Management Structure;
b) Phase B: Training and Demonstration Phase;
c) Phase C: Development of Awareness Raising and Training Material;
d) Phase D: Capacity Building Phase; and
Page 8 of 58
e) Phase E: Public Information and Wider Dissemination Phase.
The project was funded by the European Commission at a budget of 812,603 USD or
637,081.21 EUR.
2.
Objective, Scope and Methods of Evaluation
2.1. Objective and Scope of the Evaluation
The objective of this terminal evaluation was to establish the project impact, and review
and evaluate the extent to which implementation of planned activities and outputs were
accomplished. The evaluation will determine the extent to which the project has been
successful in fulfilling its objectives and obtaining the expected results and whether it has
been cost effective in producing its results.
The evaluation will cover all key activities undertaken within the framework of the project
as described in the project document. Planned outputs of the project will be compared with
actual outputs and an assessment will be undertaken on the actual results to determine the
impact of the project. The evaluation will answer the following key question:

How and to what extent has the project improved the abilities of local authorities
and the private sector in Thailand and India to: (a) manage natural and humanmade disasters; (b) and educate residents and tourists on how to properly react to
these disasters?
2.2 Methodology
This terminal evaluation was conducted as an in-depth evaluation using a participatory
approach. UNEP Evaluation and Oversight Unit (EOU), UNEP DTIE Project Manager,
key representatives of the executing agencies and other relevant staff were consulted
during the evaluation. The consultant liaised with UNEP EOU and the UNEP DTIE Project
Manager with regards to logistic and methodological issues. The draft report was
circulated to UNEP DTIE Project Manager, key representatives of the executing agencies
and the UNEP EOU.
The findings of the evaluation were based on the following:
a)
Desk top review of project documents, output, progress reports, monthly financial
reports, terminal reports, minutes of meetings and relevant correspondence.
b)
Review of specific products including publications, management and action
plans, databases and web-site updates.
c)
Telephone interviews and email correspondence with the UNEP DTIE Project
Manager and relevant staff in UNEP EOU dealing with sustainable tourism and
disaster management.
d)
Telephone interviews with relevant project partners in NSCI, IH&RA, Patong
Municipality of Phuket and Phi Phi island Authority of Krabi.
Page 9 of 58
e)
Telephone and personal interviews with relevant stakeholders involved including
the Project Coordination Unit and the Project’s Advisory Committee.
f)
Field visits and face-to-face interviews with the main project beneficiaries in
India (Table 1) and Thailand (Table 2) during the week starting 23 February
2009. Field visits were undertaken at: NSCI Headquarters in Mumbai, India;
DDPM Headquarters in Bangkok, Thailand; and Patong Municipality of Phuket,
Thailand.
Table 1: Interviews at NSCI Headquarters, Mumbai, India
Shri KC Gupta, Director General
Shri PG Dhar Chakrabarti, Executive Director, National Institute of Disaster Management,
Ministry of Home Affairs, New Delhi
Shri RP Bhanushali, Technical Advisor
Smt S Bhattacharya, Director
Shri MM Kulkarni, Director
Shri VB Patil, Director
Shri R Srivastava, Director
Shri PM Rao, Advisor
Shri MY Bhalekar, Technical Advisor
Shri TK Unnikrishnan, Assistant Director, Publications
Page 10 of 58
Table 2: Interviews Conducted in Thailand
Adthaporn Singhawichai, Director of Research and International Cooperation Bureau,
DDPM, Ministry of Interior
Luckana Manimmanakorn, Chief of International Sub-Bureau, Research and International
Cooperation Bureau, DDPM
Chatchadaporn Boonyavaha, Project Coordinating Team, DDPM
Kornisnan Wilawan, DDPM, Ministry of Interior
Srisuda Rakphao, Project Manager, Phuket
Wanchai Chaowapanja, Director
Deputy Mayor Chairat Sukban, Patong Municipality of Phuket
Tia Wongsuwan, Chief of Food Carrier Group
Jongrak Athimuttisun, Patong Community Group
Wassanaporn Palimeechai, Patong Traditional Massage Group
Nuntiya Puttarak, Patong Municipality, Phuket
Stefanos I. Fotiou, Program Officer, Tourism and Environment, UNEP DTIE
2.3 Key Evaluation Principles
In order to evaluate any outcomes and impacts that the project may have achieved this
terminal evaluation considered the difference between the answers to two simple questions
“what happened?” and “what would have happened anyway?” These questions imply that
there should be consideration of the baseline conditions and trends in relation to the
intended project outcomes and impacts. There should also be plausible evidence to
attribute such outcomes and impacts to the actions of the project.
In some cases adequate information on baseline conditions and trends was lacking. These
cases have been clearly highlighted in the terminal evaluation along with any assumptions
that were taken to enable the evaluator to make informed judgements about the
performance of the project.
2.4 Limitations and Issues Not Addressed
Field visits and face-to-face interviews were not conducted in Phi Phi Island, Krabi,
Thailand in Kanniyakumari, Tamil Nadu State, India.
3. Project Evaluation Parameters
Page 11 of 58
The success of project implementation was rated on a scale from ‘highly unsatisfactory’ to
‘highly satisfactory’ (Table 4). In particular the evaluation assessed and rated the project
with respect to the categories defined below in section 3.1.
3.1 Attainment of Objectives and Planned Results
3.1.1 Effectiveness
The effectiveness of the project will be assessed according to the five overall aims
delineated in the project brief. The specific achievement of outputs and activities is
presented in section 3.3.
The first objective of the project was to improve disaster preparedness capabilities of the
local authorities, tourism industry and residents of the disaster-prone coastal tourism
destinations for effective emergency response. This objective was achieved through
different activities. In both India and Thailand local CGs with wide community
representation were established. A range of key stakeholders involved in emergency
response were represented in the CGs. Training of 180 locals on the APELL process was
undertaken in both countries. There was also training of 150 locals on disaster prevention
and preparedness, risk assessment, emergency planning and emergency drills. This
training, mainly in emergency response involved key stakeholders and included police, fire
response, health services and civil defense. Disaster management plans were either updated
or created for each of the three tourism destinations. Drills were undertaken at the three
sites to test the disaster management plan and, involved key stakeholders with
responsibilities in disaster mitigation.
The second project objective was to raise awareness of local authorities, local residents,
tourists, and the local tourism industry about the importance of natural and human-made
disaster prevention and preparedness. This objective was achieved through the use of a
wide range of media and activities. Different brochures and handbooks outlining
emergency response were disseminated in a number of languages in the three tourism
destinations. They included a project summary brochure (200 copies in three tourist
locations); seven APELL brochures (1,400 in each location); and an APELL Handbook
(200 copies in each location). Awareness raising of locals and tourists was completed by
undertaking drills involving 800 participants in Kanniyakumari and 150 in Phuket. Three
emergencies drills (on fire, a collapsed/ building, boating accident) were carried out on Phi
Phi Island. There was large production and dissemination of written material in English,
local languages and of languages used by international tourists. Details of the material
produced are found in section 3.3.
Objective three was about improvement of local environmental management and coastal
planning capabilities of local authorities through the identification of local hazards and an
assessment of local vulnerabilities. This was achieved by conducting a workshop and
identifying disaster mitigation actions. An environmental management workshop was
attended by 70 participants in Thailand and 47 in India. The workshop identified ways in
which disaster impact could be mitigated in solid waste management, sewage treatment
and energy supply. Shelter assessment and vulnerability mapping were undertaken. This
included an assessment of potential buildings for shelters and the preparation of emergency
plans.
Page 12 of 58
Objective four ‘improvement of risk communication between the local authorities, the
private sector (tourism and other private business) and the exposed community (tourists
and local population) on local disaster issues’ was achieved through the implementation of
a range of risk communication strategies. The local CGs played an important role in raising
awareness of disaster mitigation in the local community. Training on disaster management
planning, vulnerability mapping and shelter assessments was conducted. Warning signs
were prepared and located in appropriate locations to highlight the potential of disaster and
how to respond to specific disasters. Brochures and handbooks were extensively
distributed and they communicated local disaster issues to target stakeholders.
Objective five ‘increasing the trust of tourists in the tourist destinations by disseminating
project results’ was achieved through dissemination of material on disaster awareness,
preparedness and response. Brochures, booklets, signs and general community awareness
increased the overall knowledge of tourists of the types of hazards and associated risks.
The adoption of the APELL process was an effective strategy and played a key role in
achieving the project outcomes. APELL also played a fundamental and critical role in the
overall project implementation in both countries. APELL was first developed by UNEP in
conjunction with industry, communities and governments after major industrial accidents
impacted health and environment.1 The overall purpose of APELL was to minimise the
occurrence and harmful effects of technological accidents and environmental emergencies.
This was promoted by risk awareness, risk reduction and mitigation, development of
preparedness between industry, local authorities and the local population.2 An APELL
process was developed for industry (port areas and mining), transport (dangerous goods)
and natural disasters. In 1998 an APELL Handbook was released outlining a ten step
process for integrated emergency response for local communities, governments and
emergency agencies (Table 3). After natural disasters such as the Indian Ocean tsunami in
2004, the APELL process was extended to address multi-hazard issues and was applied to
the current project.
Table 3: The APELL Ten Step Process
1. Identify Participants and Identify Their Roles
2. Evaluate and Reduce Risks
3. Review Existing Plans and Identify Weaknesses
4. Task Identification
5. Match Tasks and Resources
6. Integrate Individual Plans into Overall Plan and Reach Agreement
7. Draft Plan and Obtain Endorsement
8. Communication and Training
9. Testing, Reviewing and Updating
10. Community Education
In summary, this project was instrumental in capacity building for disaster planning and
response in both India and Thailand for both national and local governments and
1
UNEP 1988, APELL: A Process for Responding to Technological Accidents, UNEP, Paris, France.
UNEP DTIE , n. d., Explaining APELL: Background to Awareness and Preparedness for
Emergencies at the Local Level, UNEP DTIE, Paris, France.
2
Page 13 of 58
associated agencies. Disaster awareness was raised across relevant sectors and
stakeholders. Institutional capacity on disaster mitigation was increased. Local
communities (i.e. local governments, residents and local tourism industry) have improved
disaster preparedness capabilities for effective emergency response. The project has also
improved local environmental management and coastal planning capabilities of local
authorities and improved risk communication between local authorities and the private
sector. Details of specific project activities are listed in section 3.3.
3.1.2 Relevance
UNEP DTIE has a range of objectives that include supporting, inter alia, governments,
local authorities and industry to develop and implement policies, strategies and practices
that support the efficient use of natural resources, reduce pollution and risk for humans and
the environment and incorporate environmental costs. DTIE supports partnerships with
international organisations, government authorities, business and industry, nongovernment
organisations and locals.3
The 21st Session of the Governing Council Global Ministerial Environment Forum
(Decision 21/17) called for “further improvement of environmental emergency prevention,
preparedness, assessment, response and mitigation.” The UNEP DTIE APELL Program,
therefore, plays a critical role in assisting in disaster prevention and response planning in
vulnerable areas and is widely acknowledged as an appropriate method for natural
disasters. A range of organisations have contributed to the APELL process. The
International Council of Chemical Associations (ICCA) and the United States
Environmental Protection Agency (US EPA) have promoted the process. The Swedish
Rescue Services Agency has also been instrumental in producing key APELL guideline
documents.
In India the NSCI has the following mission statement: “to build a national movement on
safety, health and environment to prevent and mitigate the loss of life, human suffering and
economic losses and providing support services.”4 NSCI was set up in March 1966 as an
independent, non profit, national society and provides a range of services including:
conferences, training, national campaigns, promotional material, consultancy, publications,
national standards and safety awards.5 It has also worked extensively with a range of
international and regional authorities including: UNEP, International Labour Organisation,
World Bank, US EPA and the Thai Asian Disaster Preparedness Centre.
NSCI supported the APELL Program on Chemical Emergency Management (CEM) with
UNEP for six risk industrial areas from 1992 to 1997, contributed to the framing of the
Chemical Accidents (Emergency Planning, Preparedness and Response) Rules in 1996,
and set up the world’s first National APELL Centre in 2002. This Centre acts as a CEM
resource centre and shares experience with China, Jordan and South Africa.6 NSCI worked
3
UNEP DTIE, n.d., Sustainable Consumption and Production Branch, <http:www.unep.fr/scp/sp/>
(22 June 2009).
4
NSCI, 2009. NSCI Overview Presentation, 24 February 2009, NSCI Headquarters, Mumbai, India.
5
NSCI, n. d., Dedicated to Safety, Health and Environment, NSCI, Mumbai, India.
6
National APELL Centre, n.d., Indian Experience – Project Implementation (Phase I), UNEP DTIE’s
APELL/TransAPELL Programme Worldwide, UNEP, DTIE, NSCI.
Page 14 of 58
with UNEP DTIE throughout this project. NSCI has also been involved in disaster
mitigation for coastal industrial installations7 and hazard materials response.8
In Thailand the Department of Disaster Prevention and Mitigation was set up in 2002
within the Department of Interior.9 The Disaster Prevention and Mitigation Act 2007
provides the legislative basis for the Disaster Prevention and Mitigation Committee.10 The
objective of DDPM is to, inter alia, formulate a National Disaster Prevention and
Mitigation Plan (NDPMP); assist government agencies, local authorities and business in
preventing and mitigating disaster; and provide disaster prevention and mitigation training.
Thailand does not have the same extensive historical association with the UNEP DTIE
APELL process as India. However, DDPM enthusiastically embraced the APELL process
and worked cooperatively with UNEP DTIE throughout the project. The success of the
project has reinforced the overall relevance of APELL to DDPM and the Thai
Government.
In summary, the nature and significance of the project directly contributed to the wider
portfolio of UNEP and directly to DTIE by supporting sustainable consumption and
production and disaster management. The outcomes of the project were very consistent and
closely aligned with the operational program strategies and country priorities of India and
Thailand. The nature and significance of the contribution of the project directly contributed
to sustainable consumption and production and disaster management in both countries.
3.1.3 Efficiency
From all accounts the project was a cost effective process. The budget allocation from the
European Commission was very rigid and did not allow flexibility in improving the
financial management of the project. The budget was fixed when the project proposal was
initially submitted. There were, however, some financial challenges identified in the
documentation. The project did not build on a pre-existing tourism project on disaster
mitigation in India or Thailand. In India for instance, disaster mitigation has focused on
chemical accidents associated with industrial activities and hazard material response. In
Thailand disaster mitigation has focused on the role of local authorities and businesses in
preventing and mitigating disaster and providing disaster prevention and mitigation
training. Support for disaster mitigation and response in the tourism sector did not largely
exist. However, the project adopted the APELL process which was an identifiable earlier
initiative and was clearly appropriate to use.
UNEP DTIE role in providing financial back-stopping ensured that the partners did not
overspend their allocated budget. Throughout the two year timeframe DTIE reminded
Indian and Thai partners to stay within budget. The partners were also reminded that if
funds were to be saved, it would have to be returned to UNEP and then the EC. The project
was delayed by three months in India and unspent money was returned to UNEP DTIE.
7
NSCI, 2003. Guidelines for the Preparation of Cyclone Emergency Management Manual for Coastbased Industrial Installations, NSCI, Mumbai, India.
8
NSCI, n. d., UNEP DTIE’s APELL/Trans-APELL Programme Worldwide: An Effective Approach
for Responding to Road Transport Hazmat Emergencies – Experience of Patalganga-Rasayani
Industrial Pocket, Maharashtra, India, NSCI, Mumbai, India.
9
DDPM, 2009. Terminal Evaluation Presentation, 26 February 2009, DDPM Headquarters, Bangkok,
Thailand.
10
DDPM, n.d., Disaster Management System in Thailand, DDPM, Bangkok, Thailand.
Page 15 of 58
However, there was no evidence to suggest that this delay substantially affected the cost
effectiveness or the efficiency of the project.
Some inefficiency was noted, resulting from the challenges of working across such a wide
geographical scope. The three tourism destinations were far removed and this presented a
challenge in undertaking meetings and maintaining communication throughout the project.
Some communication issues were also apparent with the Thai partners regarding their
command of the English language.
3.2
Assessment of Sustainability of Project Outcomes
This section addresses the sustainability of project outcomes with respect to the following
conditions: institutional frameworks and governance; financial resources; socio-political;
and environmental sustainability. Obviously these conditions are not mutually exclusive
and must be treated in a holist and integrated manner. There is, however, one overarching
question: what is the probability of project outcomes and impact continuing after the
UNEP funded project ends?
3.2.1 Institutional Frameworks and Governance
With respect to institutional frameworks and governance, the project has been very
successful in gaining broad institutional support in both countries at a range of levels.
Given the high international profile of the tsunami disaster it is likely that the outcomes
and impacts of the project will be sustained at least in the short to medium term in both
countries.
This broad institutional framework was supported in India through the NSCI which has a
wide membership comprising of corporate (industries, employers organisations,
professional bodies), trade unions and individual members. NSCI has worked effectively at
international, national, regional and local levels in safety, health and environmental issues.
The Institute also has experience with the APELL process (see section 3.7) and has
developed a number of experts to implement and train in the process. NSCI has established
itself as one of the key Indian organisations in disaster mitigation and works closely with
the National Institute of Disaster Management, Ministry of Home Affairs. There was
ample evidence to suggest that accountability and transparency and technical expertise
would remain in place after the project. Stronger institutional capacity and better informed
decision making has resulted.
Institutional support for the project in Thailand was found in the DDPM within the
Ministry of Interior which has carriage of the federal Disaster Prevention and Mitigation
Act. The legislation gives DDPM the mandate to act as the principle government agency
for coordination and integration of a national disaster management system. DDPM works
closely with the Local Administrative Organizations and facilitates coordination between
the parties concerned. DDPM has established itself as the key government agency with
responsibility for disaster mitigation in Thailand. There was evidence to suggest that
accountability and transparency and technical expertise would remain in place after the
project. Strong institutional capacity and better informed decision making has resulted.
Page 16 of 58
3.2.2 Financial Resources
With the project now complete there is no doubt that the level of financial resources seen
over the two years cannot be sustained by either country. However, the project has created
momentum in disaster preparedness and mitigation. There now needs to be much greater
sustained cooperation between local government and the private sector which includes
hotels and tourism operators. Greater understanding and cooperation has been supported
between the local and national authorities. These linkages must now be more widely
explored and financial support gained for further targeted projects. A strategic assessment
should be undertaken in both countries on how to best sustain the project outcomes and
benefits beyond the medium term.
3.2.3 Socio-political Factors
The future success of the project is closely tied to the unique socio-political factors evident
in both India and Thailand. In both countries the institutional involvement at the highest
levels is now assured. NSCI and DDPM have proven to be critical in disaster response and
mitigation. There is, however, the possibility of further enhancing the working relationship
between NSCI and DDPM at an international level. Perhaps further projects on the Bay of
Bengal would provide a focus for both countries in disaster mitigation. One of the crucial
aspects of sustainability is how local governments, communities and the private sector
embrace the project’s outcomes. Increased pressure from development and expansion of
the revenue base from taxes will continue to place limitations on coastal areas identified as
having potential disaster.
Project outcomes can be supported by the encouragement of a wide range of stakeholders
to be involved in supportive programs. Links with other programs have been established?
and are relevant in supporting sustainability and socio-political factors. For instance,
Indian and UNDP collaboration on disaster risk management was undertaken in
Kanniyakumari District. UNDP project officials attended workshops run by NSCI and
UNEP for the project. Officials from this project were also invited to attend awareness
raising events for their program. This type of cross fertilization of relevant projects should
be further explored and encouraged.
In Thailand links with other projects have supported the sustainability of the project. On
Phi Phi Island, the Department of Civil Works and Town and Country Planning, Ministry
of Interior, divided the island into ten zones with associated evacuation routes. DDPM and
the Asian Disaster Preparedness Centre (ADPC) trained locals in Community-based
Disaster Risk Management (CBDRM). At PMP the Japan International Cooperation
Agency (JICA) supported an evacuation drill and the development of a community disaster
management plan.
What is less clear is how the project will continue to be supported at the grass roots or
community level. Key protagonists are crucial to ensure that this gap is closed. In
Thailand, a number of key informants were concerned that continuing work of the project
might be a domain of a few community members.
Page 17 of 58
3.2.4 Environmental Sustainability
The project’s contribution to environmental management is clear through increased data
collection and knowledge of natural processes such as meteorology, geology and earth
sciences, planning, engineering, environmental and architectural. Data such as these assists
in identifying local hazards, exposures and vulnerability and can be used effectively by
trained experts.
The success of the project will be sustained through local awareness of environmental
management as a key component on disaster reduction efforts. Workshops identified areas
such as solid waste, sewage and energy for improvement to reduce potential disaster
impact. The workshops also generated greater awareness of the dynamic nature of the
coastal zone and were useful improving tourism sector’s choice of the location of of hotels.
There was an increased awareness of the coastal ecosystem and associated defence
capabilities and mechanisms to mitigate tsunamis. This has reinforced the fact that
undisturbed natural ecosystems are the most robust with respect to tsunamis. This call for
support of the long term management of fringing mangroves, coral reefs and sea grass beds
adjacent to the coastal zone. There was evidence that some tour operators are now adopting
codes of conduct and operational protocols for tourists to protect these fragile coastal
ecosystems. This recognition has implications for wide support of targeted conservation
initiatives.
There are, however, specific activities that can undermine the environmental benefits of the
project. The reduction and removal of mangroves and the destruction of fringing coral reef
could have serious consequences. Coral reef dynamiting used by fishers could destroy
large tracts of protective coral. Removal of mangroves to increase coastal access for jetties
and marinas would be detrimental. The removal of mangroves to improve beach access for
tourists in coastal hotels could have disastrous consequences. Damaging sea grass beds
with indiscriminate boat access should be discouraged.
In summary, there are strong institutional frameworks and governance structures in place
in both countries that can sustain the project in the short to medium term. There is a high
likelihood that project outcomes will lead to sustained ecological benefits in coastal
environments of India and Thailand. The integration of disaster management into local
integrated coastal zone management would foster ecological sustainability. What is less
clear is how the disaster mitigation will be supported in the long term given the pressures
of coastal development, requirements for increased beach access and removal of
mangroves.
3.3 Achievement of Outputs and Activities
The following assessment of the project outputs and activities are presented for each of the
three tourism destinations. Within each destination an assessment is made of: local
Coordinating Groups and Project Managers; meetings, workshops and training; and
publications and infrastructure.
Page 18 of 58
3.3.1 India
Local Coordinating Groups and Project Manager
The project was implemented and supported at three different levels. At Kanniyakumari a
33 member CG11 was established in March 2007. The CG included wide of representation
from national and local governments, the private sector and nongovernment organizations.
NSCI Director General Shri KC Gupta was appointed as Chair. Administrative support
was provided by NSCI and technical support by NSCI through (Mr RP Bhanushali, Mr AY
Sundkar, Mr SS Babu and other experts as required).12 The PM at Tuticorin was Mr P
Gunasekaran with support by the General Manager of the Heavy Water Plant Tuticorin
(HWPT). The Local Project Facilitator at Kanniyakumari was Mr P Thomas with support
by District Controller/District Authority (DC/DA). The CG first met on 5 May 2007 to
discuss the requirements for a Kanniyakumari disaster management plan. A second
meeting was held 7 September 2007 to advance data gathering. As a result a disaster
management plan was developed with extensive input from the CG. The PM assisted in the
distribution of a questionnaire prepared by UNEP and SRSA to ascertain the status of
disaster management.
Meetings, Workshops and Training
A two member team from India (NSCI officer and Project Manager) received training in
Paris and Sweden in December 2006. The training focused on how to maximize
participation of local project partners. The project was launched and an APELL training
workshop was conducted in Kanniyakumari 21-23 March 2007. The workshop was
attended by 42 participants representing the CG stakeholders. International representatives
from UNEP, EC, SRSA, and IH&RA and National and local representatives attended.
Importantly, the workshop was translated into both English and Tamil. Mr SM Perumal,
Project Officer from the District Rural Development Agency, Kanniyakumari District
represented the District Controller and chaired the workshop. The workshop observed that:
the media had exaggerated the situation and as a result tourism numbers had significantly
dropped; vehicular traffic and parking needed to be regulated in the peak season; a proper
early warning system was needed; a disaster management plan should specifically consider
tourists; there should be a study of past accidents and safety of tourists at sea; and
infrastructure development should be encouraged.13 – this last part need improvement –
clearly stated observation followed with recommendations
11
District Public Relations Officer, Kanniyakumari Town Panchayat, Tashildar, Agasthiswaram, DSP,
DFO, District Health Services, Regional Transport Officer, Tamilnadu Electricity Board, PWD,
Fisheries Department, Tourism Officer, Tamilnadu, Water Administration and Drainage, Divisional
Engineer (Highways), Tamil Nadu State Transport Corporation, Poompuhar Shipping Corporation,
State Minor Port Authority, Meteorological Observatory, All India Radio, BSNL, LIC of India, Hotel
Association, Mechanised Boat Owners Association, Merchants Association, Representative of Tour
Operators, Auto Rikshaw Assocation, Tourist Guides Association, Representatives of Restaurants,
Representative of St Anthony School, Nongovernment Organisation Women Forum of Self Help
Group, National Service Scheme. NSCI, 2009.
12
NSCI, 2009. Project Implementation Presentation, 24 February 2009, NSCI Headquarters, Mumbai,
India.
13
NSCI, 2009. Project Implementation Presentation, 24 February 2009, NSCI Headquarters, Mumbai,
India.
Page 19 of 58
In April 2007 a four member team carried out a survey to assess the status of disaster
management. The team visited a number of response agencies including: fire, police,
hospital, Kanniyakumari Town Panchayat Office and the Emergency Operation Centre at
Nagercoil. The findings of the survey were presented at a workshop on risk assessment,
emergency planning and drills. This was held 2-5 May 2007 in Kanniyakumari and was
attended by 34 participants.
A shelter assessment was carried out on 12–14 September 2007 and 18 sites identified.
SRSA and IH&RA played important roles in the development of the assessment. A
vulnerability mapping exercise was undertaken 17–20 December 2007 in which SRSA
experts were involved. An integrated Emergency Response Plan was then developed with
the assistance of NSCI.
An environmental management and disaster reduction workshop took place 9–10 June
2008 with 47 participants. The workshop examined the link between environment and
disaster prevention. It also allowed hotels to gain an appreciation of environmental
management and how it could influence their sector. From 11–12 June 2008 the CG
undertook planning activities for an evacuation drill from Vivekananda Rock Memorial
due to a tsunami alert. The drill was reviewed on 11 September 2008 and implemented on
12 September 2008. Approximately 800 people were evacuated and this was successfully
achieved with the coordination of a wide range of international, national and local
stakeholders. In addition to the drill, training on the APELL process was undertaken 8–9
September 2008. Presentations on applying APELL to disaster and risk reduction were
given by representatives of UNEP, SRSA, NSCI and other key experts. The workshop was
attended by 60 participants and included a wide range of stakeholders.
Publications and Infrastructure
At Kanniyakumari two brochures were produced and translated into English, Tamil and
Hindi. The first was entitled “Disaster Preparedness and Response Guide for
Kanniyakumari Town (Tsunami, Cyclone, Fire and Stampede).”14 This brochure (20,000
copies) provided specific advice on how to prepare for these disasters and what to do
during and after the disaster. Important emergency contact numbers were provided. The
second brochure was entitled: “Tourist Emergency Resource Guide: Information for
Tourists, Kanniyakumari.”15 This brochure (50,000 copies) summarised information most
needed by tourists for a tsunami, cyclone and fire including emergency contacts. The map
included (?) in the brochure was clear and easy to follow for locals and tourists. It was
made widely available in hotels, government agencies and information centres. In addition,
20 signs were produced reproducing information on emergency response.
14
Kanniyakumari Town Panchayat Office, n. d., Disaster Preparedness and Response Guide for
Kanniyakumari Town (Tsunami, Cyclone, Fire & Stampede), KTPO, UNEP, NSCI, SRSA and
IH&RA.
15
Kanniyakumari Town Panchayat Office, n. d., Tourist Emergency Resource Guide: Information for
Tourists, Kanniyakumari, KTPO, UNEP, NSCI, SRSA and IH&RA.
Page 20 of 58
3.3.2 ANSAO, Thailand
Local Coordinating Groups and Project Manager
The ANSAO CG, which was established between January and March 2007, comprised of
22 members with six representatives from government agencies, eight from the private
sector and eight from community leader groups. The first meeting was held on 17 January
2007 on Phi Phi Island and introduced the project framework and formalised the role of the
CG and the PM. A second meeting was held on 7 August 2007 where the discussion
centred on mapping, zoning, evacuation routes, communication systems and an emergency
response unit. A final CG meeting was held on 21-22 December 2008 and delivered
summary notes and impressions of the project and handed to the Chief Executive of
ANSAO and CG members the emergency plan, drill event report and guidelines for the
establishment of a crisis communication centre for promotion of future disaster reduction
mechanisms.
The PM played a critical role in establishing the project infrastructure in the ANSAO
office in Krabi. Prior to an official meeting of the CG the PM would visit each one of the
CG members to introduce the project objectives and build up relations with members.
During workshops the PM would often revisit participants before the workshop and in the
evening to ensure active participation. The PM also conducted an assessment of the local
current disaster management plans using seven criteria (risk identification and assessment,
legal authority, organisational structure, early warning systems, communication, resources
and emergency planning and tourism related issues).
The results of the assessment were conveyed to the local community during the launching
workshop on 30 March 2007. The assessment provided baseline information and guidance
that could be used for a local self assessment in the future. The assessment questionnaire
found that most ANSAO locals were concerned about fire and boat accidents than
tsunamis, with drought the least concern. The PM was also effective in gaining special
reduced prices for venues to hold the workshops.
Meetings, Workshops and Training
The ANSAO launching workshop and APELL training workshop was held on 29-31
March 2007 on Phi Phi Island. The workshop identified local hazards and risks and
attempted to prioritise them, provided comments on the ten steps of the APELL process;
and discussed if the Island promoted sustainable tourism. The assessment showed that
locals were concerned with fire, boat accidents, tsunami, wind storms and water scarcity.
On 25 to 27 June 2007, a workshop was held at Phi Phi Island to promote better
understanding of disaster risk reduction at the local level among workshop participants.
Approximately 50 people attended each day comprising of government officials, CG
members, the private sector and sea rescue volunteers. The workshop reflected a transfer of
knowledge on risk analysis (including revision of concepts of disaster, risk mapping and
vulnerability assessment), emergency planning and drills. The APELL process was
introduced as well as the expert team from UNEP and SRSA. The use of Geographic
Information Systems (GIS) was discussed in the context of disaster management.
Page 21 of 58
An APELL demonstration was held from 17-21 September 2007 which focused on
vulnerability mapping and shelter assessment. There were a total of 22 participants at the
workshop. Experts from the SRSA and IH&RA shared their experience with local partners.
The workshop was found to be useful for local disaster management. However, the
procedures for vulnerability mapping and shelter assessment would have been improved if
the data requirements had been better explained. It was apparent that CG members needed
additional GIS training.
An Emergency Response Plan for Phi Phi Island was developed and finalised by April
2008. The Plan included information on zoning, evacuation routes, communication
systems, early warning systems, people to contact and a rescue unit. The iterative process
of the plan was recognised with its review, adjustment and testing. Fire was considered to
be the most threatening hazard to locals and a fire drill was identified to test the plan.
The Disaster Prevention and Mitigation Regional Centre 11, Surattani Province (DPMRC
11) conducted a training course for emergency response 24-26 April 2008. The training
course attracted 41 participants which were trained in fire fighting techniques, victim
search and rescue and first aid. A drill preparation meeting was held 7 May 2008 to ensure
that all participants involved understood their roles in the emergency plan. The CG and the
Emergency Response Team participated. On 15 May 2008 a fire drill was successfully
conducted. Those involved in the drill recommended that the exercise be conducted
regularly and that DPMRC 11 organise additional emergency response training courses.
A workshop attended by 59 participants was held 5-6 June 2008 focussing on an exchange
of environmental management and disaster reduction within the local tourism industry. The
workshop included a case study and field visit. A small scale drill was held on 22
September 2008 and focussed on fire, building collapse and a boating accident. A small
scale training course was run 14-16 August 2008 and focused on fire fighting and search
and rescue. A small scale drill was then undertaken on wind storms, fire and boat
accidents. From 15-16 September 2008 a workshop was held at DDPM, Bangkok to train
relevant staff from a range of government agencies in the APELL process.
Publications and Infrastructure
ANSAO developed an eight page “Tourist Emergency Resource Guide: Phi Phi Don
Island”16 (5,000 in English, 5,000 in Thai). The Guide outlined four types of risk disaster
(fire, tsunamis, boat accidents, landslides) and specifically defined safety tips for each.
Tsunami Safe Areas, Fire Safe Areas and Warning Towers were clearly delineated as ten
zones on a map. The early warning system was explained and included public radio,
warning towers and the evacuation signs to safe areas. A complete list of radio
communications and emergency contact numbers were provided.
In addition, a two page brochure was produced (35,000) entitled “Tourist Emergency
Resource Guide: Information for Tourists, Phi Phi Islands.”17 The smaller size guide
summarised the A3 guide and was much easier for tourists to use. The brochure was
translated into seven languages (5,000 copies each language): English, Thai, German,
16
PMP, n.d., Tourist Emergency Resource Guide: Phi Phi Don Island, PMP, UNEP, IH&RA, SRSA.
PMP, n.d. Tourist Emergency Resource Guide: Information for Tourists, Phi Phi Islands, PMP,
UNEP, IH&RA, SRSA.
17
Page 22 of 58
Chinese, Japanese, Korean and Swedish. Translation into several languages was expensive
and time consuming. The brochure was widely available and used extensively by foreign
tourists.
Risk communication was further enhanced with 25 signs including four for landslides, 11
for fires and 10 for sea accidents. A booklet and CD ROM entitled “Guidelines for
Establishing a Crisis Communication Centre for the Tourism Industry at the Local Level”
(100 copies) was produced and assisted in distributing capacity building material.18
3.3.3 PMP, Thailand
Local Coordinating Groups and Project Manager
Formal CG meetings were held 8 January 2007, 17 May 2007 and 29 August 2007. The
CG was comprised of 16 representatives, eight from the local government, four from the
hotel sector and four from the local community. Role and responsibilities of the CG were
first discussed, followed by disaster assessment, disaster preparedness and risk analysis.
The assessment showed that after the tsunami emergency response had slightly improved.
The PM noted that the CG could not easily meet during normal office hours because of
work commitments. As a result the PM often met with CG members individually or
informally in smaller groups. The flexibility of the role of the PM was seen as paramount.
The PM undertook an assessment of local current disaster management and plans 15
March 2007 using the seven issues identified above. An initial assessment (58
questionnaires) of local current disaster management and plans revealed that the priorities
should be landslides, road accidents and tsunamis.
Meetings, Workshops and Training
The project launch and APELL training workshop was held in Phuket 26-28 March 2007
with 145 participants from the government and private sectors and the local community.
Concern was raised about the overall costs associated with simultaneous translation on a
limited budget. Participants applied the APELL process in a role playing exercise. The
length of the three day workshop was considered by many as too long. CG members signed
a Letter of Agreement committing participants to APELL project activities.
A second workshop on capacity building for disaster prevention and preparedness for CG
members and relevant agencies was held 20-22 June 2007. The workshop included three
SRSA experts and one UNEP expert. The workshop was attended by 74 participants and
emphasized risk assessment and disaster planning. GIS was introduced as a tool to assist
risk prone areas and prepare risk assessment and shelter assessment.
An APELL demonstration workshop on vulnerability mapping was held 10-14 September
2007 and 17-18 September 2007 for shelter assessment. An SRSA expert introduced the 25
participants to apply GIS to risk zones for tsunamis, landslides and fire. Vulnerability
mapping allowed participants to designate risk areas for tsunamis, landslides and fire. The
18
UNEP DTIE 2008, Guidelines for Establishing a Crisis Communication Centre for the Tourism
Industry at the Local Level, UNEP DTIE, SRSA, IH&RA, NSCI, DDPM, PMP, ANSAO.
Page 23 of 58
shelter assessment was presented by SRSA and IH&RA experts. Field visits were
organised to landslide areas, fire risk areas and potential shelters. A tsunami evacuation
drill at Patong Beach was held 27 July 2007. Local residents, tourists, business owner and
a range of stakeholders participated in the drill.
An Emergency Response Plan was finalized at the end of April 2008. This was followed
by a workshop focusing on an exchange of environmental management and disaster
reduction within the tourism industry was held 5-6 June 2008. A total of 58 participants
represented the government and private sectors and the local community. The workshop
included a case study and field visit.
Training on the integrated emergency plan was held 19-20 September 2008 for agencies
represented on the CG. Thirty participants attended the workshop representing local
government, the private sector and CG members. A small scale drill was held with CG
members, hoteliers, the Royal Thai Navy, schools and the Patong Hospital. Approximately
150 people participated in the drill which focussed on fire, landslide and a boating
accident. A workshop was held at DDPM 15-16 September 2008 in Bangkok to train
relevant staff from a range of government agencies in the APELL process.
Publications and Infrastructure
The APELL Handbook (200 copies) and seven APELL standard brochures (1,400 copies)
of four pages each were translated and given to participants on 26 March 2007. Risk
communication and awareness raising was supported by the project through the publication
of an eight page “Emergency Resource Guide: Information to Help You Enjoy Your Stay
in Patong”19 (10,000 copies). The Guide was translated into Thai and English and provided
information on fire, landslides and tsunamis. A full page colour map provided information
on shelters, information centre and landslide, fire and tsunami prone areas. Emergency
resource information was provided.
In addition, a two page brochure was produced entitled “Tourist Emergency Resource
Guide: Information for Tourists, Patong Beach, Phuket.”20 The smaller size guide
summarised the guide and was much easier for tourists to use. The brochure was translated
into seven languages (96,440 brochures, Thai 24,500; English 24,500; German 8,400;
Swedish 8,400; Chinese 11,120; Japanese 11,120 and Korean 8,400) and made widely
available to tourists.
In addition, 12 signs were erected including five for tsunamis, two for shelters and three
for emergency information. A warning siren and three warning towers were installed on
Patong Beach using the satellites of the National Disaster Warning Centre. A CD ROM
entitled “Guidelines for Establishing a Crisis Communication Centre for the Tourism
Industry at the Local Level” (200 copies) was produced and assisted in capacity building.
3.4 Catalytic Role
19
PMP, n. d.,Emergency Resource Guide: Information to Help You Enjoy Your Stay in Patong, PMP,
UNEP, IH&RA, SRSA.
20
PMP, n.d. Tourist Emergency Resource Guide: Information for Tourists, Patong Beach, Phuket,
PMP, UNEP, IH&RA, SRSA.
Page 24 of 58
This project has a number of significant catalytic roles and supports replication at different
geographical scales in the tourism sector. Importantly, at an international scale UNEP
DTIE has forged strong working professional relationships with India through the NSCI,
Thailand through the DDPM and Sweden through the SRSA. These strong international
connections increase the likely success for the medium to long term sustainability of the
project outcomes.
At a national scale the overall adoption of the APELL framework for the project makes
replication highly feasible in both India and Thailand. Both NSCI and DDPM now have
wide ranging expertise in the APELL process, staff are now familiar with its application
and have a clear understanding of its relevance. In many cases the lessons and experiences
gained from the project can be scaled up in the design and implementation for other
projects and in other sectors.
In India the NSCI has a very strong institutional framework that supports the continuing
and ongoing application of APELL (see section 3.7). For instance, NSCI has applied the
APELL framework to ports, the mining industry and now tourism. This strong institutional
support for APELL will continue to ensure that NSCI will provide leadership and support
in disaster mitigation. Importantly, India in 2002 was the first country to develop a
National APELL Centre and there was evidence that this commitment will likely continue.
NSCI also has strong institutional cooperation with the National Institute of Disaster
Management, Ministry of Home Affairs.21 These cross institutional linkages can work to
support greater application of the APELL process across a range of sectors. At the Indian
APELL workshop, issues of sustainability and replication were specifically discussed.
Both the Ministry of Tourism and NSCI showed interest in disseminating the project
material to other hazard prone areas in India. The workshop supported training to increase
the number individuals familiar with the APELL process. Specific mention was made of
the introduction of disaster management into hotel courses.
The APELL process has not been adopted as long in Thailand as in India However, there
was strong evidence to suggest that the Thai government has widely supported its future
application in disaster mitigation. For instance, DDPM has suggested that the project
framework will be used in the other 75 provinces in Thailand. This is further evidence of
the catalytic role of the project and the potential for scaling up in design and
implementation. At the Thailand APELL workshop issues of sustainability were also
discussed. The potential application of APELL in the Community Based Disaster Risk
Management Methodology for other parts of Thailand was discussed.
The project also has a potentially strong catalytic role at the local government and
community level. Local governments and municipalities have shared experiences with
other local governments. The project played a significant catalytic role in supporting crisis
communication in the tourism sector through a publication entitled “Guidelines for
Establishing a Crisis Communication Centre for the Tourism Industry at the Local
Level”.22 This publication had wide applicability and was further evidence of increased
sustainability of the project.
21
PG Dhar Chakrabart, Executive Director, National Institute of Disaster Management, Ministry of
Home Affairs, personal communication, February 2009.
22
Allis, E., Papadopoulos, T., Rosental, A. And Arvidsson, K. 2008. Guidelines for Establishing a
Crisis Communication Centre for the Tourism Industry at the Local Level, UNEP, SRSA, IH&RA,
Page 25 of 58
In both India and Thailand the CGs ensured that a range of community partners were
effectively trained and could undertake appropriate mitigation measures for disaster
response after the project was completed. PMs played a key role in the implementation of
the overall project and its overall acceptance by the community. PMs expanded the
representation of key stakeholders involved and thus played an important catalytic role in
widening the influence of the project
Whilst there is no direct evidence of destination and tourism associations using the results
in a marketing strategy, there is evidence that the results of the project have been widely
disseminated. For instance, a range of brochures, signs and publications were made
available for international and national tourists in India and Thailand.
3.5 Assessment of Monitoring and Evaluation Systems
There is no specific mention of an M&E plan in any of the documentation and how this
might provide baselines and track progress on the project. There was no evidence that
resources were allocated for the implementation of an M&E plan. Interviews conducted in
both countries failed to mention the use or application of a M&E plan. Further, there was
no evidence to show that information generated by an M&E plan was used to adapt and
improve the project. It would seem that the project did not meet the minimum requirements
for project design of M&E and the application of the project M&E plan.
Although M&E plans were not used, there were specific relevant and transparent systems
in place to document, assess and track progress made in the project. Activity reports were
compiled for project sites and specified the project delivery into: activities/outputs; status
(complete/current); and comments. Activities/outputs included meetings, publications,
training, technical information and technical cooperation. This format made it possible to
track the development of specific activities and outputs throughout the project. There is
presently no long term monitoring of the project envisaged by either India or Thailand.
Evidence derived from an audit of the quarterly reporting for both countries showed that
the NSCI, ANSAO and PMP all provided adequate quality assurance for the financial
management of the project. The control of financial resources was transparent and
accountable. The reporting process included interim reports and quarterly reports. The
quarterly reports included an activity report, monthly timesheets and a detailed financial
report. Monthly timesheets were submitted by the PMs with details on the number of hours
worked, location and activities undertaken. Evidence of receipts and spending was viewed
and presented with the quarterly reports. Original tickets, boarding passes and receipts of
tickets purchased were verified.
An independent assessment of the project was done by the European Union Monitoring
Team (Ms Kamila Kaluszka). The Team visited PMP and ANSAO on 25 April 2007 and
26 April 2007, respectively, to monitor the implementation of the project Five aspects of
the project were monitored: relevance and quality of design; efficiency of implementation
to date; effectiveness to date; impact to date; and potential sustainability. The Team
concluded that the project was good and that it was on track to meet its aims and timelines.
NSCI, DDPM, PMP, ANSAO.
Page 26 of 58
3.6 Preparation and Readiness
The project objectives outlined in section 1.1 were clear, practicable and feasible in the two
year timeframe. Given the extensive UNEP DTIE experience in the APELL process there
was good preparation, readiness and institutional capacity at the international level. Project
management was, therefore, sound. As a result UNEP DTIE correctly identified the
relevant and appropriate counterparts in India (NSCI), Thailand (DDPM) and Sweden
(SRSA) when designing the project at the outset.
At a national level both India and Thailand were adequately prepared to oversee and
implement the project. There was evidence that partnership arrangements were properly
identified and the roles and responsibilities negotiated prior to project implementation. The
institutional capacities within NSCI to adopt the APELL process were well proven (see
section 3.7). NSCI’s track record with other APELL projects meant that these lessons
properly informed the project design. In the project design phase Thailand did not have the
same level of institutional capacity regarding the APELL process as India. However,
DDPM readily adopted the APELL process and developed expertise within the
organisation.
The designation of PMs and CGs played an important role in preparing for the project at
the local level and supported capacity building. The PMs worked effectively to prepare
stakeholders for workshops, drills and emergency planning exercises. The CGs was then
ready for the roll out of the project and could meaningfully contribute.
3.7 Country Ownership
At an international level the project supports the February 2001 Governing Council
Meeting (Nairobi, Kenya) which requested that UNEP “establish a Strategic Framework to
support the implementation of measures of Emergency Prevention, Preparedness,
Assessment, Response and Mitigation.”23 The APELL Program plays an important role in
fulfilling these objectives a key role in delivering disaster prevention, mitigation and
response in India and Thailand. The involvement of the Swedish government through the
SRSA was appropriate and timely given the overall mandate of the project.
India, through the NSCI, has a long tradition of addressing safety, health and
environmental issues. The NSCI played an important and catalytic role in the project and
had extensive experience in APELL Program. From 1992 to 1997 NSCI undertook the
APELL-LAMP project which brought together industry, local government and the
community to develop integrated community emergency response plans. Six areas were
selected to illustrate the program. Achievements from the project included: a systematic
methodology for rehearsing Emergency Plans; infrastructure and technical facilities at the
local level; and identifying the potential for national guidelines. The TRANS-APELL
Program (2000) focused on dangerous goods transport and emergency planning in local
communities. The National APELL Centre (NAC) project (2002) designed and developed
models, approaches and programs and disseminated them widely throughout India. For
instance, a NAC Newsletter was published, traffic police were trained on HAZMAT
responses and training programs were conducted. The project has important relevance to
India’s regional agreements, national development and environmental agendas.
23
More information can be obtained from the 21st Session of the Governing Council Global
Ministerial Environment Forum, Decision 21/17.
Page 27 of 58
Importantly, India showed additional country ownership by providing in kind support to
the project by allocating NSCI staff to the project at the national level. At a local
government level in kind support was provided by additional institutional support for the
PMs and CGs.
Thailand has extensive experience with international and regional agreements dealing with
disaster response and mitigation. Since 2004 Thailand has been a member of the Asian
Disaster Reduction Centre and participated with the United Nations Office of the
Coordination of Humanitarian Affairs (UNOCHA) on disaster risk assessment and the
United Nations Economic and Social Commission for Asia and Pacific (UNOCHA) on
South-South cooperation on disaster risk management. Cooperation has been supported
with the Asian Disaster Preparedness Centre (ADPC) (technical cooperation in disaster
management), the Japanese International Cooperation Agency (disaster management
capacity development); and Joint United States Military Assistance Group
(JUSMAGTHAI) (capacity building within DDPM). The project has important relevance
to Thailand’s regional agreements, national development and environmental agendas.
Additional in kind support was provided by DDPM at the national level through the
provision of personnel to assist the project. Considerable additional time and support was
provided by local government in support of the PMs, CGs, workshops and implementing
broad community awareness of the project.
3.8 Stakeholder Participation and Public Awareness
One of the strengths of this project has been the strong and effective participation of a wide
range of relevant stakeholders in both India and Thailand. The project consulted and made
use of the broad skills, experience and knowledge of the appropriate government agencies,
nongovernment organisations, community groups, private sector, local governments and
academic institutions in the design, implementation and evaluation of project activities.
Further evidence of this participation was seen in the successful implementation of a range
of workshops, fire drills and community involvement.
In India the NSCI received technical support from UNEP DTIE and SRSA. There was
evidence that the SRSA played a very strong role in the project in both countries. Their
technical support and educational role was highly valued. The IH&RA did not play an
active role in the first half of the project. This changed in the second half when greater
IH&RA involvement was required in the shelter assessment. Some members of the
IH&RA voiced concern that supplying disaster awareness brochures would scare some
guests and they did not subsequently support their wide distribution.24
Local project implementation in Kanniyakumari was critical. NSCI has the Tamil Nadu
sub-centre (National APELL sub-centre) in Tuticorin. The sub-centre has experts with the
local knowledge (language, culture and legislative framework) and provided support for
the Kanniyakumari project. The PM was based in Tuticorin and was supported locally by a
project facilitator identified by NSCI when the project commenced. In addition, a 33
member CG was established in March 2007 and represented the wide range of stakeholders
in the project.
24
Deputy Mayor Chairat Sukban, Phuket, personal communication, 27 February 2009.
Page 28 of 58
In Thailand an ANSAO CG was established in early 2007. The CG was comprised of 22
members with representatives from government agencies, private sector and community
groups. Formal CG meetings were held throughout the project. Members were comprised
of eight from the local government sector, four from the private sector and four from the
local community. Initial meetings introduced the project framework and formalised the
role of the CG and PM.
Some problems were noted with respect to stakeholder participation. Some of CG meetings
were scheduled during normal business hours and during the peak tourism season. Some
workshops were three days events. Both situations made it difficult for representatives of
the IH&RA, hotel managers and tourism operators, who were often very busy to attend.25 26
This potential weakness was overcome by the commitment of the PM.27 However, the PM
would often meet with CG members individually or informally in smaller groups after
normal business hours and on weekends. The flexibility of the role of the PM was seen as
critical to the success of the project. Some CG members felt that greater public awareness
should have been encouraged for first responders in a hazard. Those working on the beach
needed specific ongoing training in emergency drills and first aid skills.28 The project was
also effective in raising the awareness of other types of disasters and natural hazards such
as erosion and fire.29 There was also concern that there were limited personnel at the local
government level and more expertise was needed for ongoing training, master planning
and drill evacuation. 30
In summary, the project involved relevant stakeholders through information sharing,
consultation and sought their participation in project design, implementation, and
monitoring and evaluation. The degree and effectiveness of collaboration between the
various project partners and institutions during implementation of the project was
excellent. The degree and effectiveness of various public awareness activities undertaken
during the implementation of the project was highly successful. As far as could be
determined there were no opponents to the project in either country.
3.9 Financial Planning
The project was financed completely by the European Commission. As a result there was
no need to leverage funds from other partners or identify sources of co-financing. The
Indian and Thai governments were cognizant of the level of resources for the overall
activities from the commencement of the project.
Evidence derived from an audit of the quarterly reports showed the NSCI, ANSAO and
PMP all provided adequate quality assurance for the financial management of the project.
The control of financial resources was transparent and accountable. The reporting process
included: activity reports, interim reports; quarterly expenditure reports and monthly
timesheets. The quarterly expenditure reports included line items for the following: project
25
Ms Jongrak Athimuttisun, Patong Community Group, personal communication, 27 February 2009.
Ms Jongrak Athimuttisun, Patong Community Group, personal communication, 27 February 2009.
27
Wassanaporn Palimeechai, Patong Traditional Massage Group, personal communication, 27
February 2009.
28
Mr Tia Wongsuwan, Phuket, Chief of the Food Carrier Group, personal communication, 27
February 2009.
29
Ms Jongrak Athimuttisun, Patong Community Group, personal communication, 27 February 2009.
30
Deputy Mayor Chairat Sukban, Phuket, personal communication, 27 February 2009.
26
Page 29 of 58
personnel, project manager, administrative support, travel, conferences, rental of
conference rooms, participant travel, non-expendable equipment, car rental, computer,
printer, software, internet registration, publications and printing, CD burning and
translations. Monthly timesheets calculated the average of hours worked per day over each
week. As one would expect, as the project gained momentum the hours per day worked for
the PM increased accordingly.
There were some issues raised regarding the amount of funding allocated for organising
workshops and meetings. The Thai partners argued that not enough funding was allocated
and that the PM had to negotiate with hotel owners to gain discounts to facilitate
workshops. During the timeframe of the project the exchange rates increased and this
placed further financial burden on the partners. Simultaneous translation was a costly
exercise and could not be supported in all workshops. In some cases UNEP and SRSA met
the cost of sending experts to meetings with the partners. This additional funding was
covered outside the project. In Thailand the PM also noted the high cost of living and the
need for PMP officials to bargain for meeting venues, rooms, utilities, food and drinks for
workshops. The UNEP budget for a three day workshop was USD 2,826.50 and was
considered too low.
The Indian project was extended officially by three months and they did not spend all of
the allocated funds. Additional relevant activities were proposed but were evaluated as
being outside the project scope and guidelines and subsequently refused. The funding
framework was not flexible enough to use the money and a small portion was returned. In
general, appropriate standards of financial planning were applied throughout the project.
3.10 Implementation Approach
UNEP coordinated the overall implementation of the project, working closely with the
European Commission and the SRSA in planning and execution. There was evidence that
project implementation mechanisms outlined in the project document were followed during
the life of the project. UNEP was responsible for supervising NSCI and DDPM partners in
India and Thailand respectively. NSCI and DDPM were correctly identified as the key
country agencies for the project. Both NSCI31 and DDPM32 closely followed the same five
key stages in the implementation of the project (assessment, training and demonstration,
awareness raising and training materials, capacity building, public information) which
ensured parity in the overall implementation approach. Both national agencies played an
important role in working with UNEP and successfully implementing the project’s
objectives at the local government level. UNEP was also responsible for liaising with
IH&RA, ANSAO and PMP and ensured that the project was implemented consistently
across the three tourism destinations and that the project was executed according to the
original plan. There was evidence that the project was largely effective and efficient.
The APELL process adopted by UNEP DTIE played a critical role in the implementation
approach to the project. The ten step process (Table 1) provided a transparent and
accountable methodology for the implementation and supported integrated emergency
response for local communities, governments and emergency agencies. One of the key
31
NSCI, 2009. Project Implementation Presentation, 24 February 2009, NSCI Headquarters,
Mumbai, India.
32
DDPM, 2009. Terminal Evaluation Presentation, 26 February 2009, DDPM Headquarters,
Bangkok, Thailand.
Page 30 of 58
aspects of the APELL process in this project was how key locals were identified and their
expertise effectively used in the implementation.
This participatory approach outlined in the APELL process requires an effective CG to
play an ongoing and important role in the implementation of the project. There was
evidence that active representation from a wide range of key stakeholders made
implementation effective and that project managers, in some cases, were adaptable in their
role. For instance, the role of the local PMs was viewed as critical in the successful
implementation of the project and there was considerable evidence that they worked
effectively and efficiently throughout the project. The PMs worked in meetings and outside
normal office hours to ensure that the local community and key stakeholders felt
ownership of the project.
Concerns were raised about some aspects of the implementation of the project. CG
representatives that ran small businesses, hotels or tourism operators often found it difficult
to participate in workshops, meetings and drills during the peak tourism season. In many
cases the PM recognised the need to be adaptable and worked extra hours or weekends to
keep the CGs abreast of developments and ensure that they were adequately briefed on the
project.
There was also concern raised that the implementation of the project could have been
better planned in the initial stages.33 This initial preparation would have supported more in
depth training for each partner and allowed a stronger common understanding between all
partners on the project’s aims and objectives. However, this would have required further
targeted resources to instigate.
3.11 UNEP Supervision and Backstopping
UNEP DTIE provided effective supervision and administrative and financial support for
the project in both countries given the complexities involved. Supervision was evident in
all critical phases of the project and included a range of face to face meetings in France,
Sweden, India and Thailand. SRSA provided very effective backstopping and played a
critical role in fulfilling the operational or technical aspects of the project. There was
evidence to suggest that the project met all requirements for financial standards and
progress reporting. This project is rated Highly Satisfactory with respect to UNEP DTIE
supervision and backstopping.
In Phase A (Assessment Phase, Establishment of Project Management Structure) two
meetings were held that assisted the training of the local project partners and formalised
relationships through the signing of a Memorandum of Understanding between UNEP and
the various partners. The UNEP Project Coordinator attended both meetings. UNEP DTIE
held the first meeting in Paris, France (4 December 2006) and a second meeting in
Karlstad, Sweden (6-8 December 2006). The meetings were attended by representatives
from DDPM, local PMs from India and Thailand, local government officials from Patong
and the IH&RA.
33
UNEP DTIE, 2009. Final Report on Disaster Reduction through Awareness, Preparedness
and Prevention Mechanisms in Coastal Settlements in Asia: Demonstration in Tourism
Destinations, UNEP DTIE, Paris, France, page 16.
Page 31 of 58
In Phase B (Training and Demonstration) three SRSA experts and one UNEP expert were
involved in capacity building with respect to disaster prevention and preparedness. The
three workshops explained risk assessment, risk mapping, vulnerability assessment and
emergency planning and drills. In India the workshop was held on 2-5 May 2007. In
Thailand the workshops were held in Patong (20-22 June 2007) and ANSAO (25-27 June
2007).
Overall, face to face meetings were seen as very beneficial for ongoing project
management. However, the overall number of these meetings was limited given budget
restrictions. An additional two or three face to face meetings would have been useful and
relevant and should have been incorporated into the two year timeframe of the project. The
other supervisory challenge that UNEP DTIE faced was the problem of communicating in
English as the operational language of the project. This was especially problematic for
some of the Thai partners.
UNEP DTIE maintained close ongoing supervision through electronic mail and conference
calls with all project partners. Given the geographical extent of the project and the fact that
UNEP Project Coordinator had to manage five partners across local, national and
international scales, emails were considered the most successful means of communication
between UNEP DTIE and project partners. Emails were exchanged at least every other day
and there was no week during the project without email exchange. Use of telephone was
less often. However, there were at least two or three calls from UNEP DTIE to the
European partner and at least one call in a month for each of the Asian partners.
Additional important backstopping by UNEP DTIE was provided by the extensive expert
involvement of the SRSA throughout the project. SRSA was involved in Phase A
(Assessment) meetings in Paris and Karlstad and Phase B (Training and Demonstration)
meetings in India and Thailand. It were particularly effective in the APELL demonstration
activity that involved vulnerability mapping and shelter assessment. In Phase D (Capacity
Building) SRSA raised awareness of environmental conservation and disaster impact
reduction to the public and tourism sector. In Phase E (Public Information and Wider
Dissemination) SRSA sent two experts to prepare for small scale disaster response drills.
SRSA was also involved in developing APELL trainers at the national level in both
countries. Overall, SRSA was an active, effective and strong supporter of the project,
especially in providing technical expertise and training.
In summary, UNEP DTIE provided effective supervision and administrative and financial
support for the project in both countries given the complexities involved. SRSA provided
very effective backstopping and played a critical role in fulfilling the operational or
technical aspects of the project.
Page 32 of 58
4.
Conclusions and Ratings of Project Implementation Success
In conclusion, Table 4 presents to overall ratings of the project according to criterion
outlined in the project brief.
Table 4: Overall Ratings of the Project
Criterion
Evaluator’s Summary Comments
A. Attainment of project
objectives and results (overall
rating)
Sub criteria (below)
A. 1. Effectiveness
The project was successful in attaining the
five objectives outlined in the brief.
A. 2. Relevance
A. 3. Efficiency
B. Sustainability of Project
outcomes
(overall rating)
Sub criteria (below)
The project was highly effective in
improving disaster preparedness capabilities
of local authorities, tourism industry and
residents of the disaster-prone coastal
tourism destinations for effective emergency
response. Awareness of local authorities,
local residents and tourists, and the local
tourism industry about the importance of
natural and human-made disaster prevention
and preparedness was increased through a
wide range of relevant media. Local
environmental management and coastal
planning capabilities of local authorities
were supported through the identification of
local hazards and an assessment of local
vulnerabilities. Risk communication was
improved between the local authorities, the
private sector (tourism and other private
business) and the exposed community
(tourists and local population) on local
disaster issues.
The project’s aims and objectives were
highly relevant to the core business of
UNEP DTIE, NSCI and DDPM. The
application of the APELL process was very
relevant and widely supported.
The project had to deal with wide
geographical scope and a number of partners
at different scales. Some efficiency gains
could have been realised in project planning
and implementation. However, given the
overall challenges the project was run
efficiently.
UNEP DTIE, NSCI and DDPM had very
strong capacity to plan and implement the
project. With respect to institutional
frameworks and governance the project has
been very successful in gaining broad
institutional support in both countries at a
range of levels. However, the level of
financial resources over the past two years
cannot be sustained by either country and
therefore the sustainability of the project is
rated lower.
Evaluator’s
Rating
SS
HS
HS
S
MU
Page 33 of 58
Criterion
Evaluator’s Summary Comments
B. 1. Financial
The level of financial resources over two
years cannot be sustained by either country.
There is no evidence that governments, the
private sector or NGOs are willing to invest
cash or in kind contributions to sustain the
current level of the project.
The future success of the project is closely
tied to the unique socio-political factors
evident in India and Thailand. Sociopolitically both NSCI and DDPM are
recognised as national leaders in disaster
prevention and mitigation. How this
transfers to local governments, communities
and the private sector remains to be seen.
Project outcomes have been supported by
the encouragement of a wide range of
stakeholders to be involved in supportive
programs. Links with other programs have
been established and are relevant in
supporting sustainability and socio-political
factors.
Environmental sustainability is evaluated in
terms of any environmental risks that can
undermine the future flow of project
environmental benefits. The project aimed to
increase the disaster management capacity
of stakeholders, thus this criterion should be
considered as environmental risks
undermining the disaster management
capacity. This sub-criteria is not applicable
to the project
A range of relevant outputs and activities
have been undertaken in the Project.
Coordinating Groups and Project Managers
played an important role in sustaining the
activities of the project. Meetings,
workshops and training were widely
supported and successfully ran. Publications
and infrastructure such as signs were widely
disseminated.
There was no formal M&E plan for the
project but this was not a requirement of the
project design. However, there were specific
relevant and transparent systems in place to
document, assess and track progress such as
activity reports. Project delivery was
detailed as activities/outputs; status
(complete/current); and comments.
Activities/outputs included assessments of
meetings, publications, training, technical
information and technical cooperation.
There was also an evaluation by the donor
and this evaluation.
B. 2. Socio Political
B. 3. Institutional framework
and governance
B. 4. Environmental
C. Achievement of outputs and
activities
D. Monitoring and Evaluation
(overall rating)
Sub criteria (below)
D. 1. M&E Design
As discussed above there was no formal
M&E plan project but this was not a
requirement of the project design. The
activities and outputs detailed above means
Evaluator’s
Rating
MU
L
L
N/A
HS
MS
MS
Page 34 of 58
Criterion
D. 2. M&E Plan
Implementation (use for
adaptive management)
D. 3. Budgeting and Funding
for M&E activities
E. Catalytic Role
F. Preparation and readiness
G. Country ownership /
drivenness
H. Stakeholders involvement
I. Financial planning
Evaluator’s Summary Comments
that M&E design would meet basic
minimum requirements.
There was no formal M&E plan project but
this was not a requirement of the project
design. The activities and outputs detailed
above means that M&E Plan
Implementation would meet basic minimum
requirements.
An evaluation of the budget was included in
the project. Provision was made for funding
reporting activities.
This project has a number of significant
catalytic roles and supports replication at
different geographical scales in the tourism
sector. Importantly, at an international scale
UNEP DTIE has forged strong working
professional relationships with India through
the NSCI, Thailand through the DDPM and
Sweden through the SRSA. These strong
international connections increase the
likelihood for the medium to long term
sustainability of the project outcomes.
There was concern raised that the
implementation of the project could have
been better planned in the initial stages. This
initial preparation would have supported
more in depth training for each partner and
allowed a stronger common understanding
between all partners on the project’s aims
and objectives.
Both countries had strong ownership of the
project through extensive disaster response
and mitigation at the national level. Local
governments, the private sector, tourism
businesses and NGOs actively participated.
Both the national and local levels of
government actively participated in project
development,
inception
through
to
evaluation and follow up.
The project consulted and made use of the
skills, experience and knowledge of the
appropriate
government
agencies,
nongovernment organisations, community
groups, private sector, local governments
and academic institutions in the design,
implementation and evaluation of project
activities.
The reporting process included: activity
reports, interim reports; quarterly
expenditure reports and monthly timesheets.
The quarterly expenditure reports included
line items for the following: project
personnel, project manager, administrative
support, travel, conferences, rental of
conference rooms, participant travel, nonexpendable equipment, car rental, computer,
printer, software, internet registration,
publications and printing, CD burning and
Evaluator’s
Rating
MS
MS
S
S
HS
S
MS
Page 35 of 58
Criterion
J. Implementation approach
K. UNEP Supervision and
backstopping
Overall Rating
Evaluator’s Summary Comments
translations. Monthly timesheets calculated
the average of hours worked per day over
each week.
Concerns were raised about some aspects of the
implementation of the project. CG
representatives that ran small businesses, hotels
or tourism operators often found it difficult to
participate in workshops, meetings and drills
during the peak tourism season. There was also
concern that the implementation of the project
could have been better planned in the initial
stages.
UNEP DTIE was responsible for the
oversight of the project and provided support
through
development
and
approval,
including facilitating communication. UNEP
DTIE supervision was evident in all critical
phases of the project and included a range of
face to face meetings in France, Sweden,
India and Thailand. UNEP DTIE collected
and reviewed quarterly progress reports
from the PMs in India and Thailand. There
was evidence to suggest that the project met
all requirements for financial standards and
progress reporting.
The most prevalent rating in this Table was
“Satisfactory”
Evaluator’s
Rating
S
S
S
5. Lessons Learned
1.
The project outcomes suggest that the APELL process was effective in improving
the disaster management capacity of the tourism sector and that it has wider
applicability in the region. The APELL process should be adopted in other Asian
countries located on the Indian Ocean and applied to a range of disaster related
activities. Country partnerships and associated funding should be explored to
disseminate the APELL process as a matter of urgency. The ten step APELL
process explicitly supports wide stakeholder participation, public awareness and the
identification of active participants and their roles. The participatory approach was
a real strength of this project. It meant that relevant stakeholders were engaged
through information sharing and consultation, and their participation was sought in
project design, implementation, and monitoring and evaluation.
2.
UNEP DTIE, SRSA, NSCI and DDPM now have proven expertise in the delivery
of disaster planning and mitigation projects. This international and national
expertise should be further applied in a wide range of disaster mitigation projects
throughout Asia at the national, regional and local levels. The experience gained by
project partners would assist the delivery of disaster mitigation projects and would
have certain replication value throughout the region
Page 36 of 58
3.
Risk communication should be widely supported between the local authorities, the
private sector (tourism and other private business) and the exposed community
(tourists and local population) on a range of local disaster issues. This builds the
trust of tourists in the tourist destinations. Disaster management planning needs to
be widely disseminated through material on disaster awareness, preparedness and
response. Brochures, booklets, signs and general community awareness increase the
overall knowledge of tourists on the types of hazards and associated risks.
4.
Significant time is required in the initial preparation of the project. This includes
putting in place appropriate administrative frameworks, reporting mechanisms and
ensuring that each partner has a common understanding of the project’s overall
aims and objectives. UNEP should factor in additional time at the commencement
of project activities as part of the overall time for project implementation.
5.
The preparation and implementation of an M&E plan should be formally supported
at the outset of the project design. This would allow the systematic development of
baselines for existing conditions and verifiable indicators of success. Such a plan
would also support specific procedures for data collection and importantly,
monitoring of project outcomes against project objectives.
6.
In order that the project outcomes are sustainable beyond the short to medium term
there needs to be increased engagement with relevant local government authorities
responsible for disaster planning and mitigation, the private sector including hotels
and tourism operators and the local community. UNEP DTIE now has proven
expertise in the delivery of disaster response and mitigation projects. This expertise
should be further developed with other capacity building and implementation
activities undertaken by UNEP in the region.
Page 37 of 58
Annex 1: Terms of Reference
TERMS OF REFERENCE
Terminal Evaluation of the UNEP project
“Disaster Reduction through Awareness, Preparedness and Prevention Mechanisms in
Coastal Settlements in Asia –Demonstration in Tourism destinations” - CP/4020-06-05 =
CPL 5068 3977 2643
1. PROJECT BACKGROUND AND OVERVIEW
Project rationale
This goal of the project was “to minimise casualties, property and environmental damages
from natural and man-made disasters of three tsunami-hit tourism destinations
(Kanniyakumari in Tamil Nadu State, India; Patong in Phuket; and Pi Pi Island in Krabi,
Thailand) by improving the local authorities and the private sector ability to manage natural
and man-made disasters and by educating the residents and tourists on how to properly react
to disasters”.
The project was planned for implementation using the United Nations Environment
Programme’s APELL process at tourism destinations, as it has been done for other activities
or sectors, like ports and mining industry. APELL is a process designed to create public
awareness of hazards and to ensure that communities and emergency services are adequately
trained, coordinated and prepared to cope with disaster. The project activities include
assessment of local hazards, APELL demonstrations, and risk communication. The target
groups were (i) local authorities (emergency services); (ii) local tourism sector; and (iii) local
residents and tourists.
The project aimed specifically at:
1. improving disaster preparedness capabilities of the communities (local authorities,
local tourism industry and residents) of disaster-prone coastal tourism destinations for
effective emergency response through the adaptation of the existing APELL process to
tourism destinations, taking into account the challenges (cultural, linguistic, logistic)
inherent to tourism destinations;
2. raising awareness of local authorities, local residents and tourists, and of the local
tourism industry about the importance of natural and man-made disaster prevention
and preparedness (through awareness raising events and brochures production and
distribution) and about the importance of environmental management and
conservation, and its relationship with disaster reduction (through specific training
sessions to local residents including local school children);
3. improving local environmental management and coastal planning capabilities of local
authorities through the identification of local hazards, assessment of local
vulnerabilities suggestion of preventive measures, and production of vulnerability
maps with high risk zones that will allow proper risk communication, and land use
planning that will in turn promote disaster reduction and environmental conservation;
Page 38 of 58
4. improving risk communication between the local authorities, the private sector
(tourism and other private business) and the exposed community (tourists and local
population) on local disaster issues to allow better and effective response in crisis
situations leading to less loss of life and less environmental damage; and
5. increasing the trust of tourists in the tourism destinations by disseminating the project
results and by promoting the application of this proposed project approach to other
disaster-exposed tourism destinations in Asia.
Executing Arrangements
UNEP Division of Technology Industry and Economics (DTIE) coordinated the overall
project implementation, it was responsible to report and liaise with the European
Commission, supervise project partners, and worked closely with the National Safety Council
of India (NSCI), the Swedish Rescue Services Agency (SRSA), International Hotel and
Restaurant Association (IH&RA), Patong Municipality of Phuket (PMP) and with Pi Pi Island
Authority of Krabi (ANTAO-PP) to ensure that all the activities were consistent in the three
destinations. UNEP-DTIE also provided technical expertise on the APELL methodology,
supported and ran jointly with SRSA the training workshops. UNEP-DTIE invited
government authorities to participate in the project events and signed the agreements with the
project partners.
The SRSA provided the expertise needed to build the local capacity for the APELL
methodology and on the theme areas that support disaster prevention and preparedness, like
risk assessment, vulnerability mapping, emergency planning, emergency drills, evacuation
and shelter, and other issues. The SRSA followed and supported more closely the project
implementation in Thailand, supported the PMP and ANTAO-PP local project manager in
their activities.
The NSCI received technical support from UNEP-DTIE and SRSA but at the same time
guaranteed the local project implementation in Kanniyakumari. NSCI has Tamil Nadu subcenter (National APELL sub-center) in Tuticorin (only 100km from Kanniyakumari) that has
qualified people with the local knowledge (language, culture and legislative framework)
needed to properly undertake the management of the project in Kanniyakumari. The local
project manager was based in Tuticorin and was supported locally by a local project facilitator
identified by the NSCI when the project started.
The IH&RA provided technical expertise to the project and took the lead in preparing
material and ran the trainings for the tourism sectors and had a key role in the replication of
the approach to other tourism destinations. They guaranteed that expertise from the sector and
other regions would be transferred to the affected region and they ensured that the tourism
views were incorporated in the development and delivery of all materials developed in the
project.
The Patong Municipality of Phuket and the Pi Pi Island Authority of Krabi were partners
responsible for local project management, logistics and to then transfer the acquired
knowledge to their correspondents in the provinces of Phuket and Krabi. At the initial phase
of the project, the local municipalities should have identified local project managers that
would have undertaken project implementation at both locations.
Page 39 of 58
In Thailand DTIE also worked through the Department of Disaster Prevention and Mitigation
(DDPM) which was not officially a project partner but an invited participant and very active
project partner. All communications in Thailand were channeled through the DDPM
Legislative mandate



GC-23/7 - Strengthening environmental emergency response and developing disaster
prevention, preparedness, mitigation and early-warning systems in the aftermath of the
Indian Ocean tsunami disaster
GC-22/8 - Further improvement of environmental emergency prevention,
preparedness, assessment, response and mitigation
GC-22/7 - Engaging Business and Industry
Project Activities
The project duration was planned for 24 months starting on the 1st of October 2006 and
ending on the 30th of December 2008.
The project activities were organized into five phases:
Phase A: Assessment Phase and setting-up the project management structure
Phase B: Training and Demonstration Phase
Phase C: Development of awareness raising and training material
Phase D: Capacity Building Phase
Phase E: Public Information and Wider Dissemination Phase
Budget
The project was fully funded by the European Commission at a total cost of the project of
US$ 812,603
Page 40 of 58
2.
TERMS OF REFERENCE FOR THE EVALUATION
2.1. Objective and Scope of the Evaluation
The evaluation shall be conducted as an in-depth evaluation. The objective of the evaluation is
to establish project impact, and review and evaluate the extent to which implementation of
planned project activities and outputs have been accomplished. The evaluation shall
determine the extent to which the project has been successful in fulfilling its objectives and
obtaining the expected results and whether it has been cost effective in producing its results.
The evaluation will cover all key activities undertaken within the framework of the project as
described in the project document. The evaluator will compare planned outputs of the project
with actual outputs and assess the actual results to determine the impact of the project. The
evaluation will answer the following key questions:
a) How and to what extent has the project improved the abilities of local authorities and
the private sector to manage natural and man-made disasters and educating the
residents and tourists on how to properly react to these?
2.2. Method
This terminal evaluation will be conducted as an in-depth evaluation using a participatory
approach whereby the UNEP/EOU, UNEP DTIE Project Manager, key representatives of the
executing agencies and other relevant staff are kept informed and regularly consulted
throughout the evaluation. The consultant will liaise with the UNEP/EOU and the
UNEP/DTIE Project Manager on any logistic and/or methodological issues to properly
conduct the review in as independent a way as possible, given the circumstances and
resources offered. The draft report will be circulated to UNEP/DTIE Project Manager, key
representatives of the executing agencies and the UNEP/EOU. Any comments or responses
to the draft report will be sent to UNEP/EOU for collation and the consultant will be advised
of any necessary revisions.
The findings of the evaluation will be based on the following:
a) Desk review of project documents, output, half-yearly progress reports, monthly
financial reports, terminal report, minutes of meetings and relevant correspondence.
b) Review of specific products including publications, management and action plans,
database and web-site updates
c) Telephone interviews with relevant UNEP/DTIE project manager and Fund
Management Officer, and other relevant staff in UNEP dealing with sustainable
tourism and disaster management
d) Telephone interviews with relevant project partners in SRSA, NSCI, IH&RA, Patong
Municipality of Phuket and Pi Pi island Authority of Krabi.
e) Telephone and personal interviews with relevant stakeholders involved including the
Project Coordination Unit, Project’s Advisory Committee.
Page 41 of 58
f) Field visits and direct interviews with the project main beneficiaries in India and
Thailand
Key Evaluation principles
In attempting to evaluate any outcomes and impacts that the project may have achieved,
evaluators should remember that the project’s performance should be assessed by considering
the difference between the answers to two simple questions “what happened?” and “what
would have happened anyway?”. These questions imply that there should be consideration
of the baseline conditions and trends in relation to the intended project outcomes and impacts.
In addition it implies that there should be plausible evidence to attribute such outcomes and
impacts to the actions of the project.
Sometimes, adequate information on baseline conditions and trends is lacking. In such cases
this should be clearly highlighted by the evaluator, along with any simplifying assumptions
that were taken to enable the evaluator to make informed judgements about project
performance.
2.3. Project Evaluation Parameters
The success of project implementation will be rated on a scale from ‘highly unsatisfactory’ to
‘highly satisfactory’. In particular the evaluation shall assess and rate the project with respect
to the eleven categories defined below:34
A. Attainment of objectives and planned results:
1. Effectiveness: Evaluate how, and to what extent, the stated project objectives have
been met (by activities), taking into account the “achievement indicators” in the
project logframe/project document. The analysis of outcomes achieved should
include, inter alia, an assessment of the extent to which the project has directly or
indirectly assisted policy- and decision-makers to apply information supplied by this
project:
 Evaluate the immediate impact of the project on national management capacity
necessary to mitigate environmental disasters. As far as possible, assess how
the project has built the capacity of local communities (i.e. local authorities,
residents and local tourism industry) to improve disaster preparedness
capabilities for effective emergency response; raising awareness about the
importance of natural and man-made disaster preventions/preparedness an
that of environmental management and conservation. As well as the extent of
the ability of the project to help improve local environmental management
and coastal planning capabilities of local authorities; improving risk
communication between local authorities and the private sector; and how
successful was UNEP’s APELL process towards achieving these outcomes.

34
As far as possible, also assess the potential longer-term impacts considering
that the evaluation is taking place upon completion of the project and that
longer term impact is expected to be seen in a few years time. Frame
However, the views and comments expressed by the evaluator need not be restricted to these items.
Page 42 of 58
recommendations to enhance future project impact in this context. Which
will be the major ‘channels’ for longer term impact from this project at the
national and international scales?
2. Relevance: In retrospect, were the project’s outcomes consistent with the operational
program strategies and country priorities? Ascertain the nature and significance of the
contribution of the project outcomes to sustainable consumption and production and
disaster management sub-programme elements; and the wider portfolio of UNEP.
3. Efficiency: Was the project cost effective? Was the project the least cost option? Was
the project implementation delayed and if it was, did that affect cost-effectiveness?
Assess the contribution of cash and in-kind co-financing to project implementation
and to what extent the project leveraged additional resources. Did the project build on
earlier initiatives? Did it make effective use of available scientific and / or technical
information? Wherever possible, the evaluator should also compare the cost-time vs.
outcomes relationship of the project with that of other similar projects.
B. Assessment of Sustainability of project outcomes:
Sustainability is understood as the probability of continued long-term project-derived
outcomes and impacts after the UNEP project funding ends. The evaluation will identify
and assess the key conditions or factors that are likely to contribute or undermine the
persistence of benefits after the project ends. Some of these factors might be outcomes
of the project, e.g. stronger institutional capacities or better informed decision-making.
Other factors will include contextual circumstances or developments that are not
outcomes of the project but that are relevant to the sustainability of outcomes. The
evaluation should ascertain to what extent follow-up work has been initiated and how
project outcomes will be sustained and enhanced over time.
Four aspects of sustainability should be addressed: financial, socio-political, institutional
frameworks and governance, and ecological (if applicable). The following questions
provide guidance on the assessment of these aspects:
 Financial resources. To what extent are the outcomes of the project dependent on
continued financial support? What is the likelihood that any required financial
resources will be available to sustain the project outcomes/benefits once the UNEP
assistance ends (resources can be from multiple sources, such as the public and
private sectors, income generating activities, and market trends that support the
project’s objectives)? Was the project successful in identifying and leveraging cofinancing?
 Socio-political: To what extent are the outcomes of the project dependent on sociopolitical factors? What is the likelihood that the level of stakeholder ownership will
allow for the project outcomes/benefits to be sustained? Is there sufficient public /
stakeholder awareness in support of the long term objectives of the project?
 Institutional framework and governance. To what extent are the outcomes of the
project dependent on issues relating to institutional frameworks and governance?
What is the likelihood that institutional and technical achievements, legal
frameworks, policies and governance structures and processes will allow for, the
project outcomes/benefits to be sustained? While responding to these questions
consider if the required systems for accountability and transparency and the
required technical know-how are in place.
 Environmental. The analysis of ecological sustainability may prove challenging.
What is the likelihood that project achievements will lead to sustained ecological
Page 43 of 58
benefits? Are there any environmental risks that can undermine the future flow of
the project environmental benefits? The Terminal Evaluation should assess whether
certain activities in the project area will pose a threat to the sustainability of the
project outcomes. For example: construction of a dam in a protected area could
inundate a sizeable area and thereby neutralize the biodiversity –related gains made
by the project.
C. Achievement of outputs and activities:
Assessment of the project’s success in producing each of the programmed outputs, both
in quantity and quality as well as usefulness and timeliness. In particular, assess the
ability of the project to carry out all activities undertaken by this partnership.
D. Catalytic role:
The terminal evaluation will also describe any catalytic or replication effect of the
project. What examples are there of replication and catalytic outcomes that suggest
increased likelihood of sustainability? Replication approach, in the context of UNEP
projects, is defined as lessons and experiences coming out of the project that are
replicated or scaled up in the design and implementation of other projects. Replication
can have two aspects, replication proper (lessons and experiences are replicated in
different geographic area) or scaling up (lessons and experiences are replicated within
the same geographic area but funded by other sources). Specifically:

Is there demonstrable interest from the destination and the tourism associations
to make use of project results as a marketing strategy? Is there any evidence
the application of this proposed project approach to other disaster-exposed
tourism destinations in Asia?
If no effects are identified, the evaluation will describe the catalytic or replication
actions that the project carried out.
E. Assessment of Monitoring and Evaluation Systems:
The evaluation shall include an assessment of the quality, application and effectiveness
of project monitoring and evaluation plans and tools, including an assessment of risk
management based on the assumptions and risks identified in the project document. The
Terminal Evaluation will assess whether the project met the minimum requirements for
‘project design of M&E’ and ‘the application of the Project M&E plan’. UNEP projects
must budget adequately for execution of the M&E plan, and provide adequate resources
during implementation of the M&E plan. Project managers are also expected to use the
information generated by the M&E system during project implementation to adapt and
improve the project.

M&E design. Projects should have sound M&E plans to monitor results and
track progress towards achieving project objectives. An M&E plan should
include a baseline (including data, methodology, etc.), SMART indicators (see
Annex 4) and data analysis systems, and evaluation studies at specific times to
assess results. The time frame for various M&E activities and standards for
outputs should have been specified

M&E plan implementation. A Terminal Evaluation should verify that: an
M&E system was in place and facilitated timely tracking of results and progress
towards projects objectives throughout the project implementation period
Page 44 of 58
(perhaps through use of a logframe or similar); annual project reports were
complete, accurate and with well justified ratings; that the information provided
by the M&E system was used during the project to improve project performance
and to adapt to changing needs; and that projects had an M&E system in place
with proper training for parties responsible for M&E activities.
F.

Budgeting and Funding for M&E activities. The terminal evaluation should
determine whether support for M&E was budgeted adequately and was funded in
a timely fashion during implementation.

Long-term Monitoring. Is long-term monitoring envisaged as an outcome of
the project? If so, comment specifically on the relevance of such monitoring
systems to sustaining project outcomes and how the monitoring effort will be
sustained.
Preparation and Readiness
Were the project’s objectives and components clear, practicable and feasible within its
timeframe? Were the capacities of executing institution and counterparts properly
considered when the project was designed? Were lessons from other relevant projects
properly incorporated in the project design? Were the partnership arrangements
properly identified and the roles and responsibilities negotiated prior to project
implementation? Were counterpart resources (funding, staff, and facilities), enabling
legislation, and adequate project management arrangements in place?
G. Country ownership
This is the relevance of the project to national development and environmental
agendas, recipient country commitment, and regional and international agreements. The
evaluation will:
 Assess the level of country ownership. Specifically, the evaluator should assess
whether the project was effective in catalyzing action taken by the authorities in
the country that received assistance from the project.
 Assess the level of country commitment to achieving a substantial reduction of
greenhouse gasses emissions
H. Stakeholder participation / public awareness
Stakeholders are the individuals, groups, institutions or other bodies that have an
interest or stake in the outcome of the UNEP financed project. The term also applies to
those potentially adversely affected by a project. The evaluator will specifically assess
if the project involved the relevant stakeholders through information sharing,
consultation and by seeking their participation in project’s design, implementation, and
monitoring and evaluation. For example, did the project implement appropriate
outreach and public awareness campaigns? Did the project consult and make use of the
skills, experience and knowledge of the appropriate government entities, NGOs,
community groups, private sector, local governments and academic institutions in the
design, implementation and evaluation of project activities? Were perspectives of those
that would be affected by decisions, those that could affect the outcomes and those that
could contribute information or other resources to the process taken into account while
taking decisions? Were the relevant vulnerable groups and the powerful, the supporters
and the opponents, of the processes properly involved? Specifically the evaluation will:

Assess the mechanisms put in place by the project for identification and
engagement of stakeholders in each participating country and establish, in
Page 45 of 58


consultation with the stakeholders, whether this mechanism was successful, and
identify its strengths and weaknesses.
Assess the degree and effectiveness of collaboration/interactions between the
various project partners and institutions during the course of implementation of
the project.
Assess the degree and effectiveness of any various public awareness activities
that were undertaken during the course of implementation of the project.
I. Financial Planning
Evaluation of financial planning requires assessment of the quality and effectiveness of
financial planning and control of financial resources throughout the project’s lifetime.
Evaluation includes actual project costs by activities compared to budget (variances),
financial management (including disbursement issues), and co- financing. The
evaluation should:
 Assess the strength and utility of financial controls, including reporting,
and planning to allow the project management to make informed decisions
regarding the budget and allow for a proper and timely flow of funds for
the payment of satisfactory project deliverables.
 Present the major findings from the financial audit if one has been
conducted.
 Identify and verify the sources of co- financing as well as leveraged and
associated financing
 Assess whether the project has applied appropriate standards of due
diligence in the management of funds and financial audits.
 The evaluation should also include a breakdown of final actual costs and
co-financing for the project prepared in consultation with the relevant
UNEP Fund Management Officer of the project
J. Implementation approach
This includes an analysis of the project’s management framework, adaptation to
changing conditions (adaptive management), partnerships in implementation
arrangements especially those with the executing agencies, changes in project design,
and overall project management. The evaluation will:
 Ascertain to what extent the project implementation mechanisms outlined
in the project document have been closely followed. In particular, assess
the role of the various committees established and whether the project
document was clear and realistic to enable effective and efficient
implementation, whether the project was executed according to the plan
and how well the management was able to adapt to changes during the life
of the project to enable the implementation of the project.
 Evaluate the effectiveness and efficiency and adaptability of project
management and the supervision of project activities / project execution
arrangements at all levels (1) policy decisions: Steering Group; (2) day to
day project management in each of the country executing agencies and
UNEP
K. UNEP Supervision and Backstopping
 Assess the effectiveness of supervision and administrative and financial support
provided by UNEP/DTIE
 Identify administrative, operational and/or technical problems and constraints that
influenced the effective implementation of the project.
Page 46 of 58
The ratings will be presented in the form of a table. Each of the eleven categories
should be rated separately with brief justifications based on the findings of the main
analysis. An overall rating for the project should also be given. The following rating
system is to be applied:
HS
S
MS
MU
U
HU
= Highly Satisfactory
= Satisfactory
= Moderately Satisfactory
= Moderately Unsatisfactory
= Unsatisfactory
= Highly Unsatisfactory
2.4. Evaluation report format and review procedures
The report should be brief, to the point and easy to understand. It must explain; the purpose of
the evaluation, exactly what was evaluated and the methods used. The report must highlight
any methodological limitations, identify key concerns and present evidence-based findings,
consequent conclusions, recommendations and lessons. The report should provide information
on when the evaluation took place, the places visited, who was involved and be presented in a
way that makes the information accessible and comprehensible. The report should include an
executive summary that encapsulates the essence of the information contained in the report to
facilitate dissemination and distillation of lessons.
The evaluation will rate the overall implementation success of the project and provide
individual ratings of the eleven implementation aspects as described in section 3 of this TOR.
The ratings will be presented in the format of a table with brief justifications based on the
findings of the main analysis.
Evidence, findings, conclusions and recommendations should be presented in a complete and
balanced manner. The evaluation report shall be written in English, be of no more than 50
pages (excluding annexes), use numbered paragraphs and include:
i)
ii)
iii)
iv)
v)
vi)
An executive summary (no more than 3 pages) providing a brief overview of
the main conclusions and recommendations of the evaluation;
Introduction and background giving a brief overview of the evaluated
project, for example, the objective and status of activities;
Scope, objective and methods presenting the evaluation’s purpose, the
evaluation criteria used and questions to be addressed;
Project Performance and Impact providing factual evidence relevant to the
questions asked by the evaluator and interpretations of such evidence. This is
the main substantive section of the report and should provide a commentary on
all evaluation aspects (A − F above).
Conclusions and rating of project implementation success giving the
evaluator’s concluding assessments and ratings of the project against given
evaluation criteria and standards of performance. The conclusions should
provide answers to questions about whether the project is considered good or
bad, and whether the results are considered positive or negative;
Lessons learned presenting general conclusions, based on established good
practices that have the potential for wider application and use. Lessons may
Page 47 of 58
vii)
also be derived from problems and mistakes. The context in which lessons
may be applied should be clearly specified, and lessons should always state or
imply some prescriptive action. A lesson should be written such that
experiences derived from the project could be applied in other projects or at
portfolio level;
Recommendations suggesting actionable proposals for improvement of the
current project. In general, Terminal Evaluations are likely to have very few
(perhaps two or three) actionable recommendations.
Prior to each recommendation, the issue(s) or problem(s) to be addressed by
the recommendation should be clearly stated.
A high quality recommendation is an actionable proposal that is:
1. Feasible to implement within the timeframe and resources available
2. Commensurate with the available capacities of project team and
partners
3. Specific in terms of who would do what and when
4. Contains results-based language (i.e. a measurable performance
target)
5. Includes a trade-off analysis, when its implementation may require
utilizing significant resources that would otherwise be used for other
project purposes.
viii)
Annexes include Terms of Reference, list of interviewees, documents
reviewed, brief summary of the expertise of the evaluator/evaluation team, a
summary of co-finance information etc. Dissident views or management
responses to the evaluation findings may later be appended in an annex.
Examples of UNEP Terminal Evaluation Reports are available at www.unep.org/eou
Review of the Draft Evaluation Report
Draft reports submitted to UNEP EOU are shared with the corresponding Programme or
Project Officer and his or her supervisor for initial review and consultation. They may
provide feedback on any errors of fact and may highlight the significance of such errors in
any conclusions.
The consultation also seeks agreement on the findings and
recommendations. UNEP EOU collates and review comments and provides them to the
evaluators for their consideration in preparing the final version of the report.
2.4.Submission of Final Terminal Evaluation Reports.
The final report shall be submitted in electronic form in MS Word format and should be sent
to the following persons:
Segbedzi Norgbey, Chief, Evaluation and Oversight Unit
UNEP, P.O. Box 30552-00100
Nairobi, Kenya
Tel.: (254-20) 7623387
Fax: (254-20) 7623158
Email: segbedzi.norgbey@unep.org
Page 48 of 58
With a copy to:
Ruth Zugman Do Coutto, Programme Officer
Safer Production
Business and Industry Unit, Sustainable Consumption and Production Branch
UNEP/DTIE
France
Tel.: +33 1 44 37 16 34
Fax: +33 1 44 37 14 74
E-mail: ruth.coutto@unep.fr
Arab Hoballah, Head of Sustainable Production & Consumption Branch
UNEP/DTIE
Paris
Tel: 33 1 44 37 14 27
Fax: 33 1 44 37 14 74
Email: arab.hoballah@unep.fr
The final evaluation report will be printed in hard copy and published on the Evaluation and
Oversight Unit’s web-site www.unep.org/eou. Subsequently, the report will be sent to DTIE
for review. In addition the final evaluation report will disseminated to: The relevant DTIE
Focal points, Relevant Government representatives, UNEP DTIE Professional Staff, The
project’s Executing Agency and Technical Staff.
2.6. Resources and schedule of the evaluation
This terminal evaluation will be undertaken by an international evaluator contracted by the
Evaluation and Oversight Unit, UNEP. The contract for the evaluator will begin on 8th
December 2008 and end on 7th February 2009 (1 month spread over 2 months). After an
initial telephone briefing with EOU and UNEP/DTIE, the evaluator will travel to Phuket,
Bangkok and Mumbai (India) (4 days of travel and 26 days desk study). The evaluator will
submit a draft report no later than 15th January 2009 to UNEP/EOU. Any comments or
responses to the draft report will be sent to UNEP/EOU for collation and the consultant will
be advised of any necessary revisions. Comments to the final draft report will be sent to the
consultant by 30th January 2009 after which, the consultant will submit the final report no
later than 7th February 2008.
The evaluator should not have been associated with the design and implementation of the
project. The evaluator will work under the overall supervision of the Chief, Evaluation and
Oversight Unit, UNEP. The evaluator should be an international expert in disaster
preparedness and management. The consultant should have the following minimum
qualifications: (i) experience in sustainable production and consumption specifically within
the tourism sector; (ii) experience with management and implementation of development
projects in developing countries; (iii) experience with project evaluation. Knowledge of
UNEP programmes is desirable. Fluency in oral and written English is a must.
Page 49 of 58
2.7. Schedule Of Payment
Lump-Sum Option
The evaluator will receive an initial payment of 30% of the total amount due upon signature
of the contract. A further 30% will be paid upon submission of the draft report. A final
payment of 40% will be made upon satisfactory completion of work. The fee is payable under
the individual Special Service Agreement (SSA) of the evaluator and is inclusive of all
expenses such as travel, accommodation and incidental expenses.
In case, the evaluator cannot provide the products in accordance with the TORs, the
timeframe agreed, or his products are substandard, the payment to the evaluator could be
withheld, until such a time the products are modified to meet UNEP's standard. In case the
evaluator fails to submit a satisfactory final product to UNEP, the product prepared by the
evaluator may not constitute the evaluation report.
Annex 1. OVERALL RATINGS TABLE
Criterion
Evaluator’s Summary Comments
Evaluator’
s Rating
A. Attainment of project
objectives and results (overall
rating)
Sub criteria (below)
A. 1. Effectiveness
A. 2. Relevance
A. 3. Efficiency
B. Sustainability of Project
outcomes
(overall rating)
Sub criteria (below)
B. 1. Financial
B. 2. Socio Political
B. 3. Institutional framework
and governance
B. 4. Environmental
C. Achievement of outputs and
activities
D. Monitoring and Evaluation
(overall rating)
Sub criteria (below)
D. 1. M&E Design
D. 2. M&E Plan
Implementation (use for
adaptive management)
D. 3. Budgeting and Funding
for M&E activities
E. Catalytic Role
F. Preparation and readiness
G. Country ownership /
drivenness
Page 50 of 58
Criterion
Evaluator’s Summary Comments
Evaluator’
s Rating
H. Stakeholders involvement
I. Financial planning
J. Implementation approach
K. UNEP Supervision and
backstopping
RATING OF PROJECT OBJECTIVES AND RESULTS
Highly Satisfactory (HS): The project had no shortcomings in the achievement of its
objectives, in terms of relevance, effectiveness or efficiency.
Satisfactory (S): The project had minor shortcomings in the achievement of its
objectives, in terms of relevance, effectiveness or efficiency.
Moderately Satisfactory (MS): The project had moderate shortcomings in the
achievement of its objectives, in terms of relevance, effectiveness or efficiency.
Moderately Unsatisfactory (MU): The project had significant shortcomings in the
achievement of its objectives, in terms of relevance, effectiveness or efficiency.
Unsatisfactory (U) The project had major shortcomings in the achievement of its
objectives, in terms of relevance, effectiveness or efficiency.
Highly Unsatisfactory (HU): The project had severe shortcomings in the achievement
of its objectives, in terms of relevance, effectiveness or efficiency.
Please note: Relevance and effectiveness will be considered as critical criteria. The overall
rating of the project for achievement of objectives and results may not be higher than the
lowest rating on either of these two criteria. Thus, to have an overall satisfactory rating for
outcomes a project must have at least satisfactory ratings on both relevance and effectiveness.
Ratings On Sustainability
A. Sustainability will be understood as the probability of continued long-term outcomes and
impacts after the GEF project funding ends. The Terminal evaluation will identify and
assess the key conditions or factors that are likely to contribute or undermine the
persistence of benefits after the project ends. Some of these factors might be outcomes of
the project, i.e. stronger institutional capacities, legal frameworks, socio-economic
incentives /or public awareness. Other factors will include contextual circumstances or
developments that are not outcomes of the project but that are relevant to the sustainability
of outcomes..
Rating system for sustainability sub-criteria
On each of the dimensions of sustainability of the project outcomes will be rated as follows.
Likely (L): There are no risks affecting this dimension of sustainability.
Moderately Likely (ML). There are moderate risks that affect this dimension of
sustainability.
Page 51 of 58
Moderately Unlikely (MU): There are significant risks that affect this dimension of
sustainability
Unlikely (U): There are severe risks that affect this dimension of sustainability.
According to the GEF Office of Evaluation, all the risk dimensions of sustainability are
deemed critical. Therefore, overall rating for sustainability will not be higher than the rating
of the dimension with lowest ratings. For example, if a project has an Unlikely rating in any
of the dimensions then its overall rating cannot be higher than Unlikely, regardless of whether
higher ratings in other dimensions of sustainability produce a higher average.
Ratings of Project M&E
Monitoring is a continuing function that uses systematic collection of data on specified
indicators to provide management and the main stakeholders of an ongoing project with
indications of the extent of progress and achievement of objectives and progress in the use of
allocated funds. Evaluation is the systematic and objective assessment of an on-going or
completed project, its design, implementation and results. Project evaluation may involve the
definition of appropriate standards, the examination of performance against those standards,
and an assessment of actual and expected results.
The Project monitoring and evaluation system will be rated on ‘M&E Design’, ‘M&E Plan
Implementation’ and ‘Budgeting and Funding for M&E activities’ as follows:
-
-
Highly Satisfactory (HS): There were no shortcomings in the project M&E system.
Satisfactory(S): There were minor shortcomings in the project M&E system.
Moderately Satisfactory (MS): There were moderate shortcomings in the project M&E
system. Moderately Unsatisfactory (MU): There were significant shortcomings in the
project M&E system. Unsatisfactory (U): There were major shortcomings in the
project M&E system.
Highly Unsatisfactory (HU): The Project had no M&E system.
“M&E plan implementation” will be considered a critical parameter for the overall
assessment of the M&E system. The overall rating for the M&E systems will not be higher
than the rating on “M&E plan implementation.”
All other ratings will be on the GEF six point scale.
GEF Performance Description
Alternative description on
the same scale
HS
= Highly Satisfactory
Excellent
S
= Satisfactory
Well above average
MS
= Moderately Satisfactory
Average
MU
= Moderately Unsatisfactory
Below Average
U
= Unsatisfactory
Poor
HU
= Highly Unsatisfactory
Very poor (Appalling)
Page 52 of 58
Annex 2. CO-FINANCING AND LEVERAGED RESOURCES
Co-financing (basic data to be supplied to the consultant for verification)
Co financing
(Type/Source)






IA own
Financing
(mill US$)
Planned Actual
Government
(mill US$)
Planned
Actual
Other*
Total
(mill US$)
Planned
Actual
(mill US$)
Planned Actual
Total
Disbursement
(mill US$)
Planned
Actual
Grants
Loans/Concessiona
l (compared to
market rate)
Credits
Equity investments
In-kind support
Other (*)
Totals
* Other is referred to contributions mobilized for the project from other multilateral agencies, bilateral development cooperation
agencies, NGOs, the private sector and beneficiaries.
Leveraged Resources
Leveraged resources are additional resources—beyond those committed to the project itself at the time of approval—that are mobilized
later as a direct result of the project. Leveraged resources can be financial or in-kind and they may be from other donors, NGO’s,
foundations, governments, communities or the private sector. Please briefly describe the resources the project has leveraged since
inception and indicate how these resources are contributing to the project’s ultimate objective.
Page 53 of 58
Annex 3:
Review of the Draft Report
Draft reports submitted to UNEP EOU are shared with the corresponding Programme or
Project Officer and his or her supervisor for initial review and consultation. DTIE staff
and senior Executing Agency staff provide comments on the draft evaluation report.
They may provide feedback on any errors of fact and may highlight the significance of
such errors in any conclusions. The consultation also seeks agreement on the findings
and recommendations. UNEP EOU collates the review comments and provides them to
the evaluators for their consideration in preparing the final version of the report.
General comments on the draft report with respect to compliance with these TOR are
shared with the reviewer.
Quality Assessment of the Evaluation Report
All UNEP Evaluation Reports are subject to quality assessments by UNEP EOU. These
assessment and are used as a tool for providing structured feedback to the evaluator.
The quality of the draft evaluation report is assessed and rated against the following
criteria:
Report Quality Criteria
UNEP EOU
Rating
Assessment
A. Did the report present an assessment of relevant
outcomes and achievement of project objectives in the
context of the focal area program indicators if
applicable?
B. Was the report consistent and the evidence complete
and convincing and were the ratings substantiated when
used?
C. Did the report present a sound assessment of
sustainability of outcomes?
D. Were the lessons and recommendations supported
by the evidence presented?
E. Did the report include the actual project costs (total
and per activity) and actual co-financing used?
F. Did the report include an assessment of the quality of
the project M&E system and its use for project
management?
UNEP EOU additional Report Quality Criteria
UNEP EOU
Rating
Assessment
G. Quality of the lessons: Were lessons readily
applicable in other contexts? Did they suggest
prescriptive action?
H. Quality of the recommendations: Did
recommendations specify the actions necessary to
correct existing conditions or improve operations
(‘who?’ ‘what?’ ‘where?’ ‘when?)’. Can they be
implemented? Did the recommendations specify a goal
and an associated performance indicator?
I. Was the report well written?
(clear English language and grammar)
J. Did the report structure follow EOU guidelines, were
Page 54 of 58
all requested Annexes included?
K. Were all evaluation aspects specified in the TORs
adequately addressed?
L. Was the report delivered in a timely manner
Page 55 of 58
GEF Quality of the MTE report = 0.3*(A + B) +
0.1*(C+D+E+F)
EOU assessment of MTE report = 0.3*(G + H) +
0.1*(I+J+K+L)
Combined quality Rating = (2* ‘GEF EO’ rating + EOU
rating)/3
The Totals are rounded and converted to the scale of HS to HU
Rating system for quality of terminal evaluation reports
A number rating 1-6 is used for each criterion: Highly Satisfactory = 6, Satisfactory =
5, Moderately Satisfactory = 4, Moderately Unsatisfactory = 3, Unsatisfactory = 2,
Highly Unsatisfactory = 1, and unable to assess = 0.
Page 56 of 58
Annex 5:
Contact list for all Project Main stakeholders
Organization
UNEP/DTIE
(Paris)
UNEP/DTIE
(Paris)
UNEP/DTIE
(Paris)
UNEP
(Nairobi)
UNEP
(Nairobi)
Swedish Rescue
Services Agency
(SRSA)
International Hotel and
Restaurant
Association (IH&RA)
European Commission
(Donor)
Name &
responsibility
Phone number
E-mail address
Arab Hoballah
Chief, SCP Branch
+33 1 44 37 14 39
Arab.hoballah@unep.fr
Ruth Coutto
Project manager
Stefanos Fotiou
Co-project manager
Leo Heileman
Project Supervisor
Gregory Patilis
Fund Management
Officer
Lucy Halogo
+33.1.44 37 16 34
Ruth.coutto@unep.fr
+33.1.44 37 14 65
Stefanos.Fotiou@unep.fr
+33.1.44 37 14 77
Leo.Heileman@unep.fr
Rainar All
APELL educator
Janet Edwards
International
Development
Cooperation
Lars Hillstrom
Project manager
(SRSA)
Babro NaslundLandenmark
Ecologist – Natural
Disasters
Technical Expert
Mette-Lindhal
Technical Expert
Abraham Rosenthal
Director General
Sutthiya
Chantawarangul
Programme officer
Lucy.halogo@unep.org
rainar.all@srv.se
Janet.edwards@srv.se
46 54 13 50 00
lars.hillerstrom@srv.se
+46 54 13 50 50
Barbro-naslundlandenmark@srv.se
+41 (0)22 734 80 41
+66 2 305 2600
Sutthiya.chantawarangul@ec.
europa.eu
Thailand Beneficiaries
Department of
Disaster Prevention
and Mitigation (DDPM)
Department of
Disaster Prevention
and Mitigation (DDPM)
Department of
Disaster Prevention
and Mitigation (DDPM)
Patong (PMP)
Chatchadaporn
Boonyavaha
chatcha_b@hotmail.com
M. Luckana
mluckana@yahoo.com
Kanokporn Chucherd
Foreign Relations Officer
Srisuda Rakphao
Project Manager
+66 2 2430020 ext.
2415
Mobile: +66 89
9241928
+66 8 1829 6381
asparagus108@yahoo.com
srakphao@gmail.com
Page 57 of 58
Phi-Phi Island
(ANSAO)
India Beneficiaries
National Safety
Council of India
(NSCI)
National Safety
Council of India
(NSCI(
Po Or Wanchai
Project Manager
wanchaich@yahoo.com
K. C. Gupta
Director General
+91 22 2757 9924
nsci@giasbm01.vsnl.net.in
Atamaram Sundkar
Project Manager
+91 22-27579924
+91 (0)
9869113270
aysundkar@gmail.com
Page 58 of 58