Evaluation of the International Environmental Technology Centre of the United Nations Environment Programme Project Design and Evaluation Unit Corporate Planning and Accountability Service Prepared by: Ferd J.M. Schelleman, M.Sc. Nairobi, December 1998 K9935037 190899 TABLE OF CONTENTS SUMMARY I. II. BACKGROUND OF IETC 12 A. Objectives 13 B. IETC activities 14 C. Organization of IETC 15 D. Financing of IETC 19 E. Cooperation with GEC and ILEC 19 F. The IETC International Advisory Board 21 G. Main observations 21 EVALUATION ACTIVITIES A. 22 Adopting/applying and operating environmentally sound technologies for urban and lake reservoir management 23 Improving access to information on environmentally sound technologies 25 C. Quarterly IETC newsletter: 29 D. IETC support to Habitat II 32 E. Basic research guidelines to best practices in implementing urban sewage programmes 34 Environmental technology assessment (Enta) for decision makers: a leadership training programme 35 IETC assistance to the Sustainable Cities Programme of UNCHS (Habitat) 37 International Source Book on Environmentally Sound Technologies for Municipal Solid Waste Management: follow-up projects 40 Source Book on Alternative Technologies for Freshwater Augmentation: Follow-up project 42 B. F. G. H. I. III. 5 IETC'S Insight CONCLUSIONS 44 A. General conclusions 44 B. Project development 46 C. SWOT Analysis 47 2 IV. RECOMMENDATIONS 48 Annexes I. List of interviewees and persons contacted 49 II. List of reviewed documents 50 III. Terms of reference for the evaluation of the UNEP/IETC umbrella project (JC/3220-96-07) 52 3 ABBREVIATIONS ARCT EnRA EnTA ESTs GEC GTZ IAB ICLEI ICPIC IETC ILEC INTAN SCP START SWOT TIPS TRS UNCHS UNEP UNIDO African Regional Centre for Technology environmental risk assessment environmental technology assessment environmentally sound technologies Global Environment Centre German Agency for Technical Cooperation International Advisory Board (of IETC) International Council for Local Environmental Initiatives International Cleaner Production Information Clearing-house International Environmental Technology Centre International Lake Environment Committee Foundation National Institute for Public Administration Sustainable Cities Programme System for Analysis and Research Training "strength, weaknesses, opportunities and threats" Technology Information and Promotion Service CHECK Nations Centre for Human Settlements United Nations Environment Programme United Nations Industrial Development Organization 4 SUMMARY A. Introduction 1. The International Environmental Technology Centre of the United Nations Environment Programme (UNEP/IETC) started its activities in 1994 after a series of earlier discussions and implementation meetings. The IETC inauguration meeting took place on 30 October 1992. The Government of Japan then signed an agreement with UNEP to set up the International Environmental technology Centre. Close collaboration with other UNEP units, such as the Industry and Environment Centre and the Global Environmental Monitoring System (GEMS) was also sought. The budget of IETC would come from two sources: (a) The programme budget of the UNEP Environment Fund; (b) Voluntary contributions made to a technical cooperation trust fund. 2. IETC became operational in April 1994 and operates from two offices, one in Shiga, housed and supported by the International Lake Environmental Council (ILEC) and one in Osaka, housed and supported by the Global Environment Centre (GEC). Through these Japanese supporting organizations, the prefectures of Shiga and Osaka contribute to and support IETC. 3. The main objective of the Centre is to contribute to the transfer of environmentally sound technologies to developing countries and countries with economies in transition. The work of IETC work is organized around three sub-issues: (a) The improvement of access to information on environmentally sound technologies, through, for example, the maESTro database on such technologies and the Internet homepage of IETC; (b) The fostering of technology cooperation, partnerships and technology transfer, through, for example, publications, reports, and maESTro; (c) Capacity-building through the organization of workshops and training courses. B. IETC projects 4. In the IETC project action sheet, 10 activities are mentioned. Eight of these activities were developed and implemented in the period from 1 April 1996 to December 1997. These activities have been evaluated in the framework of this study. A summary of the results of these project evaluations is presented in the following table. 5 Table 1 Evaluation and scoring of IETC projects Criteria: a/ 1 2 3 4 5 6 1 b/ 2 2 1 2 2 2. Improving access to information on environmentally sound technologies 1 2 2 3 2 2 3. Quarterly IETC newsletter: 2 2 2 3 2 2 4. IETC support to Habitat II 2 2 2 1 2 N/A 5. Environmental Technology Assessment for decision-makers: a leadership training programme 2 3 2 2 3 2 6. IETC assistance to the Sustainable Cities Programme (SCP) of UNCHS (Habitat) 3 2 3 3 2 3 7. International source-book on environmentally sound technologies for municipal solid waste management 2 2 2 3 3 2 8. Source-book on alternative technologies for freshwater augmentation 3 2 2 2 2 2 Project: 1. Pilot Training programme/workshop on adopting, applying and operating environmentally sound technologies for urban and lake/reservoir management Insight a/ Criteria applied: 1 - timeliness; 2 - attainment; 3 - completion; 4 budget; 5 - impact; 6 - sustainability. b/ Explanation of scores: 1 - excellent; 2 - very good; 3 - good; 4 satisfactory; 5 - unsatisfactory. 5. In general, IETC performed very well in developing and implementing its projects. Almost all projects resulted in good final products and outputs that can be used for follow-up activities. The review of IETC activities resulted in the following conclusions: (a) Between 1 April 1996 and December 1997, IETC has developed and implemented eight internal projects. Taking the restricted number of personnel into account this is considered to be a successful accomplishment. IETC has thus certainly been effective; (b) IETC successfully developed and implemented a series of projects. These activities have resulted in products and tools that deserve maintenance and updating and that will support future activities of the Centre. These products and tools include: 6 (i) An internet homepage with access to the maESTro database, which is of good quality already serving an increasing number of users; (ii) The IETC Insight newsletter published on a regular (two to three times a year) basis with a professional outlook and providing adequate information; (iii) A number of information materials (sourcebooks, workbooks, manuals etc.) that will support the development of future IETC projects; (c) IETC is too small an office. Although the total number of staff is approximately 20, IETC currently only has two programme officers responsible for project development and implementation (not counting the Director, responsible for overall management, the Liaison Officer and the Administration/Fund Management Officer), two consultants and four support staff provided by the GEC and ILEC. Despite the small size, IETC has two offices, one in Osaka and one in Shiga. Although the housing of one branch of IETC at the premises of ILEC is logical because of IETC freshwater activities and the support provided by ILEC in this field, this split hampers effective communication between IETC staff; (d) IETC is almost fully (90 per cent) financed by the Ministry of Foreign Affairs of Japan. A major contribution from that source is logical because it was the Government of Japan that initiated IETC. Nevertheless, this dependence on one source of funding poses financial risks to IETC. Most other UNEP offices receive support (financial or in-kind in the form of seconded staff) from other countries. The funding of IETC for the coming year is again secured, but the Centre should develop activities with support from other sources and increase networking activities to ensure future financial support from other Governments and organizations to broaden its financial base; (e) The number of Professional staff responsible for the development and implementation of projects and network maintenance at IETC is almost critical. The current vacancy level at the Centre means that the number of programme officers has been reduced to an absolute minimum. This already leads to insufficient networking and has led to the abandonment of one project (annex 6 of the IETC project action sheet: Basic research to best practices in implementing urban sewage programmes). The number of Professional staff should be increased on short notice by filling the vacancies; (f) The IETC mission is to strengthen the capabilities of developing countries and countries with economies in transition, to manage large cities (urban environmental management) and freshwater lake/reservoir basins by promoting international transfer of environmentally sound technologies. The current image of IETC however focuses on the collection and dissemination of information on environmentally sound technologies (ESTs). A more central focus on urban and freshwater environmental management could be more attractive for most Governments and funding organizations. Technology information dissemination would then play an important but secondary role; (g) A clear vision of the IETC position and central focus is missing. 7 IETC has successfully developed and implemented a number of activities but these seem to be stand-alone projects with limited or no feedback or cross-fertilization; (h) Networking is a very important activity for a centre like IETC. In particular, the Director (with regard to international contacts, the United Nations, UNEP, etc.) and the Senior Liaison Officer (with regard to Japan) have to develop and maintain effective relationships with collaborating and supporting agencies and organizations. The current networking activities of IETC are at a relatively low level because of insufficient quality and quantity of personnel; (i) GEC and ILEC provide Professional staff to IETC in both offices. Staff provided by GEC and ILEC do not have sufficient experience in the environmental field, in the development and implementation of international projects and in the English language. The selection of GEC and ILEC staff provided to IETC should be done by the Director of IETC and be based on the aforementioned criteria; (j) The products and materials developed in IETC now need to be developed and incorporated into an integrated urban-environmental-management methodology and/or training programme; (k) All projects have been developed with relevant and adequate international co-operation as well with other UNEP units, such as the former UNEP Industry and Environment Centre, other United Nations organizations, such as UNCHS (Habitat) and the Sustainable Cities Programme, as well as with international expert organizations. 8 6. The above conclusions have been summarized in a "strength, weaknesses, opportunities and threats" (SWOT) analysis: STRENGTHS WEAKNESSES Good and effective cooperation with the United Nations, UNEP and other international organizations Insufficient Professional staff for the development and implementation of projects Excessive orientation on environmental technologies which is receiving less attention as a primary topic Number of effective tools available: Internet site with maESTro and EST database, newsletter and manuals and sourcebooks IETC is considered as a "Japanese" organization, which makes it difficult to receive support from other countries Cooperation with and continuous support from Japanese organisations GEC and ILEC No established network with major countries and agencies having a direct interest in IETC activities Continuous financial support from the Japanese Government providing a baseline for project development Insufficient follow-up on projects because of personnel shortages leading to underdeveloped networking GEC and ILEC support in providing staff does not match the requirements of the Centre OPPORTUNITIES THREATS Urban environmental management and freshwater management are increasingly important Risk of reduction of the contribution from the Japanese Government due to budgetary problems on the Japanese overall budget Other organizations developing international activities in the field of urban and freshwater environmental management 9 C. 7. Recommendations The following recommendations are put forward: Recommendation 1: The IETC mission should be redirected towards urban and freshwater environmental management. Without paying less attention to collection and dissemination of information on environmentally sound technologies, IETC should reorient its primary objectives towards urban and freshwater environmental management. The main reasons for this reorientation are: (a) Increased interest in the activities of the Centre in countries, with Governments and agencies potentially interested in supporting the Centre's activities; (b) Such a reorientation would provide better opportunities to develop strategic partnerships with other organizations and programmes (the Sustainable Cities Programme (SCP), the International Council for Local Environmental Initiatives (ICLEI), etc.); (c) It would provide a clear image for other organizations. The primary task of the Centre would be to provide support (information supply and dissemination, training and capacity-building) to activities of other United Nations and international programmes and organizations (SCP, ICLEI, etc.); Recommendation 2: The IETC strategy should focus on the development and implementation of one general urban and freshwater environmental management training course. This training course can be developed on the basis of available materials and experiences at IETC (workshops and training courses on environmental technology assessment, municipal solid wastes and environmentally sound technologies), with additional inputs from other UNEP and United Nations divisions, for example, in the fields of environmentally sound urban transport and environmental monitoring and organization in urban environments. The development of this training course should be initiated after consultations with and in close collaboration with UNCHS (Habitat), the Sustainable Cities Programme, the UNEP Industry and Environment Centre, and a number of other United Nations and international organizations. Recommendation 3: The IETC activities should be focused on developing the above-mentioned training course. This requires the establishment of a closer network of UNEP, United Nations and other international organizations, and finding support (financial, in kind) from relevant Governments and international organizations in the field of environment and/or development cooperation; Recommendation 4: The IETC financial basis should be broadened. IETC should intensively seek for additional funding opportunities in view of the current dependence of one major funding source and the current critical number of Professional staff. This is closely linked to the above recommendations; Recommendation 5: The IETC organization will be strengthened through the reorganization of UNEP and increased cooperation with other UNEP offices. 10 The new UNEP Division of Trade, Industry and Economics brings together a number of UNEP offices, dealing with industry and environment and ozone/energy (Paris), trade and environment (Geneva), chemicals (Geneva) and IETC. This reorganization will facilitate networking and cooperation with, for example, the industry and environment, chemicals and ozone/energy programmes, and thus contribute to the search for additional sources of funding for IETC activities, as these programmes have been more effective in this respect; Recommendation 6: The IETC staff complement should be strengthened and enhanced. The current vacancies in the Centre should be filled on as short a notice as possible. The number of Professional staff currently is at a minimum level. The staffing problem could also be considerably alleviated if the personnel provided by GEC and ILEC had the required experience in the environmental field, in the development and implementation of international projects and in the English language. The Senior Liaison Officer should also be required to have a wide international and environmental experience in order to contribute directly to the Centre's objectives and activities. The success of networking activities and development of an integrated training course will depend on sufficient quantity and quality of staff; Recommendation 7: The IETC International Advisory Board (IAB) has to be organized again. The Board should meet under the supervision of the Director of the Centre and be composed of representatives from target countries and from agencies and organizations that may support the Centre's activities. The establishment of such an Advisory Board should be coordinated with the UNEP Industry and Environment Centre. The composition of the Advisory Board should be adapted to actual needs and requirements of IETC. The Advisory Board would thus have an ad hoc character. 11 I. 8. During in 1990, the the transfer to establish BACKGROUND OF IETC the second special session of the UNEP Governing Government of Japan emphasized the importance of of environmentally sound technologies (ESTs) and a UNEP Global Environmental Technology Centre in Council, held accelerating made a proposal Japan. 9. The International Environmental Technology Centre of the United Nations Environment Programme (UNEP/IETC) started its activities in 1994 after a series of earlier discussions and implementation meetings. The UNEP/IETC inauguration meeting took place on 30 October 1992. The Government of Japan then signed an agreement with UNEP to set up the International Environmental Technology Centre. Close collaboration with other UNEP units, such as the Industry and Environment Centre and the Global Environmental Monitoring System was expressed. The budget of IETC would come from two sources: (a) The programme budget of the UNEP Environment Fund; (b) Voluntary contributions made to a technical cooperation trust fund. 10. The Government of Japan had already made a contribution of US$ 2.3 million to this trust fund. The Centre would be manned by six UNEP Professional staff members and other support staff to be appointed by the Executive Director. 11. For the purpose of supporting IETC, the Global Environment Centre (GEC) was founded on January 1992. GEC provides staff and logistical support to IETC, collaborates on joint projects and serves as a liaison between IETC and related organizations in Japan. 12. The IETC formal opening ceremony took place in April 1994 at the newlybuilt Osaka office, two years after the inauguration session and two years after the establishment of GEC. IETC would work from two offices, one in Osaka, and one in Shiga. The IETC office in Shiga receives support from the International Lake Environmental Committee (ILEC). 13. Also in 1994, the first IETC newsletter, IETC's Insight, was published. 14. It was only at the beginning of 1996 that the project action sheet for the IETC umbrella project was formally approved, with 10 annexes describing the individual projects. 15. In March 1997, the current IETC Director, Mr. John Whitelaw, was appointed. 16. Although the formal start-up of IETC activities took place at the opening ceremony and was formalized with the approval of the project action sheet in the beginning of 1996, IETC activities had been operational since the official opening in April 1994. 12 17. IETC operates from two offices, one in Shiga housed and supported by ILEC (the International Lake Environmental Council) and one in Osaka, housed and supported by GEC (the Global Environment Centre). Through these Japanese supporting organizations the Prefectures of Shiga and Osaka and Osaka City Government contribute to and support IETC. A. Objectives 18. IETC was established to contribute to the transfer of environmentally sound technologies to developing countries and countries with economies in transition. Important observations made by the Governing Council concerning the character of the Centre and the scope of activities were, among others, that: (a) IETC should have a clear international status to be reflected by the technologies disseminated and the expertise available, the organizational structure, personnel, programmes and the composition of an international advisory board of members with various backgrounds, including the Industry and Environment Centre; (b) IETC should take care of coordination of its activities with activities of the UNEP Industry and Environment and with activities taking place under the auspices of the Global Environment Monitoring System (GEMS), or with activities of other United Nations organizations; (c) IETC should be dedicated to the transfer of environmentally sound technologies to developing countries and countries with economies in transition by providing training and consulting services, carrying out research and accumulating and disseminating related information with special focus on environmentally sustainable management of large cities and freshwater lake/reservoir basins. 19. IETC would thus strengthen the capabilities of developing countries and countries with economies in transition, to manage larger cities and freshwater lake/reservoir basins by promoting international transfer of environmentally sound technologies. Its work would be organized around three sub-issues: (a) The improvement of access to information on environmentally sound technologies, with a strengthening of the capabilities of developing countries and countries with economies in transition to make informed decisions on technology choices; (b) The fostering of technology cooperation, partnership and transfer, by employing the stakeholders-dialogue approach to generate portfolios of initiatives and coalitions of resources and help to implement the countries concerned; (c) The building of endogenous capacity by organizing training courses and compiling manuals related to environmentally sound technologies and environmentally-sound-technology assessment. 20. The results and accomplishments of IETC will be evaluated against the above objectives formulated by the Governing Council when the Centre was 13 established. The evaluation will therefore also take into account IETC methodologies and partnerships with other United Nations and UNEP units or other relevant international organizations. B. IETC activities 21. The activities of IETC that will be evaluated are those initiated at its official opening in 1994. The activities are listed in the project action sheet as follows: (a) Pilot training programme/workshop on adopting, applying and operating environmentally sound technologies for urban and lake/reservoir management; (b) IETC support to the modernization of urban management programme of Indonesia; 1/ (c) Improving access to information on environmentally sound technologies; (d) Quarterly IETC newsletter IETC's Insight; (e) IETC support to Habitat II; (f) programme; Basic research to best practices in implementing urban sewage (g) Environmental technology assessment (ENTA) for decision makers: a leadership training programme; (h) IETC assistance to the Sustainable Cities Programme (SCP) of UNCHS (Habitat); (i) International source-book on environmentally sound technologies for municipal solid waste management follow-up project; (j) Source-book on alternative technologies for freshwater augmentation follow-up project. 22. All of the above activities, except activity (b), which was abandoned at a very early stage and was never initiated, have been assessed according to the criteria mentioned in the terms of reference for the present evaluation, namely: (a) Timeliness: how the project met the schedule and implementation time-table cited in the project document; (b) Achievement of objectives and results; 1/ This project was never initiated and thus has not been taken up in this evaluation. 14 (c) Attainment of outputs; (d) Completion of activities; (e) Project executed within budget; (f) Impact created by the project; (g) Sustainability. 23. Each activity has been rated on a scale of 1 (highest) to 5 (lowest) for each of the above criteria. The rating was made on the basis of the assessment elements described in section III of the terms of reference (see annex I below). C. Organization of IETC 24. IETC is a small unit with two offices, one in Osaka and one in Shiga. The IETC Osaka office is supported by the Global Environment Centre (GEC), established in 1992 by the Osaka Prefecture and Osaka City Government with the main objective of assisting IETC in its activities. The IETC Shiga office is supported by ILEC. Both GEC and ILEC provide Professional and support staff to the respective IETC offices. 25. IETC currently has the following staff: 15 Table 1 Staff of IETC Staff: Level Responsibilities: Director D1 Office management and networking Deputy director (Shiga) P5 Management of Shiga office; water projects Senior liaison officer L5 Relations with Japanese government and other organizations Administrative/Fund management officer P3 Administrative affairs of IETC Programme officer Shiga P4 Development and implementation of water projects Computer network officer P3 Vacant, temporarily occupied by consultant Programme officer P4 Vacant, temporarily occupied by consultant Information officer P4 Vacant Senior programme officer P5 Urban technology transfer (frozen, will not be filled on short notice) 4 supporting professional staff from GEC and ILEC Professional support to Osaka and Shiga office 2 supporting secretary staff from GEC and ILEC Secretariat support to Osaka and Shiga office 7 General support staff Total 20 staff (2 vacancies) G5/G4 Secretarial and administrative support, one post filled with temporary staff. Including 2 temporary consultants and supporting staff from GEC and ILEC. 26. IETC has a reasonably large staff complement which seems to be sufficient to accomplish the tasks described in the project action sheet. It should, however, be taken into account that the number of Professional staff contributing directly to the development and implementation of projects and activities is relatively limited. As the Director (responsible for overall management and contacts with external organizations) and the Senior Liaison Officer do not contribute to project implementation, IETC only has 9 Professional officers directly involved in project development and implementation. In addition, it should be noted that: (a) Two of these officers are consultants hired on a temporary basis but providing essential inputs; 16 (b) Four are support staff provided by GEC and ILEC providing important contributions to the IETC activities. As however ILEC and GEC personnel are not experienced with UNEP working methodologies and approaches and because of the differences with working methodologies in Japanese institutions, the integration of these experts in the IETC programme also requires important efforts from the other IETC Professional staff; (c) The number of vacancies is relatively high. These vacancies should be filled as soon as possible to ensure rapid development of further IETC activities. 27. During the development of the activities listed in the project action sheet, the staff of IETC has changed on several instances, which has led to problems in project development: almost all the current Professional staff (except for the Administrative/Fund Management Officer) started working for IETC after mid-1996, little more than two years ago. The current Director has been in charge only since March 1997. The consultants started activities only recently. 28. The organizational structure of IETC is presented in figure 1 on page 18 below. 17 Figure 1 Organizational structure of IETC 18 29. The Osaka office deals primarily with urban management and technologies and with information services (network and newsletter). Furthermore, the Osaka office takes care of all financial and administrative tasks of the Centre. The office receives direct support, including seconded staff, from GEC. 30. The Shiga office is primarily responsible for the development and implementation of water-management projects. The office receives direct support, including seconded staff, from ILEC. D. Financing of IETC 31. IETC receives its funds mainly from the Ministry of Foreign Affairs of Japan. The Japanese financial contribution provided through the trust fund is approximately US$ 2.5 million per year. UNEP also provides a yearly contribution. Since 1994, UNEP has contributed from the Environment Fund the amounts listed in table 2. Table 2 UNEP Environment Fund contribution to IETC Year UNEP contribution through the Environment Fund 1994 US$ 200,000 1995 US$ 1,200,000 1996 US$ 110,000 1997 US$ 405,400 1998 US$ 381,000 32. Since its establishment, the UNEP contribution to IETC has been on average US$ 450,000. This is less than 20 per cent of the contribution by the Japanese Government. 33. Contributions from other organizations and Governments have been limited to in-kind contributions to a number of projects. 34. It is clear that the existence of IETC depends almost fully on the financial contribution provided by the Ministry of Foreign Affairs of Japan. UNEP provides an important financial contribution which nevertheless covers only 20 per cent of IETC costs. The Centre has not been able to raise funding from other Governments and organizations, as opposed to in-kind contributions to some of the projects. E. Cooperation with GEC and ILEC 35. GEC and ILEC are Japanese organizations supporting IETC. Both provide the Centre with Professional staff: GEC provides two staff members at the Osaka office and ILEC provides two staff members to the Shiga office. 19 36. GEC was established to make available to UNEP the abundant accumulation of knowledge and experience in the field of environmental preservation in Japan, particularly in the Kansai region, to support UNEP contributions to urban environmental conservation in developing countries. GEC conducts the following activities: (a) Provision of general support for UNEP activities on urban environmental conservation in developing countries; (b) Conduct of surveys and research that contribute to the preservation of the global environment, especially to urban areas in developing countries; (c) Collection and dissemination of information that contributes to the preservation of the global environment; (d) Organization of training programmes, seminars and symposia; (e) Facilitation of communication with international organizations, governmental organizations and research organizations in various countries; (f) Any other activities necessary to achieve its mission. 37. The above activities clearly support the objectives of IETC. The Centre and GEC therefore have cooperated in a number of international events. GEC provides two staff members to IETC. As far as could be assessed, these staff members are often experts from Japanese companies and organizations without real experience in environmental affairs and without international experience. Their knowledge of the English language is often limited and insufficient to cooperate with international staff. 38. GEC has currently 26 employees and is housed in the same building as IETC, which facilitates communication and cooperation between the two organizations. 39. ILEC, based in Shiga, is located in the same building as the IETC Shiga office. ILEC is an international environmental non-governmental organization established in February 1986 as a follow-up of the international conference on the conservation of management of lakes organized in Shiga on the shores of Lake Biwa. Its objective is to promote the environmentally sound sustainable management of lakes and reservoirs around the world. ILEC performs activities such as the organization of international conferences and workshops and the development of training courses and manuals. 40. Since 1992, ILEC has also been designated as a supporting organization to IETC. Its activities are carried out with the advice of a Scientific Committee consisting of 19 researchers and policy makers from around the world. Like GEC, ILEC provides two staff members to IETC, in its case to the Shiga office. These staff members have similar backgrounds to those from GEC and thus have insufficient environmental and international experience and an inadequate command of English. 20 F. IETC International Advisory Board 41. IETC has an International Advisory Board operating under the supervision of the Executive Director. The International Advisory Board has met twice, the last time being in 1995. 42. The Board is composed of representatives from a range of countries from all regions of the world. The countries represented at the last meeting included Congo, India, Japan, Lebanon, Panama, the Russian Federation and the United States of America. A French expert was invited but could not attend. The Board advises the Executive Director on the general objectives of IETC and its work programme. Its members have proved to be willing to assist IETC in the development of ongoing and planned activities. 43. Some of the recommendations made at the last meeting of the Board and which are still valid related to: (a) Prioritization of activities and analysis of their interactions in order to establish a systematic and integrated approach unique to IETC; (b) Size and number of projects with preference for a small number of visible projects involving long-term commitment; (c) Attention to be paid to the critical nature of building partnerships and networking, identification of important stakeholders and development of constructive dialogues with them; (d) Regular review of activities undertaken elsewhere like the Technology Information and Promotion Service (TIPS) of the United Nations Development Programme and the System for Analysis and Research Training (START). 44. The above observations have been taken into account in the development of conclusions and recommendations. 45. As part of the development and maintenance of an effective network, it is recommended that regular meetings should be organized within the Board, with the participation of representatives from countries and organizations that may assist in the development and funding of IETC projects. The Board could be organized as an ad hoc advisory group composed of experts and officials with a background and experience in the topics at issue at a given time. The composition of the Board would thus be adjusted in line with the issues on the agenda. G. Main observations 46. Concerning the staffing and organization of IETC, the following observations could be made: (a) IETC has few Professional staff at its disposal, a factor that limits opportunities for the development and implementation of projects and activities; 21 (b) Networking and communication with other international (United Nations, United States, European Community, etc.) organizations is thereby restricted. IETC works together closely with GEC and ILEC and has developed partnerships with UNCHS (Habitat), the United Nations Industrial Development Organization (UNIDO) and universities in South Africa, Germany and Australia. There are almost no contacts with Governments and international organizations that would be able to support IETC financially. This networking and partnership-building requires considerable time and effort, as well as Professional staff to maintain efficient and regular contacts; (c) The number of support staff is not commensurate with the number of currently available Professional staff. This relationship will be improved after the vacancies for professional staff have been filled. No additional supporting staff will be required as long as IETC has less than 10 Professional staff members; (d) Despite the limited staff available, IETC has been able to develop contacts with other United Nations and international organizations within the projects conducted, which seems sufficient for a successful completion of tasks (see next chapter); (e) Although IETC has developed a close relationship with GEC and ILEC, these partnerships could still be strengthened. GEC conducted a number of activities (workshops, seminars, training courses) in developing countries that clearly overlap with the goals and objectives of IETC. GEC and IETC could thus further improve and widen their cooperation and coordination of activities. Staff provided by GEC and ILEC should meet requirements with regard to environmental and international experience and English language capabilities; (f) IETC organized two advisory board meetings. The Centre should try to develop an International Advisory Board composed of representatives from potential support organizations and organize Board meetings on a regular basis (once a year) and an ad hoc composition of experts and officials having a background and experience in the topics on the agenda. This should be coordinated with the Advisory Group of the UNEP Industry and Environment Centre. II. EVALUATION OF ACTIVITIES 47. The methodology for the evaluation of the activities of the International Environmental Technology Centre has been as follows: (a) Visit to the IETC offices in Osaka and Shiga from 12 to 21 October 1998 to study project documentation and to discuss the projects with involved IETC personnel; (b) Visit to the Ministry of Foreign Affairs and the Environment Agency of Japan in Tokyo on 19 October 1998; (c) Discussions with IETC target groups and contacts to develop an overview of project results and impacts. 22 48. For each of the activities, a scoring table has been completed applying the criteria mentioned in chapter I, section B, above. 49. Each activity-evaluation gives a short explanation of the activity with the main collaborating organizations and overview of project outputs. Then, the evaluation comments are presented as an explanation to the completed scoring table. A. Adopting/applying and operating environmentally sound technologies for urban and lake reservoir management 1. Project description 50. This project aims at providing training to elected local-government officials in local sustainable development and environmentally conscious municipal management, i.e., the adoption/application and operation of environmentally sound technologies for urban and lake/reservoir management. This topic was identified as one which was least addressed in capacity-building activities around the world. The project addresses the following needs: (a) sound way; Need for local authorities to manage cities in an environmentally (b) Need to strengthen capabilities for the adoption, application and operation of environmentally sound technologies; (c) Need for regional/national training institutes to strengthen capacities of local-government officials on the environmentally sound management of cities. 51. The main elements/outputs of the project were: (a) Study on training needs in the adoption, application and operation of environmentally sound technologies for urban and lake/reservoir management; (b) Pilot training programme, a trainers' tool-kit, a set of exercises and training materials; (c) Training manual on the management of environmentally sound technologies for local-government officials; (d) Training handbook on the management of environmentally sound technologies for councillors as part of the UNCHS (Habitat) series for elected local leadership. 52. The project was to be conducted in such a way that national training institutions or training units would be able to design, organize and implement national follow-up training to regional activities facilitated by IETC. 53. The project was implemented in close cooperation with Dresden University in Germany, which provided expertise and workshop accommodation. 23 As a follow-up to this workshop, regional workshops took place in Bucharest (May 1997) and in Perth (December 1997). These regional workshops were facilitated by Terra Nostra (Romania) and Proctor and Gamble (Australia). In 1998, a number of national follow-up workshops were organized in Australia, Mongolia, Myanmar, Samoa and other countries. 2. 54. Project outputs The project had the following outputs: (a) IETC TRS 1: Training needs in utilizing environmental technology assessment (EnTA) for decision-making. A preliminary study to strengthen capabilities in managing ESTs; (b) Pilot training held at Dresden University in Germany; (c) IETC TRS 1: Workbook for training in adopting, applying and operating ESTs a pilot programme (d) IETC Technical Publication Series No. 7: The Counsellor as Guardian of the Environment A Training Handbook for Elected Leadership on How to Utilize ESTs. 3. Timeliness Attainment 1 (a) 2 Project evaluation Completion Budget 1 1 Impact 2 Sustainability 2 Timeliness 55. The project reports and workshop were completed before the deadline of the project. The project benefited from effective cooperation with Dresden University in Germany, which also provided accommodation for the pilot training workshop. There are no remarks or comments to be made concerning the planning and completion within the project period. (b) Attainment 56. With the above-mentioned products, the project objectives have almost been met. Follow-up of the workshop through national activities on the basis of materials produced in the project was realized by a number of participants. The objective of further dissemination of the project results has thus also been realized. Currently, however, no further activities are being initiated. 24 (c) Completion 57. The project has been completed successfully and has realized most of its objectives. The reports mentioned above comply with the sub-objectives of the project and were disseminated to a range of contacts, the participants in the pilot training programme and to organizations asking for the reports as a follow-up to the IETC newsletter. (d) Budget 58. The project was completed within the budget constraints also thanks to the significant contribution from Dresden University. (e) Impact 59. The project has had a good overall impact, as the participants in the pilot training have initiated follow-up or participated in national activities. In addition, a range of organizations requested further documentation from IETC. The impact could still be increased by follow-up training activities and the development of a toolkit for trainers and train-the-trainers activities. (f) Sustainability 60. There has been a considerable follow-up of the project. IETC could initiate further follow-up projects like the one mentioned above or integrate the results of this project in an urban environmental management training activity (see recommendations on page49). 4. Overall remarks 61. The project is considered successful: the scope and objectives perfectly match the objectives of IETC. The project was developed and accomplished in close cooperation with Dresden University, thus guaranteeing up-to-date scientific input. The project also included cooperation with other United Nations organizations, such as UNCHS (Habitat). B. Improving access to information on environmentally sound technologies 1. Project description 62. In its decision 16/34, the UNEP Governing Council asked IETC to ensure information dissemination with a special focus on environmentally sustainable management of big cities and freshwater lake/reservoir basins. In order to follow up on this decision, IETC developed three interrelated work activities: (a) Survey of environmentally-sound-technology information systems, including support for the establishment on a network of providers, suppliers and users of information on environmentally sound technologies. This activity is a continuation of the survey of environmentally-sound-technology information systems. This survey will be supported by user needs assessments, expert panels, dissemination of case-studies and system benchmark studies; 25 (b) Development of a database to ensure the systematic and integrated collection and provision of objective, targeted and quality reviewed information on specific environmentally sound technologies related to large cities and to freshwater lakes and reservoirs; (c) Promotion of the transfer of selective information on scientific and technical aspects of environmentally sound technologies from industrialized nations to developing countries and to countries in economic transition. This service will have two parts: (i) a printed index and supporting database of citations and abstracts of material on environmentally sound technologies, originating principally in the industrialized countries; and (ii) a document delivery service. 63. The project was expected to deliver the following outputs: (a) Updated final report on environmentally-sound-technology information systems, including case-studies evaluating system performance; (b) Joint IETC, Industry and Environment Office and UNEP headquarters publication of an EST Information User Needs Assessment report in 1997; (c) An IETC World Wide Web homepage providing details of information systems on environmentally sound technologies and promoting a virtual electronic network of on-line information systems on such technologies using the Internet; (d) A bench-mark study evaluating existing EST information systems and providing an outline of best practices in EST information system development and operation; (e) An expert meeting in 1997 to review work undertaken and to identify strategic future directions for work and implemented actions; (f) A globally accessible database on the Internet containing useful, targeted, reviewed and objective information on environmentally sound technologies applicable to environmental issues in large cities and freshwater lakes and reservoirs, and scientific and technical abstracts and citations on environmentally sound technologies derived from the IETC Review of ESTs; (g) The IETC Review of ESTs, published quarterly and distributed to the user community. 64. The project was to be performed in close collaboration between IETC, the UNEP Industry and Environment Office and UNEP headquarters. The project document provides details on the division of responsibilities and tasks between the three partners and gives a detailed overview of the Centre's tasks in conducting and managing the project. 2. 65. Project outputs The project has delivered the following results and outputs: 26 (a) A report entitled: "UNEP Survey of information systems related to environmentally sound technologies", including chapters on a range of other internationally accessible information sources, user needs and casestudies, published by IETC, the Industry and Environment Office and UNEP headquarters; (b) An IETC World Wide Web homepage providing details of environmentally-sound-technology information systems and promoting an electronic network of such information systems on line on the Internet. 3. Timeliness Attainment 1 2 Project evaluation Completion Budget 2 3 Impact 2 Sustainability 2 66. The two main products of this project have been mentioned above: a UNEP survey concerning environmentally-sound-technology information systems and a World Wide Web homepage providing access to an environmentally-soundtechnology database. These products comply with the objectives and products mentioned in the project document. (a) Timeliness 67. According to the project document, the project started in April 1996 and would be completed on December 1997. The reports were finished in or before 1996 and published in April 1997 in one combined report. An expert meeting was held from 9 to 11 October 1995 (?). The maESTro environmentallysound-technology database and IETC homepage have been accessible since 15 February 1996. The project results were thus achieved within the time limits set in the project document. (b) Attainment 68. The objectives as formulated in the project document are generally realized. The report addresses all issues mentioned in the project document. The World Wide Web homepage and database are easily accessible from all over the world, requiring only an internet connection. The information in the database is also distributed on CD-ROM for those users not having Internet access. 69. The homepage now focuses on environmentally sound technologies. The homepage and database could be developed further by having the homepage provide more of the background information on urban environmental management available at IETC in a number of studies and reports, as well as more contact details of experts and organizations who participated in IETC events. The environmentally-sound-technology database in maESTro should provide more United States, European and indigenous environmental technologies. Currently, the database has relatively more Japanese technologies (approximately 40 per cent). IETC is already giving attention to this matter, and the situation will probably be improved in the near future. (c) Completion 27 70. With the above outputs, the project can be considered as being completed. A World Wide Web site and database, however, need constant attention and updating and can never be said to be completed. (d) Budget 71. The costs for the trust fund proved to be higher than expected. The actual costs made for equipment and professional assistance for development of the homepage and database seem to be very reasonable in view of the outputs realized. Costs for professional assistance in information technology are high. (e) Impact 72. The homepage and database have only been operational for a short period. The number of users is therefore still relatively limited but is rapidly increasing. In view of the information provided and the quality of the homepage, it is expected that the number of users will increase further. An African network has already expressed its intention to use the maESTro database as the tool for collection and dissemination of regional information on environmentally sound technologies. Recently, IETC received an expression of interest from the German GTZ to use maESTro widely. This proves the potential contribution the database may provide for organizations in developing countries and the interest that organizations already have. (f) Sustainability 73. As already mentioned above, the maintenance and updating of the homepage and database will require full attention on a continuous basis. Currently, the activities of IETC, including this project, are primarily secured by the contribution of the Ministry of Foreign Affairs of Japan. The sustainability of this activity therefore depends on the continuation of this contribution in the coming years. In view of the difficult economic situation in Japan and the risk of budget reductions, there is a danger of a shortfall in the financial resources required for this project to maintain its current quality. 4. Overall remarks 74. This project can be considered as very successful. The report and the World Wide Web homepage and database are good and useful products. The increasing number of users of the maESTro database and, especially, the use of it by the African network prove the quality of the results. 75. The database can be improved further by using it also for the dissemination of all IETC reports. This may also contribute to reducing the costs related to the dissemination of reports. As more and more organizations have Internet access, this may prove to be an effective dissemination tool. 76. In addition, closer cooperation with the UNEP Industry and Environment Office Internet site and International Cleaner Production Information Clearing-house (ICPIC) database could be developed. Currently, the maESTro 28 database certainly has cleaner production opportunities related to the main topics of IETC activities: urban environmental management and freshwater lake and reservoir management. Overlap with ICPIC will be limited as ICPIC focuses on industrial activities and processes, and the IETC database on endof-pipe treatment technologies mainly related to urban applications. C. Quarterly IETC newsletter: 1. IETC's Insight Project description 77. The priority actions of UNEP include raising public awareness and encouraging action in the area of environmental protection, as well as disseminating environmental information and data to Governments and to the United Nations system. 78. The IETC newsletter IETC's Insight responds to this priority. The project aimed at publishing eight newsletters in 1996 and 1997, one issue every quarter, and at focusing on the technical, economic, financial, social and health aspects of the management of environmentally sound technologies within the field of expertise of IETC. The newsletter was intended to serve recipients whose information needs are not covered by other sources. 79. The project for the dissemination of the IETC newsletter aimed at enabling recipients to: (a) Understand the critical tool of utilizing environmentally sound technologies in urban and freshwater management; (b) Replicate activities to promote environmentally sound technologies in the particular conditions and socio-economic context of their countries; (c) Establish direct contacts between users and owners of environmentally sound technologies to engage in technology cooperation; (d) Refer to other information sources on environmentally sound technologies and organizations concerned with the management and transfer of such technologies; (e) Follow up on the development of IETC programme activities at the offices in both Shiga and Osaka. 2. Project outputs 80. Since the beginning of 1996, nine issues of IETC's Insight have been produced and disseminated, as follows: (a) Four issues in 1996: (b) Three issues in 1997: (c) Two issues in 1998: be published shortly). spring, summer, fall, winter; spring, August and December; June and September (an additional issue will 29 81. Beside the regular newsletter content, most editions had a special section, dealing with specific environmental topics in the field of urban or freshwater environmental management: (a) Summer 1996: Sanitation?" (b) Transfer"; Fall 1996: "What is Environmental Sustainability in "Understanding Sustainable Development and Technology (c) Winter 1996: Remediation"; "Advances being made in Cold Climate Solid Waste (d) August 1997: "The Conception and Execution of Improved River Quality Using a Newly Developed Purification Method: The Simanto-gawa System"; (e) December 1997: "Gaining Common Ground to Curb Climate Change"; (f) June 1998: "Fresh Water: Sector and Those in Need"; (g) September 1998: Future Challenges". Finding a Synergy Between the Private "Water and Technology Current Trends and 82. The IETC newsletter was also used to distribute leaflets on environmental technology assessment in relation to the collaborative efforts of IETC and the Industry and Environment Office. 3. Project evaluation 83. This project was evaluated by studying the published newsletters and by sending a small questionnaire to approximately 100 recipients of the newsletters. This resulted in the following evaluation: Timeliness Attainment Completion Budget Impact Sustainability 2 2 2 3 2 2 (a) Timeliness 84. During the project period, between April 1996 and December 1997, six editions of the newsletter were published instead of eight. In 1996, it was a quarterly newsletter but in 1997 and 1998 it was published twice a year. This is considered to be a better frequency then the originally foreseen quarterly publication, especially as IETC has also other opportunities for providing information to its contacts and newsletter recipients, e.g., the World Wide Web homepage and electronic mail. (b) Attainment 85. The IETC newsletter has a professional lay-out and logical content. 30 Topics dealt with are: (a) Events and publications; (b) Background articles; (c) The World Wide Web homepage and maESTro database; (d) IETC projects. 86. Although the newsletter is of high quality and has been well received by its recipients, the following improvements should be considered: (a) Each edition should contain a list of forthcoming events for the coming year; (b) Each edition should have a list of the main IETC publications together with an order form; (c) Each edition should provide information on maESTro; (d) The current "IETC's Insight Special" could be used to provide complete guidelines for urban and freshwater environmental management. Full guidelines could be disseminated to the recipients over a number of editions, starting with an introduction to the series (explanation, content, how to use and save the material) describing the substantive content to be circulated in the issues to come. This content could be based on existing manuals and guidelines. Recipients would thus be more interested in keeping the newsletter and distributing it to interested colleagues. 87. Most recipients do not save the newsletter, so some information (events, publications) has to be repeated in each edition. (c) Completion 88. The project has been completed according to the project document, although the number of editions of the newsletter issued is lower than foreseen. (d) Budget 89. The costs have been approximately 20 per cent higher than budgeted because the distribution costs were higher than foreseen. (c) Impact 90. It is difficult to assess the impact as such an assessment would necessitate contacting many of the recipients. Most recipients browse the newsletter quickly and then put it aside. Some users copy an article and pass it on to colleagues. The use of the newsletter is as could be expected for such a type of publication. The overall impact of the newsletter is as should be expected but will be limited. It is therefore important to repeat some of the IETC information in each edition, as mentioned above. 31 (f) Sustainability 91. The publication of the newsletter depends almost fully on the financial contribution of the Japanese Government to IETC. Maintenance of the quality can only be guaranteed with sufficient professional staff at the IETC offices. 4. Overall remarks 92. IETC has developed a quality newsletter that forms an important element of its information-dissemination activities. Publication of two or three editions of this newsletter a year is considered to be sufficient. 93. Besides the publication of this newsletter, and taking the above recommendations for further improvement into account, IETC could develop a newsletter for dissemination by e-mail, which would provide regular information on IETC activities and publications to those contacts having access to e-mail. The advantages of such a newsletter would be: (a) Relative ease of production, as the lay-out could be simple; (b) More frequent dissemination of information to IETC contacts, for example, every two months; (c) It would serve as an easy answering and publication-ordering tool. 94. The main disadvantage would be that not all recipients of the printed newsletter could be addressed. The number of e-mail users is, however, increasing rapidly. D. IETC support to Habitat II 1. Project description 95. Within the framework of this project, IETC, in cooperation with GEC, contributed to the second United Nations Conference on Human Settlements (Habitat II), held in Istanbul from 3 to 14 June 1996. The IETC contribution took the form of: (a) symposium; (b) A forum on environmentally sound technologies, including a An exhibition of selected environmentally sound technologies. 96. The main objective of IETC was to contribute towards improving access to information on environmentally sound technologies and to providing heads of State and local authorities with an opportunity to access information on environmentally sound technologies and to consult with experts on selected technologies. 97. Habitat II provided IETC with an opportunity to reach a large number of decision makers and to promote the Centre. 32 2. 98. Project outputs The following project results were expected and achieved: (a) A contribution to Habitat II in the area of environmentally sound technologies for the sustainable development of cities; (b) Information on environmentally sound technologies made available to the policy makers, decision makers and technology users of the world; (c) A continuing commitment to inter-agency coordination and collaboration between UNCHS (Habitat) and UNEP and between IETC and its cooperating agencies; (d) Reports on the participants to Habitat II and on the symposium in the form of a mission report produced by a member of the IETC staff; (e) Database of contacts for IETC derived from list of participants. 3. Project evaluation 99. This project was briefly discussed with Mrs. L. Casanova of IETC at her Shiga office. In addition, various documents, including the internal evaluation and relevant articles in the IETC newsletter, were studied. Timeliness Attainment Completion Budget Impact Sustainability 2 2 2 1 2 N/A (a) Timeliness 100. The project was completed within the time-limits. (b) Attainment 101. A symposium was held featuring presentations on various environmentally sound technologies. The objectives as foreseen were therefore more or less achieved. (c) Completion 102. The project was completed with the participation of IETC in Habitat II and the symposium. (d) Budget 103. The project is completed within the budget provided. This was facilitated through close collaboration with GEC, which reduced the budget requirements for IETC. (e) Impact 104. The impact was as foreseen in the project document. 33 (f) Sustainability 105. Not applicable, as the project was a one-off event. 4. Overall remarks 106. Although the project was a small one for IETC, it is nevertheless important for IETC to promote the Centre on such occasions especially as many decision makers will be present. IETC should take part in a greater number of such international events in the field of urban and freshwater environmental management. 107. The UNEP Industry and Environment Office organizes a regular high-level seminar on cleaner production, and IETC might develop a similar event in the field of urban and freshwater environmental management in close collaboration with the Sustainable Cities Programme and/or UNCHS (Habitat). E. Basic research guidelines to best practices in implementing urban sewage programmes 1. 108. Project description The objectives of this project were: (a) To enhance the capacity of the administrators or municipal decision makers responsible for implementing sewage programmes; (b) To strengthen the ability of those officials to select, adopt, integrate and apply/manage technologies related to sewage systems in implementing urban sewage programmes. 2. 109. Because of personnel problems, this project was not implemented. 3. 110. Project outputs Project evaluation As this project was not implemented, no evaluation could be conducted. 4. Overall remarks 111. Because of personnel problems (the main expert in this field left IETC), this project was not implemented. This underlines the critical situation with regard to the staffing of IETC. UNEP and IETC management should take appropriate measures to improve the current status of IETC staffing. This matter is being addressed. F. Environmental technology assessment (EnTA) for decision makers: a leadership training programme 1. 112. Project description The EnTa project was based on a survey conducted by IETC in 1994/95 to 34 determine training needs in this field. Some 30-40 per cent of the respondents from developing countries and countries with economies in transition indicated a lack of awareness on techniques for assessing technologies. Respondents from all regions (Africa, South Asia, Latin America, Caribbean and Eastern Europe) also indicated a need for training in this field. In 1995, an expert group meeting was organized in Osaka to review this survey and an EnTA pilot training programme. 113. EnTA is an analytical tool used to help understand the likely impact on the environment of a new technology. EnTA can be utilized to strengthen local capabilities in analysing technology options. 114. The objectives of the project were: (a) To create the capability to design and implement national or local training events on EnTA for local managers of EnTA; (b) To provide basic knowledge to identify the potential impacts of consequences on the environment of technological solutions and innovations; (c) To create understanding how EnTA can contribute to a better environment for human health and well-being, and also can be related to sustainable development for their own countries. 115. The project was developed and implemented in close collaboration with the UNEP Industry and Environment Office. A pilot training workshop for the Asia/Pacific region took place in November 1995 at the National Institute for Public Administration (INTAN), Kuala Lumpur. A second regional EnTA workshop was implemented in close collaboration with the UNEP Regional Office for Africa, and Potchefstroom University, South Africa. Both workshops led to a range of national follow-up activities in China, India, Kenya, Laos, Malawi South Africa, and Viet Nam. 2. 116. Project outputs The main project results and outputs were: (a) Regional training courses on EnTA held in the Asia/Pacific region and in South Africa, in close collaboration with national organizations. Participants came from a range of countries in both regions, including Benin, Botswana, Gambia, India, Kenya, Laos, Malawi, Malaysia, Mozambique, Niger, Nigeria, Swaziland, Uganda, Viet Nam, Zimbabwe and other countries. Representatives from governmental organizations, research institutes, training centres and consultants participated; (b) Development of an evaluation report based on the immediate findings and experiences of the participants; (c) National follow-up workshops in China, India, Kenya, Laos, Malawi, South Africa and Viet Nam,; (d) A network of participants in the training course. 3. Project evaluation 35 117. The project evaluation was conducted on the basis of the project outputs (final report, evaluation report) and discussions with the IETC staff member then responsible for the project, two participants, a lecturer and a representative from Potchefstroom University. Timeliness Attainment 2 (a) 3 Completion Budget 2 2 Impact 3 Sustainability 2 Timeliness 118. The project was completed within the project schedule. It received support from the African Regional Centre for Technology (ARCT) and Potchefstroom University. (b) Attainment 119. The number of participants was quite reasonable and, with the lecturers, the foundations of a potential network of EnTA experts were established. There was, however, no indication of any follow-up to the workshop in terms of maintaining the network or follow-up EnTA activities in the participants' countries of origin. In addition, the scope of the training courses was very wide (Project reports and comments from lecturers) and was not limited to environmental assessment. Although other social issues are certainly relevant to technology assessment, IETC could have focused more on the environmental issues. (c) Completion 120. The project was completed in line with the expected results set out in the project document. (d) Budget 121. The project was completed within the budget, inter alia, because of in-kind contributions of the partners in the project. (e) Impact 122. The project attracted more than sufficient interest in the Asia-Pacific region and in African countries. The follow-up to the project and hence the final impact have also been quite good. Several participants have initiated regional or national EnTA activities. (f) Sustainability 123. The regional workshops have resulted in a number of national follow-up actions initiated by participants. As IETC has now developed a workbook for training in environmental technology assessment, even more efforts should be developed, in close cooperation with the UNEP Industry and Environment Office, to develop a series of training courses, particularly in those regions where a real need for EnTA exists because of the need to adopt and apply environmentally sound technologies. 36 4. Overall remarks 124. IETC has developed effective EnTA training courses in close cooperation with the UNEP Industry and Environment Office, INTAN, ARCT and Potchefstroom University. The lecturers came from different countries, thus providing a wide range of environmental technology knowledge and experience. The incorporation of site visits into the training courses proved effective. In general, the training course was good. 125. The overall objectives of this project, especially those follow-up in the countries of the participants, have also been number of participants have initiated EnTA follow-up actions. participants and lecturers still have contacts with each other related to achieved. A Some and IETC. 126. It is recommended that EnTA should be integrated into an overall urban and freshwater environmental training course (see recommendations on page 49 below), which should be held on a more regular basis and in different regions. Furthermore, the scope of EnTA should be focused more on environmental issues. G. IETC assistance to the Sustainable Cities Programme of UNCHS (Habitat) 1. Project description 127. This project involved support from IETC to the demonstration project "Sustainable Shenyang and Wuhan", as a contribution towards a concerted effort to promote sustainable physical, economic and social development of the municipalities of Shenyang and Wuhan by focusing on the development of local capacities for strategic planning and environmental management. 128. These projects are in support of UNEP commitment to the implementation of the Sustainable Cities Programme as a joint facility with UNCHS (Habitat). The IETC contribution was specifically in the area of promoting the transfer of environmentally sound technologies in the fields of urban and freshwater environmental management through technology cooperation, capacity-building and information dissemination and exchange. 129. The main input of IETC was to focus on: (a) concepts; (b) Provision of knowledge on environmental risk assessment (EnRA) Analytical tools for the preparation of an environmental profile; (c) Tools for the preparation and development of investment projects, including concepts and tools for EnTA. 130. IETC support focused on capacity-building and technology transfer of environmentally sound urban technologies, as well as the provision of assistance for the preparation phase of the project. 37 2. 131. Project outputs The main results that IETC intended to achieve from this project were: (a) Assistance to the cities of Shenyang and Wuhan in formulation and development of local agendas 21 to guide their (environmental) development; (b) Provision to the cities of the necessary information on environmentally sound technologies, of both the software and the hardware type; (c) Demonstration of the mechanisms for sustainable development by assisting the cities to develop local capacities for environmental planning and management and build local awareness and skills related to environmentally sound technologies; (d) Provision of information materials and training courses on EnRA and EnTA, covering both theoretical frameworks and technical systems and procedures. 3. Project evaluation 132. The project was evaluated through discussions with the responsible IETC officials, as wells as by studying the relevant IETC materials and consulting the coordinator of the Sustainable Cities Programme of UNCHS (Habitat). (a) Timeliness Attainment Completion Budget Impact Sustainability 3 2 3 3 2 3 Timeliness 133. The IETC contribution to these SCP projects was completed within the time limits set out in the project document. However, the training sessions could not be held within the original planning framework of the Sustainable Cities Programme projects. For various reasons, the planning of these projects was delayed. The training courses have now taken place at a relatively early stage of the SCP projects. It is not yet clear whether this will have a real impact on the further development of the Sustainable Cities Programme projects. (b) Attainment 134. IETC has completed its input to the Sustainable Cities Programme projects of Shenyang and Wuhan. EnRA and EnTA documents were prepared and training courses took place. It is not clear whether the results have been achieved, since the Sustainable Cities Programme projects in China have not yet been completed. 135. IETC should keep close contacts with the SCP coordinator in order to identify the final results of its input and the potential opportunities for improving its contribution to such projects. 38 (c) Completion 136. IETC has completed its contribution. Close contacts with the Sustainable Cities Programme remain necessary. (d) Budget 137. The total budget of US$ 100,000 was not spent. Only US$ 24,385 was required for the preparation and conduct of the workshops and the provision of assistance during the preparatory stage. According to the project document, the initial budget was provided for the workshops, but the latter were organized at a considerably lower cost. It was not possible to assess any difference between the planned and actual contents and quality of these workshops (programmes, lecturers, materials). The budget requirement was also lower because the two workshops were organized close to each other so lecturers could attend both workshops in the course of a single mission. (e) Impact 138. The actual results have to be observed during the coming period (final stages of the SCP projects and contacts with the officials of Shenyang and Wuhan). Municipal officials are now aware of EnRA and EnTA tools and the availability of environmentally sound technologies. In addition, they have access to information systems of IETC (maESTro) and other UNEP or international Internet sites and databases. Because of the way the workshops were organized (over a relatively short time), not all officials and technicians from Shenyang and Wuhan could attend both. As a result, the final impact of the project will be reduced. (f) Sustainability 139. IETC contributions and support of this type will have a low level of sustainability unless IETC does put more effort into developing tools, information materials and training methods for such urban management and planning projects. In view of the central goal of IETC, which is to provide support in urban environmental management and freshwater lake/reservoir management, participation of IETC in future Sustainable Cities Programme projects seems very relevant. IETC should therefore develop a common approach and even a memorandum of understanding with the Sustainable Cities Programme for more continuous and systematic cooperation with the Programme and contributions to its projects. 4. Overall remarks 140. Participation of IETC in the Sustainable Cities Programme projects is considered as one of the best means of achieving the Centre's objectives in the fields of urban environmental management and freshwater lake/reservoir management. Because of delays in the Sustainable Cities Programme projects themselves, the timing of the IETC input was not optimal. For this reason, and also because of the importance of these projects for IETC activities, a close follow-up is required. IETC should therefore maintain close contacts with the Sustainable Cities Programme coordinator and, if possible, with relevant municipal officials of Shenyang and Wuhan. IETC might check the use of the maESTro database by these municipalities by monitoring the log-ins by 39 these cities to the site. 141. IETC is already fully aware of the need to follow up on these projects. H. International Source Book on Environmentally Sound Technologies for Municipal Solid Waste Management: follow-up project 1. Project description 142. Municipal solid-waste management in developing countries involves such factors as: (a) Rapid growth of population, urbanization and expansion of squatter areas, including an increase in the quantity of solid wastes and uncontrolled discharges; (b) Lack of public awareness, education and community participation in solid-waste management, including inadequate legislation and enforcement and ineffective solid-waste-management systems; (c) Socio-economic crises causing financial and organizational weaknesses; (d) Increasing consumption and changed consumption patterns. 143. Because of these elements, IETC developed a project with the following objectives: (a) Provision of an inventory of available information sources, tailored to the needs of developing countries and countries with economies in transition; (b) Development, in a few key areas, of selected in-depth casestudies from developing countries of innovative and cost-effective technologies for environmentally sound municipal solid-waste management; (c) Development of a number of success stories that can be applied by developing countries. 144. By achieving these objectives, IETC would contribute to an accelerated flow of information on environmentally sound technologies relating to municipal solid-waste management through direct contacts between sources and users, and to enhanced know-how in the target countries. 145. The project has been conducted in close cooperation with the UNEP Industry and Environment Office and INFOTERRA, the global environmental information network of UNEP, which already published a Directory of Information Sources on Wastes and their Treatment, and was coordinated with activities of UNCHS (Habitat) and the World Health Organization in this field. 2. 146. Project outputs The project has provided the following outputs: 40 (a) International Source Book on ESTs for the Management of Municipal Solid Waste; (b) Information leaflet prepared to enhance awareness of local government officials on municipal solid-waste problems, available information sources and appropriate technologies to address these problems; (c) Report of an expert meeting that reviewed issues such as: (i) (ii) Information on how to increase domestic and international transfers of appropriate technologies; and Advice to UNEP on whether there is a need for the establishment of an information clearing-house in this area, through an article in the IETC's Insight newsletter. 3. Project evaluation 147. This project was evaluated by studying its outputs, by consulting the main IETC officials, including the responsible official who has since left the Centre, and by holding discussions with experts who participated in the expert meeting and contributed to the development of the sourcebook. Timeliness Attainment Completion Budget Impact Sustainability 2 2 2 3 3 2 (a) Timeliness 148. According to the project document, the project would start in April 1996 and be completed on 30 September 1996. The reports were finished on 1 December 1996. The expert meeting was held from 9 to 11 April 1996. (b) Attainment 149. The objectives, as formulated in the project document, were realized. The sourcebook addresses all issues mentioned in the project document. The internet homepage and database also provide information on technologies for municipal solid-waste management. (c) Completion 150. With the above outputs, the project can be considered as being completed. (d) Budget 151. The costs proved to be higher than foreseen in the project document. The budget for the project was, however, limited, originally US$ 30,000. The required budget of US$ 43,471 seems appropriate in view of the quality of the final product, namely, the sourcebook on environmentally sound technologies for municipal solid-waste management. 41 (e) Impact 152. The outcome of this project may have considerable impact. The sourcebook is of very good quality. The final impact realized, however, will depend on the dissemination activities developed and implemented by IETC. The sourcebook may certainly be used in other project, such as contributions to Sustainable Cities Programme projects and training courses for urban environmental management. The impact of the current product seems still limited as no direct follow-up projects have yet been developed. 153. The sourcebook is also available through the IETC Internet site. The users of this sourcebook, i.e., those who download the document entirely or in part, should be registered and contacted in the future for comments or follow-up actions. (f) Sustainability 154. The quality of the sourcebook combined with the available environmentally sound technologies on municipal solid-waste management in the maESTro database provide good potential for sustainable use of the outputs of this project. Because of staff problems at IETC, no follow-up has yet been organized. 155. It is therefore strongly recommended that the personnel problems of IETC should be addressed and follow-up activities initiated in this field. Integration of this sourcebook in an urban environmental management training course is recommended as one of the most logical follow-up initiatives (see the recommendations on page 49 below). 4. Overall remarks 156. The development of an international sourcebook on environmentally sound technologies for municipal solid-waste management fits in perfectly with the central objectives of IETC. The project was completed effectively with input from different international consultants and organizations. The quality of the final product deserves full attention and follow-up activities. I. Source Book on Alternative Technologies for Freshwater Augmentation: follow-up project 1. Project description 157. In 1995, IETC, in cooperation with the UNEP water programme, implemented a series of projects on the preparation of Sourcebook on alternative technologies for freshwater augmentation in five regions: Africa, Asia, East and Central Europe, Latin America and the Caribbean and small island developing States. 158. This follow-up project aims to compile the sourcebooks for each of the five regions into a consolidated format with standardized technical terminology and textual presentation. 159. The project thus addresses the following needs: 42 (a) The need for planners in the various regions to maximize and augment their freshwater resources using technologies appropriate to their respective region; (b) The need for planners in the regions to be provided with accurate information on different appropriate technologies used to maximize and augment freshwater resources. 160. The current project focuses on the compilation (editing, publication) of the sourcebooks. The evaluation also takes into account the development of the sourcebook material that took place before April 1996. 2. 161. Project outputs The following outputs have been realized: (a) Improved information exchange on appropriate technologies for augmenting and maximizing the use of freshwater resources in the regions through expert meetings, the collection of information on appropriate technologies and their dissemination through the maESTro database; (b) Enhanced regional capabilities to address freshwater-scarcity problems through the distribution of the sourcebooks. 3. Project evaluation 162. The project was evaluated through discussions with the responsible IETC officer, the study of the final products and consultations with experts involved in the development of the sourcebooks. Timeliness Attainment 3 (a) 2 Completion 2 Budget 2 Impact 2 Sustainability 2 Timeliness 163. The initial duration of the project was to be only seven months. In the end, it took 24 months to produce the sourcebooks. The initial timescale, however, seems unrealistic. From the outset, at least one year should have been allocated for the production of these sourcebooks. The eventual 24 months are considered to be too long. It would probably have been better to develop the project document and timeframe after the selection of the consultant. (b) Attainment 164. The original project objectives have been accomplished. The five sourcebooks have been printed and are of good quality. The appropriate technologies described in the sourcebooks are also made available through the IETC maESTro database. 165. The sourcebook materials were produced by hiring experts and organizing 43 expert meetings for each region. (c) Completion 166. The project was completed as foreseen, but with a considerable delay. (d) Budget 167. The project was almost completed within the original budget of US$ 25,000. Final expenditures were US$ 29,436, despite a reduced number of copies printed (300 instead of 500). The budget estimate would have been better if it could have been based on an offer from the printer. (e) Impact 168. The sourcebooks have only recently become available and their impact will depend on their distribution to users and decision makers and on the accompanying activities, such as training courses and workshops. 169. It is recommended that the information in the sourcebooks be integrated into an urban and freshwater environmental management training course to be developed and organized by IETC (see the recommendations on page 49 below). (f) Sustainability 170. The sustainability of these products will depend on follow-up activities; as mentioned above, and on the maintenance and updating of the technology database. In this respect, it is also important to initiate follow-up workshops or training courses as soon as possible, since such activities will provide a means for gathering comments and suggestions for improving the databases and new technologies. 4. Overall remarks 171. This project fits in very well with the overall objectives of IETC related to urban environmental management and freshwater lake/reservoir management. The project suffered a considerable delay partly because the original timeframe was too restricted and partly because of problems with the consultant hired for the editing. These problems could have been reduced by preparing clear work programmes with the consultant, providing for a revision of the project document and ensuring rigorous management of the project subsequently: for example, if the consultant did not produce according to the agreed conditions, payments could be reduced. 172. The final products are very useful for future IETC activities. It is recommended that the sourcebooks be integrated into a general urban and freshwater environmental management training course. III. A. 173. CONCLUSIONS General conclusions Based on the analysis of the projects performed by IETC and on general 44 discussions with IETC personnel, the following conclusions have been developed. 174. Between 1 April 1996 and December 1997, IETC developed and implemented a considerable number of projects. Two of the projects mentioned in the umbrella project document were not implemented, and eight projects were fully completed. Taking the restricted number of personnel into account, this is considered to be a successful accomplishment. IETC has thus certainly been effective. 175. IETC is only a small office. Although the total of staff complement is approximately 20 (including two consultants and six Japanese support staff), IETC currently only has two programme officers responsible for project development and implementation (not counting the Director, responsible for overall management, the Liaison Officer and the Administration/Fund Management Officer), two consultants and six support staff provided by GEC and ILEC. The limited number of international programme officers restricts the capability of IETC to maintain regular and effective contacts with other international institutes and funding organizations. 176. Despite its small size, IETC has two offices, one in Osaka and one in Shiga. Although the housing of one branch of IETC at the premises of ILEC is logical because of IETC freshwater activities and the support provided by ILEC in this field, this split hampers effective communication between IETC staff members. 177. IETC is almost fully (90 per cent) financed by the Ministry of Foreign Affairs of Japan. A major contribution from the Japanese Government is logical because of the initiation of IETC by the Japanese Government, this dependence on one source of funding nevertheless poses financial risks to IETC. Most other UNEP offices receive support (financial or in kind as seconded staff) from other countries. The funding of IETC for the coming year is again secured but IETC should develop activities with support from other sources. 178. The number of professional staff responsible for the development and implementation of projects and network maintenance at IETC is almost critical. Current vacancies at IETC have reduced the number of programme officers to an absolute minimum. This already leads to insufficient networking and has lead to abandoning one project (annex 6: Basic Research to best Practices in Implementing Urban Sewage Programme). The number of professional staff should be increased on short notice by filling the vacancies. 179. The IETC mission is to strengthen the capabilities of developing countries and countries with economies in transition, to manage large cities (urban environmental management) and freshwater lake/reservoir basins by promoting international transfer of environmentally sound technologies. The current image of IETC is focused on the collection and dissemination of information on Environmentally Sound Technologies (ESTs). Although this certainly is a central element of the IETC mission, a more central focus on Urban and Freshwater Environmental Management could be more attractive for most governments and funding organizations. Technology information dissemination would then play an important but secondary role, secondary in 45 the sense of supportive to the central mission. 180. Although the mission of IETC seems clear and provides a basis for project development, a clear vision of the IETC position and central focus is missing. IETC has successfully developed and implemented a number of activities but these seem to be stand alone projects with limited or no feedback or cross-fertilization. Follow-up to these activities have therefore been very limited or even not taken place at all. 181. Networking is a very important activity for a centre like IETC. In particular, the Director (with regard to international contacts, the United Nations, UNEP etc.) and the Senior Liaison Officer (with regard to Japan) have to develop and maintain effective relationships with collaborating and supporting agencies and organizations. Project development could also be initiated on the basis of priorities of these organizations in order to ensure financial support or secondment from these organizations. Current networking activities of IETC are at a relatively low level because of insufficient quality and quantity of personnel. 182. GEC and ILEC now provide professional staff to IETC in both offices. Staff now provided by GEC and ILEC do not have sufficient experience in the environmental field, in development and implementation of international projects and activities and in English language. The selection of GEC and ILEC staff provided to IETC should be done by the Director of IETC and be based on among others the aforementioned criteria. B. Project development 183. IETC successfully developed and implemented a series of projects. These activities have resulted in the following products and tools that warrant maintaining and updating and that will support future activities of IETC: (a) Internet homepage with the maESTro EST database of good quality already serving an increasing number of users; (b) Newsletter, IETC's Insight, published on a regular (two to three times a year) basis with a professional outlook and providing adequate information; (c) Number of information materials (sourcebooks, workbooks, manuals etc.) that will support the development of future IETC projects. 184. The projects and activities developed and implemented to date are developed as stand-alone and separate activities (except for the EST database and newsletter, which both support all other IETC activities). These activities, although mostly completed successfully, have resulted in high quality materials that now need to be developed and incorporated into an integrated urban environmental management methodology. 185. All projects have been developed with relevant and adequate international cooperation as well as with other UNEP organizations; like UNEP Industry and Environment Centre, other United Nations organizations, like UNCHS (Habitat) and Sustainable Cities Programme, and international expert 46 organizations. Through such cooperation, a basis for further development of activities and integration of experiences has been established. C. SWOT Analysis: 186. Based on the above observations and conclusions a SWOT analysis has been developed, as shown in table 3 on below. Table 3: SWOT analysis of IETC STRENGTHS WEAKNESSES Good and effective cooperation with the United Nations, UNEP and other international organizations Insufficient Professional staff for the development and implementation of projects Excessive orientation on environmental technologies which is receiving less attention as a primary topic Number of effective tools available: Internet site with maESTro and EST database, newsletter and manuals and sourcebooks IETC is considered as a "Japanese" organization, which makes it difficult to receive support from other countries Cooperation with and continuous support from Japanese organisations GEC and ILEC No established network with major countries and agencies having a direct interest in IETC activities Continuous financial support from the Japanese Government providing a baseline for project development Insufficient follow-up on projects because of personnel shortages leading to underdeveloped networking GEC and ILEC support in providing staff does not match the requirements of the Centre OPPORTUNITIES THREATS Urban environmental management and freshwater management are increasingly important Risk of reduction of the contribution from the Japanese Government due to budgetary problems on the Japanese overall budget 47 Other organizations developing international activities in the field of urban and freshwater environmental management 48 IV. RECOMMENDATIONS 187. The International Environmental Technology Centre of the United Nations Environment Programme (UNEP/IETC) started its activities in 1994 after a series of earlier discussions and implementation meetings. The IETC inauguration meeting took place on 30 October 1992. The Government of Japan then signed an agreement with UNEP to set up the International Environmental technology Centre. Close collaboration with other UNEP units, such as the Industry and Environment Centre and the Global Environmental Monitoring System (GEMS) was also sought. The budget of IETC would come from two sources: (a) The programme budget of the UNEP Environment Fund; (b) Voluntary contributions made to a technical cooperation trust fund. 188. IETC became operational in April 1994 and operates from two offices, one in Shiga, housed and supported by the International Lake Environmental Council (ILEC) and one in Osaka, housed and supported by the Global Environment Centre (GEC). Through these Japanese supporting organizations, the prefectures of Shiga and Osaka contribute to and support IETC. 189. The main objective of the Centre is to contribute to the transfer of environmentally sound technologies to developing countries and countries with economies in transition. The work of IETC work is organized around three sub-issues: (a) The improvement of access to information on environmentally sound technologies, through, for example, the maESTro database on such technologies and the Internet homepage of IETC; (b) The fostering of technology cooperation, partnerships and technology transfer, through, for example, publications, reports, and maESTro; (c) Capacity-building through the organization of workshops and training courses. 49 Annex I LIST OF INTERVIEWEES AND PERSONS CONTACTED Dr. John Whitelaw Mrs. M. Hoshino Mr. R. Rodriguez Mr. T. Hiraishi Mr. H. Sasaki Mr. J. Neate Mrs. L. Casanova Mr. V. Santiago Mr. H. Shiroi Mr. H. Ikeda Mr. Y. Shindo Mr. K. Jin Mr. C. Strohmann Prof. I.V. Springuel Mr. Mr. Mr. Ms. N. Miruka Mong'oni J.Ngari Kabugua J. Petrie B. Popescu Mr. D. Strom Mrs. S. Antkainen Mr. Mr. Mr. Mr. Mr. V. J. V. R. J. Gheorghievici Kerekes Plocq Fichelet Berloznik Nel Mr. M. Tuurman Mr. Ms. Mr. Mr. J. Eigen J. Aloisi de Larderel F. Balkau David Garceran Nieuwenburg Director IETC Administrative/Fund Management officer IETC Information Consultant Senior Adviser Senior Liaison Officer Consultant (Water programme) Deputy director IETC, Shiga office Programme Officer, Shiga office City of Osaka Global Issues Division, Japan Ministry of Foreign Affairs Deputy Director Global Issues Division, Japan Ministry of Foreign Affairs Deputy Director Planning Division, Global Environment Department, Environment Agency UNCHS Habitat South Valley University, Australia, EnTA Workshop Office of the President, EnTA Workshop Ngari Associates, Kenya, EnTA Workshop Australia, EnTA Workshop Romanian Water Authority, Romania, Freshwater Augmentation project State Government of Florida, United States of America IETC Insight Finnish Environment Institute, IETC Insight Director PPC, Romania, IETC Insight Environment Canada, IETC Insight Executive director SCOPE, IETC Insight VITO, Belgium, EnTA Workshop Potchefstroom University, South Africa, EnTA and EST Workshops Ministry of Environment, Estonia, EST Workshop Coordinator SCP UNEP IE/PAC, Paris UNEP IE/PAC, EnTA Workshop KLM/ Consultant to IETC, maESTro 50 Annex II LIST OF REVIEWED DOCUMENTS 1. IETC umbrella project, project action sheet, UNEP, 15 May 1996, with 10 Annexes describing the individual activities of UNEP/IETC (a scheduled annex 2 was cancelled: Annex 1: IETC project document: Pilot training programme/workshop on adopting, applying and operating ESTs for urban and lake/reservoir management; Annex 3: IETC project document: Improving Access to Information on Annex 4: Insight . IETC project document: Quarterly IETC Newsletters IETC's Annex 5: IETC project document: IETC support to Habitat II; ESTs; Annex 6: IETC project document: Basic research guidelines to best practices in implementing urban sewage programmes; Annex 7: IETC Project Document: leadership training programme; EnTA for decision-makers: a Annex 8: IETC project document: cities programme of UNCHS (Habitat); IETC assistance to the sustainable Annex 9: IETC project proposal: International sourcebook on ESTs for municipal solid waste management - follow-up project; Annex 10: IETC project proposal: Sourcebook on alternative technologies for freshwater augmentation - follow-up project. 2. IETC internal evaluation of umbrella project and annexes. 3. Report of the International Advisory Board meeting, Osaka, July 1995. 4. IETC internet homepage. 5. GEC internet homepage. 6. ILEC internet homepage. 7. Initiatives for Sustainable Development, Towards the 21st Century (Summary), Government of Japan, Ministry of Foreign Affairs. 8. Press release: Statement by Mr. Koichi Yamamoto, State Secretary for Environment at the sixth session of the Commission on Sustainable Development. 51 9. Statement by Mr. K. Tanaka, Vice-Minister, Environment Agency of Japan to UNEP Governing Council special session, Nairobi, 21 May 1998. 10. UNEP/IETC: Guide to Facilities, brochure. 11. Annual report 1996, Global Environment Centre Foundation. 12. Sustainable Cities Programme Internet Homepage. 13. Sustainable Cities Programme Newsletter, 10/14/1998. 14. European Commission DG XI Urban Environment Sustainable Cities progress report 1993-1996. 15. IETC's Insight: Issues of spring 1996, summer 1996, fall 1996, winter 1996, August 1997, December 1997, June 1998, September 1998. 16. maESTro documentation and CD-ROM. 17. Survey of information systems related to ESTs, UNEP Industry and Environment Centre, UNEP/IETC and UNEP headquarters. 18. IETC Report Series No.3: Environmental technology assessment (EnTA): A leadership training programme. 19. IETC Technical Publication Series No. ESTs for municipal solid waste management. 6: 20. IETC Technical Publication Series No 8.: technologies for freshwater augmentation. 52 International sourcebook on Sourcebook for alternative Annex III TERMS OF REFERENCE FOR THE EVALUATION OF THE UNEP/IETC UMBRELLA PROJECT (JC/3220-96-07) 1. Under the guidance of the Chief of the Evaluation Unit and in close collaboration with the UNEP/IETC Director, Deputy Director, other relevant UNEP/IETC officers and relevant UNEP officers in Nairobi, the evaluator shall undertake a detailed review and evaluation of the UNEP/IETC umbrella project. This evaluation will be conducted during the period from 10 October 1998 to 22 November 1998 (one month spread over six weeks). A. Background 2. The project to be evaluated has been being implemented by UNEP/IETC. This umbrella project consists of 10 annexes as follows. (a) Annex 1: Pilot training programme/workshop on adopting, applying and operating environmentally sound technologies (ESTs) for urban and lake reservoir management; (b) Annex 2: IETC support to the modernization of the urban management programme of Indonesia (Note: this annex was subsequently cancelled); (c) Annex 3: sound technologies; Improving access to information on environmentally (d) Annex 4: Quarterly IETC newsletter IETC's Insight; (e) Annex 5: IETC support to Habitat II; (f) Annex 6: Basic research guidelines to best practices in implementing urban sewage programme; (g) Annex 7: Environmental technology assessment (EnTA) for decision-makers: A leadership training programme; (h) Annex 8: UNCHS (Habitat); IETC assistance to the sustainable cities programme of (i) Annex 9: International source book on environmentally sound technologies for municipal solid waste management follow-up project (j) Annex 10: Source book on alternative technologies for freshwater augmentation follow-up project B. Scope of the evaluation 3. The scope of the evaluation will cover the key activities undertaken by the UNEP/IETC within this Umbrella Project. The consultant will compare the planned outputs of the project to the actual outputs and assess the actual 53 results to determine the impact of the project. The consultant will also highlight the lessons learned from the implementation of this project that would improve the implementation of future projects in the area of transfer of environmentally sound technologies (ESTs) and assess the appropriateness of this project in meeting the longer term objectives of UNEP and UNEP/IETC. 4. The evaluation should also determine to what extent the project has been successful in fulfilling its objectives and obtaining the expected results and whether it has been cost- effective in producing these results. C. 5. Terms of reference for the evaluator (consultant) The evaluator (consultant) shall; (a) Determine the appropriateness of the project objectives in relation to UNEP objectives by reviewing the basis on which the project was created. It should be determined whether the project has been able to fulfil the identified needs and to solve problems of the target groups; (b) Determine the added value of the project to the core programmes of UNEP, how this project compliments other UNEP activities in the area of transfer of ESTs and whether this project is consistent with the identified role of UNEP as catalyst in solving environmental problems; (c) Analyse the quality and usefulness of the project outputs and determine how these contribute to the attainment of results and overall objectives identified in the approved project proposal; (d) Assess the level of stakeholder involvement in the design and implementation of the project, identify the lessons learned and provide recommendations on how such involvement could be improved in future projects; (e) Assess the role of the project in building the capacity of the target groups in the area of transfer of ESTs and assess the long term sustainability of the benefits of this capacity building; (e) Assess to what extent UNEP/IETC utilized relevant ongoing and past research and operational activities of the UNEP, the scientific community and other donor institutions and indicate how such potential synergy may have been realised in implementing the project; (f) Review the adequacy of the monitoring and evaluations systems developed to supervise implementation of the project and based on the lessons learned, provide recommendations that could improve current procedures related to monitoring and evaluation; (g) Review the effectiveness of the organizational structure, management and financial systems which affected the implementation of the project. This will be done by investigating the staffing, administrative arrangements and operational mechanisms with emphasis on the co-ordination within and outside of UNEP. The evaluator (consultant) will solicit the views of relevant UNEP and UNEP/IETC staff members on the usefulness of the project in enhancing both UNEP and IETC work in the area of transfer of ESTs; 54 (h) Identify any technical and/or operational constraints encountered during project implementation including those that caused any delay in implementing the approved work plan. Examine the actions taken by UNEP to overcome those constraints, the lessons learned and discuss any appropriate alternative measures that could have been taken; (i) Identify and assess any measures that target groups have initiated to integrate the results and recommendations produced in implementing the project into relevant policy making and/or planning. The evaluator (consultant) should also make specific recommendations regarding potential measures of follow-up evaluation that would enable UNEP and UNEP/IETC to gauge the longer term benefits and sustainability of project activities; (j) Determine the extent to which gender considerations were incorporated into the various technical and operational aspects of the project; (k) Make other concrete suggestions or recommendations which may benefit future UNEP/IETC projects. 6. The evaluator (consultant) should examine the operational and technical advantages and disadvantages of the "umbrella" approach comparing it with the individual project format. The evaluation report should cite lessons learned and make specific recommendations regarding such "umbrella" projects in the future. D. 7. Evaluation report format The in-depth evaluation report should include: (a) Concise summary (about four pages) covering items (a)-(g) below; and (b) Detailed evaluation report (about 30 pages) addressing (a) through (g) below; (c) Rate of successfulness of the project on a scale from 1 to 5, with 1 being the highest rating and 5 being the lowest. 8. The following items will be considered for rating purposes: (a) Timeliness: how the project met the schedule and implementation time table cited in the project document; (b) Achievement of results/objectives; (c) Attainment of outputs; (d) Completion of activities; (e) Project executed within budget; (f) Impact created by the project; 55 (g) Sustainability. 9. Each of the items should be rated separately. system shall be applied: 1 2 3 4 5 - excellent very good good satisfactory unsatisfactory E. The following rating (90-100 per cent achievement) (75-89 per cent) (60-74 per cent) (50-59 per cent) (49 per cent and below) Schedule of the evaluation 10. The evaluation should run from 10 October to 22 November 1998 (one month spread over six weeks). As part of this evaluation, the consultant may visit UNEP/IETC in Osaka and Shiga, Japan to discuss the project with the relevant staff members of IETC. As an alternative, the consultant may interview the relevant staff members by telephone. 11. The consultant will present a draft of the final evaluation by 6 November 1998. The UNEP/IETC will provide written comments on the draft evaluation report to the consultant through the UNEP Evaluation Unit to the consultant by 16 November 1998. 12. The consultant will incorporate these comments and present the final version of the evaluation report to the UNEP Evaluation Unit in English by 22 November 1998. This report should be presented in written form and on a diskette in Wordperfect or MS Word formats. The core report should not exceed 30 pages. F. Qualifications 13. The consultant must be on the roster of experts in UNEP, have an advanced university degree in relevant discipline and should have demonstrated expertise in the area of international development assistance with special reference to environmental issues. Previous experience in the evaluation of United Nations programmes will be an advantage. The candidate should have at least 10 years of experience in the above-mentioned field or in related fields. 16 September 1998 ----- 56