Proposed Regional Technical Assistance for Capacity Building and Monitoring and Evaluation in Participating Countries Interdepartmental Comments on Draft Project Concept Paper Summary of Comments Received and Actions Taken Department Comments Received OGC I have reviewed the above referenced Concept Paper and only have one comment. In Section 8, Nature/Extent of Government/Beneficiary Involvement in Identifying or Conceptualizing the Assistance, no mention is made of the GMS area although Section 6.b. states that the GMS countries will be part of this RETA. This needs clarification, at least with regard to the plans for future consultation. EARD (Ying Qian) We reviewed concept paper of the above captioned proposed RETA with great interest. The success of managing for development results (MfDR) depends ultimately on the developing member countries (DMCs). ADB can play an important role in capacity development at the DMC level. Building capacity in the area of monitoring and evaluation (M&E) in the DMCs is a critical step toward that end. We think that the proposal is well formulated and EARD supports the proposed RETA. We have the following comments for your consideration in finalizing the concept paper: M&E of cross-border projects and programs was highlighted as an important element of the first pillar of the RCI strategy. The proposed RETA is expected to enhance the DMCs’ capacity in this area. Therefore, the “expected outcome” of the RETA should be expanded to include the development of capacity for the participating countries in monitoring and evaluating cross-border and sub-regional projects and programs. M&E of regional projects and programs is more complex than that of country-specific operations, and may involve different operational guidelines and methodological standards. The RETA should be designed to accommodate the special needs of the DMCs. RETA should consider to help set targets and/or benchmarks for M&E, and conduct training on diagnostic reviews. For better Actions Taken Department Comments Received Actions Taken results, consider having a focus for RETA's activities, such as M&E for a particular sector, or theme, or line ministries or a country's policy and strategy, as each has different priorities. In transport sector, the RETA could be used to develop capacity (tool or measure) for monitoring and evaluation focusing on cross-border infrastructure like roads and railways, which would be directly linked to capacity building for RCI. The proposed financing for this RETA is $600,000 from the PRC Regional Cooperation and Poverty Reduction fund (PRCF). We would like to bring to your attention that the PRCF has a ceiling of $500,000 for each project. Therefore, it might be necessary to either reduce the size of the TA (narrowing the project scope at the same time) or seek supplementary financing. We do not think an interdepartmental meeting to be necessary. EARD (Susanne M. Scheierling) In the Background section 6.a. the draft mentions the MfDR Action Plan, the RCI Strategy, and two regional cooperation programs (CAREC and GMS)--but it does not introduce a topic with its issues, problems, needs and current status etc. Thus the transition to section 6.b., where an "expected impact" is discussed, is less than smooth. One or two substantive paragraphs need to be added to section 6.a. Footnote 1 is confusing since section 6.a. introduced only two subregions so the reader expects further discussions on those, and not a later explanation that actually there are more subregions which should or could have been considered where it not for limitations of the funding source. This should be rearranged. Since section 6.a. provides very little background on issues, the importance of the outputs and the relevance of the activities presented in section 6.c. is not obvious. What would be the focus of the 2 Department Comments Received Actions Taken M&E activities? The regional projects? And among regional projects, infrastructure initiatives? One could get that impression from section 6.a., but it is not further discussed in the draft. (Also, a structure with (i)s and (ii)s on two levels should be avoided.) For some reason a group of activities is planned at country level (such as under Policy Advice), and another at regional level (such as under Training Program). This section further emphasizes that activities will be demand-driven, responding to the needs of the participating countries. However, it is not clear from the draft what this demand would be (or, at least, what examples of such a demand would be), and which countries would participate. Section 6.d. should not simply refer to sections 6.b. and 6.c. Section 8 is problematic. While Section 6.a., though without further specifying, gave the impression that there are issues to be addressed in CAREC and GMS countries, section 8 only indicates that there have been detailed discussions with PRC's MOF. While the PRC is an important country in both regional groups, it seems that it would be beneficial if the RETA plans and related potential initiatives were more widely discussed before the draft concept paper is further developed. It is not certain if an interdepartmental meeting could overcome this issue of seemingly limited government/beneficiary involvement in conceptualizing the RETA, but it may help in clarifying the next steps in this process. ERD We note that the proposed RETA is key to achieving the actions under the revised MfDR Action Plan, and we fully support the concept. In finalizing the concept paper you may consider the following points aimed at improving clarity of the concept paper: Discussions on the Outputs and Activities lists a number of capacity building initiatives aimed at the staff in the selected DMC governments. However, it is not clear as to whether the RETA will be targeting just the staff from oversight agencies (ministries of 3 Department Comments Received RSDD Actions Taken finance and planning agencies) or also the staff from key executing agencies. In general, the national systems in most DMCs are much better organized than the systems at the executing and implementing agencies, which in most cases are actually responsible for data generation and collection. Hence, benefits of the RETA may be limited if the targeted agencies do not include key executing and implementing agencies. Discussions under the Outputs and Activities mention that the Policy Advice and Training Programs will emphasize "peer-to-peer help". However, it is not clear as if the reference is to peers within a given DMC or peers across the DMCs. If the RETA is indeed aiming to cover executing and implementing agencies within a given DMC then the peer group may constitute stakeholders within a given DMC. However, if the RETA is only aimed at the oversight agencies and the peer group is to constitute stakeholders across the selected DMCs then you may consider making a specific provisions in the RETA for establishing and sustaining such a group during and after completion of the RETA implementation. You may wish to include in the concept paper perhaps also the reference to the Paris agenda and the newest international thinking on capacity building and country systems. I somehow also wonder why OED is doing such a TA, which is more operational relevant (monitoring) than resultevaluation oriented. For me this TA sounds more an activity more in the responsibility of either the regional departments or COSO. Sector and Subsector Classification: Please note that the staff instructions on project classification indicate that all operational activities supported by ADB need to identify one sector and corresponding subsector classification. The TA main outputs are policy advise on M&E, international workshops/seminars; and development of M&E knowledge sharing platform. As such, it should be classified under the ‘law, economic management and public policy’ sector with economic management as the subsector since this TA will 4 Department Comments Received Actions Taken help strengthen the project implementation of ADBfunded projects in GMS and CAREC. Thematic Classification: Please also note that the staff instructions on project classification indicate that up to three themes and corresponding subthemes can be identified for each project. We agree that this TA supports the thematic goals of governance through improving public governance (i.e. improved management of publicsector finance projects) and capacity development through organizational development (i.e. improving capacity of government staff to undertake highquality M&E work in their respective countries). We understand that the participants of the training activities will be coming from GMS and CAREC region. This alone, however, does not satisfy the criteria for the project to be classified under regional cooperation. The document briefly touches upon the training content which is supposed to be harmonization of regulations, procedures and standards for cross-border infrastructure. This, however, is not further explained or even mentioned in the project scope, outputs, activities, and impact. We therefore cannot see the direct contribution of this project on either one of the pillars of the regional cooperation strategy (which consists of infrastructure, money and finance, trade, and regional public goods). If you wish to add regional cooperation as one of the three priority themes supported by the TA, a clearer support for this goal should be evident in your concept paper and ensuing TA paper. Target classification : The impact of this TA on reducing both income and non-income poverty is more indirect. Thus, we suggest general intervention (GI) as the target classification. General comment: There is a TA in TIM on Infrastructure Project Management which is also being processed this year. You might wish to share your training/capacity building materials on M&E with them. 5 Department Comments Received CWRD (John Whittle) While the notion of improving the scope and ability of our DMCs to undertake effective monitoring and evaluation is necessary, the concept paper describes an overly ambitious TA that is unlikely to achieve the proposed outcome and impact. More specifically: Actions Taken the outcome and impact are overly ambitious given the duration, funding and regional coverage of the TA; the level and scope of training, country commitment, and institutional acceptance required to achieve the outcome would require more resources and time than that indicated in the paper; we also suggest that the coverage be substantially reduced to one selected region to ensure the outputs can be delivered. This could be on a pilot basis and then extended to other regions through a further TA; and some revision to the outcome and impact level objectives is required to be more realistic and achievable. 6 Department Comments Received Actions Taken CWRD (Juan Miranda) I have looked at this proposal. I read into this that either want to develop at the country level mini OEDs, or have better counterpart information and “partners”, or both. If this is the case, the story in my view is much more complicated. Each executing agency has its own (or should have) monitoring evaluation unit or team. There is no DMC level OED equivalent, except perhaps some sort of coordination function at the level of planning commissions. This is the case in Pakistan. They have a monitoring and evaluation team. I hope I do not offend by saying that this proposal will not add much value and may well confuse our clients. In my personal opinion, it also mixes too many things in an already fairly comprehensive OED mandate (although I am not as entitled to give an opinion on this front). I hope you can live with these comments. We value highly the work being done by the department, including the work that Ramesh does. It is true that we have far too many RETAs going on but this is not the main reason for these comments. 7 Department Comments Received Actions Taken SERD Thank you for the opportunity to review and comment on the draft concept paper. I would like to offer a few simple comments for your consideration. The paper would benefit from a clearer explanation on why this RETA is needed. It is noted that the RCI recognized the importance of the M&E of cross-border infrastructure projects and programs. The background section could briefly mention the present status of the M&E system in participating countries for cross-border infrastructure projects/programs, and how insufficient M&E capacity is affecting the level of development impact expected from these projects. It is not very clear if the RETA will support all participating countries of the GMS and CAREC programs. I assume the level of M&E capacity may vary in these countries. Is it considering under the RETA for more advanced countries to share their system and experience with others? 8