Type of Review: Annual Review Project Title: PRO-ROUTES Date started: 01 March2008 Date review undertaken: 11th August 2012 Introduction and Context What support is the UK providing? The UK is providing £76 million to the World Bank-managed multi-donor trust fund ‘High priority roads reopening and maintenance project (ProRoutes)’. What are the expected results? The headline results as set out in the World Bank’s project document are: approximately 2,900 km of the high priority road network re-opened and maintained; better access to social and administrative services and markets for people residing in the zone of influence of the roads, leading to a better standard of living and reduced rates of poverty; improved operational capacity within the Ministry of Infrastructure Public Works and Reconstruction (MITPR) and [the national roads agency] Office des Routes (OdR) for management and maintenance of the road sector; all season access on the rehabilitated roads; resurgence of local small and medium size contractors for road rehabilitation and maintenance; strengthened coordination and harmonisation in the road sector between the Government, donors and other partners, and within the donor community; and full integration of environmental and social impact management in the road sector, resulting in better socio-economic outcomes for road-side populations and reduced environmental impact. The intended impact is to, “Reduce poverty by establishing lasting access to economic and social services.” The outcome is to, “Re-establish lasting road access between provincial capitals and districts and territories in four provinces in a way that is sustainable for people and the natural environment.” The three outputs are, “Strategic roads built, upgraded, rehabilitated and maintained,” “Operational capacity for road construction, management and maintenance within the Ministry of Public Works and Infrastructure, Office des Routes and the private sector strengthened,” and “Social and environmental impacts managed.” What is the context in which UK support is provided? The DRC has a road network of 152,400 km which was largely developed during the colonial period in the 1950s. With the deterioration of the political and social situation of the country, the Congolese administration was unable to maintain the network. This brought about the collapse of the priority road network of which, at the start of the programme, 95 % was impassable. 1 The lack of transport infrastructure hampers access to basic services and to markets, and therefore limits MDG progress and economic growth. ProRoutes is currently designed as a two-phase programme. The first phase was focused on the build/maintenance of two roads as well as institutional capacity building and management of environmental and social impacts. It was originally intended to run from 2008 to 2013. The second phase added two further roads, and was approved in 2011. The end date for the new phase is 2016, and the end date of Phase 1 was brought into line with this. Section A: Detailed Output Scoring Output 1: Strategic roads built, upgraded, rehabilitated and maintained Output 1 score and performance description: C. The number of kilometres built to date is only one-third of the milestone. Only 7.5% of the milestone for maintenance has taken place. No system for routine maintenance is yet in place so the percentage of the reopened roads in good condition is likely to diminish soon if action is not taken. Progress against expected results: Expected 2012 results are: Cumulative length of roads built and upgraded: 900 km Length of roads maintained and rehabilitated in each year: 930 km Percentage of the reopened roads in good to fair condition: 95% Only 366 km of roads have been built against the target of 900 km, and only 70 km of roads are being maintained. 100% of the reopened roads were reported as being in good to fair condition in 2011, though World Bank and DFID missions have since seen some degradation and the Cellule Infrastructures is now tendering contracts for catch-up maintenance to rectify the problem. Recommendations: The World Bank mid-term review has recommended that the anticipated gap between what OdR should have delivered by the end of the programme and what they will deliver be filled by issuing further private sector contracts. We agree with this course of action. The tendering process is underway for private sector construction and maintenance contracts and contracts should be let by August 2012. We will keep a close eye on this process to ensure that rapid progress is made. Impact Weighting (%): 60% Revised since last Annual Review? No Risk: High Revised since last Annual Review? No 2 Output 2: Operational capacity for road construction, management and maintenance within the Ministry of Public Works and Infrastructure, Office des Routes and the private sector strengthened Output 2 score and performance description: A. Although FONER is functioning and has provided its funding and the amount of road works contracted to the private sector has exceeded expectations, the ability of FONER to maintain priority roads is still untested and private sector engagement in ProRoutes is weak. Moreover, the private sector development strategy is not yet being implemented, and may need further improvements. Donor coordination remains weak and the drafting of the national roads strategy and investment plan is underway but further work will be required to ensure broad buy-in from donors and investors. Progress against expected results: Expected 2012 results are: Road Maintenance Fund exists and is operational: FONER provides first tranche of funding for the maintenance of ProRoutes roads and is financing other priority national and provincial roads Annual US$ amount of road works contracted out to the private sector through the Ministry of Public Works and Infrastructure: $48m Implementation of an action plan to develop the road construction industry: 60% Sound road transport strategy and roads investment plan completed: A functioning Transport Task Force and Groupe Thématique exist and a synthesis of current and future roads investments exists, plus a national roads strategy and investment plan exist Percentage of total donor finance for the roads sector that goes to projects managed by the Cellule Infrastructure: 50% The road maintenance fund (FONER) has been in place since 2009, and has deposited the first tranche of its ProRoutes finance. FONER is financing other priority roads and has signed memoranda of understanding in North and South Kivu to this effect. The FONER operational manual is about to be finalised. Figures have not yet been provided for the amount of road works contracted out to the private sector, but in 2011 this figure considerably exceeded the target at £167m. The action plan for the private sector has been drafted, though DFID DRC believes it may need further strengthening to represent a true strategy. As of the end of 2011 it was judged to have been 10% implemented. The donor coordination group (Groupe Thématique) for the transport and infrastructure sector has been dormant for the last year, but was reinvigorated in July 2012 with a ministerial meeting. The first stage of the roads strategy (a diagnostic) has been produced, and a draft of the full strategy and investment plan was due by June 2012. This has not yet been produced and there is likely to be further work to do to ensure it has buy-in from all stakeholders in the sector. Awaiting figures on % of project spend managed by CI – Information still not available – we are chasing CI to get this 3 Recommendations: We will need to engage closely on the development of the national roads strategy and investment plan to ensure that it is not just a consultant’s report, but is a strategic plan behind which all traditional and emerging donors and other investors align themselves. DFID DRC’s Growth and Economic Advisor will review the private sector strategy and recommend whether improvements and/or further research are required. Impact Weighting (%): 25% Revised since last Annual Review? No Risk: High Revised since last Annual Review? No Output 3: Social and environmental impacts managed Output 3 score and performance description: B. There has been progress against all indicators, though all but possibly one are off track. Other significant progress not captured by the indicators has been made, however, including the signing of provincial decrees to set up environmental checkpoints to assess the volume and legality of timber and bushmeat trade. Progress against expected results: Expected 2012 results are: Effectiveness of the management of protected areas, averaged over the three areas in the programme zones: 31 Number of new protected areas created: 1 Number of Ministry of Environment and Congolese Wildlife Authority staff trained in implementing laws and accompanying local initiatives: 285 Number of local environmental management plans developed with local communities and NGOs for participatory natural resources management: 5 Condoms distributed: 4.2m The management effectiveness score, done by an environmental and social management consultant with ICCN using the Management Effectiveness Tracking Tool methodology (http://www.wdpa.org/ME/tools.aspx) for the protected areas was 21 at the end of 2011, meeting the 2011 target. 2012 data are not yet available. As of the end of 2011, no new protected areas had been created. It is unlikely this will take place in 2012. 142 staff were trained by 2011 against a target of 204. Although 2012 data are not yet available, it is unlikely this target will be achieved. There were no local environmental management plans as of the end of 2011 so this milestone is unlikely to be met. 760,000 condoms were distributed by 2011 against a target of 2.1m, due to a lack of condoms 4 being supplied and a delay in training local NGOs to distribute them. It is therefore unlikely that this target will be met. Recommendations: The DFID mission in May 2012 had some concerns about whether the environmental and social management plan was focused on the right issues, or the issues where the contractor has a comparative advantage. We will attend their next monthly meeting to discuss refocusing their efforts. Impact Weighting (%): 15% Revised since last Annual Review? No Risk: High Revised since last Annual Review? No Section B: Results and Value for Money 1. Progress and results 1.1 Has the logframe been updated since last review? No. The World Bank had agreed to review the logframe with DFID as part of their mid-term review, but this has not yet taken place. DFID has requested a number of additional impactlevel indicators as the World Bank logframe did not contain much at this level, but these have not yet been formalised and data are not yet being collected. 1.2 Overall Output Score and Description: B. Progress has been slower than anticipated across all three outputs. Construction and maintenance – the main focus of the programme – is the area that is most off track. A contingency plan has been put in place by the World Bank to address this, and we expect to see some improvement by the next annual review. This plan involves shifting a significant amount of the works originally envisaged for Office des Routes to the private sector if improvements have not been seen by December 2012. It also involves letting contracts for ‘catch-up’ maintenance works to avoid further degradation to the roads that have already been re-opened. We have some influence over the output on sector coordination and will focus our efforts here. The environmental and social management plan is now beginning to deliver but we will engage to ensure the plan is focused on the right issues. 1.3 Direct feedback from beneficiaries During DFID’s May 2012 visit we undertook a brief survey in the village of Musakité of economic impacts that gave an early indication that, where re-opening has taken place, the development impacts of the roads are significant. Before the ProRoutes road reached Musakité transport was only possible by motorbike on a rough trail. Now access to 4x2 traffic has been restored. Travel times to the main town, Kalemie, have reduced from 12 hours to 3 hours, and prices of basic commodities have halved. There was clear evidence of significant 5 indirect job creation. The number of small businesses had increased from two before the road’s reopening to 17 after the road had reached the settlement. In Kalemie the Commissioner for Tanganyika District reported that conflict had reduced in the areas where the road had reached. This was corroborated by interviews in the project area which reported that security had improved. Reasons given were improved communication and improved access for security providers. A formal evaluation would better assess and record the development impacts from the roads and we will consider how best to deliver this. 1.4 Summary of overall progress ProRoutes is considerably behind time. Only 20% of the road construction has been delivered although 70% of the original project period has elapsed. Environmental and social management is two years behind schedule. Sector coordination, private sector engagement and institutional capacity all remain weak. 1.5 Key challenges Delivery of construction works by OdR and the procurement of contracts remains a significant challenge. Construction is considerably off track, and OdR has proved itself unwilling or unable to let contracts and this has resulted in a lack of maintenance on finished sections of road and therefore some degradation which will cost more than budgeted to rectify, impacting the programme’s value for money. OdR has traditionally operated as a direct works organisation, and there are therefore institutional disincentives around letting works to the private sector. The World Bank has proposed a robust contingency plan to address this, which will shift works from Office des Routes to the private sector if significant improvements have not been demonstrated by December 2012. So far this approach seems to be bearing fruit, and during DFID’s visit to Route Nationale 5 in May 2012 we saw that progress had significantly sped up. Furthermore, tendering processes are underway to contract the first private sector firms, including to undertake catch-up maintenance works. There are specific concerns around the management of road safety in ProRoutes. During DFID’s May 2012 visit, there was no evidence of traffic calming measures near settlements or hazards, and no warning signage near schools. Several bridges had no barriers either on the approach or as part of the bridge structure. Vehicle accidents were already reported. It is worrying that a recent independent technical audit (World Bank, ProRoutes Mid-Term Review, May 2012) did not raise these issues. The World Bank and the Cellule Infrastructures responded to our comments on road safety and are putting together an action plan. We will review the action plan to ensure it is robust. 1.6 Annual Outcome Assessment The outcome indicators and milestones for 2012 are: Number of days/year when ProRoutes roads are not passable by 4x2 vehicles along the full length that has been built, upgraded, maintained or rehabilitated: milestone 67, actual 0 in 2011 (A+) Percentage increase of daily freight traffic over the baseline, averaged over the four project roads: milestone 31%, actual 26% in 2011 (B) 6 Share of total rural population in ProRoutes areas that has access to an all-season road: milestone 3.2%, no data yet available (no score) Number of people living less than 2km from ProRoutes roads: milestone 290,000, no data yet available (no score) Illegal timber as a percentage of total artisanal and industrial timber found in checks on ProRoutes roads, averaged across the four roads: milestone 78%, actual 87% in 2011 (B) Protected species as a percentage of total bushmeat found in checks on ProRoutes roads, averaged across the four roads: milestone 83%, actual 84% in 2011 (B) Average percentage of the population living along the project roads who visit a health centre each year: milestone 50%, actual 35% in 2011 (B) Annual percentage of children of school age who complete the school year, disaggregated by gender: milestone 46%, 33%, 33%, 23% in Orientale, South Kivu, Katanga and Equateur respectively; actuals 52% (Boys: 53%, Girls: 50%), 26% (Boys: 30%, Girls: 24%), 30% (Boys: 34%, Girls: 25%), 23% (No data in 2011) (B) Percentage of commercial sex workers reporting practicing safe sex in selected communities: milestone 24%, actual 14% (B) Percentage of sexually active people reporting use of condom during their last intercourse in selected communities: milestone 16%, actual 17% (A) Progress against the indicators is very mixed, scoring a B overall. This can be put down to slower progress than anticipated on road construction and environmental and social management, meaning that the impact of the road and environmental and social activities is as yet difficult to measure, and to the baselines being set and data being collected only recently. The World Bank’s mid-term review lowered the programme’s performance rating from ‘moderately satisfactory’ to ‘moderately unsatisfactory’ given the continued slow progress. This triggered the development of a robust action plan to improvement programme performance. The World Bank estimates that despite the programme’s current rating, it will reach a ‘moderately satisfactory’ rating by the end of the programme. 2. Costs and timescale 2.1 Is the project on-track against financial forecasts: No. Only 28% of Phase 1 funds have been spent to date although 70% of the original project duration has elapsed. 2.2 Key cost drivers The key cost drivers are labour, materials, equipment, fuel and management of environmental and social impacts. Prices have been driven up by the failure to let maintenance contracts in time to hand over the works. Contracts are currently being let to deliver ‘catch-up’ maintenance works but these will be more costly than they would have if maintenance had been undertaken as soon as each section of road was reopened. 2.3 Is the project on-track against original timescale: No. Only 20% of the original project has been delivered in 70% of the original timescale. The World Bank extended the project end date to 2016 as a result. 7 3. Evidence and Evaluation 3.1 Assess any changes in evidence and implications for the project Slow progress in the programme is partly due to the failure of Office des Routes to issue more than a handful of contracts. The programme was designed on the basis that an increasing amount of the works would be done by the private sector over the course of the programme, but the assumptions behind this about the willingness and ability of Office des Routes appear to be faulty. While the World Bank has put in place contingency mechanisms to address this problem by asking the Cellule Infrastructures to take on an increasing amount of the procurement for the programme, there remains a significant lack of understanding of the political economy issues that affect progress in the roads sector. We will therefore undertake a political economy analysis of either the roads sector as a whole, or the programme itself. We will discuss with the World Bank whether this can be done within the existing programme, or whether this will need to be a DFID-specific initiative. 3.2 Where an evaluation is planned what progress has been made? The World Bank will undertake an evaluation at the end of the programme. There is little information on this in the PAD, which proposes an impact evaluation but describes a monitoring system (see paragraph copied below). We will therefore consider whether DFID needs to fund additional evaluation, and of what kind. “The project will also finance a comprehensive evaluation of the impact of its activities and the performance of its implementation. The evaluation will be used to create a results framework…, complemented by a set of other indicators, in particular to measure the impact of the reopened roads on household welfare through the household income and expenditure survey (HIES), which will be conducted in selected sites using a purposive sample set. The HIES will be implemented before road works start and will be repeated after the completion of road works and periodically thereafter. It will also measure the project’s impact on selected road segments, including (i) traffic count and mix, freight volumes, trip distances, and fares; and (ii) the economic and social conditions in selected sites in the areas of influence of the project, including change in prices of commodities, nature of goods traded, marketing and provisioning conditions for agricultural and livestock produces, mobility to access employment opportunities, and opportunities to boost informal and micro-businesses. Eventually, HIES will monitor closely the evolution of conflicts along the roads reopened by the project.” 4. Risk 4.1 Output Risk Rating: The risk rating is high for all three outputs. 4.2 Assessment of the risk level The risks to programme delivery remain high. The programme is significantly behind time in terms of construction and maintenance. Although the World Bank has put a strong contingency plan in place (as discussed in Section A, Output 1 and paragraphs 1.2 and 1.5), the risk remains that contracts are not let, or are not delivered effectively. In terms of institutional capacity building, sector coordination remains weak, although the donor 8 coordination group has recently been reinvigorated. The risks that demand for maintenance funding from the national road fund outstrip available funding and capacity remain. The risk that the private sector does not have the capacity to deliver also remains, and the existing private sector development workplan may not be sufficient to build capacity. DFID DRC’s Growth and Economic Adviser is reviewing the workplan to propose improvements if necessary. The third output covers environmental and social impacts and is covered in 4.4 below. An additional risk that we currently face is that the Cellule Infrastructures is currently without a head, and the initial recruitment process was unsuccessful. A lack of capacity in the Cellule Infrastructures at a time when ProRoutes performance needs to be turned around carries a significant risk to the progress of the programme. A second recruitment phase is underway to address this. 4.3 Risk of funds not being used as intended The risk of the misuse of funds is high in the construction sector and there is an opportunity for more disclosure in the ProRoutes procurement process which could have a positive knock-on effect on the sector more broadly. We will consider undertaking a Construction Sector Transparency initiative (COST) pilot. We will also ask the World Bank to consult us on the drafting of terms of reference for all technical, financial and environmental audits. 4.4 Climate and Environment Risk One of the ProRoutes outputs, and 15% of the budget, are focused on the management of environmental and social impacts. A full environmental and social impact assessment was undertaken and a company (SOFRECO) was contracted to manage this element of the programme. Although they started slowly and their work is significantly behind track, considerable progress has been made recently, as set out in section A, output 3. The risk remains high and we will review the social and environmental management plan as part of DFID DRC’s Strategic Programme Review on climate change. DFID’s visit to Route Nationale 5 in May 2012 showed that SOFRECO’s focus was on the reduction of HIV risks and the inclusion of vulnerable groups in the benefits of the road. Some of their project activities include school building. However, this is not an activity for which SOFRECO has a comparative advantage. Other areas, such as community resource management related to charcoal production, are neglected. We therefore recommended that SOFRECO realign its activities, to include a greater emphasis on road safety and local environmental impacts including charcoal production. 5. Value for Money 5.1 Performance on VfM measures The World Bank’s project document and a subsequent update estimated that the cost to build and maintain a kilometre of road was likely to be around $44,000/km for the two roads on which works have begun. To date the costs are around $20,000/km, but it is unclear whether the same cost elements are included in these calculations, and since maintenance has not yet begun they do not include maintenance costs. It is therefore as yet difficult to assess VfM, and we will request further work on this by the World Bank. We will ask the World Bank to provide detail on how the $44,000/km and $20,000/km figures are calculated, so that we have indicators for construction and a maintenance cost schedule with which to track VfM in the 9 programme. Although economic rates of return (ERRs) were not estimated for the programme roads in the World Bank’s project document, a range of low to high scenario rates of return were estimated for other roads in DRC. Now that traffic data is available for the ProRoutes roads, we will ask the World Bank to calculate ERRs for each road using the methodology used in the project document. 5.2 Commercial Improvement and Value for Money Once private sector contracts have been let in the coming months we will be able to comment on commercial issues. 5.3 Role of project partners While ProRoutes is a Government of DRC programme led by a project implementation unit, the World Bank is managing DFID’s finance through a multi-donor trust fund. The World Bank’s communications with DFID remain weak. The monthly Consultative Group meeting between the Cellule Infrastructures, the World Bank and DFID, instituted in June 2011, has only been convened once. The timing of the mid-term review was fixed without reference to DFID’s needs, and the terms of reference for the review were not developed in consultation with DFID. Although none of these issues have broken the letter of the administrative arrangement between DFID and the World Bank, such inflexibility is not conducive to an effective partnership. We would like to ensure that the principles of our engagement are consultation and flexibility as we try to address ProRoutes’ challenges together. Given ProRoutes’ performance challenges, adherence to the spirit rather than the letter of our administrative arrangement is crucial. The World Bank’s mid-term review did not reflect on these issues or set out how the World Bank will deliver improvements, despite DFID’s requests. A further issue lies in the delays related to World Bank procedures. Works on Phase 2 of the programme have still not begun, although the additional finance was agreed by the World Bank board in July 2011. This will have a significant impact on the delivery of ProRoutes. Responses to non-objection requests that require further information also lead to numerous smaller delays. A less detailed, more strategic, ‘good enough’ approach is desirable in order to achieve the much greater prize of improved ProRoutes delivery. Finally, a more strategic, consultative approach to problem-solving could have a significant impact on ProRoutes delivery. The head of the ProRoutes project implementation unit left at the end of his contract, but due to poor management of the process no transitional arrangements were put in place. The World Bank is in a position to play a brokering, negotiating role in political issues such as this, and must do so in order to effectively manage the multi-donor trust fund. The performance of the Cellule Infrastructures, Office des Routes, FONER and SOFRECO is addressed in section A. 10 5.4 Does the project still represent Value for Money It is not yet possible to assess whether the programme represents value for money. However, the mid-term review identified that 20% of the construction output has been delivered with 28% spend. We would expect costs to be front-loaded due to the fixed costs of procurement of equipment and mobilisation of brigades. However, if spending versus delivery continues on this trajectory we will need to consider whether the programme represents VfM. The achievement of value for money depends on the results being delivered within the agreed timeframe. Benefits deferred, even at the same unit cost, impact on the rate of return. 5.5 If not, what action will you take? We will continue to review the relative proportions of progress and spend. As set out above, we will require further work on VfM metrics from the World Bank in order to assess in the 2013 annual review whether the programme represents value for money. Nevertheless, there are no immediate indications that make continuation of the project questionable on VfM grounds. 6. Conditionality 6.1 Update on specific conditions n/a 7. Conclusions and actions ProRoutes is currently off track and we therefore need to put a performance improvement plan in place. The World Bank has set out a robust plan for improving performance at the output level in the mid-term review. It is important for DFID that we go beyond delivering the project outputs to ensure that ProRoutes influences the roads sector in DRC. This will involve a much deeper understanding of dynamics and vested interests in the sector to understand why progress has been so slow. We will therefore work with the World Bank on an action plan, to be agreed between the head of DFID DRC and the head of the World Bank DRC, that goes above and beyond the mid-term review to ensure that ProRoutes provides strategic direction to the sector. Specifically, we will: Discuss with the World Bank, as manager of the multi-donor trust fund, how they can take on a brokering, negotiating role to ensure that ProRoutes leverages change in the sector. Undertake a political economy analysis of either the roads sector as a whole, or the programme itself. We will discuss with the World Bank whether this can be done within the existing programme, or whether this will need to be a DFID-specific initiative. Engage closely on the development of the national roads strategy and investment plan to ensure that it is not just a consultant’s report, but is a strategic plan behind which all traditional and emerging donors and other investors align themselves. Review the private sector strategy and recommend whether improvements and/or 11 further research are required. Focus our efforts on the output on sector coordination, over which DFID has some influence, including through active engagement in the Groupe Thématique. The following programme-focused recommendations will be followed up as a result of the annual review: Finalise logframe, including agreeing and beginning data collection on impact-level indicators. Consider the best approach to and design of an evaluation to assess and record the development impacts of the roads. Review the road safety action plan to ensure it is robust. Review the social and environmental management plan as part of DFID DRC’s Strategic Programme Review on climate change. Follow up with SOFRECO on our recommendation that they realign their activities, to include a greater emphasis on road safety and local environmental impacts including charcoal production. Ask the World Bank to develop a unit costs methodology that allows actual costs to be compared with estimated costs as set out in the original project document and the project document for the additional finance. We will also require a six-monthly report on unit costs. Ask the World Bank to calculate ERRs for each road going forward using the methodology presented in the World Bank’s project document. Consider undertaking a Construction Sector Transparency initiative (COST) pilot. Ask the World Bank to consult us on the drafting of terms of reference for all technical, financial and environmental audits. We will ask the World Bank to provide detail on how the $44,000/km and $20,000/km figures are calculated, so that we have indicators for construction and a maintenance cost schedule with which to track VfM in the programme. 8. Review Process The World Bank undertook a mid-term review of ProRoutes in March/April 2012, visiting both Phase 1 roads and meeting a range of stakeholders in Kinshasa. DFID undertook a mission to Route Nationale 5 in May 2012 to complement the World Bank mid-term review mission, in which we did not participate. The annual review draws on the World Bank’s mid-term review report, the full range of ProRoutes internal and external technical, financial and environmental and social audits, and the gap analysis undertaken by the Infrastructure Advisor from DFID’s Growth and Resilience Department. 12