Lessons from ex-post benefit-cost analysis of road projects in Norway James Odeck, Professor NTNU / Norwegian Public Roads Administration Norway The 5th Concept Symposium on Project Governance Valuing the Future - Public Investments and Social Return 20. – 21. September 2012 Symposium web-site: http://www.conceptsymposium.no/ Concept Research Programme: http://www.concept.ntnu.no/english/ Lessons from ex‐post benefit‐cost analyses of road projects in Norway James Odeck NPRA/NTNU Objectives To reveal the extent to which predictions of benefit‐cost analyses delivered to the decision makers matches outturns. In particular, we ask: 1. Do the road authorities achieve their stated road program benefits? 2. What are the magnitude of deviation between forecasts and outturns 3. What causes the deviations? 2 Background 1. The Norwegian Public Roads administration (NPRA) carries out BCA for its trunk road schemes 2. Based on information from the BCA, decision makers may prioritize projects. 3. The BCAs are conducted ex ante; therefore there is little knowledge as: (i) to what degree the predictions are correct (ii) whether the project fulfil their objectives and, (iii) whether the decision makers are supplied with appropriate information ex ante. We address these questions based on ex‐post assessments conducted since 2006 when the Ministry of Transport sanctioned such studies. 20 projects are studied to date. 3 Framework for analyses • 3 ‐ 5 projects are ex‐post evaluated every year • Requirement: • Investment cost > 200 mill NOK • Has been open for traffic in at least 5 years • The projects are chosen the year of the ex‐post assessment • The baseline for ex‐post assessments are the assessments • presented to the decision makers when the decision to fund was made (the go‐ahead decision) Focus on monetised impacts only 4 To underline what we do! We re‐do the benefit‐ analyses to explore whether the projected outturns deviate from real outcomes 5 Which estimates are compared to outturns? Project development process Project Planning Municipal External Quality Master Assurance(EQA) Plan(EIA) Project execution Zoning Plan Operations Construction construction plan 1 2 3 4 Time (Year) Actual cost Decision to build “approved BCA estimates” Outturn Different estimates 6 Compare 5 Impacts included in an Impact Assessment Cost benefit analysis Ex post evaluation Monetised impacts Travel time savings Non monetised Visual landscape Vehicle operating costs Accident costs Induced traffic Inconvenience cost (ferry) Cultural heritage Noise and local air pollution Natural resources Road maintenance costs Residual value of capital Cost of public funds Road investment costs Community life effects and outdoor recreation Natural environment Precautions taken ! Appraisal techniques develop over time in some cases causing the tools to change from before‐ to the time of ex‐ post assessment…. We use the same software package when comparing the before and ex‐post assessments The analysis period of the projects is 25 years, but the ex‐ post assessments are conducted 5 years after opening….. The same traffic forecasts are used in the ex‐post assessment as in the original analysis after year 5 8 Approach and data collection 1. Acquire project overview, project history and project development over time 2. Retrieve the database which was the basis for CBA presented to the decision makers before the go ahead‐ decision 3. Possibly redo the original CBA with a newer version of the CBA software (EFFEKT) Substitute the outturn data into the CBA in 3 4. 5. Traffic Accidents Construction costs Average speed Compare the before‐decision analysis (3) and the ex‐post 9 assessment (4)and explain differences Projects evaluated No Project 1 Rv.23 Oslofjordforbindelsen Project type Strait crossing; sub sea tunnel 2 E18 Rannekleiv - Temse Trunk road ; bypass 3 Rv.714 Hitra - Frøya Strait crossing; sub sea tunnel 4 E18 Teigeland - Håland Trunk road, mainly tunnel 5 Rv.62 Øksendalstunnelen Trunk road, 6 E8 Nordkjosbotn - Laksvatnbukt Trunk road 7 E18 Gutu - Helland - Kopstad Trunk road; 8 E39 Kleivedammen - Andenes Trunk road 9 E134 Hegstad - Damåsen Trunk road; bypass Darbu… 10 Rv.616 Kolset - Klubben Strait crossing; sub sea tunnel Project objective Reduce user costs, accidents and improve local environment Reduce user costs, accidents and improve local environment Reduce user costs, regional effects Reduce user costs, reduce risk of avalanche/land slide mainly tunnel Reduce risk of avalanche/land slide, reduce user costs Reduce accidents, improve local environment and reduce user costs bypass Sande and HolmestrReduce accidents, reduce user costs and improve local environment Reduce user costs, reduce accidents 10 Reduce user costs, reduce accidents and improve local environment Reduce user costs Projects cont. 11 Rv.580 Hop- Midttun 12 E6: Akershus grense - Patterød Trunk road within city; bypass Nestun Reduce user costs, improve local environment Trunk road Reduce user costs, accident 13 E18 Ørje - Eidsberg Trunk road Reduce user costs, accidents 14 Rv35 Grualia - Kneppe Trunk road, "new link" 15 E6 Halmstad - Patterød Trunk road Improve acessability to Gardermoen ariport, regional development Reduce user costs and accidents 16 E18 Brokelandsheia - Vinterkjær Trunk road; Bypass Søndeled 17 E39 Svegatjørn - Moberg Trunk Road; Bypass Osøyro 18 E18 Sekkelsten- Krosby Trunk Road; Bypass Askim 19 E6 Ny Svinesundforbindelse Trunk Road 20 E6 Skjerdingstad - Jaktøyen Trunk Road; Bypass Melhus? 11 Reduce accidents, user costs, improve local environment Reduce user costs, accidents and improve local environment, improve situation for Reduce user costs, accidents, improve local enironment Reduce accidents, reduce user costs especially for goods transport Reduce accidents, improve local environment, and reduce user costs RESULTS (1a): Traffic in the opening year (AADT) Deviation No Project Estimates Actual Estimates/Actual 1 2 3 Rv. 23 Oslofjordforbindelsen E18 Rannekleiv - Temse Rv. 714 Hitra - Frøya 4240 8232 353 3780 10242 512 ‐11 % 24 % 45 % 4 E134 Teigeland - Håland 1000 1367 37 % 5 6 Rv. 62 Øksendalstunnellen E8 Norkjosbotn-Laksvatnbukt 1386 2300 1345 2400 ‐3 % 4% 7 8 E18 Gutu-Helland-Kopstad E39 Kleivedammen-Andenes 12000 686 16700 924 39 % 35 % 9 E134 Hegstad - Damåsen 6740 8050 19 % 10 Rv.616 Kolset - Klubben 88 155 76 % 11 12 Rv.580 Hop‐ Midttun E18 Ørje - Eidsberg grense 19300 3288 14000 4488 ‐27 % 36 % 13 14 15 E6 Akershus grense ‐ Patterød Rv.35 Lunner ‐ Gardermoen E6 Halmstad – Patterød 21750 2354 24613 20700 1600 26954 ‐5 % ‐32 % 10 % 16 17 E18 Brokelandsheia ‐ Vinterkjær E39 Svegatjørn - Moberg 6814 2700 6018 5000 ‐12 % 85 % 18 E18 Sekkelsten- Krosby 5200 8000 54 % 19 E6 Ny Svinesundforbindelse 10282 11406 11 % 20 E6 Skjerdingstad - Jaktøyen 9100 9900 9% 12 RESULTS(1b): No Project Average traffic growth rate, the first 5 year Estimates Actual 1 Rv. 23 Oslofjordforbindelsen 1.4 % 6.8 % 2 3 E18 Rannekleiv - Temse Rv. 714 Hitra - Frøya 1.2 % 1.2 % 3.2 % 18.3 % 4 5 6 E134 Teigeland - Håland Rv. 62 Øksendalstunnellen E8 Norkjosbotn-Laksvatnbukt 1.2 % 1.0 % 1.1 % 2.3 % 5.5 % 3.7 % 7 E18 Gutu-Helland-Kopstad 1.2 % 3.0 % 8 9 E39 Kleivedammen-Andenes E134 Hegstad - Damåsen 1.0 % 3.9 % 1.2 % 2.5 % 10 11 Rv.616 Kolset - Klubben Rv.580 Hop- Midttun 1.4 % 1.7 % 1,6 % 12 13 E18 Ørje - Eidsberg grense E6 Akershus grense - Patterød 2.0 % 2.0 % 5.6 % 14 Rv.35 Lunner - Gardermoen 2.0 % 12.1 % 15 16 E6 Halmstad – Patterød E18 Brokelandsheia - Vinterkjær 2.0 % 2.2 % 4.4 % 17 18 19 20 E39 Svegatjørn - Moberg E18 Sekkelsten- Krosby E6 Ny Svinesundforbindelse E6 Skjerdingstad - Jaktøyen 1.7 % 2.0 % 13 2.0 % 1.4 % 4.1 % 3.2 % 5.7 % 2.6 % 3.4 % 3.2 % ‐0.3 % Results (2): Construction costs Project no. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Project name Rv 23 Oslofjordforbindelsen Ev 18 Rannekleiv - Temse Rv 714 Hitra - Frøya Ev 134 Teigeland - Håland Rv 62 Øksendalstunnellen E8 Norkjosbotn-Laksvatnbukt E18 Gutu-Helland-Kopstad E39 Kleivedammen-Andenes E134 Hegstad - Damåsen Rv.616 Kolset - Klubben Rv.580 Hop- Midttun E18 Ørje- Eidsberg grense E6 Akershus grense- Patterød Rv. 35 Lunner - Gardermoen E6 Halmstad - Patterød E18 Brokelandsheia - Vinterkjær E39 Svegatjørn - Moberg E18 Sekkelsten- Krosby E6 Ny Svinesundforbindelse E6 Skjerdingstad - Jaktøyen Construction costs(Mill. NOK) Estimates Actual 1560 215 589 437 215 242 325 195 262 254 322 252 296 681 295 507 248 506 14 847 678 1370 263 460 385 218 346 314 183 249 336 372 246 302 780 302 425 211 520 849 583 Deviation (Estimates/Actual) -12 % 22 % -22 % -12 % 1% 43 % -3 % -6 % -5 % 32 % 16 % -2 % 2% 15 % 2% -16 % -15 % 3% 0% -14 % Were the projects built as presented to the decision makers? Yes Yes/No No All others Oslofjord link; Only first stage built. Second stage (twin tube tunnel) was not built. Coming soon? Hitra-Frøya: Connecting road was not built; planning difficulties Teigeland- Håland 1,75 km not built to avoid cost overrun Results (3): Accident costs Reduction in accidents with personal injury, opening year 20 No 1 Project Rv. 23 Oslofjordforbinde Estimates 8.6 Actual 7.8 Deviation% -9 % 2 3 4 5 6 7 8 9 10 11 E18 Rannekleiv - Temse Rv. 714 Hitra - Frøya E134 Teigeland - Håland Rv. 62 Øksendalstunnelle E8 Norkjosbotn-Laksvatn E18 Gutu-Helland-Kopstad E39 Kleivedammen-Ande E134 Hegstad - Damåsen Rv.616 Kolset - Klubben Rv.580 Hop- Midttun 6.1 -0.2 0.0 1.8 4.0 4.0 1.1 2.1 -1.5 4.6 8.1 -0.2 0.7 1.9 3.7 14.2 1.9 4.3 -1.9 0.7 5% -6 % 252 % 69 % 105 % 24 % -85 % 12 13 14 15 16 17 18 19 E18 Ørje - Eidsberg gren E6 Akershus grense- Patte Rv.35 Lunner- Gardermoe E6 Halmstad – Patterød E18 Brokelandsheia - Vin E39 Svegatjørn - Moberg E18 Sekkelsten- Krosby E6 Ny Svinesundforbinde E6 Skjerdingstad - Jaktøy 3.8 3.1 3.7 4.3 7.4 0.7 2.8 6.3 2.7 5.4 2.4 4.0 0.8 7.6 1.4 6.0 6.4 5.1 42 % -23 % 10 % -82 % 3% 100 % 114 % 2% 89 % 33 % 0% Results 4: Cost Benefit Analysis (NPV; mill NOK) Estimates 4563 NPV Actual 4565 Ev 18 Rannekleiv - Temse Rv 714 Hitra - Frøya Ev 134 Teigeland - Håland Rv 62 Øksendalstunnellen 409 -242 -445 57 498 156 -418 67 22 % -165 % -6 % 18 % 1.2 -0.9 -0.8 0.2 1.1 0.9 -0.7 0.2 E8 Norkjosbotn-Laksvatnbukt -81 -219 168 % -0.2 -0.4 -4066 -2022 -50 % -1.0 -0.5 E39 Kleivedammen-Andenes -144 -76 -47 % -0.5 -0.3 E134 Hegstad - Damåsen Rv.616 Kolset - Klubben Rv.580 Hop- Midttun E18 Ørje- Eidsberg grense E6 Akershus grense- Patterød Rv. 35 Lunner - Gardermoen E6 Halmstad - Patterød E18 Brokelandsheia - Vinterkjæ E39 Svegatjørn - Moberg E18 Sekkelsten- Krosby E6 Ny Svinesundforbindelse E6 Skjerdingstad - Jaktøyen 303 -355 340 60 27 395 95 184 -90 165 811 -418 473 -428 203 311 72 285 768 450 62 502 1011 -186 56 % 21 % -40 % 419 % 163 % -28 % 708 % 145 % -169 % 204 % 25 % -56 % 0.9 -0.9 1.0 0.4 0.2 0.5 0.4 0.3 -0.3 0.3 0.8 -0.5 1.4 -0.8 0.5 2.0 0.4 0.3 3.3 0.8 -0.2 0.7 0.9 -0.2 Project name Rv 23 Oslofjordforbindelsen E18 Gutu-Helland-Kopstad Deviation 0% BC- ratio Estimates Actual 5.0 5.4 Conclusions/lessons learnt/challenges 1. Is NPRA achieving its stated road program benefits? 15 of 20 projects showed an improvement in NPVs than forecasted and The projects are therefore more profitable to the society than forecasted and the NPRA seems to be achieving the stated objectives Note, however that 40% of the projects have negative NPVs ! 18 2. What causes the deviations between the predicted and measured impacts? 1. The traffic growth has been higher than forecasted for all projects and the cost estimates deviates from estimates 2. 3 of the projects have been built differently than assumed. This means that: It is important to update the BCAs to the time for decision so that decision is based on best available knowledge 19 Challenges • Non‐monetised impacts are sometimes the reason for going ahead with a project. A methodology for ex‐post assessments of non‐ monetised impacts is needed • Ex post assessments of complex projects (urban projects) is needed 20 Thanks for listening! James.odeck@vegvesen.no The accuracy of traffic forecasts 22 The accuracy of cost estimates 1993 – 1996 1997 – 2003 23 2004 – 2007