Lessons from ex-post benefit-cost analysis
of road projects in Norway
James Odeck, Professor
NTNU / Norwegian Public Roads Administration
Norway
The 5th Concept Symposium on Project Governance
Valuing the Future - Public Investments and Social Return
20. – 21. September 2012
Symposium web-site: http://www.conceptsymposium.no/
Concept Research Programme: http://www.concept.ntnu.no/english/
Lessons from ex‐post benefit‐cost analyses of road projects in Norway James Odeck
NPRA/NTNU
Objectives
To reveal the extent to which predictions of benefit‐cost analyses delivered to the decision makers matches outturns. In particular, we ask:
1. Do the road authorities achieve their stated road program benefits?
2. What are the magnitude of deviation between forecasts and outturns
3. What causes the deviations?
2
Background
1.
The Norwegian Public Roads administration (NPRA) carries out BCA for its trunk road schemes
2.
Based on information from the BCA, decision makers may prioritize projects.
3.
The BCAs are conducted ex ante; therefore there is little knowledge as: (i) to what degree the predictions are correct (ii) whether the project fulfil their objectives and, (iii) whether the decision makers are supplied with appropriate information ex ante. We address these questions based on ex‐post assessments conducted since 2006 when the Ministry of Transport sanctioned such studies. 20 projects are studied to date.
3
Framework for analyses
• 3 ‐ 5 projects are ex‐post evaluated every year
• Requirement:
• Investment cost > 200 mill NOK
• Has been open for traffic in at least 5 years
• The projects are chosen the year of the ex‐post assessment
• The baseline for ex‐post assessments are the assessments •
presented to the decision makers when the decision to fund was made (the go‐ahead decision)
Focus on monetised impacts only
4
To underline what we do!
We re‐do the benefit‐ analyses to explore whether the projected outturns deviate from real outcomes 5
Which estimates are compared to outturns?
Project development process
Project Planning
Municipal
External
Quality
Master
Assurance(EQA) Plan(EIA)
Project execution
Zoning
Plan
Operations
Construction
construction
plan
1
2
3
4
Time
(Year)
Actual cost
Decision to build
“approved BCA estimates”
Outturn
Different estimates
6
Compare
5
Impacts included in an Impact Assessment
Cost benefit analysis
Ex post evaluation
Monetised impacts
 Travel time savings
Non monetised
 Visual landscape

Vehicle operating costs

Accident costs

Induced traffic

Inconvenience cost (ferry) 
Cultural heritage

Noise and local air
pollution
Natural resources

Road maintenance costs

Residual value of capital

Cost of public funds

Road investment costs

Community life
effects and outdoor
recreation

Natural environment

Precautions taken !  Appraisal techniques develop over time in some cases causing the tools to change from before‐ to the time of ex‐
post assessment…. We use the same software package when comparing the before and ex‐post assessments
 The analysis period of the projects is 25 years, but the ex‐
post assessments are conducted 5 years after opening….. The same traffic forecasts are used in the ex‐post assessment as in the original analysis after year 5
8
Approach and data collection
1.
Acquire project overview, project history and project development over time
2.
Retrieve the database which was the basis for CBA presented to the decision makers before the go ahead‐
decision
3.
Possibly redo the original CBA with a newer version of the CBA software (EFFEKT)
Substitute the outturn data into the CBA in 3
4.




5.
Traffic
Accidents
Construction costs
Average speed
Compare the before‐decision analysis (3) and the ex‐post 9
assessment (4)and explain differences
Projects evaluated No Project
1 Rv.23 Oslofjordforbindelsen
Project type
Strait crossing; sub sea tunnel
2 E18 Rannekleiv - Temse
Trunk road ; bypass
3 Rv.714 Hitra - Frøya
Strait crossing; sub sea tunnel
4 E18 Teigeland - Håland
Trunk road, mainly tunnel
5 Rv.62 Øksendalstunnelen
Trunk road,
6 E8 Nordkjosbotn - Laksvatnbukt
Trunk road
7 E18 Gutu - Helland - Kopstad
Trunk road;
8 E39 Kleivedammen - Andenes
Trunk road
9 E134 Hegstad - Damåsen
Trunk road; bypass Darbu…
10 Rv.616 Kolset - Klubben
Strait crossing; sub sea tunnel
Project objective
Reduce user costs, accidents and improve
local environment
Reduce user costs, accidents and improve
local environment
Reduce user costs, regional effects
Reduce user costs, reduce risk of
avalanche/land slide
mainly tunnel
Reduce risk of avalanche/land slide, reduce
user costs
Reduce accidents, improve local
environment and reduce user costs
bypass Sande and HolmestrReduce accidents, reduce user costs and
improve local environment
Reduce user costs, reduce accidents
10
Reduce user costs, reduce accidents and
improve local environment
Reduce user costs
Projects cont.
11 Rv.580 Hop- Midttun
12 E6: Akershus grense - Patterød
Trunk road within city; bypass Nestun Reduce user costs, improve local
environment
Trunk road
Reduce user costs, accident
13 E18 Ørje - Eidsberg
Trunk road
Reduce user costs, accidents
14 Rv35 Grualia - Kneppe
Trunk road, "new link"
15 E6 Halmstad - Patterød
Trunk road
Improve acessability to Gardermoen
ariport, regional development
Reduce user costs and accidents
16 E18 Brokelandsheia - Vinterkjær Trunk road; Bypass Søndeled
17 E39 Svegatjørn - Moberg
Trunk Road; Bypass Osøyro
18 E18 Sekkelsten- Krosby
Trunk Road; Bypass Askim
19 E6 Ny Svinesundforbindelse
Trunk Road
20 E6 Skjerdingstad - Jaktøyen
Trunk Road; Bypass Melhus?
11
Reduce accidents, user costs, improve local
environment
Reduce user costs, accidents and improve
local environment, improve situation for
Reduce user costs, accidents, improve local
enironment
Reduce accidents, reduce user costs
especially for goods transport
Reduce accidents, improve local
environment, and reduce user costs
RESULTS (1a):
Traffic in the opening year (AADT)
Deviation
No
Project
Estimates
Actual
Estimates/Actual
1
2
3
Rv. 23 Oslofjordforbindelsen
E18 Rannekleiv - Temse
Rv. 714 Hitra - Frøya
4240
8232
353
3780
10242
512
‐11 %
24 %
45 %
4
E134 Teigeland - Håland
1000
1367
37 %
5
6
Rv. 62 Øksendalstunnellen
E8 Norkjosbotn-Laksvatnbukt
1386
2300
1345
2400
‐3 %
4%
7
8
E18 Gutu-Helland-Kopstad
E39 Kleivedammen-Andenes
12000
686
16700
924
39 %
35 %
9
E134 Hegstad - Damåsen
6740
8050
19 %
10
Rv.616 Kolset - Klubben
88
155
76 %
11
12
Rv.580 Hop‐ Midttun
E18 Ørje - Eidsberg grense
19300
3288
14000
4488
‐27 %
36 %
13
14
15
E6 Akershus grense ‐ Patterød
Rv.35 Lunner ‐ Gardermoen
E6 Halmstad – Patterød
21750
2354
24613
20700
1600
26954
‐5 %
‐32 %
10 %
16
17
E18 Brokelandsheia ‐ Vinterkjær
E39 Svegatjørn - Moberg
6814
2700
6018
5000
‐12 %
85 %
18
E18 Sekkelsten- Krosby
5200
8000
54 %
19
E6 Ny Svinesundforbindelse
10282
11406
11 %
20
E6 Skjerdingstad - Jaktøyen
9100
9900
9%
12
RESULTS(1b):
No
Project
Average traffic growth rate, the first 5 year
Estimates
Actual
1
Rv. 23 Oslofjordforbindelsen
1.4 %
6.8 %
2
3
E18 Rannekleiv - Temse
Rv. 714 Hitra - Frøya
1.2 %
1.2 %
3.2 %
18.3 %
4
5
6
E134 Teigeland - Håland
Rv. 62 Øksendalstunnellen
E8 Norkjosbotn-Laksvatnbukt
1.2 %
1.0 %
1.1 %
2.3 %
5.5 %
3.7 %
7
E18 Gutu-Helland-Kopstad
1.2 %
3.0 %
8
9
E39 Kleivedammen-Andenes
E134 Hegstad - Damåsen
1.0 %
3.9 %
1.2 %
2.5 %
10
11
Rv.616 Kolset - Klubben
Rv.580 Hop- Midttun
1.4 %
1.7 %
1,6 %
12
13
E18 Ørje - Eidsberg grense
E6 Akershus grense - Patterød
2.0 %
2.0 %
5.6 %
14
Rv.35 Lunner - Gardermoen
2.0 %
12.1 %
15
16
E6 Halmstad – Patterød
E18 Brokelandsheia - Vinterkjær
2.0 %
2.2 %
4.4 %
17
18
19
20
E39 Svegatjørn - Moberg
E18 Sekkelsten- Krosby
E6 Ny Svinesundforbindelse
E6 Skjerdingstad - Jaktøyen
1.7 %
2.0 % 13
2.0 %
1.4 %
4.1 %
3.2 %
5.7 %
2.6 %
3.4 %
3.2 %
‐0.3 %
Results (2): Construction costs
Project no.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
Project name
Rv 23 Oslofjordforbindelsen
Ev 18 Rannekleiv - Temse
Rv 714 Hitra - Frøya
Ev 134 Teigeland - Håland
Rv 62 Øksendalstunnellen
E8 Norkjosbotn-Laksvatnbukt
E18 Gutu-Helland-Kopstad
E39 Kleivedammen-Andenes
E134 Hegstad - Damåsen
Rv.616 Kolset - Klubben
Rv.580 Hop- Midttun
E18 Ørje- Eidsberg grense
E6 Akershus grense- Patterød
Rv. 35 Lunner - Gardermoen
E6 Halmstad - Patterød
E18 Brokelandsheia - Vinterkjær
E39 Svegatjørn - Moberg
E18 Sekkelsten- Krosby
E6 Ny Svinesundforbindelse
E6 Skjerdingstad - Jaktøyen
Construction costs(Mill. NOK)
Estimates
Actual
1560
215
589
437
215
242
325
195
262
254
322
252
296
681
295
507
248
506
14
847
678
1370
263
460
385
218
346
314
183
249
336
372
246
302
780
302
425
211
520
849
583
Deviation (Estimates/Actual)
-12 %
22 %
-22 %
-12 %
1%
43 %
-3 %
-6 %
-5 %
32 %
16 %
-2 %
2%
15 %
2%
-16 %
-15 %
3%
0%
-14 %
Were the projects built as presented to the decision
makers?
Yes
Yes/No
No
All others
Oslofjord link;
Only first stage built.
Second stage (twin
tube tunnel) was not
built. Coming soon?
Hitra-Frøya:
Connecting road was
not built; planning
difficulties
Teigeland- Håland
1,75 km not built to
avoid cost overrun
Results (3): Accident costs
Reduction in accidents with
personal injury, opening year
20
No
1
Project
Rv. 23 Oslofjordforbinde
Estimates
8.6
Actual
7.8
Deviation%
-9 %
2
3
4
5
6
7
8
9
10
11
E18 Rannekleiv - Temse
Rv. 714 Hitra - Frøya
E134 Teigeland - Håland
Rv. 62 Øksendalstunnelle
E8 Norkjosbotn-Laksvatn
E18 Gutu-Helland-Kopstad
E39 Kleivedammen-Ande
E134 Hegstad - Damåsen
Rv.616 Kolset - Klubben
Rv.580 Hop- Midttun
6.1
-0.2
0.0
1.8
4.0
4.0
1.1
2.1
-1.5
4.6
8.1
-0.2
0.7
1.9
3.7
14.2
1.9
4.3
-1.9
0.7
5%
-6 %
252 %
69 %
105 %
24 %
-85 %
12
13
14
15
16
17
18
19
E18 Ørje - Eidsberg gren
E6 Akershus grense- Patte
Rv.35 Lunner- Gardermoe
E6 Halmstad – Patterød
E18 Brokelandsheia - Vin
E39 Svegatjørn - Moberg
E18 Sekkelsten- Krosby
E6 Ny Svinesundforbinde
E6 Skjerdingstad - Jaktøy
3.8
3.1
3.7
4.3
7.4
0.7
2.8
6.3
2.7
5.4
2.4
4.0
0.8
7.6
1.4
6.0
6.4
5.1
42 %
-23 %
10 %
-82 %
3%
100 %
114 %
2%
89 %
33 %
0%
Results 4:
Cost Benefit Analysis (NPV; mill NOK)
Estimates
4563
NPV
Actual
4565
Ev 18 Rannekleiv - Temse
Rv 714 Hitra - Frøya
Ev 134 Teigeland - Håland
Rv 62 Øksendalstunnellen
409
-242
-445
57
498
156
-418
67
22 %
-165 %
-6 %
18 %
1.2
-0.9
-0.8
0.2
1.1
0.9
-0.7
0.2
E8 Norkjosbotn-Laksvatnbukt
-81
-219
168 %
-0.2
-0.4
-4066
-2022
-50 %
-1.0
-0.5
E39 Kleivedammen-Andenes
-144
-76
-47 %
-0.5
-0.3
E134 Hegstad - Damåsen
Rv.616 Kolset - Klubben
Rv.580 Hop- Midttun
E18 Ørje- Eidsberg grense
E6 Akershus grense- Patterød
Rv. 35 Lunner - Gardermoen
E6 Halmstad - Patterød
E18 Brokelandsheia - Vinterkjæ
E39 Svegatjørn - Moberg
E18 Sekkelsten- Krosby
E6 Ny Svinesundforbindelse
E6 Skjerdingstad - Jaktøyen
303
-355
340
60
27
395
95
184
-90
165
811
-418
473
-428
203
311
72
285
768
450
62
502
1011
-186
56 %
21 %
-40 %
419 %
163 %
-28 %
708 %
145 %
-169 %
204 %
25 %
-56 %
0.9
-0.9
1.0
0.4
0.2
0.5
0.4
0.3
-0.3
0.3
0.8
-0.5
1.4
-0.8
0.5
2.0
0.4
0.3
3.3
0.8
-0.2
0.7
0.9
-0.2
Project name
Rv 23 Oslofjordforbindelsen
E18 Gutu-Helland-Kopstad
Deviation
0%
BC- ratio
Estimates
Actual
5.0
5.4
Conclusions/lessons learnt/challenges
1. Is NPRA achieving its stated road program benefits?
15 of 20 projects showed an improvement in NPVs than forecasted and The projects are therefore more profitable to the society than forecasted and the NPRA seems to be achieving the stated objectives
Note, however that 40% of the projects have negative NPVs !
18
2. What causes the deviations between the predicted and measured impacts?
1. The traffic growth has been higher than forecasted for all projects and the cost estimates deviates from estimates
2. 3 of the projects have been built differently than assumed. This means that:
 It is important to update the BCAs to the time for decision so that decision is based on best available knowledge
19
Challenges
• Non‐monetised impacts are sometimes the reason for going ahead with a project. A methodology for ex‐post assessments of non‐ monetised impacts is needed
• Ex post assessments of complex projects (urban projects) is needed
20
Thanks for listening!
James.odeck@vegvesen.no
The accuracy of traffic forecasts
22
The accuracy of cost estimates
1993 – 1996
1997 – 2003
23
2004 – 2007