Hugh Cronin
Jim Scheer
• How to determine the accuracy of CBA
• Sources of error
• Distribution of net benefits over time
• The Coquihalla Highway CBAs
• Conclusion
• Tips for dealing with error in CBA
Project
inception
Project
implementation
Project
completion
in medias res
(during)
ex ante
(before)
ex post
(after)
Study 1 – ex ante
Study 2 – in media res
Study 3 – ex post
Benefits
$394.6
$639.4
$1,307.8
Costs
$354.4
$767.6
$913.5
Net benefits
$40.2
-$128.2
$394.3
Error
Factors contributing to increasing
levels of error
Note!
Omissions
(Double
counting)
•
•
Impacts excluded (e.g. environmental)
Counting benefits in primary and
secondary markets
•
Reduces as projects evolve
(knowledge increase)
Forecasting
•
Complex/ changing project
•
Always wrong!
•
Human response unpredictable
•
•
Cognitive biases (perception, emotion)
•
Potential for ex post error
remains (counterfactual)
Also reduce over time
•
Shadow price determination
•
Valuation
•
Measurement
•
Inaccurate observation or interpretation
•
Assumptions usually
debatable
Reduce over time
Perceived as relatively less
important than other errors
Time
Distribution
ex ante
andex
ex post
post estimates
of net
Distribution
of exofante
and
estimates
of benefits
net benefits
f t (NB t )
Net benefits
Net benefits
Error
Factors contributing to increasing Note
levels of error
Omissions (Double
counting)
1.
2.
Opportunity cost of land
Cost of environmental protection
1.
2.
Low impact on result
Mitigation construction cost included but no
pollution costs etc.
Forecasting
1.
2.
Future Traffic volumes
Time and distance savings (Incl.
reduced congestion on alt. routes)
Accident rates
1.
2.
Refined over time (toll receipts)
Range of estimates and methods
3.
Range of estimates
1.
2.
3.
Value of time savings
Valuing safety benefits (accident,
injury, fatality)
Terminal value of asset
Average wages (% of wage for leisure)
New research led to vast change in
assumptions
% of construction cost accepted method
1.
Maintenance
1.
2.
Construction costs
2.
3.
Valuation
Measurement
1.
2.
3.
Misinterpretation of maint. quote (km v lane
km)
Clearer ex post (reduced strategic bias)
• Disconcertingly large errors in estimation for a
relatively straightforward project.
̵ Many potential sources of error
̵ Categorisation helps
• Importance of generating independent values for costs
(strategic bias minimisation)
• Should one in ten ex ante CBAs be subject to in medias
res or ex post follow-up and comparison to ex ante
analysis?
“Prediction is very difficult,
especially about the future.”
Niels Bohr (attributed)
• Time spent thinking about possible sources of error is time
well spent – framework available
• At pre-project stage think not only about which costs and
benefits but how they will be measured ..and ensure data
stream exists
− Programme design to enable measurement
• Understanding what factors a CBA is sensitive to (sensitivity
analysis) – presenting scenarios seems sensible!