Applying @RISK for assessing
distributions based on Risk Simulation
and Scenario Foresight: SIMSIGHT
Scientific Assistant
Inga Ambrasaite
Technical University of Denmark
Palisade Risk User Conference 2011
Amsterdam - the Netherlands
Background
• Cost-Benefit Analysis (CBA) is a generally accepted
approach to assess the various types of transport projects.
• The project alternatives are ranked and selected based on
single point estimates, such as Net Present Values (NPV),
Benefit-Cost Ratios (BCR), etc.
• However, uncertainties involved in these estimates can
lead to the risk of the selected projects not being feasible.
• The need of exploring the robustness of decision support
in relation to uncertainties is a topic of growing concern,
especially for large scale infrastructure projects.
2
DTU Transport, Technical University of Denmark
Applying @RISK for assessing distributions based on Risk
Simulation and Scenario Foresight: SIMSIGHT
12-04-2011
Research Outline
• The purpose of SIMSIGHT is to make feasibility risk and
Monte Carlo simulation when only the user-defined MIN
and MAX boundaries are known.
• The user-defined MIN and MAX boundaries are obtained in
a decision conference, where plausible scenarios are
presented and discussed.
• The MIN and MAX boundaries are the global values based
on the whole set of scenarios.
• In SIMSIGHT, overconfidence theory is
extending these MIN and MAX boundaries.
3
DTU Transport, Technical University of Denmark
applied
Applying @RISK for assessing distributions based on Risk
Simulation and Scenario Foresight: SIMSIGHT
for
12-04-2011
Overview of presentation
The presentation comprises the following:
• Introduction of the case study,
• Exploration of point estimates as BCR, NPV, etc. from a
conventional cost-benefit analysis (CBA),
• Analytical results from Reference Class Forecasting (RCF),
• The SIMSIGHT approach: applying overconfidence theory
on user inputs (MIN and MAX boundaries) to define
distributions for risk simulation,
• Conclusion and perspectives.
4
DTU Transport, Technical University of Denmark
Applying @RISK for assessing distributions based on Risk
Simulation and Scenario Foresight: SIMSIGHT
12-04-2011
Case Study: Greenland
The transportation system in Greenland is unusual and
troublesome:
• No railways, no inland waterways and almost no roads
between the towns.
• Most of the transportation takes place either by sea or
airways.
5
DTU Transport, Technical University of Denmark
Applying @RISK for assessing distributions based on Risk
Simulation and Scenario Foresight: SIMSIGHT
12-04-2011
Case Study: Greenland
Around 50 ”airports” in
Greenland for both domestic
and international connections:
• The current Dash 7 airplanes
are taken out of commission.
• The existing runways are too
short to handle newer
airplanes, except in
Kangerlussuaq.
• The international connections
are limited in Nuuk – the
capital and major city in
Greenland.
6
DTU Transport, Technical University of Denmark
Applying @RISK for assessing distributions based on Risk
Simulation and Scenario Foresight: SIMSIGHT
12-04-2011
Case Study: Greenland
Transportation plan for Greenland:
• To move the major international
airport from Kangerlussuaq to the
Greenland’s capital, Nuuk:
– Extending the existing runways to
1800 or 2200 m in Nuuk,
– Relocating the airport in Nuuk with a
new 3000 m runway.
• Based on the calculations the 2200 m
alternative seems to be the most
socio-economically profitable.
7
DTU Transport, Technical University of Denmark
Applying @RISK for assessing distributions based on Risk
Simulation and Scenario Foresight: SIMSIGHT
12-04-2011
The UNITE-DSS Modelling Framework
8
DTU Transport, Technical University of Denmark
Applying @RISK for assessing distributions based on Risk
Simulation and Scenario Foresight: SIMSIGHT
12-04-2011
Results: Cost-Benefit Analysis
The most important impacts:
Project:
Nuuk 2200 meter alternative
Purpose:
Socio-economic analysis on new runway/airport alternatives in Greenland
Project Description:
Decommissioning of the Dash-7 airplanes leaves many of the runways obsolete in
Greenland. The task is to find an overall transportation plan as concerns air, sea and road
transport in Greenland.
Basis:
Total Benefits:
Total Costs:
Benefit-Cost Ratio (BCR)
Net Present Value (NPV)
2,817.0
-1,341.5
2.10
1,475.5
Projections/Key figures:
Generel prognosis (NPI):
Growth in GDP (excl. Inflation):
Tax Distortion Ratio:
Discount ratio:
1.00%
0.50%
10%
6%
Results, schematical overview:
Principal items
Construction Costs Incl. Follow-up costs
Reinvestment costs (20 year
savings)
Operational costs (Home Rule &
AG)
User Benefits
Production Costs
Mail & Packages
Air Greenland & Road Traffic
Abandonment of Kangerlussuaq
Terminal Value (Scrap Price)
Tax Distortion
Benefit-Cost Ratio (BCR)
Internal Rate of Return (IRR)
Net Present Value (NPV)
First Year Rate of Return (FYRR)
9
Mill. DKK in
2006 Prices
-1,287.4
-54.0
-211.3
2,187.2
-1,228.6
15.3
1,348.3
686.8
55.2
-36.0
2.10
13.84%
1,475.5
16.51%
DTU Transport, Technical University of Denmark
• The construction costs.
• The user benefits:
– Consist of ticket revenue
and travel time savings.
– Based on the demand
forecasts (future number
of passengers).
Applying @RISK for assessing distributions based on Risk
Simulation and Scenario Foresight: SIMSIGHT
12-04-2011
Uncertainties within the CBA scheme
• General
tendency
of
underestimation
of
costs
(investments) and overestimation of benefits (demand
forecast/prognosis) reveals that socio-economic analyses
become overoptimistic leading to wrongful decision
support.
• To deal with this, the risk analysis and Monte Carlo
Simulation based on Reference Class Forecasting is applied
for determining the output distribution for BCR instead of
conventional single point estimate. This is presented by
the certainty values and graphs (i.e. Accumulated
Descending Graphs).
10
DTU Transport, Technical University of Denmark
Applying @RISK for assessing distributions based on Risk
Simulation and Scenario Foresight: SIMSIGHT
12-04-2011
The data fit for construction costs: Erlang
Distribution
• Based on Salling (2008).
11
DTU Transport, Technical University of Denmark
Applying @RISK for assessing distributions based on Risk
Simulation and Scenario Foresight: SIMSIGHT
12-04-2011
The data fit for demand: PERT Distribution
• Based on Salling (2008).
12
DTU Transport, Technical University of Denmark
Applying @RISK for assessing distributions based on Risk
Simulation and Scenario Foresight: SIMSIGHT
12-04-2011
Results: Monte Carlo Simulation
13
DTU Transport, Technical University of Denmark
Applying @RISK for assessing distributions based on Risk
Simulation and Scenario Foresight: SIMSIGHT
12-04-2011
SIMSIGHT approach
• Monte Carlo simulation based on Reference Class
Forecasting (RCF) produces the interval results which
contain certain advantages as decision support compared
to the point estimates.
• The RCF-technique considers the historical data:
– Looks upon historical infrastructure projects before and after
implementation.
– The usefulness of RCF depends on how the particular project
is comparable with the projects from the pool of reference
classes.
• The SIMSIGHT approach involves the combination of risk
simulation and scenario foresight based on the inputs from
users stemming from i.e. a decision conference.
14
DTU Transport, Technical University of Denmark
Applying @RISK for assessing distributions based on Risk
Simulation and Scenario Foresight: SIMSIGHT
12-04-2011
Scenario analysis
• The scenario context in SIMSIGHT is treated by conducting a
decision conference, where the participants are representing
different kinds of expertise and points of view.
• The practical outcome from participants for SIMSIGHT consists of
minimum and maximum expected values for the construction
costs and demand based on initial estimates. The lowest MIN
value stems from the worst possible scenario, whereas the
highest MAX value stems from the best possible scenario.
• The general principle of SIMSIGHT is to link these expected
values with the actual modelling. In this way, SIMSIGHT informs
the users about the risk consequences based on their own inputs
regarding the MIN and MAX boundaries for construction costs
and demand.
15
DTU Transport, Technical University of Denmark
Applying @RISK for assessing distributions based on Risk
Simulation and Scenario Foresight: SIMSIGHT
12-04-2011
Collection of the data
• Two respondents having expertise of relevance were asked
to provide the expected minimum and maximum values
for construction costs and future demand.
• For the initial construction costs estimate of 691 mill. DKK:
CC
Min, mill. DKK
Max, mill. DKK
Input #1
620
1000
Input #2
620
790
Average
620
895
• For initial future demand estimate of 43213 hours:
CC
16
Min, hours
Max, hours
Input #1
27400
45000
Input #2
33400
43200
Average
30400
44100
DTU Transport, Technical University of Denmark
Applying @RISK for assessing distributions based on Risk
Simulation and Scenario Foresight: SIMSIGHT
12-04-2011
Overconfidence theory
• Overconfidence theory states that people are unaware of
their lack to indicate a complete range of variation, see for
example Venter & Michayluk (2008):
– The indicated ranges by the two respondents are too narrow.
– Based on the theory, the ranges are only accounting for
approximately 60% variation of uncertain impacts.
17
DTU Transport, Technical University of Denmark
Applying @RISK for assessing distributions based on Risk
Simulation and Scenario Foresight: SIMSIGHT
12-04-2011
Applying user inputs for theoretical
distributions
• The confidence level of 60% represents the 20% ’tail ends’
to the given average boundaries of respondents.
• Using this information, the Erlang and PERT probability
distributions are defined with the average MIN and MAX
boundaries setting the 20% percentiles at the ends of
distributions:
– Instead of MIN, ML and MAX inputs, only the two outer
boundaries are necessary.
18
DTU Transport, Technical University of Denmark
Applying @RISK for assessing distributions based on Risk
Simulation and Scenario Foresight: SIMSIGHT
12-04-2011
Erlang Probability Distribution
19
DTU Transport, Technical University of Denmark
Applying @RISK for assessing distributions based on Risk
Simulation and Scenario Foresight: SIMSIGHT
12-04-2011
PERT probability distribution
20
DTU Transport, Technical University of Denmark
Applying @RISK for assessing distributions based on Risk
Simulation and Scenario Foresight: SIMSIGHT
12-04-2011
Results: SIMSIGHT
• The Certainty graphs depicts a
higher BCR using SIMSIGHT
than RCF input data:
– Rather conservative input
from the respondents as
concerns construction costs.
– More uncertainty as regards
the demand forecasts –>
more positive impact.
• Are only based
input parameters.
on
Average
• Certainty values in the range of
[1.5 ; 3.25].
21
DTU Transport, Technical University of Denmark
Applying @RISK for assessing distributions based on Risk
Simulation and Scenario Foresight: SIMSIGHT
12-04-2011
Conclusion
• Applying the SIMSIGHT approach provides decision
support with a focus on awareness of feasibility risk.
• Risk simulation is based on the MIN and MAX boundaries
provided by users instead of Reference Class Forecasting
technique.
• By applying overconfidence theory for extending these
MIN and MAX boundaries the reliable input for probability
distributions can be achieved.
22
DTU Transport, Technical University of Denmark
Applying @RISK for assessing distributions based on Risk
Simulation and Scenario Foresight: SIMSIGHT
12-04-2011
Perspectives
• Continuing to arrange decision conferences, where the
diverse of expertises and points of view are taken into
account to gather input parameters for analysis.
• Testing SIMSIGHT on the new cases.
• Exploring SIMSIGHT as one the approaches that can be
included in a decision conferences for the transport
planning issues.
23
DTU Transport, Technical University of Denmark
Applying @RISK for assessing distributions based on Risk
Simulation and Scenario Foresight: SIMSIGHT
12-04-2011
Thank you for your attention!
Inga Ambrasaite
Scientific Assistant
Department of Transport
Technical University of Denmark