Hands-on:
@RISK 5
Presented by
Thompson Terry
Senior Consultant
Palisade Corporation
Products Offered
Decision-Support Tools:
•
•
•
•
•
•
@RISK – Monte Carlo simulation
PrecisionTree – Decision analysis
RISKOptimizer – Genetic Algorithm optimization
TopRank – What-If scenario analysis
StatTools – data analysis
NeuralTools – neural net
-----------• @RISK for Project – schedule MC analysis
• Developer’s Kits – custom applications
2
© 2008
Risk
„
Risk: “A scenario in which there exists a
possibility of deviation from a desired or
expected outcome”
– the presence of uncertainty
– its adverse impact for any given situation
„
Threats and opportunities
risk = f ( consequence, probability )
3
© 2008
Why Model?
Precautionary measure
Better understanding of possible outcomes
Consider alternatives
Provide useful information to decision makers
4
© 2008
Point Forecasts
ƒ
ƒ
Establish a guideline of expectations
Approximation
ƒ
ƒ
ƒ
ƒ
ƒ
average
optimistic base case
most likely
not clearly defined
SWAGs
ƒ Are often wrong
5
© 2008
How MCS works…
Let’s use a Uniform distribution as an example:
f ( x ) = 1 ( max − min )
Probability
Density Function
(PDF)
11
10
9
8
7
F ( x) = P ( X ≤ x)
= ( x − min ) ( max − min )
X
66
11
10
9
7
6
5
min ≤ x ≤ max
8
max
4
© 2008
3
2
min
1
Cumulative
Distribution
Function (CDF)
6
5
4
max
3
2
1
min
0 ≤ y* ≤ 1
F −1 ( x )
11
10
9
8
7
6
5
4
3
2
x* = y* ( max − min ) + min
1
Random number (0-1)
How MCS works…
Samples
7
© 2008
Monte Carlo Simulation
Normal(0.5, 0.5)
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
2.0
X
1.5
0.5
0.0
-0.5
-1.0
0.0
1.0
0.1
8
© 2008
Basic Statistics
Probability
Normal(0.5, 0.5)
0.9
0.8
Mean = x, typically reported as the expected
value w/o probability
0.7
0.6
0.5
Variation = min. – max., offers information
about downside AND upside
0.4
0.3
0.2
Min
0.1
Max
2.0
1.5
1.0
0.5
X
0.0
-0.5
-1.0
0.0
9
© 2008
Summary Characteristics of
Probability Distributions
Lognorm(0.5, 0.5)
2.0
1.8
Measure of central tendency
(mean, mode, median and kurtosis)
1.6
1.4
1.2
Sk
1.0
0.8
ew
ne
ss
0.6
0.4
Variation
0.2
3.0
2.5
2.0
1.5
1.0
0.5
0.0
-0.5
0.0
10
© 2008
Distribution Characteristics
ƒ Center (“average”):
ƒ Mode: most frequently occurring, “most likely”
ƒ Median: 50% of outcomes on either side of this point
ƒ Mean: probability weighted average
ƒ Spread (“risk”):
ƒ Variance = sum of the squares of differences across range
ƒ Standard deviation (√ of variance) : same units as the
variable
ƒ Ranges (min-max); P10-P90 etc
ƒ Probability of achieving a target value
ƒ Shape:
ƒ Skewness (positive, symmetric/neutral, negative)
ƒ Kurtosis (broad and flat, “normal,” peaked and constrained)
11
© 2008
Traditional Approach – “Valuation”
Outcome is based on the single value for each defined assumption
Volume
Price/Mix
Point
Estimates
Cost
Discounted Cash
Flow Analysis
Project Metrics
A&P
NPV
Payback
IRR
12
© 2008
Risk Management Approach
A&P
Project Metrics
15
10
5
Risks &
Opportunities
Analysis role becomes more value added through increased collaboration
and communication with project team on key drivers and risks & opportunities
13
© 2008
3.0
2.5
2.0
1.5
1.0
0.5
0.0
-0.5
-1.0
-2.0
Cost
Key
Drivers
-1.5
Discounted Cash
Flow Analysis
45
40
35
30
25
20
15
10
5
0
Price/Mix
-5
0
12
8
10
6
4
2
0
-2
Volume
-5
@RISK
-10
Range of
Possible
Outcomes for
NPV, Payback
and IRR
-15
Value
Ranges
X1 = 0
Outcome is a range of possible values generated from applying simulation
techniques to key assumptions using business developed probabilities
@RISK:
Interface
„
Toolbar
„
Menu
„
Define Distributions
„
Simulation Settings
„
Report Settings
„
Model Window
„
Results Window
14
© 2008
Toolbars in Excel 2003
Toolbar in Excel 2007
15
15
© 2008
Menu in Excel
16
© 2008
Define Distributions
17
© 2008
Define Distributions
18
© 2008
Simulation Settings
19
© 2008
Report Settings
20
© 2008
Model Window
21
© 2008
Running Simulations
22
© 2008
Simulation Progress Window
23
© 2008
Results Summary Window
24
© 2008
Graphing Results
25
© 2008
Summary Box-Plots
26
© 2008
Tornado Graphs
27
© 2008
Exercise: Exposure in
Long-Term Decisions
„
„
The Key Result: NPV and IRR Variation
The Key Uncertain Components
–
–
–
–
–
–
–
„
Price Over Time
Quantity of Product/Service Over Time
Production in Year 1
Competitor Action
Capital Cost
Raw Materials Costs
Other Variable Costs
See RSK.Capital.Investment.xls
28
© 2008
29
© 2008
Sampling
ƒ No. of Iterations: General rules
ƒ Rapid convergence to reasonable accuracy with small
number of iterations
ƒ Inverse square root law: doubling the accuracy requires four
times as many iterations
ƒ Convergence monitoring possibilities
ƒ Display updates
ƒ Manual and automatic convergence monitoring
ƒ Sampling type
ƒ Monte Carlo = simple random number sampling
ƒ Latin Hypercube = intervals of equal probability
ƒ When running standard models <~500 iterations (e.g. during
development)
ƒ With rare events, LH will force sampling of these events
ƒ Representation not randomness is the requirement for the
sampling: randomness is a way to try to achieve this without
creating a biased sample
30
© 2008
Further use of @RISK
ƒ Sensitivity-type analyses
ƒ @RISK results in Excel
ƒ Multiple simulations, and overlaying output
graphs
ƒ RiskSimTable
ƒ Advanced analyses
31
© 2008
Sensitivity Analyses
ƒ
Sensitivity analysis: displays coefficients for tornado
graph in tabular form; often easier than tornado graphs when
there exist several outputs of interest
ƒ
Tornado diagrams: ranks sensitivity of an output to
input variables (showing by how much a one standard
deviation change in input variable affects the output, in terms
of the standard deviation of the output) (for regression graphs)
ƒ
Scenario analysis: Shows the ranges of input variables
which most explain the outputs lying within in a certain range
(e.g. >90% interval)
32
© 2008
Tornado Graphs
33
© 2008
Further use of @RISK
ƒ Sensitivity-type analyses
ƒ @RISK results in Excel
ƒ Multiple simulations, and overlaying output
graphs
ƒ RiskSimTable
ƒ Advanced analyses
34
© 2008
Statistics and Templates
ƒ Stats functions:
functions select key evaluation
metrics
ƒ RiskMean, RiskStdDev, RiskMax, etc.
ƒ Graph functions: display relevant graphs
ƒ RiskResultsGraph – formatting and
customization
ƒ RiskTemplate sheet: Excel reports with
relevant data, graphs, orientation and formats
ƒ Report Settings dialog box
35
© 2008
@RISK Statistics
„
Stats functions: select key evaluation metrics
– RiskMean, RiskStddev, RiskPercentile, RiskTarget, etc.
„
Graph functions: display relevant graphs
– RiskResultsGraph – formatting and customization
„
Use Excel’s Insert Function tool to access library
@RISK Statistics
36
36
© 2008
Templates
„
Template sheet: Excel reports with relevant data,
graphs, orientation and formats
– Report Settings dialog box
37
© 2008
Further use of @RISK
ƒ Sensitivity-type analyses
ƒ @RISK results in Excel
ƒ Multiple simulations, and
overlaying output graphs
ƒ RiskSimTable
ƒ Advanced analyses
38
© 2008
Scenario: Parametric Uncertainty
ƒ Use RiskSimtable to adjust parameters of
distributions
ƒ Change conditions
ƒ Test optimistic and pessimistic conditions
ƒ Explore alternate possibilities
ƒ Compare multiple simulation results, graphs
ƒ Random numbers, seed values and reproducibility
39
© 2008
Further use of @RISK
ƒ Sensitivity-type analyses
ƒ @RISK results in Excel
ƒ Multiple simulations, and overlaying output
graphs
ƒ RiskSimTable
ƒ Advanced analyses
40
© 2008
Terminology for Advanced Analyses
„
„
„
Goal-seek: Changes the value in a cell to obtain a
desired goal in an output
Stress analysis: Restricts samples drawn from a
distribution e.g. to sample only from the lower 5% of
one or more distributions
Advanced sensitivity analysis: Determines the
effect of an @RISK input distributions on outputs
41
© 2008
Distribution Fitting
„
„
„
„
Sample data
Density data
Continuous data
Discrete data
42
© 2008
Fitting Data
„
Tests used:
– Maximum likelihood estimation
– Chi-square
– Anderson-Darling
– Komolgorov-Smirinov
– Root Mean Square
„
„
„
Statistics
Goodness of Fit
Ranking
43
© 2008
Distribution Fitting
•
•
Description
– Effective for describing
distribution by aligning it to
existing data
– Combines effects of multiple
assumptions into one
assumption
– It is important to remove any
trend or seasonality from the
data
Examples
– Historical waste data
– Historical market prices
– Any distribution needing to be
aligned to existing data
Inputs
• Source data
44
© 2008
129
128
127
126
125
124
123
122
Fitting: Output
45
© 2008
Output: Write to Cell
46
© 2008
Fitting Distributions to Data
„
„
„
Historic data unlikely to have come from a “pure” random process e.g.
– May reflect actions that were taken at the time in response to
randomness, such as
„ Damages less than an excess amount not being recorded for
insurance purposes
„ Oil prospects below a certain estimated size not being
explored further
„ Journeys above a certain duration being cancelled
– May reflect compound random events (discrete number of events,
continuous severity of each event)
Can fit only distributions that are available in @RISK i.e. no multimode distributions
– Data may need to be disaggregated into components in order to
fit valid distributions to it (and the fitted distributions re-aggregated
through sampling during the simulation)
View Fit Summary Window when in the Fitting Results window, can
give a useful overview of all fitted distributions
47
© 2008
Notes on GOF Statistics
„
„
„
„
„
Available for sampled data only (continuous or discrete)
Test Value: Measures the deviation between the fitted and the input
data: The better the fit, the smaller the statistic
P (“significance level”): Lies between 0 and 1; the more significant the
fit, the larger the P-value
– As P decreases to zero, we become less confident that the
original data set came from the fitted distribution
– As P increases to one, we have no basis to reject the hypothesis
that the fitted distribution may have generated the data set
– Answers the question “How likely is it that a new sample of the
same size drawn from the fitted distribution would generate a
larger fit statistic?”
Critical values of the Test Value: For a specified level of significance,
what is the largest value of the Test Value that we would accept as a
valid fit?
Most P-values and critical values are actually found by very detailed
Monte Carlo simulation studies. However, not all distributions have
been analyzed in sufficient detail by this method and so sometimes an
exact P-value is not available
48
© 2008
Expert Opinion
„
„
Complex situations, expectations or
prediction
Real world data may suffer from:
– Poor documentation
– Uncertainty in measurement/instability in the underlying
processes
– Observations resulting from multiple effects, rather than a
single effect
– Sampling effects (esp. rare events or small samples sizes)
„
To cross-calibrate/bullet proof models
49
© 2008
Alternate Parameters
„
Define a variable using:
– Standard parameters
– Alternate parameters - percentiles
„
„
„
Define Distributions dialog box
Can mix parameters, percentiles
Not available for all distributions
50
© 2008
Alternate Parameters
•
•
Description:
– Specify values for
specific percentile
locations of an input
distribution as opposed
to traditional arguments
used by distribution
Examples:
– Cost Estimates
– Any distribution
needing to be defined
further
Inputs
51
• Ascending or Descending Percentiles
• Standard or Alternate Parameters
51
© 2008
Modeling
„
„
„
„
Determine outcomes which generate interest
Model causal factors, not end results
Use sensitivity analyses to determine critical
variables
Recognize impact of independent and
exclusive events
– Exclusive events imply dependence
– One outcome rules out another outcome
52
© 2008
Sources of help
„
„
„
„
„
„
„
„
„
On-line tutorials
Help menu within the software
Software manuals (PDF)
Palisade web-site www.palisade.com
Helpdesk: http://helpdesk.palisade.com/
Forum: http://forums.palisade.com/
Web encyclopedia www.wikipedia.com
Your Regional Sales Manager(s)
Palisade Training and Consulting Services
53
© 2008