Palisade Risk Conference
Monte Carlo, Las Vegas
11 November 2011
The Law of Large Numbers
vs. Project Cost Risk Models
Chris P. Caddell, PE, CCE, PMP
VP, Turner & Townsend
© Turner & Townsend plc November 11
making the difference
Outline
 Introduction
 Explanation of Law of Large Numbers
(LLN)
 Impacts due to the LLN
 Project Example
 How to Recognize &
Avoid LLN Impacts
 Questions
© Turner & Townsend plc November 11
making the difference
Author Biography
BS of Civil Engineering
Masters of Business Administration
CCE
23 Years Experience
Texas Professional Engineer
PMP
Vice President with Turner & Townsend in Houston
© Turner & Townsend plc November 11
3
making the difference
Introduction
Monte Carlo
used for 40+
years in project
cost risk
analysis
Growing
recognition of
value added by
risk analysis
Methods and
tools becoming
more
sophisticated
Quality relies
on abilities of
Risk Analyst
and model
construction
The Law of
Large
Numbers: With
more variables,
range of results
is reduced
Too many
variables in a
model results
in
unrealistically
narrow results
© Turner & Townsend plc November 11
4
making the difference
Safety Tip #1: Be aware of traffic on
the road
© Turner & Townsend plc November 11
5
making the difference
Explanation of The Law of Large
Numbers (LLN)
The LLN indicates that increasing the number of
variables in a system reduces the range of
results towards the expected value.
Example from Grinstead & Snell’s Introduction to Probability
© Turner & Townsend plc November 11
6
making the difference
Example of Dampening
Average value of dice being rolled: The more
dice you roll, the tighter the distribution of
potential outcomes
60%
55%
1 Die
50%
3 Dice
2 Dice
4 Dice
45%
40%
35%
30%
25%
20%
15%
10%
5%
0%
X< 2
© Turner & Townsend plc November 11
2≤X<3
3≤X≤4
4<X≤5
X >5
7
making the difference
Impact on Cost Risk Model Results
In cost risk analysis, the LLN indicates the larger number of
variables in the model will reduce the range of possible outcomes,
producing an overly optimistic view of the potential risk.
140
130
120
110
100
Realistic
Range for an
Early Stage
Estimate
Unrealistic
Range for an
Early Stage
Estimate
Contingency
Forecasted Project Cost
90
80
70
60
50
40
30
Appropriate
number of
variables
Base Estimate
Too many
variables in
the model
20
10
0
© Turner & Townsend plc November 11
Example of issue, not a real project
8
making the difference
Desired vs. Undesired Law of Large
Numbers Impact
Desired/Good LLN Impact: Use of truly
independent variables to simulate the effects
of the unrelated threats, opportunities, and
general uncertainty on the project cost.
Undesired/Bad LLN Impact: Use of a large
number of variables, without proper
correlation, to simulate the risk effects, many
of which are similar in nature, on the project
cost.
© Turner & Townsend plc November 11
9
making the difference
Safety Tip #2: Ensure your electrical is
not overloaded
© Turner & Townsend plc November 11
10
making the difference
Driving Factors in Law of Large
Numbers Impact
The amount of LLN impact in the cost risk
simulation is driven primarily by two factors:
Number of
independent
variables in model
Distribution of the
estimated cost across
those variables
45
40
35
Cost
30
25
20
15
10
5
0
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20
Variables
© Turner & Townsend plc November 11
11
making the difference
Number of Independent Variables
No “Industry Standard”
for the number of
variables to include in a
model
Anecdotal evidence
indicates the typical
model has 10 to 40
variables
Number of variables driven by model construction:
•Estimate Line Item Approach: Driven by level of estimate
•Risk Register Approach: Driven by number of identified
risks
•Other Approaches: Driven by how cost items and variables
are defined and level of detail
© Turner & Townsend plc November 11
12
making the difference
Distribution of Cost to Variables
Using Pareto’s 80/20
Rule, 80% of the cost
may be covered by
20% of the variables
Experience indicates
this rule is fairly
applicable
The more skewed the
cost is across the
variables, the less LLN
impact is an issue
Alternatively, if
multiple variables are
applied to cost items,
LLN impact is reduced
© Turner & Townsend plc November 11
13
making the difference
Safety Tip #3: Ensure excavations
have proper sloping
© Turner & Townsend plc November 11
14
making the difference
Project Cost Risk vs. Law of Large
Number Example
 Fictitious Project: Valued at $100 million
 Variables: +/- 20% variability for the
P10/P90 range, using a BetaPert
distribution function
 All variables are independent, i.e. no
correlations were established
 4 Cases were run:
 10 variables, each assigned $10M;
 20 variables, each assigned $5M;
 50 variables, each assigned $2M; and
 100 variables, each assigned $1M
© Turner & Townsend plc November 11
15
making the difference
Example Project: Distribution Profiles
0.30
P10/P90
Range
0.25
Relative Frequency
0.20
10 Variables
+/- 6.1%
20 Variables
+/- 4.3%
50 Variables
+/- 2.7%
100 Variables
+/- 1.9%
100 Variables
50 Variables
0.15
20 Variables
0.10
10 Variables
0.05
0.00
$80
$85
$90
$95
$100
$105
$110
$115
$120
Total Project Cost (US$M)
© Turner & Townsend plc November 11
16
making the difference
Example Project: Cumulative Curves
100%
90%
10 Variables
80%
Cumulative Probability of Project Cost (%)
20 Variables
50 Variables
70%
100 Variables
60%
50%
40%
30%
20%
10%
0%
$80
$85
© Turner & Townsend plc November 11
$90
$95
$100
$105
Total Project Cost (US$ millions)
$110
17
$115
$120
making the difference
Example: Range of Outcomes to No. of
Variables
$120
10% Probability of Cost Being At or Under
Potential Cost of Project (US$ millions)
$115
90% Probability of Cost Being At or Under
$110
$105
$100
$95
$90
$85
$80
1
5
10
15
© Turner & Townsend plc November 11
20
25
30
35
40
45
50
55
60
65
70
Number of Variables in Cost Risk Model
18
75
80
85
90
95
100
making the difference
Example Project: Cost Distribution
Alternative
In the base example, the costs were distributed evenly to the variables.
In this alternate, the cost was distributed unevenly to 20 variables to be
more realistic: 10 variables - $9M each, 10 variables - $1M each.
0.14
10 Variables - $10M
0.12
20 Variables - $5M
10 Var. - $9M, 10 Var. - $1M
Relative Frequency
0.10
0.08
0.06
0.04
0.02
0.00
$80
$85
$90
$95
$100
$105
$110
$115
$120
Total Project Cost (US$ millions)
When compared against the 10 equal variable & 20 equal variable
cases, the range distribution was closer to the 10 equal variable base
case.
© Turner & Townsend plc November 11
19
making the difference
Safety Tip #4: Scaffolding needs rails,
ladders, platforms
© Turner & Townsend plc November 11
20
making the difference
How to Recognize Undesired LLN
Impact
Experience of
Risk Analyst:
Does it pass the
“smell” test?
QC the Model:
Has it been
checked for
errors/omissions?
© Turner & Townsend plc November 11
External Metrics
& Benchmarks:
Are the results
consistent with
available data for
the project stage
/ estimate class?
21
making the difference
How to Avoid Undesired LLN Impacts
Avoid detailed models with a large number
of variables.
•Lump multiple small cost items together with
one variable for miscellaneous costs
•Only define variables for significant risks.
Correlate variables that represent similar
risks, such as
•Correlating Labor Productivity variables
•Correlating Market Pricing variables
•Correlating Scope Definition variables
© Turner & Townsend plc November 11
22
making the difference
Safety Tip #5: Edge / Fall Protection is
needed at all times
© Turner & Townsend plc November 11
making the difference
Conclusion
Project Management looks to the risk analyst to
provide a realistic assessment of the project cost
variability.
If not properly constructed, the cost risk model will
provide unrealistically narrow results, possibly driving
misguided actions by project management.
The risk analyst must ensure the model does not
include too many independent variables that
unrealistically narrow the results due to the impact of
the LLN.
© Turner & Townsend plc November 11
24
making the difference
Questions
© Turner & Townsend plc November 11
25
making the difference