Risk Modelling of a Notional Mega
Project Cost Estimate
11 Nov 2011
Overview
Mega Projects by their very nature consist of smaller individual projects that integrate
together to form a whole.
In a similar fashion the estimate for such a project also comes from a multitude of diverse
disciplines with differing sources and work practices.
This paper will use a case study to show how a risk model for the Cost Estimate was built
in Palisade @Risk to identify appropriate levels of Contingency and Management
Reserve.
Inputs to the model were taken from a variety of sources including
•
•
•
•
•
Control Estimate
Systemic risk models
Quantitative schedule risk models
Risk ranging workshops
Various risk registers
2
Contingency & Management Reserve
Contingency
Contingency is the amount of money used in the estimate to deal with the uncertainties
inherent in the estimating process. Contingency is required because estimating is not
an exact science.
• The amount of infill and concrete to cover an underground pipe, the number of manhours to complete the task, and the actual all in labour rate, are all best estimates until
the work is complete
Management Reserve
An amount added to the estimate to allow for specific risks that may or may not occur
that are within the project’s control or influence. Risk is defined as an undesirable
potential outcome and its probability of occurrence
• Does not include force majeure, currency risk, political risk etc
Scope
Both amounts are based on a defined scope. Should scope change , the estimate must
be revised to reflect such changes in scope.
Neither Contingency nor Management Reserve are a source of funds to cover scope
changes
3
Mega Project vs Project Portfolio
Key Differences
So why is a Mega Project so different to a Portfolio of Projects being executed by a
Company.
Key differences are:
• Same key personnel
• Mega project is a number of inter related projects with numerous key interfaces. Any
delay or changes to one can have a significant effect on one or the others
• Site Wide Services common to all sub projects
• Same geographical location
Effect
Must not underplay the significance of these differences. Sub projects must not be
modeled in isolation
• Liberal use of correlation between sub projects to avoid nodal bias
• Schedule must be modeled at the mega project level ensuring all interfaces are
included.
4
Identifying the Risks
The Risk Register
Prior to estimating contingency or otherwise quantifying risk impacts, the risks must first
be identified and then logged in the Risk register. Husky uses a standard 5 x 5 risk
Matrix as a Probability Impact Diagram for Project Risk. Impacts are defined specific
to project
Consequence
Very High
High
Medium
Low
Very Low
Very Low
Low
5A
4A
3A
2A
1A
5B
4B
3B
2B
1B
Likelihood of Occurrence
Medium
High
5C
4C
3C
2C
1C
5D
4D
3D
2D
1D
Very High
5E
4E
3E
2E
1E
Priority Action Setting
Critical
Immediate Action must be taken to Prevent or Mitigate the
Risk
Serious
Mitigation Action Required
Moderate
Mitigation Strategies to be investigated
Acceptable Be aware of Potential Mitigation Strategies
5
Typical Risk Register
6
Typical Boston Square
7
The Cost Model
Main input to the Cost Model was the Control Estimate. However elements of the
estimate were modelled using a variety of techniques and then brought into an
overall Cost Risk model
•
•
•
•
Risk ranging workshops using range estimating techniques
Systemic risk modelling
Quantitative schedule risk models
Specific risk modelling
8
Traditional Probability Distribution Functions
@Risk contains a wealth of distribution functions.
Most are useful to the in depth simulation requiring sophisticated tools.
Only a few are suitable for general cost modelling
9
Typical Probability Distribution Functions
Risk Uniform
Used when we have no idea what the
value is between two limits
Risk Triangle
Most Popular distribution to show
Most Likely value tapering to a Min
and Max
Risk Trigen
Modification of Triangle
Allows for a finite probability of
achieving Min & Max Values
Fundamental
Flaw of Triangle
and Trigen is
when the
distribution is
skewed
10
Most Likely =5, Min =4, Max =15
Most Likely = 5
Mean = 8
P50= 7.58
Most Likely = 5
Mean = 6.5
P50= 6.16
11
AACE International Recommended Practice No. 41R-08
RISK ANALYSIS AND CONTINGENCY DETERMINATION USING RANGE ESTIMATING
• Monte Carlo software for risk analysis requires identification of a probability density
function (PDF) for each critical item. In rare instances the behavior of a critical item is
known to conform to a specific type of PDF such as a lognormal or beta distribution,
which reflects items that may skew heavily to one side of a distribution. However in
most instances it is unlikely that the actual type of PDF that truly represents the item is
known. Thus a reasonable approximation is to use either
- Triangular Distribution
- Double Triangular Distribution
In most cases, the double triangular distribution is a better approximation since it can be
made to conform to the implicit skew of the project team’s probability assessment. The
double triangle allows the risk analyst to use the probabilities which the project team
believes are reasonable rather than letting the triangular distribution dictate a probability
which, more often than not, is invalid.
12
Probability Density
Double Triangle Method
Area = Underrun Probability
Area = Overrun Probability
a
c
Random Variable x
b
Most Likely = 5
Mean = 6.5
P50= 5.0
13
Double Triangle Method
Probability Density
F1
F2
Urun
Min
Min
-10%
Max
20%
1-Urun
ML
Urun
30%
Max
F1
F2
6
7
RF
1.00
F1= 2*Urun/Min
F2= 2*(1-Urun)/Max
RF=1+RiskGeneral(Min,Max,{0,0},F1:F2,RiskStatic(0))
14
Schedule Risk
Quantitative Schedule Risk Analysis
Husky uses Primavera P6 to schedule all
projects greater than $5MM. Oracle
Primavera Risk, formerly known as
Pertmaster, is used for quantitative
schedule risk analysis (QSRA)
To bring in the cost element of schedule
delay, the results of the QSRA were
brought into the cost model as a set of
percentiles “days delay” from P0 to
P100.
Use of the @Risk Fit Manager was used
to fit the best curve to this profile, with
options set to update the curve each
simulation. This simplified the update
process each time the schedule
model changed.
The schedule @Risk function was then
multiplied by another @Risk function
that represented the uncertainty in
“cost per day” to give an effective cost
for schedule slippage
15
Schedule Cost Risk
27/06/2013
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
55%
60%
65%
70%
75%
80%
85%
90%
95%
100%
25-Feb-13
07-May-13
23-May-13
04-Jun-13
13-Jun-13
21-Jun-13
28-Jun-13
05-Jul-13
11-Jul-13
17-Jul-13
22-Jul-13
28-Jul-13
02-Aug-13
09-Aug-13
16-Aug-13
23-Aug-13
01-Sep-13
11-Sep-13
24-Sep-13
12-Oct-13
24-Dec-13
-122
-51
-35
-23
-14
-6
1
8
14
20
25
31
36
43
50
57
66
76
89
107
180
=RiskLoglogistic(-1063,1088.2,38.361, RiskFit("Schedule","RMSErr"),
RiskName("Schedule"), RiskStatic(0))
16
Use of a Systemic Risk tool
Systemic Risk
Systemic risk drivers such as the level of project scope definition affect
individual, disaggregated estimate line items in a way that is hard to see and
predict.
• Best practice is to address systemic risk drivers using empirical knowledge
(from historical data) to produce stochastic models that link known risk
drivers (e.g. scope definition) to bottom line project cost growth.
• It was decided that this approach would be best suited for the Process Plant,
which formed the major part of the cost estimate.
• Conquest Consulting Group , who have wide experience in the use of such
tools, were engaged to construct a parametric model which used a series of
questionnaires to the project team based on :
- Contractor Organisation
- Contractor Experience
- Project Planning
- Execution Strategy
- Scope Definition
• Fit Manager was again used to integrate the results from the parametric
model into the overall model
17
Systemic Risk Questionnaire
Case Description:
Date:
2
SNAM
9/30/2010 & 10/22/2010
Enter Contractor Bid Value
Enter Execution Schedule Bid Duration
1,079,000
39.8
($ thousands)
(months)
Currency Canadian$
BID MATURITY WORKSHEET
3
Ratings: 0=N/A, 1=Greatly Exceeds, 2 = Exceeds, 3 = Meets, 4 = Does Not Meet, 5 = Fails (EXPECTATIONS)
Rating
Contractor Organization Evaluation
Overall Contractor Capabilities
3
Meets expectations with regard to overall capabilities (including financial strength, reputation, experience, resources, etc.).
added note:
Contractor Organizational Structure
3
Meets expectations with regard to the proposed organization structure to support the project.
added note:
Alignment of JV Partners
3
Meets expectations with regard to clear and documented alignment of JV Partners.
added note:
Key Project Personnel Proposed
4
Does not meet expectations with regard to the key personnel proposed for the project (years of experience, applicable experience, guarantee of
staying with the project, etc.)
added note:
Contractor Ramp-up Strategy and Capability
3
Meets expectations with regard to the proposed ramp-up strategy and capability to adequately resource the project in a timely fashion.
added note:
Contractor Resource Capability and Availability
4
Does not meet expectations with regard to resource capabilities and availability (such as number of in-house resources and capability to contract
resources).
added note:
Alignment with Subcontractors and Suppliers
3
Meets expectations with regard to alignment of the contractor with sub-contractors and suppliers.
added note:
Contractor Organization Evaluation
3.3
18
Specific Risk Input
Probability and Impact
Specific Risks consist of both a probability of occurrence, and an impact should they
occur
• Risk Probability modeled as simple binomial to simulate Yes/No
• Risk Impact modeled as a PDF to give uncertainty of the impact
• Risk outcome modeled as a product of Probability (0 or 1) and Impact
Problems
Prior to release of @Risk 5.0 there were inherent problems using this method in that
Tornado Diagram showed both Probability and Impact as two separate risk inputs
RiskMakeInput
@Risk5.0 onwards allowed for use of RiskMakeInput to combine the two together.
Use extract from Risk Register to directly map risks into the model
RiskID Description
FFH-013 Construction in advance of sufficient engineering
Score
A3
Prob
5%
Min $
2,000,000
Max $
Risk amount $
0
3,500,000
Risk Amount = RiskMakeInput( (RiskBinomial(1, Prob,RiskStatic(0))*
RiskUniform( Min,Max)),RiskName(Description))
19
Risk Results
Summary Results
Individual Risk models were established on separate MSExcel sheets and outputs
summarised on tabular results sheet
Extensive use made of the @Risk function RiskPercentile to provide tabular output that
could be copied and pasted direct into reports
$MM
Process Plant
SubSurface Development
Drilling & Completions
Infrastructure
Midstream
Business & System Management
Ow ners Costs
Additional Specific Risk
TOTAL PROGRAM
Management Reserve
Overall Total
Estimate
Risked Value
50%
$918.3
$918.3 $1,038.2
$1.6
$1.6
$1.8
$193.8
$193.8 $208.4
$124.4
$124.4 $133.4
$30.2
$30.2
$33.2
$11.3
$11.3
$12.8
$156.6
$156.6 $171.7
$.0
$.0
$22.3
$1,436.2
$1,436.2 $1,623.6
$.0
$.0
$17.9
$1,436.2
$1,436.2 $1,648.2
70%
90%
$1,107.7 $1,209.4
$2.0
$2.3
$217.0 $229.3
$137.4 $143.3
$34.6
$36.6
$13.2
$13.9
$176.4 $183.2
$28.2
$38.1
$1,693.4 $1,798.6
$30.7
$59.6
$1,717.8 $1,825.1
P50
13.1%
16.4%
7.6%
7.2%
9.9%
13.2%
9.6%
P70
20.6%
27.2%
12.0%
10.4%
14.6%
16.8%
12.6%
P90
31.7%
42.9%
18.3%
15.2%
21.2%
22.7%
17.0%
13.0%
17.9%
25.2%
14.8%
19.6%
27.1%
Note the values in the above table are for a fictitious project
20
Summary
The cost risk model built in @Risk used a variety of techniques to both represent
uncertainty from ranging workshops and from other studies. Highlights include:
• Use of the RiskGeneral function to mimic the Double Triangular Distribution
recommended by AACE for range estimating
• Use of the Fit Manager to replicate output from other models as input to the overall risk
model, in particular:
- Results from a quantitative schedule risk analysis
- Results from systemic parametric risk analysis
- Automatic recalculation of the curve fit on change of input data
• Use of RiskPercentile and other output functions to provide templated report formats
that can be copied and pasted direct into presentational materials
21
Any Questions
22