Broadleaf
Making good use of risk models
in major projects
Dr Stephen Grey
Broadleaf Capital International Pty Ltd
2006
© Broadleaf Capital International - www.Broadleaf.com.au
Dr Stephen Grey - Grey@Broadleaf.com.au
1
Broadleaf
© Copyright Broadleaf Capital International Pty Ltd, 2006
BROADLEAF CAPITAL INTERNATIONAL PTY LTD
ACN 054 021 117
23 Bettowynd Road Pymble NSW 2073
Tel: +61 2 9488 8477 Fax: +61 2 9488 9685
Cooper@Broadleaf.com.au
www.Broadleaf.com.au
This document contains substantial pre-existing Intellectual Property of value to
Broadleaf Capital International Pty Ltd (Broadleaf).
It is provided for the information of clients to whom it is released by Broadleaf, but not to be sold,
licensed or otherwise transferred, whether in its original form or as part of any further
development that they might undertake, without Broadleaf’s prior agreement.
2006
© Broadleaf Capital International - www.Broadleaf.com.au
Dr Stephen Grey - Grey@Broadleaf.com.au
2
Broadleaf
Risk models in major projects
Overview
Trends
Common problems
Sources of guidance
Geographical differences
Barriers and challenges
Way forward
2006
© Broadleaf Capital International - www.Broadleaf.com.au
Dr Stephen Grey - Grey@Broadleaf.com.au
3
Broadleaf
Current situation & trends
Project scale rising
– Marginal mineral resources
– Economies of scale
– Big packages to attract PPP and alliance
proponents
Owners’ skill base declining
– Small team or single in-house specialist
– Rely on external consultants
• Multi consultant teams
• Multi national teams
2006
© Broadleaf Capital International - www.Broadleaf.com.au
Dr Stephen Grey - Grey@Broadleaf.com.au
4
Current situation & trends
Broadleaf
Volatile escalation
– Fuel and other oil based products
– Steel
– Professional effort
Resource shortages
–
–
–
–
2006
Engineers
Machinery
Major equipment vendor capacity
Fabrication shop capacity
© Broadleaf Capital International - www.Broadleaf.com.au
Dr Stephen Grey - Grey@Broadleaf.com.au
5
Project owners’ responses
Broadleaf
More peer reviews and external checks
Closer attention to risk
Formal risk assessment
Model cost and schedule risk
Model business and economic risk
Use model to set
• Budgets
• Stretch targets
• Warning levels
Many expect software to “save the day”
2006
© Broadleaf Capital International - www.Broadleaf.com.au
Dr Stephen Grey - Grey@Broadleaf.com.au
6
Software tools
Broadleaf
Good
Inexpensive
Highly capable
Bad
Any fool can use them and many do
Easy to produce meaningless but glossy results
Ugly
Unrealistic outputs
Erroneous conclusions
Loss of faith in the process
2006
© Broadleaf Capital International - www.Broadleaf.com.au
Dr Stephen Grey - Grey@Broadleaf.com.au
7
Put tools in their place
Broadleaf
Goal
Process
maturity
Now
8
Understand the
process before
becoming immersed
in tools
Use of software tools
Only about
10% - 20% of effort
should go on modeling
2006
Interpreting
& using results
Formulating
the analysis
Modeling
Gathering
information
© Broadleaf Capital International - www.Broadleaf.com.au
Dr Stephen Grey - Grey@Broadleaf.com.au
8
Putting tools first
Broadleaf
“Add risk” to existing estimates & schedules
Hundreds of cost lines
Thousands of activities
Some parts a bit rough
Easily become
large, cumbersome
and difficult to
understand
Expedient approximations
Fixed dates that are not really fixed
“Plug” numbers
Existing
models
2006
+
Risks &
opportunities
=
© Broadleaf Capital International - www.Broadleaf.com.au
Dr Stephen Grey - Grey@Broadleaf.com.au
Risk
model
9
Putting tools first
Broadleaf
Very large models
Summary
Lots of data gathering
Validation hard
Understanding poor
Run times long
Detail is GOOD
Project uncertainty
is understood here
<10
10-100
We can add value
by examining it here
100-1000
Detail
Scale and complexity
>1000 overwhelm us here
Very large “bottom up” exercises often absorb
a lot of effort and deliver little or are abandoned
2006
© Broadleaf Capital International - www.Broadleaf.com.au
Dr Stephen Grey - Grey@Broadleaf.com.au
10
Costs
Broadleaf
Forecasting cost
Project
Forecast
project
cost
Add them up
Work
Breakdown
Structure
2006
Work
Package
estimates
© Broadleaf Capital International - www.Broadleaf.com.au
Dr Stephen Grey - Grey@Broadleaf.com.au
11
Schedule
Broadleaf
Forecasting duration
Forecast
project
duration
Project
Critical path
analysis
Tasks
2006
Task duration
estimates
© Broadleaf Capital International - www.Broadleaf.com.au
Dr Stephen Grey - Grey@Broadleaf.com.au
12
Risk
Broadleaf
Forecasting uncertainty
Forecast
uncertainty
in project
outcome
Project
Monte Carlo
simulation
WBS
Tasks
Events
2006
Uncertainty
in components
© Broadleaf Capital International - www.Broadleaf.com.au
Dr Stephen Grey - Grey@Broadleaf.com.au
13
Modeling
Broadleaf
Modeling uncertainty is, in principle, no different
to modeling anything else
It is not impossible to do it well
It easy to do it badly
We all need support
– Structured process
– Guidance on where and how to apply effort
– Appropriate tools and the skills to use them
2006
© Broadleaf Capital International - www.Broadleaf.com.au
Dr Stephen Grey - Grey@Broadleaf.com.au
14
Sources of guidance
Broadleaf
AS/NZS4360
Local standard
Generally used to guide qualitative methods
Can help with quantitative analysis too
PMBOK Chapter 11
Project Management Institute (USA and the World)
Deals with qualitative and quantitative methods
Handbook and Practice Standard in development
PRAM Guide
UK Association for Project Management
Deals with qualitative and quantitative methods
Guide to process and tools
2006
© Broadleaf Capital International - www.Broadleaf.com.au
Dr Stephen Grey - Grey@Broadleaf.com.au
15
Comparing the three
Broadleaf
Approximate relationship
AS4360
PRAM
Ch11
2006
Establish
context
Define
project
Planning
Risk
identification
Focus
PRAM
Risk
analysis
Identification
Identification
Qualitative
analysis
Risk
evaluation
Assessment
Quantitative
analysis
© Broadleaf Capital International - www.Broadleaf.com.au
Dr Stephen Grey - Grey@Broadleaf.com.au
Risk
treatment
Planning
Response
planning
Monitoring
& review
Management
Monitoring
& control
16
Major differences
Broadleaf
AS/NZS4360
True standard – core requirements
Well established in general risk management
Not yet as well established in quantitative work
Scalable – equally relevant to all sizes and types of project
PMBOK Chapter 11
Mixture of guide and standard
Mixture of process and methods
Implicit big project high tech bias
PRAM Guide
Guide not a standard
Separates process from tools
Broad applicability
Easy to read - accessible
2006
© Broadleaf Capital International - www.Broadleaf.com.au
Dr Stephen Grey - Grey@Broadleaf.com.au
17
Geographical differences
Broadleaf
Australia & New Zealand
Patchy uptake of quantitative modeling - variable quality
Sound approach to risk from AS/NZS4360
Expectation that risk management will be taken seriously
Some strong demand for realistic quantitative assessments
USA
Widespread use of models – distrust of qualitative methods
Tendency towards large models (eg. 2000 or more activities)
Belief in detail and “objective” data
UK & Europe
Long history of using risk models – North Sea oil etc.
Less obsessed with massive detail (some exceptions)
Diverse and creative
2006
© Broadleaf Capital International - www.Broadleaf.com.au
Dr Stephen Grey - Grey@Broadleaf.com.au
18
Broadleaf
Outstanding requirements
Overall approach
Integration with qualitative analysis
Integration with other processes
Estimating
Scheduling
Statement of core requirements
Guide to good practice
Engaging stakeholders and interpreting output
2006
© Broadleaf Capital International - www.Broadleaf.com.au
Dr Stephen Grey - Grey@Broadleaf.com.au
19
Broadleaf
Barriers & challenges
Prevalence of simplistic views (eg. Risk = Probability x
Impact)
Effort devoted to conventional methods creates
resistance to alternatives
Confusion between estimating rules and uncertainty
(eg. allowance for fittings)
Bottom up mind set versus top down analysis
Fear of unfamiliar ideas and methods
Obsession with the right answer instead of realism
2006
© Broadleaf Capital International - www.Broadleaf.com.au
Dr Stephen Grey - Grey@Broadleaf.com.au
20
Broadleaf
Barriers & challenges
“It’s not the way we do things”
2006
© Broadleaf Capital International - www.Broadleaf.com.au
Dr Stephen Grey - Grey@Broadleaf.com.au
21
Way forward
Broadleaf
AS/NZS4360
Consult & communicate
Establish
the context
Objectives
Stakeholders
Criteria
Key elements
Identify
the risks &
opportunities
Analyse
the risks &
opportunities
Evaluate
What can
happen?
How could it
happen?
Review controls
Likelihoods
Consequences
Significance
Rank risks &
opportunities
Screen out
minor issues
Evaluate risks
& opportunities
Treat the risks
& exploit the
opportunities
Identify options
Select the best
Develop plans
Implement
Monitor and review
Proven process for understanding and managing risk
Australia & NZ have a solid base of expertise
2006
© Broadleaf Capital International - www.Broadleaf.com.au
Dr Stephen Grey - Grey@Broadleaf.com.au
22
Systematic approach to QRA
Broadleaf
Consult & communicate
Establish the context
Judgment & experience
Qualitative context
Objectives
Identify
the risks &
opportunities
Analyse
the risks &
opportunities
Evaluate
What can
happen?
How could it
happen?
Review controls
Likelihoods
Consequences
Significance
Rank risks &
opportunities
Screen out
minor issues
Structure
Parameters
Interpretation
Stakeholders
1
Criteria
Key elements
2
Quantitative
context
Plan & implement
actions
Evaluate risks
& opportunities
Identify options
Select the best
Develop plans
Implement
5
Modeling & analysis
Scope
Measures or KPIs
3
Treat risks & exploit
opportunities
Basis for calculation
Control environment
Time
(min,likely,max)
Uncertainty
Financial
Probabilities
Risk to targets
Physicals
Distributions
Sources of
Operations
Correlations
uncertainty
Reconcile targets
and risk levels
Set or revise targets
Revise plans
Revise policy and
control environment
6
4
Quantitative validation and reconciliation
Judgment & experience
Monitor and review
2006
© Broadleaf Capital International - www.Broadleaf.com.au
Dr Stephen Grey - Grey@Broadleaf.com.au
23
Qualitative and quantitative
Broadleaf
Don’t confuse the two
Distinct views of
the same thing
Aggregate
risk
Quantitative
risk model
Must be consistent
with one another
Model components
•Description
•Priorities
•Plans
Risks
Risks
x
x
x x
x
Qualitative analysis
• cost-effective identification
• priority setting
• planning and review
• focus on risks as separate issues
Quantitative modelling
• aggregate view
• focus on overall decision making
• targets, commitments and contingencies
2006
© Broadleaf Capital International - www.Broadleaf.com.au
Dr Stephen Grey - Grey@Broadleaf.com.au
24
Good practice
Broadleaf
Consult & communicate
Establish the context
Qualitative context
Objectives
Treat risks & exploit
opportunities
Identify
the risks &
opportunities
Analyse
the risks &
opportunities
Evaluate
What can
happen?
How could it
happen?
Review controls
Likelihoods
Consequences
Significance
Rank risks &
opportunities
Screen out
minor issues
Structure
Parameters
Interpretation
Stakeholders
Criteria
Key elements
Quantitative
context
Clear
Measures or KPIs
context
Basis for calculation
Scope
Control environment
Structure (min,likely,max)
and
Financial parameters
Probabilities
Physicals
Distributions
linked
to
risks
Operations
Correlations
Time
Evaluate risks
& opportunities
Reflecting
Uncertainty
Risk to targets
significance
of
ofSources
the
risks
uncertainty
Plan & implement
actions
Identify options
Select the best
Develop plans
Implement
Reconcile targets
and risk levels
Used to
inform
Revise policy and
decisions
control
environment
Set or revise targets
Revise plans
Reconciled
withandother
views
Quantitative validation
reconciliation
Monitor and review
2006
© Broadleaf Capital International - www.Broadleaf.com.au
Dr Stephen Grey - Grey@Broadleaf.com.au
25
Conclusions
Broadleaf
Process
Get the process right then focus on tools
Good practice not yet established anywhere
Significant cultural and regional differences
AS/NZS4360 is a good foundation
Approach and culture
Use models to understand projects
Don’t expect them to make decisions for you
Value experience and judgment as well as models
2006
© Broadleaf Capital International - www.Broadleaf.com.au
Dr Stephen Grey - Grey@Broadleaf.com.au
26
Closing thought
Broadleaf
ME
MO
E
DG
JU
DE
L
When your model of your project and your
understanding of your project match up, who
can argue?
NT
PROJECT
2006
© Broadleaf Capital International - www.Broadleaf.com.au
Dr Stephen Grey - Grey@Broadleaf.com.au
27
Questions
Broadleaf
2006
© Broadleaf Capital International - www.Broadleaf.com.au
Dr Stephen Grey - Grey@Broadleaf.com.au
28
Broadleaf
Dr Stephen Grey
ph 03 9787 0520
mob 0412 223 256
Grey @Broadleaf.com.au
2006
© Broadleaf Capital International - www.Broadleaf.com.au
Dr Stephen Grey - Grey@Broadleaf.com.au
29