Risk Management Capability
The Critical Link between Quantitative
Risk Analysis and Project Approvals
- improving the performance of UK MoD
equipment procurement projects
Martin Hopkinson
© Risk Management Capability Limited 2012
UK MoD Equipment Procurement
© Risk Management Capability Limited 2012
Slide No: 2
CADMID Project Cycle
Equipment Procurement
Initial Gate
Main Gate
Demonstration and
Concept Assessment
Manufacture
In-service
Disposal
Equipment Support
© Risk Management Capability Limited 2012
Slide No: 3
Historic Schedule Performance
Project
Main Gate
Slip
Typhoon
Nov-87
41%
Brimstone
Mar-96
64%
Jul-96
141%
Astute
Mar-97
62%
ASTOR
Jun-99
51%
A400M
May-00
69%
BVRAAM
May-00
17%
Type 45
Jul-00
46%
Typhoon ASTA
Oct-00
27%
Nimrod MRA4
0
-20%
20%
40%
60%
80%
100%
120%
140%
160%
Schedule Slip
Moving average of 10 projects
Source data:
National Audit Office
Major Projects Reports
2005 - 2010
© Risk Management Capability Limited 2012
Slide No: 4
Monte Carlo Schedule Risk Models
Uncertain outcome of activity durations
estimated with probability distributions
Min
Mode
Max
Models generally
have <200 activities
Uncertain effects of risk events
estimated with probabilities and
impact probability distributions
Min
Mode
Max
Variation of milestone dates is
driven by Monte Carlo
simulation, with results typically
shown using s-curves
X%
Typically, probability distributions are defined using a
shape (e.g. Beta Pert) and a three-point estimate
© Risk Management Capability Limited 2012
Slide No: 5
Assessment Phase - Risk Reduction
Cumulative Probability
100%
90%
Main Gate
Forecast
Initial Gate
Forecast
50%
10%
Overall Project Cost (£M)
or Time (In-service date)
© Risk Management Capability Limited 2012
Slide No: 6
Three Point Confidence Forecasts
Confidence Forecasts (Percentiles)
P10
P50
P90
Cumulative Probability
100%
90%
50%
10%
Overall Project Cost (£M)
or Time (in-service date)
© Risk Management Capability Limited 2012
Slide No: 7
Risk Differentials
Risk Differential Consumed =
P50C – P50MG
P90MG – P50MG
Risk
Differential
100%
90%
Current
Forecast
50%
Forecast at
Main Gate
10%
Completion Date
© Risk Management Capability Limited 2012
Slide No: 8
Projects Passing Main Gate 2000-04
Project
Main Gate
RDC
A400M
May-00
7.3
BVRAAM
May-00
2.1
Type 45
Jul-00
6.3
Typhoon ASTA
Oct-00
4.0
Tojan & Titan
Jan-01
0.9
Bowman
Aug-01
0.0
Support Vehicle
Nov-01
4.2
NLAW
May-02
3.6
Terrier
Jul-02
18.4
CIP
Oct-02
2.3
LF Anti-tank GW
Jan-03
-0.5
Precision Guided Bomb
Jun-03
0.5
Panther
Jul-03
0.8
GMLRS
Aug-03
0.0
Naval EHF/SHF Sat Coms
Aug-03
31.0
Soothsayer
Aug-03
5.0
C-Vehicle Capability
Dec-03
0.8
-4
0
4
8
12
16
20
24
28
32
Risk differential consumed (RDC)
-4
0
4
8
© Risk Management Capability Limited 2012
12
16
20
24
28
32
Slide No: 9
Addressing Schedule Performance Risk
April 1999 Introduction of Smart Procurement and risk-based
confidence forecasts at Main Gate
April 2003 Measurements of Risk Management Process Capability
These measurements were made by auditing projects
using the Project Risk Maturity Model
© Risk Management Capability Limited 2012
Slide No: 10
Project Risk Maturity Model
Example of a Project at RMM Level 2
100%
Level 4
75%
Level 3
50%
Level 2
25%
Level 1
Stakeholders
Risk
Identification
Risk
Analysis
Risk
Responses
Project
Management
© Risk Management Capability Limited 2012
Culture
Slide No: 11
Process Capability 2004 - 2004
In the period April 2003 – March 2004, the Project Risk
Maturity Model (RMM) was used to measure process
capability on 29 of the MoD’s 30 Major Projects
RMM Level
No. of Projects
Level 4 - Natural
3
Level 3 - Normalised
4
Level 2 - Novice
16
Level 1 - Naive
1
© Risk Management Capability Limited 2012
Slide No: 12
Addressing Schedule Performance Risk
April 1999 Introduction of Smart Procurement and risk-based
confidence forecasts at Main Gate
April 2003 Measurements of Risk Management Process Capability
April 2004 Main gate approvals conditional upon demonstration of
Project Risk Maturity Model Level 3 or greater
April 2008 Project Risk Maturity Model demonstrations substituted
by Verification and Validation audit of risk models
© Risk Management Capability Limited 2012
Slide No: 13
Project Risk Maturity Model
From April 2004, projects with a value >£25m had to demonstrate
a process capability of RMM Level 3 or greater
100%
Level 4
75%
Level 3
50%
Level 2
25%
Level 1
Stakeholders
Risk
Identification
Risk
Analysis
Risk
Responses
Project
Management
© Risk Management Capability Limited 2012
Culture
Slide No: 14
Did It Work?
Did projects with a higher risk management
capability at Main Gate Approval have
better outcomes than their predecessors?
© Risk Management Capability Limited 2012
Slide No: 15
Trend in Risk Differential Consumed
Project
Main Gate
RDC
A400M
May-00
7.3
BVRAAM
May-00
2.1
Jul-00
6.3
Typhoon ASTA
Oct-00
4.0
Tojan & Titan
Jan-01
0.9
Bowman
Aug-01
0.0
Support Vehicle
Nov-01
4.2
NLAW
May-02
3.6
Terrier
Jul-02
18.4
CIP
Oct-02
2.3
LF Anti-tank GW
Jan-03
-0.5
Precision Guided Bomb
Type 45
Jun-03
0.5
Panther
Jul-03
0.8
GMLRS
Aug-03
0.0
Naval EHF/SHF Sat Coms
Aug-03
31.0
Soothsayer
Aug-03
5.0
C-Vehicle Capability
Dec-03
0.8
MTADS/PNVS
Sep-04
0.3
Watchkeeper
Jul-05
0.7
Falcon
Mar-06
0.7
Merlin Sustainment
Mar-06
0.0
Future Lynx
Jun-06
0.0
Advanced Jet Trainer
Aug-06
0.7
Future Strat' Tanker Aircraft
May-07
0.0
Jul-07
1.3
Feb-08
0.7
Queen Elizabeth Class
UK MFS Increment C
-4
0
4
8
12
16
20
24
28
32
Risk differential consumed (RDC)
Moving average of 10 projects
RMM "Level 3+" required
Source data:
National Audit Office Major
Projects Reports up to 2010
-4
0
4
8
12
© Risk Management Capability Limited 2012
16
20
24
28
32
Slide No: 16
Trend in Percentage Delay
Project
Main Gate
Slip
Typhoon
Nov-87
41%
Brimstone
Mar-96
64%
Nimrod MRA4
Jul-96
141%
Astute
Mar-97
62%
ASTOR
Jun-99
51%
A400M
May-00
69%
BVRAAM
May-00
17%
Type 45
Jul-00
46%
Typhoon ASTA
Oct-00
27%
Tojan & Titan
Jan-01
21%
Bowman
Aug-01
0%
Support Vehicle
Nov-01
63%
NLAW
May-02
54%
Terrier
Jul-02
74%
CIP
Oct-02
124%
LF Anti-tank GW
Jan-03
-12%
Precision Guided Bomb
Jun-03
6%
Panther
Jul-03
25%
GMLRS
Aug-03
0%
Naval EHF/SHF Sat Coms
Aug-03
42%
Soothsayer
Aug-03
75%
C-Vehicle Capability
Dec-03
23%
MTADS/PNVS
Sep-04
8%
Watchkeeper
Jul-05
10%
Falcon
Mar-06
12%
Merlin Sustainment
Mar-06
0%
Future Lynx
Jun-06
0%
Advanced Jet Trainer
Aug-06
20%
Typhoon Future Capabilty
Jan-07
0%
Future Strat' Tanker Aircraft
May-07
0%
Jul-07
10%
Feb-08
65%
Queen Elizabeth Class
UK MFS Increment C
-20%
0
20%
40%
60%
80%
100%
120%
140%
160%
Schedule Slip
Risk-based forecasts introduced
Moving average of 10 projects
April 04: RMM "Level 3+" required
Source data:
National Audit Office Major
Projects Reports up to 2010
-20%
0
20%
40%
60%
80%
© Risk Management Capability Limited 2012
100%
120%
140%
160%
Slide No: 17
Conclusions
1.
Project risk management can be demonstrated to work
2.
Quantitative Risk Analysis can help an organisation to
improve its record for achieving project objectives.
3.
However, the organisation may also need to implement a
process for measuring and improving process capability
4.
Setting realistic objectives at the main project approval
point is important (projects that missed their schedule
objectives continued to underperform)
© Risk Management Capability Limited 2012
Slide No: 18
Risk Management Capability Ltd
www.rmcapability.com
Free pdf downloads of papers and
risk management capability sheets
© Risk Management Capability Limited 2012
Slide No: 19