11/9/2010
Uncertainty Modeling for Performance and Value Analysis
y
Value Management Strategies, Inc.
Gregg Brink,, CVS,, PMI-RMP,, CCE/A
/
1
11/9/2010
Overview
• Background Information
– Value Engineering (VE)
– Risk Management (RM)
• Performance Assessment
–
–
–
–
Performance Assessment Process
Analytic Hierarchy Process (AHP)
Performance Assessment in the Organization
Performance Assessment of Projects/Programs
• Case Study: SR‐11 New Freeway Construction
– Background
g
Information
– Value Analysis Study
– Value Analysis Modeling
• References
2
11/9/2010
Background
• VMS specializes in Management Consulting
– Application of Value Methodology (VM)
• Value Analysis / Value Engineering
• Function Analysis
– Application of Risk Management
• Project / Program Risk Management
– Qualitative/Quantitative uncertainty modeling
• Cost validation
– Application of Decision Sciences
• Analytic Hierarchy Process
– Decision Lens
• Economic Modeling
• Decision Modeling
3
11/9/2010
The Elements of Value
Performance
Cost
VALUE
Time
Risk
4
11/9/2010
The Value Algorithm
• Value is equivalent to the relationship of the
resources needed to provide a certain level of
performance for a given function
Performance
Value =
Cost
5
11/9/2010
Expanding the Value Algorithm
Schedule as a Resource
• Schedule is an input
Schedule as a Performance Aspect
• Schedule is an output
6
11/9/2010
Incorporating Uncertainty into Value
S h d l
Schedule as a Resource
R
S h d l
Schedule as a Performance Aspect
P f
A
t
• Schedule is an input
• Risk adjusting is
multiplicative
• Schedule is an output
• Risk adjusting is
multiplicative
• Risk impacts are additive
• Risk impacts are additive
7
11/9/2010
Value Engineering Process
8
11/9/2010
Value Engineering
• Encompasses a wide
d variety off applications
l
– Construction projects
– Industrial Processes
– Organizational
g
Processes
– Strategic Planning and Decision Making
• Optimizing the relationships between
performance, cost, time, and risk
• Solution
S l ti d
driven
i
process th
thatt results
lt iin th
the
development of “value added” alternatives
9
11/9/2010
Risk Management Process
Risk
Identification
Risk
Monitoring
and Control
Risk Analysis
Risk
Response
Planning
10
11/9/2010
Changing Uncertainty Over Time Cost distribution changes over project life cycle
Actual
Cost
Planning Stages
Design Stages
Construction
Stage
End
Time
11
11/9/2010
Risk Identification
Identify Risk
Categories
(RBS)
Brainstorm
Risks by
Category
Qualify /
Prioritize Risk
Populate Risk
Register
12
11/9/2010
Risk Analysis
Describe
Risks
EEstablish
t bli h
Priorities
Run
Analysis
Identify
Impacts
Identify
Probability
13
11/9/2010
Defining Potential Outcomes
14
11/9/2010
Risk Analysis
• IInvolves
l
th
the modeling
d li off risks
i k iin either
ith a
qualitative or quantitative fashion
– Qualitative Modeling Æ Relative Indexing
– Quantitative Modeling Æ Range Estimation
– With sufficiency of data both can be simulated in
@Risk
• Requires capturing data of broad range of responses of
subject matter experts (SMEs)
• Risk Impacts are elicited in range estimates from
SMEs
– Low, Most Likely, High
15
11/9/2010
Risk Analysis
• Qualitative
l
models
d l simulate
l
the
h range off
indexed responses
• Quantitative models are constructed using a
triangular
g
distribution for range
g estimates and
discrete distributions in the modeling of
probabilistic events
– EV = (a+4b+c)/6?
– RiskTrigen(L,ML,H,10,90)
RiskTrigen(L ML H 10 90)
– RiskDiscrete(X table, P table)
16
11/9/2010
Risk Analysis
• Risks
Ri k are categorized
t
i db
by TType
– Ex: Design, Construction, ROW, Utilities
• Risk impacts are identified for cost, schedule, and
performance risks
• Risk events treated discretelyy
• Correlations Defined
– Relates Cost and Duration Relationships
– Risk Relationships
– Positive/Negative
• Risk
Ri k D
Dependencies
d i D
Defined
fi d
– Mutually Exclusive, Dependent-Inclusive, Dependent Exclusive
17
11/9/2010
Risk Prioritization
18
11/9/2010
Risk Response Planning
Identify
Response
Strategies
Secondary Risk
Analysis
Develop Action
Plans
Identify
Responsibilities
& Milestones
19
11/9/2010
Pre‐ vs. Post‐Response States
20
11/9/2010
Risk Monitoring and Control
Compile Risk Management Plan
Repeat Risk p
Management Process to Identify New Risks
Provide Status Updates
Establish Reporting Establish
Reporting
and Control Mechanisms
Review Risks at Established Dates/Milestones
21
11/9/2010
RM and VE Combined
Risk Management Process
Identify
Risks
Risk
Analysis
Information
Function
A l i
Analysis
Risk
Monitoring
and
Control
Risk
Response
Planning
Creativity
C
ea
y
Evaluation
Development
e e op e
Presentation
Implementation
pe e a o
Value Engineering Process
22
11/9/2010
Performance Assessment
• Optimal
O i l Performance
f
=O
Optimal
i l Functionality
i
li
– Optimizing performance for a project or product
delivers the desired function at least cost and duration
– Optimizing performance for an organization or
process delivers the desired function in the most
efficient and effective means
• When relating to a project or product
performance is relative to functional scope
• When relating to an organization or process
performance is relative to functional efficiency
23
11/9/2010
Performance Assessment Process
Establish
Performance
Requirements
Define
Performance
Attributes
Define
Attribute
Scales and
Weights
Baseline
Performance
Analysis
Analyze
Mutually
Exclusive
Alternatives
Strategic
Direction
Selection
24
11/9/2010
Analytic Hierarchy Process
• AHP developed
d l
db
by Dr. Thomas
h
S
Saaty
at Wharton
h
School of Business (now at University of Pittsburgh)
– Has been applied to assist organizations in allocating
billions of dollars towards the best projects, vendors,
people and organizational strategies
people,
• Enables Decision Makers to Structure Decisions
Hierarchically
– Decision goal(s) at the top, strategic objectives in
higher levels
levels, evaluation criteria in the middle level
level,
and alternative choices at the bottom
25
11/9/2010
Analytical Hierarchy Process
• P
Provides
id a structured
t t d fframeworkk ffor setting
tti priorities
i iti
on each level of the hierarchy using paired
comparisons
– Process of evaluating each pair of decision factors at a
given level on the model for their relative importance with
respect to their parent
• Consistency of judgments are tracked to validate the
decision process
– In cases where inconsistency is above 10%, it is
recommended that criteria and judgments be revisited
• R
Results
lt iin a model
d l where
h
d
decision
i i makers
k can
collectively see global priorities where the weight of
the decision is distributed from the goal downwards
26
11/9/2010
Analytical Hierarchy Process
• If the
th weight
i ht off any criterion
it i iis iincreased,
d th
then any
alternatives that perform well on the basis of that
criterion will consistently achieve higher relative
ranking scores
– This type of sensitivity analysis is portrayed graphically
using Decision Lens’
Lens software
– Helps demonstrate the impacts of changing priorities on
alternative decisions
• The software utilizes independent voting consoles
– Each participant inputs data in real time
– Consensus building discussions are held to consider and
reinforce rationale
27
11/9/2010
Performance in an Organization/Process
Define Function(s) of
Organization/Processes
Establish Requirements and Attributes
of delivering the desired function(s)
Conduct Performance Analysis
Establish Value Solutions to Enhance
Performance
f
Track Performance Over Time and
Commit to Continuous Improvement
28
11/9/2010
Performance of a Project/Program
Define Function(s) of Project/Program
Establish Requirements and Attributes
of delivering the desired function(s)
Conduct Performance Analysis
Develop Value Alternatives and Assess
P f
Performance
off Alt
Alternatives
ti
Develop Value Strategy
29
11/9/2010
Case Study Background
• SR-11 New Freeway Construction
– Begins at the proposed SR-905/SR-125 junction
– Ends at the East Otay Mesa Port of Entry on the
International Border with Mexico
– Facility length will be 5 km long
– Necessary to eliminate congestion and facilitate future
trade
– Estimated project costs: $350-$550 million
– Estimated construction duration: 650-750 working days
– Construction to be financed by generated toll revenues
30
11/9/2010
Case Study Background
31
11/9/2010
Case Study Background
Six Design Alternatives:
1) Two Interchanges
2) Single Interchange at Alta Rd.
3) Texas U-turn
4) Two Interchanges w/ CD System
5) Partial Interchange at Siempre Viva Rd.
6) No Interchange Option
32
11/9/2010
Method of Analysis
• Value
l Analysis
l
Study
d
– Performance Assessment
• Utilized Decision Lens technology
– Risk Assessment
• Cost and Performance Impacts recognized in $
• Schedule Impacts recognized in Months
– Value Engineering
• Function Analysis
• Development of Value Alternatives
• Development of Value Strategy
33
11/9/2010
Value Engineering Approach
Performance and Risk Enhanced Value Optimization
Cost
Analysis
Risk Analysis
Risk
Validation
Risk
Elicitation
Baseline Risk
Qualification
Establish RiskBased
Priorities
VE Driven
Risk Response
Planning
PostResponse Risk
Qualification
Duration
Analysis
Value a ue
Preparation
Information
Function
Speculation
Evaluation
Development
Implementation
Optimization
Duration
Analysis
Cost
Analysis
Presentation
Performance Analysis
Performance
Definition
Baseline
Performance
Analysis
Establish
PerformanceBased Priorities
Enhanced Value
Analysis
Alternative
Performance
Assessment
Performance
Driven Strategy
Analysis and
Ranking
34
11/9/2010
VE Alternatives
Developed 3 independent alternatives, 2 competing D
l
d3i d
d t lt
ti
2
ti
alternatives:
1.0) Modify Southbound SR-125 to eastbound SR-11
Direct
i
Connector to Tie
i iinto Median
di off SR-11
2.1) Modified Two-Interchange Concept
2.2) Widen Alta Road between Otay Mesa Road and
Siempre Viva Road (Single Interchange Option)
3 0) Locate CVEF between Import and Export
3.0)
Commercial Traffic
4.0) Locate Bus Transit Access (off of Siempre Viva on
West side of POE)
35
11/9/2010
Recommended Strategy
36
11/9/2010
Performance Assessment
• Performance Requirements
– Must meet December 2012 RTL Date
– Construction to begin June 2013
– Environmental Mitigation of all Impacted Areas
– Must accommodate 20 year AADT forecast
37
11/9/2010
Performance Assessment
• Performance
f
Attributes
ib
((7 Gl
Global)
b l)
– Mainline Operations
•
•
•
•
SR-11 Mainline Operations
CVEF Operations
POE Operations
Toll Operations
– Local Operations
•
•
•
•
Enrico Fermi Drive Operations
Alta Road Operations
Siempre Viva Road Operations
Other Local Road Operations
38
11/9/2010
Performance Assessment
•
•
•
•
Land-Use
d
C
Compatibility
ibili
Maintainability
Environmental Impacts
Construction Impacts
– Temporary Traffic Impacts
– Temporary Environmental Impacts
• Project Schedule
– Pre-Construction
Pre Construction Schedule
– Construction Schedule
39
11/9/2010
Performance Attribute Weighting
M i li O
Mainline Operations –
i
28 7%
28.7%
Local Operations – 21.6%
Land‐Use Compatibility –
p
y 15.8% Maintainability – 11.8%
Environmental Impacts –
11.4%
Construction Impacts – 7.3%
Project Schedule –
3.4%
40
11/9/2010
Performance Ratings
41
11/9/2010
Performance Ratings
42
11/9/2010
Risk‐Adjusted Performance
Design Alternatives ‐ Performance Rating Curves
100%
90%
80%
Probability
70%
60%
50%
40%
30%
20%
10%
0%
0.40
0.45
0.50
0.55
0.60
0.65
0.70
0.75
0.80
0.85
0.90
Performance Rating
Two Interchanges / Performance
Single Interchange at Alta Rd / Performance
Texas U-turn / Performance
Two Interchanges w/ CD system / Performance
Partial Interchange at Siempre Viva Rd. / Performance
Modified Two Interchange (VA Concept) / Performance
No Interchange Option / Performance
43
11/9/2010
Risk Assessment
• Id
Identified
tifi d P
Probabilities
b biliti and
d IImpacts
t R
Resulting
lti
from Plausible Event Risks Incurred During Project
Delivery
– Cost (Performance too!) and Schedule Risks
• Threats
• Opportunities
– Cost Risks were additive to the project Base Cost
– Schedule Risks were additive to the project Base
Duration
– Pre-Response
Pre Response and Post
Post-Response
Response States Considered
– Value Strategy is modeled in the Post-Response State
44
11/9/2010
Flowchart – An Illustration
Flowchart: SR‐11 New Freeway Construction
Costs to date
0
Design
Documentation/
PFA 1 (12 months)
Design
Approval
4
North End
B id
Bridge
STAGE 2 21
(7 months)
North End
Bridge
STAGE 1
16 (10 months)
Environmental Permits
5 (12 months)
Access Evaluation
2
(9 months)
Environmental
Documentation 3
(14 months)
North End
Road Median Widening
17 (6 months)
R/W Acquisition
6 (12 months)
Wetland Mitigation
7 (8 months)
R/W
Certification
10
North End
Road
West Side
Widening
22
(3 months)
North End
Road
and Bridge
STAGE 3
26
(7 months)
CVEF Utilities
23 ((6 months))
Hydraulics Report
8 (12 months)
AD
15
Geotechnical
9 (18 months)
PS&E
11 (15 months)
POE Area
STAGE 1
30 (7 months)
POE Area
STAGE 3
32 (6 months)
Final
Overlay
33
(3 months)
South End
STAGE 1
19 (8 months)*
South End
STAGE 2
25 (6 months)
RR Agreement
12 (6 months)
Agency Agreements
13 (9 months)
POE Area
STAGE 2
31 (6 months)
Replace 12/118 &
Widen 12/117 STAGE 1
20 (8 months)
South End
STAGE 3
29 (9 months)
Replace 12/118 &
Widen 12/117 STAGE 2
20b (8 months)
Other Agreements
14 (6 months)
Note: Not to Scale
(1) Assume single design/bid/build contract
(2) Construction closure periods: Fish windows for activities (?) Winter weather shutdown (?)
FISH WINDOWS
*Accelerated Process
45
11/9/2010
Establishing Ranges for all Key Input Variables
Labor Rate ($/Hour)
Median
10% Lower
10% Upper
$50
$45
$75
80% Confidence Interval
PROBAB
BILITY DENSITY
0.25
0.2
0.15
0.1
0.05
0
0
2
4
6
8
10
12
14
Value of Time – Auto Driver or Passenger, 4/hour
16
46
11/9/2010
Identify, Quantify, and Respond… 47
11/9/2010
Risk Register: Risk Identification
48
11/9/2010
Risk Register: Risk Quantification
49
11/9/2010
Risk Register: Risk Response Planning
50
11/9/2010
Risk Assessment
Design Alternatives ‐ Cost Range Curves
100%
90%
80%
Probability
70%
60%
50%
40%
30%
20%
10%
0%
$300
$350
$400
$450
$500
$550
$600
$650
$700
$750
Millions
Cost Range
Two Interchanges - Cost Range
Single Interchange at Alta Rd - Cost Range
Texas U-turn - Cost Range
Two Interchanges w/ CD system - Cost Range
Partial Interchange at Siempre Viva Rd. - Cost Range
Modified Two Interchange (VA Concept) - Cost Range
No Interchange Option - Cost Range
51
11/9/2010
Risk Assessment
Design Alternatives ‐ Duration Range Curves
100%
90%
80%
Probabillity
70%
60%
50%
40%
30%
20%
10%
0%
15
17
19
21
23
25
27
29
31
33
35
Duration Range
Two Interchanges - Duration Range
Single Interchange at Alta Rd - Duration Range
Texas U-turn - Duration Range
Two Interchanges w/ CD system - Duration Range
Partial Interchange at Siempre Viva Rd. - Duration Range
Modified Two Interchange (VA Concept) - Duration Range
No Interchange Option - Duration Range
52
11/9/2010
Modeling of Value
• D
Developed
l
dM
Monte
t C
Carlo
l simulation
i l ti model
d l
based on the following algorithm for Value:
• Performance Parameters estimated during the
Performance Assessment
• Cost and
d Duration parameters estimated
d in
the Risk Assessment
• Ranges of outcomes developed
53
11/9/2010
Performance / Cost
Design Alternatives ‐ Performance / Cost Index Curves
100%
90%
80%
Probabillity
70%
60%
50%
40%
30%
20%
10%
0%
0.40
0.60
0.80
1.00
1.20
1.40
1.60
1.80
2.00
Performance Rating
Two Interchanges / P/C
Single Interchange at Alta Rd / P/C
Texas U-turn / P/C
Two Interchanges w/ CD system / P/C
Partial Interchange at Siempre Viva Rd. / P/C
Modified Two Interchange (VA Concept) / P/C
No Interchange Option / P/C
54
11/9/2010
Performance / Duration
Design Alternatives ‐ Performance / Duration Index Curves
100%
90%
80%
Probabillity
70%
60%
50%
40%
30%
20%
10%
0%
0.40
0.60
0.80
1.00
1.20
1.40
1.60
1.80
2.00
Performance Rating
Two Interchanges / P/T
Single Interchange at Alta Rd / P/T
Texas U-turn / P/T
Two Interchanges w/ CD system / P/T
Partial Interchange at Siempre Viva Rd. / P/T
Modified Two Interchange (VA Concept) / P/T
No Interchange Option / P/T
55
11/9/2010
Total Value
Design Alternatives ‐ Value Index Curves
100%
90%
80%
Probabillity
70%
60%
50%
40%
30%
20%
10%
0%
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
Value Index
Two Interchanges / Value Index
Single Interchange at Alta Rd / Value Index
Texas U-turn / Value Index
Two Interchanges w/ CD system / Value Index
Partial Interchange at Siempre Viva Rd. / Value Index
Modified Two Interchange (VA Concept) / Value Index
No Interchange Option / Value Index
56
11/9/2010
REFERENCES
•
•
•
•
•
S
Stewart,
R
Robert,
b
“V
“Value
l O
Optimization
i i i ffor P
Project
j and
dP
Performance
f
M
Management”,
” JJohn
h Wiley
Wil
and Sons, Inc., Hoboken, NJ, 2010, pgs. 21-46.
Saaty, Thomas, “Decision Making for Leaders: The Analytic Hierarchy Process for Decision in
a Complex World”, New Edition 2001, (Analytic Hierarchy Process Series Vol. 2)
Hubbard, Douglas, “How to Measure Anything: Finding the Value of Intangibles in Business”,
John Wiley and Sons, Inc. Hoboken, NJ, 2007, pgs. 43-47
Stewart, Robert, Cilch, Chili, “The
The Application of Value Profiling on Public Projects”,
Projects , An An
Examination of the California Department of Transportation’s Performance Measures Process, Institute for Value Management, Bristol, England, 2006, pgs. 3-10
Stewart, Robert, Bernard, Nicole, Baza, Mark, “How VE and Decision Analysis are Improving
Mobility across Borders”, 2009 AASHTO Value Engineering Conference, pgs. 1-10
57
11/9/2010
58