Lean Aerospace Initiative Plenary Workshop
Models for Analyzing Complex System Development
Cost, Schedule, and Performance Risks
October 14, 1998
Presented By:
Tyson R. Browning
Product Development Focus Team
Research Sponsored By Lean Aerospace Initiative
Lean Aerospace
Initiative
l
Who is This For?
Decision makers needing to balance project cost
and schedule and product technical performance
l
Project planning
l
Project management and control
l
Proposal preparation
l
Design reviews
l
Project replanning
Tyson R. Browning — 2
PD Browning 101498 ©1998 Massachusetts Institute of Technology
Lean Aerospace
Initiative
l
Motivation
l
Approach and goals
l
New results
Overview
– Model one: cost and schedule risk
– Data: UCAV process
– Model two: cost, schedule, and performance (CS&P) risk
l
Product development management insights
Tyson R. Browning — 3
PD Browning 101498 ©1998 Massachusetts Institute of Technology
Lean Aerospace
Initiative
l
l
l
Research Motivation
Competitive advantage,
Maximum return on investment, and
Maximum product value
…depend on balancing…
l
l
l
Product price (or life cycle operating cost)
Delivery timing
“Technical” performance
“Overall Performance” or
“Value to Customer”
=> Establish and maintain balance of product performance
dimensions
=> Establish and maintain balance in development process
=> Understand sources and impacts of uncertainty (risk)
Tyson R. Browning — 4
PD Browning 101498 ©1998 Massachusetts Institute of Technology
Lean Aerospace
Initiative
CS&P Trade-offs
Cost
Schedule
Technical Performance
l
l
l
Different process configurations have different C&S outcomes
They may also deliver differing levels of expected performance
We need a way to help navigate these choices
Tyson R. Browning — 5
PD Browning 101498 ©1998 Massachusetts Institute of Technology
Lean Aerospace
Initiative
Research Approach and Goals
l
Observe product development processes to determine
sources of uncertainty and risk
l
Develop methodology and model for quantifying CS&P risks
in PD processes
l
Use model to provide insights for improved PD process
planning and management
Tyson R. Browning — 6
Observation
Modeling
Improvement
Understanding
PD Browning 101498 ©1998 Massachusetts Institute of Technology
Lean Aerospace
Initiative
Design Structure Matrix
Preliminary Design Activities (Example Set)
A4
A511
A512
A531
A521
A522
A5341
A532
A533
A5342
A5343
A5344
A54
A6
Prepare UCAV Preliminary DR&O
Create UCAV Preliminary Design Configuration
Prepare & Distribute Surfaced Models & Int. Arngmt. Drawings
Perform Aerodynamics Analyses & Evaluation
Create Initial Structural Geometry
Prepare Structural Geometry & Notes for FEM
Develop Structural Design Conditions
Perform Weights & Inertias Analyses
Perform S&C Analyses & Evaluation
Develop Balanced Freebody Diagrams & Ext. Applied Loads
Establish Internal Load Distributions
Evaluate Structural Strength, Stiffness, & Life
Preliminary Manufacturing Planning & Analyses
Prepare UCAV Proposal
•
•
•
•
•
•
•
1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
2
3
4
5
6
7
8
■
■
■
■
■
■
■
■
9 10 11 12 13 14
■
■
■
■
■
■ ■
■
■
■
■ ■
■
■
■ ■ ■
■ ■ ■
■ ■
■
■
■
■ ■
■
■
■
■
■ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■
Square matrix showing relation of each activity to every other
Row: information required (inputs)
Column: information provided (outputs)
Time-based ordering of activities
Counter-clockwise flow of information between activities in process
Alternating bands show concurrency
Superdiagonal marks show iteration/rework
Tyson R. Browning — 7
PD Browning 101498 ©1998 Massachusetts Institute of Technology
Lean Aerospace
Initiative
l
l
A Model to Quantify
Cost and Schedule Risk
DSM representation
Activity-based
– Stochastic durations and costs
– Learning curve effect on rework
l
Accounts for rework caused by timing of
information exchange
– Probabilistic iteration
– Variable impacts (% rework)
l
l
l
Dynamic, Monte Carlo simulation
Activity concurrency (info.-based)
Outputs: cost and schedule outcome
distributions and risk factors
Tyson R. Browning — 8
PD Browning 101498 ©1998 Massachusetts Institute of Technology
Lean Aerospace
Initiative
Model Algorithm (Simplified)
Rework Probabilities
1
2
3
4
5
6
7
A4
A511
A512
A531
A521
A522
A5341
A532
A533
Prepare UCAV Preliminary DR&O
1
Create UCAV Preliminary Design Configuration 2 .4
Prepare & Distribute Surfaced Models & Internal Arrangement
3
.5
Drawings
.4
Perform Aerodynamics Analyses & Evaluation
4 .3
.5.5
.3
.5
Create Initial Structural Geometry
5 .4
.5
.1
.4
Prepare Structural Geometry & Notes for FEM
6 .1
.4.8
.1
.1 .4
Develop Structural Design Conditions
7 .4
.1
Perform Weights & Inertias Analyses & Evaluation8
.5
Perform S&C Analyses & Evaluation
9 .4
.5.5 .5
A5342
A5343
A5344
A54
A6
Develop Balanced Freebody Diagrams & External10Applied Loads.1
.3
Establish Internal Load Distributions
11
Evaluate Structural Strength, Stiffness, & Life 12 .4
Preliminary Manufacturing Planning & Analyses 13 .5
Prepare UCAV Proposal
14 .3 .4 .4.5 .4
.3
.5
.3
.5
.1
.4
.8
9
10
12
13
14
W
WN
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
0
0
0
0
0
0
0
0
0
0
0
0
0
.1
.1
.3
.1
.2
.3
.5
.5
.3
.4
.5
.4
.4
.8
.8
.4
.4
.9
.3
.4
.8
.8
.8
.5
.4
.5
.5
.1
.5
.8
.5
.5
.1
.4
.5
.1
.0
.3
.5
.5
.8
11
.2
.3
.5
.3.5
.9
.5
8
.3
.4
.3.4
.4 .5.4 .5.4
.3
.8
.8
.8
.8
.8
Rework Impacts
l
l
l
Based on time steps
Sample activity durations (convert to time steps) and costs
Each time step
– Determine band of concurrent activities (WN)
– Do work; add time and cost (S and C)
– Check for iteration and second-order rework; add impact to W
l
Simulate hundreds of runs to generate C, S sample pairs
Tyson R. Browning — 9
PD Browning 101498 ©1998 Massachusetts Institute of Technology
Lean Aerospace
Initiative
l
l
l
l
Uninhabited Combat Aerial Vehicle (UCAV) at Boeing
Conceptual and preliminary design
Collected through interviews and surveys
Process data
–
–
–
–
–
–
l
l
UCAV Process Report
Activities
Durations
Required resources
Learning curve effect on rework
Inputs and outputs (information flow)
Sensitivities to and impacts of input changes
Serves as a process knowledge base (for
organizational learning)
Data represented using DSM
Tyson R. Browning — 10
PD Browning 101498 ©1998 Massachusetts Institute of Technology
Sample Result for Single Run of
UCAV Preliminary Design Process
Lean Aerospace
Initiative
20
40
60
80
100
120
132
1
2
3
4
5
6
7
8
9
10
11
12
13
14
l
l
l
l
Visualize critical path
Nominally sequential process
Limited concurrency (dynamic)
Waiting on results when appropriate
Tyson R. Browning — 11
PD Browning 101498 ©1998 Massachusetts Institute of Technology
Results for UCAV Preliminary
Design Process
Lean Aerospace
Initiative
Target
1.0
.8
PC(un.) ≈ 0.24
.6
.4
.2
.0
570
600
630
660
690
720
750
780
810
840
Cost ($k)
Target
1.0
0.8
0.6
0.4
• Distributions
(not point values)
and targets
—> P(unacceptable)
• with Impacts
(e.g., quadratic
loss function)
—> RC & RS
0.2
0.0
120
126
132
138
144
150
156
162
168
174
180
PS(un.) ≈ 0.91
Schedule (days)
Tyson R. Browning — 12
PD Browning 101498 ©1998 Massachusetts Institute of Technology
Joint PDF of Paired
C, S Samples
Relative Probability
Lean Aerospace
Initiative
180
170
850
160
150
140
130
Schedule (days)
Tyson R. Browning — 13
120
550
600
650
700
750
900
800
Cost ($k)
PD Browning 101498 ©1998 Massachusetts Institute of Technology
Contour Plot View of
Joint C, S PDF
Lean Aerospace
Initiative
Budget target
180
Schedule (days)
170
160
150
140
Schedule
target
130
120
Tyson R. Browning — 14
550
600
650
700
750
Cost ($k)
800
850
900
PD Browning 101498 ©1998 Massachusetts Institute of Technology
One Application: Comparing
UCAV Process Configurations
Lean Aerospace
Initiative
.4
.2
.5
.4
.4
.3
.5
.4
.5
.1
.1
.1
.3
.4
.5
.5
.5
.5
.4
.5
Tyson R. Browning — 15
.4
.4
.3
.5
.4
.5
.1
.5
.5
.1
.4
.5
.2
.1
.5
.5
.5
.4
.5
.4
.4
.3
.4
.4
.4
.5
.4
.4
.5
.4
.4
.4
.4
.4
.3
.4
.4
.5
.5
.1
.5
.5
.4
.1
.4
.4
.4
.4
.1
.5
.5
.4
.4
.5
.1
.3
.1
.4
.4
.4
.2
.5
.4
.4
.4
.5
.2
.1
.5
.5
.5
.4
.5
.4
.4
.4
.4
.5
.4
.5
.4
.4
.4
.4
PD Browning 101498 ©1998 Massachusetts Institute of Technology
Lean Aerospace
Initiative
Application: Planning a UCAV
Process Configuration
Linear Scale
Cost Risk
Plot of RC vs. RS for five process
configurations shows trade-off possibilities
Tyson R. Browning — 16
PD Browning 101498 ©1998 Massachusetts Institute of Technology
Another Application: Choosing
Cost and Schedule Targets
Lean Aerospace
Initiative
1.0
600
0.8
480
0.6
360
0.4
240
0.2
120
0.0
120
0
130
140
150
160
Schedule Target (T
Prob.
l
l
l
S
170
180
)
Risk
Different targets => different levels of risk
Choose a target with an acceptable level of risk
Other applications: “what if” analysis for adding activities,
doing iterations, other process modifications…
Tyson R. Browning — 17
PD Browning 101498 ©1998 Massachusetts Institute of Technology
Lean Aerospace
Initiative
Managerial Insights
l
Quantifies relative level of cost and schedule risk in a
process configuration (vs. a target)
l
Compare alternative process configurations and targets for
acceptable risk
l
Visualization of critical path and when activities should wait
on each other
l
Heuristic: minimize long loop iterations in process
l
Surprising result: prescribe preemptive iteration when
appropriate
– High set-up time
– Low sensitivity to input changes
– Significant learning curve effects
Tyson R. Browning — 18
PD Browning 101498 ©1998 Massachusetts Institute of Technology
Lean Aerospace
Initiative
Relationship Between Activities and
Dimensions of Product Performance
Activity 1
Cost, Sched.
Activity 2
Cost, Sched.
Activity 3
Dimension 1
Dimension 2
Cost, Sched.
Dimension 3
Activity n
Dimension m
Cost, Sched.
Activities can affect
each other
Overall Product Performance
Product Development Process
Determines
Performance dimensions
can affect each other
Premise: By decomposing a process with variance into smaller
pieces and understanding the relationships between those pieces,
we can better understand the variance of the overall process.
Tyson R. Browning — 19
PD Browning 101498 ©1998 Massachusetts Institute of Technology
Performance Measure (PM)
Tracking Methodology
Lean Aerospace
Initiative
Effective Mission Range PM Forecasting and Tracking
1050
950
Objective
850
Planned
750
650
550
Required
Warning
Threshold
450
Program Calendar Progress
•
•
•
•
•
Currently used methodology
“Events” have effect on PMs
Events result from activities
Goal of process: to meet requirements and objectives
Also: to reduce uncertainty
Tyson R. Browning — 20
PD Browning 101498 ©1998 Massachusetts Institute of Technology
Lean Aerospace
Initiative
Performance Modeling
Methodology
l
Integrate performance measure (PM) data
l
Capture how work on activities changes PMs
l
l
l
l
Separate effects on most likely value (MLV) and
uncertainty bounds (decisions vs. tests)
Plan project using anticipated effects
Manage/Control project using latest evaluations
and revised estimates
Deterministic and stochastic plans
Tyson R. Browning — 21
PD Browning 101498 ©1998 Massachusetts Institute of Technology
Types of Activity
Effects on PMs
Lean Aerospace
Initiative
Typical Effect on Uncertainty Bounds
>< Reign In
>+<
<><> Uncertain
1
<>+<>
<> Increase
2
<+>
3
t1
t2
t1
4
•
•
t2
t1
<>±<>
5
t2
<±>
6
?
?
•
± Uncertain
•
?
•
•
•
•
?
•
•
?
•
?
?
t1
t2
t1
>–<
7
t2
t1
<>–<>
8
t2
<–>
9
?
•
•
•
•
•
•
?
t1
Tyson R. Browning — 22
•
•
>±<
– Worsen
Also,
Magnitude Effects
Small
Medium
Large
•
•
?
Typical Effect on Most Likely Value
Direction Effects
T = set target
IE = make initial estimate
1-9 = table:
+ Improve
?
t2
t1
t2
t1
t2
PD Browning 101498 ©1998 Massachusetts Institute of Technology
Preliminary UCAV Data:
Effects of Activities on Six PMs
Lean Aerospace
Initiative
Magnitude
Small
Medium
Large
Direction
Performance Measures
T = set target
Example
Set
IE = initial estimate
1-9 = table
Activities
Example
Set
A4
A511
A512
A531
A521
A522
A5341
A532
A533
A5342
A5343
A5344
A54
A6
Prepare UCAV Preliminary DR&O
1
T
T
Create UCAV Preliminary Design Configuration
2
5
5
Prepare & Distribute Surfaced Models & Internal Arrangement
3
Drawings
Perform Aerodynamics Analyses & Evaluation
4
Create Initial Structural Geometry
5
Prepare Structural Geometry & Notes for FEM
6
Develop Structural Design Conditions
7
Perform Weights & Inertias Analyses & Evaluation 8
7
7
Perform S&C Analyses & Evaluation
9
Develop Balanced Freebody Diagrams & External Applied
10 Loads
Establish Internal Load Distributions
11
4
4
Evaluate Structural Strength, Stiffness, & Life
12
4
4
Preliminary Manufacturing Planning & Analyses
13
Prepare UCAV Proposal
14
T
T
T
IE
T
5
T
8
T
8
5
8
8
8
T
IE, 5
T
IE, 5
T
5
8
7
7
T
Mapping the typical, direct effects
(direction and magnitude) of activities on PMs
Tyson R. Browning — 23
PD Browning 101498 ©1998 Massachusetts Institute of Technology
Stochastic Model:
Sample Result
Lean Aerospace
Initiative
1000
MLV
E[V]
800
600
400
200
0
0
20
40
60
80
100
120
140
160
Project Time (t, days)
l
l
l
l
Simulated PM tracking chart for UCAV payload
Changes in MLV and uncertainty caused by activities in process
Goals of process are improved MLV and reduced uncertainty
Current limitation: no feedback from current performance level
Tyson R. Browning — 24
PD Browning 101498 ©1998 Massachusetts Institute of Technology
Preliminary, Example Result
(Single Run)
Lean Aerospace
Initiative
(Revise) Configuration
Good Struct. Str. & Life
Good Internal Loads
RP
Good Struct. Str. & Life
Structural Geometry
Affects Reliability
160
140
Fourth
Weights
Analysis
120
100
80
60
40
20
0
Second
Weights
Analysis
Mfg. Planning
• Compare
processes
• Add
activities
• Repeat
activities
• Repeat
processes
Project Time (t, days)
Prepare UCAV Preliminary DR&O
Create UCAV Preliminary Design Configuration
Prepare & Distribute Surfaced Models & Internal Arrangement Drawings
Perform Aerodynamics Analyses & Evaluation
Create Initial Structural Geometry
Prepare Structural Geometry & Notes for FEM
Develop Structural Design Conditions
Perform Weights & Inertias Analyses & Evaluation
Perform S&C Analyses & Evaluation
Develop Balanced Freebody Diagrams & External Applied Loads
Establish Internal Load Distributions
Evaluate Structural Strength, Stiffness, & Life
Preliminary Manufacturing Planning & Analyses
Prepare UCAV Proposal
Tyson R. Browning — 25
PD Browning 101498 ©1998 Massachusetts Institute of Technology
Lean Aerospace
Initiative
l
Options and Applications
Options:
– Repeat (iterate) all or portion of process
– Add new activities to affect PMs
l
Applications:
– Exploring CS&P implications of options
– Exploring PM contributions to RP
– Exploring activity contributions to RP
– Ensuring existence of risk reducing (value adding) chain
of activities
– Comparing processes
l
Result:
– Maintained balance and congruence of product and
process to understood customer desires
Tyson R. Browning — 26
PD Browning 101498 ©1998 Massachusetts Institute of Technology
Lean Aerospace
Initiative
l
l
l
l
l
Managerial Insights
Process cost and schedule affect product
performance
Compare processes to see which configurations
maximize overall value to customer (best balance
of C,S, and technical P)
Explore effect of management decisions on RP
Check integrity of value stream (i.e., all necessary,
value-adding activities)
And more…
Tyson R. Browning — 27
PD Browning 101498 ©1998 Massachusetts Institute of Technology