Integrating Real Options into
Manufacturing Investment Decisions:
Multi-stage Project Governance using
Business Engineering
Scott Mathews, Associate Technical Fellow
Computational Finance and Stochastic Modeling
206-655-1366
Copyright by The Boeing Company 2006
Topics for Integrating Real Options
Boeing Technology | Phantom Works
Strategic Decision and Uncertainty Modeling
• Strike-cost distribution
• Multi-stage projects
• Simulation to derive strike-cost
• Bayesian Real Options concepts
• Project governance
Copyright by The Boeing Company 2006
The Value of Project Management Flexibility
Concept of Real Options and Learning
Boeing Technology | Phantom Works
Strategic Decision and Uncertainty Modeling
Profit/Loss Profile
Lattice of iterative
program decisions
Greater
Profits


Gradual Reduction
of Uncertainty


Worse Incremental
Losses phased stream of
investments sized to
probability of success
Time or Gates
Copyright by The Boeing Company 2006
Most likely
Value (NPV Value)
Simple View of Real Options
Boeing Technology | Phantom Works
Strategic Decision and Uncertainty Modeling
Value
• Overly simplistic view of
real options – borrowed
from the capital markets
• The strike price, or
Product
Nonrecurring
Cost
product nonrecurring
cost, is a single point
Distribution of
Future Market Value
Growth of Uncertainty
t
Copyright by The Boeing Company 2006
Product Strike Cost is Also Uncertain
Boeing Technology | Phantom Works
Strategic Decision and Uncertainty Modeling
Value
• Product nonrecurring cost
is also a distribution
•
variable strike price
(product strike cost)
• If no product research is
Profits
done, the cost uncertainty
increases over time,
unless…
• Successful managers
reshape the product cost
distribution during
product development
0
Losses
Product Development Period T
Copyright by The Boeing Company 2006
Product
Cost
Distribution
Distribution of
Future Market Value
Product Development Management is Iterative
Boeing Technology | Phantom Works
Strategic Decision and Uncertainty Modeling
• Cost modeling is not a
•
random walk, but iterated
stages during product
development
Each iteration is a gate:
a discrete investment
decision to proceed or
cancel – a multistage
option
Product
Cost
Distribution
• Benefits (operating profit)
•
are at market risk –
difficult to control directly
Manager focus is on what
can be controlled –
R&D
product costs
Project
Production
Copyright by The Boeing Company 2006
Manager Controls Costs in each Iteration
Boeing Technology | Phantom Works
Strategic Decision and Uncertainty Modeling
• At each iteration manager
Product Cost
Distribution
Detail
attempts to shift and lower
expected costs
•
•
•
•
reduce cost risk
increase opportunity
decrease overall cost
minimize uncertainty
• The effect is to reshape
the cost distribution – the
essence of real options in
manufacturing is learning
about project costs
• This change in expected
Project Stage
project costs is not free
Copyright by The Boeing Company 2006
Investments Change Strike Cost Outcome
Boeing Technology | Phantom Works
Strategic Decision and Uncertainty Modeling
• Shifting cost path requires
investments across multiple
project factors
$$ Investment
• Project managers demand pilot
investments result in high
project payoffs
•
•
Some investments reduce
uncertainty while increasing project
value
Some investments result in
potentially lower cost but increase
risk
• How to decide which iterative
Project Stage
investments to make?
Copyright by The Boeing Company 2006
Increasing Project Value with 2-Stage Option
Boeing Technology | Phantom Works
• A conservative
investment reduces
strike cost uncertainty by
maturing existing
technology
• A more aggressive
•
investment decreases
strike cost by introducing
new technology
Note that new technology
falls outside strike
dispersion of old
technology
Strategic Decision and Uncertainty Modeling
2nd Stage
$$ Investment
1st Stage
Option
$ investment
2nd Stage
$$$ Investment
1st Stage
Option
$ investment
outside
Copyright by The Boeing Company 2006
More Certain
Strike
Cost
Lower
Strike
Cost
Boeing has a 2-Stage Option
Boeing Technology | Phantom Works
Strategic Decision and Uncertainty Modeling
• First stage option:
• Build a factory using existing process technology
– Advantages: mature processes reduce manufacturing risk
– Disadvantages: older technology may not be cost competitive
• Second stage option:
• Build a factory using new process technology
– Advantages:
newer technology decreases cost, more competitive
– Reduce both recurring (operating profit) and non-recurring (strike) costs
– Disadvantages:
immature processes increase manufacturing risk
– Regret: May fail to create better machine tools
• Must invest heavily in order to create new technology
• Need business engineering tools to value staged
investments with increase in option value
Copyright by The Boeing Company 2006
Modeling the Distribution Change in Strike Cost
Boeing Technology | Phantom Works
Strategic Decision and Uncertainty Modeling
• Process simulation model
•
Models process flow
including uncertainties (risk)
–
•
–
–
–
–
–
–
–
Quantity
Material costs
Machine tools
Process time
Labor resources
Quality (Yield)
Scrap
Highly flexible to consider
differing fabrication &
assembly trades
Embedded business case
–
–
–
Units
Sold
Cumulative Costs
Changing market demand
Discounted cashflows
Learning curve effects
• Able to simulate different
factory layout and tooling that
impact strike cost
-
$0B
Copyright by The Boeing Company 2006
How the Process Model Works
Boeing Technology | Phantom Works
Strategic Decision and Uncertainty Modeling
• Process steps simulated for ten year production cycle
• Analysis consists of 100’s of complete production cycles
• Each cycle is a plausible operation selection based on
uncertainty estimate of technology maturity
• Each unit produced adds to process recurring costs
• Non-recurring (strike) costs based on quantity of
machine tools required to maintain rate (balanced line)
Copyright by The Boeing Company 2006
Strike Cost used to Purchase Machine Tools
Boeing Technology | Phantom Works
Strategic Decision and Uncertainty Modeling
Typical cost is $5M - $15M.
Need 30 ~ 50 Machine Tools
Copyright by The Boeing Company 2006
Recurring Cost is Materials and Labor
One Piece Airplane Fuselage
Boeing Technology | Phantom Works
Strategic Decision and Uncertainty Modeling
Copyright by The Boeing Company 2006
First Stage Option
Boeing Technology | Phantom Works
Strategic Decision and Uncertainty Modeling
• First stage option:
• Build a factory using existing,
Strike $:
Probability
•
technology machine tools
Process simulation can derive
strike price distribution
$0B
• Applying Datar-Mathews
Method for Real Options, the
value of the first options is
calculated
Copyright by The Boeing Company 2006
Old Technology
Datar-Mathews Algorithm for Real Options
Boeing Technology | Phantom Works
Strategic Decision and Uncertainty Modeling
PV0 of cash flow @WACC
PVT of cash flow @WACC
PV0 (X) @rf
0pilot R&D
commercial
cash flow
T
time
investment
D-M Method* inputs arise
naturally in project analysis
X
• Intuitive and transparent – extension of NPV :
•
•
•
Option = PVbenefits – PVcosts
Easily integrated into spreadsheets with simulation
Extensible to many option types including contracts
Option DM method = Option BS method
*Datar-Mathews Method for Quantitative Real Option Valuation, Patent #6862579.
Copyright by The Boeing Company 2006
Second Stage Option
Boeing Technology | Phantom Works
Strategic Decision and Uncertainty Modeling
• Second stage option:
• Build a factory using new
•
Probability
•
technology machine tools
Again, process simulation can
derive strike price distribution
New strike price is substantially
less than old strike price
Strike $: New vs
$0B
• Requires investment to
development new technology
• Need new method to determine
(‘prediction classifier’ )
2nd Stage
Investment
whether second stage
investment will be a good or
bad decision, a type of project
gating with a target milestone
Copyright by The Boeing Company 2006
Old
Bayesian Option Formulation
Boeing Technology | Phantom Works
Strategic Decision and Uncertainty Modeling
• Bayes use subjective probabilities to model expectations
• Ex. simulation variables and milestones
• Bayesian methods facilitate modeling of
• contingent investment decisions (project gating)
• relationship between staged project investments and prediction
classifiers 
• Prediction classifiers  are quantified project milestones
• Bayesian methods calculate optimum staged project
investments that maximize multi-stage option value
Copyright by The Boeing Company 2006
Bayesian Decision Classification Concepts
Boeing Technology | Phantom Works
Prediction Classification
• Milestone is a threshold
• Identify well- and poor-
Strategic Decision and Uncertainty Modeling
Quantified
Milestone
performing projects
P+
• Milestone  is an
optimum Bayesian
option calculation:
• Investment $
• Strike Cost
• Option outcome (P+, P-)
P-
Copyright by The Boeing Company 2006
2nd Stage Investment Contingency Matrix
Boeing Technology | Phantom Works
Strategic Decision and Uncertainty Modeling
• Objective: determine
milestone classifier with
optimum risk-adjusted
Prevalue performance
dicted
• Not simply probability
Actual
p+
$TP
$FN
p+
p-
p$FP
$TN
Investment
$$$
• Milestone 2  is
1
Bayesian optimum
classifier when
• Maximizes benefits ($TP)
• Minimizes regrets ($FP)
2
$$
3
$
and omissions ($FN)
1
Copyright by The Boeing Company 2006
2
T
Better Project Gating Methods
Boeing Technology | Phantom Works
Strategic Decision and Uncertainty Modeling
• Quantified milestones are superior predictors of project
outcome resulting in better gating criteria
• Poor gating methods contribute to a high ‘go-ahead’ rate
of ultimately ‘challenged’ projects
• challenged projects: completed but over-budget, over schedule,
and offering fewer features and functions than originally
specified
• ‘Challenged’ projects are a type of FP phenomenon –
gated through but ultimately unsuccessful
Copyright by The Boeing Company 2006
Multi-Stage Project Options
Boeing Technology | Phantom Works
Strategic Decision and Uncertainty Modeling
• Strike-cost distribution can be simulated and managed
• Options can determine optimum multi-stage investments
to govern project
• Each stage sets project milestone expectations
• Future expectations are the project milestones
contingent on investments
• Bayesian methods and DM Real Options provide better
project governance by establishing gates balanced
between milestones and investments
Copyright by The Boeing Company 2006
Appendix
Boeing Technology | Phantom Works
Strategic Decision and Uncertainty Modeling
• Bayesian Contingency Matrix
• Better Prediction Improves Project Governance
• Illustration of ROC Graphs
• ROC Analysis of Classifiers
• Background on ROC
• Considering the Second Stage Option
• Concept of Multi-stage Options with Project Investment
•
•
Gates
DM Real Option Method
Controlled Gating Metrics for Project Governance
Copyright by The Boeing Company 2006
Bayesian Contingency Matrix
Boeing Technology | Phantom Works
Strategic Decision and Uncertainty Modeling
• Contingency Matrix
• Evaluates goodness of classifier
• Highlights misclassification
•
(errors) probabilities
Sometimes called ‘Complexity Predicted
Matrix’
Actual
P+
P-
• Issues:
•
PFP
TN
Actual
• Bayesian method relies
•
P+
TP
FN
only on probabilities (not value)
PreClassifier with fewest errors
dicted
is not always optimum
Not all errors have same
value consequences
P+
P-
Copyright by The Boeing Company 2006
P+
16%
4%
Sample Data
P15%
65%
Better Prediction Improves Project Governance
Boeing Technology | Phantom Works
Strategic Decision and Uncertainty Modeling
• Objective: establish project
outcome predictive capability
by choosing optimum milestone:
• Good classifier: TP↑ FP↓
• TP↑: surpassing milestone
Higher
Predictive Capability
100%
Milestone

P(TP)
establishes higher level of
confidence – gate go-ahead
• FP↓: failing milestone requires
steps to halt (TN) or alter to fit
challenges – gate no go
0%
P(FP)
100%
Illustration of ROC Graphs
Better Gate Classifier
ROC Graph Analysis
Copyright by The Boeing Company 2006
Minority of Projects are Managed Successfully
Boeing Technology | Phantom Works
•
Strategic Decision and Uncertainty Modeling
Standish Group CHAOS research - widely quoted
statistics in the IT industry
• Periodic MIS survey to determine:
– The scope of software project failures
– The major factors that cause software projects to fail
– The key ingredients that can reduce project failures
•
Results of 9300 IT projects in 2004:
– 29% ‘succeeded’:
– on-time and on-budget, all features and
functions as initially specified.
– 53% ‘challenged’:
Failed
1700
– over-budget, over the time estimate, and offers
fewer features and functions than originally
specified
– 18% ‘failed’:
– canceled at some point during the
development cycle.
Success
2700
Challenged
4900
http://www.standishgroup.com/sample_research/PDFpages/extreme_chaos.pdf
Copyright by The Boeing Company 2006
Analysis of Standish Data by ROC
Boeing Technology | Phantom Works
Strategic Decision and Uncertainty Modeling
• Categorizing Standish
research data and
applying ROC
analysis*, we can see
•
•
Actual
Predicted
Current poor gating
methods are little
better than Line of
Indifference
P+
29%
0%
P+
PP(TP) ≈ 100%
P53%
18%
P(FP) ≈ 75%
Standish
data
100%
Project managers
are unable to predict
good projects from
the bad projects
(random predictor)
Line of
Indifference
P(TP)
0%
P(FP)
100%
Illustration of ROC Graphs
* Some simplifying assumptions made for purposes of
categorization owing to incompleteness of Standish data.
Standish Data by ROC
Copyright by The Boeing Company 2006
High Prediction Gating Methods
Boeing Technology | Phantom Works
Strategic Decision and Uncertainty Modeling
• Objective: establish better
gates with higher predictive
capability
• Good classifier: TP↑ FP↓
• Gates are go/no-go decision
Higher
Predictive Capability
100%
P*

P(TP)
checkpoint for projects.
– TP↑: surpassing gate establishes
higher level of confidence
– FP↓: failing gate requires steps to
halt (TN) or alter to fit challenges
0%
P(FP)
100%
Illustration of ROC Graphs
Better Gate Classifier
• Standish research shows
formal governance could
increase project success rate
by 16 percentage points.
Copyright by The Boeing Company 2006
Illustration of ROC Graphs
Boeing Technology | Phantom Works
•
Starting from two distributions of
positives (red) and negatives (blue) one
can apply a threshold criterion (vertical
line) to arbitrarily separate the two.
Strategic Decision and Uncertainty Modeling
P* Threshold
•
For overlapping distributions, there is
always a tradeoff between sensitivity
(TP) and specificity (FP), since TP+FN
and TN+FP both have to add up to 1.
•
Sliding the threshold line towards the
distribution of positives will result in a
decreased probability for true positive
detection P(TP) and FPs, which is
equivalent to moving the ROC curve
(dashed) downwards.
•
If the two distributions overlap
completely, the ROC curve will be the
diagonal shown as the dot-dashed
curve. Thresholds on the diagonal
cannot distinguish outcomes (random
Copyright by The Boeing Company 2006
predictor).
ROC Graph
Excerpt from Wikipedia
ROC Analysis of Classifiers
Boeing Technology | Phantom Works
Strategic Decision and Uncertainty Modeling
• Good and bad classifiers.
100%
100%
100%
P(TP)
P(TP)
P(TP)
0%
P(FP)
100%
Illustration of ROC Graphs
Good Classifier
•High True Positive Rate
•Low False Positive Rate
0%
P(FP)
100%
0%
Illustration of ROC Graphs
Bad Classifier
•Low True Positive Rate
•High False Positive Rate
P(FP)
100%
Illustration of ROC Graphs
Bad Classifier
•High True Positive Rate
•High False Positive Rate
• An ’Indifferent’ Classifier
Copyright by The Boeing Company 2006
Background on ROC Curves
Boeing Technology | Phantom Works
Strategic Decision and Uncertainty Modeling
• ROC curves are used to evaluate the results of a prediction and were first
employed in the study of discriminator systems for the detection of radar radio
signals in the presence of noise in the 1940s, and later to study the response of
transistors. In the 1960s they began to be used in psychophysics, to assess
human (and occasionally animal) detection of weak signals. They also proved to
be useful in data mining and for the evaluation of machine learning results, such
as the evaluation of Internet search engines. They are also used extensively in
epidemiology and medical research and are frequently mentioned in conjunction
with evidence-based medicine
• In signal detection theory, a receiver operating characteristic (ROC), also receiver
operating curve, is a graphical plot of the sensitivity vs. 1-specificity for a binary
classifier system as its discrimination threshold is varied. The ROC can also be
represented equivalently by plotting the fraction of true positives (TP) vs. the
fraction of false positives (FP).
Adopted from http://en.wikipedia.org/wiki/Receiver-Operator_Characteristic
Copyright by The Boeing Company 2006
Considering the Second Stage Option
Boeing Technology | Phantom Works
Strategic Decision and Uncertainty Modeling
• Additional benefits of
second stage option
• New technology reduces unit
•
recurring cost distribution
Result is improved operating
profits
New vs Old
Probability
Rec Cost $/Unit:
$0M
• The value of the second
•
option is calculated applying a
compound Datar-Mathews
Method for Real Options
Use Bayesian approach with
DM Method to create a 2-stage
Bayesian DM Option
Copyright by The Boeing Company 2006
Reduced
unit costs
Straightforward Model Structure
for DM Method Option
Boeing Technology | Phantom Works
Strategic Decision and Uncertainty Modeling
• Transparent and intuitive
•
an extension of NPV-type models
• Either risk-neutral or risk-adverse approach
• Payoff distributions can be of any type
•
not limited to lognormal
• A variety of uncertainty evolution structures
•
•
for incomplete markets, correlated discreet time distributions
or standard Brownian motion with Weiner process
• The models typically incorporate
•
•
•
•
shifting demand curves
variable costs
competing technologies
antagonists (agents)
Copyright by The Boeing Company 2006
Relaxed Constraints for
DM Method Option Inputs
Boeing Technology | Phantom Works
Strategic Decision and Uncertainty Modeling
• The value today (S0) is not a required specific input
• The value of sigma  is not a required specific input
• The strike price can be expressed as a distribution
•
variable strike -- or 'cost-risk options‘
• Stochastic volatility is easily accommodated
•
as well as corresponding time-varying discount rates
• Extensible to compound and American option types
• Extensible to Bayesian approaches of option valuation
Copyright by The Boeing Company 2006
Controlled Gating Metrics for Project Governance
Boeing Technology | Phantom Works
Strategic Decision and Uncertainty Modeling
• If a development program adopts an iterative lifecycle approach,
then the Project Value (PV) is recalculated at each investment gate.
• Management Leverage (ML):
•
•
ML = PV/ Investment
Management Leverage at each investment gate i, taking into account
the reduced risk and lessening of uncertainty:
MLi = PVi / Investmenti
Then Real Earned Value (REVi) at each investment gate is:
REVi = PVi – Investmenti
Note how this differs from the definition of EV (Earned Value) which is
a static view of project management.
Copyright by The Boeing Company 2006