Bayesian evaluation and
selection strategies in portfolio
decision analysis
E. Vilkkumaa, J. Liesiö, A. Salo
INFORMS Annual meeting, Phoenix, Oct 14th-17th 2012
The document can be stored and made available to the public on the open internet pages of Aalto University. All other rights are reserved.
Post-decision disappointment in
portfolio selection
*
• Project portfolio selection is
important
• Decisions are typically based
on uncertain value estimates vE
about true value v
• If the value of a project is
overestimated, this project is
more likely to be selected
• Disappointments are therefore
likely
= Optimal project based on vE
= Optimal project based on v
Size is proportional to cost
18
H
E
Estimate v ($M)
16
14
C
*
12
J
*
*
G
D
*
10
E
8
B
A
6
I
*
4
F
2
0
0
2
4
6
8
10
12
14
16
18
True value v ($M)
Brown (1974, Journal of Finance), Harrison and March (1986, Administrative Science
Quarterly), Smith and Winkler (2006, Management Science)
Frequency (%)
Underestimation of costs in public work
projects (1/2)
• Flyvbjerg et al. 2002
found statistically
significant escalation
(p<0.001) of costs in
public infrastructure
Average escalation
projects
μ=27,6%
• This escalation was
attributed to strategic
misrepresentation by
project promoters
Cost escalation (%)
Source: Flyvbjerg et al. (2002), Underestimating Costs in Public Work Projects – Error
or Lie? Journal of the American Planning Association, Vol. 68, pp. 279-295.
Underestimation of costs in public work
projects (2/2)
0.035
0.03
Distribution of
maximal
escalation among 6
projects, =25%
Distribution of
maximal
escalation among 3
projects, =17%
0.025
0.02
0.015
0.01
Distribution of
escalation for
each
project, =0%,
=20%
• Cost escalation could
therefore be attributed to
ʻpost-decision
disappointment’ as well
0.005
0
-100
-80
-60
-40
• If projects with the lowest
cost estimates are
selected, the realized
costs tend to be higher
even if cost estimates are
unbiased a priori
-20
0
20
40
60
Cost escalation (%)
80
100
Bayesian revision of value estimates (1/2)
• Assume that the prior f(v) and the likelihood f(vE|v) are

E
known such that E[Vi | V  v ]   v iEf (v iE | v ) dv iE  v i

• By Bayes’ rule we have f(v|vE)  f(v)·f(vE|v)
• Use the Bayes estimates viB for selection

v  E[Vi | V  v ]   v i f (v i | v E ) dv i
B
i
E
E

• If Vi~N(μi,σi2), ViE=vi+εi, εi~N(0,τi2) , then Vi|viE~N(viB,ρi2),
where
2
2
4
σ
τ
σ
v iB  2 i 2 v iE  2 i 2 μi ,
ρi2  2 i 2 .
σi  τi
σi  τi
σi  τi
Bayesian revision of value estimates (2/2)
• Portfolio selection based on the revised estimates viB
– Eliminates post-decision disappointment
– Maximizes the expected portfolio value given the estimates viE
• Using f(v|vE), we show how to:
1. Determine the expected value of acquiring additional estimates viE
2. Determine the probability that project i belongs to the truly optimal
portfolio (= portfolio that would be selected if the true values v were
known)
Example (1/2)
•
•
•
•
10 projects (A,...,J) with costs from $1M to $12M
Budget $25M
Projects’ true values Vi ~ N(10,32)
A,...,D conventional projects
– Estimation error εi ~ N(0,12)
– Two interdependent projects: B can be selected only if A is
selected
• E,...,J novel, radical projects
– These are more difficult to estimate: εi ~ N(0, 2.82)
Example (2/2)
= Optimal project based on vE / vB
= Optimal project based on v
Size proportional to cost
*
18
18
H
14
Bayes estimate v ($M)
E
C
*
12
J
Prior mean
10
*
*
G
D
*
B
Estimate v ($M)
16
E
8
B
A
6
I
*
4
F
2
0
0
2
4
6
8
10
12
14
True value v ($M)
True value = 52$M
Estimated value = 62$M
16
18
16
H
14
12
J
Prior mean
10
*
B
I*
1
6
*
E
A
8
*
G
*
C
D
F
4
2
0
0
2
4
6
8
10
12
14
16
True value v ($M)
True value = 55$M
Estimated value = 58$M
*
18
Value of additional information (1/2)
EVI for a single project evaluation
= Optimal project based on
current information
• Knowing f(v|vE), we can
determine
– The expected value (EVI) of
additional value estimates VE
prior to acquiring vE
– The probability that project i
belongs to the truly optimal
portfolio
0.3
0.25
F
E
H
0.2
B
I
D
0.15
A
0.1
0.05
G
C
0
J
-0.05
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Probability that the project belongs to the
truly optimal portfolio
The probability that the
project belongs to the
truly optimal portfolio is
here close to 0 or 1
Value of additional information (2/2)
• Select 20
k=100out of 100 projects
k=30
76
Best 30
• Re-evaluation
strategies
78
k=100
k=30
Best 30
Random 30
96
Share of correct choices (%)
Portfolio value (% of the optimum)
97
1.
2.72
3.
70
4.
95
94
93
92
91
1
2
3
Number of evaluation rounds
4
74
Random 30
All 100 projects
30 projects with the highest EVI
ʻShort list’ approach (Best 30)
30 randomly selected projects
• Due to evaluation costs,
66
strategy
2 is likely to outperform
strategy
1
64
68
1
2
3
Number of evaluation rounds
4
Conclusions
• Uncertainties in cost and value estimates should be
explicitly accounted for
• Bayesian revision of the uncertain estimates helps
– Increase the expected value of the selected portfolio
– Alleviate post-decision disappointment
• Bayesian modeling of uncertainties guides the costefficient acquisition of additional estimates as well