Value of information
Marko Tainio
Decision analysis and Risk Management
course in Kuopio
21.3.2011
Outline of lecture
• Aim
– To give an overview on what is VOI and in what
situation it is useful.
• Content
– What is Value of Information (VOI) analysis?
– How to calculate VOI?
– How and in what situation you can use VOI?
Different definitions
• Value of information (VOI) is the amount a decision maker would be
willing to pay for information prior to making a decision.
– Definition in Wikipedia
(http://en.wikipedia.org/wiki/Value_of_information)
• Expected value of perfect information (EVPI) is the price that one
would be willing to pay in order to gain access to perfect
information
– 2nd definition in Wikipedia
http://en.wikipedia.org/wiki/Expected_value_of_perfect_information
• VOI is„a decision analytic technique that explicitly evaluates the
benefits of collecting additional information to reduce or eliminate
uncertainty”
– Yokota and Thompson, 2004
Key elements of VOI
• Decision maker
– VOI analysis is a decision analysis tool aimed to help in
decision making
• Information that contains uncertainty
– Originally VOI analysis is used in situation where there
is 2 or more available decision options and their
outcomes are uncertain
• For example: Should we vaccinate population or not
• Price for the information
– VOI also assumes that gathering of more information
is possible and that this information reduces or
eliminates uncertainty!
Calculation of VOI
(expected value of perfect information, EVPI)
Equation to calculate EVPI
EVPI = EV|PI - EMV


EVPI   max u(a, s) f ( s)ds
a A
sS 

Expected
monetary
value (EMV)


 max  u (a, s) f ( s)ds
 sS

a A
In the equation, s is the uncertain input, and
f(s) represents the probability distribution
representing prior belief about the likelihood
of s.
Yokota and Thompson (2004)
Expected value
given perfect
information
(EV|PI)
Example 1
Example with point values
• Example from wiki: http://en.wikipedia.org/wiki/Expected_value_of_perfect_information
• Suppose you were going to make an investment into only one of
three investment vehicles: stock, mutual fund, or certificate of
deposit (CD).
• Further suppose, that the market has a 50% chance of increasing, a
30% chance of staying even, and a 20% chance of decreasing.
– If the market increases the stock investment will earn $1500 and the
mutual fund will earn $900.
– If the market stays even the stock investment will earn $300 and the
mutual fund will earn $600.
– If the market decreases the stock investment will lose $800 and the
mutual fund will lose $200.
– The certificate of deposit will earn $500 independent of the market's
fluctuation.
Decision tree
$1500
(+) 50%
(+/-) 30%
Stock
Mutual fund
$300
(-) 20%
-$800
$900
(+) 50%
$600
(+/-) 30%
(-) 20%
Certificate
-$200
$500
What are the
expectations for
each vehicle?
Continuation of example
• Expectation for each vehicle:
– Expstock = 0.5 * 1500 + 0.3 * 300 + 0.2 * ( − 800) =
680
– Expmutualfund = 0.5 * 900 + 0.3 * 600 + 0.2 * ( − 200) =
590
– Expcertificateofdeposit = 500
• The maximum of these expectations is the stock
vehicle. Not knowing which direction the market will
go (only knowing the probability of the directions), we
expect to make the most money with the stock vehicle.
• Expected monetary value (EMV) = 680
Continuation of example (3/3)
• On the other hand, consider if we did know
ahead of time which way the market would turn.
Given the knowledge of the direction of the
market we would (potentially) make a different
investment vehicle decision.
• Expectation for maximizing profit given the state
of the market:
– EV | PI = 0.5 * 1500 + 0.3 * 600 + 0.2 * (500) = 1030
• That is, given each market direction, we choose
the investment vehicle that maximizes the profit.
• Hence = EVPI = EV|PI –EMV = 1030 – 680 = $350
Example 2
Decision situation
• Lets assume that you can spend time either in
Helsinki or in Kuopio;
• Only thing you care is temperature (you want to
be in warmer place all the time);
• Lets also assume that every morning you could
decide in which city you are (you have magic);
• What is the value of information for you to know
the exact temperature of every day vs. knowing
average temperature?
Decision before new information
• At the moment you know that average
temperatures for these cities are:
– Helsinki: -15 Celsius (range -10.7 to -19.3)
– Kuopio: -16 Celsius (range -14.1 to -19.9)
• Which city you would choose if you care only
from temperature and you have this data?
• In previous equation, this is EMV (Expected
monetary value)
• In next phase we add more data for the decision
situation!
Temperature data for January
Helsinki
-10,7
-14,0
-14,5
-14,3
-16,6
-11,7
-15,2
-14,7
-13,4
-17,4
-15,3
-16,2
-16,0
-17,1
-14,8
-13,8
-15,0
-15,7
-15,5
-13,1
-12,9
-12,2
-19,3
-15,8
-13,6
-16,9
-17,8
-18,3
-16,4
-12,6
-14,2
Average
-15,0
Kuopio
-15,0
-16,3
-17,2
-16,1
-16,8
-16,0
-15,6
-15,4
-16,5
-15,7
-17,3
-16,4
-17,7
-17,9
-14,8
-16,9
-14,1
-15,5
-14,2
-14,5
-15,2
-14,6
-17,5
-14,7
-14,3
-15,9
-16,6
-15,1
-17,8
-17,4
-17,0
-16,0
Difference in average temperature is 1
Celsius degree so based on that
knowledge you would prefer Helsinki
However, Helsinki is not
warmer in every day! And since
you have magic, you can choose
every morning in which city you
are!
Therefore you might want to calculate
in which days Helsinki is warmer and in
which days Kuopio is warmer to
optimise your decision every morning.
Temperature data for January
Helsinki
-10,7
-14,0
-14,5
-14,3
-16,6
-11,7
-15,2
-14,7
-13,4
-17,4
-15,3
-16,2
-16,0
-17,1
-14,8
-13,8
-15,0
-15,7
-15,5
-13,1
-12,9
-12,2
-19,3
-15,8
-13,6
-16,9
-17,8
-18,3
-16,4
-12,6
-14,2
Average
-15,0
Kuopio
-15,0
-16,3
-17,2
-16,1
-16,8
-16,0
-15,6
-15,4
-16,5
-15,7
-17,3
-16,4
-17,7
-17,9
-14,8
-16,9
-14,1
-15,5
-14,2
-14,5
-15,2
-14,6
-17,5
-14,7
-14,3
-15,9
-16,6
-15,1
-17,8
-17,4
-17,0
-16,0
Higher
-10,7
-14,0
-14,5
-14,3
-16,6
-11,7
-15,2
-14,7
-13,4
-15,7
-15,3
-16,2
-16,0
-17,1
-14,8
-13,8
-14,1
-15,5
-14,2
-13,1
-12,9
-12,2
-17,5
-14,7
-13,6
-15,9
-16,6
-15,1
-16,4
-12,6
-14,2
-14,6
9 out of 31 days Kuopio was warmer than
Helsinki.
If you take the higher temperature for
each day and calculate the average
temperature, you find out that average
temperature was -14.6 Celsius degree
Now, what is the value of information?
Before exact data you would have stayed all
the time in Helsinki and enjoyed average
temperature of -15.
With the exact data you can change the city
each day to warmer one and experience
average temperature of -14.6.
Therefore you gained 0.6 Celsius degree!
Thus, the value of information is: 0.6!
How to calculate VOI in Monte Carlo?
Iteration Decision 1 Decision 2 Max
1
1014
928
1014
2
1049
893
1049
3
835
817
835
4
812
898
898
5
1073
823
1073
6
1191
931
1191
7
1249
850
1249
8
960
966
966
9
1227
958
1227
10
1191
896
1191
11
1063
932
1063
12
881
922
922
13
1014
884
1014
14
804
926
926
15
839
844
844
16
1106
948
1106
17
945
889
945
18
840
923
923
19
1056
843
1056
20
954
968
968
21
1001
799
1001
22
908
874
908
23
1142
882
1142
24
888
966
966
25
1109
902
1109
…
…
…
…
1000
886
992
992
Mean
1000
900
1025
Calculation of EVPI in Monte Carlo model is done
similarly as in previous example with following
modifications:
- We assume that each iteration is an individual
data point;
- We consider each iteration as a separate decision
and then we maximize the benefit similarly as in
temperature example;
- The calculation is simple and can be done e.g.
with Excel (when the result sample is known).
Different variations of VOI
Expected value of perfect information
• Expected value of perfect information
assumes that uncertainty is reduced to zero
• Two analyses:
– The expected value of perfect information (EVPI)
– Expected value of perfect X information (EVPXI)
• EVPI consider whole model while EVPXI
calculates VOI for individual parameter
– E.g. the EVPXI can be calculated separately for
dose-response function, exposure estimates etc.
Probability density
Example 1: Which decision option is
better, A or B
A
B
Costs
How certain you are that A is better than B?
Probability density
Example 1: Which decision option is
better, A or B
A
B
Costs
How certain you are that A is better than B?
Imperfect information
• Expected value of imperfect information
(EVSI)
• Expected value of imperfect X information
(EVPXI)
– Imperfect information assumes that uncertainty
can be reduced but not eliminated (example 1 vs.
Example 2)
• Imperfect information is more realistic
assumption than perfect information!
Probability density
Example 2: Which decision option is
better, A or B
A
B
Costs
How certain you are that A is better than B?
Probability density
Example 2: Which decision option is
better, A or B
A
B
Costs
How certain you are that A is better than B?
How to use VOI in risk
assessment & management?
Requirements for VOI analysis
• To be able to perform a VOI analysis a
modeller needs information on the
– (i) available decision options;
– (ii) the consequences of each options;
– (iii) uncertainties and reliability of the data.
• In addition to these, both gains and losses of
the actions must be qualified with common
metrics (monetary or non-monetary).
(i) Decision options
• Decision options depend on the purpose of the
assessment
• In environmental health field decision options
could be e.g.:
– Choice between different decision options to reduce
emissions of pollutants
• Who defines the decision options depends on the
case
– Authorities that ordered assessment, modeller
himself, stakeholders etc.
(ii) Consequences of each options
• This one is the assessment model that you
have defined during the assessment
• We will not go to detailes in this lecture
(iii) uncertainties and reliability of the
data
• In VOI analysis we reduce uncertainty so the
assessment should have uncertain parameters
• Identifying and assessing uncertainties have
been considered in other lectures.
Two possible ways of using VOI in
assessment 1/2
• Guide the information gathering and model building
– The decisions can be made based on available information
or wait and collect more information
– VOI analysis can be used to inform decision maker on the
possible benefits of collecting additional information
– However, in the field of environmental health and risk
assessment, situations, where decision maker is known,
the decision maker has possibility to allocate more funding
for additional research, and more data can be collected,
are rare and this kind of exploitation of VOI analysis is
more an exception than rule
Two possible ways of using VOI in
assessment 2/2
• Guide the process of model building
– the decision maker is the modeller him/herself or the
research team who makes the decisions of the
modelling work
– Thus, VOI analysis can be used like sensitivity analysis
– The decisions that can be addressed are e.g.
• (i) should the model define uncertainties
• (ii) what are the key input parameters or assumptions in the
model
• (iii) which parts of the model should be specified more
detailed
Further reading
• Morgan M.G. and Henrion M. (1998). Uncertainty: A
guide to dealing with uncertainty in quantitative risk
and policy analyses. Cambridge University Press. 332
pp.
• Yokota F. and Thompson K.M. (2004). Value of
information literature analysis: A review of applications
in health risk management. Medical Decision Making,
24 (3), pp. 287-298.
• Yokota F. and Thompson K.M. (2004) Value of
information analysis in environmental health risk
management decisions: Past, present, and future. Risk
Analysis, 24 (3), pp. 635-650.