David H. Wolpert, Santa Fe Institute http://davidwolpert.weebly.com
Nils Bertschinger, Max Planck Institute
Eckehard Olbrich, Max Planck Institute
Juergen Jost, Max Planck Institute, Santa Fe Institute
POSITIVE VALUE OF INFORMATION
Row moves first, and Col receives a
noiseswamped
observation of Row’s move before Col moves
h m
H: 0, 0 3, 1
M: 2, 1 2, 3
POSITIVE VALUE OF INFORMATION
Row moves first, and Col receives a
noiseswamped
observation of Row’s move before Col moves
h m
H: 0, 0 3, 1
M: 2, 1 2, 3
(Unique) Nash Eq. is (H, h)
Col gets 0
POSITIVE VALUE OF INFORMATION
Row moves first, and Col receives a
noisefree
observation of Row’s move before Col moves
h m
H: 0, 0 3, 1
M: 2, 1 2, 3
(Unique) Nash Eq. is (M, m)
Col gets 3
POSITIVE VALUE OF INFORMATION
Row moves first, and Col receives a
noisefree
observation of Row’s move before Col moves
h m
H: 0, 0 3, 1
M: 2, 1 2, 3
Getting more information gains 3 for Col
NEGATIVE VALUE OF INFORMATION
Cournot duopoly game (moves are production levels)
Row moves first, and Col receives a
noiseswamped
observation of Row’s move before Col moves
h m l
H: 0, 0 12, 8 18, 9
M: 8, 12 16, 16 20, 15
L: 9, 18 15, 20 18, 18
NEGATIVE VALUE OF INFORMATION
Cournot duopoly game (moves are production levels)
Row moves first, and Col receives a
noiseswamped
observation of Row’s move before Col moves
h m l
H: 0, 0 12, 8 18, 9
M: 8, 12 16, 16 20, 15
L: 9, 18 15, 20 18, 18
(Unique) Nash Eq. is (M, m)
Col gets 16
NEGATIVE VALUE OF INFORMATION
Cournot duopoly game (moves are production levels)
Row moves first, and Col receives a
noisefree
observation of Row’s move before Col moves
h m l
H: 0, 0 12, 8 18, 9
M: 8, 12 16, 16 20, 15
L: 9, 18 15, 20 18, 18
(Unique) Nash Eq. is now (H, l)
Col gets only 9
NEGATIVE VALUE OF INFORMATION
Cournot duopoly game (moves are production levels)
Row moves first, and Col receives a
noisefree
observation of Row’s move before Col moves
h m l
H: 0, 0 12, 8 18, 9
M: 8, 12 16, 16 20, 15
L: 9, 18 15, 20 18, 18
Getting more information
costs
Col 7
NEGATIVE VALUE OF UTILITY
Change a game by imposing a tax on Row for every
move she might make. And she benefits:
L R
T: 10, 10 2, 8
B: 12, 2 0, 0
Nash eq. is (B, R)
Row gets 0
L R
T: 5, 10 3, 8
B: 4, 2 4, 0
Nash eq. is (T, L)
Row gets 5
VALUE OF PARAMETERS OF A GAME
1)
How analyze
of a
game?
2)
What is relation between a game’s parameters and
the values of those parameters to the players in the game?
3)
When is there negative val. of (info., utility) to a player?
4)
When is there negative val. of (info., utility) to
all
players?
5)
What are marginal rates of substitution among (values of)
information, utility, etc.?
DIFFERENTIAL VALUE OF INFORMATION
1)
“Value of a good” to a consumer is marginal utility of increasing
amount of just that good (per unit good).
DIFFERENTIAL VALUE OF INFORMATION
1)
“Value of a good” to a consumer is marginal utility of increasing
amount of just that good (per unit good).
2)
Value of a linear combination of goods?
 Marginal utility of increasing amount of just that linear comb
.
DIFFERENTIAL VALUE OF INFORMATION
1)
“Value of a good” to a consumer is marginal utility of increasing
amount of just that good (per unit good).
2)
Value of a linear combination of goods?
 Marginal utility of increasing amount of just that linear comb
.
3)
Value of arbitrary function f(
θ
) of vector of goods
θ
?
 Marginal utility of increasing amount of f(
θ
)
 I.e., directional deriv. of utility along
∇
f(
θ
), divided by 
∇
f(
θ
)
2
DIFFERENTIAL VALUE OF INFORMATION
1)
“Value of a good” to a consumer is marginal utility of increasing
amount of just that good (per unit good).
2)
Value of a linear combination of goods?
 Marginal utility of increasing amount of just that linear comb
.
3)
Value of arbitrary function f(
θ
) of vector of goods
θ
?
 Marginal utility of increasing amount of f(
θ
)
 I.e., directional deriv. of utility along
∇
f(
θ
), divided by 
∇
f(
θ
)
2
4)
Value of arbitrary function f of the game parameter
θ
?
Directional deriv. of utility along
∇
f(
θ
) divided by 
∇
f(
θ
)
2
E(u i
)
Θ f
•
•
•
•
Θ
E(u i
)
f
DIFFERENTIAL VALUE OF INFORMATION
Directional deriv. of utility along
∇
f(
θ
) divided by 
∇
f(
θ
)
2
1)
θ
c an be noise distribution in an information channel
2)
θ
c an be rationality parameter in a solution concept
 e.g., exponent in a QRE
3)
θ
c an be parameter specifying a utility function
4)
θ
c an be regulator’s choices , e.g., tax rates, regulations, etc.
5)
f(
θ
) can be mutual information between arbitrary nodes of a MAID
specified by
θ
, information capacity between them, rationality of a
player (or any other component of
θ
), etc.
RESULTS
Theorem :
∇
f(
θ
)
∇
V(
θ
) < 
∇
f(
θ
) 
∇
V(
θ
)
⇒
∃
changes to
θ
that reduce f (e.g., a channel’s info capacity) but increase V (e.g., expected utility)
Intuition :
If player values
anything
in addition to information, a change may reduce their information but increase their expected utility
RESULTS
Theorem :
r
( f )
62
Con (
{ r
( V i
)
}
) , Con (
{ r
( V i
)
}
) is pointed
⇒
∃
changes to
θ
that decrease f (e.g., a channel’s info. capacity) but increase V (e.g., expected utility) for all players
Intuition :
Unless a special condition relates the set of all gradients, there exists a change to game parameter that decreases information but is Paretopreferred
1)
What is relation between a game’s parameters and the values of
information to the players in the game?
2)
When is there negative val. of info. to a player?
3)
When is there negative val. of info. to
all
players?
4)
What are marginal rates of substitution among (value of)
information, rationality, utility functions, etc.?
Using differential value of information, can answer all these questions 
and more
RESULTS
Utility follower
4
3.8
3.6
3.4
3.2
3
2.8
2.6
2.4
2.2
2
0.5
0.45
0.4
0.35
eps
0.3
0.25
0.2
0.15
0.1
0.05
0
100
10
1
0.1
beta
0.01
Follower expected utility plotted against
θ
= ( channel noise, QRE exponent ( rationality value ))
Noisy leaderfollower binarymove game
 Similar for leader
DIFFERENTIAL VALUE OF INFORMATION
Directional deriv. of utility along
∇
f(
θ
) divided by 
∇
f(
θ
)
2
Unlike previous approaches, diff. val. of info. is
1)
Defined for
all
edges in an ID
2)
Defined for
any
changes to an edge in a (MA)ID – not just removal
3)
Defined separately for each equilibrium in a MAID
4)
By evaluating it for multiple f’s, allows us to compare:
•
•
value of information value of rationality, etc.
“This much extra information is equivalent to this much extra rationality”
DIFFERENTIAL VALUE OF INFORMATION
Directional deriv. of utility along
∇
f(
θ
) divided by 
∇
f(
θ
)
2
1)
Requires choosing a metric tensor over space of game param.
θ
2)
N.b.,
θ
fixes joint distribution over all variables in the MAID
3)
So natural choice for metric is Fisher information metric:
Information geometry of noncooperative games
QUANTIFYING AMOUNT OF INFORMATION
1)
2)
3)
Sample
P(x)
to get an x.
Then sample
P(y  x)
to get a y.
How much “information” is in that y, concerning that x?
Gain in accuracy of predicting x using P(x  y) rather than P(x )
Formalize this as a difference: 4)
loglikelihood of that x, evaluated under P(x  y)
minus loglikel. of that x,
mistakenly
evaluated under P(x
)
5) Average this difference over x and y:
Average amount of information in y concerning x:
Shannon’s mutual information between y and x
RESULTS
@
2 f
@✓ i
@✓ j
 r f ( ✓ )

 r V ( ✓ ) 
@
2
V
@✓ i
@✓ j
 r
V ( ✓ )

2 r
V ( ✓ ) i r
V ( ✓ ) j
=
@
2 f
@✓ i
@✓ j
 r f ( ✓ )

 r
V ( ✓ )

@
2
V
@✓ i
@✓ j
1) One of our ECS slides giving a player's mixed strategy as a function of beta and epsilon
2) The associated slide giving expected utility as a function of beta and epsilon
3) 2player game Edgeworth box plotting functions for only one of the players
(to keep the figure clear). Use this to illustrate our definition of value of info.
4) The associated mutual information and information capacity heat maps.
5) (3) again, to illustrate Prop. 1 visually.
 Formal statement of Prop. 1
6) (3) again, to illustrate Prop. 3 visually.
 Formal statement of Prop. 3
7) Edgeworth box showing functions for both players, illustrating Prop. 4 visually.
Formal statement of Prop. 4
INFLUENCE DIAGRAMS
Game against Nature
 represented as playerfixed Bayes net :
Chance Nodes
C (prefixed CPs)
model Nature, player observations.
Decision Nodes
D (player fixes
CPs at D, D’, D”) give player’s actions
The player sets CP(s) of her
decision nodes based on expected
utility of resultant full Bayes net
C
2
D’’
C
1
D’
C
3
D
PREVIOUS APPROACH TO VALUE OF INFO.
IN GAMES AGAINST NATURE
1)
Calculate maximal expected utility with
red edge – U
1
C
1
C
2
D
D’
D’’
C
3
VALUE OF INFORMATION IN GAMES
AGAINST NATURE
1)
Calc. maximal expected utility with
red edge – U
1
2)
Calc. maximal expected utility without
red edge – U
2
C
2
C
1
D’
D
D’’
C
3
VALUE OF INFORMATION IN GAMES
AGAINST NATURE
1)
Calc. maximal expected utility with
red edge – U
1
2)
Calc. maximal expected utility without
red edge – U
2
3) “ Value ” of red edge =
U
1
 U
2
C
2
C
1
D’
D
D’’
C
3
VALUE OF INFORMATION IN GAMES
AGAINST NATURE
1)
Calc. maximal expected utility with
red edge – U
1
2)
Calc. maximal expected utility without
red edge – U
2
3) “ Value ” of red edge =
U
1
 U
2
4)
Cannot measure value of edges into
chance nodes
5)
Cannot measure value of changes to
CPs of edges
6)
Cannot measure value of nodes
C
2
D’’
C
1
D’
C
3
D
MULTIAGENT INFLUENCE DIAGRAMS
MultiPlayer Game
 represented as playerfixed Bayes net :
Chance Nodes
C give states,
observations, etc. (fixed CPs)
Decision Nodes
D i
give i’s actions
(player i sets CP at D i
)
Players sets the CP(s) of their
decision nodes based on expected
utility of resultant full Bayes net.
C
2
D
1
C
1
D
2
C
3
D
1
PREVIOUS APPROACH TO VALUE OF
INFO. IN MULTIPLAYER GAMES
1)
Extensive form rep. used (not MAIDs)
2)
Provides
ordinal
“value” of different
information partitions
C
2
C
1
D
2
D
1
D
1
C
3
PREVIOUS APPROACH TO VALUE OF
INFO. IN MULTIPLAYER GAMES
1)
Extensive form rep. used (not MAIDs)
2)
Provides
ordinal
“value” of different
information partitions
3)
Cannot use for numeric analysis
4)
Difficulties comparing arbitrary
information partitions
5)
Cannot apply to games with multiple
equilibria
C
2
D
1
C
1
D
2
C
3
D
1
SHANNON INFORMATION THEORY
1)
Very deep and powerful formalization of information
2)
Arises in earlier work on games, e.g., as
•
Tool to formalize modeler’s uncertainty
•
Model of bounded rational players
•
•
 Quantal Response Equilibrum – QRE
Analogy with statistical physics
Total payoff in multistage games with multiplicative payoffs
3)
Never used before to quantify information
within
the game
(e.g., as mutual information, information capacity, etc.)
SHANNON INFORMATION THEORY
How use Shannon information theory to analyze value of information in games?
SHANNON INFORMATION THEORY
How use Shannon information theory to analyze value of information in games?
Can we answer our motivating questions if we figure out how to answer this one?
SHANNON INFORMATION THEORY
How use Shannon information theory to analyze value of information in games?
Can we answer our motivating questions if we figure out how to answer this one?
Yes
MUST BE MORE SPECIFIC ….
•
Value of information
•
{ Marginal }
value of information
•
•
•
Marginal value of information or
{ capacity between v
•
Marginal value of
{ mutual }
information
{ between v
1
and v
2
}
or
Marginal value of
{ multi}
information
{ among v
1
, v
1
and v
2
}
2
, v
3
, … }
Marginal
{ to player i }
… value of mutual information between v
1
and v
2
ROADMAP
1)
Graphical models of games
2)
Previous work involving
Shannon Info. Theory and games
3)
From marginal utility to value of information; information geometry of games
4)
Theorems and plots
NEGATIVE VALUE OF INFO. TO ALL PLAYERS
Braess’ paradox:
t = commute time for each player on indicated road,
T = total number of players on indicated road,
4000 players total,
cost = 4 for a player to try to go from A to B,
A priori
probability that road from A to B is open = .1
All players get
noiseswamped
signal of whether AtoB is open
Nash Eq
: 2000 players go from Start to A to End,
2000 players go from Start to B to End.
All players have 65 minute commute
NEGATIVE VALUE OF INFO. TO ALL PLAYERS
Braess’ paradox
:
t = commute time for each player on indicated road,
T = total number of players on indicated road,
4000 players total,
cost = 4 for a player to try to go from A to B,
A prior
i probability that road from A to B is open = .1
All players get
noisefree
signal of whether AtoB is open
Nash Eq
: Same as before when AtoB is closed (.9 probability)
When AtoB is open, all players go from Start to A to B to End,
in which case all players have 84 minute commute
NEGATIVE VALUE OF INFO. TO ALL PLAYERS
Braess’ paradox
:
t = commute time for each player on indicated road,
T = total number of players on indicated road,
4000 players total,
cost = 5 for a player to try to go from A to B,
A prior
i probability that road from A to B is open = .1
All players get
noisefree
signal of whether AtoB is open
When all players get extra information (is A to B open?), either no change in commute time (.9 probability), or all players have 19 minute longer commute (.1 probability)
VALUE OF INFORMATION
1)
What is relation between a game’s parameters and
the
to the players in the game?
2)
When is there negative val. of info. to a player?
3)
When is there negative val. of info. to
all
players?
VALUE OF … ANYTHING?
1)
What is relation between a game’s parameters and
the values of information to the players in the game?
2)
When is there negative val. of info. to a player?
3)
When is there negative val. of info. to
all
players?
Can we analyze value of tax, regulation, etc., with the same formalism?
DIFFERENTIAL VALUE OF INFORMATION
Row moves first, and Col receives a noisy
observation of Row’s move before Col moves
h m
H: 0, 0 3, 1
M: 2, 1 2, 3
RESULTS
Leader strategy
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0.5
0.45
0.4
0.35
eps
0.3
0.25
0.2
0.15
0.1
0.05
0
100
10
1
0.1
beta
0.01
Leader mixed strategy plotted against
θ
= ( channel noise, QRE exponent ( rationality value ))
Noisy leaderfollower binarymove game
 Similar for follower
RESULTS
0.5
0.45
0.4
0.35
0.3
0.25
0.2
Branch 4: CHANNELCAPACITY
UTILLEADER
UTILFOLLOWER
CHANNELCAPACITY
UTILLEADER
UTILFOLLOWER
CHANNELCAPACITY
0.15
0.1
0.05
0
0 0.05
0.1
0.15
0.2
0.25
eps1
0.3
0.35
0.4
0.45
0.5
5
4.5
4
3.5
3
2.5
2
Theorem :
∇
f(
θ
)
∇
V(
θ
) < 
∇
f(
θ
) 
∇
V(
θ
)
⇒
∃
changes to
θ
that reduce f (e.g., a channel’s info capacity) but increase V (e.g., expected utility)
RESULTS
0.5
0.45
0.4
0.35
0.3
0.25
0.2
Branch 4: CHANNELCAPACITY
UTILLEADER
UTILFOLLOWER
CHANNELCAPACITY
UTILLEADER
UTILFOLLOWER
CHANNELCAPACITY
0.15
0.1
0.05
0
0 0.05
0.1
0.15
0.2
0.25
eps1
0.3
0.35
0.4
0.45
0.5
5
4.5
4
3.5
3
2.5
2
Intuition :
If player values
anything
in addition to information, a change may reduce their information but increase their expected utility
RESULTS
0.5
0.45
Branch 4: CHANNELCAPACITY
UTILLEADER
UTILFOLLOWER
CHANNELCAPACITY
UTILLEADER
UTILFOLLOWER
CHANNELCAPACITY
5
4.5
0.4
0.35
4
0.3
0.25
0.2
3.5
3
0.15
0.1
2.5
0.05
0
0
2
0.05
0.1
0.15
0.2
0.25
eps1
0.3
0.35
0.4
0.45
0.5
Theorem :
r
( f )
62
Con (
{ r
( V i
)
}
) , Con (
{ r
( V i
)
}
) is pointed
⇒
∃
changes to
θ
that decrease f (e.g., a channel’s info. capacity) but increase V (e.g., expected utility) for all players
RESULTS
0.5
0.45
0.4
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0
0
Branch 4: CHANNELCAPACITY
UTILLEADER
UTILFOLLOWER
CHANNELCAPACITY
UTILLEADER
UTILFOLLOWER
CHANNELCAPACITY
0.05
0.1
0.15
0.2
0.25
eps1
0.3
0.35
0.4
0.45
0.5
5
4.5
4
3.5
3
2.5
2
Intuition :
Unless a special condition relates the set of all gradients, there exists a change to game parameter that decreases information but is Paretopreferred
RESULTS
0.5
0.45
0.4
0.35
0.3
0.25
0.2
Branch 4: CHANNELCAPACITY
UTILLEADER
UTILFOLLOWER
CHANNELCAPACITY
UTILLEADER
UTILFOLLOWER
CHANNELCAPACITY
0.15
0.1
0.05
0
0 0.05
0.1
0.15
0.2
0.25
eps1
0.3
0.35
0.4
0.45
0.5
5
4.5
4
3.5
3
2.5
2
Level surfaces of expected utilities and channel capacity plotted against channel parameters in noisy leaderfollower binarymove game.
RESULTS
0.5
0.45
Branch 4: CHANNELCAPACITY
UTILLEADER
UTILFOLLOWER
CHANNELCAPACITY
UTILLEADER
UTILFOLLOWER
CHANNELCAPACITY
5
4.5
0.4
0.35
4
0.3
0.25
0.2
3.5
3
0.15
0.1
2.5
0.05
0
0
2
0.05
0.1
0.15
0.2
0.25
eps1
0.3
0.35
0.4
0.45
0.5
Theorem :
@
2 f
@✓ i
@✓ j
 r f ( ✓ ) 
 r V ( ✓ ) 
@
2
V
@✓ i
@✓ j
@
2 f
 r f ( ✓ )

@
2
V
 r
V ( ✓ )

2 r
V ( ✓ ) i r
V ( ✓ ) j
=
@✓ i
@✓ j
 r
V ( ✓ )

@✓ i
@✓ j
⇒
The set {
θ
: no changes to
θ
reduce f while increasing V} has measure 0
RESULTS
0.5
0.45
0.4
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0
0
Branch 4: CHANNELCAPACITY
UTILLEADER
UTILFOLLOWER
CHANNELCAPACITY
UTILLEADER
UTILFOLLOWER
CHANNELCAPACITY
0.05
0.1
0.15
0.2
0.25
eps1
0.3
0.35
0.4
0.45
0.5
5
4.5
4
3.5
3
2.5
2
Intuition :
Unless a special condition between Hessians and gradients holds
for all
values of game parameter
θ
, there exists a game parameter
θ
and changes to it that decrease information but increase expected utility
RESULTS
0.5
0.45
Branch 4: CHANNELCAPACITY
UTILLEADER
UTILFOLLOWER
CHANNELCAPACITY
UTILLEADER
UTILFOLLOWER
CHANNELCAPACITY
5
4.5
0.4
0.35
4
0.3
0.25
0.2
3.5
3
0.15
0.1
2.5
0.05
0
0
2
0.05
0.1
0.15
0.2
0.25
eps1
0.3
0.35
0.4
0.45
0.5
Theorem : 
r
( f )
62
Con (
{ r
( V i
)
}
) , Con (
{ r
( V i
)
}
) is pointed
⇒
∃
changes to
θ
that
increase
f (e.g., a channel’s info. capacity) but increase V (e.g., expected utility) for all players
RESULTS
0.5
0.45
0.4
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0
0
Branch 4: CHANNELCAPACITY
UTILLEADER
UTILFOLLOWER
CHANNELCAPACITY
UTILLEADER
UTILFOLLOWER
CHANNELCAPACITY
0.05
0.1
0.15
0.2
0.25
eps1
0.3
0.35
0.4
0.45
0.5
5
4.5
4
3.5
3
2.5
2
Intuition :
Unless a special condition relates the set of all gradients, there exists a change to game parameter that increases information and is Paretopreferred
RESULTS
•
•
•
0.5
0.45
Branch 4: CHANNELCAPACITY
UTILLEADER
UTILFOLLOWER
CHANNELCAPACITY
UTILLEADER
UTILFOLLOWER
CHANNELCAPACITY
5
4.5
0.4
0.35
4
0.3
0.25
0.2
3.5
3
0.15
0.1
2.5
0.05
0
0
2
0.05
0.1
0.15
0.2
0.25
eps1
0.3
0.35
0.4
0.45
0.5
Issue is Paretoimproving change to
θ
,
not
Paretoimproving change to mixed strategy profile
Not uncommon that the “special condition” does hold.
Example
: constantsum games .
Generically, if number players < dim(
θ
),
∃
both changes to
θ
that: i) Increase f and are Paretoimproving ii) Decrease f and are Paretoimproving.