Game and Evolutionary Game in
Communication Networks
Yuedong Xu
2013.12.04
Outline
• Game Theory: A Premier
• Evolutionary Game
• Applications to Networks
• Potential Research Fields
Using as less math
as possible !
Game Theory: A Premier
• What is “game” about?
• Game of Chicken
– driver who swerves away looses
2
2
• What should drivers do?
– To swerve or to stay?
Game Theory: A Premier
• What is “game” about?
• Game of Chicken
– driver who swerves away looses
2
2
Driver 2
swerve
stay
Driver 1
swerve
stay
0, 0
5, -1
-1, 5
-10, -10
Drivers want to
do opposite of
one another
Game Theory: A Premier
• A Game consists of
– at least two players
– a set of strategies for each player
– a payoff for each strategy profile
• Basic assumption (rationality of players)
• Nash Equilibrium
– no player can improve its payoff by unilaterally
changing its strategy
• Pareto optimality, price of anarchy
Game Theory: A Premier
Classification 1:
• Non-Cooperative (Competitive) Games
– individualized play
• Cooperative Games
– play as a group
• Repeated, Stochastic and Evolutionary Games
– not one shot
Game Theory: A Premier
Classification 2:
Non-cooperative
Cooperative
Static
Dynamic (repeated)
Strategic-form
Extensive-form
Perfect information
Imperfect information
Complete information
Incomplete information
Game Theory: A Premier
Internet Application
v
C(x) = x
C(x) = 1
C(x) = 0
s
C(x) = 1
t
C(x) = x
w
Selfish Routing game
Game Theory: A Premier
Internet Application
P2P Networks: Bittorrent, Xunlei, Pplive, PPStream, QQLive …
Game Theory: A Premier
Internet Application
Internet Ecosystem (Business Models)
Game Theory: A Premier
Internet Application
Cloud Computing game
Game Theory: A Premier
Internet Application
Online Social Networks
Game Theory: A Premier
Internet Application
Network Security Game
Game Theory: A Premier
Wireless Application
802.11 multiple access game
Game Theory: A Premier
Wireless Application
3G/4G Power Control Game
Game Theory: A Premier
Wireless Application
?
Blue
Green
?
Packet forwarding game
Game Theory: A Premier
Wireless Application
Cognitive radio network game
Game Theory: A Premier
Wireless Application
E
Wireless jamming and eavesdrop games
Outline
• Game Theory: A Premier
• Evolutionary Game
• Applications to Networks
• Potential Research Fields
Recap
• Classical game theory (CGT)
– Outcome depends on strong rationality
assumption
– Each individual uses a strategy that is the "best
response" to other players’ choice
– Question: what is the meaning of a symmetric NE
∗
(𝑠𝑖∗ , 𝑠−𝑖
), given a large number of players ?
Follow the crowd!
Evolutionary game theory
• Evolutionary game theory (EGT)
Evolutionary game theory differs from classical game theory by focusing
more on the dynamics of strategy change as influenced not solely by the
quality of various competing strategies, but by the effect of frequency
with which the various competing strategies are found in the population.
– refinement of CGT
– game in a population
– dynamics of strategy adoption
– mutual learning among players
Evolutionary game theory
• Evolutionary game theory (EGT)
– Usually two types of game: games against the field
and games with pairwise contests
A game against the field is one in which there is no specific “opponent”
for a given individual - their payoff depends on what everyone in the
population is doing. Ex: Choice of Gender
A pairwise contest game describes a situation in which a given
individual plays against an opponent that has been randomly selected
(by nature) from the population and the payoff depends just on what
both individual do. Ex: Hawk-Dove Game
Evolutionary game theory
• A profile of evolutionary game
Given a set of pure strategy S. A population profile is a vector x that
gives a probability x(s) with which each strategy s ∈ S is played in the
population.
• Payoff (fitness)
Consider a particular individual in the population with profile x. If
that individual uses a profile σ={p s , s ∈ 𝐒}, the individual’s payoff is
denoted as 𝜋(σ, 𝒙). The payoff of this strategy for a pair-wise game is
𝜋 σ, 𝒙 =
𝑝 𝑠 𝑥(𝑠 ′ )𝜋 𝑠, 𝑠 ′ )
𝑝 𝑠 𝜋 𝑠, 𝒙 =
𝑠∈𝑺
𝑠∈𝑺 𝑠 ′ ∈𝑺
Evolutionary game theory
• Evolutionary stable strategy (ESS)
An evolutionarily stable strategy is a strategy which, if adopted by a
population of players, cannot be invaded (or replaced) by any
alternative strategy that is initially rare.
• Theorem (ESS)
Evolutionary game theory
• Example (Hawk-Dove Game)
– H: aggressive; D: mild
– Population strategy 𝒙 = (𝑥, 1 − 𝑥)
– Mixed strategy (H,D) of an individual 𝜎 = (𝑝, 1 − 𝑝)
– Payoff matrix (v<c):
– Suppose the existence of an ESS 𝒙∗ = (𝑝∗ , 1 − 𝑝∗ )
Evolutionary game theory
• Example (Hawk-Dove Game) ‘cont
– In the population, the payoff of a mutant is
Evolutionary game theory
Evolutionary game theory
• ESS
– no statement of dynamics
– monomorphic / polymorphic
• Replicator Dynamics
– individuals, called replicator, exist in several different
types (e.g Hawk and Dove)
– each type of individual uses a pre-programmed
strategy and pass it to its descendants
– individuals only use PURE strategy in a finite set
– the population state is (𝑥1 , 𝑥2 , … , 𝑥𝐾 ) where 𝑥𝑘 is
fraction of individuals using strategy 𝑠𝑘
Evolutionary game theory
• Replicator Dynamics
𝑥𝑖 = 𝑥𝑖 (𝜋𝑖 𝑠𝑖 , 𝒙 − 𝜋 𝑡 )
– Fixed point: 𝑥𝑖 = 0
– Stability of fixed point:
Lyapunov stability vs asymptotic stability
– Stability proof:
Lyapunov function and Engenvalue approach
Evolutionary game theory
• ESS vs NE in associated two-player game
– An ESS is a (mixed) NE
– A NE might not be an ESS
• Asymmetric NE in monomorphic population
• Unstable NE
Evolutionary game theory
• Replicator dynamics and NE
– In a two-strategy game
• Any NE is a fixed point of replicator dynamics
• Not every fixed point corresponds to a NE
• Replicator dynamics and ESS
– ESS is an asymptotically stable fixed point
– Two strategy pair-wise contest
ESS  Asymp. Stable f.p.⊆ sym. NE⊆ f.p.
– More than two strategies
ESS ⊆ Asymp. Stable f.p. ⊆ sym. NE ⊆ f.p.
Outline
• Game Theory: A Premier
• Evolutionary Game
• Applications to Networks
• Potential Research Fields
Peer-to-peer file sharing
Wireless networks
Peer-to-peer file sharing
• File  Piece (e.g. chunk, block)
– A content is split in pieces
– Each piece can be independently downloaded
• Leecher
– A peer that is client and server
– In the context of content delivery
• Has a partial copy of the content
• Seed
– A peer that is only server
– In the context of content delivery
• Has a full copy of the content
Peer-to-peer file sharing
Seed
t=0
t=T
t=2T
time
• Great improvement
over customer-server
mode
• Ideal system: single
chunk, fully cooperative
• Big System: many peers,
many chunks, stochastic
system
Which peers shall I serve
in each time slot?
Peer-to-peer file sharing
• If no good incentive strategy
– Slow service
– Even overwhelmed by requests
• Incentive model
– A strategy is the behavior (providing/rejecting a
service) of a peer against other peers
– A policy is the set of rules of for incentivization
– A point is a utility measure of peers
– A system is robust : convergence and cooperation
Q. Zhao, J. Lui, D. Chiu“A Mathematical Framework for Analyzing Adaptive
Incentive Protocols in P2P Networks”, IEEE/ACM Trans. Networking, 2012
Peer-to-peer file sharing
• Incentive model (’cont)
– Strategy = type of peer
– Finite strategies
• {cooperator, defector, reciprocator}
Always serve
Always reject
Serve cooperators and reciprocators
with certain probabilities,
reject defectors
Peer-to-peer file sharing
• Incentive model (’cont)
– System description:
At the beginning of each time slot, each peer randomly
selects another peer to request for service. The selected
peer chooses to serve the request based on his current
strategy. A peer obtains α points if its request is served
and loses β (=1) points if it provides service to others.
– Incentive scheme (esp. for reciprocators)
𝒈𝒊 (𝒋) - Prob. that a type i peer provides service to a type j peer
Peer-to-peer file sharing
• Utility model
– After a long way, the points gained by a type-i peer
𝑃𝑖 𝑡 = 𝛼
3
𝑗=1 𝑥𝑗
𝑡 𝑔𝑗 𝑖 −
3
𝑗=1 𝑥𝑗
𝑡 𝑔𝑖 𝑗
𝑃 𝑡 = 𝛼 − 1 𝒙𝑇 𝑮𝒙
Type-i payoff
Network payoff
• We can now study
– equilibrium state (given G)
– is the equilibrium stable?
– how to reach this equilibrium?
– how good is the incentive scheme
Is this enough?
Peer-to-peer file sharing
• Learning model in P2P networks
– Current best learning model
At the end of each slot, a peer chooses to switch to
another strategy s’ with certain prob. To decide which
strategy to choose, the peer learns from other peers.
𝑥𝑖 𝑡 + 1 = 𝑥𝑖 𝑡 − 𝛾𝑥𝑖 𝑡 𝑃ℎ 𝑡 − 𝑃𝑖 𝑡 ,
𝑥ℎ 𝑡 + 1 = 𝑥ℎ 𝑡 + 𝛾
3
𝑖=1,≠ℎ
𝑖 ≠ ℎ,
𝑥𝑖 𝑡 (𝑃ℎ 𝑡 − 𝑃𝑖 𝑡 )
Needs to compute the gains of all other peers !
Peer-to-peer file sharing
• Learning model in P2P networks
– Opportunistic learning model
At the end of each slot, each peer chooses another peer
as its teacher with certain prob. If the teacher is of a
different type and performs better, this peer adapts to
the teacher’s strategy with another prob.
𝑥𝑗 𝑡 + 1 = 𝑥𝑗 𝑡 + 𝛾𝑥𝑗 𝑡 (𝑃𝑗 𝑡 − 𝑃 𝑡 )
Simpler !
Peer-to-peer file sharing
• Now we can study
– Robustness of incentive scheme
Prob. That reciprocators serve other types of peers!
• Mirror incentive policy
– reciprocators are tit-for-tat
• Proportional incentive policy
– A reciprocators always serves any other reciprocator
• Linear incentive policy
Each scheme generates a different matrix G !
Peer-to-peer file sharing
Peer-to-peer file sharing
• In relation to EG
– pair-wise contest population game
– peers players; chunk exchange2 players games
𝑥𝑖 𝑡 = 𝛾𝑥𝑖 𝑡 (𝑃𝑖 𝑡 − 𝑃 𝑡 )
Opp. Learning
𝑥𝑖 𝑡 = 𝛾𝑥𝑖 𝑡 𝑃ℎ 𝑡 − 𝑃𝑖 𝑡 , 𝑖 ≠ ℎ
𝑥ℎ 𝑡 = 𝛾 𝑃ℎ 𝑡 − 𝑃 𝑡
Curr. Best Learning
𝜋𝑖 (𝑠𝑖 , 𝒙) = 𝑃𝑗 𝑡
After some efforts
𝑥𝑖 = 𝑥𝑖 (𝜋𝑖 𝑠𝑖 , 𝒙 − 𝜋 𝑡 )
Replicator dynamics
Large-scale wireless networks
• Random multiple access (slotted ALOHA)
– A node transmits with prob. p in each slot
– Simultaneous transmission  collisions
Large-scale wireless networks
• Power control game
(signal to noise interference ratio, SINR)
– Large power  better throughput
– Large power  more interference to other
receivers
Large-scale wireless networks
Large-scale wireless networks
• Sad facts:
– Selfishness is unsuccessful
– Optimal cooperation is hard in a large distributed
networks (bargaining, Shapley value)
What if wireless nodes learn from each other
in local interactions?
• Evolutionary game kicks in!
H. Tembine, E. Altman, “Evolutionary Games in Wireless
Networks”, IEEE Trans. Syst. Man Cyber. B, 2010
Large-scale wireless networks
• Challenges
– Standard EGT: a player interacts with all other
players (or average population)
– Large-scale wireless networks:
• no longer strategic pair-wise competition
• random number of local players
Non standard EGT
 Standard EGT
• non-reciprocal interactions
– Finite strategies of a player
{transmit, stay quiet} in multiple access game
{high power, low power} in power control game
Large-scale wireless networks
• WCDMA power control game
– SINR with distance r between transmitter and
receiver of node i is given by
PL
PH
PH
Pi : the strategy of node i (i.e., PH or PL)
x : the proportion of the population choosing PH
g : channel gain, r0 is the radius-of-reception circle of receiver
α : the attenuation order with value between 3 and 6, σ : the noise
power, and β : the inverse of processing gain
I(x) : total interference from all nodes to the receiver of node i
Large-scale wireless networks
• WCDMA power control game
– Payoff of node i is as follows:
R : transmission range
wp : cost weight due to adopting power
Pi (e.g. energy consumption)
ζ(r) : probability density function given the density of receiver
Large-scale wireless networks
• WCDMA power control game
– Existence of uniqueness of ESS
• Replicator dynamics
This function is continuous and strictly monotonic,
which is required for the proof of stability based on
sufficient condition
Large-scale wireless networks
• Some other related works
– Extensions to EGT
E. Altman, Y. Hayel. “Markov Decision Evolutionary Games”, IEEE Trans. Auto.
Ctrl. 2010
X. Luo and H. Tembine. “Evolutionary Coalitional Games for Random Access
Control”, IEEE Infocom 2013 (mini)
– Applications
P. Coucheney, C. Touati. “Fair and Efficient User-Network Association
Algorithm for Multi-Technology Wireless Networks”, IEEE Infocom 2009 (mini)
S. Shakkottai, E. Altman. “The Case for Non-cooperative Multihoming of Users
to Access Points in IEEE 802.11 WLANs”, IEEE Infocom 2006
C. Jiang, K. Liu, “Distributed Adaptive Networks: A Graphical Evolutionary
Game-Theoretic View”, IEEE Trans. Signal Processing, 2013
Large-scale wireless networks
• Summary
– P2P : practical problem  EG theory
– WCDMA: EG theory  practical problem
Two different styles !
– Common Challenges:
• difficult to find important problem
• difficult to have theoretical contributions to EGT
Thank you!
Q&A