Towards Realistic Models for
Evolution of Cooperation
LIK MUI
… about procedure …
• Briefly go over the paper
– Clarify major points
• Describe simulations (not in paper)
RoadMap

•
•
•
•
Introduction
Cooperation Models
Simulations
Conclusion
Evolution of Cooperation
• Animals cooperate
• Two questions:
– How does cooperation as a strategy becomes
stable evolutionarily?
– How does cooperation arise in the first place?
Darwinian Natural Selection
“Survival of the fittest”
• If evolution is all about individual survival,
how can cooperation be explained?
• Fittest what?
Fittest what ?
• Individual
– Rational agency theory (Kreps, 1990)
• Group
– Group selection theory (Wilson, 1980)
• Gene
– Selfish gene hypothesis (Dawkins, 1979)
• Organization
– Classic organizational theory (Simon,
1969)
RoadMap

• Introduction
• Cooperation Models
•
•
•
•
•
Group Selection
Kinship Theory
Direct Reciprocity
Indirect Reciprocity
Social Learning
• Simulations
• Conclusion
Group Selection
• Intuition: we ban cannibalism but not
carnivorousness
• Population/species: basic unit of natural
selection
• Problem: explain war, family feud,
competition, etc.
Kinship Theory I
• Intuition: nepotism
• Hamilton’s Rule:
c
r
b
– Individuals show less aggression and more
cooperation towards closer kin if rule is satisfied
– Basis for most work on kinship theory
• Wright’s Coefficient of Related: r
– Self: r=1
– Siblings: r=0.5
– Grandparent-grandchild: r=0.25
Kinship Theory II
• Cannot explain:
– Competition in viscuous population
– Symbioses
– Dynamics of cooperation
Direct Reciprocity
• Intuition: being nice to others who are nice
• “Reciprocal Altruism”
– Trivers (1971)
• Tit-for-tat and PD tournament
– Axelrod and Hamilton (1981)
• Cannot explain:
– We cooperate not only with people who cooperate
with us
Indirect Reciprocity
• Intuition: respect one who is famous
• Social-biological justifications
– Biology: generalized altruism (Trivers, 1971, 1985)
– Sociobiology: Alexandar (1986)
– Sociology: Ostrom (1998)
• 3 types of indirect reciprocity:
– Looped
– Observer-based
– Image-based
Indirect Reciprocity: Looped
• Looped Indirect Reciprocity
– Boyd and Richerson (1989)
Indirect Reciprocity: Observers
• Observer-based Reciprocity
– Pollock and Dugatkin (1992)
Indirect Reciprocity: Image
• Image (reputation) based Reciprocity
– Nowak and Sigmund (1998, 2000)
Social Learning
• Intuition: imitate those who are successful
• Cultural transmission
– Boyd and Richerson (1982)
• Docility
– Simon (1990, 1991)
Critiques of Existing Models
• Many theories each explaining one or a
few aspects of cooperation
• Unrealism of model assumptions
Unrealism for Existing Models
• asexual, non-overlapping generations
• simultaneous play for every interaction
– c.f., Abell and Reyniers, 2000
• dyadic interactions
• mostly predetermined behavior
– c.f., May, 1987 (lack of modeling stochasticity)
• discrete actions (cooperate or defect)
• social structure and cooperation
– c.f., Simon, 1991; Cohen, et al., 2001
• extend social learning
– c.f., Simon, 1990
RoadMap

• Introduction
• Cooperation Models
• Simulations
• Nowak and Sigmund Game
• Prisoner’s Dilemma Game
• Simon’s Docility Hypothesis
• Conclusion
Nowak and Sigmund Game
Interact
Not interact
Donor
-C
0
Recipient
B
0
Interact
Not interact
Donor
A
-A
Recipient
0
0
• Payoff Matrix
C = 0.1
B = 1.0
• Image Adjustment
A=1
Abundance
Using Global Image: 1 Run
100
90
80
70
60
50
40
30
20
10
0
t=0
-5 -4 -3 -2 -1 0 1 2 3 4 5 6
Using Global Image: 100 Runs
Dynamics using Global Reputation
8
6
Strategy, K
4
2
0
-2
-4
0.3
-6
0.25
Num ber of Generations
50,000
Frequency
0.2
0.15
0.1
2.5
0.05
2
0
1
1.5
2
3
4
5
6
7
Payoff
Strategy, K
1
0.5
0
Num ber of Generations
50,000
8
9
10
11
12
Using 10 Observers/Interactions
0.3
0.18
n=50
n=20
0.16
0.25
0.14
0.2
Frequency
Frequency
0.12
0.1
0.08
0.06
0.15
0.1
0.04
0.05
0.02
0
0
-5
-4
-3
-2
-1
0
1
2
3
4
5
6
-5
-4
-3
-2
-1
Strategy, K
1
2
3
4
5
6
0.3
0.35
n=200
n=100
0.3
0.25
0.25
0.2
Frequency
Frequency
0
Strategy, K
0.2
0.15
0.15
0.1
0.1
0.05
0.05
0
0
-5
-4
-3
-2
-1
0
1
Strategy, K
2
3
4
5
6
-5
-4
-3
-2
-1
0
1
Strategy, K
2
3
4
5
6
Evolutionary PD Game
• Repeated Prisoners’ Dilemma Game
• Agent Actions:
Action = { cooperate, defect }
• Payoff Matrix:
C D
C 3/3 0/5
D 5/0 1/1
PD Game Agent Strategies
• All defecting (AllD)
• Tit-for-tat (TFT)
• Reputational Tit-for-tat (RTFT): using
various notions of reputation
Base Case: PD Game
Group Reputation (base: min_gr >= 0)
10000
12000
12100
12200
12500
120
100
TFT Count
80
60
40
20
0
1
4
7
10 13 16 19 22 25 28 31 34 37 40 43 46 49
Generation
Simple Groups: social structures
• Group structure affects members
– Interactions, observations, and knowledge
– Persistent structure
• Groups actions
– Observed indirectly through member's
actions
Group Membership
• Member agents
– Have public group identity
– Directly associated with one environment
• Group Structure is a Tree
– Least common ancestral node (LCAN)
– Events occur with respect to a shared
environment
Shared Environment Example
Agents
A1,A2
A3,A4
A5,A2
A1,A3
A5,A3
Group
G1
G2
G1
G0
G0
A0
G0
A3
A4
A1
A2
G1
G2
A5
G3
PD Game with Group Reputation
(varying encounters per generation EPG)
Group Reputation (min_gr >= 0.5)
100
200
500
1000
1200
120
100
TFT Count
80
60
40
20
0
1
4
7
10 13 16 19 22 25 28 31 34 37 40 43 46 49
Generation
PD Game with Group Reputation
(100 EPG; varying Inter-group interaction probability)
Group Reputation (min_gr >= 0.5, varying ip)
0.1
0.3
0.325
0.35
1.0
120
100
TFT Count
80
60
40
20
0
1
4
7
10 13 16 19 22 25 28 31 34 37 40 43 46 49
Generation
Groups/Organizations:
bounded rationality explanation
• Docility
– Cooperation (altruism) as an explanation for the
formation of groups/organizations
• Why individuals “identify” with a group?
– boundedly rational individuals
– increase their survival fitness
(Simon, 1969, 1990, 1991)
PD Game with Docility
(50 cooperators and 50 defectors; 100 EPG; 1.0 IP)
Varying intergroup docility, intragroup docility = 1.0
0.0
0.4
0.41
0.425
1.0
120
Cooperator Count
100
80
60
40
20
0
1
4
7
10 13 16 19 22 25 28 31 34 37 40 43 46 49
Generation
Conclusion
• Reviewed 5 major approaches to study
evolution of cooperation
• Provided 2 main critiques for existing
models
• Constructed model extensions addressing
the critiques
Implications for Computer Science
• Artificial intelligence
– Benevolent agents are not good enough
(c.f., multi-agents systems)
– Learning theory can be used to study evolution of
cooperation
• Systems
– Improve system design by understanding the
dynamics of agents
– Accountability substrate needed for distributed
systems
Future Plan
•
•
•
•
Extend the simple group social structure
Overlapping generations
Sexual reproduction
Extend social learning using
realistic/robust learning model
Modeling Diploid Organisms
Modeling Diploid Organisms
Modeling Diploid Organisms
Parental Chromosomes
One of 2 Child Chromosomes
Simulation Demo
• Recall PD payoff matrix:
C
D
C
D
R/R
S/T
T/S
P/P
• PD strategies: viewed as a probability vectors
–
–
–
–
–
Strategy:
TFT:
AllD:
AllC:
STFT:
<PI, PT, PR, PP, PS>
< 1, 1, 1, 0, 0 >
< 0, 0, 0, 0, 0 >
< 1, 1, 1, 1, 1 >
< 0, 1, 1, 0, 0 >
Simulation: a search problem
• Search Optimal PD Strategy
– Search space: I, T, R, P, S probabilities