Calderon, J. P., & Zamara, R. (2006). How Learning Affects the Evolution of Strong
Reciprocity. Journal of Adaptive Behavior, 14(3), 211-221.
Summary by Sarah Llanes and Zeke Hindle (Fall, 2011)
The ultimatum game consists of two players who are randomly selected to participate as
either a proposer or a responder. A proposer offers a percentage of M (total money in the split
between the two players), while the responder decides whether or not to accept the offer. If the
offer is accepted then the players split the money evenly; if not then no one receives anything. A
homo economicus is a selfish being trying to maximize his or her own benefits by offering the
smallest possible percentage of a certain amount to her opponent, as well as accepting anything
greater than nothing (aka the selfish rational solution). However plausible this may seem, recent
studies show that humans do not actually act in this way. In fact, humans are more likely to offer
at least half of the amount of money possessed, with responders rejecting any offers below 30%.
A substantial amount of previous research provides many explanations for this example of strong
reciprocity, particularly towards non-kin and those who we may never interact with again. One
such explanation is the influence of learning, or plasticity. Beneficial traits that are acquired
throughout a person’s lifetime have the ability to be passed on to future generations; it has been
demonstrated that natural selection is what gives people the ability to acquire these beneficial
traits, not the traits themselves. The present study revamped a model that supported the idea that
strong reciprocity can evolve from selection acting on the individual and not the group as
(Sanchez and Cuesta, 2005).
“As the population becomes more plastic the diversity of behaviors increases, and
evolution takes the population toward more altruistic behaviors.” (11) After a fixed amount of
games have been played, two agents are randomly selected to compete for reproduction. The one
with the highest fitness (most money) wins, while the other is discarded and replaced with the
offspring of the winner. Once a new generation begins, all agents begin with the same zero
fitness (a clean slate), until the next reproduction competition. The benefit of this “clean slate”
allows those players who didn’t do particularly well in previous rounds to learn from them and
thus have a change to do better in subsequent games. Although agents were allowed to adapt in
their lifetime (learn from their past mistakes within the current generation), they were unable to
pass any beneficial traits to their offspring. This is the most important characteristic of this
model: the fact that agents had no memory or repeated interactions with other agents truly tested
whether or not strong reciprocity was evolving through exclusive individual selection.
Agents change every time they are involved in a game, adapting their behavior in the
direction that would help them to perform better if they were to meet the same partner over and
over again. Three types of plasticity were explored:
1) Phenotypes free to change as much as they want from their genome; plastic throughout
their lifetimes; change after each game. Offspring inherit genome, not phenotype.
2) Behavior only allowed to adapt within a certain range (limited to how much it can
change)
3) Agents allowed to adapt a certain amount of times, after which the agent then becomes
rigid; offspring inherits original genes, able to adapt while parent stays rigid
It is important to note that phenotypes are exact replicas of their genomes, yet phenotypes change
throughout time. It is the genome that offspring inherit from their agent, not the phenotype.
Results demonstrated that rigid agents not only do in fact converge to the rational
solution predicted by game theory, but also that a population initially diverse in behaviors will
eventually converge to selfish behaviors. This solution is not entirely stable, though, for the
offers and acceptances fluctuate throughout each generation (or the evolutionary process).
Results also showed that the behavior of those agents who were plastic throughout their lifetimes
no longer converged to the rational solution--learned behavior differed from what was innate in
them. For example, they learned to accept (want) more than what they would initially be content
with. This demonstrates that plasticity does indeed cause the spread of learnt behaviors.
Furthermore, a positive interaction was found between adaptability and the average offers and
acceptance thresholds: as adaptability for agents increases, so does the average offers and
acceptance thresholds of the population.
The present study concludes that plasticity is in fact beneficial in the evolution of strong
reciprocity. Not only does it increase the diversity of behaviors within a population, but it is also
evolutionary stable (does not disappear if genetically determined).
Outline for Article
I.
Definitions of terms for this specific article and experiment format
A. Phenotypic plasticity, lifetime adaptations, ontogenic adaptations, learning
1. The aforementioned terms are defined as “changes that an individual
undergoes within his/her lifetime and that will not going to be [sic]
inherited directly by the offspring”.
B. Strong Reciprocity
1. The evolved behavior of humans to “cooperate with non-kin, including
people that they will never meet again, and punish non-cooperators even
at a personal cost”.
C. Ultimatum Game
1. Two players are randomly selected to participate on being the proposer
and one being the responder, the participants do not know each other,
and only play with each other once.
2. Proposer offers a percentage of M (total money in the split between the
two players).
3. Responder decides whether or not to accept the offer.
4. If the offer is accepted then the players split the money evenly, if not
then no one receives anything.
D. Homo Economicus
1. “A self-interested being who will try to maximize his or her own
monetary utility”.
2. offer the smallest percentage of M and will accept anything above the
zero value
II.
III.
3. However, many studies have shown that humans do not act in the
aforementioned manner
Introduction
A. Previous Literature Findings
1. Studies (Bethwaite and Tompkinson, 1996; Fehr and Schmidt, 1999)
show that strong reciprocity is found in the ultimatum game and
“preferences [potentially are] based on a utility function other than
maximizing payoff”.
2. The Sanchez and Cuesta’s (2005) agent-based evolutionary game theory
model predicts the emergence of strong reciprocity from selection acting
on the individual and not the group, in addition the appearance of
altruistic behaviors is found regardless of repeated interactions,
reputation, or personal gains.
Model
A. The model formed by the authors/experiments was similar to the model
implemented by Sanchez and Cuesta (2005), however the agents and genetic
algorithim was altered to allow for plasticity.
B. Rigid Agents
1. (Go) - Indicates amount of money affered by proposer.
2. (Gr) - Indicates amount of money accepted by responder.
3. [0, M] – Range of integer values of money.
4. Genes are randomly selected forcing the behavior of the agent, hence the
term “Rigid Agent”.
C. Genetic Alogorithim
1. Two agents randomly selected, and the one with the higher fitness (more
relative money) is replicated (offspring) while the other (less relative
money) is discarded and leaves the game.
2. After each reproduction, “the fitness of all agents is reset to 0”, and then
players continue to play the ultimatum game as defined by the above
rules.
D. Plastic Agents
1. These agents are allowed the ability to adapt during their lifetimes
“without directly passing this adaptation on to their offspring”.
2. The offspring only inherit the genome and not the phenome.
3. “Agents change every time they are involved in a game, adapting their
behavior in the direction that would help them to perform better if they
were to meet the same partner over and over again”.
a. Type 1 Plasticity
1. Agents may freely adapt during their lifetimes.
b. Type 2 Plasticity
1. Agents only allowed to adapt within a certain range.
c. Type 3 Plasticity
1. Agent is allowed to adapt a given number of times [An] and
then becomes rigid.
2. Offspring follows same limitations.
d. Inherited Plasticity
IV.
V.
Results
A. Rigid Agents
1. Results converge “to the rational solution predicted by game theory”
meaning that the agents proposed very little and willing to accept almost
anything.
B. Plastic Agents
1. Type 1 Plasticity
a. Plasticity causes the spread of learnt behaviors.
2. Influence of Plasticity Types 2 and 3
a. As adaptablity of the agents increases (Am) the average offers and
acceptance thresholds increase.
b. As number of games before a given agent becomes rigid (An)
increases, the offers made are more generous and thresholds less
strict.
3. Plasticity as an Evolutionarily Stable Strategy
Discussion and Conclusion
A. Author Notes
1. Strong Reciprocity can evolve from selection acting on the individual
and not the group as previously thought before the research of Sanchez
and Cuesta (2005); reputations are irrelavent
2. Learning can be both beneficial or detraimental to the individual and
offspring and could entail an evolutionary trade-off.
3. Costs of plasticity
a. Stochasticity
b. Time-wasting cost due to the learning process.
4. Benefits of plasticity
a. Takes place within the lifetime of agent.
b. Might be evolutionarily stable.
c. Increases diversity of behaviors and genetic diversity, hence the
level of cooperation amongst group members of the population.
d. Changes overall (average) behavior of population from selfish to
altruistic in nature.
5. Conclusion
1. “Plasticity is evolutionarily stable as (depending on the
mechanism by which it is passed on to offspring) it does not
disappear from the population if it is genetically determined”.
Test Questions
1) Sanchez and Cuesta’s Model of evolutionary game theory (2005) predicts the emergence of
strong reciprocity based upon selection acting on:
a) the individual
b) the group
c) both a and b
2) Over time plasticity will evolve behaviors that can be described as:
a) selfish
b) altruistic
c) destructive
d) both a and b
3) By definition, strong reciprocity takes into account all of the following except:
a) cooperation with kin
b) cooperation with non-kin
c) cooperation with people that will never interact again
d) cooperation even at personal cost
4) Plasticity can be defined as changes that an individual undergoes within his/her lifetime that
will be directly inherited by the offspring of said individual. T or F
5) Altruistic punishment is positively correlated with the evolution of plasticity and strong
reciprocity. T or F
6) The evolutionary costs of plasticity are higher than the benefits of plasticity that’s why we’re
are all selfish and mean people. T or F
Answer Key
A
B
B
T
T
F