Digitized by the Internet Archive
in
2011 with funding from
Boston Library Consortium Member Libraries
http://www.archive.org/details/basinsofattractiOOelli
a
DEWEY
HB31
.M415
I
working paper
department
of economics
BASINS OF ATTRACTION, LONG RUN EQUILIBRIA,
AND THE SPEED OF STEP-BY-STEP EVOLUTION
Glenn Ellison
96-4
Aug. 1995
massachusetts
institute of
technology
50 memorial drive
Cambridge, mass. 02139
J';
BASINS OF ATTRACTION, LONG RUN EQUILIBRIA,
AND THE SPEED OF STEP-BY-STEP EVOLUTION
Glenn Ellison
96-4
Aug. 1995
MASSACHUSl
MAR I 3
^ 96
library
Basins of Attraction, Long
Run
Equilibria,
and the
Speed of Step-by-Step Evolution
Glenn Ellison 1
Department of Economics
Massachusetts Institute of Technology
August 1995
4 would
like to
thank Sara Fisher Ellison, Drew Fudenberg, David Levine, Eric Maskin, Roger
Myerson, Georg Noldeke, Tomas Sjostrom and Peter Sorensen
for their
comments. Financial support
was provided by National Science Foundation grants SBR-9310009 and SBR-9496301.
Abstract
The paper
provides a general analysis of the types of models with c-perturbations which
have been used recently to discuss the evolution of social conventions.
of the size and structure of the basins of attraction of
measures
dynamic systems, the radius and
bound the speed with which evolution occurs. The
coradius, are introduced in order to
main theorem
Two new
uses these measures to provide a characterization useful for determining
long run equilibria and rates of convergence. Evolutionary forces are most powerful
evolution
may
proceed via small steps through a series of intermediate steady states.
number of applications
games
is
when
are discussed.
The
generalized to the selection of
selection of the risk
—dominant
dominant equilibrium
equilibria in arbitrary games.
in 2
A
x2
Other
applications involve two-dimensional local interaction and cycles as long run equilibria.
Keywords: Learning, long run equilibrium, equilibrium selection, waiting times, radius,
coradius, risk dominance, local interaction,
JEL
Classification No.:
Markov
C7
Glenn Ellison
Department of Economics
Massachusetts Institute of Technology
77 Massachusetts Avenue
Cambridge,
MA 02139
,
process.
Introduction
1
Multiple equilibria are
common
in
economic models, and when they are present, both aggre-
gate welfare and the distribution of payoffs often varies dramatically with the equilibrium
on which players coordinate. Beginning with the work of Foster and Young (1990), Kandori,
Mailath, and
Rob (KMR)
(1993), and
Young (1993a), a
large
number
of recent papers have
explored the extent to which evolutionary arguments can provide a rationale for regarding
some
equilibria as being
more
likely
literature in
two ways.
run and the
medium run behavior
than others. This paper attempts to contribute to
First, it develops
a general technique
of such models, providing intuition for
of previous analyses are what they are, and allowing for
behavior. Second,
interest.
it
discusses a
number
is
a crucial determinant of whether evolutionary change
possible for change to be effected gradually via
KMR,
the results
interest outside the
will
it is
occur rapidly
a number of small
and Young (1993a) develop
lit-
a discussion of when evolutionary
produce rapid change (and hence are practically relevant), in which
of
why
more thorough descriptions of
Most prominent among these (and hopefully of broader
The papers
both the long
of specific topics which should be of independent
erature on the development of conventions in games)
forces
for analyzing
this
explicit
is
argued that
whether
it
is
steps.
models of processes by which
conventions might be established as large populations of boundedly rational players interact
and adjust
their play over time.
Each makes the assumption that there
is
some source of
persistent noise in the population, e.g. player turnover, trembles, etc., with the central ob-
amounts of noise may dramatically
servation being that small
the system. While any behavior
become much more
likely
is
in theory possible
alter the long
once noise
is
run behavior of
introduced, some actions
than others, which suggests the definition of "long run equilibria"
as states which continue to be played in the long run with nonvanishing probability as the
amount of
noise tends to zero.
The most
striking result
which these papers obtain
is
that
such an analysis not only rules out unstable mixed equilibria, but usually provides an argu-
ment
for selecting
an unique equilibrium even
in
games with multiple
strict
Nash
equilibria.
For the case of 2 x 2 games, these papers find that for a variety of dynamics the unique
long run equilibrium involves the population coordinating on the equilibrium Harsanyi and
Selten (1988) term risk-dominant.
this conclusion
is
affected
A
number of more
recent papers have explored
how
by the assumptions about the nature of the social interactions
1
and learning
and what similar analyses have to say about bargaining,
rules,
signalling,
and
1
other classes of games with multiple equilibria of inherent interest to economists.
The
part of this paper attempts to develop a
first
new framework
for analyzing
models
In previous papers, the standard approach has been to rely on a
with persistent noise.
tree construction algorithm developed
by
KMR, Young
(1993a), and Kandori and
Rob
(1992) (building on the work of Freidlin and Wentzell (1984)) to identify the long run
This method of analysis has a number of significant limitations which provide
equilibria.
The
the motivation for this paper.
analysis
provides
it
is
first
obvious limitation of the technique
limited in scope in addressing only long run behavior
anything about how rapidly the long run equilibrium
is
reached.
2
As
is
that the
and not saying
Ellison (1993) argues,
convergence times for models in this literature are at times so incredibly long that long
may have
run behavior
relevant time horizon.
very
to
little
do with what what happens within any economically
Another drawback
in theory universally applicable,
it
is
that while the tree construction algorithm
has in practice proven
difficult
is
to apply to complicated
models and when developing generalizations. Finally, the nontransparency of the algorithm
has
make
the whole literature seem a bit mysterious
that the typical paper writes
down a model,
the feeling
is
says something about trees and after several
pages of calculations gives an answer but provides
is
— a common frustration
little
understanding of how the answer
connected to the assumptions of the model.
The framework developed
in this
paper provides a method for analyzing both the long
run and the medium run behavior of models with small persistent noise. While the method
is
not universally applicable,
and may
facilitate
it
provides an intuitive understanding of the literature to date
both more thorough analyses and analyses of more general and complex
models in the future.
One can
First,
it is
think of the main theorem of the paper as incorporating three main insights.
noted that virtually
'Sec, for example, Bergin
all
of the models in the literature share a
and Lipman (1993), Binmore and Samnelson(1993),
common
structure
Elliaon (1993),
Kandori
and Rob (1992, 1993), Noldeke and Samnelson (1993, 1994), Robson and Vega Redondo (1994), and Young
(1993b).
2
The papers
of Ellison (1993),
Binmore and Samuelson (1993), and Robson and Vega Redondo (1994)
are notable exceptions to the tendency to rely excusively of the Freidlin- Wentzell construction
ignore convergence rates.
and thereby
and thus can be addressed within the context of a simple
(albeit abstract) model.
Next,
the paper points out a relationship between the long run behavior of models with noise
and the speed with which evolutionary change occurs which
clear intuition for
is
simply that
if
why
the long run equilibrium of a
a model contains a state or set of states
in the sense that play tends to
is
model
remain
in
system begins
most of the time. In
in
any other
ft
which
Q.
for
it is.
is
The observation
both very persistent
a long time whenver
ft
be reached relatively quickly
ft will
state, then in the long
light of this observation,
what
is
a neighborhood of
reached, and sufficiently attractive in the sense that
after the
at least occasionally provides
run states near
ft will
occur
an understanding of what makes evolution
evolution fast or slow will provide also an algorithm for identifying long run equilibria.
The
final crucial observation,
which should be of interest to the broadest audience,
concerns the speed with which evolutionary change occurs Specifically,
critical
may
determinant of whether evolutionary change
will
occur rapidly
argued that a
whether the process
proceed by a (possibly very long) series of gradual changes from one nearly stable state
more dramatic changes are
to another, or whether
think about
why
required.
A
evolve into a bat.
If
one
the process of growing a wing required ten distinct
independent genetic mutations and a creature with anything
viable,
biological analogy helps
evolutionary mechanisms of the former type are more plausible. Think of
how a mouse might
all
is
it is
we would have
less
to wait a very, very, long time until one
ten mutations simultaneously.
If
than a
full
wing was not
mouse happened
to have
instead a creature with only one mutation was able
to survive equally (or had an advantage, say, because a flap of skin on
its
arms helped
it
keep cool), and a second mutation at any subsequent date produced another viable species,
and so on, then evolution might take place
in
a reasonable period of time.
paper focusses on the development of conventions in games,
on the speed of evolution which
to assess arguments in
this
paper provides
I
While
this
hope that the observations
may be of use more generally
in helping
a variety of areas of economics where "evolutionary" changes are
discussed.
The main theorem
ization of the long run
focusses on two
of the paper combines
and formalizes these observations a character-
and medium run behavior of models with
new measures,
referred to as the radius
scribe the size of the basin of attraction of a limit set
noise.
The
formalization
and modified coradius, which
and
its
de-
relation to the basins of
— and which can be used to provide bounds on the speed
attraction of the other limit sets
with which evolution occurs.
The
radius measure, denoted by #(ft),
obtained by simply
is
counting the number of "mutations" needed to escape the basin of attraction.
an upper bound on the speed with which evolution away from
notion of persistence.
The modified coradius measure,
on the speed with which evolution toward
ft will
as to capture the fact that
more complicated so
be reached with few mutations and
if it is
occur.
ft will
ft
Cft*(ft), provides a lower
The measure
arise
itself is
it
number
via a
the paper shows that
ft is
is
also
if it
can
of smaller
a state which
is
occur most of the time, the main theorem of
the unique long run equilibrium of a model
In this case the modified coradius
bound
somewhat
more quickly both
possible to reach
will
provides
can occur, capturing the
transitions between intermediate limit sets. Formalizing the intuition that
both persistent and sufficiently attractive
It
> CR'(Q).
if iZ(ft)
shown to provide a bound on how rapidly
ft will
be reached.
One can
theorem as making a number of contributions to the
see this
the most obvious benefit
is
that
is
provides a tool with which one can analyze
Second, what
as well as long run behavior.
benefit
it
may be
will
is
is
unexpected
allow virtually
all
going on in the literature.
probably not be surprised to learn that long run equilibria can in some
cases be identified by looking at simple measures of persistence
think
medium run
the most important and unexpected
the intuition which the theorem provides for what
While the reader
literature. First,
is
that the
main theorem of
this
paper
is
and attractiveness, what
I
sufficiently powerful so as to
of identifications of long run equilibria which have been given in the past
to be rederived as Corollaries (albeit sometimes with a great deal of work).
of this broad applicability
literature, all that is
is
An
implication
that behind most of the mysterious tree constructions in the
happening
is
that there
is
a nearly stable state (or a
which requires a large number of mutations to escape (and hence
is
set of
persistent)
such states)
and to which
the system returns relatively quickly after starting at any other point (either because
it
can
be reached with few mutations or by a sequence of smaller steps through intermediate limit
sets).
Finally,
I
hope that the theorem
will
identify the long run equilibria of complex
and general models. The examples and
which are presented are intended in part to
most
usefully applied.
prove valuable also in helping researchers to
illustrate
corollaries
where (and how) the theorem may be
Following the main theorem, the paper explores a number of different topics which
should be of independent interest (at least to those
development of conventions
games.) The
in
first
who have
of these involves generalizing the famous
result that risk-dominant equilibria are selected in 2
reviews negative results (providing a
selected equilibrium
presents the
new
x 2 games. The discussion here
new example which
the long run equilibrium concept by showing
how
is
first
nonrobustness of
illustrates the
outside the class of 2 x 2 games the
may depend on which Darwinian
positive result which
followed the literature on the
behavior rule
the main item of interest.
is
chosen), and then
The
result
is
that in
any game for which an equilibrium satisfying Morris, Rob and Shin's (1995) refinement of
dominance, ^-dominance,
risk
that this selection
many
is
exists, that equilibrium
quite robust in that
local interaction settings.
the main theorem
Two
may
model, and
holds both for any Darwinian dynamic and in
it
trivial
KMR
proof of this result serves also to illustrate
how
aid in the derivation of general results.
additional learning in
cal interaction
The
selected by the
is
games
topics
which are discussed at some length are
lo-
models and the extent to which models of evolution with noise should be
thought of as a method of equilibrium selection. The primary result on local interaction
is
that ^-dominant equilibria are selected also in a two-dimensional local interaction model,
with convergence to the long run equilibrium being relatively
fast.
This latter result
is
noteworthy because the model lacks the powerful "contagion'' dynamics which one might
have thought drove the
model serves also to
most
likely to
fast evolution of
clarify
my
earlier
one dimensional
comment
and the
The
that the techniques developed here are
prove useful in analyzing complex models
to a single limit set (and paths to it)
local interaction models.
ability to
— the
ability to limit one's focus
account for the impact of transitions
through intermediate limit sets are each most valuable when considering models for which
the unperturbed dynamics feature a large
The main comment
select
and
cycles.
is
may be something
illustrated via a simple example,
is
that the term long
of a misnomer in that an evolutionary model can easily
a cycle or other behavior even in games with an unique Nash equilibrium.
The
The
states
of this paper on the equilibrium selection interpretation of evolu-
tionary models with noise, which
run equilibrium
number of steady
final
first
two sections of the paper are concerned with
contains a discussion of the applicability of the
its
relationship to the literature.
main theorem which attempts to
catalog both the extent to which the results of previous papers could have been derived
main theorem
as applications of the
rates)
which case
(in
it
provides intuition and convergence
and the extent to which the main theorem would have allowed
much more
Using the former criterion the theorem
easily.
latter situation
is
common. The
less
is
results to be derived
very widely applicable; the
contains an alternate proof of the long run
final section
equilibrium part of the main theorem based on a Freidlin- Wentzell "tree surgery" argument.
In light of this
argument one can interpret
this part of the
theorem also as
illustrating
how
a systematic tree surgery argument can be applied to models in which transitions through
This section
intermediate limit sets are important.
the techniques of
Young
(1993a), Kandori and
Rob
will clarify
how
this
paper builds on
(1992), Ellison (1993), Evans (1993),
Samuelson (1994), and others, and suggests how "tree surgery" arguments might be applied
in other contexts.
2
Model
One
of the primary goals of this paper
to provide an improved understanding of the
is
behavior of evolutionary models with noise. As a
that
if
one abstracts away from the
the literature can be
fit
into a
details
it
first
becomes
framework which
is
step toward this goal,
I
clear that virtually all of the
note here
models
in
remarkably simple. Because the economic
motivation for this abstract model would otherwise be a total mystery to anyone not familiar
with the literature,
I
begin this section with a concrete example and then describe the
general framework within which the
2.1
An example
main
results of the
paper
will
be formulated.
— a model of the evolutions of conventions
The primary motivation
for the recent literature
suggestion of Foster and
Young
(1990),
KMR
in
games
on models of evolution with noise
is
the
and Young (1993a) that we might better
understand the behavior of players in games (especially those with multiple equilibria) by
explicitly
as
modeling the process by which "social norms" or "conventions" might develop
boundedly rational players adjust to their environment over time.
present one such example which
framework discussed
Let
G
later
be a symmetric
is
is
meant to
illustrate the
In this section
I
type of model which the abstract
intended to encompass.
mxm game.
We
will
6
suppose that conventions for how to play
in
this
game develop within a
to play
G
in periods
t
write
Z
t
0, 1,2,
who
period not knowing
at time
=
his
states of the system.
It
.
.
.
•
•
i
players as they are repeatedly
We
will be.
all
can describe the play of the population
such strategy
profiles,
and
a
1
elements of
refer to the
simplifies notation at times to pick
x
mN
We
each of the players.
5 /v) giving the strategies of
states so that each can be represented also by
place and zero in
randomly matched
with each player choosing a single action to use in each
,
opponent
by a vector (si,$2>
for the set of all
N
society of
Z
will
as the possible
an (arbitrary) ordering of the
vector, zt , equal to
one
in a given
the rest.
Rather than suppose that the players choose their actions rationally, one might imagine
some simple boundedly
instead that the players react to their environment by following
One
rational rule.
particular
example which has been extensively studied
response dynamic" under which each player at time
used at time
-
t
1
by his potential opponents.
t
is
the "best-
plays a best response to the strategies
We
can incorporate this or any other
deterministic behavior rule in which players react only to actions in the previous period by
specifying that changes in play over time are described by the appropriate "deterministic
evolutionary dynamic'', a function b
When
one
is
be made that
trying to
Z
—
Z
=
such that zt +i
model the behavior of a large
society,
b(zt).
an argument can certainly
not plausible to assume that the specified boundedly rational learning
it is
rule will describe everyone's play.
will
:
More reasonably, we might assume instead
that there
always be noise on top of this due to trembles in action choices, players reacting to
idiosyncratic payoff shocks, the replacement of old players with
the prevailing norms, and other factors.
and Young (1993a)
will typically
is
amount of noise
we can more
is
which
fixed e
is
we know
initial
little
conditions (which
we
about), this dependence
introduced into the model. As a result, we
Canning (1992),
KMR,
usually think of as a result
is
may
eliminated
if
a sufficient
obtain a model in which
confidently predict the long run course of play.
The most common
some
crucial observation of
players unfamiliar with
that while long run behavior an evolutionary model without noise
be dependent upon
of historical accidents
The
new
€
(0,
specification of noise in the previous literature
^), the evolution of play over time
defined by composing the deterministic
under which each player's strategy at time
t
+
is
dynamic
1 is left
is
to suppose that for
described by a stochastic process
b
with a random mutation process
unaltered with probability
1
- me,
and
drawn independently from a
is
distribution placing equal probability on each of the
To describe such a model mathematically, note
possible strategies with probability me.
may be
that given our notation with z t as a vector, the deterministic dynamics
=
as a linear transformation z t +i
the tj tn place
if b(i)
—
is
described by a
which has a one
1
x
mN
The
is
vector v t whose
in that place.
We
with c(i,j) the number of players
is
t
tn
matrix which has a one
from
t
to j.
c(i,j)
=
2.2
The general model
=
d(b(i),j) with
discuss the
away from many
=
e
c(,,i)
who
(l
- (m - ljc)*-*^,
play differently in the states j and
I will later
It is
more abstract model
small (with high values
is
use the fact that this cost function
in the context of
A
i.e.
nice side benefit of this generality
first
state space of the
descriptors of
is
clarify the structure of these
d(x,y)
of the form
+ d(y, z).
which the general characterizations
The model
abstracts
a framework which
that the framework becomes quite simple
models.
models of learning and evolution with noise can be seen
main elements of the general model defined
as combining various specifications of three
The
is
encompass most of the models which have previously been
Essentially, (most) previous
below.
<
d(x,z)
details of the evolutionary process in order to obtain
and therefore helps
Note that
b(i).
standard to refer to c(i,j) as the
d satisfying the triangle inequality,
sufficiently general so as to
studied.
a Markov
v t P(e),
of the behavior of evolutionary models of noise will be developed.
is
is
in
have
a measure of the relative likelihood of transitions when €
cost of a transition
now
mN
element gives the probability of the state zt
of c(i,j) corresponding to very unlikely transitions).
I
x
the probability transition matrix given by
P.j(e)
c(i,j)
mN
probability distribution over the possible states at time
vt+ i
where P(e)
an
and a zero there otherwise. The stochastic model
j
process on the state space Z.
t
P
z t P, with
represented
main element of the model
is
a
finite set
Z
which
I
will refer to as the
model. In the typical model the state space consists of a set of possible
how a game
is
being played. For example, a state z €
8
Z
might be the current
summary
strategy profile of the population, a
such as the number of players using
statistic
each strategy, or a more detailed description of play in which the current strategy profile
is
augmented with
on past play or on who was matched with whom.
historical information
The second main element
model
of the
space Z. In the typical model
P
is
P
a Markov transition matrix
on the state
describes the implications of a particular specification of
the process by which boundedly rational players adjust their play over time. For example,
besides the best-response dynamics
decision rules where players
make
could be used to capture a variety of more complex
it
use of
more information
which are stochastic because players adjust their play
or rely on information the acquisition of which
matching process. Note that the one
based. For example,
if
players are
observations, the state should incorporate
Finally, the third
we
In particular,
are given a
outcome of the random
P
that
is
is
Markov, so that
on which players decisions are to be
to play a best response to their
what each player saw
main element of the model
random perturbations.
here
critical restriction
assumed
historical data) or
each period with some probability
in
affected by the
in the state space all information
one must include
interval [0,«),
is
example
(for
I will
Markov process
is
most recent k
in the previous k periods.
a specification of some type of small
assume that
t
belonging to some
Z
and with transition
each
for
{z\} with state space
probabilities
Prob{zt<+1
such that
in
e
(i) for
with P(0)
that for
all
each
=
>
P; and
<
we
provide of
Pxx =
oo (with
i
(iii)
mutations.
=
z}
=
P„,(e)
is
ergodic;
there exists a cost function c
6 Z, lim ( _o Pzz'i.t)!^*'2
*
for sufficiently small e if c(z, z
medium run and
exists
1
)
=
P(e)
is
continuous
x
Z
—* TZ + U oo such
and
is
strictly positive if
The
characterizations
Z
:
(ii)
oo).
long run behavior depend on the specification of the
random perturbations only through the
cost function as
z'\z\
the transition matrix P(()
pairs of states z,z!
0(2,^)
will
e
=
cost function, so
we
will
tend to think of the
a primitive of the model concerning the relative likelihood of the possible
For example, instead of simply choosing the cost of a transition to be the
number of independent random trembles necessary
for
it
to occur, a cost function could
be chosen so as to incorporate state-dependent mutation rates with unbounded likelihood
ratios (as in Bergin
and Lipman (1993)), correlated mutations which occur among groups
of players, or the impossibility of
some mutations
(in
which case the cost
is
set to infinity).
medium run behavior
Descriptions of long run and
2.3
In this paper long run behavior will be described in the
any fixed
in the literature. Specifically, for
e
>
0,
the
manner which has become standard
Markov process corresponding
model with £-noise has an unique invariant distribution
(given by
/x'
=
/*'
to the
limj_oo vo-P'(e))
which we think of as describing the expected likelihood of observing each of the states
given that evolution has been going on for a long, long time. While
which
is
of most interest, //
is difficult
to compute,
and hence
it
has become standard in
=
the literature to focus instead on the limit distribution p' defined by n"
motivation for this
when
that n*
is
is
easier to
lime _o A4 '- 3 The
compute and provides an approximation
small. Also following the literature
( is
this distribution
it is
we
will call
to
jj.'-
a state z a long run equilibrium
if/z*(*)>0. 4
The most
basic result
on what long run equilibria look
like
(found in Young (1993a))
is
that the set of long run equilibria of the model will be contained in the recurrent classes of
s
(Z,P(0)).
The
recurrent classes are often referred to as limit sets, because they indicate
the sets of population configurations which can persist in the long run absent mutations.
They may take the form of steady
between which the system
consisting of the state
A
shifts
in
states, of deterministic cycles, or of
a collection of states
randomly. The prototypical example
which
all
players play
A
in a
model where
is
the singleton set
G
has {A, A) as a
symmetric Nash equilibrium, and where players do not switch away from a best response
unless a mutation occurs.
While
common
it is
because the
this is not
in the literature to
medium run
examine only the long run behavior of models,
As
not an important consideration.
is
Ellison (1993a)
and others have noted, a description of the implications of evolution which includes only
an identification of the long run equilibria may be misleading in that what happens in the
"long run"
3
if
often very different from
The asaumpttoas on the nature
does in fact
exist.
To see
what we would expect to observe
of the perturbations
this note that for
we have made
any two states z and *
in, say,
the
first
ate sufficient to ensure that the limit
Lemma
3.1 of
Chapter 6 of Freidlin and
«
Wentzell (1984) expresses the quantity
-fat ratio of polynomials in the transition
the transition probabilities themselves are asymptotically proportional to powers of
will
*
5
have a limit as
t
—•
there exists
s
>
fl
C Z
t,
Because
each of these ratios
0.
Such states are also at times referred to as being stochastically
Recall that
probabilities.
is
a recurrent class
such that Prob{z?+>
=
if
Vw €
ui'|z?
0, Prob{z?+1 €
= w) >
10
0.
stable.
ft|z,°
= w} =
1,
and
if
for
all
w, w' 6
Q
Many
trillion periods.
because this
authors have nonetheless focussed on the long run, no doubt in part
what the standard techniques allow one
is
to
do most
easily.
The main theorem
of this paper provides a general characterization of the rate at which systems converge, with
my hope
being that this will induce future researchers to provide more complete analyses.
In particular, writing
first
W(x,Y, e)
for the
expected wait until a state belonging to the set
reached given that play in the e-perturbed model begins in state x, and supposing
the set of long run equilibria of the model, this paper uses
of
Y
how
quickly the system converges to
convergence
is
fast
and we
will
regard
ft
its
long run limit.
max r g£ W(x, ft, f)
If this
maximum
is
ft is
measure
as a
wait
is
small,
not just as the collection of states which continue to
occur frequently in the long run, but also as a good prediction for which behaviors we might
expect to see in the
medium
run. If the
maximum
wait
is
large,
one should be cautious
drawing conclusions from an analysis of long run equilibria. Note that in practice
waiting times will tend to infinity as
e
goes to zero.
being fast or slow in a given model will depend on
all
these
Whether we regard convergence
how
in
quickly the waiting times blow
as
up
as e goes to zero.
3
The
radius and coradius of basins of attraction
This section defines two new concepts, the radius and coradius of the basin of attraction of
a limit
set,
which are central to
models with
this paper's characterization of the behavior of evolutionary
noise. Essentially, the
main theorem of this paper provides a sufficient condition
long run equilibria by formalizing a very simple intuitive argument relating
for identifying
long run behavior and waiting times:
persistent once
it is
reached, and which
relatively quickly after play begins in
that states in
ft
if
a model has a collection of states
is
ft
which
very
is
sufficiently attractive in the sense of being reached
any other
state, then in the long
run we
will
observe
occur most of the time. The nontrivial work involved in developing this
observation into an useful characterization of behavior consists of developing measures of
the persistence of and of the tendency of sets to be reentered which are reasonably easy to
compute. The particular approach
I
take
is
develop measures which
reflect critical
of the size and structure of the basins of attraction of a model's limit sets.
as surprising
is
aspects
What
I
see
not that a characterization along these lines can be developed, but rather
that a fairly simple the characterization of this form turns out to be sufficiently powerful to
11
identify the long run equilibria of
I
most of the models which have been previously analyzed.
begin with a measure of persistence. Suppose (Z, P(0))
the model above, and
set of initial states
let ft
{z €
Z|Prob{3T
zt
s.t.
g
Write D(ft)
ft
with probability one,
t
be a
finite
=
main theorem
is
simply the number of
sequence of distinct states (z\, *2
< T, and zt i D(Q). Write
5(ft,
Z-
of cost can be extended to paths by setting c(z\ , z 2 ,
radius of the basin of attraction of
out of D(Q),
1}.
=
min
r T )€5(n,z-D(n))
we
define set-to-set cost functions
c(X,Y)=
=
win
zj) with
.
.
,
minimum
ft
1
c i zu *t+i )•
cost of
I
define the
any path from
ft
as the radius of the largest neighborhood
it
Note that the calculation of the radius of
ft
it.
all
c(zi,z2 ,...,zT ),
the basin of attraction of a limit set, the longer will
from
zt 6 Z?(ft) for
c(zi,z 2 ,. ..,zr).
from which the Markov process must converge back to
ft
ft,
such paths. The definition
all
= X^i
zj-)
6
z\
c(ft,Z-I>(ft)).
Geometrically, one can think of the radius of
of
,
out of
by
(n,... I* T )€S(-*,r)
then #(ft)
.
to be the
ft, i2(ft),
(zi
if
.
ft
i.e.
R(il)
Note that
. .
the set of
Z?(ft)) for
i.e.
z)
"mutations" necessary to leave a basin of attraction. Formally, define a path from
Z?(ft) to
for the
> T\z =
V*
ft
of persistence which appears in the
Markov process of
the base
its limit sets.
from which the Markov process converges to
D{Q) =
The measure
be a union of one or more of
is
full
dynamic system;
ft.
In practice, the least costly path from
it
typically suffices to
ft.
The
take for the
is
the radius of
Markov process
to escape
does not require that one explore the
examine the dynamics
ft
larger
out of
-D(fi) is often
neighborhood of
in a
a direct path (w,z 2 ).
In each of the examples presented in this paper, the cost function c(x, y) takes the form
c(x,y)
=
min{j|/>„(o)>o}d(z t y), with d satisfying the triangle inequality.
simple method for proving that A(ft)
with c(w,z2 )
=
minu/€fl c(w, z)
To
=
k
is
to exhibit states
k and to show both that min w€ nc(w, z) < k
< k
see that this
is
=>
minu,60.c(u»,z')
sufficient,
< min w€ o c(w,z)
one shows that the
showing inductively that there exist
12
for all z'
least costly
w 1 ,w2 ,...,u>t-\
6
ft
w € ft,
=> z €
path
is
In this case, a
and z2 $ D(il)
£>(ft),
with
and that
PZX '(0) >
0.
a direct path by
such that c(w, zi,z2 ,...,zt)
>
c(wi,22,--.,2t)
>
•••
> C(WT-1,ZT). 6
The second property
theorem
is
ft
which
will play
a
critical role in
the length of time necessary to reach the basin of attraction of
One simple way
state.
of a union of limit sets
to put a (not particularly tight)
bound on
ft
our main
from any other
this waiting time
is
count the "number of mutations" which are required. Formally, define a path from x to
to be a finite sequence of distinct states (zi,Z2,-
and zt 6
ft.
Write S(x,ft) for the
of attraction of
ft,
such paths.
= max
min
is
&
x, z t
ft
for all
t
< T,
define the coradius of the basin
c(z!,z2 ,... ,2j).
simplifies the calculation of the coradius to keep in
formula above
I
—
ft
CiZ(ft), by
Cft(ft)
It
set of all
,zt) with z\
to
mind
that the
always achieved at a state which belongs to a limit
coradius of the basin of attraction of
when
maximum
Intuitively, the
set.
states are, in a cost-of-paths sense,
ft is
small
it is
easy to see that the basin of attraction of
all
in the
close to that set.
When
the coradius of
be entered
is
fairly
small
soon after play starts in any other state. Given any
a nonnegligible probability of reaching D(Q) within some
this does not occur,
is
ft is
then the period
T
state
is
finite
ft will
initial state, there
number
T
of periods. If
again an inital condition from which there
a nonnegligible probability of reaching D(Q) by period 2T, and so on. The bound on
waiting times provided by the coradius, however,
is
often not very sharp.
the computation does not reflect the fact that evolution toward
by the presence of intermediate steady states along the path.
ft
may be
In particular,
greatly speeded
Defining a variant of the
coradius concept which takes this into account will allow us to achieve a tighter
bound on
return times, producing more widely applicable characterization of long run equilibria.
The
inductive step consists simply of noting
arg min,/ go e( »',*;),
c(w, ii, ij)
and
that
6
*tgmin{»<|p
=
c(w, zi)
+ c(n,z 7 =
>
c(v>,zi)+d(wi,Z2)-d(wl,z[)
=
c(w,*i)
>
d(wl,zt)>c(wi,zi).
v>l
for
,(o)>o} d(w\z[),
-i-
)
c(w,
n) +
+ d(w', zi)-c(wi,z[)
13
d(x[,zt)
ij
€ argmin{,|p,
,(<»>o} d(z, *a).
fi
€
The
we
variant
will use
based on a modified notion of the cost of a path which allocates
is
a portion of the total cost of getting to
ft
to the necessity of leaving the basin of attraction
of each limit set the path passes through.
to
C
Li,Li,...,L T
ft, let
Specifically, for (zi,Z2>- ••>-*r)
a path from x
be the sequence of limit sets through which the path passes
ft
consecutively, with the convention that a limit set can be appear on the
but not successively. Define a modified cost function,
c*,
list
multiple times,
by
r-l
c"{z l ,z 2 ,. ..,zt)
=
c(z u z 2 , ...,zT
)~Y, R(Li).
i=2
The
definition of modified cost can
m
c (x,ft)
be extended to a point-to-set concept by setting
=
min
(z\
The
*r)€S(x,n)
c'(z u z2 ,...,zT ).
modified coraditts of the basin of attraction of
ft is
then defined by
CR m (ft) = maxc M (x,ft).
Note that CR'(ft) < CR(ft).
The
definition of the modified coradius
steady states on expected waiting times.
is
To
compare the two simple Markov processes
meant to capture the
get
some
in the
effect of
intuition for this
it is
intermediate
instructive to
diagram below. In the diagram arrows
are used to indicate the possible transitions out of each state and their probabilities (with
remaining probabilities). In the Markov process on the
null transitions accounting for the
left,
both the cost and the modified cost of the path
until such
a transition occurs
Markov process on the
occurs
is
only 7
+
7
+
right
=
7
is still
x
A
The modified
cost of the path
three,
is
and the expected wait
some
€
•
- •
x
y
from x to
w
also one.
€
*•
w
biological analogy provides
is
three, the expected wait until a transition
3
e
•
w)
In contrast, while the cost of the path (i, y, z, w) in the
is jy.
7-
(x,
intuition for the result.
€
»>
•
z
»•
•
w
Think of the graphs
as
representing two different evironments in which three major genetic mutations are necessary
14
to produce the
more
fit
w
animal
mouse.) The graph of the
from animal
a situation in which an animal with any one or two
left reflects
of these mutations could not survive.
(growing an entire wing?)
mutation on
its
own
In this case, getting the large mutation
provides an increase in fitness which allows that mutation to take over
itotv seems
change from
plausible
when such gradual change
is
From
the comparison of the two
Markov processes above,
mediate limit
in the
modified coradius definition
is
relatively
more
possible.
Why
can speed evolution.
sets
we need
a very unlikely event. In the process on the right, each single
is
Clearly, the large cumulative
the population.
(For example, to produce a bat from a
x.
it
should be clear that inter-
the particular correction for this embodied
approprate
is
much
less obvious.
While Theorem
1
shows formally that the modified coradius can be used to bound return times, an example
may
be more useful in presenting the basic intuition. In the three state Markov process
shown below, suppose that c(x,y) >
with R{il)
=
oo
and CR'(Q)
=
ry
c(x,y)
and c(y,il) > r y so that
+
-
c{y,tt)
ry .
fi is
the long run equilibria
To compute W(x,fi,t) (which
will
turn out to be the longest expected wait) note that
W(x,Q,e) =
with
Nx and Ny
being the number of times x and y occur before
the system starting in state x.
of the run of z's before an i
x
—
E(Nx ) + E(Ny ),
—
We
can write
E(NX )
y transition occurs
Q
is
reached conditional on
as the product of the expected length
and the number of times the transition
y occurs. Hence,
r,>
r^l-r +—
,
E(NX ) =
(l
+
.
/
c( y ,n)\
€
J
„
Similarly,
we can
number of x -» y
negligible
c
-(«K«.»)+e(v.n)-i-»).
write
E(Ny )
as the product of the length of each run of y's
transitions. This gives
compared to
E(NX )-
E(Ny ) ~
is
*(
exactly € _CA n ).
»),
which
is
and the
asymptotically
Hence we have
W(x,n,€) ~
which
f~(
r » +c (i'' n ) -r
e
-M*.v)+c(v,n)-r,)
j
Looking carefully at
this calculation,
one can see that the
subtraction of the radius of the intermediate limit set reflects the fact that because most of
15
the time the system
in state x the waiting
is
time
is
and the probability of a transition from y to
leave i
essentially
ft
determined by the wait to
conditional on a transition out of y
While the unconditional probability of the transition
occurring.
probability
is t
c
c
is e ("
n ), the conditional
^ ,n )- r v.
e'v
.
*.•
V
ft
The main theorem
4
We
are
now
in
a position to present the theorem which contains the main general
results of
medium run
behavior
The theorem
this paper.
models with noise (using the radius and modified coradius measures).
in evolutionary
hope that
this
theorem
medium run and
the
provides a description of both long run and
seen as valuable in providing both a general tool for analyzing
is
intuition for
begin with the theorem and
some of
its
Theorem
limitations via a
run
(a) All of the long
(b)
its
why
and then
proof,
number
the results of previous papers are what they are.
For any y #
Note that as a
SI,
illustrate the use of the
equilibria of the
if iZ(ft)
>
theorem and discuss
Cfl'(ft), then
system are contained in
we have W(y,il,e) = 0(€~ CR '^)
corollary, the
I
of simple examples.
In the model described above
1
I
same
results hold if R(il)
ft.
as € -* 0.
>
Cft(ft).
Proof
Write
€- perturbed
the
all
/*'(*) for
y
£
ft.
A
the probability assigned to state z in the steady-state distribution of
model. For part (a)
it
suffices to
show that
e
/j'(y)/jj (ft) -»
standard characterization of the steady-state distribution
E #
H*(y)
(
/j
E
(ft)
#
of times y occurs before
of times
ft
il is
occurs before y
is
is
r
See
e.g.
RHS
is
Theorem
at
reached starting at y
reached starting in
most W(y,il,e), while the denominator
6.2.3 of
Kemeny and
Snell (1960).
16
for
that
with the expectation in the denominator being also over starting points in
ator of the
as € ->
is
fi'
ft.
7
bounded below
The numeras €
—
by
W(w,Z -
a nonvanishing constant times min^gn
spent in D(Cl)
is
spent in
D(il),e) (because almost
Hence to prove both part
fi.)
(a)
and part
all
(b) of the
of the time
theorem
it
show that
suffices to
W(w,Z-
Z?(fi), e)
~
e"
fl(n)
Vw
€
fi,
and
W(y,n,e) = 0(e- CR 'M)
The
€
of these asymptotic characterizations of waiting times, that
first
-R(0) for
€
u>
we may
that
Vy g H.
17 is
find constants
T
,zj with zj G
Z—
w=
z\, zj,
ities
on the path
. . .
obvious.
fairly
is
and
D(Q),e) < /f€-* (n) note
W(w,Z -
To
see that
cj
such that for any
W{w,Z— D(Sl),e) ~
w
€ D(Q) there
D(il) such that the product of
path
the transition probabil-
all
R n '. Conditioning on the outcome of the
'
at least cit
exists a
first
first
T
periods
we
have
W(w,ZTaking the
maximum
<T + (l- c^^Max^^W^Z - Z?(fl),e)
D(Q),e)
of both sides over
all
w
€ D(£l) yields
Max^njWKZ - D(Cl),e) < (T/a)*- ™.
1
To
see that
W(u>,
W(w,Z Z-
D(il),e)
> k(- R
less
hitting
n)
for
any
>
E(#
>
l/MaXj, € nProb{Z
D(Sl),e)
Given that the system returns to
on
^
times in
in
a finite
fi
w
€
before
—
note that
fi,
Z-
D(il)
D(ft)
is
reached
is
clearly of exact order e
=
w)
reached before returning to
number of periods
(in
R W, and hence we have
il\zo
expectation) conditional
than R(il) mutations occuring, the probability of going from
ft is
\zq
w
to
Z- D(il)
without
the desired lower bound for
W(u;,Z-.D(ft),Oaswell.
It
remains now only to show that
Maxy(U W(y,n,t) = 0(€- CR 'M).
To
begin,
we note that
if
we
define set-to-set expected waiting times
W(X,Y,e) = m*xW(z,Y,€),
17
on a worst case
basis,
=
y}.
and write £
for the
union of limit sets of (Z, f(0)), then because in the limit as
expected wait until a limit
Write W(Q,e) for
For each y €
reached
set is first
is finite, it will
suffice to
e
—
the
show that
Max y€ £_ n W(y,ft,<0.
£—
ft, let
y
=
zi,Z2,
•
•
,ZT De a P at h from y to
ft
of
minimum
possible
modified cost. Because any element of a limit set can be reached at zero cost from each
other element of the same limit set,
L\,L 2 ,...,L T through which
it
we may choose
passes
distinct,
is
this
path so that each of the limit
and such that the path
is
contained in
each of these limit sets for a set of successive periods. (Remember that y € L\
path to have
minimum
modified cost
it
sets
).
For this
must be the case that
r-l
= ]£e(Z,„I, +1 ),
c(zi,z 2 ,...,zt)
1=1
and
zt € £,
if
must
it
also
be the case that
=
c*(zt,zt +i,...,ZT)
c"(I,,ft).
(Otherwise, a lower modified cost path can be constructed by concatenating (z\,...,zt )
with a
minimum
Now, the
to reach
£-
crucial element of the proof
Q from
wait to reach
of
modified cost path from I, to
is
ft.)
to simply
L\ as the expected wait to reach any other limit
from that point. Writing q{z\y)
ft
expand the expected waiting time
Ly reached
is
z (after starting at y),
has the longest expected wait to reach
ft
set, plus
the expected
for the probability that the first
and assuming that y
is
the element of L\ which
we have
z€C-Li
Writing
min/gi, Ejgt, q(z\l) we have
qt 2 for
W(L u Sl,e) < W(L lt C - L u e) +
q 12
W(L 2 ,a,c) +
Repeating this process we get
W(y,ft,£)
<
W(L u C-L u( + (l-q12 )W(Sl,e)
)
18
(1
-
element
qi2)W(£l,e).
+qn(W(L 2 ,£ -L 2 ,e) + (l+9i2923(W(L 3 ,£ - L 3 ,e) +
923) W(ft, e))
(1
-
ft4)TT(fi,c))
+
+ 912923
W(I
=
X
..qr -2r-l(W(L r -l,C
,£ - Ii.e)
+
+
•
(1
"
+
fl 2
W
r
- I r -1,«) +
r
912923
•
•
9r-lr)W (fi,f)
•
it
suffices to
subsets of e values for which W(y,Sl,c)
=
W(fi,«). For such
that
show that
912923 •••9r-lr
To show
on the
that W(il,
RHS
To do
e)
= 0(e~ CR 'W)
of the above equation
so, consider the
we have qu+i ~
e
c(
is
it
e's
this holds
on each of the
the expression above gives
9r-lr
now
suffices to
show that each of the expressions
-c **( n )).
0(€
minimum
- R L >)
c(I,,Ii + i) and W(I,,Z-D(£,),e) ~ e
elements which appear in the expressions. Because the
D(U) to Z,+i
cost path from each z 6
" flr-lr^fl.e))
+ 9i2923W(£ 3 ,£ - L 3 ,e)
(£i,£ - &a,«)
W(Q,e) = 0(e~ CR 'W)
To show
(1
has cost
(
i <> i *+i) _fl ( i '). Because the unperturbed Markov process converges to a
limit set in finite time
from any point and the probability of converging to
bounded away from zero
for
any y & D(Li), we also have W(Li,C
-
L,
Z,,e)
is
uniformly
~ W(L
t
,Z -
D(Li),€).
Putting these together we have
9,i+l
•
fC(Lf,L <+ ,)-«(LO
9r-lr
_
.
. .
( c(Lr-l,Lr)-R(Lr-l)
,-c«(^,n)
with the last line following from our earlier discussion of the nature of
cost paths.
0(€~ CR 'W),
For each
:'
this expression is
0(c~
CRm W)
minimum
modified
and hence we have W(y, fi,e)
=
the second main component of the proof.
QED.
At
this point,
analysis
it
I
would
like to give
provides with the
some motivation
now standard
for the
theorem by contrasting the
techniques developed in
KMR, Young
and Kandori and Rob (1992). These techniques provide an algorithm which
19
(1993a),
in principle will
run equilibria of any model which
find the long
of the algorithm requires one to
tree
on the
set of limit sets
The obvious drawback
first
model and
find the
of this paper's characterization of long run equilibria in comparis
restricted applicability
its
identify all of the limit sets of the
direct application
which minimizes a particular cost function.
ison with the standard techniques
Section 9,
A
our framework.
fits in
that
may
not universal.
it is
However, as
is
discussed in
not be such a serious limitation from a practical
viewpoint in that the set of models to which the main theorem applies appears to contain
most models
niques.
for
which economists have previously succeeded
The main theorem of
this
paper has several potential advantages.
why
the theorem provides a clear intuition for
papers are often regarded as being mysterious.
find surprising about the intuition provided
long run equilibrium can be derived from
in previous papers. Finally, the
to models which contain a large
by
it
My
this
guess, in fact,
paper
is
standard techniques are
number of limit
and most
Second,
limit.
works, whereas the results of previous
is
sets or
that what readers will
not that a characterization of
but rather that there
it,
standard tech-
First,
theorem provides a convergence rate as well as a long run
concretely, the
on
in applying the
is
nothing more going
ill-suited to direct application
when one wants
to derive theorems
which apply to classes of models broad enough as to include members for which the
sets differ. It
my hope
is
that the fact that the techniques of this paper
analyses will be seen as one of
The
its
may
limit
facilitate
such
primary contributions. 8
basic plan for the remainder of this section (and for the remainder of the paper)
is
to present a sequence of examples which are intended in varying proportions to illustrate
the use of the main theorem and to present applications which are of interest in their
right.
I
own
begin with a simple example which illustrates the use of the radius, coradius and
modified coradius measures, and which provides some geometric intuition.
Example
the
1 Suppose
N players are
repeatedly randomly matched to play the
game
G
as in
model of Section 2 with uniform matching best-reply dynamics and independent random
mutations,
'The sense
Theorem
with with player
i.e.
which
in
2 clearly
this
i
in period
paper might
shows that
all
facilitate
t
playing a best response to the distribution of
such analyses deserves some discussion.
models to which Theorem
1
The proof
with the standard techniques via "tree surgery" arguments, but that this could have been done
systematic way.
I
would interpret
this result as indicating that
one can alternately think of
in
this
contributing to the analyses of such models by systematizing the use of tree surgery arguments.
20
of
applies not only could have been handled
a fairly
paper as
strategies in the population in period
other two strategies with probability
N
sufficiently large
for
N
—
t
If
t.
we have fi*(A) =
1.
G
If
ABC
we have p'(B) =
sufficiently large
with probability
1
G
the
is
2e,
game shown on
the
is
—
1
and playing each of
the left below, then for
game shown on
the right below, then
ABC
1.
A
7,
7
5,
5
5,
A
7,
7
5,
5
5,
B
5,
5
8,
8
0,
B
5,
5
8,
8
1,
o
C
0,
5
0,
C
0,
5
0,
1
6,
6
6,
his
6
Proof
The diagrams below show
in (<74,<7fl,<7c:)-space.
The
the best-response regions corresponding to each of the games
deterministic dynamics in each case consist of immediate
jumps
from each point to the vertex of the triangle corresponding to everyone playing the best
response.
From the
minimum
distance between
region where
state
A
is
figure
on the
A
and any point not
the best response),
for
N
large
and the
in
D(A)
about |jV. The
is
and D( A) (here the distance from
R(A) > CR{A)
obvious that in the
left, it is fairly
B
to
D(A))
first result
is
(here
D(A)
maximum
approximately
follows from
desired for players not including their
own
is
game R(A),
the
approximately the
distance between any
\N
Theorem
argument requires only a straightforward but tedious accounting
if
first
1.
.
For this reason,
Formalizing this
for integer
problems (and
play in the distribution to which they play a
best response).
In the figure
on the
fN, while CR*(B)
is
right, the modified coradius
about |JV because
the modified cost of the indirect path
follows
suffice
from Theorem
— CR(B)
is
1.
Note that
measure
is
useful. Here,
this is the modified cost of the
(C,A,B)
in this case
approximately jN.
21
is
R(B)
is
about
path (A,B) and
only about gjV. Hence, the result again
a simple radius- coradius argument does not
QN,Q,lN)
K7fl(A)H^
05
(|iV,0,|AT)
1^ |^ (0) i^ i N)
\-R(B)
-T^ 1^ 7$f
{Qf
^
s_
N)
QED.
The bound on
the convergence rate in these games
reaching the long run equilibria are at most of order
is
that the expected waits until
(~W a N
)
and
nothing particularly interesting about this result, other than that as
with uniform matching the convergence time
The
reader should keep in
simplicity,
this
more evident
will
N
in
paper
is
most
like to
theorem of
rapidly increasing in the population size.
The power
useful.
of the modified coradius measure
=
1
this
It is
much
states.
examples which point out a couple of weaknesses of the theorem
it
works.
I
paper does not provide a characterization which in theory can identify the
all
(which turns oat to be
is
systems.
To
see this look at the three state
easy to verify that R(x)
< CR"(z) =
equilibrium
is
remind the reader that unlike the Freidlin-Wentzell characterization, the main
long run equilibria of
R(z)
is
that the above examples were chosen largely for their
models with many steady
this time, providing
below.
There
.
typical in models
probably help the reader to develop a more complete understanding how
would
left
is
5)
and that they are not representative of the situations where the radius-coradius
approach of
At
mind
is
e-( 2 /
5.
The theorem
y).
More
=
5
< CR'(x) =
11,
R(y)
Markov process on the
=
2
< CR'(y) =
5,
and
thus does not help us find the long run equilibrium
generally the theorem will never apply unless the long run
also the state with the largest radius.
22
Next, the theorem provides only an upper bound on the expected wait until the system
reaches the long run equilibrium, and this
illustrated by the four-state
because
it
wait until
takes
^
fi is
why
the modified coradius
bound do not take
In particular, the calculation
sets.
new
"worst case" occurs whenever a transition to a
minimum
modified cost path.
bound
full
£
A
bounds
any computation which does not require a
in
only
2t*'
that the
is
done always assuming that the
is
limit set does not correspond to the
In this case convergence
failure to provide tight
is
advantage of the relationships
faster than such
is
calculation indicates, because the transitions from y or z which do not reach
us back to x.
the
periods to reach
not tight here
is
fl is
is
has modified cost three.)
fl
reached conditional on starting at x, however,
general calculations which derive this
between the limit
(Any path from x to
3.
This point
In this system,
right above.
periods to reach the set {y, z) and then a further
Intuitively, the reason
Q.
=
not necessarily tight.
is
Markov process on the
long run equilibrium and CR*(£l)
The expected
bound
some
a worst case
fl
never take
situations seems unavoidable in
full identification
of
all
of the limit sets and an
analysis of the relationships between them.
The example on the
left
above also provides a nice opportunity to point out a feature
of the theorem which has not been emphasized so far
a union of limit
2
>
that
1
sets rather
= CR({x,y}), and
all
than a single limit
that R({y,z})
=
10
— that the
set. In this
>
5
in the
theorem can be
example note that R({x,y})
= CR({y,z). The
useful is not clear to
me, the
fact that the
practical importance because in models involving extensive
form games,
Samuelson (1993, 1994) and Huck and Oechssler (1995),
appears to be
23
is
the unique
make arguments
theorem applies to unions of
it
—
theorem then implies
long run equilibria are contained in both {x,y} and {y, z}, so that y
long run equilibrium. While the extent to which being able to
is
fi
like this
limit sets
e.g.
is
of
Noldeke and
common
for the
»
set of long
run equilibria to be a union of several limit sets between which the system can
move with one
or
more mutations.
Generalizing risk-dominance: ^-dominance
5
In 2 x 2 games, the models of
KMR
and Young (1993a) provide an elegant and robust
characterization of the long run impact of mutations on the development of social conventions - societies are led to adopt the risk-dominant equilibrium. This section discusses the
generalization of this result, with the primary positive result being that a refinement of
dominance, j-dominance, provides a
risk
sufficient condition for
determining the long run
equilibrium.
For the purposes of this section
G be a symmetric mxm game with
Let
detail.
Z
,
•
•
•
t
SNt)- Let
implications of
results
the set of pure strategies. Following
i
be the period
(•sn, sit
5
N players are uniformly randomly paired to play G in periods =
suppose that
zt 6
necessary to specify a KMR-style model in a bit more
it is
some
state which
P
we
Z
be the Markov transition matrix on
set of behavioral rules. Let the
from each player independently following
and choosing a strategy
at
random with
Let
1,2,
assume corresponds simply to an action
will
KMR
profile
capturing the reduced form
perturbed process P(t) be that which
his behavior rule
probability
e
(with
all
with probability
1
—
e
strategies having positive
probability.)
Also following
(Tn
KMR,
this section will focus
on a particular
be the mixed strategy in which the probability of action
times the number of players j
j4 t
with
Sj t
which are a best response to an for some
».
Darwinian
if
in play over
if z"
time
is
said to be
has more players playing strategy
BR{z
t
)).
The
definition
is
=
s.
Let
BR(z
t,
being being played
s
is
^-j-
denote the set of strategies
t )
The (unperturbed)
process governing changes
whenever BR(zt )
a singleton,
BR(z
is
1
t
)
than does z t (or z has
motivated by the supposition that
to switch strategies after period
class of behavior rules. Let
if
player
i
all
PZtZ >
>
only
players playing
has the opportunity
a reasonable thing for a myopic player to do would be to
play a best response to the strategies used by the other players in the most recent period. 9
Recall that in a symmetric
'One example
of a Darwinian
2x2 game
dynamic
is
with pure strategy equilibria (A, A) and (B, B),
the best-response dynamic under which
response to the previous period's strategy distribution,
24
i.e.
Sit+i
€ argmax,
g(s,ff, t ).
all
players play a best
the equilibrium (A,
A)
is
said to be risk-dominant
\A + \B. The main
strategy
result of
KMR
run equilibrium of the above model involves
rium. Note that this selection
if
that for
is
the best response to the mixed
is
N
sufficiently large the
holds for
it
it-nearest neighbor
matching on a
Given that Harsanyi and Selten's definition of
circle
generalizations to explore was to see
Young ( 1993a) immediately showed
risk
ization, but that
that in 3 x 3
games the
would
that
it
first
did not, exhibiting a 3 x 3
Example
games the
C
is
and
are
all
differs
players have
assumed
direction for work on
what was
game where
selected.
in
which
.
One problem noted by
Ellison (1993)
dependent on the nature of the matching
illustrating
selected equilibrium
the unique long run equlibrium is
playing
exists.
may
2 For the model described above with
being used
titled
between the uniform and two neighbor matching models.
differ
now an example
Darwinian dynamic which
a local interaction
Harsanyi and Selten's definition not the sought after general-
selected equilibrium
may be
like to present
that in 3 x 3
1
it
in
their definition corresponded to
if
no general and robust answer
rule, in particular
I
is
than evo-
dominance appeared in a book
10
the risk dominant equilibrium was not selected by his dynamics.
turns out that not only
faster
and best-reply dynamics.
General Theory of Equilibrium Selection in Games, the obvious
is
much
showing that risk-dominant equilibria are selected also
model involving
It
is
Ellison (1993) further demonstrates the robustness of this
lution in the reverse direction.
A
Darwinian dynamics,
all
even those which are rigged so that evolution toward one strategy
selection in
unique long
players playing the risk-dominant equilib-
all
quite robust in that
is
A
A
from
A
to switch to
an even more basic nonrobustness
differ for different
—
Darwinian dynamics. 11
G the first 3x3 game discussed in Example
under the best-reply dynamic, and
B
under the
dynamic only when both
B
and
the best-reply
C
are
as their best response, in which case only the players
A.
Proof
The
for the
of
results are clear
from an examination of the basins of attraction of the equilibria
two models pictured below. The diagram on the
Example
1)
shows the basins
R(A) > CR(A). The basin
10
for the best- reply
of attraction of
The example being the game on the
"Hahn
left in
B
is
Example
left
(repeated from the discussion
dynamic from which
it is
larger under the other dynamic, because
1
(1995) has noted the same result for 2 x 2 games in a two population model.
25
easy to see that
some points which were formerly
starting at
5, the unperturbed dynamics now reach
edge where
and
B
is
players play
all
A
B
just to the right of the basin of attraction of
two steps
in
or 5, and then to B.
^ 3^y
s
05
While robust
results
and Selten (1988)
call
(lN,0,lN)
Nj i N)
MNAN)
C
i
(0i
exist), the
main
result of this section does provide a robust
equilibrium. Morris,
strict best
Recall that in
Nash equilibrium (A, A) the
the
\A +
better response than s to
\s for
all
Rob and Shin (1995)
call
is
is
if
also a
A
is
a
Nash
a symmetric equilibrium (A, A) p-dominant
—
that the selection of risk-dominant equilibria in 2 x 2
Corollary 1 In the model above, suppose (A, A)
sufficiently large the limit distribution
=
games, Harsanyi
such that (s,s)
s
generalizes to the selection of ^-dominant equilibria
satisfies fi*(A)
N
a refinement of risk dominance. The following Corollary contains
the main result of this section
N
x
dominant equilibrium
risk
pure strategies
N
response against any mixed strategy placing probability at least p on A.
Note that —dominance
for
we have R(B) > CR(B)
(IjV,0,|iV)
generalization for a substantial class of games.
games
to a state on the
can not be obtained generally (or even for the class of games for
which risk-dominant equilibria
a
result
moving
(|iV,|jV,0)
B r-Ci?(A)H^
A is
As a
first
the long run equilibrium.
(|AT,|iV,0)
if
—
[t*
is
whenever they
exist.
a ^-dominant equilibrium ofG. Then
corresponding to any Darwinian dynamic
1.
Proof
The proof is
detail).
quite easy (with
Note that since
at least 5
on
A
A
is
length here being due to
its
its
being presented in extreme
a strict best response to any distribution placing probability
there exists a q
<
\
such that
distribution placing probability at least q on A.
26
A
The
is
also
first
a
strict best
response to any
step of the proof
is
to
show that
R(A) >
(1
- q)(N -
To
1).
players playing A, each player sees at least
A
each of them has
q(N -
< c(A,z)
of the
1)
at cost zero has strictly
Vz,z' with c(A,z)
the argument above shows that any such z
< q(N -
in
is
players play
A
C R(A) < q(N-l)+l,
has (Tit(A)
>
from any state x to a state
play as in
some
q for
in
all i,
1
N
has at least
N
1
-
1)
+
and P„.(0) >
1
sufficiently large so that (1
+
+
(i.e.
1
we
0). Iterating
N - (q(N -
show R(A) >
q(N —
1)
+
1
in
is
1)
+
1).
at least
q(N-l) + l
D(A). The
direct path
players play
state which has positive probability in the unperturbed
1)
1)
others playing A. Thus,
because any state in which
first
q(N -
more players playing A
and hence as above
which the
from z to D(A) of cost at most q(N Taking
± A
D(A). From the argument following the
definition of the radius, these conditions are sufficient to
Next, observe that
z
and the Darwinian property implies that
as an unique best response
any state which can be reached
have shown c(A,z')
whenever
see this, note that
A
while the rest
dynamics
is
a path
1.
- 2q)(N -
1)
>
1,
R(A) > CR(A) and Theorem
applies.
QED.
I
fact that the proof of the Corollary is fairly trivial will
hope that the
be taken as
evidence that the main theorem of the paper can facilitate the development of general
Because
results.
itself is trivial, I
I
realize that
would
one might be tempted to conclude instead that the result
like to
note that both Kandori and Rob's (1992) result on pure
coordination games and their (1993) result on risk-dominance in games with the total
bandwagon property are immediate
different payoff to coordinating
corollaries of Corollary
1.
In a
game where
game with
^-dominant. 13
the total
The
bandwagon property,
result of
Kim
of symmetric I player, two action
games
a
is
also ^-dominant.
dominant equilibria are automatically
also follows
KMR
model
in a class
from the argument above (given the
13
Note that what Kandori and Rob (1993) show
satisfying
risk
(1995) on the behavior of the
appropriate definition of ^-dominance).
1
is
on each of the available actions and the payoffs are zero
whenever the players do not coordinate, the pare to optimal equilbrium
In a
there
is
that risk dominant equilibria are selected in games
both the total bandwagon property and the monotone share property.
Evidently, the latter
restriction is unnecessary.
The
appropriate definition for the situation being that a symmetric equilibrium
player's strategy
is his
is
^-dominant
if
each
unique best response to any distribution of his opponents' play induced by random
27
Local interaction: two dimensional lattices and fast vs.
6
slow evolution
This section discusses the behavior of models with local interaction.
The most notable
that ^-dominant equilibria are selected and evolution
a model with a two
result
is
is
fast in
dimensional interaction structure. The result should be of interest both because
it
the role of "contagion" dynamics in producing fast evolution, and because
illustrates
much more
clearly
than have any of the previous examples where and how the modified
coradius computation can be a powerful tool.
in the
paper and
how
works.
both
late
details of
it
it
clarifies
in this section
I
hope that the
fact that the
example occurs
does not deter the reader from looking at the
Evolutionary models with local interaction are intended to capture social situations in
which players interact most often with a small stable
Ellison (1993) argued that such
exist in the real world, but
is
fast
enough
The
on a
first
circle
to
because
only in the context of such models that convergence
selection
new result presented here is
arguments plausible.
that for ^-nearest neighbor local interaction models
x 2 games again
generalizes to the
^-dominant equilibria when they exist (indicating a further robustness of the
both because of
its
main theorem helps
and that evolution remains
inherent interest, and because
in developing generalizations
its
and
relatively fast.
I
include the result
very quick proof illustrates
how
in providing convergence rates.
state this result formally suppose as in Ellison (1993) that players 1,2,...
arranged sequentially around a
dynamic
in
which player
i
Again take the
€- perturbed
t
+
the
14
,N
are
Consider a 2k neighbor version of the best-reply
circle.
in period
by taking the average of the play
1
plays a best response to the distribution a, t formed
in period
t
by players i—k,i—k+l,...,i—l,i+l,...,i+k.
process to be that given by independent
random
trembles.
The
is
Corollary 2 Suppose (A, A)
matching within a population
14
it is
make evolutionary
selection in these games),
result
models are interesting not only because such relationships
the selection of risk-dominant equilibria in 2
selection of
To
set of friends, colleagues, or neighbors.
in
is
a ^-dominant equilibrium of G.
which at
Ellison (1993) only provides
neighbor model, and does so by a
For
N
sufficiently large
least half of the players are playing the equilibrium strategy.
an (-order approximations to the second largest eigenvalue
much
longer argument.
28
for the
two
the limit distribution p* corresponding to the
dynamics has n"(A)
=
and for
1,
all z
2k neighbors on a
£ A we
have W(z, A,c)
circle
=
model with
best-reply
0{€-( k+l) ).
Proof
The argument
Ellison (1993).
R(A) > CR(A)
that
Any
cluster of players playing
N
For
>
l,2,...,Jfc+
among
1,
a second mutation
+
l)
2
path from
2) a
+
1,
.
1
.
.
,
adjacent players play
A
lies in
grow contagiously. Hence CR(A) < k
will
+
l)(k
+
A
{k
players (k
+
state in which k
follows directly from the proof of
(ik
A
among
+
l)(Jb
out of
D(A)
players k
+
2).
+
+
Hence, R(A)
,2(k
+
> k+
of
1
D{A) because
the
1.
requires a mutation
2, ...
Theorem
1), etc.,
among
players
and a k
2 and again
+
2
nd
Theorem
1
applies.
QED.
Note that an analysis which
tighter
is
tailored
more to a given model's dynamics may provide
bounds on convergence times. For example,
involving a 2 x 2
neighbors play
in
which
A Theorem
asymptotically e~
My
game
s
,
1
A
is
in a
model of eight neighbor matching
the best response whenever three of a player's eight
can be used to show that the expected wait to reach
but rather
«~ 3
A
is
.
primary motivation for discussing the model above and the one that follows
desire to
comment on what
it is
not
makes evolution
fast in local interaction
is
a
models. As noted
above, the dynamics of the ^-nearest neighbor on a circle model feature the rapid contagious
expansion of relatively small clusters of players playing the ^-dominant equilibrium. Such
dynamics are present, however, only because of the extreme overlap between a player's
neighbors and his neighbors' neighbors in this model, and one might wonder whether fast
evolution would go away
The remainder of this
of extreme overlap
if
we eliminated
section
is
this unreasonable aspect of the social interaction.
concerned with one model which eliminates this assumption
— a model in which the players are situated at the vertices of a two-
dimensional lattice and interact with their four nearest neighbors. 15
The
best-reply dynamics of such a
dimensional models. Note
l4
first
model
differ in interesting
ways from those of one
that unlike one-dimensional models there
is
no strong force
See Blume (1993) for a discussion of a two-dimensional Ising model which has a somewhat analagous
conclusion about long run behavior, and Andeilini and Ianni (1993) and
nonergodic models with a two-dimensional interaction structure.
29
Blume (1994)
for analyses of
which ensures homogeneneity of actions here.
Instead there
may be many
steady states
which different actions are taken by different segments of the population. For example,
in
in the
2x2
standard
coordination
game where
B
coordinating on A, one from coordinating on
state
for
is
a cluster of four adjacent players
the rest of the population plays B.
two of
the players receive a payoff of two from
and zero from miscoordinating, one steady
(in
a "square" configuration) to play
A
Players in the cluster are satisfied with
A
while
because
their four neighbors belong to the cluster, while each player outside the cluster
happy playing
B
because at least three of his four neighbors
Other steady states are obtained,
for
is
also outside the cluster.
is
example, by having variously sized separated blocks
A
of players playing A, or by having vertical or horizontal "stripes" where
is
played while
the remainder of the population plays B.
The
fact that small clusters
do not grow contagiously
in the
model without noise makes
an interesting pattern of evolution in the model with noise. Rather than seeing conta-
for
gious growth triggered by a few mutations
agglomeration
arise
in
we
will instead
observe a proces of evolution by
which small clusters of players playing the ^-dominant equilibrium
from mutations, subsequently grow (and sometimes shrink) slowly as mutations occur
grow together and take
at the edge of the cluster, until eventually adjacent clusters start to
over the population.
to
will
make
The Corollary shows
evolution fast
much about
— evidently the
that this pattern of growth
is
sufficiently strong
fast evolution of local interaction
the contagious spread of strategies as
it is
about the
models
is
not so
ability of strategies to
gain footholds in small areas.
It is
because this model has so
many steady
the ideal setting in which to illustrate
The
fact that this
very
difficult to
how
states (and cycles) that
compute the long run equilibrium by
order Freidlin-Wentzell tree (a construction whose
same
see
why
first
states (and cycles)
it is
also
useful.
would make
step would be the enumeration of
number
Note that
of limit sets
in
it
minimum
directly constructing the
time, the presence of a large
what makes the modified coradius calculation
we have seen
that
the main theorem of this paper can be applied.
model has a large number of steady
of the limit sets.) At the
I feel
is
all
precisely
most of the examples
so far a radius-coradius theorem would have sufficed, and hence one can not
the modified coradius
is
important. Here, both the identification of the long run
equilibrium and the demonstration that evolution
30
is
fast will
more thoroughly
exploit the
fact that the intermediate limit sets exist
provides a plausible
Let
me now
way
for
and a step-by-step path through these
change to occur.
give a brief formal specification of the model, which will be followed by a
somewhat lengthy though not too complicated proof that ^-dominance
to determine the long run equilibrium,
N1N2
limit sets
and that evolution
N\ x N2
players are located at the vertices of an
state z of the system
a function z
is
{1,.
:
..
,N\} x
again sufficient
relatively fast.
is
lattice
Suppose that
on the surface of a torus.
—
{1,...,./V2 }
The
action taken by the player at location {t,j}.
is
5 with
A
z(i,j) giving the
deterministic best-reply dynamics with
nearest neighbor matching are obtained by assuming that in each period each player plays a
best response to the strategies used by his four immediate neighbors in the previous period,
z t +i
i.e.
=
b(z t ) satisfies z t +i(i,j) £
-
putting probability \ on each of zt (i
Argmax, € 5</(s, o~ijt) where
1, j),
+
z t (i
1, j),
zt (i,j
-
<T, Jt
1),
the distribution
is
and z
t
(i,j
+
1),
and we
assume again that the perturbed process incorporates independent e-probability mutations.
Corollary 3 Suppose (A, A)
the limit distribution
is
the
^-dominant equilibrium ofG. For Ni > 3 and N2 >
corresponding to the best-reply dynamics of the nearest neighbor
fi'
=
matching model on an Ni X iV2 torus has n*(A)
0(«- 3 ) as
€
3,
and for any z
1,
^ A we have W(z, A, c) =
— 0.
Proof
First,
show that R(A) > min(JVi,
I
To
jVj).
see this,
tance measure which provides a lower bound on the
D(A) from any
play
has
A and
all
point. Let g{z) be the smaller of the
number of columns
the
players playing
A
>
g(z).
in
which
all
of
them play
A
-
follows that
<
min(iv" 1 ,iV2 ) then g(zr)
g(z)
c(i4,z2 ,...,ZT)
to
>1=>:6
That g(z) >
in
which
all
players
a given row (column)
row (column) have
From
two
at least
this,
we know
Further, as each "mutation" can only break up one row and one
g(z)
A
If
in b(z) as well.
>
path from
easiest to define a dis-
of mutations needed to escape
players play A.
all
column, we know that g(z')
if
is
number of rows
in z, then all players in that
neighbors playing A, and hence
that g(b(z))
number
it
Z — D(A)
with cost
c(z, z
less
1
).
Applying
>
this relationship repeatedly,
1.
than min(7V 1 ,iV2 ),
To
it
establish that there
is
it
no
thus suffices to show that
D(A).
1
=^
intuition for this result
z
is
G D(A) follows from a straightforward
that
if all
explicit calculation.
The
players in a "cross" pattern play A, the set of players
31
playing
A
To
population.
that z(t,l)
z(i,j)
expand out from the center of the cross
will
= A
=
>
A
=>
1
for all
with
for all (i,j)
i
and has
proof,
I
t
+j <
k,
= A
Each transition so
3.
=
i.
three or
less, I will
Let z\
far has cost zero. Let z 2 +k l
that c(zi + k x , Z2+ki)
<
2.
+j <
i
k
+
It
rest of the
set.
Let zt
To do
conclusion
=
6
t_1
(zi) for
t
be a state in which players
turns out that this
is
minimum
=
will
index
+
2,...,ki
(1, 1)
1.
and (2,2)
population plays as in the state b(z l+kl
define the path with at
A
addition
so, I explicitly construct
Let k\ be the
So that the path being constructed
clusters of players playing
The
1.
).
Note
have a modified cost of
most one other transition which
best response or a transiton which escapes a distinct limit set whose cost
that limit set.
If in
follows by induction.
show next that CR'(A) <
both play A, and in which the
j € {1,...,JV2 }.
for all i,j with
a member of a stable limit
is
all
loss of generality
easy to see that b(z) also has the same "cross"
it is
each x £ D(A) a path from x to D{A).
such that b kl (zi)
encompasses the entire
it
and assume without
1
G {1,...,^} and
b(z)(i,j)
D(A) then
z G
To complete the
for
= A
z(l,j)
of players playing
that g(z)
>
see this formally, suppose g(z)
until
is
is
not either a
the radius of
indeed possible, because successively larger stable
can be formed by adding single mutations to the edge of the
cluster.
Formally,
let
fc 2
be the
minimum index such
that b
k2
(z 2 +k l
)
is
a member of a limit
set.
A
odd
Note that as the players (1,2) and (2,1) each have two neigbors playing
period, and players (1, 1)
and (2,2) each have two neighbors playing
along the way, one of these pairs plays
i
=
1,2,
.
.
.
,k2-
A
in b kl (z2+k x
)
all
in
each even period
as well. Let ^2+fci+i
Note that each of the added transitions
Let Z2+k 1 +k1 +i be defined so that
A
will
in each
=
&'( z2+i,) for
have cost zero.
of players (1,1), (1,2), (2,1), and (2,2) play
A
with the remainder of the players playing as in b(z2+k l +ki )- This transition has modified
cost at
most one. Repeating the process of adding successive best responses we obtain a
state z 2 +kl +kI +k3 belonging to a limit set in which all four of those players play A. (This
sequence of transitions described in this paragraph should be skipped with £3 set to zero
the four players already play
The
state z 2+tl+fc7+fc3
A
in z2+fc,+fc}.)
must
either have all players in rows
have players (1,1),. ..,(1, a) and
of Z2+ki+k 3 +k,(l,a
+
1)
if
(2, 1),.
..
,(2,a) playing
and 22+t + fc, +fc,(2,a
,
+
32
1) different
A
for
1
and 2 playing A, or
some a >
2,
with one
from A. Let z2+ 1+k3+fc3+1 be
fc
defined by adding a single mutation to b(z2+ki +fcj+* 3
mentioned above plays A. Let z2+ k 1 +k1 +k3+2,
ers
responses such that z 2+ iei+ e2+ jC3+ iei
i
one of players (l,a
at least
A, and
+
1)
have both playing
will
process,
all
)
is
and
A
if
players in the
first
a
z 2 +k 1
we eventually obtain a path
member
again a
(2,
+
.
1)
playing
so that an extra one of the two playz 2+ki+k2 +k3 +kA be a sequence of best
of a limit set. This limit set will have
A
if
z 2+kl+kj+k3 had neither playing
+k J +k 3 had one of them playing A. Repeating this
of modified cost at most three to a limit set in which
two rows play A. Following the same process
we obtain a path of modified
cost at
two columns also play A. By the
attraction of A. Hence,
most three to a state
result
CR'(A) <
3,
in
for the first
which
all
two columns
players in the
above on "crosses", such a state
is
first
in the basin of
which completes the proof.
QED.
While
and
is I
this
example does a nice job of
illustrating the
think representative of the situations where
keep in mind also that the modified coradius
many
to be applied to
Some
7
is
it
is
power of the modified coradius
most
useful, the reader should
needed in order to allow the main theorem
of the less complicated models which have appeared in the literature.
of these cases will be pointed out in Section 8.
Another example: cycles as long run equilibria
The example
of this section concerns one of the the
most interesting aspects of stochastic
learning models, their ability to provide a rationale for selecting between multiple strict
equilibria.
Such an interpretation has been encouraged by Young's (1993a) proof that
weakly acyclic games
all
in
long run equilibria are Nash equilibria and also by the reason-
ableness of the selection criterion in a variety of settings. 16 In light of this, the following
example
is
of interest as a simple illustration of the fact that beyond weakly acyclic games,
the term "long run equilibrium"
come may very
well be
example, (A, A)
in
is
may be something
of a misnomer in that the selected out-
a cycle which does not involve the unique Nash equilibrium.
the unique Nash equilibrium, while the long run equilibrium
which the players synchronously alternate between
B
and C.
"Sec, for example, Noldeke and Samuelson (1993, 1994) and Young (1993b)
33
is
In this
a cycle
Example 3
Suppose
N players are
repeatedly randomly matched to play the
game
G shown
below as in the model of section 2 with the uniform matching best-reply dynamics.
for
N sufficiently
=
large fi'(B)
Then
ABC
fi"(C)
A
1,
B
0,
C
0,
=
\.
0,
I
0,
-4,-4
3,
3,
3
4,-4
3
Proof
Again, the answer
space pictured below.
is
is
obvious given the structure of the best response regions in {a a, &b,&c)-
The radius R{{B, €})
approximately ^N. For
N large Theorem
is
approximately f JV. The coradius
gives
1
m
fi
(B)+fi*(C)
=
1,
and the
CR({B, C})
result then
follows by symmetry.
3
(
<N,\N,0)
\-R({B,C})-\\
(0,fJV,fiV)
QED.
Looking at the dynamics above,
it is
tempting to view the result merely as a curiosity
attributable to an unreasonable specification of the deterministic dynamics.
instead chosen an unperturbed dynamic in which play changed
players
had a longer memory,
the simple example I've given
for
not be robust, however, the problem
34
we had
more gradually or where
example, then the cycle would not be a limit
may
If
is
set.
While
fundamental.
What
selected by models with noise
is
is
determined simply by the
between the basins of attraction, and there
is
a necessary condition
for
is
not hard to write
it is
and
relations
no reason to think that being an equilibrium
having a large basin of attraction. Given that cycles appear as
limits not only of the best-reply
play,
is
sizes of
dynamic, but also of learning processes such as
down a
variety of
models
fictitious
which the "long run equilibrium"
for
a cycle.
8
Relationships with the literature
One
of the primary motivations for this paper was a desire to develop a framework which
would provide a more
models with
noise.
to discuss in
some
intuitive
To
and thorough understanding of the behavior of evolutionary
assist the reader in assimilating the ideas of this paper, it is useful
detail the relationship
between
this
paper and the existing
This section focusses on two topics: the extent to which the main theorem of
is
applicable to models which have been previously studied and
how
literature.
paper
this
the techniques of this
paper are derived from and build on the previous work of several authors.
Obviously, the framework of this paper
applicable.
The question
dried as
might seem at
make
it
is
only useful to the extent that
of where the framework
first.
is
applicable, however,
Depending on why we care about
is
it
is
widely
not as cut and
applicability,
it
may
sense to interpret this question in two very different ways: asking for which models
the long run equilibria could have been identified using the main theorem of this paper,
or alternatively asking where the theorem further
equilibria easier (or
I
will discuss
I
more general) than
it
makes the
would have been with the standard techniques.
answers to both forms of the question in
begin by addressing the
first
this section.
form of the question, with the motivation being that
such cases the results of this paper both provide intuition for
what
is
it is,
and allow
for
we can say
is
With
that the theorem of this paper
One should keep in mind
the easiest
way
is
main theorem of
this
35
one
applicable to almost every model for
identified.
17
For example, this includes
here that this definition of applicability allows
to see that the the
the long run equlibrium
this first interpretation of "applicable,"
which long run equilibria have previously been
1
why
in
a more complete characterization of behavior by providing
a bound on the rate of convergence.
thing
identification of long run
all
me to include many models where
paper applies involves essentially constructing the
of the cases where long run equilibria are identified in Kandori, Mailath,
Evans (1993), Noldeke and Samuelson (1993, 1994), Robson and Vega-
Ellison (1993),
and Young (1993a,
(1994), Samuelson (1994)
Redondo
and Rob (1993),
not applicable include both models which do not
models which do
fit
fit
b).
18
Cases where the theorem
into the
framework of Section 2 and
but for which the theorem simply lacks power. Examples of the
type include models where the noise
is
is
sufficiently restricted so as to render the
first
model
non-ergodic (as in Anderlini and Ianni (1993) and some of Noldeke and Samuelson 's (1994)
and papers which employ variants of the solution concept
signalling games),
and Samuelson (1993) who look
examples of Markov processes
lacks
power
(as
the literature
at
for
an
N—
oo limit). While
games discussed
in
Binmore
easy to construct simple
which the radius-modified coradius calculation simply
was done at the end of Section
— the only examples
it is
(as in
I
know
4),
such examples appear to be quite rare in
of are (some of the) differentiated price oligopoly
Kandori and Rob (1992),
for
which the tree analysis exploits
details of
the relationships between the limit sets which are not captured by the modified coradius
calculation.
my hope
It is
that the techniques of this paper will not only allow for a
more complete
understanding of the previous literature, but will also allow the literature to grow by
away from
itating the analysis of interesting models which economists have to date shied
because of their complexity. For this reason,
it is
facil-
useful also to discuss "applicability" in
terms of where the techniques of this paper make the analysis easier. The cleanest examples
I've
found of
The
result
ing
KMR's
this
type are, not coincidentally, those that
on ^-dominance with
original
its trivial
I
have discussed in
this paper.
proof generalizes several previous results includ-
theorem on 2 x 2 games, Kandori and Rob's (1992) result that pareto
optimal equilibria are selected in pure coordination games, and Kandori and Rob's (1993)
conclusion that risk dominance
in
games
minimum
"Most
in
satisfying their total
a sufficient condition for being a long run equilibrium
bandwagon and monotone share
order Freidlin-Wentzell tree
properties. 19 Similarly, the
ud reading the radios and modified coradius off of the tree diagram.
R > CR theorem. Young (1993b) and the 3 x 3 game
of these papers, in fact, require only a
Young (1993a)
are examples where the modified coradius
from the minimum cost
tree.
The
rederived by showing that their
"The
is
results of the
lemmas
paper) that
necessarily
and can be
easily
determined
Samuelson and Noldeke-Samuelson analyses are most
easily
follow from a modified coradius computation.
result also generalizes the result of
i-
is
Maruta (1995) (which was developed without knowledge
-dominant equilibria are selected
in
coordination games.
36
of this
result
on one dimensional
in Ellison (1993).
games (such
and
as
M
local interaction
Generally,
it
is
one to derive results
in allowing
games with j-dominant
in the analysis of
models generalizes the analysis of 2 x 2 games
equilibria as
for
broader classes of
opposed to pure coordination games)
models with many steady states that
I
would expect the techniques
of this paper to prove most useful.
In trying to understand the
is
useful also to note
been developed
of this paper
in
how
this
approach to analyzing models with noise developed here,
paper builds on and combines ideas and techniques which have
previous papers.
— that
it
The
most basic idea behind the main theorem
single
the long run equilibrium concept
necessary for transitions between limit sets
—
new
is
is
closely related to waiting times
as a basis for
an algorithm, but has
been clearly recognized as an intuitive description of the long run equilibrium concept from
the very beginning (see
e.g.
KMR).
Rather than discussing waiting times, the previous literature on identifying long run
equilibria has followed instead the basic
directly or via "tree surgery"
approach of analyzing
minimum order
trees (either
arguments a la Young (1993a)). In trying to understand how
the argument of this paper works,
it is
useful to note (as
follows) that the long run equilibrium part of the
is
discussed in the section which
main theorem of
this
paper can also be
derived fairly easily via a "tree surgery" argument. This argument can be seen as combining
two
distinct ideas: that evolution with noise tends to favor limit sets
and that the presence of intermediate steady
attraction,
We
may
speed evolution.
can find each of these ideas in some form in the previous literature.
with a large enough basin of attraction
of places.
for long
states
with larger basins of
An
is
That a
selected has been previously noted in a
explicit statement of the fact that R(ft)
> CR(il) being a
set
number
sufficient condition
run equilibrium was independently given by Evans (1993) (see
Lemma
3.1),
with
a tree surgery argument which establishes the result (though
it
being contained in the proof of Theorem
Other authors have noted
1
of Ellison (1993).
the limit set with the largest basin of attaction
contexts, e.g. Kandori
their total
and Rob (1993) note that
bandwagon and monotone share
intermediate steady states
20
Here
it is
may speed
instructive also to
is
is
not stated explicitly)
the long run equilibrium in particular
this is the case in 3
properties.
x 3 games
The second idea,
satisfying
that the presence of
evolution, has not previously been stated so explicitly,
compare the
tree surgery proof in Ellison (1993) with the fully constructive
proof which was given in the working paper version of that paper only for 2-neighbor interaction.
37
but does clearly play a prominent role
one
(in
my
the work of Samuelson (1994)
and Noldeke and
In those papers, the long run equilibria are identified using a
Samuelson (1993, 1994).
lemma which
in
words) states that a component
in the limit distribution
if fl(ft)
>
2
and
ft
of limit sets receives probability
ft
can be reached from any other limit set
CR'(Q) =
via a chain of single mutations (a condition which ensures that
1.)
of this paper can be seen as building on this work by noting that a path
deserves "credit" for R(x) mutations (rather than one)
i,
and
in
combining
this idea
where the nonselected limit
The
tion of
final
sets are not so unstable as to
medium run behavior
in the
the limit set
can be applied in models
be upset by just a single mutation.
is its
provision of a general descrip-
form of a characterization of waiting times. While no
previous papers have provided any general discussions of
ic
it
analysis
more generally
when passing through
with the previous one so that
departure of this paper from the literature
The
medium run
behavior, the top-
has been discussed in a few particular models. Ellison (1993) argues that convergence
rates are an important consideration
computation and
in
terms of
N—
<•
and discusses convergence rates both via an eigenvalue
The two subsequent
oo asymptotics of waiting times.
papers which explicitly discuss convergence rates, those of Binmore and Samuelson (1993)
and Robson and Vega-Redondo (1994) both take the approach of using minimum T-period
bound waiting times. Though neither paper
transition probabilities to
a
level of generality
which
is
theorem at
states a
greater than that necessary to apply to the model in question,
the arguments they give are easily generalized to establish that the waiting time to reach
a long run equilibrium
which
ft
satisfies R(il)
> CR(£l)
is
at
most 0(€~ CR^
^).
One can
think of the recursive waiting time computation of this paper as achieving a tighter bound
by taking into account the
9
A
effects of intermediate
steady states.
Freidlin-Wentzell 'tree surgery' argument
In this paper,
I
have argued that the techniques developed here have several advantages over
the Freidlin-Wentzell approach. That the main theorem of this paper provides
and that
it is
controversial.
valuable to
know how
In addition
I
new
intuition
quickly long run equilibria are reached should not be
have argued that the main theorem of
this
paper
finding long run equilibria in complex or incompletely specified models. While
I
facilitates
hope that
the examples presented here provide convincing evidence of this, one could debate this
38
point by saying that the Freidlin-Wentzell approach
in principal
be applied to any of the models
What I would
part of the
like to
note here
main theorem can
in this paper.
that in fact even
is
completely general and thus could
is
more
is
true
— the long run equilbrium
be derived as an only somewhat involved
in its full generality
application of "tree-surgery" arguments (to borrow a phase from
The primary motivation
Freidlin-Wentzell methodology.
for
Young (1993a))
doing so
is
to the
the hope that
21
presenting this connection will improve the understanding of both methodologies.
In
addition, doing so allows a slight extension of the long run equilibrium part of the theorem
to provide partial characterizations
and
hope
I
suggestive of
is
how
similar tree surgery
arguments may be crafted to solve problems where the main theorem of this paper
fails
to
apply
Theorem
x
$
ft
/x*(ft)
2 In the model described in Section
we have #(ft) > c'(x,ft) then
>
in
0. If
if
for some limit set
R(Q) =
ft
and some
>
c*(x,ft) then /i'(x)
state
implies
0.
Note that part (a) of Theorem
c*(z, ft)
=
/i*(x)
2,
< CR'(£l) Vz g
ft,
1
from the
follows
and hence the
combination with the fact that p'(Z)
first
=
1
conclusion of this theorem because
first
conclusion implies fi"(z)
implies that /x"(ft)
=
=
Vz £
ft,
which
1.
Proof
The argument given here
(1993),
and Young (1993a)
of n".
The argument
construction of the
follows Foster
in relying
differs
minimum
on
and Young (1990), Kandori, Mailath, and Rob
Freidlin
somewhat
and Wentzell's (1984)
in that
it
is
tree characterization
based not as much on an explicit
order trees involved as on a "tree surgery" argument which
recognizes that for the purposes of this theorem
we care only about whether the minimum
order tree has a certain property, and this can be examined by considering modifications
The argument
of trees without the property.
(1993) and the main
An
x-tree
exists k with
t
is
k
t
Lemma of
a function
(z)
=
x.
t
generalizes the proof of
1
in Ellison
Evans (1993).
:
Z
—
Z
such that t(x)
Define the cost of an x-tree
'In particular, one should note
Theorem
how
=
t,
x and such that Vr
c(r),
by
c(t)
=
^
x there
J2 z ^ x c(z,t(z)).
the arguments here can be seen as combining the surgery technique
inherent in Ellison (1993) and developed generally in Evans (1993) with an extension of the single transition
technique used by Samuelson (1993) and Samuelson and Noldeke (1993, 1994).
39
.
A
consequence of Freidlin and Wentzell's results
minimum
order trees. Specifically, i
exists an u>-tree
such that
t
To show that x £
given an x-tree
Suppose
cost.
minimum
t'
ft is
one can
not a long run equilbrium
is
x-trees
all
have
t'
c(t')
>
(for
w
some
€
ft)
modified cost. This cost
is
at
when
x),
most
zj,...,zr(=
we may
define a w'-tree b
Y =
U'_i D(Li). Combining several maps
that for
if
z
£
some
we have
>
it
{zi,zj,. ..,zt},
k
b (z)
:
c*(x,ft),
like
=
-,L T
choice of path
show how,
I
ft'
that above
=
be the
ft)
with
ft
set of
we may assume
that
of (Z,P(0)) and for any state
=
such that c(z,6(z))
Z?(ft')
e {z\,zi,... ,zt}
and such that b(w)
>
be a path from x to
c"(x,ft). Let (L\,L,2,-
—
D(ft')
ft'
fi(ft)
*"))
these limits sets are distinct. Note that for any limit set
w' e
for
construct an w-tree t" which has a strictly lower
With an appropriate
limit sets crossed along this path.
some w ^ x there
if
c(t).
not a long run equilibrium
an x-tree. Let (zi(=
t' is
that long run equilibria correspond to
is
we may choose
Vz
b
w'. Let
5*
Y — Y
:
such
€ Y, such that c(z,6(z))
for all z
=
w.
Define an u>-tree t" by
t"(z)
To
see that this
with that of
Because
t'
is
{
it
the definition of
zt for
some
£ {1,2,..., T
b(z)
ifzey,z$r{z 1 ,z2 ,...,zr -,}
tf{z)
otherwise
t
Y
reaches either an element of
an x-tree (and
is
=
if
note that for any state z the path
u;-tree
until
('
the second set
to
a
is
=
z
z t+ i
z\
—
(z, t"(z),
reached, the path clearly leads to w.
1}
n (z),
t'
.)
.
coincides
or an element of {zi,zj,. ..,zj}.
one of these
x), this ensures that
—
If
the
sets will
first set is
be reached.
If
reached then, by
the second set will be reached within k periods and again the path leads
b,
uj.
Because the trees
the definition oft",
the costs of
**
and
:'
and
t" are identical at all states
we may cancel the
t".
In addition,
covered by the "otherwise" line of
costs of transition for all such states
we know
c(z, b(z))
=
Vz €
Y
with z
when comparing
g
{z\, z 2 ,
.
.
.
,
zj}.
Using these facts we can write
c(t")-c(t')=Y
t
Now, consider each
at
most c*(x,ft)
of the
+ Y^Zl
c(z t ,zt+l )-
two sums on the
R(Li).
right
,
£c(z,t(z)).
hand
side of this expression.
To bound the second, note
40
that, because
The
t' is
first is
an x-tree,
corresponding to any limit set
k
that (w',t'(w'),...,t' (w'))
least fl(ft').
If ft'
€ {L 2
.
,
is
.
.
,
#
a path from
Lr =
appears as a term in the second
for distinct limit sets.
with x
ft'
ft},
sum on
Hence, that
sum
Z?(ft')
ft'
The
out of D(Q').
c(t")
c(i')
<
<:'(*, ft)
+
As
is
at least
is
at
R(Q) + £'=2 #(£«') Hence, we know
r-l
£
R(Li) - (R(Q) +
£
<
£(!<))
0.
i=2
second conclusion of the theorem, note simply that
for the
some
cost of such a path
a lower cost u;-tree as desired.
t" is
the argument above shows that the
for
such
ft'
the right hand side. Further, the terms are distinct
«=2
Thus,
and a w' €
the cost of each of the transitions in such a path
r-l
-
>
there exists a k
w
€
minimum
cost x-tree
is
if
c"(x, ft)
=
.R(fl),
then
no cheaper than than a w-tree
ft.
QED.
10
Conclusion
In this paper,
I
have discussed the behavior of stochastic models both in general and
With regard
several particular examples.
in
to the former, the paper outlines an approach
which involves describing the basins of attraction with two new measures, the radius and
coradius.
The main theorem
viously studied
is
applicable to
of the models which have been pre-
and expands our understanding of these models
a clear intuitive argument which
may
Freidlin- Wentzell tree constructions
converges.
many
The approach
the games involved
is
eliminate
much
in
two ways:
of the mystery
left in
and provides a measure of the rate
at
it
provides
the wake of
which a model
tractable in the examples discussed here despite the fact that
may have
best response cycles,
and that
in
one case the dynamics are
so complex as to render even a listing of the stable limit sets difficult.
As the
literature
on
stochastic evolution continues to grow, economic interest continues to
draw researchers
to
more complex games. Among the recent notable examples are Young's (1993b)
analysis of
bargaining, Noldeke and Samuelson's (1994) analysis of signaling games, and Binmore and
Samuelson's (1993) "muddling" model.
I
hope that both the argument of
this
paper
will
spur further research into such topics in the future both by reducing the burden of carrying
out proofs and by making analyses more complete and more transparent.
41
This paper
is
about models of evolution with
also a paper
primary result of the paper
is
noise.
In this area, the
that the selection of risk dominant equilibria in 2 x 2
generalizes to the selection of ^-dominant equilibria whenever they exist.
the long run equilibrium
is
more robust
to the specification of the
games
In these cases,
matching process and the
dynamics than can be generally assured. Several other remarks are also worth restating. In
games which are not weakly
when the Nash equilibrium
acyclic, long run equilibria
is
Despite lacking contagion dynamics, models with
unique.
two dimensional local interaction can be
like
long run limits and in that convergence
very rapid.
The paper may be
of
more general
stances in which evolutionary change
is
need not be Nash equilibria, even
is
one dimensional local models both
interest for the insight
likely to rapid.
it
in their
provides into the circum-
The argument
that shifts from one
equilibrium to another are most likely to be observed in systems which are amenable to
may be
gradual change
applicable in a wide variety of economic contexts
gradual change takes the form of shifts which occur
and where
it
first in
small subsets of the population
involves a continuous variable adjusting slowly between
In the future, the results of this paper might be extended in a
Among the most
— both where
two extremes.
number of
directions.
promising topics to explore are the possibility of extending the applicability
of the theorem by grouping limit sets, the possibility of developing tighter bounds on
waiting times by taking advantage some specific features of the dynamics rather than always
assuming the worst case, and the use of tree surgery arguments
theorem of
this
paper does not apply.
42
in situations
where the main
MIT LIBRARIES
3 9080 00940 0554
References
Anderlini, L.
and
Ianni, A. (1993):
"Path Dependence and Learning from Neighbors,"
mimeo, Cambridge University.
Bergin,
and Lipman, B. (1993): "Evolution with State- Dependent Mutations," mimeo,
J.,
Queens University.
Binmore, K., and Samuelson, L. (1993): "Muddling Through: Noisy Equilibrium Selection,"
mimeo, University of Wisconsin.
Blume,
nomic Behavior
Blume,
"The
L. (1993):
Statistical
Mechanics of Strategic Interaction," Games and Eco-
5, 387-424.
L. (1994):
"The
Statistical
Mechanics of Best- Response Strategy Revision," mimeo,
Cornell University.
Canning, D. (1992): "Average Behavior in Learning Models," Journal of Economic Theory
57, 442-472.
Ellison,
G. (1993):
"Learning, Local Interaction, and Coordination," Econometrica 61,
1047-1071.
Evans, R. (1993): "Observability, Imitation and Cooperation in the Repeated Prisoners'
Dilemma" working paper, University of Cambridge.
,
Foster, D.,
ical
and Young, H.
P. (1990): "Stochastic
Evolutionary
Game
Dynamics," Theoret-
Population Biology 38, 219-232.
Freidlin, M.,
and Wentzell, A. (1984): Random Perturbations of Dynamical Systems. New
York: Springer Verlag.
Hahn,
S. (1995):
"The Long Run Equilibrium
in
an Asymmetric Coordination Game,"
mimeo, Harvard University.
Harsanyi,
J.,
Cambridge:
Huck,
S.,
and Selten, R. (1988):
MIT
A
General Theory of Equilibrium Selection in Games.
Press.
and Oechssler,
J. (1995):
"The
Indirect
43
Approach to Explaining
Fair Allocations,"
mimeo, Humboldt University.
Kandori, M., Mailath, G., and Rob, R. (1993): "Learning, Mutation, and Long
libria in
Games," Econometrica
CARESS
A
General
Run Technology
Choice,"
Working Paper 92-06R.
Kandori, M., and Rob, R. (1993): "Bandwagon Effects and Long
University of
Equi-
61, 29-56.
Kandori, M., and Rob, R. (1992): "Evolution of Equilibria in the Long Run:
Theory and Applications,"
Run
Tokyo Discussion Paper 93-F-2.
Kim, Y. (1995):
"Equilibrium Selection in n- Person Coordination Games,"
Games and
Economic Behavior, forthcoming.
Maruta, T. (1995): "On the Relationship between Risk- Dominance and Stochastic
ty,"
Stabili-
mimeo, Kellogg-MEDS.
Morris,
S.,
Rob, R., and Shin, H. (1995): "p-Dominance and Belief Potential," Economet-
rica 63, 145-157.
Noldeke, G., and Samuelson, L. (1993):
"An Evolutionary Analysis
ward Induction," Games and Economic Behavior
5,
of
Backward and
For-
424-454.
Noldeke, G., and Samuelson, L. (1994): "Learning to Signal in Markets," mimeo, University
of Wisconsin.
Robson, A. and Vega Redondo, F. (1994):
Games with Random Matching," mimeo,
"Efficient Equilibrium Selection in Evolutionary
University of Western Ontario.
Samuelson, L. (1994): "Stochastically Stable Sets in
Games with
Alternative Best Replies,"
Journal of Economic Theory 64, 35-65.
Young, H.
P. (1993a):
"The Evolution of Conventions," Econometrica
Young, H.
P. (1993b):
"An Evolutionary Model
ory 59, 145-168.
44
61, 57-84.
of Bargaining," Journal of Economic The-
Date Due
,*'
Lib-26-67