The Evolution of Cooperation in a Hierarchically Structured,
Two-Issue Iterated Prisoner’s Dilemma Game:
An Agent-Based Simulation of Trade and War
1 October 2004
Abstract:
The study of the emergence of cooperation in anarchy has neglected to address the
implications of multiple issues linked within a hierarchical structure. In many social situations,
actors simultaneously (or sequentially) interact in several issue areas. For example, countries
typically interact with their neighbors on both security issues and economic issues. In this
situation, there is a hierarchy among the issues in that the costs and benefits in one issue (war)
surpass the costs and benefits in the other issue area (trade). This article explores the impact of
multiple, hierarchically structured issue areas on the level of cooperation in an iterated prisoner’s
dilemma game using a computer simulation. Three findings emerge from the analysis. First,
while altering the payoffs in only one issue area has a marginal impact on overall cooperation,
sustaining high levels of cooperation across the population requires improving incentives to
cooperate in both issue areas. Second, cooperation can be sustained even when the payoffs are
based on relative power. Third, the rate of learning is critical for the emergence and spread of
cooperative strategies.
David L. Rousseau
Assistant Professor
Department of Political Science
235 Stiteler Hall
University of Pennsylvania
Philadelphia, PA 19104
E-mail: rousseau@sas.upenn.edu
Phone: (215) 898-6187
Fax: (215) 573-2073
and
Max Cantor
Department of Political Science
and the School of Engineering and Applied Science
University of Pennsylvania
Philadelphia, PA 19104
E-mail: mxcantor@sas.upenn.edu
1
INTRODUCTION
Since Axelrod’s (1984) landmark study, we have learned a lot about the evolution of
cooperation and the utility of agent-based computer simulations to explore the evolutionary
process.1 One issue that has been neglected in this growing literature is the impact of multiple
issue areas on the emergence of cooperation. In many social situations, actors simultaneously (or
sequentially) interact in several issue areas. For example, countries typically interact with their
neighbors on both security issues and economic issues. In this example, there is a hierarchy
among the issues in that the costs and benefits in one issue (war) surpass the costs and benefits in
the other issue area (trade). This article explores the impact of multiple, hierarchically structured
issue areas on the level of cooperation in an iterated prisoner’s dilemma game using a computer
simulation. Three findings emerge from the analysis. First, while altering the payoffs in only one
issue area has a marginal impact on overall cooperation, sustaining high levels of cooperation
across the population requires improving incentives to cooperate in both issue areas. Second,
cooperation can be sustained even when the payoffs are based on relative power. Third, the rate
of learning is critical for the emergence and spread of cooperative strategies.
While our findings are generalizable to any hierarchically linked issue areas, we focus on
the implications of multiple issue areas in international relations. Over the last several centuries
the sovereign state has emerged as the dominant organizational unit in the international system
(Spruyt 1994). During this evolutionary period, sovereign states and their competitors have
struggled to identify an optimal strategy for maximizing economic growth and prosperity. In
general, states have pursued some combination of three general strategies: (1) war, (2) trade, or
(3) isolation. For example, realists such as Machiavelli (1950) argued that military force is an
effective instrument for extracting wealth and adding productive territory. In contrast, liberals
such as Cobden (1850, 518) argued that international trade was the optimal strategy for
maximizing economic efficiency and national wealth. Finally, economic nationalists such as
Hamilton (Earle 1986) rejected this liberal hypothesis and argued that isolation from the leading
2
trading states rather than integration with them would enhance economic development. The three
policies are not mutually exclusive. For example, List promoted both military expansion and
economic isolation for Germany in the 19th century (Earle 1986). However, in general, trade and
war are viewed as hierarchical because states rarely trade with enemies during military conflict.
While some instances have been identified (e.g., Barbieri and Levy 1999), many systematic
studies indicated that military conflict reduces international trade (e.g., Kim and Rousseau 2004).
The importance of issue linkage has long been recognized. Axelrod and Keohane (1986,
239) argued that issue linkage could be used to alter incentive structures by increasing rewards or
punishments. Although this idea of issue linkage has been explored with rational choice models
(e.g., Morgan 1990; Lacy and Niou 2004) and case studies (e.g., Oye 1986), modeling issue
linkage as a two-stage game within an N-person setting has largely been neglected.2
Conceptualizing trade and war as generic two stage game raises a number of interesting
questions that will be explored in this article. If trade and war are the two most important issues
in international relations, how does the hierarchical linkage between the two issues influence the
amount of cooperation in the international system? What factors encouraged (or discouraged) the
rise of a cooperative order? What strategies might evolve to facilitate the maximization of state
wealth in a two issue world? Finally, how stable are systems characterized by hierarchical
linkages?
COOPERATION IN ANARCHY
Many interactions among states are structured as “Prisoner’s Dilemmas” due to order of
preferences and the anarchical environment of the international system. The Prisoner’s Dilemma
is a non-zero-sum game in which an actor has two choices: cooperate (C) with the other or defect
(D) on them. The 2x2 game yields four possible outcomes that can be ranked from best to worst:
1) I defect and you cooperate (DC or the Temptation Payoff "T"); 2) we both cooperate (CC or
the Reward Payoff "R"); 3) we both defect (DD or the Punishment Payoff "P"); and 4) I cooperate
and you defect (CD or the Sucker's Payoff "S"). The preference order coupled with the
3
symmetrical structure of the game implies that defection is a dominant strategy for both players in
a single play game because defecting always yields a higher payoff regardless of the strategy
selected by the opponent. Therefore, the equilibrium or expected outcome for the single play
prisoner’s dilemma game is “defect-defect” (i.e., no cooperation). This collectively inferior
equilibrium is stable despite the fact that both actors would be better off under mutual
cooperation. The problem is neither actor has an incentive to unilaterally alter their selected
strategy because they fear exploitation.3
Many situations in international economic and security affairs have been models as
iterated Prisoner’s Dilemmas. In the trade arena, researcher tend to assume states have the
following preference order from “best” to “worst”: 1) I raise my trade barriers and you keep yours
low; 2) we both keep our trade barriers low; 3) we both raise our trade barriers; and 4) I keep my
trade barriers low and you keep yours high. In the security sphere, scholars typically assume that
states possess the following preference order: 1) you make security concessions, but I do not; 2)
neither of us makes security concessions; 3) both of us make security concessions; and 4) I make
security concessions but you do not. While scholars have identified particular instances in which
preference orders may deviate from these norms (e.g., Conybeare 1986 and Martin 1992), the
Prisoner’s Dilemma preference order is typical of many, if not most, dyadic interactions in the
two issue areas.4
Students of game theory have long known that iteration offers a possible solution to the
dilemma (Axelrod 1984, 12). If two agents can establish a cooperative relationship, the sum of a
series of small "Reward Payoffs" (CC) can be larger than a single "Temptation Payoff" followed
by a series of "Punishment Payoffs" (DD). The most common solution for cooperation within an
iterated game involves rewards for cooperative behavior and punishments for non-cooperative
behavior -- I will only cooperate if you cooperate. The strategy of Tit-For-Tat nicely captures the
idea of conditional cooperation. A player using a Tit-For-Tat strategy cooperates on the first
move and reciprocates on all subsequent. Axelrod (1984) argues that the strategy is superior to
4
others because it is nice (i.e., cooperates on first move allowing a CC relationship to emerge),
firm (i.e., punishes the agent's opponent for defecting), forgiving (i.e., if a defector returns to
cooperation, the actor will reciprocate), and clear (i.e., simple enough for the agent's opponent to
quickly discover the strategy). Leng (1993) and Huth (1988) provide empirical evidence from
international relations to support the claim that states have historically used reciprocity oriented
strategies to increase cooperation. Although subsequent work has demonstrated that Tit-For-Tat
is not a panacea for a variety of reasons (e.g., noise, spirals of defection, sensitive to the
composition of the initial population), reciprocity oriented strategies are still viewed as viable
mechanisms for encouraging cooperation under anarchy.5
Although the emergence of cooperation can be studied using a variety of techniques (e.g.,
case studies, regression analysis, laboratory experiments), in this article we employ an approach
that has been widely used in this literature: an agent-based computer simulation (Axelrod 1997;
Macy and Skvoretz 1998; Rousseau 2005). Agent-based simulations are “bottom-up” models that
probe the micro-foundations of observable macro-patterns. Agent-based models make four
important assumptions (Macy and Willer 2002, 146). First, agents are autonomous in that no
hierarchical structure exists in the environment. Second, agents are interdependent in that their
welfare depends on interactions with other agents. Third, agent choices are based on simple rules
utilizing limited information rather than complex calculations requiring large amounts of data.
Finally, agents are adaptive or backward looking (e.g., if doing poorly in last few rounds, change
behavior) as they alter characteristics or strategies in order to improve performance. Agent-based
models are ideally suited for complex, non-linear, self-organizing situations involving many
actors. Macy and Willer claim that agent-based models are “most appropriate for studying
processes that lack central coordination, including the emergence of organizations that, once
established, impose order form the top down” (2002, 148).
As with all methods of investigation, computer simulations have strengths and
weaknesses.6 On the positive side of the ledger, five strengths stand out. First, as with formal
5
mathematical models, simulations compel the researcher to be very explicit about assumptions
and decision rules. Second, simulations allow us to explore extremely complex systems that often
have no analytical solution. Third, simulations resemble controlled experiments in that the
researcher can precisely vary a single independent variable (or isolate a particular interaction
between two or more variables). Fourth, while other methods of inquiry primarily focus on
outcomes (e.g., do democratic dyads engage in war?), simulations allow us to explore the
processes underlying the broader causal claim (e.g., how does joint democracy decrease the
likelihood of war?). Fifth, simulations provide a nice balance between induction and deduction.
While the developer must construct a logically consistent model based on theory and history, the
output of the model is explored inductively by assessing the impact of varying assumptions and
decision rules.
On the negative side of the ledger, two important weaknesses stand out. First, simulations
have been criticized because they often employ arbitrary assumptions and decision rules (Johnson
1999, 1512). In part, this situation stems from the need to explicitly operationalize each
assumption and decision rule. However, it is also due to the reluctance of many simulation
modelers to empirically test assumptions using alternative methods of inquiry. Second, critics
often question the external validity of computer simulations. While one of the strengths of the
method is its internal consistency, it is often unclear if the simulation captures enough of the
external world to allow us to generalize from the artificial system we have created to the real
world we inhabit.
Given that we are primarily interested in testing the logical consistency and
completeness of arguments, the weaknesses are less problematic in our application of agent-based
model. That is, we are interested in probing whether traditional claims (e.g., increasing the
reward payoff increases cooperation or relative payoffs decrease cooperation) produce expected
patterns when confronting multiple, hierarchically structured issues. While we use the trade/war
setting to illustrate the applicability of the model, we do not claim to be precisely modeling the
6
international trade or interstate conflict. This is not to say that using agent-based simulations to
model real world interactions is impossible or unimportant. We simply wish to use the technique
to evaluate general claims about factors promoting or inhibiting cooperation.
OVERVIEW OF THE MODEL
In our agent-based model, the world or "landscape" is populated with agents possessing
strategies that are encoded on a string or “trait set” (e.g., 00010010100110011001). Over time
the individual traits in the trait set (e.g., the “0” at the start of the string) change as less successful
agents emulate more successful agents. The relatively simple trait set with twenty individual
traits employed in the model allows for over 1 million possible strategies. Presumably, only a
small subset of these possible combinations produces coherent strategies that maximize agent
wealth.
The structure of our model was inspired by the agent-based model developed by Macy
and Skvoretz (1998). They use a genetic algorithm to model the evolution of trust and strategies
of interaction in a prisoner’s dilemma game with an exit option. Like us, they are interested in
the relative payoff of the exit option, the location of interaction, and the conditionality of
strategies. Our model, however, differs from theirs in many important respects. First, unlike
their single stage game, our model is a two stage prisoner’s dilemma game that links two issues in
a hierarchical fashion (i.e., both a “war” game and a “trade” game). Second, our trait set differs
from theirs because we have tailored it to conform to standard assumptions about trade and war.
In contrast, their trait set is more akin to “first encounter” situations (e.g., do you greet partner?
do you display marker? do you distrust those that display marker?). Third, our model allows for
complex strategies such as Tit-For-Tat to evolve across time.
Our simulation model consists of a population of agents that interact with each other in
one of three ways: 1) trade with each other; 2) fight with each other; or 3) ignore each other.
Figure 1 illustrates the logic of each of these encounters. The game is symmetrical so each actor
has the same decision tree and payoff matrices (i.e., the right side of the figure is the mirror image
7
of the left side of the figure). Each agent begins by assessing the geographic dimension of the
relationship: if the agents are not immediate neighbors, then the agent skips directly to the trade
portion of the decision tree. If the two agents are neighbors, the agent must ask a series of
questions in order to determine if it should enter the war game. If it chooses not to fight, it asks a
similar series of questions to determine if it should enter the trade game. If it chooses neither war
nor trade, it simply exits or "ignores" the other agent. Both the war and the trade games are
structured as prisoner's dilemma games.
**** insert Figure 1 about here ****
The model focuses on learning from one's environment. In many agent-based
simulations, agents change over time through birth, reproduction, and death (Epstein and Axtell
1996). In such simulations, unsuccessful agents die as their wealth or power declines to zero.
These agents are replaced by the offspring of successful agents that mate with other successful
agents. In contrast, our model focuses on social learning.7 Unsuccessful agents compare
themselves to agents in their neighborhood. If they are falling behind, they look around for an
agent to emulate. Given that agents lack insight into why other agents are successful, they simply
imitate decision rules (e.g., don't initiate war against stronger agents) selected at random from
more successful agents. Over time repeatedly unsuccessful agents are likely to copy more and
more of the strategies of their more successful counterparts. Thus, the agents are “boundedly
rational” in that they use short cuts in situations of imperfect information in order to improve their
welfare (Simon 1982).8
The fitness of a particular strategy is not absolute because its effectiveness depends on
the environment in which it inhabits. Unconditional defection in trade and war is a very effective
strategy in a world populated by unconditional cooperators. However, such an easily exploited
environment begins to disappear as more and more agents emulate the more successful (and more
coercive) unconditional defectors. While this implies that cycling is possible, it does not mean it
8
is inevitable. As the simulation results demonstrate, some populations are more stable across
time because they are not easily invaded by new strategies. .
In the simulation, trade and war are linked in four ways. First, agents cannot trade with
other agents if they are in a crisis or at war. A crisis emerges when one side exploits the other in
the war game (i.e., DC or CD outcome in the war game). A war occurs when both sides defect in
the war game (i.e., DD). Second, agents can adopt strategies that either mandate or prohibit
going to war with current trading partners. Third, while agents can fight and trade with
immediate neighbors, they can only trade with non-contiguous states. Although this rule is
obviously a simplification because great powers are able to project power, it captures the fact that
most states are capable to trading but not fighting at a great distance. Fourth, while an "exit"
option exists with respect to trade (i.e., you can choose not to enter a relationship with someone
you don't trust), an agent cannot exit from a war relationship because agents can be a victim of
war whether or not they wanted to participate.
THE WAR-TRADE-ISOLATION SIMULATION
Figure 2 illustrates the basic structure of the war-trade-isolation simulation. The
population consists of a wrapping 20 x 20 landscape or grid of agents. Each of the 400 individual
agents is surrounded by eight immediate neighbors (referred to as the Moore 1 neighborhood in
the simulation literature). Each agent in the landscape has a set of traits that determine the
agent’s characteristics and rules for interacting with other agents. In Figure 2, we see the
distribution of the Trait #1 shown in black over the entire landscape. The structure of the actor
and its northeastern neighbor are highlighted in the Figure in order to illustrate the properties of
the agents. In the Figure, the actor's War Character (Trait #1) is "Cooperate" (or "0" shown in
white in the figure) and the neighbor's War Character is "Defect" (or "1" shown in black). The
Figure indicates that war-like agents tend to cluster spatially and are in a minority in this
particular landscape. The simulation is executed for a number of iterations or "runs." In each
iteration, the actor interacts with neighbors, updates their power based on outcomes of
9
interactions, and (possibly) updates traits by adopting those of more successful neighbors.9
Therefore, the trait landscapes update slowly over time as the more successful agents are
emulated by their less successful neighbors.
**** insert Figure 2 about here ****
Figure 3 displays the payoff structure for each interaction. During each iteration of the
simulation, the agent interacts with the eight members of its Moore 1 neighborhood. Nine
outcomes of the interactions are possible: ignoring (exiting prior to trade), trading (DC, CC, DD,
CD), or fighting (DC, CC, DD, CD). For example, if both states cooperate in the trade game they
each receive a payoff of "1". Conversely, if they both defect in the war game they each receive a
payoff of "-5". The exit payoff always falls between the DD payoff and the CC payoff in the
trade game. The ExitPayoff parameter, which varies from 0 to 1, determines the exit payoff
between these endpoints. So a setting of 0.50, which is the default in the simulation, sets the exit
payoff as the midpoint between the DD minimum (0) and the CC maximum (1). As with all
parameters in the model, the user is free to manipulate the parameter in order to explore the
model.10
**** insert Figure 3 about here ****
Four points about the payoff structure should be highlighted. First, the payoffs in the
model must be symmetrical because the user specifies a single matrix for all actors. While this is
an obvious simplification, it is a standard assumption in most of the formal and simulation work
employing a prisoner's dilemma.11 Second, the gains from the temptation payoff and the losses
from the sucker's payoff in the trade game are always less than for the analogous payoffs in the
war game. This captures the hierarchical nature of the relationship between trade and war.
Moreover, it reflects the widely held belief that the stakes are lower in the economic realm
compared to the military realm (Lipson 1984; Stein 1990, 135; Rousseau 2002). Third, the
simulations in this paper always set the DD outcome in the trade game to zero and the CC
outcome in the war game to zero. While this is not necessary, it implies that peacetime and no
10
trade are the status quo outcomes in which states neither lose nor gain. Fourth, the payoffs in the
simulations presented below meet the two requirements for a prisoner's dilemma game (i.e.,
DC>CC>DD>CD and CC>(DC+CD)/2).
Trait Sets
Each agent has a "trait set" that defines it properties and specifies many (but not all)
interaction rules. In our model, each agent's trait set has 24 traits (four of which are currently
blank in that they do not affect behavior). Thus, agents can search over a million possible strings
in order to locate the most useful strategy. In general, the first half of the trait set governs
behavior in the war game and the second half of the trait set governs behavior in the trade game.
Table 1 summarizes the trait names, trait numbers, attributes, and trait descriptions. For example,
the War Character trait (#1) determines a state’s preferred strategy in the war game. Some
agents prefer to defect in order to exploit their partner or to protect themselves from others. Other
agents prefer to cooperate when entering the war game.
**** insert Table 1 about here ****
The War: Unconditional trait (#2) determines whether or not the War Character trait is
applied unconditionally in all circumstance. If the attribute specifies a conditional strategy, the
agent may defect in general unless another trait specifies when it should cooperate. For example,
an agent with War Character trait equal to 1 (i.e., defect) may cooperate when facing a stronger
agent (War: Power trait (#6)=0), a trading partner (War: Interdependence trait (#7)=1), or a
member of the same type (War: Type trait (#8)=1).
The War: Behavior w/ Me trait (#3) is the first of three traits that allows agents to use the
past behavior of the other agents to determine whether or not to cooperate. A simple Tit-For-Tat
decision rule involves cooperating on the first move of the interaction and reciprocating on all
subsequent moves (i.e., cooperate if they cooperate and defect if they defect). Tit-For-Tat
requires that the actor record one piece of historical information: what did the opponent do in the
last round? If the War: Behavior w/ Me attribute equals 0, then the state does not use any
11
historical information to guide current behavior. However, if the attribute equals 1, the state does
look at past behavior. But how far back should one look? Although the simplest form of Tit-ForTat looks back only one round, more complex strategies are available (e.g., Tit-For-2-Tats).12 In
our simulation, we address this question by creating a Memory trait (#23) that determines how far
back an agent should look. It is conceivable that a bit of memory (e.g., 1 or 2 rounds) helps
cooperation emerge but that a lot of memory (9 or 10 rounds) severely limits the establishment of
cooperative relations (Hoffmann 2000, 3.3). This trait, like all the others, evolves over time
allowing agents to locate an efficient memory length.
Should agents only look at dyadic behavior? For example, does the United States only
assess China on the basis of American-Chinese interactions? Or does its assessment of China
include its behavior with Japan, Taiwan, and South Korea? The War: Behavior w/ Common
Neighbors trait (#4) and the War: Behavior w/ All trait (#5) gradually increase the number of
others agents considered in the decision process. In the simulation, agents have perfect
information on the past behavior of other agents and can decide whether or not to include this
information in the decision calculus.13
The sequence of trade traits directly parallels the sequences of the war traits just
discussed with a single exception: the Trade: Neighbors trait (#18). This trait, which becomes
important in conditional strategies, determines if neighbors should be treated differently. The fact
that states can only go to war with immediate neighbors, implies that they might be more
concerned about trade (and the potential for relative gains) with these potential adversaries.
After the trade traits, the trait set contains two traits related to agent “type”. Agents have
a type if their War Character and Trade Character are identical. That is, the model allows for a
cooperative type and a non-cooperative type (agents with mixed traits do not have a type). This
raises two interesting questions. First, should you signal your type to others? For example, if
you are a cooperative agent, do you want to signal this fact to others? If a cooperative agent
signals its type to another cooperative agent, the probability of establishing a mutually
12
cooperative relationship is enhanced. However, if a cooperative agent signals its type to a noncooperative agent, it opens itself up for exploitation. Conversely, if a defector displays its type, it
could both deter attacks on it by other defectors and warn cooperators of coming exploitation.
Second, should agents use type to help them determine an optimal strategy? The Display Type
trait (#21) and the Detect Type trait (#22) determine whether or not an agent uses type when
choosing whether to defect or cooperate.
Finally, the Military Propensity trait (#24), which is the only continuous gene other than
Memory trait, determines how likely a state is to initiate an unprovoked attack on another state.
While defensive realists and offensive realists may both possess the "defect" attribute on the War
Character trait, only the offensive realists would be expected to have a high Military Propensity
probability. A landscape populated by agents with high Military Propensity is expected to have
high rates of war. The intriguing question is whether states with a high military propensity thrive
over time.
In the simulation, traits can be changed in one of two ways: 1) mutation and 2) learning.
In the default simulations, there is a 0.001 probability that each trait mutates (i.e., flips from “1”
to “0”) in each iteration. While raising the mutation rate increases the likelihood of an agent
zeroing in on an optimal strategy by chance, it introduces volatility into the landscape because
quite fit agents can suddenly become ill equipped to deal with the neighborhood. In terms of
learning, the simulation permits three decision rules: (1) always copy from the most successful
agent in the neighborhood; (2) if below the mean power in the immediate neighborhood,
randomly select anyone above the mean power level in the neighborhood; and (3) if the worst in
the neighborhood in terms of power, randomly select anyone else in the neighborhood. The
“select the best” rule causes rapid convergence and the “if the worst” rule causes very slow
convergence. The intermediate “if below look above” rule is employed in the default simulations.
Once a “better” agent is selected for imitation using any one of the decision rules, agents copy on
a gene by gene basis with a 0.50 probability.14 While copying trait by trait rather than by blocks
13
slows the identification of an optimal strategy, it conforms to the idea that states “muddle”
through rather than “optimize.”
The War and Trade Modules
While Figure 1 provided a rough overview of the model, we did not spell out the precise
logic of each step in the decision calculus. The model contains four detailed decision trees, which
appear in Figures 4 through 7, labeled the "War Choice" module, the "War Strategy" module, the
"Trade Choice" module, and the "Trade Strategy" module. The "choice" modules specify the
rules for entering either the war game or the trade game. The "strategy" modules specify the rules
for choosing a particular strategy (i.e., cooperate or defect) once the actor has entered a game.
**** insert Figures 4 through 7 about here ****
The War Choice Module in Figure 4 identifies the sequence of questions an agent asks in
order to determine if war is preferred. Agents with unconditional strategies (as determined by
trait #2) immediately cooperate or defect based on their War Character trait (#1). Agents with
conditional strategies begin by examining the behavior of the other state with the agent itself,
with common neighbors, and with all other agents in the landscape. For example, if the agent has
a "1" on the War Behavior w/Me trait (#3) and a Memory trait (#23) of 10, then the agent looks to
the last ten iterations between the agent and the other. If the other has defected 5 of the last 10
times with the agent, there is a 50% change that the agent will defect. This probability will be
influenced in an analogous manner by the War Behavior w/ Common Neighbors and War
Behavior w/ All traits (#4 and #5 respectively). For example, if an actor has a “1” for all three
traits, then the probability of “trust” is simply one minus the sum of the average rates of defection
divided by three.
The War Strategy Module in Figure 5 identifies three sequential questions asked by the
agent in order to determine if a conditional strategy is appropriate for the interaction. The first
question focuses on the current balance of power: is the other agent weaker than me? If the other
14
agent is weaker, then the agent must ask if it treats weaker agents differently (i.e., trait #6=1).
For example, suppose an agent had a War Character trait indicating "cooperate" but a War:
Power trait indicating "treat weaker states differently." In this case, the agent would cooperate
with stronger agents but attempt to exploit weaker agents by defecting. A similar procedure is
followed for the trade question (Did I trade with the other agent in the last round?) and the type
question (Is the other agent a similar type?). The Trade Choice and Trade Strategy modules
shown in Figures 6 and 7 follow the same basic logic as the war modules.
THE HYPOTHESES
The simulation model contains approximately 30 parameters that can systematically
varied by the user. These parameters correspond to the independent variables in the causal
relationship because varying the parameter (e.g., the CC payoff in the trade game from low to
medium to high) may have a significant impact on a particular dependent variable. The
dependent variables included in the model and stored in the output files are 1) the number of wars
per agent in the landscape, 2) the number of trades per agent in the landscape, 3) the mean level
of power in the landscape; 4) the inequality of power in the landscape; 5) the prevalence of each
trait in the population; 6) the percentage of trade with neighbors; 7) the clustering of CC agents;
8) the percentage of dyads “exiting”; and 9) the average military propensity. Although the
simulation model can be used to test a wide variety of hypotheses, we will limit our analysis to
four due to space due limitations.
Hypothesis #1 predicts that increasing the gains from trade relative to the gains from war
will increase the amount of cooperation in the trade and war games. Historically, the gains from
trade were limited by the high cost of transportation and limited specialization. However, the
industrial revolution dramatically reduced transportation costs and increased specialization in
labor and capital.15 As the gains from trade increased during this era, the international economic
system experienced an increase in the growth of trade and treaties designed to lower trade
barriers. The annual average growth in international trade grew from 1.10 percent in 1720-1780
15
to 4.84 percent in 1840-1860 and 3.24 percent in 1870-1900 (Rostow 1978, 67). The signing of
the Anglo-French Cobden-Chevalier Treaty in 1860 was followed by a series of similar
agreements which paved the way for the most open trading era in European history (Kindleberger
1975). Although many bumps appeared along the way (e.g., the structural adjustments of 18731896 and the global depression of the 1930s), international trade and investment has grown
rapidly over the last one hundred and fifty years (Held 1999, 149, 189). Within the simulation,
we test Hypothesis #1 by increasing the CC trade payoff from 1 to 3 and decreasing the cost of
the sucker’s payoff from -4 to -1. By decreasing the gaps between the two highest payoffs and
the two lowest payoffs, we should decrease both the incentive to exploit and the fear of being
exploited (Jervis 1978; Rousseau 2002). Moreover, by keeping the war payoffs constant, the rise
in the gains from trade inherently makes trading more attractive than in the baseline situation. .
Hypothesis #2 predicts that offense dominance increases the average number of wars
(Jervis 1978; Van Evera 1984); the rise in conflict should simultaneously decrease the average
amount of trade cooperation. When military technology favors the offense, attacking is relatively
easy and defending is relatively difficult. The attacking state can exploit the advantage of
surprise and dictate the course of the conflict to the retreating enemy. According to Jervis,
offense dominance increases the reward of the temptation payoff and/or the cost of the sucker's
payoff (i.e., increases the gap between the DC and CC payoff and/or increases the gap between
the DD and CD payoff (1978, 171). By increasing the temptation payoff, we increase the
incentive to unilaterally defect in the hope of exploiting another state. By increasing the cost of
the sucker’s payoff, we make it very difficult for a state to choose cooperation because it fears the
dire cost of exploitation. In the simulation, we test Hypothesis #2 by doubling both the positive
DC payoff and the negative CD payoff in the war game.
Hypothesis #3 predicts that a shift from absolute to relative payoffs will decrease the
amount of trade and increase the amount of war. In the default simulation, the trade payoffs are
absolute values based in the user input. For example, the CC payoff in the default trade game
16
provides each party with 1 power unit. This absolute payoff assumption has been implicitly
adopted by the vast majority of liberal and neo-liberal scholars (e.g., Axelrod 1984 and Baldwin
1993). However, economic nationalists such as List and dependence theorists such as Frank
(1979, 173) argue that the gains in trade accrue disproportionately to the stronger actor in the
interaction. If true, trade should increase inequality in the landscape and increase the amount of
war as stronger agents can use their economic power to augment their military capability. We
can test this hypothesis by turning the UseAbsolutePayofffs function “off” in the parameter table.
Now, if agent A has 1000 power units and agent B has 3000 power units, mutual cooperation in
the trade game would result in a 0.25 payoff for agent A and a 0.75 payoff for agent B.
Hypothesis #4 predicts that faster learning will increase cooperation in the trade game
and decrease conflict in the war game. As describe above, the simulation permits three learning
rules: 1) “only learn from the best”; 2) “if below mean look above mean”; and 3) “if the worst,
learn from anyone.” Given that there are over a million possible strategies possible with a string
of 20 traits, more efficient learning will allow agents in the landscape to diffuse successful
strategies across the population faster. If cooperation tends to emerge in small clusters as
Axelrod (1984) suggests, then cooperative cluster should perform better than average and an
“only learn form the best” rule should spread the successful strategy more quickly.
THE RESULTS OF THE TRADE-WAR SIMULATION
Figure 8 displays the output of a typical simulation using the default settings and the
random seed set to 1. Panel (A) displays the “trait landscape” for the twenty traits which were
listed in Table 1.16 The landscape in the upper left hand corner indicates that the War Character
trait (#1) is just about evenly split between defectors (attribute=1, shown in black) and
cooperators (attribute=0, shown in white). However, the panel reveals that the traits tend to
cluster spatially so that those with the “cooperate” attribute tend to cluster near each other as do
agents with the “defect” attribute.
17
Panel (B) displays the prevalence of wars in the landscape. In this panel, actors are
shown as nodes in a network (rather than as squares on a lattice as in the previous panel). Each
black line connecting two agents indicates that they are engaged in a war (i.e., a DD outcome in
the war game). Each red line represents agents in a crisis (i.e., either a CD or DC outcome). The
limited number of lines connecting nodes indicates that the number of wars in the landscape is
relatively low (0.9 wars per dyad at this snapshot at iteration 5000).
Panel (C) displays the prevalence of trade (show by green lines connecting agents) and
indicates that trade is just about as infrequent as war (0.08 trades per dyad at this snapshot at
iteration 5000). The vast majority of agents in this simulation have chosen to “exit” in order to
isolate themselves from the anarchic system. Both Panels (B) and (C) color code the agents
based on their War Characters and Trade Characters (green for Ctrade-Cwar; red for Dtrade-Dwar;
blue for Dtrade-Cwar; and black for Ctrade-Dwar). The color patterns in the panels clearly indicates
that cooperative agents tend to cluster spatially (even when they are not trading with each other).
Although trade is very limited, it is apparent that trade tends to cluster in small groups.
Panel (D) displays a landscape of the average level of power. States receive an
endowment of power at the initialization of the simulation. The initial power is a random number
drawn from a uniform distribution bounded by a minimum and maximum. In the default
simulation, these bounds are set at 10,000 in order to begin the simulation with an even
distribution of power. In worlds in which total economic growth is positive, the average power
should increase across time. As an actor’s wealth increases above the initial baseline, the color of
the landscape Panel D of the figure becomes darker blue. If the actor’s wealth falls, it becomes
increasingly red. The prevalence of red in the panel indicates that wealth has declined drastically
for most of the agents in the landscape.
Panel (E) tracks the percentage of individual traits across time. Although any (or all) of
the 22 traits can be tracked simultaneously across time, we have selected four traits (#1, #2, #12,
and #13) for the graphs used in this section. The rising red and aqua lines, which track the
18
percentage of agents having #1 War Character “defect” (red) and #12 Trade Character “defect”
(aqua) indicate that defectors have slowly become a majority over time. The falling blue and
black lines, which track the percentage of agents having #2 War Unconditional “yes=1” (blue)
and #13 Trade Unconditional “yes=1” (black), indicate that states are increasingly adopting
conditional strategies. For example, an agent with a trade character “defect” and a conditional
strategy may cooperate with weaker states or states with a similar type. The panel indicates that
learning is slow and differentiation is only beginning to emerge after 5000 iterations. The
variance is also increasing over time as the lines begin to oscillate more.
Panel (F) tracks three of the nine available dependent variables across time: 1) the
average number of successful trades (CC) per dyad shown in red; 2) the average number of wars
(DD) per dyad shown in blue; and 3) the Gini coefficient shown in aqua. The Gini coefficient
measures the power inequality in the landscape on a scale from zero (perfect equality) to one (one
actor has all the power in the system). For comparative purposes, the U.S. Census Bureau
calculated a Gini coefficient of 0.456 for the United States in 1994. In the panel, the Gini
coefficient rises to above 0.90, indicating that a few agents control most of the wealth in the
system.
Panel (F) also indicates that warfare rose early in the simulation and then slowly declined
across time. In contrast, trade rose slowly to about 0.10 trades per dyad and then oscillated
between 0.10 and 0.05 for the rest of the simulation. The gradual decline of war occurred
because agents engaging in this activity tended to do worse in terms of accumulating power than
agents that avoided war. For example, the military propensity is a randomly distributed trait at
the start of the simulation with a mean of 0.50. This implies that agents initiate lots of
unprovoked attacks. This turns out to be a costly policy, so slowly over time this trait is driven
toward zero. Thus, the total amount of war in the system declines as agents learn to be more
selective in their war involvement. However, war never totally disappears and (as we shall see)
can spike up again when the situation is favorable to the use of military force.
19
**** insert Figure 8 about here ****
Hypothesis #1 predicts that increasing the gains from trade will increase the average
number of trades and decrease the average number of wars. Operationally, this involves
increasing the trade reward payoff from -1 to +3 and shifting the trade punishment from -4 to -1.
As with several of the hypotheses, we probe this claim graphically by comparing representative
runs using graphs similar to those shown in Figure 8. In the graphics we typically use a random
seed of 1 in order to isolate the impact of the change in parameter settings. Given that each
simulation run represents just one of many possible outcomes, we also report T-tests based on
thirty runs with random seeds lasting 5,000 iterations. This allows us to determine whether the
differences in the average amount of trade or the average amount of war are statistically
significant.
**** insert Figure 9 about here ****
The representative run in Figure 9 indicates that making trade more attractive has a
moderate impact on trade. Panel F shows that the number of trades has increases slightly above
the number of wars per dyad. Panel F also shows that trade helps reduce inequality in the system
as shown by the Gini coefficient peaks at just below .80 before falling slightly. The slight
increase in trade has not influenced the average amount of wealth in the landscape; Panel D
shows that the average power has fallen for the vast majority of actors over the course of the 5000
runs. T-tests based on 30 runs indicate that average trades per dyad rises from 6.3% to 15.3% and
the average number of wars per dyad falls slightly from 12.6% to 12.0%. However, only the
increase in trade is statistically significant (at better than the .001 level of significance).
Hypothesis #2 predicts that increasing offense dominance in war will increase the
average number of wars and decrease the average number of trades. Operationally, this involves
doubling the temptation payoff (10 to 20) and the sucker’s payoff (-11 to -22). Given that the
graphical output does not change dramatically, we will focus on the average results from 30 runs
of the simulation. Offense dominance increases the average number of wars per dyad as expected
20
from 12.6% to 15.6%. However, the average number of trades actually rises from 6.3% to 7.5%.
Both results are statistically significant at better than the .001 level. Why might trade increase in
the offense dominant world? Although the differences are small, the rise in trade appears to be
due to the fact that the prevalence of warfare increases the amount of poverty in the system (mean
power falls from 1264 to 949). War is a great gamble because agents that are prone to warfare
either win big gains or suffer big losses. Over time, this violent environment seems to favor those
agents that shun warfare for trade.
Hypothesis #3 predicts that relative payoffs (i.e., strong states get more gains from trade)
will decrease the average amount of trade, decrease average wealth, and increase inequality.
Operationally, this test involves weighting the payoffs by the ratio of the agent A’s power divided
by the sum of agent A’s power and agent B’s power. As expected, the average number of CC
trade with relative gains is significantly less than with absolute gains (0.044 vs. 0.082; significant
at better than the 0.001 level). However, contrary to expectations, relative payoffs increase
average wealth (1265 vs. 1355; significant at better than the 0.001 level). Inequality remains
virtually unchanged. This finding is consistent with earlier analysis of Axelrod’s computer
tournament (Busch and Reinhardt 1993). We will return to the unexpected impact of relative
payoffs in a moment.
Hypothesis #4 predicts that faster agent learning will result in more trade and less war.
Operationally, the hypothesis is tested by shifting the learning strategy from “if below mean look
above mean” to “copy from the best.” That is, all agents learn from the most successful agent in
their local (i.e., Moore 1) neighborhood. Figure 10 displays the results of the simulation using the
“learn from the best” update rule. Panel (A) indicates that fast learning creates more
homogeneity in neighborhoods. In comparison to “snowy” images in Panel (A) of Figure 9, the
current distribution of agents in Panel (A) of Figure 10 reveals large clusters of black agents and
large clusters of white agents with clear boundaries between the two. Although the snap shots at
iteration 5000 in Panels (B), (C), and (D) do no reveal striking differences from previous figures,
21
the time series presentations in Panel (E) and (F) indicate a much more dynamic world when
learning is more efficient. Panel E shows great oscillations in the dominant traits. This occurs
because strategy strings with a slight advantage can spread rapidly throughout the population.
For example, the emergence unconditional traders about two-thirds of the way through the
simulation (see Panel E) triggers a rapid increase in average trade per dyad (see Panel F).
However, this trading world collapses with the sudden proliferation of unconditional (black)
defectors (aqua) in the landscape. Across the 30 runs of the simulation, we find that the average
number of trades increases from 6.3% in the baseline to 18.8%, the average number of wars
decreases from 12.6% to 10.1%, and average wealth rose from 1,264 to 2,983.
It is important to note that simply raising the number of iterations with the default update
rule does not produce the same results as utilizing the more rapid learning rule. Slow learners do
not simply need more time. One possible explanation for this if learning is too slow, the agents
are always optimizing for an environment that is rapidly becoming passé. Like generals that are
always “fighting the last war,” slow learning may never be able to respond to the environment
quickly enough to do well.
**** insert Figure 10 about here ****
The analyses of Hypotheses #1 through #4 indicate that sustaining cooperation in a
hierarchically structured, multiple issue game is very difficult. Altering incentives in a single
game (e.g., the second stage trade game) is not sufficient for establishing a stable cooperative
relationship. Weighting the payoffs by relative power has only a marginal impact on cooperation.
Finally, while increasing the rate of learning increases cooperation, it has the untended
consequence of increasing volatility. This raises an interesting question: what factor or
combination of factors can produce a stable world dominated by cooperation rather than
defection?
Our final test varies four factors simultaneously: 1) raising the benefits of trade; 2) raising
the costs of war; 3) introducing relative payoffs; and 4) assuming rapid learning. This
22
combination is not without historical precedence in the modern international system. The spread
of the industrial revolution in the 1800s reduced the cost of trade by dramatically lowering
transportation costs (Rosecrance 1985), increased the cost of war by permitting the mass
production of technologically advance weapons (McNeill 1982), and increased interstate
communication through steam shipping, railroads, telegraph and eventually radio (Held et al.
1999). Operationally, the simulation parameters were set as follows: raise the CC payoff in the
trade game from 1 to 3; lower the DD payoff in the war game from -5 to -10; turn relative gains
“on”; and set learning to “high.”
Figure 11, which displays snap shots at iteration 5000 at the top and time series analysis
in the lower two panels, reveals that the interaction produces several interesting patterns. First,
Panel (B) indicates that war (black lines) is relatively rare despite the existence of a number of
crises (i.e., DC or CD outcomes show in red). Thus, raising gains from trade and raising the costs
of war appears to dampen the escalation of conflict. Second, Panel (C) indicates a large amount
of spatially clustered trade. The densely packed green lines within the clusters indicate that all
the agents are trading with all the other cooperative agents in their neighborhood. Third, Panel
(D) appears to indicate that trade does not pay: the power level of the cluster of red (Dtrade-Dwar)
agents in Panel (C) appear to have higher power (shown in blue) in Panel (D). However, this is
just an artifact of the snapshot. In this run the simulation, trade collapses just before the end of
the simulation as show by the falling red line in Panel (F). Prior to this collapse, wealthy traders
dominated the left hand side of the landscape. Fourth, Panel (F) indicates that the rise in trade is
directly correlated with a fall in inequality in that the Gini coefficient (show in aqua) never peaks
as high as in previous charts. Fifth, Panel (F) shows that the total trade reaches levels far higher
than in previous charts and the number of wars remains relatively low. However, trade is prone
to oscillation as it reaches “tipping” points as shown by the collapse of trade after iteration 4600.
Despite the oscillation, the mean level of trade remains relatively high after it becomes
entrenched in the system at round iteration 2000.
23
Across the 30 runs of the simulation, we find that the average number of trades per dyad
rises from 6.3% in the baseline to 34.9%, the average number of wars falls from 12.6% to 8.3%,
the average wealth rises from 1,264 to 12,790, the Gini coefficient falls from 0.75 to .44, and
clustering of Ctrade-Cwar agents rises from 0.43 to 0.67. All these changes are statistically
significant at better than the .001 level of significance.
CONCLUSIONS
The simulation reveals that it is difficult to establish cooperation in a two issue world in
which the issues are linked hierarchically. Significantly increasing cooperation requires
increasing the rewards for cooperation or the punishment for defection in both issue areas. In a
world in which a wide variety of strategies are available (over 1 million), no single strategy
becomes permanently dominant in the landscape. Rather, we see shifting across strategies as
agent take advantages of the existing population. However, the cooperation does emerge when
we simultaneously alter incentives (i.e., payoffs in each game) and learning patterns.
The simulation is not designed to produce a realistic model of international politics for a
particular era. Rather, we have developed a generic two stage game in which cooperation in the
primary game influenced the probability of cooperation in the second stage of the game.
Although the language of “war” and “trade” has been applied to illustrate the dynamics of the
model, the findings should be generalizable to similarly structured issue areas (e.g., war is simply
mutual defection in the first stage of a generic two-stage game).
However, we believe the model does shed some light on the emergence of the liberal
trading ordering in the 19th century for several reasons. First, discussions of the expansion of
international trade in the era rarely emphasize the key role of the industrialization of warfare. If
warfare had not become increasingly expensive, one wonders if the decline in transportation costs
would have trigger such a shift to “trading” strategies. Second, nationalists and dependency
theories often assume if payoffs are a function the relative power of states, cooperation should be
slow to emerge. Moreover, the exploited states should not benefit from the trading system. In
24
fact, relative gains increased average wealth and reduced income equality as trade expanded in
the system. While we are agnostic as to the division of gains in the real world (which was
probably driven by demand factor rather than relative power), we have shown that relative gains
do not by themselves inhibit cooperation. Third, the simulation reveals that rapid learning is a
double edged sword. While it increases the probability of identifying a reciprocity oriented
strategy that encourages cooperation, it also undermines the stability of this equilibrium by
facilitating the rise and spread of strategies that can take advantage of the status quo. Finally, the
simulation illustrates the importance of modeling cooperation within a spatial context. The model
demonstrated that cooperation tended to cluster spatially. This finding is consistent with others in
the simulation literature.17 Historically, we know that trade has tended to cluster spatially due to
the diffusion of “ideas” (Kindleberger 1975, 51), the relatively high levels of trade between
neighboring states, and the growing inclusion of Most Favored Nation (MFN) clauses in bilateral
trade agreements (Pahre 2001).18 Similarly, the security community associated with the
democratic peace emerged first in the North Atlantic and may be spreading to other regions as the
democratization spreads in a succession of waves (Huntington 1991).
Was the emergence of a liberal international order inevitable? The simulation results
suggest that the answer is no. The establishment of the liberal order required the simultaneous
convergence of factors across issue areas. Is the current equilibrium likely to be extremely
stable? The current “information revolution” is likely to reinforce the incentives established
during the industrial revolution. The gains from specialization continue to increase and the cost
of warfare has reached new heights. From smart bombs to unmanned aircraft, war remains a
technologically sophisticated and capital intensive operation. Thus, by increasing the benefits of
trade, the costs of war, and speed of learning, we expect the information revolution to reinforce
the liberal trading world.
25
Bibliography
Axelrod, Robert. 1984. The Evolution of Cooperation. New York: Basic Books.
Axelrod, Robert. 1997. The Complexity of Cooperation: Agent-Based Models of Competition and
Collaboration. Princeton, NJ: Princeton University Press.
Axelrod, Robert and Lisa D’Ambrosio. 1994. “Annotated Bibliography on the Evolution of
Cooperation.” Unpublished manuscript. University of Michigan.
Axelrod, Robert and Robert O. Keohane. 1986. “Achieving Cooperation Under Anarchy:
Strategies and Institutions.” In Kenneth A. Oye (ed.) Cooperation Under Anarchy.
Princeton, NJ: Princeton University Press.
Axtell, Robert. 2000. “Why Agents? On the Varied Motivations for Agent Computing in the
Social Sciences.” Working Paper No. 17. Washington, DC: Brookings Institution, Center on
Social and Economic Dynamics.
Baldwin, David A. (ed.) 1993. Neorealism and Neoliberalism: The Contemporary Debate. New
York: Columbia University Press.
Barbieri, Katherine and Jack S. Levy. 1999. “Sleeping with the Enemy: The Impact of War on
Trade.” Journal of Peace Research 36/4 (July), 463-479.
Bearce, David H. and Eric O’N. Fisher. 2002. “Economic Geography, Trade, and War.” Journal
of Conflict Resolution 46/3 (June), 365-393.
Bendor, Jonathan. 1993. "Uncertainty and the Evolution of Cooperation," Journal of Conflict
Resolution 37/4 (December), 709-734.
Binmore, Ken. 1998. “Review of The Complexity of Cooperation: Agent-Based Models of
Competition and Collaboration.” The Journal of Artificial Societies and Social
Simulation 1/1 (January).
Busch, Marc L. and Eric R. Reinhardt. 1993. “Nice Strategies in a World of Relative Gains: The
Problem of Cooperation Under Anarchy.” Journal of Conflict Resolution 37/3
(September), 427-445.
26
Cederman, Lars-Erik. 2001. "Modeling the Democratic Peace as a Kantian Selection Process."
Journal of Conflict Resolution, 45/4 (August), 470-502.
Cobden, Richard. 1850/1878. Speech before the Peace Society in Wrexham, Wales, 14 November
1850. Reproduced in John Bright and James E. Thorold Rogers (eds.) Speeches On Questions
of Public Policy by Richard Cobden. London, MacMillan, 1878.
Conybeare, John. 1986. "A Comparative Study of Anglo-Hanse, Franco-Italian, and HawleySmoot Conflicts." In Kenneth A. Oye (ed.) Cooperation Under Anarchy. Princeton, NJ:
Princeton University Press.
Dawkins, Richard. 1976. The Selfish Gene. New York: Oxford University Press.
Earle, Edward Meade. 1986. “Adam Smith, Alexander Hamiliton, and Friedrich List: The
Economic Foundations of Military Power.” In Peter Paret (ed.) Makers of Modern Strategy:
From Machiavelli to the Nuclear Age. Princeton, NJ: Princeton University Press.
Epstein, Joshua and Robert Axtell. 1996. Growing Artificial Societies: Social Science From the
Bottom Up. Washington, DC: Brookings Institution Press.
Frank, Andre Gunder. 1979. Dependent Accumulation and Underdevelopment. New York:
Monthly Review Press.
Held, David, Anthony McGrew, David Goldblatt, and Jonathan Perration. 1999. Global
Transformations: Politics, Economics, and Culture. Stanford, CA: Stanford University
Press.
Hoffmann, Robert. 1999. “The Independent Localisations of Interaction and Learning in the
Repeated Prisoner’s Dilemma.” Theory and Decision 47/1, 57-72.
Hoffmann, Robert. 2000. “Twenty Years On: The Evolution of Cooperation Revisited.” Journal
of Artificial Societies and Social Simulation 3/2.
Huntington, Samuel P. 1991. The Third Wave: Democratization in the Late Twentieth Century.
Norman, OK: University of Oklahoma Press.
27
Huth, Paul K. 1988. Extended Deterrence and the Prevention of War. New Haven: Yale
University Press.
Jervis, Robert. 1978. "Cooperation Under the Security Dilemma," World Politics 30/2 (January),
167-214.
Johnson, Paul E. 1999. "Simulation Modeling in Political Science." American Behavioral
Scientists 42/10 (August), 1509-1530.
Kim, Hyung Min and David L. Rousseau. 2004. “The Classical Liberals Were Half Right (or Half
Wrong): New Tests of the "Liberal Peace," 1960-88." Unpublished manuscript.
Kindleberger, Charles P. 1975. “The Rise of Free Trade in Western Europe, 1820-1875.” The
Journal of Economic History 35/1 (March), 20-55.
Lacy, Dean and Emerson M. S. Niou. 2004. “A Theory of Economic Sanctions and Issue
Linkage: The Role of Preferences, Information, and Threats.” Journal of Politics 66/1
(February), 25-42.
Leng, Russell. 1993. "Reciprocating Influence Strategies in Interstate Crisis Bargaining." Journal
of Conflict Resolution 37/1 (March): 3-41.
Lipson, Charles. 1984. "International Cooperation in Security and Economic Affairs." World
Politics, 37:1-23.
Lustick, Ian S., Dan Miodownik, and Roy J. Eidelson. 2004. “Secessionism in Multicultural
States: Does Sharing Power Prevent or Encourage It?,” American Political Science Review
98/2 (May), 209-230.
Machiavelli, Niccolo. 1950. The Prince and Discourses. New York: The Modern Library.
Macy, Michael and John Skvoretz. 1998. “The Evolution of Trust and Cooperation between
Strangers: A Computational Model.” American Sociological Review 63/5 (October), 638-661.
Macy, Michael W. and Robert Willer. 2002. “From Factors to Actors: Computational Sociology
and Agent-Based Modeling.” American Review of Sociology. 28, 143-66.
28
Martin, Lisa L. 1992. Coercive Cooperation: Explaining Multilateral Economic Sanctions.
Princeton, NJ: Princeton University Press.
McNeill, William H. 1982. The Pursuit of Power: Technology, Armed Force, and Society Since
A.D. 1000. Chicago: University of Chicago Press.
Morgan, T. Clifton. 1990. “Issue Linkages in International Crisis Bargaining.” American Journal
of Political Science 34/2 (May), 311-333.
Oye, Kenneth A. 1986. “The Sterling-Dollar-Franc Triangle: Monetary Diplomacy 1929-1937.”
In Kenneth A. Oye (ed.) Cooperation Under Anarchy. Princeton, NJ: Princeton
University Press.
Pahre, Robert. 2001. “Most-favored-nation Clauses and Clustered Negotiations.” International
Organization 55/4 (Autumn), 859-890.
Rogowski, Ronald.1989. Commerce and Coalitions: How Trade Affects Domestic Political
Alignments. Princeton, NJ: Princeton University Press.
Rosecrance, Richard. 1985. The Rise of the Trading State: Commerce and Conquest in the
Modern World. New York, NY: Basic.
Rostow, W.W. 1978. The World Economy: History and Prospect. Austin, TX: University of
Texas Press.
Rousseau, David L. 2002. “Motivations for Choice: The Salience of Relative Gains in
International Politics.” Journal of Conflict Resolution 46/3 (June), 394-426.
Rousseau, David L. 2005, forthcoming. Democracy and War: Institutions, Norms, and the
Evolution of International Conflict. Stanford, CA: Stanford University Press.
Simon, Herbert A. 1982. Models of Bounded Rationality. Cambridge, MA: MIT Press.
Snyder, Glenn H. and Paul Diesing. 1977. Conflict Among Nations: Bargaining, Decision
Making, and System Structure in International Crisis. Princeton, NJ: Princeton University
Press.
29
Spruyt, Hendrik. 1994. The Sovereign State and Its Competitors. Princeton, NJ: Princeton
University Press.
Stein, Arthur A. 1990. Why Nations Cooperate: Circumstance and Choice in International
Relations. Ithaca, NY: Cornell University Press.
Van Evera, Stephen W. 1984. "Causes of War." Ph.D. Dissertation, University of California,
Berkeley.
30
Location:
Are you a
neighbor?
No
Location:
Are you a
neighbor?
Figure 1: Overview of Trade-War Model
Yes
Yes
Situation:
What is the
current state
of relations?
Crisis
or War
Situation:
What is the
current state
of relations?
Crisis
or War
Peace
War Choice:
Should I
fight with
the other?
Peace
Yes
War Strategy:
Cooperate
or Defect?
Cooperate
Defect
No
Trade Choice:
Should I
trade with
the other?
No
R
R
S
T
T
S
P
P
Cooperate
War Strategy:
Cooperate
or Defect?
Yes
Defect
War Choice:
Should I
fight with
the other?
No
Exit
Exit
Yes
Trade Strategy:
Cooperate
or Defect?
No
No
Trade Choice:
Should I
trade with
the other?
Yes
Cooperate
Defect
R
R
S
T
T
S
P
P
31
Cooperate
Defect
Trade Strategy:
Cooperate
or Defect?
Figure 2: Landscapes and Actors
Moore 1
Neighborhood
Actor
Power
Trait 1
Trait 2
Trait 3
.
.
.
Trait 22
Trait 23
Trait 24
20x20 Landscape for Trait 1
(Population of 400 agents)
32
Neighbor
11,263
0
1
0
0
5
0.76
Power
Trait 1
Trait 2
Trait 3
.
.
.
Trait 22
Trait 23
Trait 24
8,200
1
1
1
1
7
0.23
Figure 3: Payoff Structure for Agents
Trade Payoffs
Other
Cooperate
Defect
1
Cooperate
1
Self
4
-4
0
-4
Defect
4
0
War Payoffs
Other
Cooperate
Defect
0
Cooperate
0
Self
10
-11
-
-11
Defect
10
-5
Notes: Self payoff is located in the lower left of each cell.
Other payoff is located in the upper right of each cell. The
“exit” payoff is equal to the Reward payoff plus the Punishment
payoff times the ExitPayoff parameter set by the user. In
the default simulation the ExitPayoff is 0.50, implying an
exit payoff of 0.50.
33
Figure 4: The War Choice Module
Is my war
strategy
Yes (G2=1)
unconditional
What is
strategy?
Cooperate
Defect (G1=1)
No
(G2=0)
Add # of
Defection
Fear Bin
How often
has the other
defected in war
with “me” during the last
X (G23) rounds?
Yes (G3=1)
Continue
Add # of
Defection
Fear Bin
How often
has the other
defected in war with
“common neighbors”
in the last X (G23)
No (G3=0)
Yes (G4=1)
Add # of
Defection
Fear Bin
Continue
War Choice
Module
Do I use war
behavior with
“neighbors”
marker?
No (G4=0)
Continue
How often
has the other
defected in war with
“all others” in the
last X (G23) rounds?
Do I use war
behavior with
“me” marker?
Yes (G5=1)
Do I use war
behavior with
“all others”
marker?
No
Calculate
Probability of
Fear to continue
(continues to War
Strategy Module)
34
Defect With
Probability
1-p (Trust)
Figure 5: The War Strategy Module
War Strategy
Module
6=
G
Ye
s(
No
No
Is the other state
weaker than me?
same
Yes
Did I trade with
the other state
last round?
Yes
Is the other
agent my type?
Yes
No
Do I treat
weaker states
differently?
=1
G6
Do I treat
stronger states
differently?
s(
Ye
0)
(continues from War
Choice Module)
)
No
No
1)
No
Do I treat Yes (G7=1)
trade partners
differently?
G
=0
G8
s(
Ye
Do I treat
non-trade
partners
differently?
8=
Yes (G7=0)
)
Ye
s(
If Cooperator (G1=0), then defect
If Defector (G1=1), then cooperate
Do I treat
other types
differently?
No
No
No
Do I treat
my type
differently?
Not Sure
If Defector (G1=1),
then defect
What is my
war
strategy?
35
If Cooperator (G1=0),
then cooperate
If Cooperator (G1=0), then defect
If Defector (G1=1), then cooperate
Figure 6: The Trade Choice Module
Is my trade
strategy
Yes (G13=1)
unconditional?
No
What is
trade
strategy?
Cooperate (G12=0)
Defect (G12=1)
(G13=0)
Add # of
Defection
Trust Bin
How often
has the other
defected in trade
with “me” during the
X (G23) rounds?
Yes (G14=1)
(G14=1)
Continue
Add # of
Defection
Trust Bin
No
How often
has the other
Yes (G15=1)
defected with
“common neighbors”
(G =1)
in the last X (G23) rounds? 15
How often
has the other
defected with
“all others” in the
last X (G23) rounds?
Do I use trade
behavior with
“neighbors”
marker?
Trade Choice
Module
No
Continue
Add # of
Defection
Trust Bin
Do I use trade
behavior with
“me” marker?
Yes (G16=1)
(G16=1)
Do I use trade
behavior with
“all others”
marker?
No
Continue
Calculate
Probability of
Trust To continue.
(continues to
Strategy Module)
36
Exit With
Probability
1-p (Trust)
Figure 7: The Trade Strategy Module
Ye
s(
G
Do I treat
stronger states
differently
in trade?
No
Trade Strategy
Module
Is the other state
weaker than me?
No
Yes
Do I treat
weaker states
differently
in trade?
Yes
Do I treat
neighbors
differently
in trade?
Yes
Do I treat
my type
differently
in trade?
No
=1
G 17
s(
Ye
17 =
0)
(continues from Trade
Choice Module)
)
No
Do I treat
other types
differently
in trade?
No
Am I a
neighbor of
the other state?
No
No
)
If Defector (G12=1),
then defect
Yes (G18=1)
1)
=0
G 19
s(
Ye
Do I treat
non-neighbors
differently
in trade?
19 =
Yes (G18=0)
Is the other
agent my type?
No
Not
Sure
What is my
trade
strategy?
37
No
Ye
s(
G
If Cooperator (G12=0), then defect
If Defector (G12=1), then cooperate
If Cooperator (G12=0),
then cooperate
If Cooperator (G12=0), then defect
If Defector (G12=1), then cooperate
Figure 8: Output From The Baseline Model
A) Distribution of traits in
the landscape at iteration
5000 (Attribute=1 shown
in black).
B) Distribution of War
shown by black lines
connecting nodes.
C) Distribution of Trade
shown by green lines
connecting nodes.
E) Change in the Percentage of States Having a Traits #1
(red), #2 (blue), #12 (aqua), & #13 (black) Across Time.
D) Spatial Distribution of Wealth
(Blue above 10000 and Red
Below 10000 Power Units)
F) Average Number of Wars (blue), Trade (red), and
Gini Coefficient (aqua) Across Time
38
Figure 9: H1: Increasing the Gains From Trade
A) Distribution of traits in
the landscape at iteration
5000 (Attribute=1 shown
in black).
B) Distribution of War
shown by black lines
connecting nodes.
C) Distribution of Trade
shown by green lines
connecting nodes.
E) Change in the Percentage of States Having a Traits #1
(red), #2 (blue), #12 (aqua), & #13 (black) Across Time.
D) Spatial Distribution of Wealth
(Blue above 10000 and Red
Below 10000 Power Units)
F) Average Number of Wars (blue), Trade (red), and
Gini Coefficient (aqua) Across Time
39
Figure 10: Rapid Learning Representative Runs
A) Distribution of traits in
the landscape at iteration
5000 (Attribute=1 shown
in black).
B) Distribution of War
shown by black lines
connecting nodes.
C) Distribution of Trade
shown by green lines
connecting nodes.
E) Change in the Percentage of States Having a Traits #1
(red), #2 (blue), #12 (aqua), & #13 (black) Across Time.
40
D) Spatial Distribution of Wealth
(Blue above 10000 and Red
Below 10000 Power Units)
F) Average Number of Wars (blue), Trade (red), and
Gini Coefficient (aqua) Across Time
Figure 11: Relative Gains, Rapid Learning, Increased Trade Benefits, & Increased War Costs
A) Distribution of traits in
the landscape at iteration
5000 (Attribute=1 shown
in black).
B) Distribution of War
shown by black lines
connecting nodes.
C) Distribution of Trade
shown by green lines
connecting nodes.
E) Change in the Percentage of States Having a Traits #1
(red), #2 (blue), #12 (aqua), & #13 (black) Across Time.
41
D) Spatial Distribution of Wealth
(Blue above 10000 and Red
Below 10000 Power Units)
F) Average Number of Wars (blue), Trade (red), and
Gini Coefficient (aqua) Across Time
Table 1: Trait Set for Each Agent
Trait Name
War Character
Trait # Attribute
1
0
1
War Unconditional
2
0
1
War -- Behavior w/ Me 3
0
1
War -- Behavior w/
4
0
Common Neighbors
1
War -- Behavior w/ All 5
0
1
War -- Power
6
0
1
War -- Interdependence 7
0
1
War -- Type
8
0
1
blank
9
0
1
blank
10
0
1
blank
11
0
1
Trade Character
12
0
1
Trade Unconditional
13
0
1
Trade -- Behavior w/ Me 14
0
1
Trade -- Behavior w/
15
0
Common Neighbors
1
Trade -- Behavior w/ All 16
0
1
Trade -- Power
17
0
1
Trade -- Neighbors
18
0
1
Trade -- Type
19
0
1
blank
20
0
1
Display Type
21
0
1
Detect Type
22
0
1
Memory Length
23
--
Description
Cooperate in war
Defect in war
Do not use unconditional approaches in war
Use unconditional strategies in war
Don't use past behavior in war with me
Use past behavior in war with me
Don't use past behavior in war with common neighbor
Use past behavior in war with common neighbor
Don't use past behavior in war with all others
Use past behavior in war with all others
Treat stronger states differently in war
Treat weaker states differently in war
Treat non-trading partners differently in war
Treat trading partners differently in war
Treat other type differently in war
Treat similar type differently in war
not currently in use
Military Propensity
Continuous variable measuring probability of initiating conflict
24
--
not currently in use
not currently in use
Cooperate in trade
Defect in trade
Do not use unconditional approaches in trade
Use unconditional strategies in trade
Don't use past behavior in trade with me
Use past behavior in trade with me
Don't use past behavior in trade with common neighbor
Use past behavior in trade with common neighbor
Don't use past behavior in trade with all others
Use past behavior in trade with all others
Treat stronger states differently in trade
Treat weaker states differently in trade
Treat non-neighbors differently in trade
Treat neighbors differently in trade
Treat other type differently in trade
Treat your type differently in trade
not currently in use
Do not display type marker
Display type marker
Ignore marker
Attend to marker
Integer recording number of previous rounds used by the agent
42
ENDNOTES
1
For a review of this literature, see Hoffman (2000) and Axelrod and D’Ambrosio (1994).
2
Bearce and Fisher (2002) model trade and war using an agent-based model. However, our model
differs from theirs in terms of the number of actors, the structure of interaction, the role of
learning, the evolution of strategies, the number of stages, and basic interaction rules.
3
Defect-Defect (DD) is a Nash equilibrium because no player can unilaterally switch strategies
and reap a higher payoff.
4
The simulation model described below allows the user to explore any preference order buy
simply altering the payoffs in the input screen (e.g., a Stag Hunt in stage 1 and a Prisoner’s
Dilemma in stage 2). However, we focus on a two-stage Prisoner’s Dilemma game because we
consider it the modal case in a two issue area situation.
5
For a critique of Axelrod’s conclusions, see Binmore (1998). For a more positive evaluation of
Axelrod’s contribution, see Hoffmann (2000).
6
For a more extensive discussion of strengths and weakness of agent-based modeling, see
Lustick, Miodownik, and Eidelson (2004), Rousseau (2005), Johnson (1999), and Axtell (2000).
7
Many authors prefer to restrict the use of the term “gene” to situations involving death and
reproduction. For these individuals, the learning model proposed here would be more
appropriately labeled a “meme” structure (Dawkins 1976). In order to make the model as
accessible as possible, we will simply talk about the evolution of traits.
8
A fully rational agent would be able to select a strategy through optimization. In our model,
agents “satisfice” based on the mean of the neighborhood (i.e., if power if below the mean, copy
from anyone above the mean). This implies that agents simply have to rank order neighbors in
terms of wealth. Moreover, agents have imperfect information about the strategies employed by
others and the effectiveness of individual strategies.
9
The model allows both local and long distance interaction. However, the results presented here
involve local learning and interaction only.
43
10
The model can be downloaded from www.xxx.edu.
11
For an interesting exception, see the "trade war" analysis by Conybeare (1986), the sanctions
game by Martin (1992), and some of the games explored by Snyder and Diesing (1977).
12
These strategies have typically been discussed with respect to the "echo effect." If both players
are using a Tit-For-Tat strategy, it is quite easy to become trapped in a punishment spiral (i.e.,
you defect because I punished you, I defect because you punished me, you defect because I
punished you, ...). Axelrod (1984, 38) argues that more lenient strategies such as Tit-For-2-Tats
and 90% Tit-For-Tat can reduce the likelihood of spirals, particularly under uncertainty.
13
Research has shown that “uncertainty” or noise can sharply reduce cooperation (Bendor 1993;
Axelrod 1997). Many different sources of uncertainty exist within the iterated prisoner’s
dilemma, including (1) uncertainty over payoffs, (2) uncertainty over relative power, (3)
uncertainty over specific plays, and (4) uncertainty over strategies. We plan to explore the
relative impact of different types of uncertainty in future analysis.
14
If the UpdateProbability parameter is set to 1.0, the agents will copy all of the traits of the more
successful agent. Thus, the most rapid learning occurs with learning rule (a) and update
probability 1.0.
15
Rogowski (1989, 21) claims that railroads decreased the cost of land transportation by 85-95
percent and steam ships decreased the cost of water transportation by 50 percent.
16
The four traits that are currently not used in the model do not appear in the figure. Therefore,
the number of the traits runs from left to right: Row 1: {1, 2, 3, 4, and 5}; Row 2: {6, 7, 8, 12, and
13}; Row 3: {14, 15, 16, 17, and 18}; Row 4: {19, 21, 22, 23, and 24 (not shown)}.
17
Hoffmann (1999) examines the interaction between the location of learning (local versus
distant) and the location of interaction (again local versus distant). He concludes that cooperation
is enhanced by local learning and local interaction. In his simulation of the democratic peace,
Cederman (2001) found that the democratic peace tended to appear in clusters that grew as
44
alliances and collective security were added to the baseline simulation. In sum, the theoretical
literature and the empirical analysis to date supports the expectation of clustering.
18
Pahre focuses on clusters of negotiations rather than spatial clustering. However, the MFN
treaties were regionally clustered in Western Europe.
45