Thesis Summary:
Empirical Game-Theoretic Methods for
Strategy Design and Analysis in Complex Games
Christopher Kiekintveld
University of Michigan
Computer Science and Engineering Building
2260 Hayward Street
Ann Arbor, MI 48109-2121
Motivation
only able to sample a small fraction of the full TAC SCM
strategy space.
Complex games like TAC SCM are not easily reduced to
explicit, tractable game models of the form common in the
game theory literature, and they often defy exact analytic
solution using conventional tools. I believe that the insights
of game theory can still be a valuable guide to developing
strategies in complex domains. However, there is a pressing
need for solution techniques based on game-theoretic principles that can meet the challenges of these complex domains.
The goal of my thesis work is to develop game analysis techniques capable of informing strategy design in large, complex games. Broadly defined, games are situations where
multiple players make interacting decisions; each players’
choice depends on the choices of the other players. The formal study of games has origins in economics, but in recent
years game theory has drawn increasing interest in computer science. Computer science offers a rich set of tools
for advancing the art of game analysis, including simulation
methods that can be used to explore strategic interactions in
games.
Game theory has achieved significant success and popularity, but practitioners have had mixed success applying
the theory to real strategy design problems (Roth 2002). An
acute challenge in game theory applications is the size and
complexity of real games, which often require players to
choose among very large sets of distinct courses of action. A
second (related) challenge is that players and analysts typically face substantial uncertainty about the outcomes due to
computational and observational limitations. Large games
exacerbate these uncertainties to the extent that they arise
from resource limitations. This form of uncertainty is particularly difficult to characterize due to the intricacies of gathering evidence about game outcomes.
Consider, for example, the Trading Agent Competition
Supply Chain Management game (TAC SCM) (Arunachalam & Sadeh 2005; Kiekintveld et al. 2006). This is a
game of significant interest among researchers, and gametheoretic investigation of this game forms part of my thesis. The game is an hour-long simulation of a supply chain
scenario played by automated agents. The space of possible agent designs is extremely large and – despite countless
hours of investigation – no optimal solution is known or on
the horizon. Like many real games of interest, TAC SCM
is defined in a compact form, by a set of game rules and
game server that implements those rules.1 We can gather evidence about the SCM game by simulating game instances,
but even using thousands of hours of simulation time we are
Research Overview
I adopt the framework of empirical game theory, an emerging methodology for principled analysis of large games.
This methodology makes explicit the process of exploring
and modeling the game as part of the analysis. The typical approach is to use simulation to derive an empirical estimate of the game form, which may be noisy and incomplete.
Various game-theoretic solution concepts can be applied to
the empirical game, including equilibrium models. The outcome of this analysis may be used to drive further simulation
and experimentation.
My thesis contributes to the growing body of evidence
that empirical game-theoretic analysis can yield useful
strategic guidance in very complex domains. I apply these
techniques to analyses of the TAC SCM game (Wellman
et al. 2005a; Vorobeychik, Kiekintveld, & Wellman 2006;
Jordan, Kiekintveld, & Wellman 2007) and a four player
variant of chess called chaturanga (Kiekintveld, Wellman,
& Singh 2006). As an example, the study in (Vorobeychik, Kiekintveld, & Wellman 2006) provides strong evidence that a rule change in the SCM game intended to curb
undesirable early purchasing behavior by the agents had little chance to have the desired effect, based on rational agent
responses. Other examples of successful applications of empirical game theory from the literature include continuous
double auctions (Phelps et al. 2005) and simultaneous ascending auctions (Reeves et al. 2005).
The preceding studies propose a wide variety of candidate algorithms for analyzing games, and further examples
can be found in the literature. To date, there is very little evidence to support comparison of the candidate approaches to
determine which yield the best results under varying conditions. The remainder of my thesis develops an experimental
c 2007, Association for the Advancement of Artificial
Copyright Intelligence (www.aaai.org). All rights reserved.
1
Contrast this with the explicit enumerations of actions and outcomes common throughout the literature on game theory.
1935
framework for evaluating candidate algorithms, and applies
this framework to test specific hypotheses about the relative
performance of candidates on different classes of games.
I propose evaluating candidate algorithms by analyzing
meta-strategy games. Each candidate is a meta-strategy:
a high-level strategy that selects a specific strategy to play
given (limited) observations of any game instance drawn
from some class of games. Since the performance of the
selected strategy depends on the context of strategies chosen by the opponents, we must specify how the opponents
select strategies. If we assume that the opponents are also
using one of the candidate meta-strategies, this induces a
game between the candidates that I refer to as the metastrategy game. This game can be analyzed using common
solution concepts including dominance and Nash equilibria.
Intuitively, we are interested in the possibility that a player
analyzing the game in one way would have an incentive to
switch to an alternative method of analysis. This approach is
similar in spirit to evaluations of single-agent learning methods. A key distinction is that this approach accounts for the
possibility of strategic interactions between meta-strategies
by considering them to be participants in a game.2
I use apply this framework to explore two issues of interest. The first is motivated by the uncertainty present in
empirical estimates of games due to sampling noise and incomplete data. In principle, solutions concepts applied to
these estimates should yield better and more robust answers
if they account for this uncertainty in some way. However,
the exact distribution of noise is unknown. Nevertheless,
some methods that incorporate models of noise (such as Nash or quantal-response equilibrium (McKelvey & Palfrey
1995)) may serve as good heuristic approximations under
many conditions. I hypothesize that solution concepts that
yield broader, distribution-based predictions of the outcome
will be more robust to the presence of significant noise than
point-based solutions.
My second experiment is motivated by the observation
that many real games seem to exhibit high degrees of structure that could be exploited to more efficiently explore these
games. I test the ability of several candidates to exploit unknown independence structure in games to direct sampling
towards the most relevant regions of the outcome space. I
hypothesize that adaptive learning methods are one class
of methods that is able to efficiently exploit independence
structure, including noisy variations. A second class of candidates that should demonstrate strong performance on this
class of games are variants of the best-first search for equilibrium first proposed in (Wellman et al. 2005b).
International Joint Conference on Automomous Agents and
Multi-Agent Systems. To appear.
Jordan, P. R.; Kiekintveld, C.; and Wellman, M. P. 2007.
Empirical game-theoretic analysis of the TAC supply chain
game. In Sixth International Joint Conference on Automomous Agents and Multi-Agent Systems. To appear.
Kiekintveld, C.; Miller, J.; Jordan, P.; and Wellman, M. P.
2006. Controlling a supply chain agent using value-based
decomposition. In Seventh ACM Conference on Electronic
Commerce.
Kiekintveld, C.; Wellman, M. P.; and Singh, S. 2006. Empirical game-theoretic analysis of chaturanga. In AAMAS06 Workshop on Game Theoretic and Decision Theoretic
Agents.
McKelvey, R. D., and Palfrey, T. R. 1995. Quantal response
equilibria for normal form games. Games and Economic
Behavior 10:6–38.
Phelps, S.; Marcinkiewicz, M.; Parsons, S.; and McBruney,
P. 2005. Using population-based search and evolutionary
game theory to acquire better-response strategies for the
double-auction market. In IJCAI-05 Workshop on Trading
Agent Design and Analysis.
Reeves, D. M.; Wellman, M. P.; MacKie-Mason, J. K.;
and Osepayshvili, A. 2005. Exploring bidding strategies
for market-based scheduling. Decision Support Systems
39:67–85.
Roth, A. E. 2002. The economist as engineer: Game theory, experimental economics and computation as tools of
design economics. Econometrica 70(4):1341–1378.
Vorobeychik, Y.; Kiekintveld, C.; and Wellman, M. P.
2006. Empirical mechanism design: Methods, with application to a supply-chain scenario. In Seventh ACM Conference on Electronic Commerce.
Wellman, M. P.; Estelle, J.; Singh, S.; Vorobeychik, Y.;
Kiekintveld, C.; and Soni, V. 2005a. Strategic interactions
in a supply chain game. Computational Intelligence 21:1–
26.
Wellman, M. P.; Reeves, D. M.; Lochner, K. M.; and Suri,
R. 2005b. Searching for Walverine 2005. In IJCAI-05
Workshop on Trading Agent Design and Analysis.
References
Arunachalam, R., and Sadeh, N. M. 2005. The supply
chain trading agent competition. Electronic Commerce Research and Applications 4:63–81.
Davis, G.; Benisch, M.; Carley, K.; and Sadeh, N. 2007.
Factoring games to isolated stategic interactions. In Sixth
2
Tournaments and factorial analysis are two alternative evaluation criteria that appear in related literature; each is less comprehensive than the full game-theoretic analysis.
1936