M.l.T. LIBRARIES - DEWEY Digitized by the Internet Archive in 2011 with funding from Boston Library Consortium Member Libraries http://www.archive.org/details/necessarysufficiOOsmit Q&tm ? working paper department of economics Necessary and Sufficient Conditions for the Perfect Finite Horizon Folk Theorem Lones Smith No. 93-6 May 1993 massachusetts institute of technology 50 memorial drive Cambridge, mass. 02139 Necessary and Sufficient Conditions for the Perfect Finite Horizon Folk Theorem Lones Smith No. 93-6 M.I.T. LIBRARIES May 1993 - DEWEY RECBVH} Necessary and Sufficient Conditions Horizon Folk Theorem for the Perfect Finite Lones Smith* Department of Economics Massachusetts Institute of Technology- MA 02139 Cambridge, May 7, 1993 Abstract Benoit and Krishna (1985) proved a folk finite- horizon n-player perfect theorem that assumed that every player has distinct Nash payoffs in the stage game and (essentially) that infinite-horizon folk theorem. Abreu, it satisfy the conditions of the Dutta and Smith (1993) have cently provided the necessary and sufficient condition infinite-horizon folk theorem. We do prove that NEU (NEU) is re- for the necessary for the finite-horizon folk theorem, but more importantly, this note substitutes the distinct Nash payoff condition requirement with a weaker necessary and sufficient condition, that players have recursively distinct Nash folk payoffs. As a consequence, we show how the n-player theorem might even obtain offs in if finite-horizon only one player has distinct Nash pay- the stage game. Conversely, when the stage but not our condition, the folk theorem's failure is game satisfies NEU rather dramatic: In particular, at least one player's limit equilibrium payoff set is a singleton. 'The motivation for this note, namely seeking the necessary conditions for a folk theorem, stemmed from collaboration with Dilip Abreu and Prajit Dutta, and in particular from their earlier work Abreu and Dutta (1991). 1. INTRODUCTION For a given n-player normal form game G, where the objective function is G(5, T) be the T-fold repeated the average discounted game, the perfect finitely-repeated let folk theorem is perfect equilibrium payoffs includes any feasible payoff vector of G enough for large in the finitely-repeated Prisoners' 'defect' every period for any T< T< oo and S G(S, T) obtains folk theorem (1986), and and < 1. condition for when G (i) if the set of subgame strictly individually rational It has been long known that Dilemma, the only subgame perfect equilibrium oo. This is is the ineluctable consequence of a simple and Krishna this fact, Benoit theorem led to correctly conclude that a perfect folk satisfies the sufficient for conditions for the infinite-horizon — namely, the full-dimensionality condition of Fudenberg and Maskin (ii) Recently, BK] were of stage payoffs. For this said to hold backwards induction argument. Perhaps motivated by (1985) [hereafter, sum game each player has distinct Nash payoffs in G. Abreu (i), et al. (1993) discovered which they termed NEU, an (essentially) necessary and sufficient for non-equivalent utilities, that neatly supplants the full-dimensionality condition. While we do prove that NEU is nec- essary for the finite-horizon folk theorem, this note primarily provides a necessary — recursively distinct Nash payoffs — for the finite-horizon and sufficient condition folk theorem that replaces The thus extending contribution of this note tended to finish the is BK to a complete characterization. purely conceptual: The characterization work of BK, thus completing the perfect information rem program. For just class of (ii), as is in- folk theo- NEU only differed from full-dimensionality by a nongeneric games, so too, within the class of stage games with recursively distinct Nash payoffs, the measure with distinct Nash payoffs for all players is zero. In addition to the stated goal of this note, our condition helps explain how finite-horizon subgame perfect equilibria behave as the horizon grows. The logic of the Smith (1990/92) is BK folk theorem, as best exemplified in Krishna (1989) or rather simple: Late in the repeated game, because all players have distinct Nash payoffs, the behaviour of any one of them can be leveraged by threatening to finish off with a fixed number (say S) of plays of that player's worst Nash payoff, rather than cycle through all the players' best Nash payoffs. from the end of the game, the infinite-horizon punishments work perfectly And because S is fixed independent of the horizon length, the effect Away well. on the average made payoff can be arbitrarily small. This essentially Rather than formally define our condition, we three-player example 2,1,-1 -1,-1,-1 1,-1,-1 2,2,2 2,2,2 2,2,2 -1,-2,-1 -2,-1,-1 1,-2,-1 2,2,3 2,2,2 -1,-1,0 -1,1,-1 -2,1,-1 0,0,-1 chooses rows (actions U, 1 weakly dominated M,D), 2 chooses columns (actions strictly prefers for player 1 (resp. player 2). Nash (pure or mixed) payoffs are 1 and 2 have unique Nash payoffs, m, G Thus it is easy to see (2, 2, 2) satisfies 1 BK does not satisfy the G r), to R, while action for all three players. is Nonetheless, a folk theorem does obtain! For (ii). L £, convex combinations of all Furthermore, the mutual minimax payoff Because players condition with the following 2,2,2 that the only (2, 2, 3). it 2,2,2 (resp. r) is and motivate game G. and 3 chooses matrices (actions L, R). Player 3 D their proof. 2,2,3 game, In this first is NEU, and thus the infinite-horizon folk theorem applies to G. Next, player 3 enjoys the distinct Nash (extremal) of the game: We so that his behaviour is need only threaten to switch from (U, £, By choosing S S-period phase. periods, say S', as irrespective of and 2. It and payoffs of 2 we wish 3, large enough, 3 just prior to this what players G(R) player 3 plays L). which is 1, We differs from can induce player 2 to play £, m, or r for as G(r,R) for player 1. It in G(£, L). We near the end of the game! That same proof (2, 1). R player 3 plays 1 2's Nash (i.e. when So player S' (and hence S) large enough, many periods, say S", as 1 does, yielding a has the unique optimal payoff of unique optimal payoff of 2 the many induced for players In particular, players 2 R) irrespective of what player (r, L) for the for as unique Nash payoff in G(R) By choosing prior to this penultimate 'Nash' phase. 1 his is R £, have now leveraged the behaviour of player 2 near the end of the game! Iterate this process. to play When Nash phase. and 2 do, a new game G(R) 1 L) to (M, willing to play has the unique Nash equilibrium payoff vector payoff from player is leveraged near the end 0, new we wish we just and 3 are willing (one-player) which differs game from l's have now also leveraged the behaviour of G satisfies a folk theorem now follows by as before. *But observe that no one player can simultaneously minimax the other two, a fact of some importance later on in our proof of necessity. We Nash summarize the above procedure by saying that payoffs. If G satisfies NEU, G has recursively distinct a sufficient condition it is for the BK long as such a recursive procedure eventually leverages the behaviour of then a perfect folk theorem obtains. Moreover, all So players, G satisfies NEU, our new condition if necessary too for the perfect finite-horizon folk theorem. Namely, is result: chain of recursive reductions, we cannot leverage the behaviour of if for any such all players, then a folk theorem does not obtain. The intuition behind the necessity length T, a player's set of or it is not. subgame evident that It is if is perfect equilibrium payoffs enough T, while leveraged, then he entertains a unique payoff for all SPE for instance, player 3 has a multivalued The SPE special payoffs in payoffs for G T. Clearly in the latter case, G game has our condition, The of our condition Indeed, fail. two Nash we focus on the folk theorem and of NEU 2. 4 and if T> T, while players 2 and all 8, respectively. Nash Thus, for generic games satisfying payoffs. stage game, and precisely define our iterative We established in section 3.1. is which distinct in one player has distinct Nash payoffs, equilibria. players enjoy distinct all In section 2, procedure. at least payoff set for we with no discounting, might suggest the nongeneric way Nash payoff requirement can then the T> behaviour cannot be his if cannot possibly hope for anything more. In the above game only receive distinct either point- valued, is the behaviour of a player can be leveraged, then his payoffs are multivalued for large 1 any given horizon also rather simple. For prove the necessity in section 3.2. THE STAGE GAME 2.1 Basic Definitions Consider a is player A— \ write finite set of actions, i's finite E. Let 7Tj(/i) Mi be — best response for i payoffs is F' : /j, i's A = G= II" A{ mixed . (A,-, Player strategies i's and l)-profile of mixed strategies = {w G F 1, . . . , n), where A{ utility function is let \i M = II"M,. = (/ij, . which minimax player when being minimaxed. Normalize € M}, = i 7r,-; expected payoff under the mixed strategy for i's co{n(fi) and the set of player Let ml,- be an (n F= normal form n-player game 7r,(m') = . . i, for all 7r, : Simply /lx„) 6 M. and m\ a i. Define so that the set of feasible and strictly individually rational : m, > 0, for all i} ^ 0, to sidestep trivialities. . Denote the = X' set of players ,i m} av = {ii,i2, when 2.2 (a, 1 ,a,- 2 ,. . ,a im — € ) MM x M • • G The game Mx = >, X \ X' obtained from utilities (or satisfies NEU) if game G no two players' utility functions are equivalent, i.e. for all i and j, ni(-) 7Tj(-). has recursively distinct Nash payoffs for if some h > 1, there an increasing sequence of h nonempty subsets of players from I, namely {0=Zo£Z £I2 £---QIh 1 Mj and actions a%g E and £ bj Mj for each g = =Z}, 1, 2, Nash payoff vectors y{a Ig ofG(ai g ) and y(bIg o(G(bIg ) lg \Ig -i, ) i ) (1) .. . ,h, with some pair of different exactly for players i.e. y(ai.)i for all im Games has non- equivalent G The game 2. in M x • for players not a positive affine transformation of exists x l2 m)-player Properties of Stage von Neumann Morgenstern is . actions the actions of players X' are fixed to aji Two 1. {l,2,...n}. For a given strict subset of players QX and corresponding choice of (possibly mixed) G(aj') be the induced (n let asJ = G Xg \Z ff -i. Even if Ir Q * y(bx,)i (2) I, call such a sequence {Ig } a Nash decomposition of G. Observe that if all players have distinct Nash payoffs as Krishna (1985), then our condition holds. The converse is assumed in Benoit and not true, as illustrated earlier. It later proves fruitful to consider the case distinct Nash Nash equilibrium Nonetheless, it is payoffs y(aj G does not have recursively Nash decomposition and correspond- payoffs. In that case, for every ing choice of where and y(bj ) conceivable that the set of all ), J/, £ X always obtains. such Nash decompositions could include every player. In fact this cannot occur. Lemma 1 There sively distinct Nash is a well-defined payoffs. maximal set of players J C X who have recur- . The proof is in the appendix. FINITELY-REPEATED GAMES 3. NEU 3.1 Sufficiency of We and Recursively Distinct Nash Payoffs games with will analyze finitely-repeated perfect monitoring, allowing each player to condition his current actions on the past actions of simplicity, public randomization is players. Also, for allowed: In every period players publicly observe the realization of an exogenous continuous its all random variable and can condition on outcome. Denote by a, = {an, . expected payoff in period function in G(5, T) The set of is subgame q,t) a behavior strategy for player , . t given the strategy profile a. the expected discounted Beyond the replacement differs and by Each player sum of his payoffs: perfect equilibrium payoffs i jz$r ^ 7r i«( a )- V(5, T). is from Theorem 3.7 of Benoit and Krishna (1985) on a few it is So objective _1 Nash payoff requirement, Theorem of the distinct admits payoff discounting. As such, i's 7r,t(a) his fronts. 1 First, it a uniform folk theorem, meaning that the discount factor and horizon length can vary independently over the relevant range. Second, NEU replaces full-dimensionality, as already noted. Third, unlike the (more intuitive) use of long deterministic cycles in Benoit rely and Krishna (1985), we simply upon correlated outcomes. 2 Theorem 1 (The Folk Theorem) and has recursively distinct VueF* and Ve > 0, 3T Nash payoffs. Then for < G satisfies NEU Suppose that the stage game oo and S < 1 the finitely -repeated so that T >T and 8 <E [S game , 1] G(8, T), => 3v G V{6,T) with \\v-u\\ <e. Proof: We suppose modify the argument first that mixed strategies are observable, and discuss how to later. Fix a Nash decomposition (1) of players for which inequality (2) obtains, and define c 9 2 = mn CT xV \\y^)i - y(6r,)«ll > ° It is also possible to take advantage of the correlating device to produce an exact, rather than an approximate, folk theorem. We omit such a laborious exercise. for = g means 1, 2, . . > Let the p h. , . be the largest payoff range for any player in G. By of public randomizations, let y9 (resp. z 9 ) be a payoff vector according equal weight to the preferred (resp. and y(bx Xg Xg -\ \ for players in ) is Xg \ Xg -\. Then, without payoff discounting, any player willing to suffer through k periods of any action profile is strictly alternative Nash payoff vectors amongst y(aj less preferred) that at least fg {k) = +1 [knp/cg \ if periods of y9 are switched to z 9 -kp + fg (k)yf>fg (k)zf. For any m> 0, - s g (m) for g g = = h 1, 2, . — . . — 2, l,h , . . , . + fg (m = where each y9 phase vectors x ,x . , rationality), x\ and t s (m) . . < ,x n h periods, have shown that et al. (1993) 2 + r) such that for x\ (payoff These inequalities are all and u E F* NEU i ^ i.e. h s h (m)) -\ ...,u;y ,..., y ;...;y\...,y lasts s g (q the (3) = S\(m) The T-period equilibrium outcome sequence h. h l 0, in fh(m) and s g+ i(m) Define to(m) 1. it, Abreu = recursively define Sh(m) , ) is + • • + s g (m) for is l played T — t^q + r) times. implies the existence of feasible payoff we have j, asymmetry), and x) < u,- 3> x' (strict individual (target payoff domination). later essential. We now explicitly describe the players' strategies which support this equilibrium. 3 For ease of exposition, late deviations are those occurring during the final q + r + th(q + r) periods of the repeated game; all others are called early deviations. 1. if 2. some player For g [If 3. 3 to T — th(q + r) — Play u until period = in Xg in Xg [If deviates late, start > h,h-l,...,l: Play y9 some player 1. > [If in periods player j ^ i deviates early, start 3; T-t g (q + r), ... ,T-t g _i{q + r) - 1. g' < g, start 5.] deviates, start 4.] Then set j <— i. i and j denote arbitrary players. Also, for clarity, we use the simple notation j <— i "assign j the value i." Also, steps always follow sequentially, unless otherwise indicated. Below, mean i 5.] deviates late, where Play m' for q periods. any player Bracketed remarks refer to off-path play, i.e. following deviations. Play xi for 4. r periods. [If any player Then in Xg> deviates late, start 5.] ' < g" The g', return to step T — t g >-i(q + r) — 1. Play z 9 until period 5. deviates early, restart 3; if some player i and set g' <— g" restart 5.] [If some Then go verification that these strategies constitute a suitable choices of Remark: If the q, r, and To 5 found is minimax strategy m' in the strategies, then unobservable deviations of m'j must somehow be deterred. show how, 4 (1993) al. player in lg » deviates, where to step 2. subgame perfect equilibrium for appendix. <0> some punishers playing completely entails mixed 2. by all players j within the support Fudenberg and Maskin (1986) and Abreu et an infinite-horizon context, to make players indifferent in across their pure strategies: After playing probabilistically replaced by one of n — m ! play proceeds to step , but with x' possible substitute reward phases, with 1 make each minimaxing probabilities chosen as to 4, player j ^ indifferent over all i actions in the support of m'-. This technique applies here equally well. In our finitely repeated context, we must from any mixed strategies required to maintain the incentives of the recursively distinct Nash payoff vector y 9 = than y9 and yet still , for playing may a 9 y /2 z 9 /2 that NEU. strictly is strictly preferred to z 9 5 . First replace y 9 with the Since g' > g, Second, to reward player j the support of a% fashion analogous to Abreu et that will maintain 3.2 Necessity of Abreu all incentives. NEU , 6 Mj et al. (1993) first where Nash outcome among > \ I9'_i. Finally, it is affect the possible to define in a (1993) a proper probabilistic transition function We omit the tedious algebraic verification. have already established the (near) necessity of That proof NEU for the easily extends to our finite-horizon case. authors discovered the idea, while the latter modified their argument to handle stage games that do not The , , and Recursively Distinct Nash Payoffs infinite-horizon folk theorem. The al. Xg \ Ig _i G Tg \Ig -\ g' > g, we any such substitution cannot possibly I5 new worse for every player j E probabilistically replace y 9 in step 2 with player fs best incentives of the original deviant in 5 For this we provide another way minimaxing players that takes advantage of the less preferred action in y{a Ig ) and y{bxg )- 4 in step 2. payoffs and not of + also deter late unobserved deviations but only the weaker NEU assumption. be modified to fg (k) = [2knp/cg \ + 1. satisfy full-dimensionality, definition of fg {k) will have to For consider the infinitely-repeated game with stage game G(<5, T). Clearly, u if G as a sequence of games a subgame perfect discounted average payoff for G(8,T), is then by concatenation of the equilibria, it is also one for the infinitely repeated game with stage game G. This establishes Theorem 2 (Necessity of NEU) Suppose that no two players can be simulta- neously held to their lowest feasible payoff in G. Then NEU necessary for the is conclusion of the folk theorem. If a game G, however, does not have recursively distinct Nash payoffs, the conse- quences are rather stark. The following result has the flavour of the "zero-one" laws of probability theory. Namely, in a T-period repeated subgame , F* of G, or some players receive a unique equilibrium payoff. no middle ground. is Theorem game G(5, T) either the set of perfect equilibrium payoffs of all players tends to the strictly individually rational payoff set There game 3 (Necessity of Recursively Distinct perfect equilibrium of G(5,T), for Nash In any sub- Payoffs) any T, players in X J \ receive a payoff Nash equilibrium payoff of G. equal to their unique payoff of G(S,T) X\J has a unique subgame perfect equilibrium equal to his Nash payoff in G, for T = 1,2, ,T (Since J\ J ^ 0, we know be true for To Assume Proof: in 1\ J that every player in . this to = 1.) Then . . . in the first period of G(5, Tq must play a Nash equilibrium of G(aj), for any aj E Mj, As a simple set V(5,T) as grows. to say, but otherwise, Corollary Tg -i = i g for proji g (V(5,T)) g is = induction, <) we can If many offer some possible insight into the behaviour of the payoff Nash decompositions we can sometimes make 1,2, . . . exist, , h, then there exist single valued for all periods we have little rather sharp predictions. unique Nash decomposition of G, and 7/ there exist a all players T. all corollary, T 1), because they By have a unique (subgame perfect) continuation payoff by assumption. the result obtains for + times T\ T < Tg , < T2 < • • it • < satisfies Ig \ Th such that and otherwise multi-valued. CONCLUDING REMARKS 4. This note has illustrated an easily- verifiable, and yet necessary and sufficient replacement of the distinct Nash payoff requirement of Benoit and Krishna (1985). Although its theoretical advantage is nongeneric, theorems only imply that players' behaviour is hopefully underscores that folk it iteratively leveraged near the end of the repeated game. This conceptual contribution might have generic application in other circumstances. The analysis has also offered insight into the behaviour of fulfilling our condition: are singletons. When not all effective minimax payoff to their level. Proof of If for Lemma (1993), who has pointed might not be so dramatic: minimax payoff level, but their (newly defined) 6 APPENDIX 5. 5.1 Wen NEU out that the consequences of the partial failure of down not partially players can be leveraged, their payoff sets This contrasts with recent work by Players can be held games only 1 = some Nash decomposition we have Xh X, then unambiguously J = X, and we are done. Otherwise, let £,(G) be the set of all Nash payoffs for player Consider the alternative unique nested sequence of {0 = Jo £ Jx £ J2 £ • • £ h' Jh> nonempty If i £ no such T\Jg for whom {£,(G(axJ),aj G MXg players Jg+ \ Jg exist, set h' = g. Note that 9 it may Nash decompositions {Xg }. For This equals i. {Xg } satisfying sets Jg , the set Jg+i \ Jg is all does not consist of a singleton. not be possible to find only two Nash equilibria providing i not coincide with any Nash decomposition {Tg }. utilities to game G. i distinct payoffs simultaneously for all players for all ) in the = J £ X}, given by the following (well-defined) recursion: Given players i mm a _ . /( Jg +\ \ Jg Still, clear that . That we can show X/, C maa; j€ /(,)mai , j.7ri(aj,a_ J ), where I(i) axe ( ) it is E Jh all ,. is, {Jg } that Jh< need = Xh Moreover, for any players with equivalent . such nested sequence Xg For . {£i(G{ax,.)),ai,, e Mr Mf} € {Si{G(ax')),ai' if Theorem 1 proceed recursively as we ensure subgame perfection. deterrents are strict by some discounting. Given continuity of discounted sums only check that any for all G level of discounting 8 [5 By yield a higher payoff. for 1], and select q In light of x\ r. + l where ^ 2, for any > 0, let >/?«' It will 1. q satisfy strict follow that for this 7 +1 (4) the best payoff vector for player is simultaneously renders steps 3 and 4 a large without they will thus remain in 6, some 5 < 1, the "unimprovability" criterion, the profile will be a w\ and positive margin, say we perfect equilibrium. We now for all j, , First note that no history consisting of isolated deviations does a one-shot deviation profile, after subgame some does not consist of a singleton, then neither does 5.2 Verification of Strategies for We are eventually included in > £ 1" I' >) if Jg players in any (1), all < G. Since x\ in i u,- too, (4) deviations from steps strict deterrent to Next, step 4 deters deviations by the punishers from step 3 r. if r 1 is enough that + qwj for all j ^ Next, This i. we for all i + r + t h (q + = £ lg and g , if To j3] + rxij + 1 - (q - l)iij + 1 > x3j. verify that step 5 deters all late deviations. Indeed, this Given q and context > possible because x is consequence of inequality [q rx) 'The inequality - t (4) q g (q + r)]{-p) + s g (q + r)y? > s g {q + r)z° + 1 l,2,...,h. + t^(p + q)- For above program T> is feasible for To, each deterrent — in particular, why the left-hand side random an immediate which upon substitution yields r as defined above, the > p+ that obeying the r) (3), is is not any repeated remains (q+ l)x\ strict for all — reflects the fact correlating device generating x' might sometimes require worst possible outcome. 10 i to play his S G for [Sq, 1], some S < 1. Finally, for resulting average payoff vector is T < v G F*, oo and 5o we have ||u — < v\\ 1 < large enough, e, since if p and the q are fixed. References Abreu, Dilip and Prajit Dutta, Games: A New Condition," 1991. University of Rochester mimeo. — — and Lones , "Folk Theorems for Discounted Repeated , Condition," 1993. Smith, "The Folk Theorem for Repeated Games: A NEU MIT mimeo. Benoit, Jean-Pierre and Vijay Krishna, "Finitely Repeated Games," Econometrica, 1985, 53, 905-922. Fudenberg, Drew and Eric Maskin, "The Folk Theorem in Repeated Games with Discounting or Incomplete Information," Econometrica, 1986, Krishna, Vijay, "The Folk Theorems for 54, 533-554. Repeated Games," 1989. mimeo 89-003, Harvard Business School. Smith, Lones, "Folk Theorems: Two-Dimensionality 1990/92. is (Almost) Enough," MIT mimeo. Wen, Quan, "Characterization of Perfect Equilibria in Repeated Games," 1993. University of Windsor mimeo. 11 579 :i 2 / Date Due Lib-26-67 MIT LIBRARIES DUPL 3 TOfiO 00A3E7S0 fl