NOTES ON P.-L. LIONS’ LECTURES AT THE COLLÈGE DE FRANCE These notes relies on P.L. Lions’ lectures at the Collège de France in on Mean Field Games. They do not claim to be the transcription of the talks, which are available on the Collège de France website (http://www.college-de-france.fr). 1. Master equation for Mean Field Games 1.1. The master equation as a mean field limit. Let us consider a N −Player differential games, where each player i controls his dynamics p √ dXti = αti dt + 2νdWti + 2βdBt In the above equation, αi : [0, T ] → Rd is the control of Player i, which is assumed to be adapted to the filtration generated by the d−dimensional Brownian motions W i and B. All the W i and B are supposed to be independent. The players aim at minimizing the cost function, given by ˆ T N i N N Li (Xs , αs )ds + Ūi (XT ) Ji (x, t, α) = E t (X 1 , . . . , X n ) LN i where X = and where : RN d × Rd → R and ŪiN : RN d → R are respectively the running cost and the terminal cost of player i. From now on we assume that, for player i, N the other players are undistinguishable: then LN i and Ūi are symmetric with respect to the (Xj )j6=i and, more precisely, have supposed to have the following form: 1 X 1 X i δxj , αi ) and ŪiN (x) = Ū (xi , δxj ) LN i (x, α ) = L(xi , N −1 N −1 j6=i j6=i Rd Rd Rd In the above expressions, L : × P2 × → R and Ū : × P2 → R, where P2 is the set of d Borel measures on R with finite second order moment. Let us introduce the convex conjugate of the map L with respect to its last variable: L∗ (x, m, p) = sup hα, pi − L(x, m, α) α∈Rd and set H(x, m, p) = L∗ (x, m, −p). In this framework, a Nash equilibrium payoff (uN i )i=1,...,N is a solution of the following system of Hamilton-Jacobi equations: X X ∂ 2 uN 1 X ∂uN N i i + ν ∆ u + β Tr( ) − H(xi , δxj , Dxi uN xj i i ) ∂t ∂x ∂x N − 1 k l j k,l j6=i X ∂H 1 X N N − h (xj , δxk , Dxj uj ), Dxj ui i = 0 (1) ∂p N −1 j6 = i k6 = j 1 X N u (x, T ) = Ū (x , δxj ) i i N −1 j6=i Indeed, by verification techniques we get: Lemma 1.1. If (uN i ) is a solution to the above system, then the feedback strategies X 1 αi∗ (x, t) := −Dp H(xi , δxj , Dxi uN i ) N −1 j6=i 1 i=1,...,N provide a feedback Nash equilibrium for the game. Namely JiN (x0 , t0 , (αj∗ )j=1,...,N ) ≤ JiN (x0 , t0 , αi , (α̂j∗ )j6=i ) for any i and any control αi adapted to the filtrations generated by (W j )j=1,...,N and B. Derivative in the space of measures: In order to introduce the formal limit of the maps let us recall two notions of derivative in the Wasserstein space P2 . The first one requires to work on densities. The second one—which is more general—requires to work in an extended space. Let U : P2 → R. A first possible definition consists in restricting the function U to the elements m of P2 which have a density in L2 (Rd ). Then, assuming that U is defined in a neighborhood in L2 (Rd ) of P2 ∩ L2 (Rd ), we can use the Hilbert structure on L2 (Rd ). We denote δ2 U δU the gradient of U in L2 (Rd ) (this is again an element of L2 (Rd )) and δm by δm 2 the second order derivative (which can be identified with a symmetric bilinear form on L2 (Rd )). The second notion relies on the representation of U as a function on random variables. More precisely, fix a probability space (Ω, F, P) (we assume that it is a standard probability space without atom). We consider the extension Ũ of U to L2 (Ω, Rd ) defined by Ũ (Y ) = U (L(Y )), where L(Y ) is the law of the random variable Y . The we can consider the derivative of Ũ in the space L2 (Ω, Rd ). It s known that, if Ũ is of class C 1 , then the random variable ∇Ũ (Y ) is adapted to Y : namely, there is a Borel measurable map Dm U (L(Y ), ·) ∈ L2 (Rd , Rd , L(Y )) such that ∇Ũ (Y ) = Dm U (L(Y ), Y ). Here is the relation between the two first order derivatives defined above: δU m − a.e. (2) Dm U (m, ·) = Dx δm To explain this relation, let us compute the action of a vector field on a measure m and the image by U . For a given vector field B : Rd → Rd and m ∈ P2 absolutely continuous with a smooth density, let m(t) = m(x, t) be the solution to ∂m ∂t + div(Bm) = 0 m0 = m (uN i ), This expression directly gives the equality d δU U (m(h))|h=0 = h , −div(Bm)iL2 (Rd ) . (3) dh δm Coming back to the L2 (Ω, Rd ) setting, m(t) can be represented by solving the differential equation Y 0 (t) = B(Y (t)), Y (0) = Y0 in L2 (Ω) (where L(Y0 ) = m): then m(t) = L(Y (t)) and thus d d U (m(h))|h=0 = Ũ (Y (h))|h=0 = h∇Ũ (Y0 ), B(Y0 )iL2 (Ω) = hDm U, BiL2m (Rd ) . dh dh So, for any vector field B, δU δU hDm U, BiL2m (Rd ) = hDm U, mBiL2 (Rd ) = h , −div(Bm)iL2 (Rd ) = hDx , mBiL2 (Rd ) , δm δm which implies (2). Below we need to consider more singular evolution equations for the density m(t): the heat equation and the transport equation by a Brownian drift. Action of the heat kernel: Let us now assume that the curve m(t) is driven by the heat equation: ∂m ∂t − ∆m = 0 m0 = m 2 Then U (m(t)) = U (m + t∆m + o(t)) = U (m) + th δU , ∆miL2 (Rd ) + o(t), δm which proves that d δU U (m(t))|t=0 = h , ∆miL2 (Rd ) . dt δm Using the relation (2), we also obtain δU δU Dx m h , ∆miL2 (Rd ) = −hDx , Dx miL2 (Rd ) = −hDm U, i 2 d δm δm m Lm (R ) We can write U (m(t)) in another way: let Y0 ∈ L2 (Ω, Rd ) be such that L(Y0 ) = m √ and let (Bt ) be a d−dimensional standard Brownian motion independent of Y0 . Then L(Y0 + 2Bt ) = m(t), so that √ √ U (m(t)) = Ũ (Y0 + 2Bt ) = Ũ (Y0 ) + 2hDm U (L(Y0 ), Y0 ), Bt iL2 (Ω) + D2 Ũ (Y0 )(Bt , Bt ) + o(t). Note that, as Y0 and (Bt ) are independent, hDm U (L(Y0 ), Y0 ), Bt iL2 (Ω) = E [hDm U (L(Y0 ), Y0 ), Bt i] = hE[DU (L(Y0 ), Y0 )), E[Bt ]i = 0. 2 U (m, Y ), As for the second term, as D2 Ũ (Y0 ) = Dmm 0 D2 Ũ (Y0 )(Bt , Bt ) = tD2 Ũ (Y0 )(B1 , B1 ) To summarize (for some Y0 of law m and B1 normalized Gaussian vector independent of Y0 ): Dx m d δU U (m(t))|t=0 = D2 Ũ (Y0 )(B1 , B1 ) = h , ∆miL2 (Rd ) = −hDm U, i 2 d. (4) dt δm m Lm (R ) Action of a common noise. Let us now consider √ the case of a common noise. Let m ∈ P2 be a smooth density and consider m(x, t) = m(x − 2Bt ) where (Bt ) is a standard Brownian motion. We want to expand E[U (m(t))]. First we have by Ito’s formula, √ 2 m(x, t) = m(x) − 2hDx m, Bt i + hD Xxx mBt , Bt i + o(t) √ 2 = m(x) − 2hDx m, Bt i + Btk Btl hDxx m ek , el i + o(t). k,l Therefore (keeping in mind that Bt is a constant vector before taking the expectation) X δU √ 2 E[U (m(t))] = U (m) + h E[Btk Btl ]hDxx m ek , el i i , − 2hDx m, E[Bt ]i + δm k,l i 2 X h k l δ U + E Bt , Bt (hDx m, ek i, hDx m, el i) + o(t) δm2 k,l ! X δ 2 U ∂m ∂m δU = U (m) + t h , ∆mi + ( , ) + o(t) δm δm2 ∂xk ∂xk k So d X δ 2 U ∂m ∂m d δU E[U (m(t))]|t=0 = h , ∆mi + ( , ). dt δm δm2 ∂xk ∂xk k=1 3 √ √ On another hand, if Y0 is a random variable with law m, then Y0 + 2B̂t has law m(· − 2Bt ) (where B̂t is a Brownian motion which lives in a new probability space (Ω̂, F̂, P̂)). Hence i h i h i h √ √ Ê[U (m(t))] = Ê Ũ (Y0 + 2B̂t ) = U (m) + Ê hD̃U (Y0 ), 2B̂t i + Ê D2 Ũ (Y0 )(B̂t , B̂t ) + o(t) = U (m) + t d X D2 Ũ (Y0 )(ek , ek ) + o(t) k=1 So d d X X δ 2 U ∂m ∂m d δU 2 E[U (m(t))]|t=0 = , ∆mi + ( Dmm U (m)(ek , ek ) = h , ). dt δm δm2 ∂xk ∂xk k=1 (5) k=1 N Formal asymptotic of the (uN i ). From now on we assume that, for any i = 1, . . . , N , ui is symmetric with respect to permutation on {1, . . . , N }\{i}. The Master Equation (also called (MFGf) for MFG fundamental) arises when one considers the limit of the uN i as N → +∞. Namely, one expects that 1 X δxj , t) uN i (x, t) ∼ U (xi , N −1 j6=i where U : Rd (M F Gf ) × P2 × [0, T ] → R satisfies the master equation (or MFG fondamental) δU ∂H ∂U − H(x, m, Dx U ) − h , −div(m (x, m, Dx U ))i + (ν + β)∆x U ∂t δm ∂p X δ 2 U ∂m ∂m δU δ +(ν + β)h , ∆mi + β ( , ) − 2βh Dx U, Dx mi = 0 2 δm δm ∂xi ∂xi δm i in Rd × P2 × (0, T ) U (x, m, T ) = Ū (x, m) in Rd × P2 This equation can also be written in terms of derivatives of the map Ũ (x, Y, t) := U (x, L(Y ), t) on Rd × L2 (Ω, Rd ) × [0, T ]: ∂H ∂ Ũ − H(x, m, Dx Ũ ) − hDm Ũ , (x, m, Dx Ũ ))i + (ν + β)∆x Ũ + νD2 Ũ (Y0 )(B1 , B1 ) ∂t ∂p ! d d X X 2 Dmm Ũ (m)(ek , ek ) = 0 in Rd × L2 (Ω) × (0, T ) hDm Ũ , ek i + +β 2 k=1 k=1 Ũ (x, Y, T ) = Ū (x, Y ) for (x, Y ) ∈ Rd × L2 (Ω) Heuristic derivation: At least at a formal level, ∂U ∂t arises as the limit of P as the limit of H(xi , N 1−1 j6=i δxj , Dxi uN ). i The first non obvious term is X ∂H 1 X N h (xj , δxk , Dxj uN j ), Dxj ui i . ∂p N −1 j6=i Heuristically, 1 ∂H (xj , ∂p N −1 ∂uN i ∂t , H(x, m, Dx U ) k6=j X δxk , Dxj uN j ) behaves like the vector field k6=j ∂H (·, m, Dx U (·, t)). ∂p Let us now fix a smooth vector field B : Rd → R and consider the expression X X d N hB(xj ), Dxj uN i = u (x, δxj (s) , t)|s=0 , i ds i j6=i j6=i 4 where (xj (s)) is solution to x0j (s) = B(xj (s)). This means that the measure mN x is transported by the vector field B. So, in view of (3), X δU , −div(Bm)i . hB(xj ), Dxj uN i i∼h δm j6=i This shows that X ∂H δU 1 X ∂H N h (xj , δxk , Dxj uN , −div(m (x, m, Dx U ))i. j ), Dxj ui i ∼ h ∂p N −1 δm ∂p j6=i k6=j We now study the term X N ∆xj uN i . As ∆xj ui ∼ ∆x U , we have to analyze the quantity j X ∆xj uN i . It comes from the action of the independent noises (Wti ) on the system. If mN x is j6=i √ has law mN the empirical measure of a vector x ∈ RN d , then mN x ? G(t) where G(t) is x+ 2(W i ) t the heat kernel. In view of the discussion in (4), we expect X δU ∆xj uN , ∆miL2 (Rd ) . i ∼h δm j Let us finally discuss the limit of the term X Tr( k,l ∆xi uN i +2 X k6=i Tr( ∂ 2 uN i ) that we rewrite ∂xk ∂xl X ∂ ∂uN ∂ 2 uN i i )+ Tr( ) ∂xk ∂xi ∂xk ∂xl (6) k,l6=i d X X ∂ ∂uN i The first term gives ∆x U . The second one is equal to 2 . This behaves like the l ∂xl ∂x i k l=1 k6=i ∂U sum over l of the transports of the quantity ∂x l along the vector fields Bl = el : hence in view of the discussion above, XX X δ ∂U ∂ ∂uN i 2 Tr( , −div(mBl )i ) ∼ 2 h ∂xk ∂xi δm ∂xl l k6=i l X δ ∂U ∂m δ , − l i = 2h Dx U, −Dx mi = 2 h δm ∂xl δm ∂x l This can be rewritten in terms of Ũ as ˆ Xˆ Xˆ X δ ∂U ∂m ∂ δ ∂U δ ∂U −2 ( l) l = 2 ( ( l ))m = 2 div( ( l ))m = 2 hDm Ũ , el i l δm ∂x Rd δm ∂x ∂x Rd ∂x δm ∂x Rd l l l The last term in (6) is due to the action of the common noise: in view of the discussion in (5) it yields to d X X ∂ 2 uN δU δ 2 U ∂m ∂m i Tr( )∼h , ∆mi + ( , ). ∂xk ∂xl δm δm2 ∂xk ∂xk k=1 k,l6=i 5 To summarize, we expect that the system X X ∂ 2 uN 1 X ∂uN N i i + ν ∆ u + β Tr( ) − H(x , δxj , Dxi uN x i i i ) j ∂t ∂x ∂x N − 1 k l j k,l j6=i X ∂H 1 X N N (xj , δxk , Dxj uj ), Dxj ui i = 0 − h ∂p N −1 k6=j j6=i X uN (x, T ) = Ū (x , 1 δxj ) i i N −1 j6=i has for limit δU ∂U + ν∆x U + νh , ∆mi + β ∂t δm ! X δ 2 U ∂m ∂m δ δU ∆x U + 2h Dx U, −Dx mi + h , ∆mi + ( , ) δm δm δm2 ∂xk ∂xk k δU ∂H −H(x, m, Dx U ) − h , −div(m (x, m, Dx U ))i = 0 δm ∂p U (x, m, T ) = Ū (x, m) which is the claimed equation. 1.2. The master equation contains the MFG system (β = 0). We assume here that β = 0. The master equation becomes δU ∂H ∂U ∂t − H(x, m, Dx U ) − h δm , −div(m ∂p (x, m, Dx U ))i δU (M F Gf ) +ν∆x U + νh , ∆mi = 0 in Rd × P2 × [0, T ] δm U (x, m, T ) = Ū (x, m) in Rd × P2 Let us fix the initial repartition m̄ of the players and let m(t) = m(x, t) be the solution to ∂H ∂m − ν∆m − div(m (x, m, Dx U )) = 0 in P2 × [0, T ] ∂t ∂p m(0) = m̄ We set v(x, t) = U (x, m(t), t). Then we claim that the pair (v, m) is a solution of the MFG system ∂v ∂t + ν∆v − H(x, m, Dv) = 0 ∂m ∂H − ν∆m − div(m (x, m, Dv)) = 0 ∂p ∂t v(x, T ) = Ū (x, m(T )), m(x, 0) = m̄(x) Proof. Indeed, ∂v ∂U δU ∂H = +h , ν∆m + div( (x, m, Dx U ))i ∂t ∂t δm ∂p = H(x, m, Dx U ) − ν∆x U 1.3. The master equation contains the system for a finite number of players ν = 0. We now assume that ν = 0. In this case, the master equation not only can be understood as a limit of a Nash equilibrium when the number of player tends to infinity, but actually contains this system. Let us understand why. 6 Let U be a solution of the master equation, which, when ν = 0, becomes δU ∂H ∂U − H(x, m, Dx U ) − h , −div(m (x, m, Dx U ))i + β∆x U ∂t δm ∂p X δ 2 U ∂m ∂m δ δU ( , ) − 2βh , ∆mi + β Dx U, Dx mi = 0 +βh (M F Gf ) 2 ∂x ∂x δm δm δm i i i in Rd × P2 × [0, T ] U (x, m, T ) = Ū (x, m) in Rd × P2 N N For any i = 1, . . . , N , let us set uN j (x, t) = U (xi , m(xj )j6=i , t), where, as usual, m(xj )j6=i is the P N 1 N empirical measure mN j6=i δxj . We claim that the (ui ) is a Nash equilibrium (xj )j6=i := N −1 payoff, i.e., satisfies system (1) which, as ν = 0, is given by X ∂ 2 uN 1 X ∂uN i i + β Tr( ) − H(xi , δxj , Dxi uN i ) ∂t ∂x ∂x N − 1 k l k,l j6=i X ∂H 1 X N (xj , δxk , Dxj uN − h j ), Dxj ui i = 0 ∂p N − 1 k6=j j6=i 1 X N δxj ) ui (x, T ) = Ū (xi , N − 1 j6=i Proof. We have ∂uN ∂U N N N N i = (xi , mN (xj )j6=i , t), H(xi , m(xj )j6=i , Dxi ui ) = H(xi , m(xj )j6=i , Dx U (xi , m(xj )j6=i , t)) ∂t ∂t X ∂H N N We have also seen that the expression h (xj , mN (xj )j6=i , Dxj uj ), Dxj ui i is the derivative of ∂p j6=i ∂H N N uN i with respect to the vector field B((xj )j6=i ) := ∂p (xj , m(xj )j6=i , Dxj uj ) acting only on the variables (xj )j6=i . In view of (3), we get X ∂H δU ∂H N N h (xj , mN , −div(m (x, m, Dx U ))i (xj )j6=i , Dxj uj ), Dxj ui i = h ∂p δm ∂p j6=i P ∂ 2 uN i Finally the sum k,l Tr( ∂xk ∂x ) is resulting from a common noise acting on all the variables, l and has to be split into the sum of 3 terms X X ∂ ∂uN ∂ 2 uN i i ). ∆xi uN + 2 Tr( ) + Tr( i ∂xk ∂xi ∂xk ∂xl k6=i k,l6=i δ These expressions are respectively equal to ∆x U (xi , mN (xj )j6=i , t), −2h δm Dx U, Dx mi and, by (5), d X δ 2 U ∂m ∂m δU , ∆mi + ( , ). h δm δm2 ∂xk ∂xk k=1 The above statement is not correct for ν 6= 0. To see this, let us assume, to simplify the expressions, that H = 0, β = 0, Ū = Ū (m) and let us check that the family (uN i ) is not a solution of the Nash system. In this case, U depends only on m and therefore solves ( ∂U δU + νh , ∆mi = 0 in P2 × [0, T ] ∂t δm U (m, T ) = Ū (m) in P2 7 The Nash system is just N X ∂ui + ν ∆xj uN i =0 ∂t j6=i uN (x, T ) = Ū (mN i (xj )j6=i ) However it is not true that X δU , ∆mi| m=mN δm (xj )j6=i j6=i ´ in general. For instance, if we assume that U (m) = R2d φ(x, y)m(x)m(y)dxdy (where, the map φ : R2d → R is assumed to be symmetric), then X N φ(xk , xl ), uN i (x) = U (m(xj )j6=i ) = ∆xj U (mN (xj )j6=i ) = h k,l6=i so that X ∆xj uN i =2 j6=i X ∆x φ(xj , xk ) + j,k6=i X Tr( j6=i ∂2φ (xj , xj )) ∂x∂y On another hand, as U is bilinear, ˆ X δU N (m(xj )j6=i ) = 2 φ(x, y)mN φ(x, xj ), (xj ) (y)dy = 2 δm Rd j6=i so that h δU , ∆mi| = 2 m=mN δm (xj )j6=i = 2 Xˆ j6=i Rd Xˆ j6=i Rd φ(x, xj )∆m(x)dx |m=mN (xj )j6=i =2 ∆x φ(x, xj )m(x)dx |m=mN (xj )j6=i 8 X j,k6=i ∆x φ(xj , xk ) 6= X j6=i ∆xj uN i . Appendix A. Organization of PLL courses at the College de France A.1. Organization 2006-2007. A.2. Organization 2007-2008. • 09/11/2007 Behavior as N → ∞ of symmetric functions of N variables. Distances on spaces of measures. Eikonal equation in the space of measures (by Lax-Oleinik formula). Monomial on the space of measures. Hewitt-Savage. • 16/11/2007 A proof of Hewitt-Savage. • 23/11/2007 1rst hour: A remark on quantum mechanics (antisymmetric functions of N variables). 2nd hour: extensions on the result about the behavior as N → ∞ of symmetric functions of N variables. - other moduli of continuity (|uN (X) − uN (Y )| ≤ C inf σ maxi |xi − yσ(i) |). - relaxation of the symmetry assumption: symmetry by blocs. - distances with weights (replacing 1/N by weights (λi )). Discussion on the differential calculus on P2 : functions C 1 over P2 defined through conditions on their restriction to measures with finite support. • 07/12/2007 1rst hour: Back to the differential calculus on P2 ; application toPlinear transport equation, to 1rst order HJ equations (discussion on scaling (1/N ) i H(N Dxi uN ) discussion on the restriction to subquadratic hamiltonians). 2nd hour: second order equations. Heat equations (independent noise, common noise); case of diffusions depending on the measure. • 14/12/2007 Discussion about differentiability, C 1 , C 1,1 on the Wasserstein space. Wasserstein distance computed by random variables. A.3. Organization 2008-2009. • 24/10/08 • 31/10/2008 Differentiability on P2 through the representation as a function of random variables . Definition of C 1 , link with the differentiability of functions of many variables. Structure of the derivative: law independent of the choice of the representative, derivative as a function of the random variable. First order Hamilton-Jacobi equations in the space of measures. Definition with test functions in L2 (Ω). Lax-Oleinik formula. Uniqueness of the solution. • 07/11/2008 First order Hamilton-Jacobi equations in the space of measures: comparison. Limit of HJ with many variables: Eikonal equation, extension to general Hamiltonians, weak coupling. Discussion about the choice of the test function: is it possible to take test functions on L2 (Ω) which depend on the law only? • 14/11/2008 1rst hour: 2nd order equations in probability spaces. Back to the limit of equations P ∂ 2 uN = 0: different expressions for the limit. (A) ∂t uN − ∆uN = 0 and (B) ∂t uN − i,j ∂x i ∂xj 2nd hour: strategies for the proof of uniqueness for the limit equation (A): (1) by verification—restricted to linear eq, (2) in L2 (Rd )—requires coercivity conditions which 9 are missing here, (3) Feng-Katsoulakis technique—works mostly for the heat equation and relies on the contracting properties of the heat eq in the Wasserstein space. • 21/11/2008 ´ (Digression: Back to the family of polynomials: restriction to U (m) = Πk Rd φk (x)m(x).) Analysis of the “limit heat equation” in the Wasserstein space (case (A)): explanation of the fact that it is a first order equation - interpretation as a geometric equation. Back to uniqueness: use of HJ in Hilbert spaces (cf. Lions, Swiech). Key point: diffusion almost in finite dimension. Proof of uniqueness by using formulation in L2 (Ω). 1 P N Nonlinear equations of the form (∗) ∂t u − N i F (N D2 uN i ) = 0. Heuristics for the limit by polynomials. (Digression: Structure of the second order derivative. Key property1: U 00 (X)(Z, Z) = 00 U (E[Z|X], E[Z|X]). ) Limit equation of (*): ∂t U − E1 [F (E2 [U 00 (G, G)])] = 0. Uniqueness: as before. Beginning of the case of complete correlation. • 28/11/2008 Analysis of “limit heat equation” in the Wasserstein space (case (B)). Discussion on the well-posedness. Remark on the dual equation. • 05/12/2008 2nd hour: back to the system of N equations and link with Nash equilibria. Ref. Bensoussan-Frehse. Uniqueness of smooth solutions; existence: more difficult, requires conditions in x of the Hamiltonian (growth of ∂H ∂x ). Problem: quid if N → +∞. ∂uN ∂uN Key point: one needs to have | ∂xjj | ≤ C/N and | ∂xji | ≤ C. Known for T small or special structure of H. Open in general. One then expect that uN i → U (xi , m, t). Derivation of the Master equation for U (without common noise). Discussion on the Master equation; uniqueness. No maximum principle. Derivation of the MFG system from the Master equation. Direct derivation of the MFG system from the Nash system: evolution of the density of the players in the RN d system for the Nash equilibrium with N players when starting from an initial density m0 ; cost of a player with respect to the averaged position of the ∂ 2 uN j | ≤ C/N 2 . other players. Propagation of chaos under the assumption | ∂xj ∂x k • 19/12/2008 Analysis of the MFG system for time dependent problems: second order. Existence: H Lipschitz or regularizing coupling. Discussion on the coupling: local or nonlocal, regularizing. Case H Lipschitz + coupling of the form g = g(m, ∇m) with a polynomial growth in ∇m. A priori estimates for (m, u) and its derivatives. Case of a regularizing coupling F = F (m) without condition on H (here H = H(∇u)): a priori estimates by Bernstein method (sujet de la thèse de 3ième cycle de PLL). • 09/01/2009 Existence of solutions for the MFG system: by strategy of fixed point and approximation. Starting point: H Lipschitz and regularizing coupling. Other cases by approximation. Description of “la olla”. 1Does not seem correct: not even true for U (X) = E[φ(X)]. 10 Discussion on uniqueness for the system MFG. Two regimes: monotone coupling versus small time horizon. • 16/01/2009 A.4. Organization 2009-2010. • 06/11/2009 Presentation of the MFG system. 1rst hour: Maximum principle in the deterministic case for smooth solutions: if u0 ≤ v0 , then u ≤ v. Proof by reduction to a time-space elliptic equation with boundary conditions Dirichlet and nonlinear Neumann (+ discussion on the link with Euler equation). Proof that this is an elliptic equation. 2nd hour: generalization to the case where the initial condition on u is a function of m. Discussion of the maximum principle when the coupling f growth: not true in general. Discussion of the maximum principle when the continuity equation has a right-hand side. • 13/11/2009 Comparison principle when in the second order setting with a quadratic hamiltonian. Quadratic Hamiltonian: change of variable and algorithm to build solutions. Conjecture: non comparison principle for more general Hamiltonians. • 20/11/2009 Comparison principe: second order setting with a quadratic hamiltonian and stationary MFG systems. Comments on the convergence of the MFG system as T → +∞ convergence: convergence of mT (t), uT (t)− < uT (t) >, and < uT (t) > /T . Claim that uT (t) − λ̄(T − t) converges. Ergodic problem: comparison in the determinist setting: if f1 ≤ f2 , then λ̄ ´ 1 ≤ λ̄2 . When H(x, ξ) ≥ H(x, 0) for all ξ, then m = [f −1 (x, λ)]+ where λ is such that m = 1. Then u = constant in {m > 0}; solve H(x, Du) = λ in {m = 0} with boundary conditions. Justification by ν → 0+ for instance. Comparison in the second order setting: quadratic H quadratique. Planification problems. Approach by penalization. Link with Wasserstein. • 27/11/2009 Link between MFG with optimal control of (backward) Fokker-Plank equation: ∂t m + ∆m + div(mα) = 0, m(T, x) = m1 (x) where α = α(x, t). Pb: minimize ˆ Tˆ ˆ mL(x, α)dxdt + Ψ(m) + Φ(x, m(0, x))dx 0 Q 1 2 km Q m0 k22 . − Planing pb: Φ = Derivation of the optimality conditions. Generalization to the case L(x, α, m) which is a functional of m. Approach by optimal control of the planing problem. Leads to controllability issues. Discussion of the polynomial case. 2nd hour: First order planning pb : existence of a smooth solution. Step 1: link with quasilinear elliptic equation with nonlinear boundary conditions. Step 2: L∞ estimates on w := ∂t u + H(x, Du) (i.e., estimate on m): extension of Bernstein method by looking at the equation satisfied by w. Step 3: L∞ estimate on u. Indeed u is smooth and solves ∂t u + H(Du) = f (m) where f (m) is bounded. So it is a forward and backward solution which gives the result. 11 • 04/12/2009 Panning problem : remainder of the link with quasilinear elliptic equation with nonlinear boundary conditions Strategy for the Lipschitz estimate: again Bernstein. Difficulties = constant are subsolution and boundary conditions. Trick: use |Du|2 + γu2 . Step 1: t = 0 or t = T . TO BE COMPLETED (20mn). Hour 2 : end of the proof (TO BE COMPLETED). (18mn) : discussion on the assumptions. Difficulties when f is decreasing. • 11/12/2009 First hour: Back to the first order planning problem. ˆ ˆ ∂u Dual problem, i.e., optimal control of HJ equation. Namely inf G( +H(Du))− u ∂t ˆ (m1 u(T ) − m0 u(0)). Computation of the first variation, and shows the link with the MFG system. Comment on the fact that f = f (m) has to be strictly increasing. Generalization to second order problems. Counter-examples: (i) (reminder when H at most linear (first or second order): existence of solutions). In this case there is no existence of solution for the dual problem (at least for small time). (ii) Regularity? Normalization: H(0) = 0, H 0 (0) = 0, f (1) = 0, A = H 00 (0) > 0, f 0 (1) = a > 0. Then m = 1, u = 0 is the unique solution for m0 = mT = 0. One linearizes to get ∂t v´ − ν∆v´= an, ∂t n + ν∆n + div(ADv) = 0 with n(0) = n0 and n(T ) = nT where n0 = n1 . Stability requires that A > 0. Proposition: the linearized (periodic) problem is well-posed iif A > 0, a ≥ 0, ν ≥ 0. Proof for first order, straightforward; for second order, Fourier. Second hour: end of the proof. Second order planning problem. Approach by optimization (optimal control of FokkerPlanck equation). Yields to existence and uniqueness of very weak solutions. Main issue: regularity? Understood when H = 21 |p|2 . Theorem: when H = 12 |p|2 , and f non decreasing with polynomial growth, then there is a unique smooth solution. Generalization to the case |H 00 (p) − I| ≤ √ C 2 (conj. could be generalized to the case cI ≤ H 00 ≤ CI). Proof by 1+|p| the Hopf-Cole transformation. • 18/12/2009 Congestion problem. • 08/01/2010 Back to the congestion problem. • 15/01/2010 A.5. Organization 2010-2011. • 05/11//2010 Uniqueness of MFG equation for H = H(Du, m): Different approaches: monotony, continuation, reduction to an elliptic equation. • 12/11/2010 Uniqueness of MFG equation for H = H(Du, m): different approaches (continued): linearization, problems with actualization ???? Master equation (MFGf)2: 2Warning: missing term in the MFGf. 12 • • • • • • • (1) Heuristics: Master equation a limit system of Nash equilibria with N players as N → +∞ (2) The Master Equation contains the MFG equation (when β = 0) (3) Back to the uniqueness proof: U is monotone (4) Back to N → +∞ 19/11/2010: 1rst hour: Back to the Master equation3. Check that when ν 6= 0 the equation does not match with Nash eq for N players. Link with optimal control problems in the case of separate variables (discussion on the case of non separate variables). 2nd hour: TO BE DONE 26/11/2010 Erratum on the master equation. Interpretation of the Master Equation as a limit as N → +∞: good explanation of the various second order terms. 1) Interpretation in term of optimal control problem (β = 0) 2) Uniqueness related to the convexity of F and Φ 3) General principle for the link between optimal control and the Master Equation in infinite dimension 03/12/2010 System derived from Hamilton-Jacobi: Propagation of monotony FINIR 10/12/2010 System derived from Hamilton-Jacobi: - Propagation of monotony for second order systems - Propagation of smoothness, method of characteristics 17/12/2010 P 2 0 aα,β ∂x∂ αUxβ Propagation of monotony for ∂U ∂t + (H (DU )D)U = f (x) + 07/01/2011 0 Existence and uniqueness of a monotone solution for ∂U ∂t + (H (DU )D)U = f (x) Remarks on semi-concavity for HJ equations 12/01/2011 A.6. Organization 2011-2012. • 28/10/2011 n n Analysis of equation : ∂U ∂t + (U.∇)U = 0 (where U : R × (0, +∞) → R ). - case U0 = ∇φ0 : then U = ∇φ with φ sol of HJ equation. - case U0 monotone, bounded and Lipschitz continuous: existence and uniqueness of a monotone, bounded and Lipschitz continuous sol, which is smooth if U0 is smooth. Generalization to ∂V ∂t +(F (V ).∇)V = 0, provided F and V0 monotone (since U = F (V ) the initial equation) Explicit formula: linear case, method of characteristics: solution is given by U = (U0−1 + tId )−1 as long as there is no shock. Quid in general? i Propagation of the condition ∂U ∂xj ≤ 0, j 6= i. • 04/11/2011 Back to the system ∂U ∂t + (U.∇)U = 0. ∂Ui i Propagation of the condition ∂U ∂xj ≤ 0, j 6= i. Consequence: ∂xi is a bounded measure. A striking identity: if U is a classical solution of div(F (U ) det(∇U )) = 0. • 25/11/2011 3Warning: missing term in the MFGf. 13 ∂U ∂t +(F (U ).∇)U = 0, then ∂ ∂t det(∇U )+ • 09/12/2011 Propagation of monotony with second order terms. • 16/12/2011 Analysis of ∂U ∂t + (F (U ).∇)U = 0. Following Krylov idea: introduce W (x, η, t) = U (x, t).η. • 06/01/2012 Analysis of ∂U ∂t + (F (U ).∇)U = f (x): existence of a smooth global solution under monotonicity assumptions. A priori estimates. • 13/01/2012 14