>> David Wilson: So I think in the interests of time we'll just go ahead and get started.
We're a little bit, ten minutes over time.
So I guess everyone knows that the nature of this little Saturday workshop each fall is
that each of the constituent participating states, Oregon and UVC, and University of
Washington nominates a representative.
And it's my pleasure to introduce the representative from Oregon. Son Luu Nguyen. Son
is just joining us this fall from the past two years as a post-doc at Carlton University in
Ottawa. And he's now a post-doc at Oregon State University. And with that -- and who
works in numerical methods for stochastic differential equations and optimization and
control.
So Son.
>> Son Luu Nguyen: So thank you for giving me an opportunity to talk in seminar.
So I'm going to talk about linear quadratic Gaussian mixed game with continue
parameterized minor player. This one is joint work with Dr. Huang in Kato University.
So I'm going to talk about the midfield dynamic games model. So this model contains
two kinds of players. The measure players we have strong influence to other players and
the minor players have quicker influence to other players.
So here's contents of my talk. So this talk is related to the stochastic game with large
number of [inaudible] iterating players, and it is motivated from the complex system
arising in engineering and cycle academics problems.
For example, in [inaudible] dynamics with many [inaudible] many wireless [inaudible] with
management or surface [inaudible] [inaudible] individual radiation bricks by joining the
group.
So we can see these pictures. This is a joined group to produce bricks from [inaudible].
So these pictures are taken from National Geographic.
So for this kind of system, there are two key features. The first one is that each
[inaudible] receive a negligible influence from other individual. And the effect of the
overall population is significant to each agent.
And because the number of the players is very large, so we cannot expect that player
can collect the information of all other players.
So for the control problem we want to develop a low complexity solution that X player
needs to use its own state information.
So that's [inaudible] centralized [inaudible]. So the class of games we consider here
involve a method player and minor player. So the major player has a significant role in
affecting others and minor players have quicker influence.
And the minor players are parameterized by continuing parameter. So this result is
motivated by societal economic problems. So it describes interaction with one or more
last cooperations and many smaller competitors.
So the objective of my talk is to design -- so centralize [inaudible] and [inaudible] and
[inaudible] equilibrium property. So the method we use is backwards SDE. Backwards
stochastic differential equation.
So let's consider the dynamics of the players. So the dynamics of the measured player is
described by a linear stochastic difference equation like this and the second equation is
dynamics of [inaudible] minor player.
So X stands for the average state of minor players. So it is average space of -- we want
to accept N. And use -- and you are control. And the minor players are parameterized
by smart parameter data sub I and we assume that's a Brownian motion W knot and WI
are independent standard Brownian motion and the metrics B and F and D are
continuous, constant matrix and have combatibility mentioned and so the cost for the
measured player have quadratic form. And the cause of the minor player have this one.
With midfield coupling term. So X and here stands for the midfield term.
So appearance of X sub zero in cause of the minor player show the strong influence of
the major player to the minor players.
So for the game problems with larger number of players. So the centralized epsilon S
equilibrium was considered by one Peter Canes and [inaudible] in a series of paper.
And so this centralized control for large populations stochastic multi-agent system was
considered by Lee and Tang in 2008 and the midfield equilibrium condition for non-linear
system was considered by [inaudible] and Lyons in 2006 and 2007, as application for
Markov effects in these tree dynamics with many firms was considered by (inaudible).
So more related to our talk, it's the case that the parameter set theta, when the set is
finite, then the problem was considered by Juan in 2010 and in this paper he used the
Markov state of [inaudible] approach.
So say he aggregate own parameters, [inaudible] equals parameters in the same set. So
the system is reduced to finite dimension problems.
So in our case, when the parameter set is infinite, then it's not applicable. So we choose
the midfield process as a random process and we use backwards SD approach by
[inaudible] and the [inaudible] multi minor major players was considered in [inaudible] in
2002 [inaudible] and follow players.
And we also used a maximum principle of SDE. So another maximum principle for
midfield problems was considered by Anderson and [inaudible] in 2010.
So we assumed that initial state X sub zero are independent and have uniformly bounded
second moment. And the joint empirical distribution of the parameter and the expectation
of the initial states converse quickly to some distribution.
And we assume that some metrics A, B, F, D are continuous functions for, with respect to
theta and the parameter set theta is a complex set in D dimensional space.
So when the number of the player tends to infinity then the midfield term tends to
stochastic process. And based on the linear property of the equations of the player, then
we assume that the limit has this form.
F1, F2 and GSD are deterministic function and this function will be determined later. And
after we approximate the midfield term by stochastic process Z then we get a limiting
problem.
So we get a limiting problem for the major player. So in here the midfield term is
replaced by both stochastic process. So we get a stochastic control problem with a
random coefficient Z.
So using the method by this much in 1976, we can [inaudible] problem and we can show
that the ultimate control for the limiting problem has this form. You may not have this
form. This control depends on the state of the major player. And another process, a new
sub zero, a new sub zero is a stochastic process, and it is of unique solution to a given
for backwards SDE.
And the metrics did not see here is the solution of the equation.
And now we apply that ultimate control and we want to show that the optimal dynamics
have this form. So this form is similar to the form of the process Z with the three
coefficient functions.
So the first function F knot F knot of 1 is function of F1. And we can see that the second
coefficient FX knot bar is a function of F2 and the last function G of X knot bar is function
of G.
And we can compute this operator directly. And here the [inaudible] fee sub zero is a
unique solution of given of the system so we can compute the dynamics of the
measured -- the optimal dynamics of the measured player expressly.
So after we can solve the limiting for the problem for the measured player, we can apply
the optimal dynamics and the process Z to obtain -- limiting optimal control for the minor
player.
So in here the midfield term is replaced by stochastic process Z and is a cross-function X
sub zero is replaced by optimal dynamics X knot bar. And the midfield term here is
replaced by Z.
So we get another optimal control problem with random coefficient Z and X knot bar. So
by similar way we can solve that problem and get the optimal control of UI bar in this
form. And UI bar still depends on the state of the agent I and it also depends on another
stochastic process, but this stochastic process is known.
And we can represent the optimal dynamics for the agent I in this form. So the difference
here is that the dynamics of the agent I also depends on the Brownian motion WI and
also depends on the initially state of the agent I.
And the coefficient functions can be computed by similar way. So the first coefficient
function is a function of F1. The second coefficient function is function of F2 and the
kernel J here is the function of Z.
And so the third function here and the last kernel can be computed directly. And here phi
is a solution of the audio system. All these coefficient functions are linear continuous
function.
So after we solve the limiting control problems, we can obtain the optimal dynamics for
the minor player and we have a consistency condition. So Z of T should be the limit
when N tends to infinity of 1 of N of the optimal states. Dynamic states. And by the
formula for the dynamics of the ultimate control for minor player, we get the second
equation.
So based on this equation we can find the process Z. So we denote the operator gamma
1 equals the average of gamma theta 1 with distribution theta.
And for us another term. The function of gamma 2 have this form and the function of
gamma 3 -- I'm sorry -- lambda is average of lambda theta. Then it's cheaper function F1
F2 Z should be the solution of this system of equations. So the solution of this system is
the consistent solution to the [inaudible] certainty equivalence equation system.
And under [inaudible] Simpson we can show that this system has unique solution F1 F2
G. And when you can solve this system you can obtain the approximation for the midfield
process.
Now we can apply the midfield approximation to the finite system with last number of
minor player. So we have seen that -- we define the ultimate control for the finite system
by this formula.
This is similar to the control of limiting problems. But the difference here is that we only
have finite number of minor player.
So each player can use information from its own state and the major players Brownian
motion.
And after this control is applied we get dynamics for the major player and the minor
players. And we get a midfield here.
And we can estimate the difference between the approximation process ZT and the
midfield from the -- finite minor player. So it is, this difference tends to 0 when N tends to
infinity. And we can show that the set of control U hat J has epsilon asymptotic nice
equilibrium property.
So it means that the cross-function for the Jth player, if we change the control of the Jth
player, then if you move of the cross functions for the Jth player is between the ultimate
control for the limiting problems and the error should be epsilon. And epsilon tends -- is
very close to 0. It tends to 0 when the number of player tends to infinity.
So let's consider scalar model. So we consider a scalar model for any player with certain
coefficients here. Here we consider dynamics for the measured player at this equation
and the equation for the minor players by this equation, and we don't put the midfield
term dynamics of the players. And we have seen the parameteric set is one-to-one and I
here has a uniform distribution of native one-to-one.
So we can compute -- we assume that the cross function of the major player and minor
player by this formula. And we can compute directly the approximation of the midfield
process.
So we can compute F1 and F2 by numerical method and here's a kernel J and we can
obtain the midfield approximation.
So when we can solve this one we can construct the centralized strategy for each player.
So for the case that the parameter set is infinite set, then Markov state augmentation
cannot be applied, and we can design the strategy based on the function space and we
can solve the system of deterministic equations so it is computation efficient.
So next we consider a midfield model with a stronger influence of the major player. We
consider the dynamics of the major player and the minor player with stronger influence of
the major player.
So appearance of the major player is the dynamics of the minor player shows a stronger
influence of the major player.
And in this case we assume that A, B, F and G and D are fixed metrics and it doesn't
depend on parameters. So in this K, problem is complicated, more complicated in the
sense that if we change -- if we change the control of the major player a little bit to you
know plus delta U, then the dynamics of the major player will change delta X knot and
because appearance of the major player in the dynamics of the minor player, can the
state of the minor player change delta XI and because all minor players will be changed,
so it causes a change in the midfield term.
So when N tends to infinity, we approximate the midfield term by Z. But now it's more
complicated. When U is changed by the delta U, then X 0 also change and the midfield
change and it is described by this system.
So to sum this problem, we need more flexible definition for the control problem. So we
say that X knot bar, unit bar and equilibrium solution for with respect to process Z star, if
we have this inequality. X knot bar satisfy the dynamic equation and the cross-function
for unit bar is smaller than any other variation of unit bar.
And the equations of variations is given in this equation. And the cross function is the
same as -- the cross function is the first part. So we can -- by this way we can also
construct the limiting problems, limiting control problems with the same limiting process Z
bar and we need to find -- we want to find equilibrium solution X knot bar unit bar with
respect to Z bar. And when we can solve this problem, we can construct the limiting
control problem for the minor player.
And for the minor player, we have X knot bar is an optimal dynamics from the first
problem and Z bar is given here. So by the same way we can have this representation of
the X knot bar. X knot bar also have this form.
X was the first function is the function of F1 and the second function is F2 and last
function is function of J.
And similarly, the dynamics of the minor player also have this representation. So first
coefficient function of the minor player also is a function of F1 and the second function is
the function of F2 and the fourth kernel here is a function of G.
And the third function and the last kernel can be given expressly by this function. So,
again, we have reached a state of all minor players and we get another equation to
obtain the limiting process Z. And we obtain a fixed point operator and from that we can
solve for the limiting midfield process and then we can process similarly and we can
obtain the epsilon nice equilibrium property and we can estimate the performance of the
approximation.
So this method of reduce computation -- so deterministic function space and for certain
problems we can compute directly by numerical method. So thank you for your attention.
[applause]