>> Gireeja Ranade: I'm actually pretty excited to be giving this talk here today because the
problem that I'm going to be talking about is a problem that I started thinking about actually
during the beginning of my PhD. And I had this problem and I really, really wanted to be able to
present it to my qual committee during my qualifying exam. I worked really hard but I couldn't
really get a result and so I said okay. Maybe it won't happen for my qual, but maybe in my
dissertation I can say something about it. One year went by; two years went by. Dissertation
came and went, still no result. And so finally, now here is a postdoc at MSR and working with
Yuval and Jian, I'm able to say something about the converse result here. I'll tell you about this
long-term problems that I've been thinking about. Before I get into details, the problem that
I'm going to be talking about today is really motivated by developments in the internet of
things. Today we have these control systems like package delivering drones or self driving cars
or, you know, autonomous home systems that sometimes cats ride on, that are interacting
physically with the real world. And control is a natural way of trying to understand and model
these systems. But when we are dealing with these systems, we might be controlling them or
observing them over unreliable or noisy communication channels or the models themselves
might not be precise. They might be changing rapidly and so we need to really be able to
understand kind of the fundamental limits on systems like this. For example, if we were to
think about a drone that we are trying to control, but maybe we only have some kind of
unreliable measurements about its actual position, yet we want to have it fly in a certain way.
How can we understand systems like this? In today's talk I'm going to be talking about a very
simplified model for problems like this. And we will explore what we will say about this model.
This simplified model is going to take the following form. I'm going to have a system whose
state I'm going to monitor and I'll make all of this precise. This state will be given maybe some
noisy version of this that will be given to a controller. And the controller, then wants to put an
input back into the system so as to stabilize the system or have the system go according to the
trajectory that the controller wants it to go in. To make this somewhat more precise, let's think
of x being the state of the system and n as a time index and I really want to think of x as being
the error from the desired trajectory. We really want x to be small. We want it to be going to
zero. Let's say we have some initial state that is random variable x not is just a normal zero,
one. And the system evolves according to the following equation, that x at time n plus one is A
times x sub n minus this control u of n. So A is a scalar. All of the system variables here are
scalars, is known to us. And this control u can basically be any function of this y which is what
the controller gets to observe about the state x. And you can use memory. It can be linear,
nonlinear, whatever. So now the question is really what is this observation and how is this
observation, what is this kind of thing we are going to consider. In this set up I want to consider
this channel as being multiplicative noise channel, so I want to think of y as being zn times x sub
n where zn is drawn IID each time from let's say this normal 1 sigma squared distribution. And
sigma squared is let's say known to us. And these kinds of multiplicative noise channels can
model synchronization errors in systems or they can model quantization errors, for example.
For particular kinds of communication that is happening over, for instance a fast fading channel.
These are the kinds of things that can be modeled and that we are trying to capture in this kind
of system. Yeah?
>>: And zn is independent of the text approaches?
>> Gireeja Ranade: Zn is independent of all of the x's and they are all IID overtime.
>>: If it was an additive problem it would be a simple solution.
>> Gireeja Ranade: Yes. The additive problem is very well understood. That comes down to
basically being, and filter. And the additive noise problems actually have been widely studied
and are quite understood, but multiplicative problems are problems where we don't have a
precise model and are significantly more challenging. And this is some work that is kind of
moving in that direction. Finally, with all of this, what is our goal? Our goal is going to be to
make sure that our system error does not grow too large. In fact, in particular, we want to look
at the second moment of the system. And we want to make sure that this is always finite. This
will be our goal going forward. There has been a lot of work thinking about problems that are
similar to this in the control literature. I don't have time to go through all of this, but I want to
point out, for example, the common filtering work is obviously very relative, which considers
additive noises. But the one piece of work I do want to highlight is the case where the system
state x is observed over a rate limited channel. The controller gets a finite number r bits about
the system state and the system evolution is exactly the same. Here we have a very nice and
clean result that says that this system is stabilizable in a mean squared sense if and only if the
information transfer across this channel is larger than log A which is the growth of the system.
And there's a nice intuition about this that I can explain later if people are interested. But with
this background I want to now move towards the bulk of the talk. I'll first talk about a linear
achievable strategy to say how can we possibly stabilize this system and think about the most
simple thing we can do to do this. And then I'll move to the second part of the talk which will
be the converse result, which will be kind of the main heft in the talk. Let's try and understand
this problem and let's say what can a controller do. And let's consider the most simple linear
strategy, by which the controller says what do I have. I have y. Let me scale it somehow,
appropriately, and what's the best I can do? Let me not even try to use memory, nothing. Let's
do the simplest possible thing. Let's evaluate this. We just substitute this into our equation.
We get the following evolution. Now when we think about this we are interested in the second
moment, so let's square this. When we square this we see that what is really of interest is this
quantity the expected value of A minus d times zn squared. This we know exactly how to
calculate. We can just rewrite this as A minus d squared plus d squared sigma squared. I want
now to have this system be stable, so I want this factor to be less than one. In fact I wanted to
be as small as possible. So what do I do? I differentiate. I can find the optimal d. It is nothing
complicated, simple stuff. And I solve the question I find that d is equal to the optimal d is A
over one plus sigma squared. Further, I can say well when is this factor less than one, which will
tell me when the system is stabilized using this strategy. And I get some condition like this.
This is great but I want to actually rewrite this condition in a different form and I want to say
that this system is mean square stabilizable will using this linear strategy if and only if log A is
now smaller than half log one plus sigma squared. And this formula is kind of cute. It reminds
us of the half log one plus s and r formula from information theory. So this is kind of nice. This
cuteness brings me to one of the nicest pictures of the talk. For those of you who are
wondering where the title is coming from, we are saying in this regime the system is definitely
not any kind of tiger. It's actually kind of cute and came if log A is bounded in this particular
fashion. Now that we have this, I guess we can move to the more ferocious part, which is really
the converse. What kind of converse with we want to have, would we expect to have? We
only consider it in the past achievability we considered this memoryless linear simple strategy.
We can actually improve that to show that actually this particular linear strategy is the optimal
linear strategy. Memory doesn't help in the linear case. But we don't know if that bound was
coming from the fact that we were only looking at linear strategies or if some sort of nonlinear
memory full strategy can do much better. So what we would really like is something that says if
log A or if A is larger than a threshold, then your second moment must be unbounded. It had
better explode to infinity. It turns out I'm going to prove a slightly different result is actually a
stronger result that says that I'll give you a threshold for which stability in any reasonable sense
is not possible. And so the statement of the result that we have is the following. If A is larger
than some threshold that depends on sigma, then for any control strategy we want to look at
the probability of x being bounded within the box. And this probability of it being bounded
itself is going to go to zero for any box that you choose. And so this obviously implies that your
second moment is going to be exploding because you cannot even be within a certain box of
your choosing. So forget anything about the second moment. This then brings me to the next
nicest picture in the talk. This is not a system you want to try to tame. Everyone with me until
now? Okay. How do we go about proving this result and that's what I'll be talking about for the
rest of the talk. The first step that I really want to do is I want to think about a reduced system.
I want to start ignoring this growth factor A. I am going to drop the A here and consider the
system with A equals one and for the rest of the talk I will explore only this particular system.
Here what we can see is that if I want the original system to be bounded within the box says M,
I want this non-growing system to fit into an exponentially shrinking box. So I want to instead
look at this probability that XM with this system is now in this box that keeps shrinking by a
factor A as time goes on. Now that we are looking at this modified system, given that we have
all of these past results on rate limited channels and how to understand them, it's natural to
think you know you have some kind of channel here and additive noise is well understood as
was pointed out. Can't you build on some of those techniques to get somewhere? So the
channel that we have is really something like Yn and Z times Xn so you can say that log Yn is just
log Xn plus log Zn and if we can bound the rate of this channel that might give us an approach.
But it turns out that in our particular problem, the range of Xn is actually unbounded and we
don't know what this U can do to this distribution of X. It can change things in any way and so
getting a bound on the rate across this channel is going to actually be, like we can do that. So
the standard approach of having the rate limited channel is actually going to be hard and we
can't use this approach. Yeah?
>>: Can you say at least that the posterior distribution of Xm is always Gaussian?
>> Gireeja Ranade: No you can't because you're adding basically a correlated random variable
U and U will be dependent on X. So you don't know what the controller can do.
>>: So the conditional distribution of Xn given the entire sequence of U's?
>> Gireeja Ranade: The conditional distribution of Xn given the entire sequence of U's will be
Gaussian.
>>: Then you take that mixture of Gaussians and you use [indiscernible]
>> Gireeja Ranade: But yeah, but you don't actually have Gaussianity and so you can't actually
say anything. Okay. What is the strategy that we want to look at? This was one of the key new
ways of looking at the problem. The approach of this problem is different from the way control
theorists and information theorists have been looking at this problem so far. What the proof
does is actually it looks at this object which is the density of the system state X at time n. I'm
just conditioning on basically everything that the controller knows up to time n. I'm looking at
this object and remember, what do we want to do? We want to look at the probability that Xn
is going to be in these intervals that are shrinking exponentially. If we can say that actually this
density is upper bounded and we can give some exponential upper bound on this, then we can
say that the controller cannot know with this exponential precision where the state is. And if
the controller doesn't know where the state is, it's not going to be able to hit it and control it.
This is the approach that we are going to take to prove that the converse style result. To do this
we really want to understand the effect of these observations because that's what's going to
make this density become peaky. What do we want to really understand? We want to
understand how does this density change every time the controller gets a new observation?
Because that's how the controller can do better. So I really want to compare this pink density
density based on everything the controller new at time n minus one to the black density which
is now with the new observation included. And I can just rewrite this in this form and now it's
natural to want to use Bayes rule to kind of separate these effects out of. When I use Bayes
rule I have this pink term which is kind of the term that I want to relate this to, but then I have
this denominator and it's like the density of y conditioned on all the paths and I don't really
know how to deal with this. It's something complicated. So we said okay. Let's try to avoid
getting into this mess. And note that here there is no term X, so the idea is let's take the ratio
of densities and so that this denominator term will just cancel out. Because I like being able to
deal with this recursive format, this term I'm going to be able to deal with eventually. So I take
the ratio to be able to remove this denominator. Now I want to look at the ratio of the
densities of f of little x given my observations divided by f of little x at some w given my
observations. And I'm just taking the logs here to make things a little bit easier. Now my
denominators have canceled out and what I have here in pink is this nice recursive term that I
want to recursive on to get my bound. Now there is this blue term here which is basically the
conditional density of I given what X is. But what do we know about y? I is just a Gaussian
times X, so if I know that Xn is equal to some little x the density of Y's conditional density is now
just the conditional density of a Gaussian, which is really easy for me to handle and deal with. I
can actually explicitly write this thing up. This I can deal with. This is recursive. We are excited.
Everyone is excited. We can almost solve this problem. But now what we need to do is we
need to bound this term. And we have these things in the denominator. X is going towards
zero and now we have these things that are getting small that are in the denominator. Finding
a bound is suddenly going to be a bit of a problem. So this is where the second idea in the talk
comes in, which is that we have a genie that provides side information that helps bound this
term in the denominator. It's interesting how this works. Let me make exactly what is
happening precise. Remember that we are proving a converse bound. We give the controller
some extra information and prove that the controller cannot do anything. That's always fine.
You can always give this extra information. So what is this side information we give? The side
information is essentially a quantized version of the log of Xn. What we do is we find Xn and we
localize it in the binary interval between say 2 to the minus Kn and 2 to the minus Kn plus 1. A
genie comes along and observes this and hands this Kn to the controller. And remember, what
was the channel that the controller already had? It already had log Yn is log Xn plus some log
Zn. So it already had a kind of noisy version off of the log effect. So it's not like this genie is
actually giving that much extra information. But the clever way in which its used allows us to
actually get the bound that we need.
>>: So this [indiscernible] actually depended on sigma. The Y depended on sigma?
>> Gireeja Ranade: The Y depended on sigma and sigma will show up in the bound. I'm
somewhat suppressing that right now just to be able to give the flavor of the proof, but the
sigma does show up. These constants will all depend on sigma. Basically, the precision of the
operation will dictate the Kn, yeah, exactly. But what this gives us now is this bound on the
magnitude Xn. And the way the proof is going to proceed is it's going to use these Kn's in a key
way, because now, what did we want to do? We wanted to measure how fast Xn was
approaching zero. And these Kn's are basically exactly measuring that. First they give us a
lower bound on Xn, which is what we wanted. But secondly, the problem is now becoming, is
we want to measure basically how the Kn's are proceeding. And if we want and exponential
bound on X, what we really want is we want now a linear bound on how the Kn's are increasing.
And really what we will be doing is we will be using kind of the properties of the density of Xn
to bound Kn and then use Kn to bound Xn and so it's kind of this back-and-forth between Xn
and Kn that is going to let us get to the conclusion that we want to. But now thanks to our nice
and friendly genie, we can go back to the proof of where we were and actually have a bound on
this term. This trouble has now gone away. And now with some algebra that I'm really not
going to talk about so much, we can have the kind of exponential bound that we wanted, which
is to say that the density of X can be bounded basically by 2 to the Kn and there is some
constants here and the constant will actually depend on the Z's and the sigmas. But now the
problem is we still don't quite understand these. These K's are still random. The question is
now reduced to understanding basically how do these Kn's behave. And here I'll introduced the
last lemma in the talk, which is the Kn's will actually grow at most linearly. In fact the precise
statement will be that the probability that Kn minus K not is larger than some constant times n.
This probability has to do the zero. And the way that we are going to prove this lemma is to
look at the increments. We are going to look at Kn plus one minus Kn and the probability that
this is very large. Basically, in one step these K's grow by a lot. We will show that this actually is
kind of like a geometric. This has these exponentially dying tails and once we have this we can
have that these Kn's grow at most linearly.
>>: If this is very precise then you sort of know K and you can ignore Y? Presumably, it's not
safe enough to do it. Is it? What's [indiscernible] anymore? If I have the distribution of Kn, it's
possible.
>> Gireeja Ranade: Yeah. These constants are all not actually constants. There is a
randomness. These are random constants that depend on the Z's and the Y's. So just in the
spirit of giving a flavor to the proof, I'm hiding that fact, but there is a lot more work that needs
to go into actually showing that these things work. But yeah. We can actually completely
ignore the Y's because of that. To give now a flavor of how this proceeds, is this is the
probability we want to look at which is the probability that Kn plus one minus Kn is larger than
L. So what does this mean? Kn is basically looking at the order of magnitude of Xn. This means
that if Kn plus one minus Kn is very large, it means that Xn plus one must've been much smaller
than Xn. Xn plus one suddenly became much closer to zero. In fact, he became closer to zero
by this factor of 2 to the Kn plus L.
>>: [indiscernible]
>> Gireeja Ranade: Closer by 2 to the L. And when is that going to happen? What does that
mean? That means that basically at time n the controller really hit it home. The controller got
really, really close to Xn. So basically, this Un at time n was really close to Yn, so this was also
bounded by 2 to the 2 minus Kn minus L. But now let's look at this probability. This is where all
of our work on bounding the density of Xn is going to come in handy because U is now
measurable by Fn. You would just depend on all of the observations of the past and so this
probability I can just rewrite as an integral of the density of X. And this integral is going to be
over and interval J that is basically around X and is of this length.
>>: Around?
>> Gireeja Ranade: Around?
>>: Around Un.
>> Gireeja Ranade: Un is varying around Xn.
>>: [indiscernible] is deterministic now.
>> Gireeja Ranade: Okay, yes. Yes, okay. So we are looking basically at this integral in this
small interval and now we have a bound on this density. Basically, we know that this guy is
bounded by something like this. We know that the length of J is of this order. And so now we
have this expression. These Kn's cancel and we get now that the probability we wanted is
decaying as 2 to the minus L, just like we wanted. This gives us exactly the lemma and once we
have this we have now proven that the Kn's increase that most linearly with this known rate
which means that the Xn's can decay at most exponentially and once we have that, basically,
that gives the converse result. And so this bound is far from tight and there are still some open
questions, but yeah.
>>: That answers my question. I thought in an earlier slide you presented and only if…
>> Gireeja Ranade: For the linear strategy.
>>: Ah, okay.
>> Gireeja Ranade: If and only if for linear strategies.
>>: So you don't know that linear is optimal. You know that it is approximately optimal in the
sense that you have a converse…
>> Gireeja Ranade: I don't even know that it is approximately optimal. I have other reasons to
believe that it is going to be, that you might not be able to do that much better than linear. But
I don't have a proof for it at all. I don't know that it is optimal. Like non-linear controllers can
improve my a little bit, but I don't know that they can improve by a lot. Yeah, I don't know the
answer to that.
>> Yuval Peres: Okay. Let's thank the speaker. [applause]. Any other questions?
>>: Is this available online?
>> Yuval Peres: On request.
>> Gireeja Ranade: Yes, basically, yeah.
>> Yuval Peres: Any other questions?
>>: In your motivation you talk about model misspecification, which in your model I would
interpret it as something like the constant A is not known, but you know that it belongs to some
interval of values. Can you say anything about robust control policies the stabilize the tiger for
a range of optical models [indiscernible]?
>> Gireeja Ranade: Thinking about actually A not being known precisely, is very, very
challenging and there aren't really any results that I know. I know of one classical result that
talks about a very limited case of that. I tried thinking about it a lot, but I don't have a result
here. The model misspecification motivation really comes from, for example, if you think about
the standard common filtering model. You assume that your Y is C times X plus some may be
additive noise. And here we are thinking maybe this C is unknown, this observation gain is
unknown. And I have the results that talk about cases where the gain on the controller is
unknown, so you have some distribution on it but you don't know precisely what it is. It might
be changing. In that regime I also can talk about some robust control notions. And maybe I can
tell you about it off-line at some point.
>> Yuval Peres: We can take other questions in the coffee break. [applause]