Reprint from Proc. 6th IFAC Symposium on Identification and
System Parameter Estimation, Washington, D.C., June 1982.
LIDS-P-1216
SOME CRITICAL QUESTIONS ABOUT DETERMINISTIC AND STOCHASTIC
ADAPTIVE CONTROL ALGORITHMS*
Michael Athans
Laboratory for Information and Decision Systems
Massachusetts Institute of Technology
Cambridge, Massachusetts 02139
Lena Valavani
Laboratory for Information and Decision Systems
Massachusetts Institute of Technology
Cambridge, Massachusetts 02139
Summary
The purpose of this informal paper is to discuss certain robustness
issues associated with existing adaptive control algorithms.
A modeling
framework for incorporating high-frequency unknown dynamics in the adaptive
control framework is suggested.
Possible fundamental limitations of existing adaptive control algorithms are also discussed, with emphasis upon their
closed-loop stability properties in the presence of unmodeled high-frequency
dynamics.
INTRODUCTION
1.1
One should not blame the theory for this
state of affairs.
Elegant and useful theoretical advances have been made in the last
decade, and especially in the past three
years, that have unified diverse approaches.
The difficulty appears to be that some of
the hypotheses needed to rigorously prove
the theoretical results are too restrictive
from a practical point of view.
Hence, new
advances in the theory are necessary, by
making different assumptions which better
reflect the desired properties of physical
control systems.
Background
The development of a systematic design
methodology for the synthesis of practical
self-adjusting control systems which can
maintain first stability and second performance improvement, in the presence of rapid
and large variations in the open-loop dynamics, represents a very important generic
goal in control systems engineering, in view
of its wide applicability to industrial and
defense applications.
The
so-called "adapdefense applications.
The so-called
"adaptive control problem" has received attention
by theoreticians and practitioners alike for
the past 25 years.
By practical we mean that the adaptive con- trol loop must adjust its bandwidth (crossover frequency) in such a manner so that it
does not excite unmodeled high frequency
About a dozen books and
hundreds of articles have been devoted to
the subject; different philosophies have
been
deeoe
mdlreeec
dtive
dynamics. To put it another way the adaploop must remain stable in the presence
of unstructured modeling uncertainty which
always exists and cannot be adequately
and cannot be adequately
modeled in any physical system.
On the
been developed (model reference adaptive
control, self-tuning regulators, dual-control methods, multiple-model adaptive control etc.) and a variety of (mostly academic)
examples have been simulated.
other hand, the adaptive control system must
also be able to provide performance improvement in the case of plant structured uncertainty, typically exhibited when the parameters in the differential equations that
are used to model the plant in the low frequency region vary within a bounded set.
The adaptive system must exhibit good command-following and disturbance-rejection
properties in the low frequency region where
the structured model uncertainty predominates.
In spite of the intense research activity,
it is the opinion of the authors (who have
actively contributed to the literature) that
there is a significant gap between the available methodologies and the potential applications.
To put it bluntly, we do not believe
that any of the available adaptive control
algorithms can be routinely implemented on a
real system and guarantee even the stability
of the closed loop process in the presence
of the inevitable unmodeled high frequency
dynamics.
* Research supported by NASA Ames and Langley Research Centersunder grant
NGL-22-009-124.
186
We believe that there may exist a fundamental conflict in many adaptive control
schemes. To compensate for structured uncertainty and performance the adaptive
scheme may wish to increase the cross-over
frequency. On the other hand, the presence
of unstructured uncertainty places an upper
bound on the crossover frequency in order
to maintain stability. Thus, the soughtfor practical adaptive algorithms must be
"smart" enough to recognize this fundamental conflict, and adjust their cross-over
frequency.
investigation into the nature and properties
of several available direct control algorithms. The focus of the analytical effort
was to understand
As we have alluded above, the mathematical
assumptions that have led to all available
adaptive control algorithms have taken into
account the existence of structured uncer-
To gain basic understanding it was assumed
that the controlled plant was a simple first
order system; the rationale was that if undesirable performance and robustness charac
tainty but they have neglected completely
teristics were encountered for first-order
(a) the dependence of the closed loop
adaptive system bandwidth upon the
amplitude and frequency content of
the reference input signal,
(b) the robustness of the adaptive control system to unmodeled high frequency dynamics,
(c) the impact of sensor noise.
the issue of unstructured uncertainty;
systems one could certainly conjecture that
unhappily, the available algorithms (that
we have investigated) are very vulnerable
to the presence of unmodeled high-frequency
dynamics because the closed loop system
becomes unstable.
the same problems would arise in more inA recent paper [2], describes an analytical
technique, based upon linearization, which
we call the final approach analysis, that
If classes of practical adaptive control
algorithms were available, then numerous application areas would benefit in both the
military and commercial sector. Advances
in microprocessor technology allow the engineer to implement in real-time the noninee,
toimplement
in real-time the non-with
linear, time-varying algorithms necessary
to
the
implement
adaptive
dynamic compencompento implement the adaptive dynamic
tor necessary to stabilize and improve the
performance of a plant with poorly understood characteristics.
1.2
Current Research at MIT: Theme
and Philosophy
For the past 2 years an intensive study of
can be used to analyze the dynamic properties of several available direct adaptive
t
ofies
several available direct adaptinuoustime case and the discrete-time case.
In
particular this method can be used to predict the behavior of the adaptive systems
dit the behavior of the adaptive systems
respect to parameter convergence, sensitivity to unmodeled dynamics, and impact
of observation noise.
As explained in more detail in [2], the
final approach analysis method is valid
during the final stage of adaptation in
which the output error is small. During
this phase one can linearize the general
nonlinear time-varying differential (or difference) equations linking the dynamics of
ferene) equations linking
characteristics of existing direct adaptive
the dynamics of
control algorithms has been conducted by
Drs. Athans, Stein, Valavani and Sastry as-
the output error to those of the parameter
adjustment algorithm.
One then obtains a
set of linear differential or difference
udet Theequations which are either time-varying or
behavior of existing direct adaptive control
time-invariant depending upon the nature of
sisted by several students.
The initial
s
s
emphasis was to understand the transient
dynamics
observation
noise. to unmodeled
algorithmsand
and
their robustness
dynamics
and
observation
noise.
It
to analyze
the
It then
then becomes
becomes possible
possib) inputs
and outputs
behavior of the linearized dynamics using
available results in linear system theory.
When the resultant dynamics are timeinvariant, even simple root-locus type of
plots can be used to predict the asymptotic
performance of the adaptive system with respect to oscillatory behavior and possible
instability in the presence of unmodeled
dynamics.
The first phase of this research was devoted
to digital simulation studies and a brief
paper [1] described the simulation results.
It is self evident from the simulation results that no consistent pattern with respect to the adaptation process could be
predicted. Nonetheless the simulation resuits confirmed our suspicions that the
class of adaptive algorithms considered were
characterized by
(a) high-frequency control signals
characteristic of a high-bandwidth
system,
(b) the extreme sensitivity of the algorithm to unmodeled high-frequency
dynamics which can result in unstable
closed loop behavior
(c) lack of robustness to observation
noise.
The final approach analysis has been used
to analyze the behavior of the adaptive
systems when the algorithms of Narendra and
Valavani [3], Feuer and Morse [4], Narendra,
Lin, Valavani [5], Morse [6], Narendra, Lin
[7], Landau and Silveira [81,[9] and Goodwin
Ramadge, and Caines [10] were employed. For
the base line first-order example considere
all algorithms considered were found (in
different degrees) to suffer from the viewpoint of yielding high-bandwidth closed-
Motivated by the simulation results of [1],
made
initiate
analtical
a decision was decision
made to
to was
initiate an
ananalytical
loop systems which can excite unmodeled
high-frequency
dynamics and lead to closedloop instability.
187
Since the final approach analysis is based
upon dynamic linearization under the assumption that the output error is small, it cannot predict the dynamic behavior of the
adaptive system during its transient (startup) phase. The simulation results of [1]
suggest that even more complex dynamic effects are present. Thus, we view the final
approach analysis as a necessary, but by no
means sufficient, step in the analysis and
design of adaptive algorithms.
The assumed SISO model structure can be
readily extended to the multi-input-multioutput (MIMO) case. Thus we shall assume
that the transfer matrix G(s) from the control vector u(t) to the output vector y(t)
is defined as follows:
(a) Unstructured modeling uncertainty
reflected at the plant input
G(s) = ZO(s,8)[I+L(s)]
(2.2)
(b) Unstructured modeling uncertainty
In spite of its limitation, we believe that
the final approach analysis is a useful tool
since it can predict undesirable characteristics of wide classes of adaptive algorithms
in the final phase of the adaption process;
these undesirable characteristics are apt to
persist (or get even worse) in the transient
start-up phase. Moreover, the final approach
analysis can suggest ways of modulating the
control gains, in a nonlinear manner, to improve performance while retaining the global
stability properties of the algorithms in the
absence of high frequency modeling errors
(structured uncertainty).- At this stage of
understanding the resultant transient startup characteristics can be evaluated only by
simulation; analytical insights are needed.
reflected at the plant output
G(s) = [I+L(s)]G (s,8)
2.3 Information on Unstructured Uncertainty
To the best of our knowledge, the existing
theoretical literature on adaptive control
(deterministic and stochastic adaptive control algorithms) has dealt exclusively with
structured uncertainty. As pointed out in
Section 1, such adaptive algorithms can
excite high frequency dynamics and lead to
the instability of the adaptive loop.
MODEL ASSUMPTIONS AND FUNDAMENTAL
QUESTIONS
2.1
Introduction
To avoid such an undesirable behavior the
mathematical assumptions must include in-
-
In this section we shall present what we
believe should be the structure of the model
for the plant to be controlled. This is nec-an
essary so as to clearly define what we mean
by structured and unstructured modeling uncertainty, and to define future research
directions.
2.2
Frequency Domain Description
-formationon the unstructured uncertainty
I(s) or L(s).
The question is: what level of
one provide to the mathematics
about the unstructured uncertainty.
Our suggested approach is to take the point
of view that we know very little about the
unmodeled high frequency dynamics. At sufficiently high frequencies I(s), and the
elements of L(s), will be characterized by
+1800 phase uncertainty.
We also do not
know very much about the order of such highfrequency dynamics (in real life every physical system is described by partial differen-
In the single-input-single-output (SISO) case
we assume that the unknown plant has the
transfer function, from the scalar control
u(t) to the scalar output y(t), given by
g(s) = g (s8)IHl+Z(s)]
(2.3)
The transfer matrix G (s,e) represents the
structured modeling uncertainty at low frequencies for values of 8 in a closed'bounded
set, 0.
The transfer matrix L(s) represents
a multiplicative modeling error, due to high
frequency dynamics etc., and will be used to
represent the unstructured modeling uncertainty.
(2.1)
tial equations and, hence, is infinitedimensional).
The parameter vector 8 in the transfer function g
e(st8)
gives rise to the structured
ton gives
g0(s,) rse to the structured
modeling uncertainty.
Our assumption is that
We claim that the only reasonable information
that a control engineer has about high-frequency errors is as follows:
g0(s,8) is a good model for the plant for low
frequencies.
The parameter e will be assumed
to be in a closed bounded set; the relative
degree of g0 (s,8) will not change as the pameter vector 8 changes.
We may even know a
nominal value of 8 which can be used in the
adaptive control algorithm before any real
time measurements (initialization problem).
The transfer function I(s) in Eq. (2.1) gives
to the unstructured modeling uncertainty.
Note that it represents a multiplicative per-
turbation to g (s,8) [11]; 2(s) represents
0°thunb
oden d
t
the unmodeled high frequency dynamics that
are alway s present in any physical system
(high frequency resonances, small time delays,
non-minimum phase zeroes at high frequencies
etc. etc.).
188
(a) They are negligible at low frequencies
(b) There is a frequency, w , in which
the magnitude of the unstructured
modeling uncertainty becomes significant (near unity).
(c) They are dominant at high frequencies.
In our opinion, it
is
imperative that
the
mathematical representation of the unstructured uncertainty be of such form, so that
the adaptive algorithm will not try to estimate it.
The way that one can model the existence of
e(s) or L(s) in the statement of the adaptive
control problem will be to assume that there
exists a scalar function of frequency m(w)
IQ(j)|l<
JZ~jw)J<
for
for all
all
w
m(X),
then Eqs. (2.6), and (2.11) imply that in
a well behaved closed-loop adaptive control
system we must have*
'
(2.4)
for the SISO case, and
aa
(L(jw))< m(w),
max -
for all w
< w
W
(2.5)
(2.12)
c
u
The bottom line of this discussion is that
the gain crossover frequency w of the adaptive loop transfer matrix cannot exceed the
uncertainty crossover frequency w . Otherwise, there always exist an unstructured
't*
illustrated in Fig. 2.1 is small at low frequencies, and is a monotonically increasing
function of w.
The frequency w
at which
i m( U)=1
(2.6)
m(~u)=
(.
l
will be referred to as the uncertainty crossover frequency.
perturbation L(s), that satisfies Eq. (2.5),
which will cause the closed-loop system to
be unstable.
2.4
I 5X "(muw)
lo
Fundamental Limitations
Adaptive Control
(?) of
We are now in a position to discuss in a
more precise manner some of the fundamental
questions that arise in adaptive control
which have not been addressed to date.
2.4.1
Existence of a Non-Adaptive
Stable
Compensator
0
/1
/
u
/.W
u icERA
rAi wY
.C
QoSro
/./F.s vEa
.CRossoEVueiY
To establish a realistic problem framework
we must assume that there exists a nonadaptive compensator, denote by K*(s), which for
all allowable values of 8e_, where 8 is the
closed bounded set constraining the range of
values of e, results in a stable closed loop
~
system. Such a K*(s) may be found by a
minimax type of design. Note that the actual gain crossover frequency will depend
upon the specific value of 8.
We denote
this dependence by w (8).
Thus, w (8) is
FIG
2.1
a
i.~
c-
defined by
The existence of the unstructured uncertainty
places an upper bound on the bandwidth of the
[G (jw (8),8)K*(jw (8)))=1
c-
max-O
(2.13)
closed-loop system, whether the closed-loop
system is adaptive or not.
We define by w* to be the maximum gain crossover .frequency for all 8Oe
w* = max w (8)
(2.14)
c
To illustrate this point, and to motivate
the basic questions that arise, let us sup-
By the assumption of guaranteed stability
pose that we design a multivariable adaptive
we must have
C
in th
sence of unstructured uncertainty
in the presence
of unstructured uncertainty
algorithm converges to a series compensator
K(s,0) (where the dependence on 8 stresses
the fact that the adaptive system explicitly
or implicitly adjusts compensator parameters
to "take care" of the structured uncertainty).
The loop transfer matrix is then
G(s)K(s,O)
(2.7)
-From the sufficient conditions for stability
using singular value tests we know that the
closed loop system would be stable if
c* <
u
(2.15)
c
u
We let
e* denote the "worst" parameter
value which defines (2.14), i.e.
w
(e*) =
(2.16)
c c
We remark that the crossover frequency w*
can be used as a bound on the maximum performance that can be expected by a non-adaptive controller.
It will define the upper
-1
amax (L(jw))<amin (I+[G (juw,)K(jW,6)]
- or, in view of Eq. (2.5), if
)
(2.8)
frequency range of good command input following and disturbance rejection.
It is
useful to think of w* as arising from the
"worst" possible value that can be achieved
the parameter vector e, i.e. _=8*.
~
mw)< a (jw(2.9)
K-1
)< amin(I
[%o (jws,6cjw,8
-by
)
(2.9)
for all frequencies w.
When the loop gain
S
is large, Eq.
(2.9) implies that we must have
G
(G (jW,0)K(jw,0)),<1
max -o
)
m(U)
2.4.2
(2.10)
The Need for Adaptive Control
If we define the crossover frequency, w , by
(G (jwtu,0)K(jw ,6))=1
C
(2.11)
To justify the implementation of an adaptive
controller, of whatever type, one must assume that the performance of the best nonadaptive controller discussed above is not
good enough for the application at hand.
*Tne notation G(A) is used to denote the singular values of a square matrix A.
*Typically one would select w -0.5w
have some reasonable phase margin
at wc-
raxG-0
K
C6
1
189
to
°
(45 )
To be specific one can assume that the control
system must be able to track reference inputs
and reject disturbances in the frequency range
Every adaptive algorithm proposed in the literature (dual control, self-tuning regulators,
model reference adaptive control), whether it
W* <w< W
(2.17)
c
u
Thus, in many applications whenever the parameter vector e is not at its worst value, one
would like to design an adaptive algorithm
which after the transient adaptation phase,
will converge to a compensator with cross'*,so that the
over frequency larger than
adaptive control system willChave "better"
command-following and disturbance-rejection
properties than the (guaranteed stable) nonadaptive design.
carries out an explicit parameter identification or an implicit parameter adjustment,
must rely on some form of output signal(s)
correlation. As we remarked above, when the
closed-loop system breaks into instability,
the plant outputs contain all sorts of signals
about which we know nothing. It is not at all
clear how the necessary output correlation
algorithm can even be designed to quickly separate the relevant information in the output
signals from those that the algorithm knows
about, and at the same time carry out rapidly
2.4.3
the necessary (explicit or implicit) change
in the controller parameters in the correct
direction!'! The results in [2] demonstrate
that many of the existing indirect adaptive
control algorithm cannot handle this phenomenon, and the closed-loop system breaks into
total instability.
C
CONCLUDING REMARKS
3.
Some Potential Problems
We next discuss how adaptive control algorithm
can get in trouble, with respect to closedloop stability. Our arguments here are
heuristic and intuitive; to be sure they must
be made rigorous so as to clearly delineate
possible problem areas associated with any
adaptive control algorithm.
The discussion has brought to the surface a
6Suppose
that
the aalau
e
pa te
whole variety of fundamental issues that must
Suppose that the adaptive
be dealt with when one includes high frequency
control algorithm converges at some time t
unstructured uncertainty in the adaptive conc1
in
a
)
by
K(s,e
denoted
to a compensator
stable
resung
manner,and that
- thetrol problem formulation. It is self evident
stable manner, and that the resuling crossthat adaptive control algorithm must be able
over frequency is denoted by ucl' having the
dover frequency is denoted by Xcl having the
to control accurately the cross-over frequency
desired property
of the closed loop system to avoid instability.
Also, rapid changes in the parameters asso(2.18)
< w<
W* < W
Suppose that the actual value of the parameter
vecto
-vector
is e1 e
c
cl
~u
ciated with the structured uncertainty can
ciated with the strbility of the adaptive
Now let us suppose that after this convergence
control loop.
has occurred, that at time t <t the parameter
control loop.
vector 8 jumps-from 81 to
Now,every
will
adaptive
control
Both
algorithm
frequency-domain and time-domain tools
Now, every adaptive control algorithm will
must be used.
take some time to process the measurements,
detect the change in parameters
or implicitly) angd change the compensator.
or implicitly) and change the compensator
output variables, that may be due to temporary
excitation of high frequency dynamics, must
isHowever, at time 2 the loop transfer matrix
G. (s,e*)K(s,6
)
(2.19)
-0
- -1
and it may well happen that the resultant
cross over frequency, say Wc2' i.e.
be quantified. It may be necessary to impose
constraints on the speed that parameter variations can occur in the structured uncertainty, and to relate the speed of convergence
of the adaptive algorithm to that of parameter
variations. This in turn would require a comprehensive analysis of the nonlinear timevarying differential or difference equations
that are characteristic of popular adaptive
control algorithms.
To ensure global stability results in the presence of unstructured
uncertainty one may have to adapt the conic
sector stability results of Safonov [12] to
the nonlinear time-varying equations that
describe the adaptive control system. In the
absence of global stability results, one
max -O
c2
- -(2.20)
exceeds the uncertainty crossover frequency
w , i.e.
(2.21)
XU <<
(2.21)
u
c2
Then at time t=t +E the adaptive loop becomes
unstable.
This iy itself is not catastrophic
as long as the adaptive system can recognize
this situation and quickly change the compensator (in this case to the safe one K*(s)).
should develop local ones, which means that
should develop local ones, which means that
one may want to further limit the range and
The problem as we see it, arises with the
nature of the information available to the
explicit or implicit adaptive controller to
rapidly detect
this
The ability of adaptive al-
gorithms to tolerate unknown signals in the
n the
gorithms to tolerate unknown signal
(explicitly
rate of parameter variations in the structured
uncertainty. At any rate, the "black
uncertainty.
parameter change and
change the compensator accordingly. When the
closed loop system becomes even temporarily
unstable, its high frequency dynamics in
L(s) get excited, and hence the measured outputs contain signals (the modes of L(s)) for
which by assumption we know nothing about,
in view of the magnitude and phase uncertainty
for w>w .
u
190
At any rate, the "black box"
approach associated with model reference adaptive control systems, in which no specific
assumptions are made upon the nature of the
abandoned.
We believe that the explicit inclusion of the
unstructured uncertainty into the adaptive
control problem will lead to new insights into
the desirable and undesirable attributes of
existing adaptive control algorithms and,
in addition, will point the way to the development of more robust adaptive control
algorithms in the presence of unmodeled
high-frequency dynamics. More research is
also needed to properly account for stochastic additive disturbances as well as for
stochastic additive sensor noise.
4.
10. G.C. Goodwin, P.J. Ramadge, and P.E.
Caines, "Discrete-Time Multivariable
Adaptive Control," IEEE Trans. Automat.
Contr., Vol. AC-25, pp. 449-456,
June 1980.
11. JC Doyle and G Stein, "Concepts for
a Classical-Modern Synthesis," IEEE
Trans. Automat. Contr., Vol. AC-26,
pp. 4-17, February 1981.
ACKNOWLEDGMENT
The authors are greatful to the many disucssions with Mr. C.E. Rohrs and Professors
G. Stein and S.S. Sastry on adaptive control.
:*
5.
REFERENCES
1.
C. Rohrs, L. Valavani and M. Athans,
"Convergence Studies of Adaptive
Control Algorithms, Part I: Analysis,"
in Proc. IEEE CDC Conf., Albuquerque,
New Mexico, 1980, pp. 1138-1141.
2.
C. Rohrs, L. Valavani, M. Athans and
G. Stein, "Analytical Verification of
Undesirable Properties of Direct Model
Reference Adaptive Control Algorithms,"
LIDS-P-1122, M.I.T., August 1981; submitted to IEEE Trans. Autom. Control;
also Proc. 20th IEEE, Conf. on Dec.
and Control, San Diego, CA, Dec. 1981.
3.
K.S. Narendra and L.S. Valavani, "Stable
Adaptive Controller Design-Direct Control," IEEE Trans. Automat. Contr.,
Vol. AC-23, pp. 570-583, Aug. 1978.
4.
A. Feuer and A.S. Morse, "Adaptive Conof Single-Input Single-Output
Linear Systems," IEEE Trans. Automat.
Contr., Vol. AC-23, pp. 557-570,
August 1978.
5.
K.S. Narendra, Y.H. Lin and L.S.
Valavani, "Stable Adaptive Controller
Design, Part II: Proof of Stability,"
IEEE Trans. Automat. Contr., Vol. AC-25,
pp. 440-448, June 1980.
6.
A.S. Morse, "Global Stability of Parameter Adaptive Control Systems," IEEE
Trans. Automat. Contr., Vol. AC-25,
pp. 433-440, June 1980.
7.
K.S. Narendra and Y.H. Lin, "Stable Discrete Adaptive Control," IEEE Trans.
Automat. Contr., Vol. AC-25, pp. 456461, June 1980.
8.
I.D. Landau and H.M. Silveira, "A Stability Theorem with Applications to
Adaptive Control," IEEE Trans. Automat.
Contr., Vol. AC-24, pp. 305-312,
April 1979.
-.
I.D. Landau, "An Extension of A Stability Theorem Applicable to Adaptive
Control," I--E Trans. Automat. Contr.,
Vol. AC-25, pp. 814-817, Aug. 1980.
,trol
12. M.G. Safonov, Robustness and Stability
of Stochastic Multivariable Feedback
System Design, Ph.D. dissertation,
M.I.T., Cambridge, MA, September 1977;
also M.I.T., Press 1980.
191