Linear Riccati Dynamics, Constant Feedback, and Controllability in
Linear Quadratic Control Problems
July 2001
Revised: December 2005
Ronald J. Balvers
Department of Economics
P.O. Box 6025
West Virginia University
Morgantown, WV 26506-6025
Phone: (304) 293-7880
Email: rbalvers@wvu.edu
Douglas W. Mitchell
Department of Economics
P.O. Box 6025
West Virginia University
Morgantown, WV 26506-6025
Phone: (304) 293-7868
Email: dmitchel@wvu.edu
ABSTRACT
Conditions are derived for linear-quadratic control (LQC) problems to exhibit linear evolution of
the Riccati matrix and constancy of the control feedback matrix. One of these conditions
involves a matrix upon whose rank a necessary condition and a sufficient condition for
controllability are based. Linearity of Riccati evolution allows for rapid iterative calculation,
and constancy of the control feedback matrix allows for time-invariant comparative static
analysis of policy reactions.
JEL classification: C61
Keywords: Controllability, Riccati Equation, Linear Quadratic Control.
Linear Riccati Dynamics, Constant Feedback, and Controllability in
Linear Quadratic Control Problems
1. Introduction
This paper presents some conditions for linear evolution of the Riccati matrix, for
constancy of the control-feedback matrix, and for controllability or lack thereof, in the standard
linear quadratic control (LQC) problem. Computational aspects and simplification of the LQC
problem have figured prominently in the recent literature on dynamic linear economies [see
Amman (1997), Anderson et al. (1997), Anderson and Moore (1985), Binder and Pesaran
(2000), Blanchard and Kahn (1980), Ehlgen (1999), Klein (2000), Ljungqvist and Sargent
(2000), and Sims (2000)]. Linearity of Riccati evolution allows for rapid iterative calculation,
and constancy of the control feedback matrix allows for time-invariant comparative static
analysis of policy reactions.
2. The control problem
A typical statement of the finite horizon linear quadratic control problem is:
(1a)
(1b)
,
subject to
,
where K and A are n x n , C is n x k, yt is n x 1 , and ut is k x 1. The cost matrix K is positive
definite, and C has full column rank; the transition matrix A need not have full rank. If the
original problem statement has control costs, one can augment the state vector with the costed
controls [see Chow (1975)], putting all costs on the state vector and thus giving the problem
formulation in equations (1). It is well known [Chow (1975)] that the optimal controls are given
by:
(2)
t#T,
,
(3)
,
where the symmetric n x n matrix
,
t # T,
is positive definite. The nonlinear dynamic matrix
equation (3) is the Riccati equation.
The system is said to be controllable if and only if the state vector can be driven from any
value to any value within n periods. The well-known necessary and sufficient condition for
controllability [e.g., Anderson and Moore (1979), (1990), and Söderström (1994)] is that
.
3. A Preliminary Lemma
Write
with C1 being q x k where q / n - k, and with C2 being k x k. Since C
was assumed to be of full column rank, there is at least one k x k sub-matrix of C that is
invertible. Proper prior arrangement of the
vector (and concomitant arrangement of C, A, and
K ) is thus sufficient to guarantee that C2 is invertible. Define the n x q matrix M as
, so that
.
2
We can now prove the following identity:
LEMMA (IDENTITY RE -EXPRESSING A FUNCTION OF C IN TERMS OF M).
For C (which is n x k with full column rank) and M (which is n x q (q = n k) with full column rank) such that
, and D any (symmetric)
positive definite n x n matrix, the following identity holds:
(4)
.
Proof. Consider a Cholesky decomposition such that
, with
full rank (such a
decomposition exists for any positive definite matrix). Then we can premultiply equation (4) by
, postmultiply by
, and rearrange to obtain
(5)
.
Note that both sides of equation (5) express symmetric idempotent matrices. Define the n x n
matrix
. It is easy to confirm
. Hence, defining the n x n matrix
that
, we have:
(6)
.
3
Next show that Y has full rank: (i) Q has full rank n and C has full column rank k, so Y1 = QC
has rank k; (ii) Q has full rank n and M has full column rank q, so Y2 =
has rank q;
, so all the k columns of Y1 are
(iii)
independent of all the q columns of Y2. Given (i), (ii), and (iii), Y has full rank. Thus, from
equation (6), Z = 0 implying that by the definition of Z equation (5) must hold; but equation (5)
is equivalent to equation (4) in the statement of the Lemma.
€
4. Conditions for Linear Riccati Evolution, Constant Feedback, and Controllability
Apply the Lemma to the Riccati equation (3), setting
which is in fact positive
definite. This yields the following alternative statement of the Riccati equation:
(7)
,
.
This allows us to obtain Theorem 1:
THEOREM 1 (LINEAR DYNAMICS OF
). Linear evolution of
constancy of the control feedback matrix
implied by each of (a)
for all
, and (b)
strongly, (b) implies constancy of
Proof. (a) Post-multiply equation (7) by C. With
, are both
. More
.
this gives
# T. Using this equation and its transpose in equations (2) and (3) gives
4
, and
for all t
, and
(8)
,
.
(b) Use a standard matrix inversion identity [e.g. Söderström (1994), pp. 156-157] on
equation (7) to obtain
(9)
,
,
and post-multiply this by M to obtain
when
which is constant for t # T. Equation
equation (7) gives
(2) with constant
. Using this in
gives constant
Linear evolution of
for t # T. €
in equation (8) gives obvious computational advantages.
Constancy of the control feedback matrix
implies that comparative static analysis of policy
reactions can be conducted in straightforward fashion, with the results not dependent on time to
horizon. This Theorem also implies a result for stabilizability [Ljungqvist and Sargent (2000),
p.61]:
5
COROLLARY (SUFFICIENT CONDITION FOR STABILIZABILITY ).
is sufficient for stabilizability, and moreover, under
this condition optimal stabilization drives the state variables to their
target values of 0 in two periods.
. For period T the cost
The Corollary follows trivially from the immediate constancy of
matrix is
; for period T-1 it equals its steady state value; and with more than two periods
is driven to its desired value 0 in two periods. For
to the horizon, the cost is unchanged; thus
the latter to occur, two things are necessary: A and C, and hence M, must be such that it is
feasible to do so; and K must be such that it is desirable to do so. The corollary shows that the
condition
meets both of these criteria.
THEOREM 2 (RELATION OF
(a) Full row rank of
TO
CONTROLLABILITY ).
implies controllability; (b)
implies, but is not implied by, lack of controllability.
Proof. (a) Consider a “canonical” form of the decision problem in equations (1) with
such that
and
. Any possible form of the
problem in equations (1) can be put in this canonical form by setting
, where
and
6
. It is easy
to check that
, full
row rank (n) of which is equivalent to controllability (so controllability of the canonical problem
is equivalent to controllability of the original problem). Now
, where we partition A* as
with
being q x k. Thus we
, the system is controllable. (Of course this requires q # k --
need to show that, if
equivalently, n # 2k.) Let
=
.
implies full row rank of W*. So the canonical problem
Clearly full row rank of
and hence the original problem are controllable.
(b) We show that
implies
, where the
latter defines noncontrollability. Consider again the canonical form referred to above. Since
=
which now equals zero, this directly implies lack of controllability for the
canonical problem (sub-vector
controlling
since
cannot be controlled directly since
, nor indirectly by
). As shown above, lack of controllability of the canonical problem
implies lack of controllability of the original problem.
Part (b) further asserts that lack of controllability need not imply that
show this by example: Let
and take
to be idempotent and such that
7
. We
.
Then the system is noncontrollable since
, since C and AC each have k columns.1 €
Note that
counterexample: let
construction of M. If
does not imply lack of stabilizability. We can show this by
, in which case
since
by
the state equation is stable in the absence of time-varying control,
and hence trivially is stabilizable.
1
A numerical example is:
.
8
References
Amman, H. M., 1997, Numerical methods for linear-quadratic models, in H.M. Amman, D.A.
Kendrick, and J. Rust, eds., Handbook of Computational Economics (North-Holland,
Amsterdam).
Anderson, B. D. A. and J. B. Moore, 1979.Optimal filtering (Prentice Hall, Englewood Cliffs).
Anderson, B. D. A. and J. B. Moore, 1990. Optimal control: linear quadratic methods
(Prentice Hall, Englewood Cliffs).
Anderson, E., L. P. Hansen, E. R. McGrattan and T. Sargent, 1997. Mechanics of forming and
estimating dynamical linear economies, in H.M. Amman, D.A. Kendrick, and J. Rust,
eds., Handbook of Computational Economics (North-Holland, Amsterdam).
Anderson, G. and G. Moore, 1985. A linear algebraic procedure for solving linear perfect
foresight models, Economics Letters 17, 247-252.
Binder, M. and M. H. Pesaran, 2000. Solution of finite-horizon multivariate linear rational
expectations models and sparse linear systems, Journal of Economic Dynamics and
Control 24, 325-346.
Blanchard, O. J. and Kahn, C. M., 1980. The solution of linear difference models under
rational expectations, Econometrica 48, 1305-11.
Chow, G. C., 1975. Analysis and control of dynamic economic systems (Wiley, New York).
Ehlgen, J., 1999. A nonrecursive solution method for the linear-quadratic optimal control
problem with a singular transition matrix, Computational Economics 13, 17-23.
Klein, P., 2000. Using the generalized Schur form to solve a multivariate linear rational
expectations model, Journal of Economic Dynamics and Control 24, 1405-24.
Ljungqvist, L. and T. J. Sargent, 2000. Recursive macroeconomic theory (MIT Press,
Cambridge).
Sims, C. S., 2000, Solving linear rational expectations models, Working paper, Yale University.
Söderström, T., 1994. Discrete-time stochastic systems: estimation and control (Prentice-Hall,
New York).
9