6.241 Dynamic Systems and Control
Lecture 3: Least Square Solutions of Linear Problems
Readings: DDV, Chapter 3
Emilio Frazzoli
Aeronautics and Astronautics
Massachusetts Institute of Technology
February 9, 2011
E. Frazzoli (MIT)
Lecture 3: Least Squares
Feb 9, 2011
1/5
Outline
1
Least Squares Control
E. Frazzoli (MIT)
Lecture 3: Least Squares
Feb 9, 2011
2 / 5
Least Squares Control
Consider a system of m equations in n unknowns, with m < n, of the form1
y = A� x.
In these conditions, there are in general many possible values for x that make
A� x = y . It may be of interest to find such a solution x of minimum
(weighted) Euclidean norm, i.e., of minimum �x�.
Example
This situation occurs often in the context of control. For example, consider the
simple discrete-time system x[i + 1] = ax[i] + bu[i], with initial condition
x[0] = 0. It is desired to bring this system to x[N] = y , while minimizing the cost
�N
2
i=1 |u[i]| . This problem can be written as
min �u�2 ,
u
s.t. y = [b, ab, a2 , ....][u[N − 1], u[N − 2], . . .]�
1 We use the notation y = A� x so that A is still a “tall” matrix columns are possibly
infinite-dimensional.
E. Frazzoli (MIT)
Lecture 3: Least Squares
Feb 9, 2011
3/5
Constructing all possible solutions
It turns out that a x̌ = A(A� A)−1 y is a solution. (Check by substitution).
However let the null space of A� , N (A� ), be the subspace containing all
solutions of the homogeneous equation A� x = 0
Then, if x̌ is a solution of y = A� x, then so is x̌ + xh , where xh ∈ A.
In addition, should x1 and x2 be two solutions of y = A� x, then their
difference must be in N (A).
The trick is again that of finding among all such solutions the one with
minimum norm—which is in fact x̌.
E. Frazzoli (MIT)
Lecture 3: Least Squares
Feb 9, 2011
4/5
A general formulation
Consider the formulation of the problem in terms of Gram product:
y =� A, x �.
Let x be a solution of y =� A, x �. Decompose it into its projection onto
R(A) and on its orthogonal component, i.e., x = xA + xA⊥ .
Hence xA = Aα, xA⊥ = x − xA . Since it must be
�ai , x − Aα� = 0,
i = 1, . . . , n,
i.e.,
� A, x − Aα �= 0.
With some algebra, we get
α =� A, A �−1 � A, x �,
and finally
xA = A � A, A �−1 � A, x �= A � A, A �−1 y .
Since �x�2 = �xA �2 + �xA⊥ �2 , it is clear that one would choose xA⊥ = 0, and
hence x̌ = xA .
E. Frazzoli (MIT)
Lecture 3: Least Squares
Feb 9, 2011
5/5
MIT OpenCourseWare
http://ocw.mit.edu
6.241J / 16.338J Dynamic Systems and Control
Spring 2011
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms .