Miscellaneous Results, Solving Equations,
and Generalized Inverses
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
1 / 51
Result A.7:
Suppose S and T are vector spaces. If S ⊆ T and dim(S) = dim(T ),
then S = T .
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
2 / 51
Result A.8:
Supposem×n
A and b satisfy
m×1
Ax + b = 0 ∀ x ∈ Rn .
m×n
m×1
Thenm×n
A =m×n
0 and b = 0.
m×1
m×1
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
4 / 51
Corollary A.1:
Ifm×n
B andm×n
C satisfy Bx = Cx ∀ x ∈ Rn , then B = C.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
6 / 51
Corollary A.2:
Supposem×n
A has full column rank. Then
AB = AC =⇒ B = C.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
8 / 51
Lemma A.1:
C0 C = 0 =⇒ C = 0.
Proof of Lemma A.1:
Let ci denote the ith column of C. Then the ith diagonal element of
C0 C is c0i ci .
∵ C0 C = 0, c0i ci = 0 ∀ i = 1, . . . , n.
Now c0i ci = 0 ∀ i =⇒ ci = 0 ∀ i =⇒ C = 0.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
10 / 51
Another Result on the Rank of a Product
Suppose rank(m×n
A ) = n and rank( C ) = k. Then
k×l
rank( B ) = rank(ABC).
n×k
Proof: HW problem.
Corollaries:
If A is full-column rank, rank(AB) = rank(B).
If C is full-row rank, rank(BC) = rank(B).
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
11 / 51
Solving Equations:
Consider a system of linear equations
Ax = c,
wherem×n
A is a known matrix and c is a known vector.
c
Copyright 2012
Dan Nettleton (Iowa State University)
m×1
Statistics 611
12 / 51
We seek a solution vector x that satisfies Ax = c.
If m = n so thatm×n
A is a square and ifm×n
A is nonsingular, then
A−1 Ax = A−1 c =⇒ x = A−1 c
is the unique solution to Ax = c.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
13 / 51
Ifm×n
A is singular or not square, Ax = c may have no solution or
infinitely many solutions or a unique solution.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
14 / 51
A system of equations Ax = c is consistent if there exists a
solution x∗ such that Ax∗ = c.
A systems of equation Ax = c is inconsistent if Ax 6= c ∀ x ∈ Rn .
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
15 / 51
Result A.9:
A system of equations Ax = c is consistent iff c ∈ C(A).
Provide an example m×n
A, c 3 Ax = c is inconsistent.
m×1
Provide an example m×n
A, c 3 Ax = c is consistent.
m×1
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
16 / 51
Generalized Inverse
A matrix G is a generalized inverse (GI) of a matrix A iff
AGA = A.
Every matrix has at least one GI. We will use A− to denote a GI of
a matrix A.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
19 / 51
If A is nonsingular, then A−1 is the unique GI of A.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
20 / 51
Result A.10:
Suppose rank(m×n
A) = r. If A can be partitioned as

A=
C
r×r
D
r×(n−r)
E
F
(m−r)×r

,
(m−r)×(n−r)
where rank(r×r
C) = r, then
"
G=
n×m
#
C−1 0
0
0
is a GI ofm×n
A.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
22 / 51
Proof of Result A.10:
"
AG =
#"
#
C−1 0
E F
"
=
I
EC−1
"
∴ AGA =
C D
I
0
0
#
0
.
0
#"
#
0 C D
EC−1 0
E F
"
=
C
D
#
E EC−1 D
.
We need to show EC−1 D = F.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
23 / 51
First note that
rank(r×r
C) = r =⇒ rank([C, D]) = r
∵ [C, D] has at least r LI columns and at most r LI rows.
(rank([C, D]) ≥ r and rank([C, D]) ≤ r =⇒ rank([C, D]) = r.)
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
24 / 51
Now rank
"
#!
C D
E F
= r =⇒ each row of [E, F] is a LC of the rows of
[C, D].
Thus ∃ a matrix K 3
K[C, D] = [E, F]
⇐⇒ [KC, KD] = [E, F]
⇐⇒ KC = E, KD = F.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
25 / 51
Now KC = E ⇐⇒ K = EC−1 . Together with KD = F, this implies
EC−1 D = F.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
26 / 51
Permutation Matrix
A matrixn×n
P is a permutation matrix if the rows of P are the same as the
rows ofn×n
I but not necessarily in the same order.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
27 / 51
Example:


0 1 0



P=
1 0 0 .
0 0 1




1 2 3 4
5 6 7 8







If A = 
5 6 7 8  , then PA = 1 2 3 4  .
9 10 11 12
9 10 11 12
Order of rows of PA are permuted relative to order of rows of A.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
28 / 51
Example:


"
#
0 1 0


1
2
3

P=
1 0 0 , B = 4 5 6 .
0 0 1
"
#
2 1 3
.
Then BP =
5 4 6
Order of columns of BP are permuted relative to order of columns
of B.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
29 / 51
A Permutation Matrix is Nonsingular
The rows (and columns) of a permutation matrix are the same as
those of the identity matrix. Thus, a permutation matrix has full rank
and is therefore nonsingular.
Furthermore, if P = [p1 , . . . , pn ] is a permutation matrix,
P−1 = P0 ∵ p0i pj =
c
Copyright 2012
Dan Nettleton (Iowa State University)

0
if i 6= j
1
if i = j.
Statistics 611
30 / 51
Result A.11:
Suppose rank(m×n
A) = r. There exist permutation matricesm×m
P and Q such
n×n
that
"
PAQ =
C D
#
,
E F
where rank(r×r
C) = r. Furthermore,
"
G=Q
C−1 0
0
0
#
P
is a GI of A.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
31 / 51
Proof of Result A.11:
Because rank(A) = r, there exists a set of r rows of A that are LI.
Let P be a permutation matrix, 3 the first r rows of PA are LI.
Let H be the matrix consisting of the first r rows of PA. Then
rank(H) = r.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
32 / 51
This implies that ∃ a set of r columns of H that are LI.
Let Q be a permutation matrix 3 the first r columns of HQ are LI.
Then the submatrix consisting of the first r rows and first r
columns of PAQ has rank r.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
33 / 51
Thus we can partition PAQ as
"
By Result A.10,
#
C−1 0
0
0
c
Copyright 2012
Dan Nettleton (Iowa State University)
"
#
C D
E F
, where rank(r×r
C) = r.
"
is a GI for PAQ =
C D
E F
#
.
Statistics 611
34 / 51
PAQ =
"
#
C D
"
⇐⇒ A = P
−1
C D
#
Q−1
E F
E F
"
#
C−1 0
∴ AQ
PA =
0
0
"
#
"
#
"
#
−1
C
D
C
0
C
D
= P−1
Q−1 Q
PP−1
Q−1
E F
0
0
E F
"
#"
#"
#
C−1 0 C D −1
−1 C D
=P
Q
E F
0
0 E F
"
#
−1 C D
=P
Q−1 = A.
E F
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
35 / 51
Use Result A.11 to find a GI for


1 1 3 4



A=
2 2 6 8  .
3 3 0 12
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
36 / 51
Algorithm for finding a GI of A:
1. Find an r × r nonsingular submatrix of A, where r = rank(A). Call
this matrix W.
2. Compute (W −1 )0 .
3. Replace each element of W in A with the corresponding elements
of (W −1 )0 .
4. Replace all other elements in A with zeros.
5. Transpose resulting matrix to get A− .
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
42 / 51
Result A.12:
Let Ax = c be a consistent system of equations, and let G be any GI of
A. Then Gc is a solution to Ax = c, i.e., AGc = c.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
43 / 51
Result A.13:
Let Ax = c be a consistent system of equations, and let G be any GI of
A. Then x̃ is a solution to Ax = c iff ∃ z 3 x̃ = Gc + (I − GA)z.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
45 / 51
Prove that a consistent system of equations
Ax = c
has a unique solution if and only if A is full-column rank and infinitely
many solutions if and only if A is less than full-column rank.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
48 / 51