Lecture 7
Intersection of Hyperplanes and
Matrix Inverse
Shang-Hua Teng
Elimination Methods for 2 by 2
Linear Systems
• 2 by 2 linear system can be solved by eliminating
the first variable from the second equation by
subtracting a proper multiple of the first equation
and then
• by backward substitution
• Sometime, we need to switch the order of the first
and the second equation
• Sometime we may not be able to complete the
elimination
Singular Systems versus
Non-Singular Systems
• A singular system has no solution or
infinitely many solution
– Row Picture: two line are parallel or the same
– Column Picture: Two column vectors are colinear
• A non-singular system has a unique solution
– Row Picture: two non-parallel lines
– Column Picture: two non-colinear column
vectors
Gaussian Elimination in 3D
2x  4 y  2z  2
4 x  9 y  3z  8
 2 x  3 y  7 z  10
• Using the first pivot to eliminate x from the
next two equations
Gaussian Elimination in 3D
2x  4 y  2z  2
y  z  4
y  5 z  12
• Using the second pivot to eliminate y from
the third equation
Gaussian Elimination in 3D
2x  4 y  2z  2
y  z  4
4z  8
• Using the second pivot to eliminate y from
the third equation
Now We Have a Triangular System
2x  4 y  2z  2
y  z  4
4z  8
• From the last equation, we have
Backward Substitution
2x  4 y  2z  2
y  z  4
z  2
• And substitute z to the first two equations
Backward Substitution
2x  4 y  4  2
y  2  4
z  2
• We can solve y
Backward Substitution
2x  4 y  4  2
y
 2
z  2
• Substitute to the first equation
Backward Substitution
2x  8  4  2
y
 2
z  2
• We can solve the first equation
Backward Substitution
x
y
 1
 2
z  2
• We can solve the first equation
Generalization
• How to generalize to higher dimensions?
• What is the complexity of the algorithm?
• Answer:
Express Elimination with Matrices
Step 1
Build Augmented Matrix
2x  4 y  2z  2
4 x  9 y  3z  8
 2 x  3 y  7 z  10
Ax = b
[A b]
 2 4 2 2
A b   4 9  3 8 
 2  3 7 10
Pivot 1: The elimination of column 1
 2 4 2 2
 4 9 3 8 


 2  3 7 10
 2 4 2 2
 0 1 1 4


 2  3 7 10
2 4  2 2 
0 1 1 4 


0 1 5 12
 
  2
 
 1 
Pivot 2: The elimination of column 2
2 4  2 2 
0 1 1 4 


0 1 5 12
 2 4  2 2
0 1 1 4


0 0 4 8
Upper triangular matrix
 
 
 
 1
Backward Substitution 1: from the
last column to the first
Upper triangular matrix
 2 4  2 2
0 1 1 4


0 0 4 8
1 0 0  1
0 1 0 2 


0 0 1 2 
 2 4  2 2
0 1 1 4


0 0 1 2
 2 0 0  2
0 1 0 2 


0 0 1 2 
 2 4  2 2
0 1 0 2


0 0 1 2
2 4 0 6
0 1 0 2


0 0 1 2
Expressing Elimination by
Matrix Multiplication
Elementary or Elimination Matrix
Ei , j
• The elementary or elimination matrix Ei , j
That subtracts a multiple l of row j from row i
can be obtained from the identity matrix I
by adding (-l) in the i,j position
 1 0 0
E3,1   0 1 0
 l 0 1
Elementary or Elimination Matrix
 a1,1 a1, 2 a1,3   1 0 0  a1,1 a1, 2 a1,3 

 



E3,1 a2,1 a2, 2 a2,3    0 1 0 a2,1 a2, 2 a2,3 
 a3,1 a3, 2 a3,3   l 0 1  a3,1 a3, 2 a3,3 



a1,1
a1, 2
a1,3


a
a
a
2
,
1
2
,
2
2,3


 la1,1  a3,1  la1, 2  a3, 2  la1,3  a3,3 
Pivot 1: The elimination of column 1
 
  2
 
 1 
 2 4 2 2
 4 9 3 8 


 2  3 7 10
2 4  2 2 
0 1 1 4 


0 1 5 12
Elimination matrix
 1 0 0  2 4  2 2 
  2 1 0  4 9  3 8  



 0 0 1  2  3 7 10
 2 4 2 2
 0 1 1 4


 2  3 7 10
1 0 0  2 4  2 2  2 4  2 2 
0 1 0   0 1 1 4    0 1 1 4 


 

1 0 1  2  3 7 10 0 1 5 12
The Product of Elimination Matrices
1 0 0  1 0 0  1 0 0
0 1 0  2 1 0   2 1 0


 

1 0 1  0 0 1  1 0 1
0 0
1 0 0  1 0 0  1
0 1 0    2 1 0     2 1 0 


 

0  1 1  1 0 1  1  1 1
Elimination by Matrix Multiplication
 1 0 0  2 4  2 2   2 4  2 2
  2 1 0  4 9  3 8    0 1 1 4


 

 1  1 1  2  3 7 10 0 0 4 8
Linear Systems in Higher Dimensions
1
1

1

1
1 1 1
0



2 3 4

2
x 
  5
3 6 10 

 
4 10 20
 9
Linear Systems in Higher Dimensions
1
1

1

1
1
0

0

0
1 1 1 0
2 3 4  2
3 6 10  5

4 10 20  9
1
0

0

0
1 1 1 0
1 2 3  2
2 5 9  5

3 9 19  9 
1 1 1 0
1 2 3  2
0 3 6  3

0 0 4 0
1
0

0

0
1 1 1 0
1 2 3  2
0 3 6  3

0 3 10  3
Linear Systems in Higher Dimensions
1
0

0

0
1 1 1 0
1 2 3  2
0 3 6  3

0 0 4 0
1
0

0

0
0 0 0 1
1 0 0 0 
0 1 0  1

0 0 1 0
1
0

0

0
1 1 1 0
1 2 3  2
0 3 6  3

0 0 1 0
1
0

0

0
1 0 0 1
1 0 0 0 
0 1 0  1

0 0 1 0
1
0

0

0
1
0

0

0
1 1 0 0
1 2 0  2
0 3 0  3

0 0 1 0
1 1 0 0
1 2 0  2
0 1 0  1

0 0 1 0
Booking with Elimination Matrices
1
 1

 1

 1
0 0 0 1
1 0 0 1
0 1 0 1

0 0 1 1
1 0
0 1

0  2

0  3
1
0

0

0
1 1 1 0  1
2 3 4  2 0


0
3 6 10  5
 
4 10 20  9 0
0 0 1 1
0 0 0 1
1 0  0 2

0 1  0 3
0 0 0 1
1 0 0 0
0 1 0  0

0  1 1  0
1 1 0  1
2 3  2 0

5 9  5  0
 
9 19  9 0
1 1 1 0  1
1 2 3  2 0

0 3 6  3  0
 
0 3 10  3 0
1 1 1 0 
1 2 3  2
2 5 9  5

3 9 19  9
1 1 1 0 
1 2 3  2
0 3 6  3

0 3 10  3
1 1 1 0 
1 2 3  2
0 3 6  3

0 0 4 0 
Multiplying Elimination Matrices
1 0 0
 1 1 0

 1  2 1

 1  3  1
0 1
0 1
0 1

1 1
1 1 1 0  1
2 3 4  2 0

3 6 10  5 0
 
4 10 20  9 0
1 1 1 0 
1 2 3  2
0 3 6  3

0 0 4 0 
Inverse Matrices
• In 1 dimension
3x  9
1
x  3 9  3
1
1
3 3  33  1
Inverse Matrices
• In high dimensions
Ax  b
Can we write?
1
xA b
1
Is there a matrix A such that?
1
1
A A  AA  I
Inverse Matrices
• In 1 dimension
1
0 does not exist
1
a exists iff a  0
• In higher dimensions
1
When does A not exist?
singular matrices! !!
Some Special Matrices and
Their Inverses
I 1  I
1
d1

1 / d1

 
 








d n 
1 / d n 
Inverses in Two Dimensions
1
a b 
1  d  b
 c d   ad  bc  c a 




Proof:
0 
1  d  b  a b 
1 ad  bc

I






ad  bc 
ad  bc  c a   c d  ad  bc  0
0 
a b  1  d  b  
1 ad  bc
I
 c d  ad  bc  c a    ad  bc  0

ad  bc





Uniqueness of Inverse Matrices
BA  I and AC  I then
B  C
Proof :
B  BI  B AC   BAC  BA C  IC  C
Inverse and Linear System
if A is invertible then
Ax  b
has a unique solution given by
1
A b
Proof :
1
1
A Ax  A b
1
Ix  A b
1
xA b
Inverse and Linear System
• Therefore, the inverse of A exists if and
only if elimination produces n non-zero
pivots (row exchanges allowed)
Inverse, Singular Matrix and
Degeneracy
Suppose there is a nonzero vector x such that Ax = 0
[column vectors of A co-linear] then A cannot
have an inverse
Proof :
1
1
A Ax  A 0
x0
Contradiction:
So if A is invertible, then Ax =0 can only have
the zero solution x=0
One More Property
 AB
1
Proof
1
B A

1

B 1 A1  AB  B 1 A1 A B  B 1B  I
So
 ABC 
1
1
1
C B A
1
Gauss-Jordan Elimination for
Computing A-1
• 1D
ax  1 implies
xa
1
• 2D
 a11 a12   x1  1
 a11 a12   y1  0
a a   x   0 and a a   y   1
 21 22   2   
 21 22   2   
then
 a11 a12   x1 y1  1 0
 a a   x y   0 1 

 21 22   2 2  
Gauss-Jordan Elimination for
Computing A-1
• 3D
 a11 a12 a13   x  1  a11 a12 a13   y  0
a a a   1   0 , a a a   1   1  and
 21 22 23   x2     21 22 23   y2   
a31 a32 a33   x3  0 a31 a32 a33   y3  0
 a11 a12 a13   z  0
 a a a   1   0 
 21 22 23   z 2   
 a31 a32 a33   z3  1 
then
 a11 a12 a13   x1 y1 z1  1 0 0
 a a a   x y z   0 1 0 
 21 22 23   2 2 2  

 a31 a32 a33   x3 y3 z3  0 0 1
Gauss-Jordan Elimination for
Computing A-1
• 3D: Solving three linear equations defined
by A simultaneously
• n dimensions: Solving n linear equations
defined by A simultaneously
A
1
A I   I , A
1

Example:Gauss-Jordan
Elimination for Computing A-1
 2 1 0 
1 0 0
 1 2  1 X  0 1 0




 0  1 2 
0 0 1
• Make a Big Augmented Matrix
 2  1 0 1 0 0



1
2

1
0
1
0


 0  1 2 0 0 1
Example:Gauss-Jordan
Elimination for Computing A-1
 2  1 0 1 0 0


  1 2  1 0 1 0
 0  1 2 0 0 1
2  1 0
1 0 0


0 3 / 2   1 1 / 2  1 0
0  1 2
0 0 1
2  1
0
1
0 0


0 3 / 2   1 1 / 2 1 0
0 0 4 / 3 1 / 3 2 / 3 1
Example:Gauss-Jordan
Elimination for Computing A-1
2  1
0
1
0 0






0
3
/
2

1
1
/
2
1
0


0 0 4 / 3 1 / 3 2 / 3 1
2  1
0
1
0
0 










0
3
/
2
0
3
/
4
3
/
2
3
/
4


0 0 4 / 3 1 / 3 2 / 3 1 
2 0
1 / 2
0 3 / 2 1


0 3 / 2  0 3 / 4 3 / 2 3 / 4 
0 0 4 / 3 1 / 3 2 / 3 1 
Example:Gauss-Jordan
Elimination for Computing A-1
2 0
1 / 2
0 3 / 2 1


0 3 / 2  0 3 / 4 3 / 2 3 / 4 
0 0 4 / 3 1 / 3 2 / 3 1 
2 0 0 3 / 4  1 / 2  1 / 4 






0
1
0
1
/
2
1
1
/
2


0 0 1 1 / 4  1 / 2  3 / 4 