CHAPTER 1. MATRICES AND
GAUSSIAN ELIMINATION
Introduction
What is Linear Algebra?
■ Linear Algebra deals with linear equations and linear functions which
are represented through matrices and vectors.
Ref: https://misa-xu.github.io/space/
1
Why Study Linear Algebra?
■ Applications in communications
– Multivariate signal processing and filter design
– Multiple-input multiple-output (MIMO) system
– Estimation and detection
– Array signal processing
■ Applications in data science and machine learning
– Vectorized code: Computation of many values simultaneously
– Image recognition: Matrix representation of color image
– Dimensionality reduction: Matrix fatorization
– Data and learned model representation
– Word embeddings
2
1.2 The Geometry of Linear Equation
Geometry of Linear Equations (by rows)
2𝑥 − 𝑦 = 1
■ Given a system with two equations and two unknowns !
𝑥+𝑦 =5
■ Look at the system by rows
– Each equation is represented by a straight line in the 𝑥 − 𝑦 plane
– The problem is to find the point of intersection of both lines
3
Geometry of Linear Equations (by columns)
2𝑥 − 𝑦 = 1
■ Given a system with two equations and two unknowns !
𝑥+𝑦 =5
■ Look at the system by columns
– Two separate equations are realized as one vector equation
2
−1
1
𝑥+
𝑦=
1
1
5
– The problem is to find the combination of the column vectors on
the LHS that produces the vector on the RHS
LHS: left-hand side
RHS: right-hand side
4
Geometry of Linear Equations (3-dim)
■ Given a system with three equations and three unknowns
2𝑢 + 𝑣 + 𝑤 = 5
* 4𝑢 − 6𝑣
= −2
−2𝑢 + 7𝑣 + 2𝑤 = 9
■ Look at the system by rows
■ Look at the system by columns
2
1
1
5
4 𝑢 + −6 𝑣 + 0 𝑤 = −2 = 𝒃
−2
7
2
9
5
Geometry of Linear Equations (𝑛-dim)
■ With 𝑛 equations in 𝑛 unknowns, there are
– Row picture: 𝑛 planes
■ Each equation produces (𝑛 − 1)-dimensional plane in 𝑛 dimensions
– Column picture: 𝑛 vectors plus a vector 𝒃 on the RHS
■ The 𝑛 equations ask for (if there exists a unique solution)
– Row picture: an intersection of 𝑛 planes
■ Every new plane (i.e., equation) reduces the dimension by one
■ When a point lies on all the planes, it is the solution! (i.e., the set of
equations is said to be consistent)
– Column picture: a linear combination of the 𝑛 columns that
equals 𝒃
■ For certain equations, it is impossible to find the above case, i.e., the
singular case
6
The Singular Case (by rows)
𝑢 + 𝑣+ 𝑤=2
!2𝑢
+ 3𝑤 = 5
3𝑢 + 𝑣 + 4𝑤 = 6
𝑢 + 𝑣+ 𝑤=2
!2𝑢
+ 3𝑤 = 5
3𝑢 + 𝑣 + 4𝑤 = 7
■ No solution as in Fig. 1.5(b) (inconsistent equations)
■ An infinity of solution as in Fig. 1.5(c) (end view: three planes have a whole line in
common)
■ No solution as in Fig 1.5(d)
7
The Singular Case (by columns)
■ Three columns on the RHS are in the same plane, which is solvable
only for 𝒃 in that plane
1
1
1
2 𝑢+ 0 𝑣+ 3 𝑤=𝒃
3
1
4
2
𝒃= 5
6
2
𝒃= 5
7
8
1.3 An Example of Gaussian Elimination
Linear Equation and Gaussian Elimination
■ A basic problem in linear algebra is to solve a linear system
𝑎''𝑥' + 𝑎'(𝑥( + ⋯ + 𝑎') 𝑥) = 𝑏'
𝑎('𝑥' + 𝑎((𝑥( + ⋯ + 𝑎() 𝑥) = 𝑏(
⋮
𝑎*'𝑥' + 𝑎*(𝑥( + ⋯ + 𝑎*) 𝑥) = 𝑏*
– These are 𝑚 linear equation in 𝑛 unknowns 𝑥', … , 𝑥) . The 𝑎+, and
𝑏, are constant, and if 𝑏, = 0, 𝑗 = 1, … , 𝑚 the system is
homogeneous. Otherwise, it is called inhomogeneous.
■ The locus of one equation 𝑎+'𝑥' + 𝑎+(𝑥( + ⋯ + 𝑎+) 𝑥) = 𝑏+ is an 𝑛 − 1
dimensional plane.
– E.g., for 𝑎𝑥 + 𝑏𝑦 + 𝑐𝑧 = 𝑑, the locus of (𝑥, 𝑦, 𝑧) satisfying the
equation is a plane in three-dimensional space ℝ-
Locus: a set of all points whose location is determined by the equation
9
Example: Locus of Equation
■ For 𝑎𝑥 + 𝑏𝑦 + 𝑐𝑧 = 𝑑, the locus of (𝑥, 𝑦, 𝑧) satisfying the equation is a
plane in three-dimensional space ℝ-
𝑎⃗𝚤 + 𝑏⃗𝚥 + 𝑐𝑘 is normal to the plane
■ For 𝑎𝑥 + 𝑏𝑦 = 𝑐, the locus is a line in ℝ(
!
slope − " if b ≠ 0,
otherwise vertical
10
Example: Locus of Equation
■ In parameter form, a line can be described by one parameter
𝑥!
𝑐!
𝑑!
⋮ =𝑡 ⋮ + ⋮
𝑥"
𝑐"
𝑑"
■ Example
𝑥 =𝑡+1
– - 𝑦 = 2𝑡
𝑧 = 3𝑡
■ A two-dimensional plane by two parameters
𝑥!
𝑐!
𝑒!
𝑑!
⋮ =𝑠 ⋮ +𝑡 ⋮ + ⋮
𝑥"
𝑐"
𝑒"
𝑑"
11
Example: Locus of Equation (cont.)
■ Consider the plane in ℝ- given by
𝑥 + 2𝑦 + 3𝑧 = 1
*4𝑥 + 5𝑦 + 6𝑧 = 4
7𝑥 + 8𝑦 + 9𝑧 = 8
⇒
𝑥 + 2𝑦 + 3𝑧 = 1
* −3𝑦 − 6𝑧 = 0
−6𝑦 − 12𝑧 = 1
so that there are no points of intersection of the three planes.
■ If the “8” in the original set is changed to “7”, then we have
𝑥 + 2𝑦 + 3𝑧 = 1
* −3𝑦 − 6𝑧 = 0
−6𝑦 − 12𝑧 = 0
and the locus of intersection is the line:
𝑧 = 𝑡,
𝑦 = −2𝑡,
𝑥 =1+𝑡
■ If the “9” in the original equation is changed, then the intersection is a
single point
12
Example: Locus of Equation (cont.)
■ Gaussian elimination provides a systematic way of doing such analysis
– These involve row operation and back substitution
■ Row operation:
1. Interchange two rows
2. Multiply a row by a nonzero number
3. Add a multiple of a row to another row
13
Example: Gaussian Elimination
2𝑢 + 𝑣 + 𝑤 = 5
■ Original system: - 4𝑢 − 6𝑣
= −2
−2𝑢 + 7𝑣 + 2𝑤 = 9
■ Forward elimination step
pivot
𝟐
1 1 5
𝟐 1
1
𝟐 1
1
5
5
4 −6 0 −2 ⇒ 0 −𝟖 −2 −12 ⇒ 0 −𝟖 −2 −12
−2 7 2 9
0 8
3
0 0
𝟏
14
2
– With a full set of pivots 2, 8, and 1, there is only one solution
■ Backward elimination step (i.e., back-substitution)
– 𝑤 = 2 at the third equation
– 𝑣 = 1 by substituting 𝑤 = 2 into the second equation
– 𝑢 = 1 by using 𝑣 = 1 and 𝑤 = 2
■ By definition, pivots cannot be zero because we need to divide by them
p. 13
14
Example: Gaussian Elimination
pivot
1𝑥 + 2𝑦 + 3𝑧 = 1
*4𝑥 + 5𝑦 + 6𝑧 = 4
7𝑥 + 8𝑦 + 9𝑧 = 7
⇒
pivot
𝑥 + 2𝑦 + 3𝑧 = 1
* −3𝑦 − 6𝑧 = 0
−6𝑦 − 12𝑧 = 0
⇒
𝑥 + 2𝑦 + 3𝑧 = 1
* −3𝑦 − 6𝑧 = 0
0 =0
■ Back substitution:
– 𝑧 is “free variable”
𝑧=𝑡
𝑦 = −2𝑡
– *
𝑥 = 1 + 4𝑡 − 3𝑡 = 1 + 𝑡
15
1.4 Matrix Notation and Matrix Multiplication
Matrix Notation
■ For 𝑚×𝑛 matrix
→
→
rows
→
𝑎''
𝑎('
⋮
𝑎*'
↑
■ Column vector and row vector
𝑎'
⋮
and
𝑎*
𝑎'(
𝑎((
⋮
𝑎*(
𝑎')
𝑎('
⋮
⋯ 𝑎*)
↑ columns ↑
𝑎'
⋯
⋯
𝑎+,
↑
𝑖th row
↑
𝑗th column
⋯ 𝑎)
16
Matrix Algebra
■ Addition of 𝑚×𝑛 matrices is point-wise
1 2 3
1 −1 2
2 1 5
– Ex.
+
=
4 5 6
−2 0 3
2 5 9
■ Multiplication by scalar
1 2 3
2 4
6
– Ex. 2
=
4 5 6
8 10 12
■ Inner product (dot product) of vectors
𝑏'
𝑎' ⋯ 𝑎) ⋅ ⋮ = 𝑎'𝑏'+ 𝑎(𝑏( + ⋯ + 𝑎) 𝑏)
𝑏)
17
Matrix-Vector Multiplication
■ Row-wise multiplication of 𝑚×𝑛 matrix and 𝑛 component column vector
𝒗' ⋅ 𝒘
𝒗' →
𝒘
𝒗( ⋅ 𝒘
𝒗( →
↓ =
← 𝑚 component column vector
⋮
⋮
⋮
𝒗* ⋅ 𝒘
𝒗* →
– 𝒗' ⋯ 𝒗* : 𝑛 component row vectors
– 𝒘: 𝑛 component column vector
■ Example (row-wise)
1 2 3 1
−2
4 5 6 0 = −2
7 8 9 −1
−2
(row at a time)
18
Matrix-Vector Multiplication
■ Column-wise multiplication of 𝑚×𝑛 matrix and 𝑛 component column vector
𝒖' 𝒖( ⋯ 𝒖) 𝑤'
⋮ = 𝑤'𝒖' + 𝑤(𝒖( + ⋯ + 𝑤) 𝒖) ← 𝑚 component column vector
↓
↓
↓
𝑤)
– 𝒖' ⋯ 𝒖) : 𝑚 component column vectors
– 𝒘: 𝑛 component column vector
■ Example (column-wise)
1 2 3 1
1
2
3
−2
4 5 6 0 = 1 4 + 0 5 + (−1) 6 = −2
7 8 9 −1
7
8
9
−2
19
Matrix-Matrix Multiplication
■ Multiplication of 𝑚×𝑛 matrix and 𝑛×𝑝 matrix (in that order) gives 𝑚×𝑝 matrix
𝒖! ⋅ 𝒗! 𝒖! ⋅ 𝒗# ⋯ 𝒖! ⋅ 𝒗%
𝒖! →
𝒗! 𝒗# ⋯ 𝒗%
𝒖# ⋅ 𝒗! 𝒖# ⋅ 𝒗# ⋯ 𝒖# ⋅ 𝒗%
𝒖# →
↓
↓
↓ =
⋮
⋮
⋮
⋮
⋮
𝒖$ →
𝒖$ ⋅ 𝒗! 𝒖$ ⋅ 𝒗# ⋯ 𝒖$ ⋅ 𝒗%
■ Example
1 2 3
4 5 6
■ Formal rule
𝑎!! ⋯ 𝑎!"
⋮
⋮
𝑎$! ⋯ 𝑎$"
𝑏!!
⋮
𝑏"!
⋯
⋯
−1 2
10 −4
4
0 =
22 −4
1 −2
𝑐!!
𝑏!%
⋮ = ⋮
𝑐$!
𝑏"%
⋯
⋯
𝑐!%
⋮
𝑐$%
where 𝑐&' = ∑"()! 𝑎&( 𝑏('
20
Property of Matrix Multiplication
■ If dimensions are appropriate so that the multiplication is defined:
𝑨𝑩 𝑪 = 𝑨(𝑩𝑪)
Usually 𝑨𝑩 ≠ 𝑩𝑨
𝑨 𝑩 + 𝑪 = 𝑨𝑩 + 𝑨𝑪
𝑨 + 𝑩 𝑪 = 𝑨𝑪 + 𝑩𝑪
– Ex. 1 2
3
0
4 −1
1
−2
=
0
−4
1
0
and
3
−1
1 1
0 3
2
3
=
4
−1
4
−2
■ You must be careful in doing matrix multiplication
– Ex. 1 2
−1
−1
3 0 = 2 and 0 1
1
1
1 0
■ 𝑛×𝑛 identity matrix 𝑰 = 0 1
⋮ ⋱
0 0
■ If 𝑨 is 𝑛×𝑛, then 𝑨𝑰 = 𝑰𝑨 = 𝑨
2
−1
3 = 0
1
−2
0
2
−3
0
3
⋯ 0
⋱ 0 functions like number 1
⋱ ⋮
⋯ 1
21
Elementary Matrix (3×3 Case)
■ 𝑬89 matrix formed by interchanging 𝑖th and 𝑗th rows in 𝑰
0 1 0
– Ex. 𝑬:; = 1 0 0
0 0 1
■ 𝑬8 (𝑐) matrix formed by multiplying 𝑖th row by nonzero number 𝑐 in 𝑰
1 0 0
– Ex. 𝑬; (4) = 0 4 0
0 0 1
■ 𝑬89 (𝑐) matrix formed by replacing 𝑖th row in 𝑰 by 𝑖th row + 𝑐 times 𝑗th row
1 4 0
– Ex. 𝑬:; (4) = 0 1 0
0 0 1
■ Each row operation in an arbitrary matrix can be accomplished by multiplying on the
left by the corresponding elementary matrix
– Ex. 𝑬:;
1
4 −1
6
2
0
7
3
0
8
1
4
2 = 0
0
9
4
1
0
0 1
0 −1
1 6
2
0
7
3
0
8
−3
4
2 = −1
6
9
2
0
7
3 12
0 2
8 9
22
1.5 Triangular Factors and Row Exchanges
Model System of 𝑨𝒙 = 𝑩
■ Multiplication of elementary matrices
– 𝑬+, 𝑬+, = 𝑰
– 𝑬+ 𝑐 𝑬+ 1/𝑐 = 𝑰
– 𝑬+, (𝑐) 𝑬+, (−𝑐) = 𝑰
■ We can write the system below
𝑎''𝑥' + ⋯ + 𝑎') 𝑥) = 𝑏'
⋮
𝑎*'𝑥' + ⋯ + 𝑎*) 𝑥) = 𝑏*
in matrix form 𝑨𝒙 = 𝒃, where
𝑎'' ⋯ 𝑎')
⋮ ,
𝑨= ⋮
𝑎*' ⋯ 𝑎*)
𝑥'
𝒙= ⋮ ,
𝑥)
𝑏'
and 𝒃 = ⋮
𝑏*
23
Forward Elimination
■ Consider Gaussian elimination applied to an 𝑛×𝑛 matrix 𝑨
■ By row operations, 𝑨 can be brought to upper triangular form
𝑎!!
∗
𝑎##
⋱
0
𝑎""
1 2 3 *+, +-.*/01+2 1 2
3
– Ex. 4 5 6
0 −3 −6
7 8 9
0 0
0
– Ex. Using elementary matrices gives
1 2 3
1 2
3
𝑬3# −2 𝑬3! −7 𝑬#! (−4) 4 5 6 = 0 −3 −6
7 8 9
0 0
0
1 2 3
1 2
3
Thus, 4 5 6 = 𝑬#! (4) 𝑬3! 7 𝑬3# 2 0 −3 −6
7 8 9
0 0
0
24
LU Decomposition
■ Observation: In our algorithm, we began with top row, and worked our
way down. Thus, in 𝑬-( −2 , 𝑬-' −7 , 𝑬('(−4) (and hence in
𝑬(' 4 , 𝑬-' 7 , 𝑬-( 2 ) no changes are made above the diagonal.
Their product is therefore also lower triangular.
𝑳 = 𝑬(' 4 𝑬-' 7 𝑬-( 2
1 0 0 1 0 0 1 0 0
1 0 0
= 4 1 0 0 1 0 0 1 0 = 4 1 0
0 0 1 7 0 1 0 2 1
7 2 1
■ Thus, if there are no row exchanges we may write
𝑨 = 𝑳𝑼
↑
lower triangular with 1! s on diagonal
↑
upper triangular matrix
■ This is the LU decomposition
Appendix: Product of lower triangular matrices
25
LDU Decomposition
■ From here, we can go to a more symmetric form. Suppose the
diagonal entries in 𝑼 are not zero
𝑑' 𝑢'( ⋯ 𝑢')
1
0
𝑑(
⋮
1
𝑨=
⋱
⋱
∗
0
1
𝑑)
■ Then, we may write
1
𝑨=
0
1
⋱
∗
1
𝑑'
⋮
0
⋯ 0
⋱
⋮
⋯ 𝑑)
1
2!"
3!
⋯
1
0
⋱
2!#
3!
⋮
1
■ This is the LDU decomposition
↑
↑
lower triangular with
upper triangular with 1! s on diagonal
↑
1! s on diagonal
diagonal matrix
26
Row Exchanges and Permutation Matrices
■ Suppose we need row exchanges. These are done by 𝑬+, matrices. A
product 𝑷 of such matrices is called a permutation matrix.
1 1 1 𝑬$! 5( 𝑬"! 5' 1 1 1 𝑬"$ 1 1 1
– Ex. 𝑨 = 1 1 3
0 0 2
0 3 6
2 5 8
0 3 6
0 0 2
⇔ 𝑬(-𝑬-' −2 𝑬(' −1 𝑨 = 𝑼
■ We wish to write 𝑷𝑨 = 𝑳𝑼 where 𝑷 is a permutation matrix. This
means that row exchange must be done first
1 1 1 𝑬$! 5' 𝑬"! 5( 1 1 1
𝑬(-𝑨 = 2 5 8
0 3 6
1 1 3
0 0 2
𝑬-' −1 𝑬(' −2 𝑬(-𝑨 = 𝑼
𝑬(-𝑨 = 𝑬(' 2 𝑬-' 1 𝑼 𝑳
𝑷
• Nonsingular 𝑨: There is a permutation matrix 𝑷 that reorders the rows of A to avoid zeros in the pivot positions.
Then, 𝑨𝒙 = 𝒃 has a unique solution
• Singular 𝑨: No 𝑷 can produce a full set of pivots (elimination fails).
27
1.6 Inverses and Transposes
Inverses and Transposes
■ If 4!
𝑨 is an 𝑛×𝑛 matrix and 𝑩𝑨 = 𝑰, then 𝑩 is the inverse of 𝑨, denoted by 𝑩 =
𝑨
– Not all matrices have inverses. An inverse is impossible when 𝑨𝒙 is zero
and 𝒙 is nonzero
■ Note 1. The inverse exists if and only if elimination produces 𝑛 pivots
■ Note 2. The matrix 𝑨 cannot have two different inverses. Suppose 𝑩𝑨 = 𝑰
and also 𝑨𝑪 = 𝑰. Then 𝑩 = 𝑪.
■ Note 3. If 𝑨 is invertible, the one and only solution to 𝑨𝒙 = 𝒃 is 𝒙 = 𝑨4! 𝒃
■ Note 4. Suppose there is a nonzero vector 𝒙 such that 𝑨𝒙 = 𝟎. Then 𝑨
cannot have an inverse
■ Note 5. A 2×2 matrix is invertible if and only if 𝑎𝑑 − 𝑏𝑐 is not zero
1
𝑎 𝑏 4!
𝑑 −𝑏
=
𝑐 𝑐
𝑎𝑑 − 𝑏𝑐 −𝑐 𝑎
■ Note 6. A diagonal matrix has an inverse provided no diagonal entries are
zero
■ A product 𝑨𝑩 of invertible matrices is inverted by 𝑩4! 𝑨4!
28
Calculation of Inverse: Gauss-Jordan Method
■ To understand 𝑨5' better, suppose we use a sequence of row
operation to bring 𝑨 to upper triangular form. Then,
𝑬𝑨 = 𝑻
– 𝑬 is the product of elementary matrices
– Now, 𝑬 has an inverse as we saw in the previous slide
– Thus, 𝑨 = 𝑬5'𝑻. Now if 𝑨 is nonsingular, then 𝑻 has nonzero
pivots
■ Therefore, we can bring 𝑨 to 𝑰 by a sequence of row operations so
h𝑨 = 𝑰
𝑬
h is the product of elementary matrices
– 𝑬
h 5'𝑰 = 𝑬
h 5', and so
– Hence, 𝑨 = 𝑬
h 5' ⇒ 𝑬
h = 𝑩𝑬
h 5'𝑬
h ⇒ 𝑬
h=𝑩
𝑰 = 𝑩𝑨 = 𝑩𝑬
29
Example: Gauss-Jordan Method
2
1 1
■ For 𝑨 = 4 −6 0 , find 𝑨5'
−2 7 2
−
30
Summary
■ If 𝑩𝑨 = 𝑰, then 𝑨𝑩 = 𝑰 as well and we write 𝑨5' = 𝑩
■ In this case, 𝑩 = 𝑬: ⋯ 𝑬' product of elementary matrices so
𝑬: ⋯ 𝑬'𝑨 = 𝑰
5'
5'
■ 𝑨 = 𝑬'5' ⋯ 𝑬5'
: 𝑰 = 𝑬' ⋯ 𝑬:
– So, a nonsingular matrix is the product of elementary matrices
■ We can also see the familiar algorithm: If a sequence of row
operations bring 𝑨 to 𝑰, the same sequence on 𝑰 gives 𝑨5'
𝑬: ⋯ 𝑬'𝑨 = 𝑰 ⇒ 𝑨5' = 𝑬: ⋯ 𝑬' ⇒ 𝑨5' = 𝑬: ⋯ 𝑬'𝑰
■ If 𝑨 has an inverse, we say 𝑨 is invertible so invertible = nonsingular
– 𝑨 is nonsingular ⇔ 𝑨 has an inverse
31
Transpose of Matrix
■ The transpose of 𝑨5 of matrix 𝑨 switches rows and columns:
1 4
5
1 2 3
– Ex.
= 2 5
4 5 6
3 6
■ An 𝑛×𝑛 matrix is symmetric if 𝑨5 = 𝑨 (𝑎'& = 𝑎&' for every 𝑖, 𝑗)
■ An 𝑛×𝑛 matrix is skew-symmetric if 𝑨5 = −𝑨 (𝑎'& = −𝑎&' for every 𝑖, 𝑗)
■ Some rules for matrix algebra:
– 𝑨𝑩 5 = 𝑩5 𝑨5
– If 𝑨 is invertible, 𝑨4! 5 = 𝑨5 4!
– If 𝑨 and 𝑩 are 𝑛×𝑛 invertible, 𝑨𝑩 4! = 𝑩4! 𝑨4!
– Any square matrix 𝑨 ∈ ℝ"×" can uniquely be written as sum of a symmetric
and a skew- symmetric matrix, known as Toeplitz decomposition:
𝑨=
1
1
𝑨 + 𝑨< + (𝑨 − 𝑨< )
2
2
symmetric
skew-symmetric
32
Appendix: Locus
■ Definition: Consider any set of equations in the
variables 𝑥', … , 𝑥) . The locus in ℝ) of this set of equations is
the set of vectors that satisfy all of the equations. The plural of
a locus is loci.
■ Definition: A line in ℝ( is the locus of the equation 𝑎𝑥 + 𝑏𝑦 = 𝑐
for 𝑎, 𝑏, 𝑐 ∈ ℝ. In general, the line is affine. The line is linear if
𝑐 = 0.
■ Definition: A plane in ℝ- is the locus of the equation 𝑎𝑥 + 𝑏𝑦 +
𝑐𝑧 = 𝑑 for 𝑎, 𝑏, 𝑐, 𝑑 ∈ ℝ. In general, the line is affine. The plane is
linear if 𝑑 = 0.
■ Definition: A hyperplane in ℝ) is the locus of the equation
𝑎'𝑥' + 𝑎(𝑥( + ⋯ + 𝑎) 𝑥) = 𝑐. It has dimension 𝑛 − 1. In general,
its is affine. The hyperplane is linear if 𝑐 = 0.
33
Appendix: Product of Lower Triangular Matrices
■ Suppose 𝑨 = 𝑎&' and 𝑩 = 𝑏&' are 𝑛×𝑛 lower triangular matrices
– By definition 𝑎&' = 𝑏&' = 0 if 𝑖 < 𝑗
■ Let 𝑪 = 𝑐&' = 𝑨𝑩. Then, it is sufficient to show that 𝑐&' = 0 when 𝑖 < 𝑗
"
𝑐&' = ^ 𝑎&( 𝑏('
()!
■ Case I: 𝑘 ≤ 𝑖 < 𝑗
– 𝑏(' = 0 for 𝑘, 𝑗 ∈ {1,2, … , 𝑛} with 𝑘 < 𝑗
■ Case II: 𝑖 < 𝑘
– 𝑎&( = 0 for 𝑖, 𝑘 ∈ {1,2, … , 𝑛} with 𝑖 < 𝑘
■ Hence, for 𝑖 < 𝑗, we have
&
"
𝑐&' = ^ 𝑎&( 𝑏(' + ^ 𝑎&( 𝑏(' = 0
()!
𝑏!" = 0 in Case I
()&7!
𝑎#! = 0 in Case II
34
Appendix: Product of Upper Triangular Matrices
■ Suppose 𝑨 = 𝑎&' and 𝑩 = 𝑏&' are 𝑛×𝑛 lower triangular matrices
– By definition 𝑎&' = 𝑏&' = 0 if 𝑖 > 𝑗
■ Let 𝑪 = 𝑐&' = 𝑨𝑩. Then, it is sufficient to show that 𝑐&' = 0 when 𝑖 > 𝑗
"
𝑐&' = ^ 𝑎&( 𝑏('
()!
■ Case I: 𝑖 > 𝑗 ≥ 𝑘
– 𝑎&( = 0 for 𝑖, 𝑘 ∈ {1,2, … , 𝑛} with 𝑖 > 𝑘
■ Case II: 𝑘 > 𝑗
– 𝑏(' = 0 for 𝑘, 𝑗 ∈ {1,2, … , 𝑛} with 𝑗 > 𝑘
■ Hence, for 𝑖 > 𝑗, we have
'
"
𝑐&' = ^ 𝑎&( 𝑏(' + ^ 𝑎&( 𝑏(' = 0
()!
𝑎!" = 0 in Case I
()'7!
𝑏!" = 0 in Case II
35