CHAPTER 1
Matrix Algebra ( Linear Algebra )
A matrix is a collection of elements arranged in an array of rows and
columns.
2 1 3

1 2 4 
Ex： A  
Amn  [aij ]
Ex： a12  1
i  1,2, . . m
. , , j  1,2,..., m
, a23  4
 a11 
a 
 12 
  .  column vector
 . 
a 
 n1 
An1
Amn
1 0
B

0 1 
,
 a11 a12
a
a 22
 21
 .
.
 .
.
a
 n1 a n 2
A1n  [a11 , a12 ,..., a1n ] row vector
... a1n 
... a 2 n 

... .  square matrix
... . 
... a nn 
Matrix Operations：
1.Addation：adding elements in corresponding
position.(element-by-element)
1 2 3
Ex： A23  

 2 0 4
,
2 4 5
B23  

1 1 7 
1  2 2  4 3  5 3 6 8 
 A B  


2  1 0  1 4  7 3 1 11
2.Subtraction：(element-by-element)
1  2 2  4 3  5  1  2  2


2  1 0  1 4  7  1  1  3
1 2
1 1 2 

, M 23  


0 4 
 4 5 6
Ex： A  B  
Ex： M 22
1
Note: Only matrices of the same size can be added or subtracted.
3.Multiplication：
1 2 3 3 6 9 


2 0 4 6 0 12
Ex： 3  A  3
Ex： M 23
Multiply each a ij by 3
 3 5
1 1 2



 , N 32  0 1
2
0
3


2 4
 1  3  1  0  2  2 1  5  1  1  2  4   7 14 
 P22  M  N  


2  3  0  0  3  2 2  5  0  1  3  4 12 22
註： Pqt  M qr  N rt
1 4 
1 0 1
4 1


Ex： A  2 3 , B  2 1 1 , C  





 5 2
1 0
1 0 3
A  B  Can’t do it !
1  4  4  5 1  1  4  2  24 9
A  C  2  4  3  5 2  1  3  2   23 8

 

1  4  0  5 1  1  0  2   4 1
Note：乘法性質
1 0
2 1
, B


 2 1
 3 4
Ex： A  
1  2  0  3 1  1  0  4 2 1
A B  


2  2  1  3 2  1  1  4 7 6
 2  1  1  2 2  0  1  1  4 1
B A 


3  1  4  2 3  0  4  1 11 4
 A  B  B  A (也有相等的時候)
2
1 0
1 0
1 0
, B
→ A B  


  B  A (a  b  b  a )
 2 1
0 1 
 2 1
ex： A  
2. AB  AC 
 B  C (ab  ac  b  c) ( a  0 )
1 1
1 1
1 0
, B
, C



1 1
0 0
0 1 
Ex： A  
1 1
1 1
AB  
 AC  


1 1
1 1
but B  C
3. A( BC )  ( AB)C
Identity Matrix：單位矩陣 ( always square )
I×A=A×I=A
I 22
1 0


0 1
I 33
1 0 0
  0 1 0


0 0 1
( 1 on diagonal and 0 elsewhere )
Ex：
2 5 8 
A  3 6 9 


4 7 10
 A  I 33
2 5 8 
 3 6 9 


 4 7 10 
1 0 0   2 5 8 
0 1 0    3 6 9   I  A

 

0 0 1   4 7 10 
1 0 0 
 2 1 3 
   2 1 3
0
1
0
Ex： B  I 33  

  4 6 5
 4 6 5 
0 0 1 
3
4. Transposition：
Ex： A23
1 3
1 2 1
列  行  A' 32  2 0




3 0 4 
1 4
Ex： B24
1
3
 1 3 4 5



B
'

42

4
 2 4 2 7

5
Note：1. ( A  B)'  A' B'
2
4

2

7
2. ( AB)'  A' B'
3. ( AB)'  B' A'
4. ( A' )'  A
5. Matrix Partitioning：
Ex： A44
B43
 a11 a12
a
a 22
  21
a31 a32

a41 a 42
 b11 b12
b
b
  21 22
b31 b32

b41 b42
a14 
a24   A11

a34   A21

a44 
a13
a23
a33
a43
b13 
b23   B11

b33   B21

b43 
A12 
A22 
B12 
B22 
Then,
A
A  B   11
 A21
A12   B11
A22   B21
B12   A11  B11  A12  B21

B22   A21  B11  A22  B21
A11  B12  A12  B22 
A21  B12  A212  B22 
Diagonal Matrices:
1.Always square ( m=n )
2.with one or more nonzero elements on the main diagonal ([\])
and zeroes elsewhere.
 2 0
Ex： 

0 0 22
,
1 0 0
 0 2 0


0 0 5 33
4
1 2 3
If M  4 5 6 ,


7 8 9

If M  Mˆ  M ,
1 0 0
then M̂  0 5 0


0 0 9
0 2 3
 
then M  4 0 6


7 8 0
Ex：
 3 0 0 
 1 2 3  
   3 10 3    1   3  2   5  3   2 

0
5
0
 4 1 2  
  12 5 4    4  1   2 

 


 0 0 2 
2 0 1 2 3  2 4 6
2  1 2 3







0 3 4 1 2 12 3 6
3  4 1 2
Ex： 
Vectors：
1.Summation Vectors：
1
1
e 
: 

1
or
1
1
1  
: 

1
1
1 2 3   1  2  3  6 
Ex： 
 1 
 
4 5 6 1 4  5  6 15

1 1 
1 2 3
  1  4 2  5 3  6  5 7 9
4 5 6
2.Inner Product：
u  v  u  v  uv
5
1 
4


Ex： u  2 , v  2
 
 
 3
1 
5
 u  v  3 2 1 1  15  2  4  21
4
3.Outer Product:
u  v  uv
1 
4


Ex： u  2 , v  2
 
 
 3
1 
1 
 4 2 1


 u  v  u  v   2 4 2 1   8 4 2
 3
12 6 3
Linear Equation Systems
Linear －Contain only variables to its first power
－ 2 x  5 , 3x1  2 x2  1
－No x 2 , x 3 , e x , ln t, sin x
－No ( x1  x2  x3 )!
 2 x  5  0 point
 2 x1  3x2  7  0 line
 3x1  x2  x3  5  0 plane
Ex：Job Retraining Programs
Total budget = $1,000,000
Program1：$2500/人  expect to earn $10000/yr
Program2：$3500/人  expect to earn $20000/yr
6
Target tax revenue = $115000/yr (from program1 and program2)
Tax rate = 2.5%
5x  7 x2  2000
Ans： 2500x1  3500x2  1000000
 1
 Ax  B
(1 0 0 0x0
1 1 5 0 0 0
2  2 0 0 0x0
2 )  0.0 2 5
 x1  2 x2  460
x 
5 7
2000
, X   1 , B  
A


 460 
1 2
 x2 
Inverse：反矩陣
If AA1  A1 A  I , then A1 is the inverse of A .
1
1
2 x  6  (2 x)  2 1 (2 x)  1  x  (6)  3
2
2
1
1
Ax  B  A ( Ax)  Ix  x  A B
(1) Existence？
3  31 
1
3
,
5  51 
1
5
,
1
0 (x)
0
Some matrices may not have inverses , we call such matrices singular,
otherwise , nonsingular (inverse exists！)
(2) Uniqueness？
Proof：Suppose B and C are inverses of A
 AC  BA  I
 B( AC )  ( BA)C
 B  I  I C
BC
How to fine the inverse A 1 ？
5 7
 A 1  ?

1 2
Ex： A  
A  A 1  I
7
s
5 7  s t  1 0

 


1 2 u v  0 1
t
Let A 1  

u v 
5s  7u  1
1
2
u 
, s

3
3
 s  2u  0
5s  7v  0
5
v

3
 t  2v  1
a
, t
2

 A 1   3
1

3
7
3
 7
3 
5 

3 
a 
In general , A   11 12   A 1  ?
a 21 a 22 
s
t
Let A 1  

u v 
a
A  A1  I   11
 a21
a12 
a22 
 s t  1 0 
u v    0 1 

 

 a s  a12u  1 ( a22 )
  11
a21 s  a22 u  0 ( a12 )
a t  a12 v  0 ( a22 )
  11
 a21t  a22 v  1 ( a12 )
 ( a a  a21a12 )  s  a22
a22
  11 22
 s
a11a22  a21a12
 ( a11a22  a21a12 )  t  a12
t
u 
 a 21
a11a 22  a 21a12
 A1 
,
1
a11a22  a21a12
v
a12
a11a22  a21a12
a11
a11a 22  a 21a12
 a22
 a
 21
a12 
a11 
1 3

2 4
Ex： A  
8
 A 1

 2
4

3


1



4  6  2 1  
1

1 3 
A
 ,
2 4

2
1 3 

 
2 4  1

3
2   1 0 

1  0 1 
2 
A  4  6  2
a12 
a
A   11

a21 a22 
Note： A1 
3
2

1 
2 
,
A  a11a22  a21a12
1
 Some matrix
A
How to fine the determinant A ？
2 6
A
 
4
9


A  2  9  4  6  6
1 4 2 
A  2 0 3


 3 5 7 
 A  1  0  7  2  5  2  3  3  4 - 2  0  3 - 3  5  1 - 7  2  4  -15
Def：
1. Minor of an element a ij = m ij = the determinant of the matrix
remaining when row i and column j are removed.
1 3 
Ex： A  
  m11  4  4
2 4
1 4 3
A   2 5 1   m 23  6  12  6


 3 6 6 
9
1 4 
A
  m 22  1  1
3
6


Def 2：Cofactor of an element a ij = A ij  (1) i j  mij
1 3
  A 12  (1)
2 4
Ex： A  
1 2
4 2 3
A   0 5 2   A 31  (1)


1 4 1 
Note：For any n  n matrix
 2  2
31
( m12  2 )
 (11)  11
( m31  2  2  3  5 )
A
n
(1)
A   a ij  A ij ：for any row i
j1
n
(2)
A   a ij  A ij ：for any column j
i 1
1 3
  A  4  6  2
2 4
Ex： A  
Row 1：1  A 11  3  A 12  (1) 11  4  3  (1) 12  2  4  6  2
Column 2：3  A 12  4  A 22  6  4  (1)
2 2
 1  6  4
1 4 2 


Ex： A  2 1 5  3  A 31  3  A 33  3  18  3  (7)  33 (Row 3)


 3 0 3 
Def 3：The adjoint of a matrix A ＝ adj A   A ij 
10
( Cofactors of A )
1 3
1 2 
 4 3
 A  
 adj A  



2 4
3 4 
 2 1 
Ex： A  
1 4 2
Ex： A   2 0 1 


 3 5 6 
 5 14 4 
 adj A   9 0 3 


10
7 8 
1
 adj A
A
Note： A1 
A ：
Properties of
1.
1 2 3
 A   4 0 5


 2 1 6 
A  A
1 3
  A  4  6  2
2
4


Ex： A  
1 2 
A  
  A  4  6  2
3 4 
2. If any row or column of A contains all zeroes, than
A  0.
1 0 5 


Ex： A  2 0 9  0


 3 0 18
3. If B is obtained by multiplying some row or column of A by a
constant C, than |B|=C |A|.
4. If any 2 rows or columns of A are interchanged, than |A| changes sign.
1 3 
2 4
B

,

1 3 
2 4


Ex： A  
A  4  6  2 ,
B 642
11
5. If two or more rows or columns in A are equal, than | A |=0.
1 1
0
Ex： A 
3 3
,
1 1
4
B 2 2
9
0
3 3 100
6. If 2 rows or columns in A are proportional, then | A |=0.
 2 3
Ex： A  
0
4 6
 1
A 9

 2
4
0
8
5
9 0

1 7
7. Evaluation of the determinant by alien cofactors always yields a
value of zero.
n
 a i j  A  i j 0
 j1

n
 a A 0
ij
ij

i 1
n
 a ij  A ij
 j1
A 
n
 a A
ij
ij

i 1
1 1 1 


Ex： A  2 0 6


 3 7 1 
f o r a n y i i
f o r a n y j j
for any i
for any j
A  2  ( 1 )  ( 6 ) 0 6 ( 1 ) ( 4)  1 2
Note： adj A   A ij 
1 5 
1 6 
 6 5  A11


A

adj
A

,

5 3 
 3 1    A
3 6 



  12
Ex： A  
12
A 21 
A 22 
Theorem： A1 
adj A
A
 a11
a
proof： A  adj A   21
 :

a n1
 n
  a1i A1j
 j1
 n
  a 2i A1j
  j1

:
 n

a ni A1j

 j1
a12
a 22
:
an2
... a1n   A11
... a 2n   A 21

:   :
 
... a nn   A n1
n
 a1i A 2 j
j1
n
a
j1
2i
A2 j
:
n
a
j1
ni
A2 j
A12
A 22
:
An 2
... A1n 
... A 2n 
: 

... A nn 

A nj 
j1
 A
n

...  a 2i A nj   0
j1
 :
 
:
  0
n

...  a ni A nj 
j1

n
a
...
1i
0
A
...
0
...
 A I
So, A  adj A  A  I  A 
adj A
adj A
 I  A1 
A
A
( AA-1  A-1 A  I )
1 3
 A1  ?
Ex： A  

3 7 
 7 3
adj A  

 3 1 
 7 3  7 3 


adj A  3 1   2
2
A1 



A
2
 3 1 
2 
 2
A  7  9  2
,
Note：1. ( AB)1  B 1 A1 ( A1 B 1 )
13
0

0
: 

A 
Is B1 A1 the inverse of AB ?
( M  M 1  M 1  M  I )
( AB )  ( B 1 A1 )  A  ( BB 1 )  A1  A  A1  I
( B 1 A1 )  ( AB )  B 1  ( A1 A)  B  B  B 1  I
1 1 
Ex： B  
  B  0  2  2
 2 0
 0 1
adj B  

 2 1 
,
,
1 2 
B  

1 0 
0 1 
2
B 
1 1 
2

1
1 
 3
 7 1
1  3 1  4
4
1
AB  
 ( AB)  



7 
4  17 7   17
17 3
4
4

0 1 
2
B A 
1 1 
2

1
1
3   3
1 
 7
2  4
4
 2

3
7 
1   17
2 
4
4
 2
2. ( A)1  ( A1 )
3 
 7
1 3
1 3
2
2
1



A 
 (A ) 
Ex： A  


3
1 
3 7 
3 7 
2
 2
3 
3 
 7
 7
2
2
2
2
1
1


A 
 ( A ) 
3
3
1 
1 
2
2
 2
 2
Note：1. For any pair of square matrices A and B , |AB|=|A|‧|B|.
2. If matrix B is obtained from A by adding a multiple of one row
of A to another row of A, then
1 3 
Ex： A  

2 4
,
1 3 
B

 4 10 

14
B  A .
A  4  6  2
,
B  10  12  2
2 3
1 1
Ex：
3 2
1
2
4
2
5
2
1
0 5 8
0 2 4

0 1 11
3 1
1
0
1
1
7
3
4
5
 (1) 2
3 1
8
4
7
3
1 11 4
47 13
 (1) 0 18
1 11
5
4
 235  234  1
Gauss-Jordan Method for Computing Inverses
A →
A-1 =?
[ A
I ]
BA BI
A-1A A-1I
[ I
A-1 ]
Def：Elementary Row Operation (e.r.o.) .
Note：Performing Elementary Row Operations to a matrix is equivalent to
pre-multiply the matrix by another matrix.
1 0 1 1 1 1
1 2 1 1 -1 1
= 
, 
Ex： 




 1 1 =  1 1 
0
3
1
1
3
3
0
1

 
 


 
 

 2 1 1
Ex：  4 6 0 


 2 7 2 
15
1 0 0  1


0 1 0   0

0 0 1  0

2 1 1
 4 6 0

 2 7 2
1 0

 0 1

0 0

3
1
2
8
8
0 0  1
2
 
2 1 0   0
 
1 0 1  0
 
1
1
2
2
3
1
0 
16  1 0 0
1
1
0   0 1 0
4
8
 
1
1
1  0 0 1
 
3
8
1
4
1
8
3
4
1
2
1
1
2
1
0
1
2
1
4
1
5
3 
16
8
3
1 
8
4
1
1 

e.r.o. :
1. Multiply some row by a constant.
2. Add some multiple of one row to another row.
Using Matrix Partition to find inverses：
1
3
Ex： A  
 1

6
2 0
4 4
3 1
5 7
A
Suppose A=  11
 A 21
AA-1=I






1
2 
2

8
A12 
A 22 
 AR  I
A
  11
 A 21
R 12 
R 22 
A12   R11
A 22   R 21
R12   I 0 

R 22  0 I 
A11R11  A12 R 21  I
A11R 12  A12 R 22  0
A 21R11  A 22 R 21  0
A 21R12  A 22 R 22  I
R
So , A  R   11
 R 21
-1
R
, then A-1=R=  11
 R 21
R12  [A11  A12 A 22 1A 21 ]1

R 22  
A 22 1A 21R11
16
R11A12 A231 

A 22 1[I  A 21R12 ]
0

1
1
0
4
8

1
1
1

1
2
0
Gramer’s rule：
Ex：5X1+7X2=2000
2000 7
460 2
X1 =
5 7
1 2
; X1+2X2=460
5 2000
1 460
, X2 
5 7
1 2
 X1 
 b1   2000
5 7 
A
,
X=
,
B=
X 
 b    460 

1 2 

 2
 2 
AX=B
 A 1AX  A 1B  IX  A 1B
1
X
adj A  B
A
 X1 
 A11 A 21
X 

 2   X  1  A12 A 22
 
A
 

Xn 
 A1n
1
 X i
(A1i b1  A 2i b 2 
A
Ex：X1=
1
(A1i b1  A 2i b 2 
A
Gramer’s rule： X i 
A Bi
A
A n1 
A n 2 


A nn 
 b1 
 
 b2 
 
 
 b n 
 A ni b n )
 A ni b n )
Where ABi is obtained from A by replacing the ith
column of A by B.
 a11
a
A   21


a n1
a1i
a 2i
a ni
a1n 
 a11

a
a 2n 
, A Bi =  21




a nn 
a n1
a12
a 22
b1
b2
an2
bn
17
a1n 
a 2n 
 A Bi  b1A1i  b 2 A 2i 


a nn 
 b n A ni
In AX=B, where A is n×n ：
If | A |≠0 , then “ unique solution ”.
If | A |=0 , then either no solution or infinite # of solution ( depends
on B ).
Theorem：A -1 exists iff | A |≠0
Note：A -1=
1
adj A
A
, If | A |＝0, A -1 does not exist
 sin gular.
The Geometry of Vectors：
Def：Given n vectors V1 ,V2 , … ,Vn , each containing m elements, and
given n scalars α1, α2 , … , αn , then
n
V0=α1V1+α2V2+…+αnVn=  i Vi is called the Linear Combination
i 1
( L.C ) of { V1,V2,…,Vn}.
1 
 3
Ex： V1    , V2   
 2
  4
1 
 3    7
V0  2V1  3V2  2    3     
 2
 4  16 
If αi≧0, i=1,2, … ,n , then V0 is called a nonnegative L.C. of
{ V1,V2,…,Vn}.
Ex：V0=2V1+4V2
n
If αi≧0 , i=1,2, … ,n , and   i
 1,
then V0 is called a Convex
i 1
Combination (C.C.) of { V1,V2,…,Vn }.
18
1
3
2
3
Ex：V0= V1+ V2
 a11 x1  a12 x 2  b1

 a 21 x1  a 22 x 2  b 2
Ax  B
a
A   11
a 21
a12 
x 
b 
, x   1 , B  1

a 22 
x 2 
b2 
V1 1 V2  2
V0
x 
  A 1 A 2   1   B  A 1 x1  A 2 x 2  B ( L.C. of columns of A )
x2 
( A i  i th column of A ) 向量 常數
In general , n×n system of equations , Ax=B  A1x1  A2 x 2    An x n  B
( B is a L.C. of {A1 , A2 , … , An}) .
 C1 
C 
Length of a vector C=  2  .
  
 
C n 
C 
C   1   C  C12  C 22
C 2 
C  C12  C22    C2n 
,
n
C
i 1
 C1 
 d1 
 C1  d1 
C 
d 
C  d 
2
2
1


Distance between 2 vectors C 
and D 
, C-D   1
  

  
 
 


C n 
d n 
C n  d n 
C  D  (C1 d1 ) 2  (C2 d 2 ) 2    (C n d n ) 2 
n
 (C
i 1
i
 di )2
C+D
C
C
C-D
C-D
D
D
19
2
i
Angle between 2 vectors C and D：
C  D  C  D  cos 
C
D
 cos  
θ
1 
- 1
, D    , cos 

  2
1
Ex： C  
(1)(-1)  (-2)(1)
(1)  (2)  ( 1)  (1)
2
2
2
2

CD
C  D
1
10
Area enclosed by 2 vectors：
C 
d 
C   1 , D   1
C 2 
d 2 
C
D
 2
7 
C1
C2
d1
 C1d1  C 2 d 2
d2
7 2
 38
Ex： C    , D    , Area 
2 6
 6
 2
Area = | C D|. (Why?)
(2.6,1.5)
1 0 
  ,    basis (基底) :
0 1 
20
2 6
1 
0 
1 5   2.6  0   1.5  1  (natural basis)


 
 
-2 
1 
0
 3   (2)  0   3  1 
 
 
 
1 3 
Ex：      basis?
  2  4 
 -2 
 1 
 3 
 3   1  2   1  4 


 
 
-2=α1+3α2
3=2α1+4α2
α1=
17
2
, α2=
7
2
=> NO!
1 3 
Ex：   ,    basis?
2 6 
  2
1 
3
3   1 2   2 6
 
 
 
Note：
1.The set of Vectors must be non-parallel in order to be a basis.
2. The set of Vectors must be Linearly independent in order to be a basis.
3.The set of Vectors must span the whole vector space in order to be a
basis.
21
Linear Independence and Linear Dependence
Def：The vectors V1 ,V2 , … ,Vn in m-dimensional space are linearly
dependent ( L.D.) if there are n scalars α1, α2 , … , αn , Not all zero ,
such that α1V1＋α2V2＋…＋αnVn＝0. If the only set of αi’s such
that α1V1＋α2V2＋…＋αnVn＝0 holds is α1＝α2＝ … ＝αn＝0,
then V1,V2 , … ,Vn are Linearly Independence( L.I. ).
 1 
 3 
Ex： V1    , V2   
 2 
 4 
   3 2  0
1 
 3  0 
1      2         1
2
 4  0 
 21  4 2  0
 1   2  0
 V1 , V2 are L.I.
 1 
 -2 
Ex： V1    , V2  

 2 
 -4 
   2 2  0
1 
 2  0
1      2         1
2
 4  0
 21  4 2  0
 choose 1  2 ,  2  1
1  6 ,  2  3
 V1 , V2 are L.D.
Theorem：Let V be an n-dimensional space and let S={X1,X2,…Xn} be a
set of vectors in V.
Then：(1) if S is L.I. , then S is a basis for V.(∵|S|=n)
(2) if S spans V,….., S is L.I.
22
Theorem：If S={X1,X2,…Xn} is a basis for V , then any vector in V can be
represented as
one and only one L.C. of X1,X2,…Xn.
Def：The rank of a Matrix A , R(A) or ρ(A) is the max. number of L.I.
columns or rows in A .
 1 2 3 
Ex： A  
  R(A)  2
 2 1 1 
 1 2 3 
Ex： B  
  R(B)  1 (
 2 4 6 
不能成為倍數)
 1 2 2 
Ex： C  
  R(C)  2
 2 7 4 
 1 2 3 
Ex： A   2 5 6   R(A)  2


 2 4 6 
Note：(1) R(A)=R(A’)
(2) If A is m×n , then ρ(A)≦min(m,n)
Quadratic Forms：
-important in minimization and maximization
2-Var 二次式：
 X1 

X2 
Let X= 
Q(X1,X2)=6 X12 +4X1X2+3 X 22
 X1
6 2
, A= 

2 3
 Q(X , X )  X 'AX
1 2
a12 
a
A=  11
 (a12 =a 21 ) Symmetric
a 21 a 22 
 6 2   X1 
X2   
 
2 3 X 2 
3-Var 三次式：
23
Q(X1,X2,X3)=3 X12 -4 X 22 +5 X 32 +2X1X2-6X1X3+8X2X3
 X1 
 3 1 3
X   X 2  , A=  1 -4 4 
 X3 
 3 4 5 
 Q(X1 , X 2 , X 3 )  X'AX
Q(X1,X2)≧0 for all X1,X2 ?
.……… ≦ ………………
Ex：2 X12 +3 X 22 ≧0
Q(X1,X2) = a11 X12 +2a12X1X2+a22 X 2 2
= a11 X12 +2a12X1X2+
a12 2 2
a 2
X 2  a 22 X 2 2  12 X 2 2
a11
a11
a12
a12
a11a 22  a12 2
2
X1X 2  ( X 2 ) ]+[
= a11[ X +2
] X 22
a11
a11
a11
2
1
a1 1a 2-a2
 The sign of Q(X1 ,X 2) depends on the signs of a 1 and
1
a11
2
1 2
.
Note：
(1) Let A=[A1,A2,…,An] , then , |A|≠0 if and only of A1,A2,…,An are L.I.
(2) If A n is a Linear Combination of A 1,A 2,…,A n-1, then A 1,A 2,…,A n-1, A n
are L.D.
An is a L.C. of A 1,A 2,…,A n-1
 A n  1A1   2 A 2 
 1A1   2 A 2 
 A1 ,A 2 ,
  n 1A n 1
  n 1A n 1  A n  0
,A n-1 ,A n are L.D.
24
(3) The max. number of L.I. Vectors in m-dimensional space is m .
1   3
Ex：     L.I
 2  4
1 
1  3  - 6 
 
 2  4  7 N o t L . I. 
     
 3 
4
;
5




1
2
1




6 
- 1 
7 
1
6
Not L.I.
Def：A set of vectors S={X1,X2,…Xn} in a Vector Space V is called a basis
if (1) S spans V and (2) S is L.I.

1 
0
0
1 

0
1
 2
0







Ex： S   X1 
, X2 =
, X3 
, X4 =  
1 
-1
 2
0







 

0
2
1 
1 






The show (2)： 1X1   2 X 2  3 X3   4 X 4  0
1 
0
0
1   0 
0
1
 2
0 0






 1
 2
 3
 4       0
1 
 1
 2
1   0 
 
 
 
   
0
2
1 
0  0 
 4
 1

 2  2 3


 1   2  2 3

2 2   3   4
0
0
=0
0
 1   2   3   4  0
The show (1)：
25
 S is L.I.
a 
1 
0
0
1 
b
0 
1
 2
0








Let X 0 
 1
 2
 3
 4  
c 
1 
 1
2
0
 
 
 
 
 
d 
0 
2
1 
1 
Definitions：
1.If Q is always positive (Q>0) for all values of Xi’s (except when all Xi’s are
zero) , then Q is positive definite (PD).
2……………….negative(Q<0)………………………………………………
……….. negative definite (ND).
3……………… nonnegative (Q≧0)………………………………………...
……. positive semi-definite (PSD).
4.……………….nonpositive (Q≦0)………………………………………...
…….. negative semi-definite (NSD).
Def：Leading Principal Minors of a matrix A.
The kth order leading principal minor of a matrix is the determinant of the
matrix made up of the first k rows and columns.
 a11 a12
Ex： A  a 21 a 22

a 31 a 32
a13 
a 23  , n=3
a 33 
 a11 a12
A3  a 21 a 22
a 31 a 32
A1  a11  a11
A2 
a11
a 21
a12
 a11a 22  a 21a12
a 22
26
a13 
a 23  = A
a 33 
A1  a1 1 1
1 2 4 
Ex： 2 1 0 


 0 3 3 
A2 
1 2
 1  4  5
2 1
1 2
A3   2  1
 0 3
4
0

3
Def：Principal Minors of a matrix A.
A principal minor of a matrix is the determinant of the matrix remained
after a particular set of rows and the corresponding columns are removed.
A kth-order principal minor of A is the determinant of the matrix remaining
when all but k rows and corresponding columns are removed.
 a11
Ex： a 21

 a 31
a12
a 22
a 32
a13 
a 23 
a 33 
 a1 1
a
 21
 a 3 1
a 1 2 a 1 3
a 2 2 a 2 3
a 3 2 a 3 3
(1,2) |A1(1)|=|a33|=a33
(1,3) |A1(2)|=|a22|=a22
(2,3) |A1(3)|=|a11|=a11
a
(3) |A2(3)|=  11
a 21
a12 
a 22 
 a11
|A3|= a 21
 a 31
a
(1) |A2(1)|=  22
a 32
a 23 
a 33 
a
(2) |A2(2)|=  11
a 31
a13 
a 33 
a12
a 22
a 32
a13 
a 23  =|A|
a 33 
Note：LPM  PM
Theorem：
1.A quadratic form, X’AX , is
(a) PD if and only if all LPMs of A are positive ( |A1|>0 , |A2|>0 , …. ,
|An|>0 ).
(b) ND if and only if the LPMs of A alternate in sign beginning negative.
( |A1|<0 , |A2|>0 , |A3|<0,…..)
27
A
2.A quadratic form, X’AX, is
(a) PSD if and only if all PMs of A are nonnegative.
( all |A1|≧0 , all |A2|≧0 , … , all |An|≧0 )
(b) NSD if and only if all odd-order PMs of A are nonpositive.
( all |A1|≦0 , all |A3|≦0 , all |A5|≦0 , …………)
and all even-order PMs of A are nonnegative.
( all |A2|≧0 , all |A4|≧0 , all |A6|≧0 , …………)
Ex：
Ex： 2X12  6X1X2  5X22
2X12  4X1X 2  6X1X 3  8X 2 X 3  X 2 2  4X 32
 X1 
X ' AX , X   
X 2 
 2  3  X 1 
[ X 1 X 2 ]
 
 3 5   X 2 
 X1
X2
 2 2 3   X1 
X 3   2 1 4   X 2 
 3 4 4   X 3 
Ex1：X12+6X22
 X1
1 0   X1 
X2  
 
0 6   X 2 
A1  1  0
A2 
1 0
60
0 6
 all LPM>0
 PD
Ex2：-X12-6X22
 X1
 1 0   X1 
X2  
 
 0 6   X 2 
A1 | 1| 1  0
A2 
1 0
60
0 6
 , ,.....
 ND
Note：If Q(X1,X2) is PD , then –Q(X1,X2) is ND.
Ex3：X12-4X1X2+4X22+X32
PSD
28
PD
NSD
ND
 X1
X2
A1  1
 1 2 0   X1 
X 3   2 4 0   X 2 
 0 0 1   X 2 
A2 
1 2
0
2 4
 Neither PD nor ND.
A1 (1)  1 , A1 (2)  4 , A1 (3)  1  all  0
A 2 (1) 
4 0
1 0
1 2
 4 , A 2 (2) 
 1 , A 2 (3) 
0
0 1
0 1
2 4
1 2 0
A 3  2 4 0 =0
0 0 1
 all  0
 all  0  PSD
Ex4：-X12+4X1X2-4X22-X32
 1 2 0   X1 
 X1 X 2 X3   2 4 0  X 2 
 0 0 1  X 2 
A1  1
A2  0
 Neither PD nor ND.
A1 (1)  1 , A1 (2)  4 , A1 (3)  1  all  0
A 2 (1) 
1 2
1 0
4 0
 0 , A 2 (2) 
 1 , A 2 (3) 
4
2 4
0 1
0 1
1 2 0
A 3  2 4 0 =0  all  0  NSD
0 0 1
Ex：X12-6X22
 X1
1 0   X1 
X2  
 
0 6   X 2 
A1  1
A2 
1 0
 6
0 6
A1 (1)  1 , A1 (2)  6
 Neither PD nor ND
 Neither PSD nor NSD
 indefinite
29
 all  0