Chapter I
Vectors and Matrices
In this chapter we present the general concepts of
vectors and matrices.
1.1- Vectors.
let us begin by defining a vector, a set of N number
which we shall write in the form
 x1 
 x 
x 2 
(1)
  


 xN 
A vector of this type is called a column vector. If N
numbers are arranged in horizontal array.
x  ( x1 , x2 , ..., x N )
(2)
x is called a row vector.
We write “x” to denote a quantity having the form in (1).
Lowe-case letters such as x, y, z or a, b, c will be
employed throughout to designate vectors. When discussing
a particular set of vectors, we shall use superscripts, thus
x1 , x 2 , etc.
The quantities x i are called the components of x, while
N is called the dimension of the vector x. One dimensional
vectors are called scalars.
-1-
By x we shall denote the vector whose components
are the complex conjugates of the elements x.
*Vector Addition.
Two vectors x and y are said to be equal if and only if
their components are equal xi  yi for i  1, 2, ..., N . The
sum of two vectors, x and y, is written x  y and defined to
be the vector
 x1  y1 
 x y 
2 
x y 2





 xN  y N 
It showed be noted that the plus sign connecting x and
y is not the same as the sign connecting x i and y i .
*Scalar Multiplication.
Multiplication of a vector x by a scalar c1 is defined by
means of the relation
 c1 x1 
 c x 
c1 x  x c1   1 2 
  


 c1 x N 
i- Multiplication of Vectors.
Let A and B are 1  n and n  1 row and column
vectors, respectively. Denoting these vectors by x T and y we
have
-2-
n
x T y   xi yi .
i 1
The result of such an operation is (complex) number, and it
follows directly from the above equation that
i- x T y  y T x ,
ii- x T ( y  z )  x T y  x T z ,
iii- ( x )T y   ( x T y )  x T ( y ) .
*Example 1.1.
 2 
 1 i 




If x   3 i  and y   2  . Find
 3i 
 1 i 




b- y T y ,
a- x T y ,
c- ( x, y) ,
d- ( y, y)
*Solution.
a- x y   2
T
3i
 1 i 
1  i   2   2 (  1  i )  6 i  (1  i ) ( 3  i )


 3  i 
 4i .
3
b- y T y   yi2  y12  y22  y32  12  8 i .
i 1
c- ( x, y )   xi yi  2 (  1  i )  3 i ( 2 )  (1  i ) ( 3  i )  2  2i .
d- ( y, y )   yi  yi  16 .
-3-
ii-The Inner Product of Two Vectors.
We now introduce a most important scalar function of
two vectors x and y, the inner product. This function will be
written ( x, y) and defined by the relation.
N
( x, y )   xi yi .
i 1
The following properties of the inner product are
derived directly from the definition:
1- ( x, y )  x T y ,
2- ( x, y)  ( y, x ) ,
3- ( x, y  z )  ( x, y)  ( x, z ) ,
4- ( x  y, z  w )  ( x, z )  ( x, w )  ( y, z )  ( y, w ) ,
5- ( c1 x, y )  c1 ( x, y ) ,
6- ( x,  y )   ( x, y) .
*Orthogonality.
If two real vectors are connected by the relation
( x, y )  0 , they are said to be orthogonal.
*Theorem 1.
  be a set of real vectors which are mutually
Let x i
orthogonal; that is ( x i , x j )  0 , i  j and normalized by the
condition, that
( xi , x j )  1
Then if a particular vector x has the representation
-4-
m
x   ci x i ,
i 1
We have the following values for the coefficients
ci  ( x , x i )
And, in addition
m
( x, x )   ci2 .
i 1
Exercises.
1- Show that we have commutatively x  y  y  x and
associatively x  ( y  z )  ( x  y )  z .
2- Define subtraction of two vectors directly, and in terms
of addition; that x  y is a vector z such that y  z  x .
3- If x is real, show that ( x, x )  0 unless x  0 .
4- Show that ( u x  v y, u x  v y )  u 2 ( x, x)  2 u v ( x, y )  v 2 ( y, y )
is non-negative definite quadratic form in the scalar
variables u and v if x and y are real and hence that
( x, y ) 2  ( x, x) ( y, y ) for any two real vectors x and y
(Cauchy’s inequality)
5- In N-dimensional Euclidean space, consider the
coordinate vectors
-5-


1
y 



1
0

0


,






2
y 



0
1
0

0



 , ...,





N
y 



0
0

1






Show that y i are mutually orthogonal and are in addition
normalized. A set of this type is called orthogonal if
N
x   ci y i determine the value of ci .
i 1
1.2- Matrices.
Let us now introduce the concept of a matrix. An array
of complex numbers written in the form
a12

a1 N 
 a11
 a
a22
 a2 N 
21

A
(1)
 

 


a
a

a
N
1
N
2
N
N


will be called a square matrix. When other types of matrices
are introduced subsequently, we shall use an appropriate
modifying adjective.
The quantities ai j are called the elements of A and the
integer N is the dimension. The quantities ai1 , ai 2 , ..., aiN are
said to constitute the ith column. Through, matrices will be
denoted by upper-case letters X , Y , Z and A, B, C .
We can use the notation
-6-
A  ( ai j )
(2)
The simplest relation between matrices is that of
equality; two matrices are called equal if and only if their
elements are equal.
The sum of two matrices A and B is written A  B and
defined by the notation
A  B  ( ai j  bi j )
Multiplication of a matrix A by scalar c1 is defined by
the relation
c1 A  A c1  ( c1 ai j ) .
~
Finally, by A we shall mean the matrix whose
elements are the complex conjugates of A. When the
elements of A are real, we shall call A real matrix.
*Matrix Multiplication-Vector by Matrix.
Consider the linear transformation of the form
N
Yi   ai j x j ,
i  1, 2, ..., N
(1)
j 1
where the coefficients ai j are complex quantities, this
system can be written in the form
Y  Ax


where Y  



y1 
 a11
a
y2 
 , A   21
 
 


yN 
 aN1
-7-
a12

a22



aN 2

a1 N 
 x1 


a2 N 
 , x   x2  .
 
  



aN N 
 xN 
This relation defines multiplication of a vector x by a matrix
A.
*Matrix Multiplication-Matrix by Matrix.
Now, let us see how to define the product of a matrix
by a matrix. Consider a second linear transformation
Z By
which converts the components of y into the components of
Z. in order to express the components of Z. in terms of the
components of x, where, as above, Y  A x , we write
N
N
 N
 N  N

Z i   bi k y k   bi k   ak j x j      bi k ak j  xi .
k 1
k 1

 j 1
 j 1  k 1
If we now introduce a new matrix C  ( ci j )
N
ci j   bi k ak j ,
i, j  1, 2, ..., N
k 1
we may write
Z Cx
since, formally
Z  B y  B ( A x )  ( B A) x .
*Notes:
The multiplication of the matrices in general is not
commutative. In other words in general A B  B A .
If A B  B A , we shall say that A and B commute.
-8-
*Associativity.
For all matrices A, B, and C
( A B ) C  A ( B C ) . To show that, we see that
we
have
( A B ) C  [ ( ai k bk L ) CL j ] ,
A ( B C )  [ ai k ( bk L cL j ) ] ,
which establishes the equality of ( A B ) C and A ( B C ) .
*Quadratic Forms as Inner Products.
The quadratic form
Q ( u, v )  a u 2  2 b u v  c v 2
can be written in the form
U ( au  bv )  V (bu  c v ) .
Hence, if the vector x and the matrix are defined by
u
a b
x   , A 

v
b c
we see that Q ( u, v)  ( x, a x ) .
Similarly, given the N-dimensional quadratic form
Q ( x) 
N
 ai j xi x j
i , j 1
where without loss of generality we may take ai j  a j i , we
can write
N

N

N

Q ( x )  x1   ai j x j   x2   ai j x j   ...  x N   ai j x j 
 j 1

 j 1

 j 1

where x has the components x i and A  ( ai j ) .
-9-
*The Transpose Matrix.
The transpose matrix A  [ ai j ] is A  ( a j i ) consider
a set of vectors { x } and another set { y } , and form all inner
products ( x, y ) composed of one vector from one set and
one from the other.
Suppose now that we transform the set { x } by means
of matrix multiplication of A, obtaining the new set { A x} .
Forming inner products as before, we obtain the set of values
( A x, y ) ,
Observe that
N
( A x, y )  y1   ai j x j
 j 1

N

y

2   ai j x j

 j 1

  ...

N

 y N   ai j x j 
 j 1

 ( x, A y ) .
In other words, as far as the inner product is concerned the
effect of the transformation A on the set of x ’s is equivalent
to the transformation A as an induced transformation or
adjoint transformation.
*Symmetric Matrices.
A matrix A is said to be symmetric iff A  A i.e.
( ai j )  ( a j i ) .
-10-
*Exercises.
 cos  sin  
T ( )  

 sin  cos 
show that
T (1 ) T ( 2 )  T ( 2 ) T (1 )  T (1   2 ) .
1- If
2- Let A be a matrix with the property that ai j  0 if i  j ,
a diagonal matrix, and let B a matrix of similar type.
Show that AB is again a diagonal matrix, and that
AB  BA.
3- Let A be a matrix with the property that ai j  0 , j  i , a
triangular or semi diagonal matrix, and let B be a matrix
of similar type. Show that AB is again a triangular
matrix, but that AB  BA, in general.
a2 
b2
 a
 b
, B 1
4- Let A   1

  a2 a1 
  b2 b1
again a matrix of same type.
5- Consider the identity matrix
0
1


I



 0
1 

 show that AB is

I
defined
by
Explicitly , I  (  i j ) , where  i j is defined by the
 0,
relation  i j  
 i,
i j
i jN
N
i j   ik  k j .
i 1
-11-
,
show
that
2
 N

6- Show that ( A x, A x )     ai j xi  .
i 1  j 1

7- Use the relation
 A B   | A| | B |
N
to
express
( a12  a22  a32  a42 ) ( b12  b22  b32  b42 ) as a sum of four
squares.
8- Consider the linear fractional transformation
a z  b1
w 1
 T1 ( z ) .
c1 z  d1
If T2 ( z ) is a similar transformation, with coefficients
a1 , b1 , c1 , d1 replaced by a2 , b2 , c2 , d 2 , show that
T1 ( T2 ( z ) ) and T2 ( T1 ( z ) ) are again transformation of
the same type.
9- Show
that
( A  B )  B   A ,
( AB )  B  A ,
( A1 A2 ... An )  An ... A2 A1 , ( A ^ )  ( A ) ^ .
10- Show that A B is not necessarily symmetric if A and B
are symmetric.
11- Show that A B A is symmetric if B is symmetric.
12- Show that ( A x, B y )  ( x, A B y ) .
13- Show that | A|  | A| .
*Orthogonal Matrices.
Definition 1:
A real matrix A is called orthogonal if its transpose A
coincides with the inverse A1 ; thus
-12-
A  A1 or A A  A A  1
(2)
An orthogonal matrix has the following properties
1- The rows (columns) of an orthogonal matrix are
orthogonal in pairs
Indeed, if A  [ ai j ] , then from (2) we have
n
 ai k a j k  0
for
i j
for
i j
k 1
and
n
 ak i ak j  0
k 1
*Example 2.
a12 
a
Let A   11
be an orthogonal matrix, then
a22 
 a21
A A  A A  I .
Then the rows (columns) of A are orthogonal.
*Solution.
Since,
a
A   11
 a21
a12 
a
, A   11

a22 
 a12
a21 
a22 
 ( a11 ) 2  (a12 ) 2 a11 a21  a12 a22
A A  
2
2
 a21 a11  a21 a12 ( a21 )  ( a22 )
the rows of A is ( a11 , a12 ) , ( a21 , a22 )
 1

 0
( a11 a22 )  ( a21 a22 )  a11 a21  a12 a22  0 .
-13-
0
1 
Which is orthogonal, similarly the columns are orthogonal.
2- The sum of the squares of the elements of each row
(column) of an orthogonal matrix is equal to unity.
From (2), for i  j , we obtain
n
n
k 1
k 1
 ai2k   ak2 j  1
3- The determinant of an orthogonal matrix is equal to  1 .
Thus, on the basis of (2) we have
det A det A  det 
whence, since det A  det A , and det   1, it follows
that
( det A ) 2  1 
det A  1 .
4- The transpose and the inverse of an orthogonal matrix are
also orthogonal matrices
Since A is orthogonal, then
A  A1
and
A A  A A   .
Prove that A 1 is orthogonal matrix i.e. ( A1 )1  ( A1 ) 1 ?
Since
( A 1 )  ( A )  A  ( A 1 ) 1
and
( A1 )1 A1  ( A ) A1  A  A1   .
( A 1 ) ( A 1 )  ( A 1 ) ( A )  A 1  A  
similarly we can prove that A is orthogonal matrix.
-14-
1.3- Eigenvalues and Eigenvectors.
*Definition.
Let A be any n  n matrix, a nonzero n  1 matrix X
such that A X   X is called an eigenvector of A and the
scalar  is called the eigenvalue of A, associated w it X.
*Example 1.
2 6
4
1
For A  
,
and
,
x

z


2
 1  we have
1
3


 
 
 20 
4
Ax  

5

 2   5x
 10 
 
and
4
1
A z        for any  .
8
1
Thus, x is an eigenvector of A associated with eigenvalue
  5 and z is not eigenvector of A.
 3 
Y 
 is an eigenvector of A associated with the

1


eigenvalue   0 .
The defining equation A x   x , is equivalent to
 x  Ax

(    A) x  0 .
This equation gives some light on the problem of how to find
eigenvectors. Since it shows that how to find eigenvectors.
Since it shows that eigenvectors are solutions of certain
homogeneous system of linear equation.
-15-
If we know the eigenvalues, we could solve the
homogeneous systems (    A ) x  0 , for the eigenvectors. If
we knew the eigenvectors then we could fined the
eigenvalues by comparing the entries of x and A x . If we
know neither, and this is the usual case, then we the
following.
The homogeneous system (    A ) x  0 , has a
nonzero solution if, and only if det (    A ) x  0 , that is, the
matrix    A is singular. The scalar  is an eigenvalue of A
if and only if det (  j   A ) x  0 .
*Example 2.
 3 6
det ( 5   A )  det 
 1 2

0 ,

and
 2 6 
det ( 0   A )  det 
  0,
 1  3 
i.e.   5,   0 are eigenvalues of A, but
 1 6 
det ( B   A )  det 
  6  0 ,

1
0


i.e. B is not eigenvalue of A.
If we think of  as a parameter, then det (    A ) is a
polynomial in  . This polynomial is called the characteristic
polynomial of the matrix A
-16-
6 
  2
det (    A )  det 

 1   3 
 (   2 ) (   3)  6   (   5 )  0
  0,   5 .
We fined the eigenvalues and eigenvectors by the following
three steps.
1- Find the characteristic polynomial
C (  )  det (    A ) .
2- Find the eigenvalues of A by finding the roots  j of the
characteristic equation C (  )  0 .
3- For each eigenvalue  j , find an associated eigenvector of
A by solving system ( ; I  A ) x  0 .
*Example 3.
Find the eigenvalues and eigenvectors of
 0 1 0 
A  0 0 1 .


 2 1  2 
*Solution.


C (  )  det (  I  A )  det  

-17-
1 0 0   0 1 0  
0 1 0  0 0 1 

 

 0 0 1   2 1  2  
0 
  1
 det  0

1 


  2  1   2 
  [  (   2 )  1]  2  0
 2 (   2 )    2  0  (   2) ( 2  1)  0
1  1, 2  1, and 3  2 . These roots are the eigenvalues
of A. We find the eigenvectors from the relation
(  I  A ) x  0 , ( 1 I  A ) x  0
1  1  ( I  A ) x  0
then
 1 1 0 
  0
1 1  x  0


  2  1 3 
for which the augmented matrix we reduce rows of this
matrix as follows.
 1 1 0 0 
 1 1 0 0 
R1  R3
 0
1  1 0  2

 0 1  1 0 




  2  1 3 0 
 0  3 3 0 
 1 1 0 0 
R2  R3
3

 0 1  1 0 


 0 0 0 0 
The system equivalent to this final matrix is
x1  x2  0 
x1  x2 ,
x2  x3
-18-
there are many solution to this system, if we set x3  1 , we
find x1  1, x2  1 .
so that
X 1  1, 1, 1 
T
is an eigenvector of 1  1 if c  0 then c x1 is also an
eigenvector for   1,
for   1
  1  1 0 0   x1 
  I  A  X  0   0  1  1 0   x2 
  2  1 1 0   x3 
 1 1 0 0 
 1 1 0 0 
 2 R1  R3
 0  1  1 0  
 0  1  1 0 




  2  1 1 0 
 0 1 1 0 
 1 1 0 0
 0  1  1 0

 0
0
0 0
x1   x2 ,
x2   x3
R2  R3
if x3  1  x2  1,

  x1  x2  0



 x2  x3  0

x3  1
X 2  1,  1, 1  is an eigenvector of A for   1.
T
*Example 4.
Find the eigenvalues and all eigenvectors of
-19-
5 4 2
B 4 5 2 .


 2 2 2 
*Solution.
The characteristic polynomial of B is
C (  )  det (  I  B )
2
  5 4

 4  5 2

  2
2 2




 3  12 2  21  10
The only rational possibilities for the eigenvalues are factors
of the constant term  1,  2,  5 and  10 from synthetic
division
10) 1  12 21  10
10  20
1 2
and
2) 1  12
10
1
0
21  10
2  20
1 2
1
2
8
we see that C (10 )  0 , C ( 2 )  8 , thus 10 is an eigenvalue
of B but 2 is not, then
3  122  21  10  (  10) (2  2   1)  0
(   10 )(   1) 2  0
-20-
1  10 ,
 2  1,
 3  1.
The augmented matrix for the system
(10 I  B ) x  0
1 4 0 
 5 4 2 0 
 1
  4 5  2 0  R

R1  R 2
1R 2
  4 5  2 0  4




  2  2 8 0 
  2  2 8 0 
1 4 0 
 1
 1 1  4 0  1 R R
2
1
2
R

R
 0
 13  0 9  18 0  
9 
9

18
0




  2  2 8 0 
 0 0 0 0 
 x1  2 x3  0
 1 0 2 0 

 0 1 2 0 
 


x  2 x  0
 0 0 0 0 
3
 2
From this row reduction we see that the general solution of
the system (10 I  B ) x  0 ; that is, the most general
eigenvector of B associated with the eigenvalue 10 is
 2c 
2
x   2c   c  2  .


 
 1 
 1 
In order to find the eigenvectors associated with repeated
eigenvalue   1, we consider the system ( I  B ) x  0 for
which the augmented matrix and row reduction are as
follows:
-21-
  4 1  2 0 
2 2 1 0
 4 4 2 0 
  0 0 0 0 




  2  2  1 0 
 0 0 0 0 

 2 x1  2 x2  x3  0
The general solution of this
arbitrary constants, is
c1





x
c2
 c1 



  2 c1  2 c2 

system which involves two
1 
 0 

0  c2  1 



  2 
 2 
and these are all the eigenvectors of B for   1.
Note that in the above example we found a set of
eigenvectors with two degrees of freedom, associated with
double eigenvalue   1. This frequently happens, but is not
always the case.
Note that the matrix
3 1 0
J  0 3 1 ,


 0 0 3 
has   3 as a triple root but the set of eigenvectors has only
one degrees of freedom.
*Theorem 1.
The eigenvalues of a real symmetric matrix are real.
-22-
*Proof.
Assume the contrary. Since A is a real matrix, it
follows from the characteristic equation | A   I | 0 , that the
conjugate of any complex characteristic root  is also a
characteristic root we obtain this result and further
information from the fact that if the equation
A x   x , holds, then the relation
A x   x , is also valid.
From these equations, we obtain the further relations
( x , A x)   ( x , x ) ,
( x, A x )   ( x, x ) .
Since
A is symmetric matrix, which implies that
( x , A x )  ( A x , x )  ( x, A x ) , the forgoing relations
yield
0  (    ) ( x, x )
whence    .
This means that the characteristic vectors of a real
symmetric matrix A can always be taken.
*Theorem 2.
Eigenvectors associated with distinct characteristic
roots of a real symmetric matrix A are orthogonal.
*Proof.
From A x   x ,
Ay   y
   , we obtain ( y, A x )   ( y, x ) , ( x, A y )   ( x, y ) ,
-23-
since ( x, A y )  ( A x, y )  ( y, A x) ; subtraction yields
0  (    ) ( x, y )
whence ( x, y)  0 .
*Exercises.
1- Determine if either X or Y is an eigenvector for A and if
so, determine the eigenvalue.
 11  9 
1
1
a- A  
,
X

,
Y


1
2
 4 2 
 
 
2 1 1
 1 
 1 
Y  0 
b- A   2 3 2  , X   1  ,






 1 1 2 
 0 
 1 
for the matrices in Exercises 2-10 find the characteristic
equation, the real eigenvalues and the corresponding
eigenvectors.
 1 3
 16  14 
2- 
.
3  14 9 



 1 5 
2 1 1
 1 0 4 
4-  2 3 2  .
5-  0 5 4  .




 1 1 2 
  4 4 3 
 5 1 1 
2 1 0


6-  1 5  1 .
7-  0 2 1  .




 1  1 5 
 2 1 0 
-24-
 3 3 2 
 1 1  3 
2 1 5 
8-  3 2  3  . 9-   1 1 1  . 10-  1  2  1  .






 5  1 2 
  2  3  3 
  3 1  1 
11- Consider the matrices A of Exercises 2, 8, 9, and 10 let P
be a matrix whose columns are the eigenvectors of A.
compute P 1 A P .
12- What are the eigenvalues of
a- a diagonal matrix?
b- a triangular matrix?
13- Show that   0 is an eigenvalue of a matrix A iff A is
singular.
14- Show that A and
B  P 1 A P
characteristic polynomial.
15- The characteristic polynomial
0 1 0
B   0 0 1  is


 3 5 2 
have the same
of
the
matrix
3  2 2  5   3  0 , show that
B3  2 B 2  5 B  3 I  0 .
16-
Show
3 0
J  0 3

 0 0
that
0
1

3 
the
is
characteristic
(   3)3
polynomial
and that the set of
eigenvectors of J has only one degree of freedom.
-25-
of