Supplement: Orthogonal Vectors
1. Inner Product
Definition of inner product in
Rn :
Let
 u1 
 v1 
u 
v 
2
u    and v   2 
  
  
 
 
u
 n
v n 
then we define another inner product (similar to the one in section II) by
u  v  u1v1  u 2 v2    u n vn .
Definition of length in
Let
u
u
Rn :
be a vector in R n . Then, the length of
u
u 
u u
e 
u
u
is
.
can be standardized by
where
,
e  1.
Important results:
Let u , v and
w
are vectors in R n and

u  u  0,

u v  vu .
for
c
is a scalar, then
u  0 ; u u  0 u  0.
1

(u  v)  w  u  w  v  w

(cu )  v  u  (cv)  c(u  v)
Important results (Cauchy-Schwarz inequality):
Let u , v are vectors in R n , then
uv  u v
.
Definition of angle between two vectors in
The angle between two vectors
cos  
u
and
v
Rn :
are
uv
, 0  
u v
.
Example:
Let
1 
1 

 
u  
 0  , v  0  .


0 

1 

Then,
cos( ) 
uv
11  0  0  0 1
1


u v
2
12  0 2  0 2 12  0 2  12
.
Thus,
 

4
.
Definition:


u
and
v
in R n are said to be orthogonal if u  v  0  
u
and
v
in R n are said to be parallel if u  v  u v
2


2 .
  0 or  
Important results:
Let
u

uv  u  v
 if
and
u
v
and
be vectors in R n . Then,
v
are orthogonal  u  v
2
 u
2
 v .
2
2. Orthogonal Vectors:
Motivating Example:
1 2 


S

u
,
u

 ,    and W  span(S ) . Then, find a set of
Let
1
2
0 3 
orthogonal vectors which also spans W.
[solution:]
2
3
 
u2
v2  u2  projection of u2 on v1 

v1  u1
1 
0 
 
2
0 
 
Suppose the set of orthogonal vectors is v1 ,v2  . From the above figure, we can
obtain
1
v1  u1    .
0 
2 2 0
v2  u 2  projection of u 2 on v1           ,
3 0 3
3
where
projection of u 2 on v1 
u 2 cos 
v1
v1 
u 2 v1 cos 
v1
2
v1 
u 2  v1
v1
v1  v1
 2  1  3  0  1 2

    
1

1

0

0

 0   0 
and  is the angle between the vector
Note: For two vectors
u
and
u2
and
v1 .
v , the projection of u
on
v
is
u
u  projection of u on v
v
v
projection of u on v
projection of u on v 
u cos 
v
v
u v cos 
v
where  is the angle between the vector
u
and
2
v
u v
v
vv ,
v.
Example:
1 1  1  
      


S

u
,
u
,
u

0, 2, 2  and
1
2
3
Let
 0   0   3  
      
set of orthogonal vectors which also spans W.
4
W  span (S ) .
Then, find a
[solution:]
1 
2
 
0 
u2
v2  u2  projection of u2 on v1 

v1  u1
1 
0 
 
0 
Suppose the set of orthogonal vectors is v1 , v 2 , v3 . From the above figure, we can
obtain
1
v1  u1  0
.
0
1 1 0
v2  u2  projection of u2 on v1   2  0  2
,
0 0 0
where
projection of u 2 on v1 
u 2 cos 
v1
v1 
u 2 v1 cos 
v1
2
v1 
u 2  v1
v1
v1  v1
1 1
1

1

2

0

0

0

   

 0  0
 1  1  0  0  0  0    
0 0
Similarly,
v3  u3  projection of u3 on v1   projection of u3 on v2 
1 1 0 0
      

 2    0    2   0 

 3
 
0 
 
0
 
 3

5
where
projection of u3 on v1 
u3 cos1 
v1
v1 
u3 v1 cos 
v1
2
v1 
u3  v1
v1
v1  v1
1 1
1

1

2

0

3

0

   

 0  0
 1  1  0  0  0  0    
0 0
projection of u3 on v2 
u3 cos 2 
v2
v2 
u3 v2 cos 
v2
2
v2 
u3  v2
v2
v2  v2
0  0 
 1  0  2  2  3  0    

 2  2
 0  0  2  2  0  0    
0 0
, 1 is the angle between the vectors u3 and v1 , and  2 is the angle between the
vectors u3 and v2 .
0 
v3  u 3 - (projectio n of u 3 on v1 , v 2 )  0
3
u3
projection of u3

0 
projection of u 3 on v 2  2
0
1
projection of u 3 on v1  0
0
Therefore,
1 0 0 
      
0, 2, 0  is a set of orthogonal vectors which spans W.
 0   0   3  
      
6
Gram-Schmidt Process:
For a set of vectors
u1 , u2 ,, un
set of orthogonal vectors
and W  spanu1 , u 2 ,, u n  , a
v1 , v2 ,, vn
which also spans W:
v1  u1
v 2  u 2  projection of u 2 on v1 
v3  u 3  projection of u 3 on v1   projection of u 3 on v 2 

vi  u i  projection of u i on v1   projection of u i on v 2     projection of u i on vi 1 

v n  u n  projection of u n on v1   projection of u n on v 2     projection of u n on v n 1 
That is,
v1  u1
u2  v1
v1
v1  v1
v2  u 2 

vi  ui 
ui  vi 1
u v
u v
u v
vi 1  i i 2 vi 2    i 2 v2  i 1 v1
vi 1  vi 1
vi 2  vi 2
v 2  v2
v1  v1

vn  u n 
un  vn1
u v
u v
u v
vn1  n n2 vn2    n 2 v2  n 1 v1
vn1  vn1
vn  2  vn  2
v2  v2
v1  v1
Note: Let v1 , v2 ,, vn be an set of orthogonal vectors. Then,
w1 
v
v1
v
, w2  2 ,, wn  n
v1
v2
vn
,
are a set of orthonormal vectors.
Definition of orthogonal basis and orthonormal basis:
Let S  v1 , v2 ,, vn  and T  w1 , w2 ,, wn  be two bases for a vector
7
space V. Then, suppose S is a set of orthogonal vectors and T is a set of
orthonormal vectors, then S is called an orthogonal basis and T is called
an orthonormal basis.
Important Result:
Let
T  w1 , w2 ,, wn 
n
be an orthonormal basis in R . If
v  a1 w1  a2 w2    an wn , w  b1 w1  b2 w2    bn wn ,
then,
v  w  a1b1  a2 b2    an bn .
8