Solutions
Block 3: S e l e c t e d Topics i n L i n e a r Algebra
Unit 7: The Dot ( i n n e r ) Product
3.7.1
I n o u r d i s c u s s i o n of t h e game of mathematics a t t h e s t a r t of t h i s
course, we i n d i c a t e d t h a t one o f t e n t r i e s t o a b s t r a c t t h e c r u c i a l
a s p e c t s of a well-known system i n o r d e r t o s e e what inescapable
consequences follow from t h e most "reasonable" s e t of axioms.
With t h i s i d e a i n mind, l e t
.us observe m a t a s of t h i s moment,
w e have n o t r e q u i r e d our g e n e r a l i z e d v e c t o r space t o have t h e
e q u i v a l e n t of a " d o t " product d e f i n e d on it. That i s , we have
t a l k e d about b a s e s of a v e c t o r space and about l i n e a r transformat i o n s , b u t we have n o t r e q u i r e d t h a t t h e r e be defined an o p e r a t i o n
which allows us t o a s s i g n t o each p a i r of v e c t o r s a number.
Thus, i f w e were i n t e r e s t e d i n developing t h e anatomy of a v e c t o r
space s t i l l f u r t h e r , it would next be reasonable t o d e f i n e t h a t
p a r t of t h e s t r u c t u r e which i s c h a r a c t e r i z e d by what t h e d o t
product does f o r o r d i n a r y 2- and 3-dimensional v e c t o r spaces.
What mathematicians f i r s t n o t i c e d i n t h i s r e s p e c t was t h e importance of l i n e a r i t y among t h e v a r i o u s a t t r i b u t e s of t h e d o t product.
For t h i s reason, they i n s i s t e d t h a t a r u l e t h a t assigned t o ordered
p a i r s of v e c t o r s a r e a l number n o t even be considered an " i n n e r "
( d o t ) product u n l e s s it possess t h e property of l i n e a r i t y . With
t h i s i n mind, they d e f i n e d a b i l i n e a r * f u n c t i o n on a v e c t o r space
t o be any mapping
such t h a t f o r a , 6, YEV and c E R
*We
s a y b i l i n e a r b e c a u s e t h e domain o f f i s o r d e r e d p a i r s .
{ ( a , b ) : a € A , ~ E B ) ) . Hence
* * R e c a l l t h a t b y A x B we mean
S x S denotes t h e s e t o f a l l ordered p a i r s of elements i n
S.
Solutions
Block 3: S e l e c t e d T o p i c s i n L i n e a r Algebra
U n i t 7: The Dot ( i n n e r ) Product
3.7.1 continued
Note #1
I n keeping w i t h t h e n o t a t i o n o f t h e u s u a l d o t p r o d u c t , one o f t e n
writes
a-B r a t h e r than f ( a , B ) .
With t h i s i n mind,
(1) becomes
rewritten a s
1.
(a
2.
a.
+
8 ) * y = a - y + Boy
( 6 + y) = a - y + a - y
a - (cB) = ( c a ) * B = c ( a
3.
B
Note #2
a - B i s 2 number, n o t a member
I t i s c r u c i a l t o remember t h a t
of
v.
Note # 3
L a t e r w e s h a l l add t h e requirement t h a t a-B=
B-a
, but
for
now t h i s p r o p e r t y i s n o t imposed on o u r d e f i n i t i o n o f a b i l i n e a r
function.
1, and
For t h i s r e a s o n
I n o t h e r words, i f w e assume t h a t
2, a r e n o t redundant.
a-B
=
B-a
f o r a l l a , BEV,
then
a
( B + y)
=
a-6
+
a-y
.
S i n c e , however, w e do n o t assume
a-B =
6-a
, we
cannot
d e r i v e ( 2 ) from (1).
Note # 4
There a r e , i n g e n e r a l , many d i f f e r e n t ways ( u s u a l l y i n f i n i t e l y
many ways) t o d e f i n e f :V x V
obeyed.
+
R such t h a t p r o p e r t y
(1) i s
(We s h a l l d i s c u s s t h i s i n more d e t a i l l a t e r ) .
Thus,
when w e t a l k a b o u t a.6 w e a r e assuming t h a t w e have chosen
one p a r t i c u l a r b i l i n e a r f u n c t i o n .
Solutions
Block 3: S e l e c t e d Topics i n L i n e a r Algebra
U n i t 7: The Dot ( i n n e r ) Product
3.7.1 continued
With t h i s a s background, w e now t r y t o e x p l o r e why t h e l i n e a r
What we s h a l l show i s t h a t many of
t h e u s u a l s t r u c t u r a l p r o p e r t i e s o f a d o t p r o d u c t follow j u s t
properties a r e s o important.
a.
by l i n e a r i t y .
+
w
O* = a
In particular:
-(a + 6 )
= a . d + a 0 6 .
Since a
'
8
is, i n any e v e n t , a number (i.e . ,
l e t us d e n o t e it by b.
a member o f R)
-
W e t h e n conclude from ( 2 ) t h a t
and s i n c e b i s a number, w e conclude by elementary a r i t h m e t i c
Thus, w e have shown t h a t
from ( 3 ) t h a t b = 0 .
I n summary, w e have shown i n p a r t ( a ) t h a t t h e u s u a l p r o p e r t y
a
6 = 0 h o l d s f o r any b i l i n e a r f u n c t i o n .
b.
Given,
(a3a3 + a a ) , w e may t r e a t a3a3 + a 4 a 4 a s a
(alal + a 2a 2 )
4 4
s i n g l e v e c t o r whereupon w e may u s e 1. t o conclude t h a t
= (alal
+
a2a2) - y
[where y = a3a3 + a 4 a 4 j
and t h i s by 3. e q u a l s
+
*We h a v e r e s o r t e d t o t h e n o t a t i o n 0 t o r e i n f o r c e t h e i d e a t h a t
a . B i n an o p e r a t i o n on v e c t o r s n o t numbers.
+
**It i s n o t a n o v e r s i g h t t h a t we h a v e y r i t t e n 0 r a t h e r t h a n 0
on t h e r i g h t s i d e o f ( 4 ) .
Namely, a . 0 i s a number, n o t a
vector.
Solutions
Block 3: S e l e c t e d Topics i n L i n e a r Algebra
U n i t 7: The Dot ( i n n e r ) Product
3.7.1 continued
Replacing y by a3a3
+
a 4 a 4 , (5) becomes
By 2. w e know t h a t
a l e ( a3a 3
+
a 4 a 4 ) = a l - ( a3a 3
+
al. ( a 4 a 4 ),
and t h i s , i n t u r n , by 3. i s
Similarly
From ( 7 ) and (8) w e see t h a t (6) may be r e p l a c e d by
S i n c e e q u a t i o n ( 9 ) i n v o l v e s o n l y numbers*, w e may u s e t h e r u l e s
o f o r d i n a r y a r i t h m e t i c t o r e p l a c e (9) by
I n summary, t h e n , p r o v i d e d t h a t w e keep t h e o r d e r " s t r a i g h t "
i s n o t n e c e s s a r i l y t h e same a s a 4 - a l ) , j u s t a s
(i.e. ,
al' a4
w e had t o do i n o u r s t u d y o f t h e c r o s s - p r o d u c t , we see from (10)
t h a t t h e l i n e a r i t y o f t h e d o t p r o d u c t i s what a l l o w s us t o
t r e a t i t a s w e t r e a t t h e analogous e x p r e s s i o n s of numerical
arithmetic.
"That i s , w h i l e a a ' a 3 , and a 4 a r e v e c t o r s , a
numbers.
That i s t ' w % y we h a v e p a r e n t h e s i z e d a L,
t o emphasize t h a t a 1
a2 i s
number.
S.3.7.4
a
,
e t c . are
namely
la,, ' e t c . ,
L
Solutions
Block 3: S e l e c t e d T o p i c s i n L i n e a r Algebra
U n i t 7: The Dot ( i n n e r ) P r o d u c t
3.7.2
a. Once a g a i n w e r e s t r i c t o u r a t t e n t i o n t o n = 2 f o r computational
s i m p l i c i t y , b u t o u r r e s u l t s h o l d f o r a l l n.
More i m p o r t a n t l y ,
t h e t e c h n i q u e w e s h a l l u s e h e r e i s t h e same one t h a t i s used i n
t h e higher dimensional cases.
Keep i n mind t h a t o u r main aim i n t h i s e x e r c i s e i s t o show that
a p a r t i c u l a r b i l i n e a r f u n c t i o n i s completely determined once
w e know i t s e f f e c t on each p a i r o f b a s i s elements.
So suppose t h a t V = [ul.
chosen b a s i s f o r V.
V. u21 where {ul, u 2 1 i s an a r b i t r a r i l y
Now l e t a and B
be a r b i t r a r y elements o f
Say,
From (1) and p a r t ( b ) o f t h e p r e v i o u s e x e r c i s e w e conclude that S i n c e x , x , y , y a r e known once a and B a r e g i v e n , w e see from
1 2 1 2
( 2 ) t h a t a.B i s determined once t h e v a l u e s of u l - u l , u ~ - uu2 - ~
u l I~
and u 2 - u 2 a r e f i x e d .
Note #1
With a l i t t l e i n s i g h t , e q u a t i o n ( 2 ) g i v e s u s a h i n t as t o why
o u r m a t r i x coding system i s used i n y e t a n o t h e r c o n t e x t i n
vector spaces.
Namely, i f w e l e t
-P
t
x
= [xl
,xzl
and
=
w e s e e t h a t equation (2) says t h a t
Solutions
Block 3: S e l e c t e d T o p i c s i n L i n e a r Algebra
U n i t 7: The Dot ( i n n e r ) P r o d u c t
3.7.2
continued
A s a check on t h e e q u i v a l e n c e of
( 2 ) and ( 2 ' ) , w e have
e
.-.
and i t i s e a s i l y s e e n t h a t t h e number named by ( 3 ) i s t h e same
a s t h a t named by ( 2 ) .
Note #2
The c o n v e r s e o f Note #1 i s a l s o t r u e .
d e n o t e any 2 by 2 m a t r i x .
f u n c t i o n f on V x V.
Namely, suppose w e l e t
W e may now u s e A t o i n d u c e a b i l i n e a r
Namely, i f V = [u1,u21
, we
define f a s
- u2-u1 -- a21; and u 2 * u 2 = a 2 2 .
W e t h e n u s e ( 2 ) t o e x t e n d t h e d e f i n i t i o n t o a l l of V. T h i s i s t h e e s s e n c e o f p a r t ( b ) wherein we i l l u s t r a t e o u r l a s t remark more c o n c r e t e l y . b. L e t V = [u1,u21,
B = (1,4), then
then i f a = (3,2) [ i . e . ,
3ul
+
2u21 and
Solutions
Block 3: S e l e c t e d T o p i c s i n L i n e a r Algebra
Unit 7: The Dot ( i n n e r ) P r o d u c t
3.7.2
continued
I n " l o n g hand" o u r m a t r i x A codes t h e b i l i n e a r f u n c t i o n d e f i n e d
by
ul.ul
= 1, u - u
1
2
= -1, u2-u1 = -5,
and u .u = 6.
2 2
W e would t h e n o b t a i n
which checks w i t h ( 4 )
.
Note # 3
What w e have e s s e n t i a l l y shown i s t h a t i f dim V = n ( a l t h o u g h
w e used n = 2) t h e n t h e r e a r e a s many b i l i n e a r f u n c t i o n s which
can b e d e f i n e d on V ( r e l a t i v e t o a f i x e d b a s i c { u l r . . . , u n
a s t h e r e a r e n by n m a t r i c e s .
Namely, i f A = [ a i j ]
by n m a t r i x , w e d e f i n e a b i l i n e a r f u n c t i o n by u i - u
j
whereupon
is
the b i l i n e a r
f u n c t i o n induced by A.
}
i s any n
= a
ij
Solutions
Block 3: S e l e c t e d Topics i n L i n e a r Algebra
Unit 7: The Dot ( i n n e r ) P r o d u c t
3.7.2
continued
Note # 4
S i n c e t h e r e a r e many d i f f e r e n t b a s e s f o r t h e same v e c t o r , two
different
n by n m a t r i c e s may code t h e same b i l i n e a r f u n c t i o n
b u t with respect t o d i f f e r e n t bases.
When we become concerned
w i t h t h i s problem w e have y e t a n o t h e r s i t u a t i o n i n which w e must
d e f i n e what w e mean f o r two m a t r i c e s t o be e q u i v a l e n t .
That i s ,
i n t h e same way t h a t we have i d e n t i f i e d d i f f e r e n t m a t r i c e s a s
being e q u i v a l e n t ( i n t h i s case, c a l l e d s i m i l a r ) i f they denote
t h e same l i n e a r t r a n s f o r m a t i o n o f V b u t w i t h r e s p e c t t o ( p o s s i b l y )
d i f f e r e n t b a s e s , w e might want t o i d e n t i f y d i f f e r e n t m a t r i c e s a s
b e i n g e q u i v a l e n t i f t h e y d e f i n e t h e same d o t product.
Certainly
t h e s e two d e f i n i t i o n s of " e q u i v a l e n t " need n o t b e t h e same.
T h a t i s , two m a t r i c e s may code t h e same l i n e a r t r a n s f o r m a t i o n
b u t n o t t h e same d o t p r o d u c t .
Hopefully, o u r t r e a t m e n t i n t h e f i r s t two e x e r c i s e s makes i t c l e a r
t h a t one does n o t need t h e b i l i n e a r f u n c t i o n t o b e commutative i n
o r d e r f o r it t o have a n i c e s t r u c t u r e .
W e observe, however, t h a t
t h e " u s u a l " d o t p r o d u c t does obey t h e commutative r u l e .
f o r any "arrows"
and
z,
g
z
=
That i s .
g.
Thus, t o c a p t u r e , even more, t h e f l a v o r o f t h e o r d i n a r y d o t
p r o d u c t w e add a n o t h e r p r o p e r t y t o t h e b i l i n e a r f u n c t i o n f , namely
that f ( a ,B)
a-B = B-a
= f ( B , a ) ; o r i n t h e Language of t h e d o t p r o d u c t ,
.
W e c a l l t h e r e s u l t i n g b i l i n e a r form a symmetric b i l i n e a r form,
and w e n o t i c e t h a t n o t h i n g new happens t h a t d i d n ' t a l r e a d y o c c u r
i n o u r p r e v i o u s t r e a t m e n t e x c e p t now a l l of o u r m a t r i c e s which
r e p r e s e n t t h e d o t p r o d u c t a r e a l s o symmetric.
I n o t h e r words,
f o r a symmetric b i l i n e a r f u n c t i o n w e know t h a t ulWu2 = u2*u1,
Hence t h e m a t r i x
and even more g e n e r a l l y t h a t u i a u = u * u
j
j i'
i s a symmetric m a t r i x ( t h a t i s , we g e t t h e same m a t r i x i f w e
i n t e r c h a n g e rows and columns). I n any e v e n t , it i s t h e s t u d y
S.3.7.8
Solutions
Block 3: S e l e c t e d Topics i n L i n e a r Algebra
U n i t 7: The Dot ( i n n e r ) P r o d u c t
3.7.3 c o n t i n u e d
of symmetric b i l i n e a r f u n c t i o n s t h a t makes t h e s t u d y o f symmetric
matrices especially important.
a.
I f w e assume t h a t V = [u1,u21
A
=
s
t h e n t h e symmetric m a t r i x
]
codes t h e symmetric b i l i n e a r f u n c t i o n d e f i n e d by
W e t h e n have t h a t
Had w e wished t o a r r i v e a t ( 3 ) u s i n g m a t r i x n o t a t i o n , w e would
have
which checks w i t h ( 3 )
.
Solutions,
Block 3: S e l e c t e d T o p i c s i n L i n e a r Algebra
U n i t 7: The Dot ( i n n e r ) P r o d u c t
3 . 7 . 3 continued
b. W e know from o u r g e o m e t r i c t r e a t m e n t of t h e d o t p r o d u c t e a r l i e r i n
o u r c o u r s e t h a t t h e l e n g t h o f t h e p r o j e c t i o n o f v2 on vl i s g i v e n
by
s o t h a t t h e v e c t o r p r o j e c t i o n o f v2 o n t o v 1 i s given by
Pictorially,
( F i g u r e 1j
Solutions
Block 3 : S e l e c t e d Topics i n L i n e a r Algebra
Unit 7: The Dot ( i n n e r ) P r o d u c t
3 . 7 . 3 continued
W e may now d i v i d e vl
and v2* by t h e i r magnitudes t o determine an
orthonormal b a s i s f o r t h e s p a c e spanned by vl and v2.
Before p r o c e e d i n g t o t h e a c t u a l i l l u s t r a t i o n of t h i s e x e r c i s e ,
l e t us o b s e r v e t h a t w h i l e i t was c o n v e n i e n t t o u s e geometry t o
c o n s t r u c t v2*, i t was n o t n e c e s s a r y t o do s o . W e could have done
t h e same t h i n g a l g e b r a i c a l l y ( a x i o m a t i c a l l y ) .
t h e a l g e b r a i c approach remains v a l i d f o r n
approach doesn ' t
.
2
More i m p o r t a n t l y ,
3 while t h e geometric
The a l g e b r a i c d e t e r m i n a t i o n of v2*
comes from o b s e r v i n g t h a t
+
OP = xvl (where x i s t o b e d e t e r m i n e d ) , t h a t xvl
+
v2* = v
( o r v2* = v2
-
xvl) and t h a t v2*
s o t h a t v2* must be g i v e n by
which a g r e e s w i t h t h e g e o m e t r i c s i t u a t i o n .
Turning now t o o u r s p e c i f i c e x e r c i s e , w e have
Hence,
v1 = 0.
Namely, Solutions
Block 3: S e l e c t e d T o p i c s i n L i n e a r Algebra
U n i t 7: The D o t ( i n n e r ) P r o d u c t
3.7.3
continued
Then,
Pictorially,
z
(Figure 2 )
v2* i s t h e v e c t o r p r o j e c t i o n
The s p a c e spanned by vl
and P2.
+
+
of OP2 o n t o OP1.
and v2 i s t h e p l a n e d e t e r m i n e d by 0 , P1,
Solutions
Block 3: S e l e c t e d Topics i n L i n e a r Algebra
U n i t 7: The Dot ( i n n e r ) P r o d u c t
3.7.3 c o n t i n u e d
A s a check t h a t vl
and v2* i s an o r t h o g o n a l b a s i s f o r t h e s p a c e
spanned by vl and v 2 , w e need o n l y check t h a t v2* belongs t o
t h i s s p a c e , and vl
That vl
v1
'
v
2
*
v2* = 0.
In particular,
= 0 may b e checked a t once from (6).
v2* = ( 1 , 2 , 3 )
. -7
(11,29, -23)
A s t o whether v2* b e l o n g s t o t h e s p a c e spanned by vl and v 2 , w e
need simply row-reduce t h e m a t r i x
;t
1
-
J - c -v1 -v2
t o obtain
from which w e conclude t h a t S ( v l r v 2 ) = {xl ( l r 0 , 1 9 )
+
x2 (0,1,-8)
orI
{
.
(x1tx2, 19x1 - 8 ~ I ~ )
In particular,
++z
Hence,
(v11v2)
= -23.
(11,29, -23) ES (vl,v2)
*
.
1
T h e r e f o r e , v2* = - ( I l l29 ,-23) ES
7
1
Solutions
Block 3: S e l e c t e d Topics i n L i n e a r Algebra
Unit 7: The Dot ( i n n e r ) Product
3.7.3 c o n t i n u e d
Note:
Do n o t l e t t h e f a c t t h a t w e a r e now d e a l i n g w i t h d o t p r o d u c t s make
you f o r g e t t h a t one u s e s t h e row-reduced m a t r i x t e c h n i q u e q u i t e
In particular, w e
a p a r t from any knowledge o f a d o t p r o d u c t .
a n a l y z e t h e s p a c e S ( v1 , ~ 2 ) i n t h e same way a s w e would have i n
t h e f i r s t t h r e e u n i t s o f t h i s Block. What w e s h o u l d b e b e g i n n i n g
t o f e e l now i s t h e number o f d i f f e r e n t ways i n which m a t r i x
n o t a t i o n i s used t o code completely d i f f e r e n t a s p e c t s o f v e c t o r
spaces.
To conclude t h i s p a r t o f t h e e x e r c i s e w e need o n l y d i v i d e b o t h
v1 and v2* by t h e i r magnitudes and we have o b t a i n e d t h e d e s i r e d
orthonormal b a s i s .
v2*
= Jv2*
. v2*
To t h i s end w e have:
= - dl21
7
+
841
+
529
=
;
.
Hence, w e l e t
Then,
S(v1,v2) = [u1,u21
c.
where u
1
ul = u2
-
u2 = 1 and ul
u2 = 0 .
A t f i r s t g l a n c e it might s e e m t h a t p a r t ( b ) was an i n t e r r u p t i o n
of o u r t r a i n of t h o u g h t i n going from p a r t ( a ) t o p a r t ( c ) ,
b u t t h i s is hardly t h e case.
Rather what w e wanted t o show i s
t h a t t h e t e c h n i q u e used i n p a r t ( b ) which i s s o obvious from a
g e o m e t r i c p o i n t o f view works, word f o r word, i n t h e more
abstract situations.
I n p a r t i c u l a r , i n t h i s e x e r c i s e we want t o e x p r e s s t h e s p a c e
V = [ul,
u ] d e s c r i b e d i n p a r t ( a ) by r e p l a c i n g u2 by a n
2
a p p r o p r i a t e u2*, where u 1 u2* = 0 .
Solutions
Block 3: S e l e c t e d Topics i n L i n e a r Algebra
Unit 7: The Dot ( i n n e r ) P r o d u c t
3 . 7 . 3 continued
To t h i s end, w e need simply t a k e
That is,
where
Moreover, w e see from (10) t h a t
Note #1:
A s w e may see from ( l l ) ,t h e m a t r i x of o u r symmetric b i l i n e a r
f u n c t i o n r e l a t i v e t o t h e b a s i s {ul*, u2*) i s g i v e n by
T h a t i s , t h e m a t r i x A of p a r t ( a ) and t h e m a t r i x B o f ( 1 2 ) ,
code t h e same symmetric b i l i n e a r f u n c t i o n b u t w i t h r e s p e c t t o a
different basis.
The m a t r i x B shows us why {ul*, u2*) i s a " b e t t e r " b a s i s t h a n
{ul, u 1 a t l e a s t w i t h r e s p e c t t o t h e given b i l i n e a r f u n c t i o n .
2
Solutions
Block 3: S e l e c t e d Topics i n L i n e a r Algebra
Unit 7: The Dot ( i n n e r ) P r o d u c t
3.7.3
continued
Namely from (12) we s e e t h a t
I n m a t r i x form,
(13) may b e d e r i v e d from
Note #2:
W e may u s e (11) t o f i n d a m a t r i x P such t h a t
PAP^
= B.
That i s ,
w e may d i a g o n a l i z e a symmetric m a t r i x A by a p p r o p r i a t e l y choosing
P. T h i s c h o i c e o f P comes from t h e following.
1. W e know t h a t ul*
ul* = u
ul, SO r e l a t i v e t o {u1 , ~ 2 1a s a
1
b a s i s w e may code t h i s p i e c e of i n f o r m a t i o n by w r i t i n g
S i n c e u2* = -2ul
+
u2, w e may r e p r e s e n t u2*
u2* by
P u t t i n g ( 1 4 ) and (15) i n t o a s i n g l e form we have
Solutions
Block 3: S e l e c t e d Topics i n L i n e a r Algebra
Unit 7: The Dot ( i n n e r ) Product
3 . 7 . 3 continued
where
and
Note 8 3 :
The p r o c e d u r e used i n o b t a i n i n g t h e d i a g o n a l m a t r i x B from t h e
symmetric m a t r i x A g e n e r a l i z e s t o n-dimensional space.
The
s t r a n g e p a r t , a t l e a s t i n t e r m s of o u r u s u a l e x p e r i e n c e , i s t h a t
t h e d i a g o n a l i z e d m a t r i x may i n c l u d e n e g a t i v e e n t r i e s a s w e l l a s
z e r o among i t s d i a g o n a l e n t r i e s .
For example, w i t h r e s p e c t t o
(12) , n o t i c e t h a t -1 a p p e a r s on t h e d i a g o n a l .
What t h i s means i s t h a t i f w e s t i l l want t o i d e n t i f y v
v with
t h e " l e n g t h " o f v , t h e n some symmetric b i l i n e a r f u n c t i o n s produce
t h e " d i s t u r b i n g " r e s u l t s t h a t a v e c t o r can have an imrnaginary
l e n g t h and t h a t a non-zero v e c t o r can have a z e r o l e n g t h .
While
it i s u n d e r s t a n d a b l e t h a t such a r e s u l t may s e e m d i s t u r b i n g , i t
i s i m p o r t a n t t h a t w e t r y t o u n d e r s t a n d t h e s t r u c t u r e o f mathe-
m a t i c s t o an e x t e n t which a l l o w s us t o a c c e p t such r e s u l t s i n a
n a t u r a l way.
I n e s s e n c e when w e s a y t h a t w e want t o t h i n k o f
v
v a s t h e s q u a r e of t h e magnitude of v , we have t o r e a l i z e
t h a t u n l e s s o u r a x i o m a t i c approach h a s c a p t u r e d e v e r y p e r t i n e n t
i n g r e d i e n t o f t h e system w e a r e t r y i n g t o c h a r a c t e r i z e , i t i s
p o s s i b l e t h a t c e r t a i n c o n c e p t s may p o s s e s s s t r a n g e p r o p e r t i e s
i n t h e g e n e r a l i z e d , a b s t r a c t system.
This happened t o us i n
Block 1 when w e i n t r o d u c e d modular a r i t h m e t i c a s an example
t h a t obeyed much o f t h e s t r u c t u r e of o r d i n a r y a r i t h m e t i c , y e t
was d i f f e r e n t enough t o c a u s e us some s t a r t l i n g r e s u l t s .
Solutions
Block 3: S e l e c t e d Topics i n L i n e a r Algebra
U n i t 7: The Dot ( i n n e r ) Product
3.7.3 c o n t i n u e d
I n any e v e n t , l e t us o b s e r v e t h a t t h i s e x e r c i s e shows u s t h a t
r e l a t i v e t o o u r p r e s e n t axioms, a symmetric b i l i n e a r f u n c t i o n
may d e f i n e a d o t p r o d u c t f o r w h i c h v
v < 0 or v
v = 0 even
i f v # 0.
Note #4:
P u r s u i n g t h e p r e v i o u s n o t e i n somewhat more d e t a i l , we s e e from
(13) t h a t r e l a t i v e t o t h e b a s i s (ul*,
u2*1
That i s ,
hence
Thus, u s i n g ( 1 6 ) , w e s e e t h a t i f v = xul*
+
yu2*, t h e n
I n o t h e r words, t h e v e c t o r s v , which a r e non-zero s c a l a r
m u l t i p l e s o f ul* +_ u2*, a r e d i s t i n g u i s h e d by t h e f a c t t h a t t h e y
a r e n u l l vectors (v is called a n u l l vector i f v # 0 but v
v =
0)
W e may u s e (10) t o i n v e s t i g a t e how n u l l v e c t o r s look r e l a t i v e t o
t h e b a s i s {ul, u 2 )
.
Namely,
solutions
Block 3: S e l e c t e d T o p i c s i n L i n e a r Algebra
U n i t 7: The Dot ( i n n e r ) Product
3.7.3
continued
while
Check:
(-u1
+ u2)
'
(-u1
+
-
u 2 ) = u1 ' u1
2u1
u2
+
U2
'
u2
Note #5:
The m a t e r i a l d i s c u s s e d i n Notes 3 and 4 l e a d s t o an a p p l i c a t i o n
of symmetric b i l i n e a r f u n c t i o n s i n t h e s t u d y of t h e a l g e b r a i c
t o p i c known a s q u a d r a t i c forms.
In n variables a quadratic
form i s any e x p r e s s i o n of t h e t y p e
I n t h e s p e c i a l c a s e n = 2,
(17) becomes
The c o n n e c t i o n between q u a d r a t i c forms and symmetric b i l i n e a r
f u n c t i o n s s h o u l d become c l e a r r a t h e r q u i c k l y once w e r e a l i z e
t h a t i f w e l e t a12 = 1 b12,then
Solutions
Block 3 : S e l e c t e d Topics i n L i n e a r Algebra
Unit 7 : The Dot ( i n n e r ) Product
3 . 7 . 3 continued
I n o t h e r words, any e q u a t i o n o f t h e form
allxl
2
+
b12x1x2
+ a 2 2 ~22 = m
may be viewed a s t h e v e c t o r e q u a t i o n
where v
v i s d e f i n e d by t h e symmetric m a t r i x
R e l a t i v e t o o u r p r e s e n t e x e r c i s e , we have t h a t
is equivalent t o solving v
v = m where v = x u
+ x2u2.
That
is,
More c o n c r e t e l y i f w e l e t m = 0 i n ( 1 9 ) , t h e n w e want t o s o l v e
t h e equation
T h i s can b e done r a t h e r simply h e r e ( s i n c e t h e r e a r e o n l y two
v a r i a b l e s ) by completing t h e s q u a r e t o o b t a i n
Solutions
Block 3: S e l e c t e d ~ o p i c si n L i n e a r Algebra
Unit 7: The Dot ( i n n e r ) P r o d u c t
3 . 7 . 3 continued
The key c o m p u t a t i o n a l p o i n t i s t h a t r e l a t i v e t o t h e b a s i s
{ul*, u2*1 e q u a t i o n (20) may b e w r i t t e n a s
where now v = ylul*
+
y2u2*.
I n summary, t h e p r o c e s s of r e p l a c i n g t h e symmetric m a t r i x A by
t h e d i a g o n a l m a t r i x B a l l o w s us t o r e p l a c e a q u a d r a t i c form by
an e q u i v a l e n t form i n which o n l y t h e s q u a r e terms a r e p r e s e n t .
a.
Since t h e matrix
i s symmetric, it codes a symmetric b i l i n e a r f u n c t i o n .
In parti-
c u l a r , i f V = [ u l r u 2 ] , t h e n t h e m a t r i x A codes t h e symmetric
b i l i n e a r f u n c t i o n d e f i n e d by
where w e have w r i t t e n ( 2 ) i n a form which we hope s u g g e s t s t h e
m a t r i x A g i v e n i n (1). We t h e n have
Solutions
Block 3: S e l e c t e d Topics i n L i n e a r Algebra
Unit 7 : The Dot ( i n n e r ) Product
3.7.4
continued
I n p a r t i c u l a r , we s e e from (3) t h a t if ( x l l x 2 ) = (ylry2) then
By way of review, t h e r i g h t s i d e of ( 4 ) i s known a s a q u a d r a t i c
form.
as v
We may always view
'
v where t h e d o t product i s defined by t h e symmetric
matrix
with a , b, and c a s i n ( 5 ) .
b.
So f a r we have t h a t
and
We a l s o know t h a t i f f u l l u21 i s a b a s i s f o r V s o a l s o i s
{ul + xu2, u21 where x i s any r e a l number. So what we t r y t o
do i s choose x s o t h a t
To s o l v e (7) we s e e t h a t
so that
Solutions
Block 3: S e l e c t e d T o p i c s i n L i n e a r A l g e b r a
U n i t 7: The Dot ( i n n e r ) P r o d u c t
3.7.4
continued
Using ( 6 ) , w e see t h a t ( 8 ) becomes
W e now r e p l a c e ul by
a n d w e choose a s o u r new b a s i s f o r V,
From ( 9 )
W e a l s o know t h a t
s i n c e t h i s i s how ul* was chosen. Check: ul*
'
u2 = (ul
4
- u )
5 2
Eul*, u21
.
Solutions
Block 3: S e l e c t e d Topics i n L i n e a r Algebra
Unit 7: The Dot ( i n n e r ) Product
3.7.4
continued
Finally
Hence, from
(lo),
( l l ) ,and (12) w e conclude t h a t t h e m a t r i x o f t h e
g i v e n d o t p r o d u c t r e l a t i v e t o t h e b a s i s (ul*,
u21 i s
I n o t h e r words, i f VEV i s g i v e n by v = ylul* + y u , t h e n
2 2
1
2
v . v = - 5
Y12 + 5y2 and t h e yly2 t e r m i s missing. Again, by
way of review, (13) may b e o b t a i n e d from (1) a s f o l l o w s . We
know t h a t
and t h a t
(14) and (15) may b e combined i n t o t h e s i n g l e form
Solutions
Block 3: S e l e c t e d T o p i c s i n L i n e a r Algebra
Unit 7: The Dot ( i n n e r ) P r o d u c t
3.7.4
continued
2
c . What ( b ) shows i s t h a t t h e e q u a t i o n 3xl + 8x1x2 + 5x22 = m i s
equivalent t o
5 Y1
2+5y22=mwherethex'sandy'sarere-
-
l a t e d a s follows.
I f v = ylul*
where xl
+
= yl
y2u2*, t h e n
and x2 = y 2
4
- 5
yl.
In particular,
Hence, v = y u *
11
+
y u * is a n u l l v e c t o r ++y2 =
2 2
1 +- F
yl
u
v i s a non z e r o s c a l a r m u l t i p l e o f ul*
+
u2*
++
v i s a non z e r o s c a l a r m u l t i p l e of (ul
-
45- u 2 )
v = k'(ul
constant. ++
Check: -
u2) o r k ( u l
-
+ -15 u 2
3
u 1 , k an a r b i t r a r y non-zero
5 2
Solutions
Block 3: S e l e c t e d Topics i n L i n e a r Algebra
Unit 7: The Dot ( i n n e r ) P r o d u c t
3.7.4
continued
3.7.5
(Review o f t h e L e c t u r e )
A s d e s c r i b e d i n t h e l e c t u r e , t h e p r o c e s s o f f i n d i n g an o r t h o g o n a l
b a s i s may b e extended t o an n-dimensional space. The procedure
i s i n d u c t i v e . Namely, w e have a l r e a d y s e e n t h a t i f
V = [u1,u2,...,u
1 t h e n we may r e p l a c e u2 by
n
...,un I ,
u2* = 0. Once w e
1
know t h a t ul and u2 a r e o r t h o g o n a l w e u s e t h e f a c t t h a t i f u
3
i s r e p l a c e d by u3* where u * had t h e form:
3
whereupon, V = [ul,u2*,
b u t now u
and u
t h e n u1,u2, and u * span t h e same s p a c e a s t o u
3'
3
ltU2'
W e t h e n t r y t o d e t e r m i n e xl and x i n such a way t h a t ul
2
u3 *
= u2 ' u3* = 0.
.
A s l o n g a s n e i t h e r ul
done.
2
u2 = 0, t h i s can always be
Namely, g i v e n t h a t
and t h a t ul
obtain
ul n o r u
u
2
= 0, w e may d o t both s i d e s of
( 2 ) by ul t o
solutions
Block 3: S e l e c t e d T o p i c s i n L i n e a r Algebra
Unit 7: The Dot ( i n n e r ) P r o d u c t
3.7.5
continued
whereupon t h e c o n d i t i o n t h a t ul
u3* i s t o e q u a l 0 a l l o w s u s t o
conclude from ( 3 ) t h a t
S i m i l a r l y , i f w e had d o t t e d b o t h s i d e s of ( 3 ) w i t h u2 and s e t
u2
u3* e q u a l t o 0, w e would have o b t a i n e d
So, p u t t i n g ( 4 ) and (5) i n t o ( 2 ) would y i e l d
Equation (6) r e p r e s e n t s a g e n e r a l i z a t i o n of o u r approach i n
E x e r c i s e 3.7.3.
Notice t h a t
i s t h e v e c t o r p r o j e c t i o n o f u3 o n t o ul,
while
i s t h e v e c t o r p r o j e c t i o n of u3 o n t o u 2 ' The procedure i n d i c a t e d
i n ( 6 ) i s known a s t h e Gram-Schmidt O r t h o g o n a l i z a t i o n P r o c e s s .
...,
Suppose {ul,
u } i s an o r t h o g o n a l s e t
n
a r e a l l d i f f e r e n t from
and t h a t ul
ul,
and un-1
Un-l
z e r o . Given un, w e l e t
I t works a s f o l l o w s .
.
...,
We t h e n d o t b o t h s i d e s w i t h uk where k = 1,
this yields
..., o r
n
-
1 and
Solutions
Block 3: S e l e c t e d Topics i n L i n e a r Algebra
Unit 7: The Dot ( i n n e r ) P r o d u c t
3.7.5 c o n t i n u e d
S i n c e uk
uk # 0 b u t uk
ui = 0 f o r i # k, we may s e t uk
u *
n
equal t o 0 i n (8) t o o b t a i n
P u t t i n g t h e r e s u l t o f ( 9 ) i n t o ( 7 ) w e have t h a t i f ui
f o r i # j; i, j = 1,
...,n
-
1 then i f
. uj
- 0
Looking a t (10) one dimension a t a t i m e , n o t i c e t h a t t h e c o e f f i c i e n t
of uk i s t h e v e c t o r p r o j e c t i o n of u
I n o t h e r words, t h e
n o n t o uk.
c o e f f i c i e n t of uk h a s i t s numerator e q u a l t o (minus) u
un and
k
i t s denominator e q u a l t o uk
uk. With t h i s a s background, we
proceed w i t h t h e p r e s e n t e x e r c i s e .
..
With V = [u1,u2, u31, t h e m a t r i x
d e f i n e s t h e symmetric b i l i n e a r f u n c t i o n given by
u1
'
u1 = 1 u1
'
u2 = 1
u1
. u3
Hence t o f i n d an o r t h o g o n a l b a s i s f o r V w e b e g i n by l e t t i n g
Solutions
Block 3: S e l e c t e d Topics i n L i n e a r Algebra
Unit 7: The Dot ( i n n e r ) P r o d u c t
3.7.5 c o n t i n u e d
and
Hence,
V = [ul*,
u2*, u31 and ul*
u2* = 0 .
S i n c e ul* and u2* a r e o r t h o g o n a l , w e u s e t h e Gram-Schmidt process t o replace u
u3* = u
3
-
3 by
u1 ' u3
( ul** u1 * ) ul*
-
u2*
(u2*
u
u2
Using ( l l ) , (13) , and (14) w e have
Thus, u s i n g ( 1 7 ) , w e see from (16) t h a t
From (13) , ( 1 4 ) , and (18) we conclude t h a t an o r t h o g o n a l b a s i s
f o r V i s g i v e n by
Solutions
Block 3: S e l e c t e d T o p i c s i n L i n e a r A l g e b r a
U n i t 7: The Dot ( i n n e r ) P r o d u c t
3.7.5
continued
Check:
b.
From ( 1 7 ) w e a l r e a d y know t h a t ul*
-
ul* = 1 and u2*
u2* = 1.
From ( 1 8 ) ,
Hence, r e l a t i v e t o {ul*,
Therefore,
u2*, u3*
1 ,v
= x u
* +
11
x u
2 2
* +
x3u3*+
Solutions
Block 3: S e l e c t e d Topics i n L i n e a r Algebra
Unit 7: The Dot ( i n n e r ) P r o d u c t
3.7.5 c o n t i n u e d
C.
Given t h a t v = 3ul*
obeyed.
Hence v
+
4u2*
+
5u3* w e s e e t h a t e q u a t i o n (20) i s
v = 0 , and, a c c o r d i n g l y , v i s a n u l l v e c t o r .
Again u s i n g ( 1 3 ) , ( 1 4 ) , and (18) w e have
Check:
Hence, p u t t i n g t h e s e t h r e e r e s u l t s i n t o a s i n g l e m a t r i x e q u a t i o n
yields
Solutions
Block 3: S e l e c t e d Topics i n L i n e a r Algebra
Unit 7: The Dot ( i n n e r ) Product
3.7.5
e.
continued
The equation
X1
+
2xZ2 + 4xj2
+
2x x
1 2
+
2x1x3
+ 6x2x3
= m
is e q u i v a l e n t t o s o l v i n g
v = m relative t o
Namely (21) evolves from computing v
{u1,u2,u3)
y = xlul
(i.e.,
v
and o b t a i n i n g
+
x2u2
+
x3u3) and (22) evolves from s o l v i n g
v = m r e l a t i v e t o {ul*, u2*, u3*)
where v
- ylul*
+
y2u2*
+
; namely,
y3u3*.
We now d e f i n e an i n n e r , o r a d o t , product on a v e c t o r space V t o
be any mapping of ordered p a i r s of elements of V i n t o t h e r e a l
numbers, say, f :VXV
+
R which has t h e following p r o p e r t i e s .
For vlr v2, v3€V and C , E R
(Y)
f(vl,
V2)
(ii) f (v1,v2
= f(v
+ v3)
1tV2)
= f(vlIv2)
+ f (vlIv3)
(iii)f (vlr cv2) = c f (vlrv2)
( i v ) f ( v l , v l ) 2 0 ; and f ( v l , v l )
= 0
++
v1 = 0.
Solutions
Block 3: S e l e c t e d Topics i n L i n e a r Algebra
Unit 7: The Dot ( i n n e r ) P r o d u c t
3.7.6
continued
R e w r i t t e n i n terms o f t h e d o t n o t a t i o n we have:
(i)
v l - v 2 = v 2
(ii) v1
(iii)v
1
( i v ) v1
v1
( v 2 + v3) = v1
v2
(cv2) = c ( v l
v2)
.
> O ;
v1 -
vl.
+
v1
v3
v l = o + + v1=
o.
When a symmetric b i l i n e a r f u n c t i o n s a t i s f i e s p r o p e r t y ( i v ) , t h e
From a n o t h e r p o i n t of view,
function is called positive definite.
what p o s i t i v e d e f i n i t e means i s t h a t when w e r e p l a c e t h e symmetric
m a t r i x by t h e e q u i v a l e n t d i a g o n a l m a t r i x , e v e r y e n t r y on t h e
d i a g o n a l i s a p o s i t i v e ( r e a l ) number.
The c r u c i a l p o i n t i s t h a t once w e have a p o s i t i v e d e f i n i t e ,
symmetric, b i l i n e a r f u n c t i o n , t h e n w e can i d e n t i f y t h e t r a d i t i o n a l
d e f i n i t i o n o f a d o t p r o d u c t [namely t h e d e f i n i t i o n which s a y s t h a t
+ anbn] w i t h o u r "new"
(bl ,
,bn) = a 1b 1 +
(al ,
,an)
definition.
...
...
...
More s p e c i f i c a l l y , and w e s h a l l i l l u s t r a t e t h i s more c o n c r e t e l y
i n t h e p r e s e n t e x e r c i s e , i f w e have a p o s i t i v e d e f i n i t e , symmetric,
b i l i n e a r f u n c t i o n d e f i n e d on V (and t h i s u s u a l l y i s a b b r e v i a t e d by
s a y i n g t h a t w e have an i n n e r p r o d u c t d e f i n e d on V ) , t h e n w e may
u s e t h e Gram-Schmidt O r t h o g o n a l i z a t i o n p r o c e s s t o f i n d an o r t h o g o n a l
b a s i s f o r V.
f a c t t h a t uk*
Of c o u r s e , t h a t much w e could do b e f o r e ; b u t now t h e
uk* i s p o s i t i v e f o r each member o f o u r o r t h o g o n a l
b a s i s , means t h a t
is real.
I n t h i s c a s e w e can l e t
...,
] and {wl,
w 1 i s n o t only o r t h o g o n a l
n
n
b u t a l s o orthonormal, meaning i n a d d i t i o n t o wi
w = 0 if i # j
j
T
h
i
s
s
h
a
l
l
b e summarized
t h a t wk
wk = 1 f o r each k = l , . .. , n .
whereupon V = [wl,...,w
a s a n o t e a t t h e end of t h i s e x e r c i s e .
Solutions
Block 3: S e l e c t e d T o p i c s i n L i n e a r Algebra
U n i t 7: The Dot ( i n n e r ) P r o d u c t
3.7.6
continued
Turning t o t h i s e x e r c i s e w e have t h a t
codes t h e symmetric b i l i n e a r f u n c t i o n , d e f i n e d on V = [u1,u2,u31,
by
Now j u s t a s b e f o r e w e l e t
and
Then [u , u 1 = [ul*,
1 2
W e next let
Now,
u2*1, and ul*
u2* = 0 -
Solutions
Block 3: S e l e c t e d Topics i n L i n e a r Algebra
Unit 7 : The Dot ( i n n e r ) P r o d u c t
3 . 7 . 6 continued
and
Consequently ( 5 ) may b e r e w r i t t e n a s
r
Matrix Check
1
Solutions
Block 3: S e l e c t e d Topics i n L i n e a r Algebra
Unit 7: The Dot ( i n n e r ) Product
3.7.6
continued
From ( 6 ) w e conclude t h a t
where
In particular,
( 6 ) t e l l s us t h a t our b i l i n e a r form i s p o s i t i v e
d e f i n i t e s i n c e each v = V has t h e form
whereupon
Finally, i f we let
Solutions
Block 3: S e l e c t e d Topics i n L i n e a r Algebra
Unit 7: The Dot ( i n n e r ) P r o d u c t
3.7.6
continued
w e may v e r i f y t h a t {wl, w2, w3) is an orthonormal b a s e s f o r V.
I n p a r t i c u l a r , r e l a t i v e t o {wl, w2, w3}
y3) = x 1Y 1
+
,
(xl, x 2 , x3)
(ylf y 2 ,
X2Y2 + X3y3'
Check:
Note :
T h i s i s t h e c r u c i a l exercise i f w e a r e t o i d e n t i f y o u r d e f i n i t i o n
of a d o t p r o d u c t w i t h t h a t g i v e n i n t h e t e x t ( f o r t h a t m a t t e r ,
w i t h t h a t given i n most t e x t s ) . N o t i c e t h a t i n t h e t e x t , one
i d e n t i f i e s vectors with n-tuples,
d e f i n e d by
whereupon t h e d o t p r o d u c t i s
The key p o i n t i s t h a t t h e r e a r e a s many d i f f e r e n t ways o f r e p r e s e n t i n g an n-dimensional s p a c e V i n terms of n - t u p l e s a s t h e r e
What t h i s e x e r c i s e shows
a r e ways o f choosing a b a s i s f o r V.
i s t h a t i f o u r symmetric b i l i n e a r f u n c t i o n i s p o s i t i v e d e f i n i t e
( i . e . , a d o t p r o d u c t ) , t h e n w e can always c o n s t r u c t an o r t h o I f ul,
un d e n o t e s t h i s orthonormal b a s i s ,
normal b a s i s f o r V.
S. 3.7.37
...,
Solutions
Block 3: S e l e c t e d Topics i n L i n e a r Algebra
Unit 7: The Dot ( i n n e r ) Product
3.7.6
continued
then r e l a t i v e t o t h i s b a s i s , o u r d o t product i s a s given by ( 7 ) .
I n o t h e r words;
sinceu
u
= landui
u = O i f i f j.
i
j
Notice, however, t h a t i f a l l we know i s t h a t ul,.
i
..,un
i s an
a r b i t r a r y b a s i s f o r V, then we a r e n o t allowed t o conclude t h a t
r e l a t i v e t o t h i s b a s i s ( x ~ ~ . - . ~ x( y~l I). . . , y n ) = xlyl +
+
xnyn s i n c e we do n o t know t h a t ui
u = 1 o r t h a t ui
u for
i
j
i # j.
...
The key p o i n t i s t h a t once we know t h a t we have a p o s i t i v e
d e f i n i t e , symmetric, b i l i n e a r f u n c t i o n then r e l a t i v e t o t h i s
t h e r e i s an orthonormal b a s i s (which may be found by t h e
Gram-Schmidt o r t o g o n a l i z a t i o n p r o c e s s ) .
t h i s b a s i s , then=
If
we now agree t o use
may t h i n k of t h e d o t product i n terms of t h e
u s u a l n-tuple d e f i n i t i o n .
By t h e u s u a l use of t h e c r o s s product we know t h a t ul x u 2 i s
p e r p e n d i c u l a r t o t h e plane determined by ul and u2. We a l s o
know t h a t
-t
On t h e o t h e r hand,
Solutions
Block 3: S e l e c t e d T o p i c s i n . L i n e a r Algebra
Unit 7: The Dot ( i n n e r ) Product
3.7.7
continued
Comparing (1) and ( 2 ) w e have t h a t -ul
-
1
u2
+
u3 i s p a r a l l e l
1
t o ( i.e., i s a s c a l a r m u l t i p l e o f ) ul x u 2 ' Hence -u 1 - 2- u 2+ u 3
i s p e r p e n d i c u l a r t o t h e p l a n e determined by ul and u2. I n f a c t ,
i f w e want t o make t h e g e o m e t r i c i n t e r p r e t a t i o n complete, o u r
c o n s t r u c t i o n of u2* i n t h e l e c t u r e was e q u i v a l e n t t o t a k i n g t h e
component of u2 which was p e r p e n d i c u l a r t o ul.
C l e a r l y ul and
u2* determined t h e same p l a n e a s d i d ul and u
2'
I t i s worth n o t i n g t h a t t h e u s u a l geometric procedure seems much
e a s i e r t o h a n d l e t h a n t h e more a b s t r a c t Gram-Schmidt Orthogonal i z a t i o n P r o c e s s , b u t t h e f a c t remains t h a t t h e l a t t e r method
a p p l i e s t o a l l v e c t o r s p a c e s i n which an i n n e r p r o d u c t i s d e f i n e d ,
w h i l e t h e more i n t u i t i v e geometric t e c h n i q u e a p p l i e s o n l y t o 2and 3-dimensional s p a c e .
You may r e c a l l i n o u r f i r s t U n i t i n t h i s Block, we emphasized
t h e f a c t t h a t t h e r e w e r e i n f i n i t e dimensional v e c t o r s p a c e s and
t h a t one such s p a c e was t h e set of continuous f u n c t i o n s on an
interval [a,bl.
I n t h i s e x e r c i s e w e want t o p o i n t o u t t h a t t h e
i d e a of a d o t p r o d u c t and t h e a s s o c i a t e d concept of " d i s t a n c e "
make s e n s e i n t h i s r a t h e r a b s t r a c t model of a v e c t o r s p a c e . I n
f a c t , t h e s t u d y o f o r t h o g o n a l f u n c t i o n s ( o f which t h e s p e c i a l
c a s e of F o u r i e r S e r i e s w i l l b e d i s c u s s e d i n t h e n e x t and f i n a l
u n i t ) makes e x c e l l e n t s e n s e i n terms of t h e g e n e r a l i z e d n o t i o n
of " d i s t a n c e " .
More s p e c i f i c a l l y , w e know t h a t s i n c e f ( x ) i s continuous on [ a , b ] ,
jabf ( x ) dx e x i s t s ; and w e a l s o know t h a t t h e p r o d u c t of two
i n t e g r a b l e f u n c t i o n s i s an i n t e g r a b l e f u n c t i o n . Consequently,
b
s i n c e b o t h f and g a r e continuous on [ a , b] , $ f ( x ) g ( x ) dx i s a
w e l l - d e f i n e d number.
Thus, t h e d e f i n i t i o n
Solutions
Block 3: S e l e c t e d Topics i n Linear Algebra
Unit 7: The Dot ( i n n e r ) Product
3.7.8
continued
i s a t l e a s t a f u n c t i o n which maps ordered p a i r s of continuous
f u n c t i o n s , d e f i n e d on [ a , b ] , i n t o t h e r e a l numbers; and accordi n g l y t h e d e f i n i t i o n given above i s a t l e a s t e l i g i b l e t o be
considered a s a p o s s i b l e d o t product.
Our aim i n t h i s e x e r c i s e
i s t o show t h a t indeed (1) does d e f i n e a d o t product; and we do
t h i s by showing t h a t t h e d e f i n i t i o n given i n (1) s a t i s f i e s
, (ii), (iii), and ( i v ) r e q u i r e d of any d o t product.
p r o p e r t i e s (i)
Given t h a t
then
(ii) ( c f )
g = [[cf
(iii) f . g =
=
(x, I g ( x ) d x
6"
£(x)g(x)dx
P.
(XI
f (XI
-
Solutions
Block 3: S e l e c t e d Topics i n L i n e a r Algebra
U n i t 7: The Dot ( i n n e r ) P r o d u c t
3.7.8
continued
and s i n c e [ f ( x ) 1
2
2
0 f o r a l l x,
2
k [ f ( x )1 dx > 0 f o r a l l x and
2
O* ++f = 0; f
f > 0 and f . f = 0
$ [ £ ( x ) ~ d x = 0 ++f(x)
++ f = 0.
Hence, s i n c e ( i ), (ii), (iii)and ( i v ) a r e a l l obeyed
f
g , d e f i n e d by f
g = $f
( ~ ) ~ ( x ) idsx a p o s i t i v e d e f i n i t e ,
symmetric, b i l i n e a r f u n c t i o n .
I n o t h e r words, it i s an i n n e r
product.
Note :
Once w e have a d o t p r o d u c t d e f i n e d on a v e c t o r s p a c e , i t makes
sense t o t a l k about orthogonality.
I n p a r t i c u l a r , suppose t h a t
fl,...,fn
a r e any l i n e a r l y independent f u n c t i o n s which a r e cont i n u o u s on [ a , b l and t h a t f o r i # j,
i # j
fi
'
f
j
= 0.
That is, f o r
I
W e t h e n have a s i m p l e way o f computing t h e c o e f f i c i e n t s cl,
...,
and cn i n an e x p r e s s i o n such a s
g(x) = clfl(x)
+
... + cn f n ( x ) .
Indeed w e need o n l y employ t h e u s u a l t e c h n i q u e of d o t t i n g b o t h
s i d e s o f ( 3 ) w i t h f k ( x ) ( f o r any k = 1 , 2 , 3 , .
t o f i n d each ck.
.., n )
i n order
In particular, t h i s leads t o
Each term on t h e r i g h t s i d e of ( 4 ) i s z e r o e x c e p t f o r c k f k
= 0 whenever i # j.
j
I f w e rewrite ( 4 ) i n terms o f d e f i n i t i o n ( l ) ,we o b t a i n
since fi
f
*Here we u s e t h e f a c t t h a t f i s c o n t i n u o u s f o r o t h e r w i s e we
c o u l d a l l o w f ( x ) t o b e unequal t o 0 a t , f o r example, a f i n i t e
number o f p o i n t s i n [ a , b ] and y e t
fk,
Solutions
Block 3: S e l e c t e d Topics i n L i n e a r Algebra
Unit 7: The Dot ( i n n e r ) Product
3.7.8
continued
from which w e conclude t h a t
. ..
[Notice t h a t f k # 0 s i n c e f
,£ 1 i s l i n e a r l y independent.
2
hence by ( i v ) ,
[ f k ( x ) 1 dx # 0 and, consequently ( 6 ) i s w e l l -
$
.
defined ]
I f a l s o w e assume t h a t t h e f ' s a r e normalized, i . e . ,
f
f =
2
d b [ f (XI I dx = l * , t h e n ( 6 ) may b e f u r t h e r s i m p l i f i e d t o r e a d
Using t h e r e s u l t of ( 7 ) i n ( 3 ) w e o b t a i n
A s i s u s u a l l y the c a s e , very few s e r i o u s problems a r i s e a s l o n g
a s w e l o o k a t f i n i t e l i n e a r combinations. The deep s t u d y b e g i n s
when we assume t h a t we a r e d e a l i n g w i t h an i n f i n i t e sets o f
f u n c t i o n s d e f i n e d and c o n t i n u o u s on an i n t e r v a l [ a r b ] . For now,
j u s t a s i n o u r s t u d y o f power series i n P a r t 1 of t h i s c o u r s e ,
w e must a s k t h e q u e s t i o n o f whether e v e r y continuous f u n c t i o n
d e f i n e d on [ a , b ] can be r e p r e s e n t e d a s a convergent series i n
.
*If f
f # 1 , we a r e i n no g r e a t t r o u b l e .
Namely, i f
f = k $ 1 (k $ 0 ) , then s i n c e f
f 2 0, k 2 0, & is real.
f
whereupon
We t h e n r e p l a c e f by f /
11
&
i s then c a l l e d a weighting f a c t o r f o r f .
Block 3: S e l e c t e d Topics i n L i n e a r Algebra
Unit 7: The Dot ( i n n e r ) P r o d u c t
3.7.8
continued
which t h e series i s an i n f i n i t e l i n e a r combination of elements of
o u r s e t S. Once w e have answered t h i s q u e s t i o n ( h o p e f u l l y , i n t h e
a f f i r m a t i v e ) we must t h e n a s k whether t h e p r o c e s s of d o t t i n g t e r m
by term a s w e d i d i n t h i s e x e r c i s e i s v a l i d .
I n o t h e r words, a s
u s u a l , such s t a t e m e n t s a s " t h e i n t e g r a l of a sum i s t h e sum o f
t h e i n t e g r a l s " depends on t h e f a c t t h a t o u r sum i n v o l v e s o n l y a
f i n i t e number of terms.
The d i s c u s s i o n h i n t e d a t h e r e forms t h e f o u n d a t i o n of such t o p i c s
a s r e p r e s e n t a t i o n s by F o u r i e r series. We s h a l l n o t go i n t o t h e
t h e o r y o f F o u r i e r series i n t h i s c o u r s e , b u t w e s h a l l , i n t h e
n e x t u n i t , t a l k a b o u t a few p r o p e r t i e s o f F o u r i e r series.
3.7.9
a.
(optional)
Suppose
then
a r e o r t h o g o n a l w e have t h a t ul
Since {ul,...,un)
T h i s , coupled w i t h t h e f a c t t h a t ul
j = 2,. ..,n.
u = 0 for
j
0 = 0,
r e d u c e s (21 t o
and ul
u a r e r e a l numbers, w e conclude from ( 3 )
1
1
t h a t e i t h e r cl = 0 o r ul
u1 = 0.
Now, s i n c e c
But ul
ul = 0 ++ u1 = 0 ( i . e . , a d o t p r o d u c t i s p o s i t i v e
definite).
Hence, i f ul # 0, t h e n cl = 0.
W e may r e p e a t t h i s
p r o c e d u r e by d o t t i n g b o t h s i d e s of (1) w i t h u2, u 3 , . . . ,
...,un) i s any
clul + ...
'nun
and we conclude t h a t i f (ul,
orthogonal v e c t o r s , then
o r un;
s e t of non-zero
= O c + c =
=
..'
+
C
n
= 0.
I n o t h e r words, any set of non-zero orthogonal v e c t o r s i s
l i n e a r l y independent.
T h i s does n o t mean t h a t one needs t h e
concept of a d o t p r o d u c t t o s t u d y l i n e a r independence b u t r a t h e r
Solutions
Block 3:. S e l e c t e d T o p i c s i n L i n e a r Algebra
Unit 7: The Dot ( i n n e r ) Product
3.7.9
continued
t h a t once one knows about a d o t p r o d u c t , t h e r e a r e e a s i e r ways
o f t e s t i n g f o r l i n e a r independence.
Along t h e s e same l i n e s , n o t i c e how w e used t h i s i d e a when w e
-t
3
d e a l t w i t h 1, $, and k components i n E
Quite generally, i f
.
where ul,
..., and
un a r e o r t h o g o n a l , w e may compute cl by d o t t i n g
b o t h s i d e s of ( 4 ) w i t h ul.
Since
w e deduce from ( 4 ) t h a t
I f w e now assume, i n a d d i t i o n , t h a t {ul,
t h e n ul
...,un}
i s orthonormal,
ul = 1, whereupon ( 5 ) y i e l d s
What e q u a t i o n ( 6 ) c o r r o b o r a t e s i s o u r u s u a l p r o p e r t y of
3-dimensional orthonormal c o o r d i n a t e systems. Namely, i f
Iu1,...,un}
i s orthonormal and i f v i s any l i n e a r combination
of u l , . . . ,
and un; t h e n w e f i n d t h e ui-component of v merely
by d o t t i n g v w i t h ui.
b.
I n t h i s p a r t of t h e e x e r c i s e we a r e t r y i n g t o show how t h e
concept o f an o r t h o g o n a l complement e x t e n d s t h e i d e a o f p e r p e n d i c u l a r i t y a s s t u d i e d i n t h e lower dimensional c a s e s .
For example,
i n 2-space w e know t h a t t h e 1-dimensional subspaces a r e l i n e s .
Any two n o n - p a r a l l e l l i n e s span t h e p l a n e , b u t w e can always
p i c k o u r l i n e s t o be p e r p e n d i c u l a r .
I n a s i m i l a r way, given a
p l a n e W i n 3-space t h e n any v e c t o r i n 3-space i s t h e sum o f
two v e c t o r s , one of which l i e s i n W and t h e o t h e r o f which
doesn't.
We can always choose t h e second v e c t o r t o be p e r p e n d i -
c u l a r t o W.
Our p o i n t i s t h a t t h i s i d e a e x t e n d s t o any v e c t o r s p a c e on which
a d o t product is defined.
The b a s i c i d e a i s a s f o l l o w s :
Solutions
Block 3: S e l e c t e d Topics i n L i n e a r Algebra
Unit 7: The Dot ( i n n e r ) P r o d u c t
3.7.9
continued
(i) Suppose t h a t W i s any subspace o f V.
s e t W by
P
W
P
We t h e n d e f i n e a new
= { V E V : ~ w = 0 f o r each we^)
i s a s u b s e t of V, b u t a s w e
P
ask you t o show i n t h i s p a r t of t h e e x e r c i s e , W i s a l s o a
P
To show t h i s , a l l w e have t o do i s prove t h a t t h e
subspace o f V.
C l e a r l y , by i t s v e r y d e f i n i t i o n , W
and t h a t e v e r y
sum of two m e m b e r s o f W i s a l s o a member o f W
PI
P
s c a l a r m u l t i p l e o f an element o f W belongs t o W
P
P
Computationally, t h i s i s c e r t a i n l y e a s y enough t o do. Namely:
.
v , v EW +
1 2 P
VEW
P
v
w = v2
1
w
+
v2
w = 0 f o r each weW
.w
= 0 f o r e a c h WEW
+
v1
+
(vl
+
+
v
w = 0 f o r each WEW
+
c(v
+
(cv)
v2)
W)
w = 0
f o r each WEW
= 0 f o r each wsW, where c i s any s c a l a r
w = 0 f o r each WEW
(ii) I f veW t h e n v
w = 0 f o r each weW. I n p a r t i c u l a r , t h e n ,
P
i f v a l s o b e l o n g s t o W, t h e n by v i r t u e o f i t b e i n g a member o f
.
v
v = 0. S i n c e o u r d o t p r o d u c t i s , by d e f i n i t i o n , p o s i t i v e
P
definite, v
v = 0 + v = 0. Hence, veWnW + v = 0, o r
P
W
L-
@
Wp.
That i s , each veV may
(iii) W e now show t h a t V = W
b e e x p r e s s e d i n one and o n l y one way a s a sum o f two v e c t o r s ,
one o f which i s i n W and t h e o t h e r o f which i s p e r p e n d i c u l a r t o
W (i.e.,
in W ).
P
Solutions
Block 3 : S e l e c t e d Topics i n L i n e a r Algebra
Unit 7: The Dot ( i n n e r ) Product
3.7.9
continued
= (01 , it i s s u f f i c i e n t t o show t h a t each VEV may be
P
w r i t t e n i n a t l e a s t one way i n t h e r e q u i r e d form.*
Since W n W
W e i n t r o d u c e t h e Gram Schmidt process t o s o l v e our problem.
Namely, we begin by c o n s t r u c t i n g an orthonormal b a s i s f o r W , say
{wl,...,w r 1. We may now use t h e Gram-Schmidt Orthogonalization
Process t o augment t h i s b a s i s t o become a b a s i s f o r V.
That i s
v
= [wl,
...,
vl,...
Wr.
IV
n-r
I
I
where dim V = n.
..
and
= [vl,.
,v
I where v
1'""
P
n-r
To prove t h i s a s s e r t i o n we f i r s t n o t i c e
The key l i e s i n t h e f a c t t h a t W
v
a r e a s given by ( 8 ) .
n-r
t h a t it i s t r i v i a l t o show t h a t vl,
.... and vn-r
a l l belong t o W
P'
Namely each of t h e s e v e c t o r s i s by c o n s t r u c t i o n orthogonal t o each
of t h e v e c t o r s wl..
, and wr; hence t o t h e space spanned by
..
wl,....
and wr.
A s a computational review, we have t h a t f o r each
wEW,
Hence, vi
E
W
P'
* R e c a l l t h a t i f W1.nW2 = ( 0 ) a n d wl + w = w ' + w ' , w h e r e
wlYw1'~W1
a n d w2,
* E W ~ . then w = w "and
l w = 2w2'.
Namely
2
+
w2 = w '
w '2+
w - w ' =lw '
w '&W2
w
Hence,
1
1
1
2
since w ' - l w
i s i n w2?
S i n c e w12- w ' 2 ~w e h a v e w h a t wl - w 1I =
2
W1 n w Z 2 ; a n d s i n c e W
=
0
1 w 1J = 0 f r o m w h i c h
W1
A s i m i l a r a r g u m e n t s h o w s t h a t w2 = w 2 ' .
we c o n c l u d e t h a t w =lwl
.
+
y2
1 .
-
Solutions
Block 3: S e l e c t e d T o p i c s i n L i n e a r A l g e b r a
U n i t 7 : The Dot ( i n n e r ) P r o d u c t
3 . 7 . 9 continued
Conversely i f w
(8) t h a t
i s any e l e m e n t of W
P
P
t h e n s i n c e W c V , w e see from
P
w = 0 f o r e a c h WEW, w e know t h a t i n p a r t i c u l a r
= w
w, = 0. Thus, f o r example, i f w e d o t b o t h
P
W~ ' W1
s i d e s o f ( 9 ) w i t h wl w e o b t a i n
Now, s i n c e w
-
P
. w1 =
-
P
...
( C W
11
and s i n c e {wl.
+
... + crwr
....wr,
from (10) t h a t 0 = cr
0 , s o t h a t by ( 9 ) ,
+
alvl+
... + an - r vn-r
vn - r 1 i s o r t h o n o r m a l , w e c o n c l u d e
S i m i l a r l y , w e may show t h a t c 2 =
-- Cr --
vl...,
and t h i s shows t h a t Cvl,
...
...,vn - r 1
span W
proof t h a t
3.7.10
P
.
T h i s completes o u r
(optional)
W e have
u4
-
u1 = 1, u 4
. u2 = 2,
u4
u3 = 1,
U4
u 4 = 9 J
W e f i r s t let
wl
Solutions
Block 3: S e l e c t e d Topics i n L i n e a r Algebra
U n i t 7: The Dot ( i n n e r ) Product
3.7.10
continued
Then by t h e Gram-Schmidt p r o c e s s w e have
o r from (1),
[Check:
u
*
u
1
2
*
=
= 1
1
;?. ul) = u1
(u2
1
;?.(4)
= 1
1 = 0.1
-
u1
-
. u2
1
- 7
u,
-
With ul* a s i n ( 2 ) and u2* a s i n ( 3 ) ' w e n e x t d e f i n e u3* by
u3* = u
-
' U *
l
ul*
ul*. ul*
U
3
-
u
3
- u *
u2*. u2 *
u*.
Solutions
Block 3: S e l e c t e d T o p i c s i n L i n e a r A l g e b r a
U n i t 7: The Dot ( i n n e r ) P r o d u c t
3.7.10
continued
~ u t t i n tgh~e r e s u l t s o f (5), ( 6 )
,
( 7 ) and ( 8 ) i n t o ( 4 ) y i e l d s
Finally,
27
4
u2*
u2* [by ( 8 ) ] =
u4
u3* = [by ( 9 ) 1 u4
= u4 u 3 - -
11
-27 u1 -
(u3
11
2 7 u4
'
u1 -
11
= 1 - 3 (1) - l
o (2)
27
10
27
u2)
10
27
u4 '
2
Solutions
Block 3: S e l e c t e d Topics i n L i n e a r Algebra
U n i t 7: The Dot ( i n n e r ) P r o d u c t
3.7.10
continued
P u t t i n g t h e s e r e s u l t s i n t o (10) y i e l d s
Summarizing, w e have from ( 2 1 , (31, (91, and (11)
u * = -ul
4
-
u
2
+
2u3
+
u,,
Solutions
Block 3: S e l e c t e d Topics i n Linear Algebra
Unit 7: The Dot (inner) Product
3.7.10 continued
Note : I f we w r i t e (12) i n matrix form w e have Matrix (13) t e l l s us t h a t
Solutions
Block 3: S e l e c t e d Topics i n L i n e a r Algebra
U n i t 7: The Dot ( i n n e r ) Product
3.7.10
Hence,
continued
ul*,
u 2 * , u3*, u4*
i s an o r t h o g o n a l b a s i s f o r V; and
T h i s i s much n e a t e r t h a n t h e corresponding r e s u l t u s i n g
u , u , u , and u4 as a b a s i s - i n which c a s e w e would a l s o have
1 2 3
had t o worry a b o u t t h e terms i n v o l v i n g x1y2, x y , x1y4, x2y1,
1 3
X2Y3f
X 2 Y 4 r X3Y1r
X 3 Y 2 t X3Y4'
X4Y1, X4Y28 and X4Y3-
MIT OpenCourseWare
http://ocw.mit.edu
Resource: Calculus Revisited: Complex Variables, Differential Equations, and Linear Algebra
Prof. Herbert Gross
The following may not correspond to a particular course on MIT OpenCourseWare, but has been
provided by the author as an individual learning resource.
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.