Chapter 5 Inner Product Spaces
5-1 Inner Products and Norms
Inner product <x,y>: V is a vector space over F, and <x,y> has the following characteristics:
(a) <ax+bz,y>=a<x,y>+b<z,y>, (b) <x,y>=<y,x>*, (c) <x,x> is positive if x≠0.
Eg. Let u=(u1,u2,u3) and v=(v1,v2,v3). Determine which of the following are inner products on R3.
(i) <u,v>=u12v12+u22v22+u32v32, (ii) <u,v>=2u1v1+u2v2+3u3v3, (iii) <u,v>=u1v1-u2v2+u3v3. [2003交大電
信所]
(Sol.) Only (ii) is the inner product. (i) does not fulfill (a), (iii) does not fulfill (c) if u1v1 +u3v3<u2v2.
Eg. The following calculations fulfill the definition of the inner product:
n
(a) x=(a1,a2,…,an), y=(b1,b2,…,bn), <x,y>= ai bi , (b) A,B Mnxn(F), <A,B>=tr(B*A),
i 1
(c) f, g V, <f,g>= f x g ( x)dx .
a
0
In Matlab language, we can use the following instructions to obtain the inner product and the outer
product of two vectors:
>>A=[4,-1,3]; B=[-2,5,2]; C=dot(A,B); D=cross(A,B)
A=
4
-1
3
B=
-2
5
2
-14
18
C=
-7
D=
-17
Norm ||.||: A real-valued function on V and satisfies the following conditions for all x, y V and c F.
(a) ||cx||=|c|.||x||, (b) ||x||≧0, and ||x||=0 if x=0, (c) ||x+y||≦||x||+||y||.
Eg. Show that ||-x||=||x||. [1993 台大電研]
(Proof) ||-x||=|-1|.||x||=||x||
Eg. The following calculations fulfill the definition of the norm:
(a) V=Mmxn(F), ||A||= max |Aij|, (b) V=C[0,a >0] , ||f||= max f (t ) , (c)
t[ 0 ,a ]
i, j
Euclidean norm of a vector: x V, ||x||=
V=Fn,
n
2
x a i
i 1
1
2
x, x
Theorem (a)| <x,y>|≦||x||.||y|| (Cauchy-Schwartz Inequality), (b) ||x+y||2 + ||x-y||2 =2||x||2 +2||y||2.
(Remark) By definition of the Euclidean norm.
In Matlab language, we can use the following instructions to obtain the Euclidean norm of a vector:
>> x=[3 4 5]; c=norm(x)
c=
7.0711
Orthogonal vectors x and y: x⊥y <x,y>=0.
Orthonormal vectors x and y: ||x||=||y||=1 and <x,y>=0 if x≠y.
Eg. Show that if {v1, v2, v3, …, vn} is an orthogonal set of non-zero vectors in an inner product
space, then v1, v2, v3, …, and vn are linearly independent. [2015 中央電研固態組、生醫電子組]
(Proof) Let y=
be a linear combination of v1, v2, v3, …, and vn.
Since {v1, v2, v3, …, vn} is an orthogonal set of non-zero vectors in an inner product space, we have <
vi, vj>=0 if j≠i but < vi, vi>≠0 in case of j=i.
Consider an inner product <y,vj>=<
, vj >=
< vi, vj >=ai< vi, vi>
Set y=0 and then we have ai=0 because < vi, vi>≠0 for each i. It implies that v1, v2, v3, …, and vn are
linearly independent.
m
Theorem V is an inner product space and S={x1,…, xm}. If y= a i xi , and then (a) aj=<y,xj>/<xj,xj>
i 1
for all j if S is orthogonal, and (b) aj=<y,xj> for all j if S is orthonormal.
m
m
y, x j
i 1
i 1
xj, xj
(Proof) (a) y , x j ai xi , x j ai xi , x j a j x j , x j a j
Eg. (8,-7)=8(1,0)-7(0,1) where 8=<(8,-7),(1,0)> and -7=<(8,-7),(0,1)>
Theorem A linear operator T: V→W, let β={x1,…,xm} and orthogonal set β’={y1,…,ym} are the bases
T ( x j ), y i
of V and W, respectively. Then the ij-entry Tij=
. If β’ is an orthonormal basis, then
yi , yi
Tij= T ( x j ), y i .
m
m
m
k 1
k 1
k 1
(Proof) ∵ T ( x j ) Tkj y k , ∴ T ( x j ), y i Tkj y k , y i Tkj y k , y i Tij y i , y i
Tij
T ( x j ), y i
yi , yi
Eg. For a linear operator T(a1,a2)=(a1-a2,a1,2a1+a2), let β={(1,2),(2,3)} be a basis for R2, and
β’={(1,0,0),(0,1,0),(0,0,1)} be an
orthonormal
basis
for R3, then we have
T(1,2)=(-1,1,4)=-1(1,0,0)+1(0,1,0)+4(0,0,1) and T(2,3)=(-1,2,7)=-1(1,0,0)+2(0,1,0)+7(0,0,1)
T
'
1
1
4
1
2
7
,
where
T11=-1=<T(1,2),(1,0,0)>,
T21=1=<T(1,2),(0,1,0)>,
T31=4=<T(1,2),(0,0,1)>, T12=-1<T(2,3),(1,0,0>, T22=2=<T(2,3),(0,1,0)>, and T32=7=<T(2,3),(0,0,1)>
5-2 Gram-Schmidt Orthogonalization Process
Gram-Schmidt Orthogonalization Process: Let S={y1,y2,y3,…,yn} be a linearly independent subset
y 2 , x1
of V and S’={x1,x2,x3,…,xn} be an orthogonal subset of V. Set x1=y1, x2= y2 x1 , x3=
x1 , x1
y3-
k 1 y , x
y3 , x1
y3 , x2
k
j
x j . Then Span(S)=Span(S’).
x1 x2 , …, and xk=yk-
x1 , x1
x2 , x2
j 1 x j , x j
Eg. For V=R3, β={(1,1,0),(2,0,1),(2,2,1)}, find an orthogonal basis for V by the Gram-Schmidt
orthogonalization process. [2007 台科大電研]
(Sol.) Let x1 (1,1,0), x2 (2,0,1)
x3 (2,2,1)
(2,0,1), (1,1,0)
(1,1,0) (2,0,1) 22 (1,1,0) (1,1,1)
(1,1,0), (1,1,0)
(2,2,1), (1,1,0)
(2,2,1), (1,1,1)
(1,1,0)
(1,1,1) ( 13 , 13 , 23 ) .
(1,1,0), (1,1,0)
(1,1,1), (1,1,1)
Then β’={x1,x2,x3} is an orthogonal basis.
Eg. Let the vector space P2 have the inner product <p,q>=
1
1
p( x)q( x)dx . Apply the
Gram-Schmidt process to transform the standard basis S={1,x,x2} into an orthonormal basis.
[2005 北科大電腦通訊所]
1
y 2 , x1
x 1dx 1 =x-0=x,
(Sol.) S={1,x,x2}={ y1, y2, y3}. Let x1= y1=1, x2= y2 x1 =x- 11
x1 , x1
1 1dx
1
1
1
x 2 1dx
x 2 xdx
y3 , x1
y3 , x2
1
1
1
1
x3= y3 x1 x2 =x2- 1
1 - 1
x =x2- -0= x2- .
3
3
x1 , x1
x2 , x2
1 1dx
x xdx
1
1
1
45 2 1
1
3
∴ Orthogonal basis: {1,x,x2- } Orthonormal basis: {
,
x,
(x - )}
3
3
8
2 2
Sperp: S⊥={x V: <x,y>=0 for all y S} is an orthogonal set of S.
Eg. For V=C3, S=Span{(1,0,i),(1,2,1)}, compute S⊥.
(Sol.) Suppose S⊥=Span{(a,b,c)},
a, b, c , 1,0, i 0 a ci
1 i
⊥
a, b, c , 1,2,1 0 b 1 i c , S = Span{( i, 2 ,1)}
2
Eg. Show that (a) {x1,x2,…,xk} is an ordered basis for W (subspace of V) and {x1,x2,…,xk,xk+1,…,xn}
is an ordered basis V, then {xk+1,…,xn} is an order basis of for W⊥.
(b) W is a subspace of V, then dim(V)=dim(W)+dim(W⊥). [台大電研]
n
(Proof) (a) x V x x, xi xi
i 1
If x W , then x, xi 0 for 1 i k . Therefore, x
n
x, x
i k 1
(b) According to (a), dim( V )=n=k+(n-k)=dim( W )+dim( W )
i
xi Spanx k 1 , , x n
Eg. Show that (a) W1 W2 W1 W2
and (b) W1 W2 W1 W2 . [1990, 1999 台大電
研]
(Proof) W1 W2 W1 W2 W1 W1 W2 W2 V
W1 and W2 W1 W2 , W1 W2 W1 and W2 W1 W2 W1 W2
x W1 W2 , then y W1 and z W2 y , x z, x 0
cy dz W1 W2 , cy dz, x c y, x d z, x 0
x W1 W2 W1 W2 (W1 W2 ) , ∴ W1 W2 W1 W2
On the other hand, U 1 U 2 U 1 U 2 U 1 U 2 U 1 U 2
Set U 1 W1 , U 2 W2 W1 W2 W1 W2
5-3 Various Matrices
Adjoint matrix: A* is the complex conjugate transpose of A. Note: A*≠adj(A) and (AB)*=B*A*.
1 i 2
1 i 3i
Eg. For A=
, then A*=
.
4
3i 4
2
Eg. For T: C2→C2 by T(a1,a2)=(2ia1+3a2,a1-a2), if β is the standard ordered basis, find T*(a1,a2).
[1998 台大電研]
(Sol.) T 1,0 2i,1 2i (1,0) 1 (0,1) and T 0,1 3,1 3 (1,0) (1) (0,1)
2i 3
2i 1
T
T*
, ∴ T*(a1,a2)=( -2ia1+a2,3a1-a2).
1 1
3 1
Theorem For A Mm×n(F), x Fn, y Fm, then <Ax,y>=<x,A*y>.
(Note: T x , y x, T * y for x, y V , T : V V )
Eg.
1 1 x u
x y u
x 1 2i u
x u 2iv
2i 3 y , v 2ix 3 y , v xu yu 2ixv 3 yv y , 1 3 v y , u 3v
Theorem (a) A Mm×n(F), Rank(A*A)=Rank(A). (b) A Mm×n(F), if Rank(A)=n, then A*A is
invertible.
(Proof) (a) A*Ax=0 Ax=0
1. " ": Ax 0 implies A* Ax 0
2. "": 0 A* Ax, x
n
Ax, A** x
m
Ax, Ax m , ∴ Ax=0
(b) A M mn F A* A M nn F
Rank(A*A)=Rank(A)=n, ∴ A*A is invertible
Orthogonal matrix A: AAt=AtA=I. Unitary matrix A:. AA*=A*A=I.
1 / 2
Eg. A=
1 / 2
1/ 2 t 1/ 2 1/ 2
t
t
, A =
, AA =A A=I, ∴ A is orthogonal.
1/ 2
1
/
2
1
/
2
i / 2
Eg. A=
i / 2
i / 2
i / 2
, A*=
i/ 2
i/ 2
i / 2
, AA*=A*A=I, ∴ A is unitary.
i / 2
Normal Matrix A: AA*=A*A.
i i
i i
Eg. A=
, A*=
, AA*=A*A, ∴ A is normal.
i i
i i
Eg. T is a normal operator on V. Show that (a) ||T(x)||= ||T*(x)||, (b) T(x)=λx T * x x , (c) λ1
and λ2 are distinct eigenvalues of T with corresponding eigenvector x1 and x2. Then x1 and x2 are
orthogonal. [2012 台大電研]
(Proof) (a) T x T x , T x T *T x , x TT * x , x T * x , T * x T * x
2
2
(b) Suppose T x x for some x V . Let U T I ,
0 U x U * x (T * I )x , ∴ T * x x .
(c) Let T x1 1 x1 , T x2 2 x2 ,
1 x1 , x2 1 x1 , x2 T x1 , x2 x1 , T * x2 x1 , 2 x2 2 x1 , x2
1 2 , x1 , x2 0 .
Self-adjoint (Hermitian) matrix A: A=A*.
2 5i
1
Eg.
is a self-adjoint matrix.
2 5i 3
(Note: A self-adjoint matrix is also a normal matrix, but a normal matrix may not be self-adjoint)
Theorem If A is a self-adjoint matrix, all eigenvalues of A are real. [2000 台大電研]
(Proof) x Ax A* x x for some x≠0, λ is real.
Gramian matrix A: B Mm×n(F) such that A=BBt then A is called the Gramian matrix.
A is symmetric.
Theorem A is a Gramian matrix
all eigenvalues of A are nonnegative.
(Proof) (1) At=(BBt)t=BBt=A, ∴ A is symmetric. (2) For some x 0, and Ax=λx,
then
Ax, x x, x x, x BB t x, x B t x, B t x 0, x, x 0, 0
Eg. Which of the following matrices is Hermitian? Which is normal?
i
2 i
1
B 2
A
2 i 1
1
2
1
i
1
3 1 i i
0
2 C i
0
2 i D 1 i 1 3 [交大電子所]
i
i
1 2 i
3 1
0
2
(Sol.) A=A* and D=D*, ∴ A and D are both Hermitian matrices,
1 i
1 i
CC*=C*C, BB*=
, B*B=
, ∴ C is normal but B is not normal.
i 1
i 1
5-4 Special Characteristics of Matrices
Ax, x >0 for all x≠0, then A is positive definite.
Positive definite: If
Eg. If A is a positive-definite matrix, then all the eigenvalues of A are positive.
(Proof) For some x≠0 and Ax=λx, positive definite:
Ax, x x, x x, x 0 0 .
Eg. Let A be a complex normal (or real symmetric) n×n matrix with eigenvalues: λ1, λ2, …, λn,
n
n
i 1
i 1
2
n
and then show that (a) tr(A)= i and tr(A*A)= i , (b) det(A)= i . [1998 台大電研]
i 1
n
(Proof) Let A=QDQ-1 and D be diagonal, tr(A)=tr(QDQ-1)=tr(DQQ-1)=tr(D)= i
i 1
n
det A det QDQ 1 det Q det D det Q 1 det Q det D det Q 1 det D i .
i 1
2
1
Eg. Let λi be the eigenvalues of A=
- 1
0
1 -1 0
3 4 2
, 1≦i≦4, (a)
4 1 2
2 21
4
i
i 1
? (b)
4
i
?
i 1
( c ) I s A p o s i t i v e definite? [台大機研]
4
(So1.) ∵ A is a real symmetric matrix, ∴ (a)
i tr(A) = 2+3+1+1=7, (b)
i 1
4
=det(A) = 10,
i
i 1
(c) Method 1: λ<0 A is not positive definite.
2 1 -1
2 1
Method 2: det([2])=2>0 and det(
)=5>0, but det( 1 3 4 )=-38<0 Not positive definite.
1
3
- 1 4 1
A is unitarily equivalent to B: P fulfills B=P*AP and P-1=P*.
Theorem (a) A is complex normal A is unitary equivalent to a diagonal matrix. (b) A is real
symmetric A is orthogonally equivalent to a real diagonal matrix.
(Proof) AA* PDP * PDP *
*
PDP * PD * P * PDD * P *
A PDP * , A* A PDP * PDP * PD * DP * . DD * D * D, A* A AA*
*
Projection: V=W1⊕W2, x1 W1, and x2 W2, x=x1+x2. If T(x)=x1, then T is projection on W1. That is,
R(T)=W1={x: T(x)=x} and N(T)= W2. (Note: T2=T if T is a projection.)
Orthogonal Projection: If R(T)⊥=N(T) and N(T)⊥= R(T) for T: V→V be a projection.
Theorem T is a linear operator on V. Then T is an orthogonal projection. T2=T=T*.
Eg. Let T: R2→R2 be a linear operator. For any z R2, T(z)=p, where p is the projection of z on the
line x=y. Find [T]. [交大電信所]
1
2 } and the standard basis of R2 is { 1 , 0 }.
(Sol.) The orthonormal basis of line x=y is {
0 1
1
2
1
1
2 >= 1 , so the orthogonal projection of 1 onto the line x=y is
Inner product: < ,
0
2
0 1
2
1 1
1
2 = 2 = 1
1
1 2
2
2 2
Inner
1
2
1 1
0 + 2
0
1
1
0
2 >= 1 , so the orthogonal projection of 0 onto the line x=y is
product: < ,
1
2
1 1
2
1 1
1
1
2 = 2 = 1 1 + 1 0 , ∴ [T]= 2
2 , and we have [T]2=[T]=[T]*
1
1
1 1 2 0 2 1
2
2
2 2
1
0 0
Eg. Find the orthogonal projection of the vector v= 0 onto the subspace S=Span( 1 , 1 ).
2
1 1
[交大電子所]
0 0
0 0
(Sol.) Transform the orthogonal basis { 1 , 1 } into the orthonormal one: { 1 , 1 }.
2
2
1
1
1 1
2
2
1
< 0
2
0
1
0
1
,
>=-√2, so the orthogonal projection of the vector v= 0
onto 1 is
2
1
2
1
2
0 0
-√2 1 = 1 .
2
1
1
2
1 0
1
0
1
< 0 ,
>=-√2, so the orthogonal projection of the vector v= 0
onto 1 is
2
2 1
2
1
2
0 0
-√2 1 = 1 .
2
1 1
2
0
0 1 0 0 1
1
1 1 0 0
1
1
∵ 0 -(-√2
)-(-√2
)= 0 - 1 - 1 = 0 , ∴ v= 0 = 0 + 1 + 1
2
2
1
1 2 1 1 0
2
2 0 1 1
2
2
1
0 0
and { 0 }⊥S=Span( 1 , 1 )
0
1 1
Rotation: orthogonal basis β={x1,x2} for W, and θ is real and fulfills T(x1)=x1cosθ+x2sinθ,
T(x2)=-x1sinθ+x2cosθ, and T(y)=y, y W⊥, where W is a 2-dimensional subspace. And then T is a
⊥
⊥
rotation of W about W , where W is the axis of rotation.
Reflection: T(x)=-x, x W and T(y)=y, y W⊥. And then T is a reflection of W about W⊥, where
W is a 1-dimensional subspace.
Eg. T: R2→R2 by T(a,b) = (-a,b). Let W=Span({e1}) and W⊥=Span({e2}). It is reflection of R2 about
W⊥ (the y-axis).
Theorem T is an orthogonal operator on a 2-dimensional real inner product space. If det(T)=1,
then T is a rotation. If det(T)=-1, then T is a reflection.
cos sin
(Proof) By definition, T
is a rotation, ∴ det(T) = 1.
sin cos
Choose z1 , z 2 ,... be an orthogonal basis for V and z1 W,
T
1 0 0
0 1
det(T) = -1.
0
0 0 1
Theorem The composition of a reflection and a rotation is a reflection.
(Proof) det(T1T2)=det(T1)det(T2)= -1
normal on V over F C
Spectral Theorem If T is
, and λ1, …, λk are distinct eigenvalues of
self - adjoint on V over F R
T. Let Wi be the eigenspace of T corresponding to λi, and Ti be an orthogonal projection on Wi.
(Ti(x)=xi Wi). Then (a) V=W1⊕W2⊕W3…⊕Wk, (b) Wi⊥= W j (direct sum), (c) TiTj=δijTi, (d)
ji
I=T1+T2+…+Tk, (e) T=λ1T1+λ2T2+…+λkTk.
0, i j
(Proof) (c) Let x 0 and T j ( x) x j , then Ti Tj ( x) Ti ( x j )
i j Ti ( x) TiT j i j Ti
xi , i j
(d) x Ix x1 ... xk T1 ( x) T2 ( x) ... Tk ( x) (T1 T2 ... Tk )( x) , ∴ I T1 T2 ... Tk .
(e) For x V , then x x1 ... xk , where xi Wi ,
T ( x) T ( x1 ) ... T ( xk ) 1 x1 ... k xk 1T1 ( x) ... k Tk ( x) (1T1 ... k Tk )( x) ,
∴ T 1T1 ... k Tk
Theorem (a) F=C, T is a unitary operator T* is normal and |λ|=1 for all eigenvalues. (b)
T=-T* each λ is pure imaginary. (c) T is a projection each λ is either 0 or 1.
5-5 Bilinear Forms
Bilinear form, H: If (a) H(ax1+bx2,y)=aH(x1,y)+bH(x2,y) and (b) H(x,ay1+by2)=aH(x,y1)+bH(x,y2) for
x1, y1, x2, y2, x, y V, a, b F.
Theorem For each H B(Fn), ! A Mn×n(F) fulfills H(x,y)=xtAy for all x, y Fn.
a b
a b
Eg. Define H : R 2 R 2 R by H ( 1 , 1 ) 2a1b1 3a1b2 4a 2 b1 a 2 b2 for all 1 , 1 R 2 ,
a2 b2
a2 b2
a1
b1
2 3
t
then let x , y , A
H ( x, y ) x Ay [a1 a 2 ]
a
b
4
1
2
2
2 3
4 1
Symmetric bilinear form: H(x,y)=H(y,x) for all x, y V.
Quadratic form: K: V→F if H B(V) fulfills K(x)=H(x,x)=xtAx.
Eg. Let f(t1,t2,t3)= 2t1 t 2
2
f (t1 , t 2 , t 3 ) t1 t 2
2
t3
0
2 3
t1
6t1t 2 4t 2 t 3 K ( t 2 ) , and then A= 3 1 2
t 3
0 2 0
0 t 1
2 3
3 1 2 t is a quadratic form.
2
0 2 0 t 3
b1
b .
2
Eg. There are two bases (x1,y1) and (x2,y2) in R2. If the equations of the same ellipse represented
by two distinct bases are described as follows, respectively: 2x12-4x1y1+5y12-36=0 and
x22+6y22-36=0. Please find the transformation matrix between these two bases. [2004 台大電研]
x
2
(Sol.) Let K 1 2 x1 2 5 y1 2 4 x1 y1 , A
2
y1
2 a1
a
2
0 1 a , a 0,
4 a2
1
a2
a
1
2
b
4 2 b
1 1 A 1I 1
a
2
2 2
2
, det
2 5 1 6, 1, 6
5
b
1
2 6 A 2 I 1
1 0 1 b , b 0
2
b2 2 1 b2
b2
Q
2
5
1
5
x
∴ 1
y
1
1
5 fulfills Q 1 AQ Q t AQ 1 0 for 1 2, 1 1 .
0 6
2
5 1 5 2
5
2
5
1
5
1
5 x 2 K x2 x 2 6y 2 2 x1 2 4 x1 y1 5 y1 2 36 0 x 2 2 6 y 2 2 36 0
2
2
y
2 y 2
2
5
Eg. [t1,t2,t3]t R3, 3t12+3t22+3t32-2t1t3+2√2 (t1+t3)+1=0. Find a basis β for R3 such that the above
equation is simplified.
t1
3 0 1
2
2
2
(Sol.) Let K t 2 3t1 3t 2 3t 3 2t1t 3 , then A 0 3 0 . It is found that an orthogonal
t 3
1 0 3
0
matrix Q 1
0
1
0
3 0 0
1
1
2
1 1
t
0 fulfills Q AQ 0 4 0 for 1 ,
0 ,
0 .
2 2
1
0
0 0 2
1
1
2
1
2
0
1
2
t1 0
∴ t 2 1
t 3 0
1
2
0
1
2
1
2 s1
0 s 2 K ( x) 3s12 4s2 2 2s3 2
1
s3
2
3t1 3t 2 3t 3 2t1t3 2 2 (t1 t 3 ) 1 0 3s1 4s2 2s3 4s3 1 0 .
2
2
2
2
2
2
0
You can add this document to your study collection(s)
Sign in Available only to authorized usersYou can add this document to your saved list
Sign in Available only to authorized users(For complaints, use another form )