Lecture 20: 6.1
Inner Products
Wei-Ta Chu
2011/12/5
Definition
An inner product on a real vector space V is a function
that associates a real number ⟨u, v⟩ with each pair of
vectors u and v in V in such a way that the following
axioms are satisfied for all vectors u, v, and w in V and all
scalars k.
⟨u, v⟩ = ⟨v, u⟩
⟨u + v, w⟩ = ⟨u, w⟩ + ⟨v, w⟩
⟨ku, v⟩ = k ⟨u, v⟩
⟨u, u⟩ ≥ 0 and ⟨u, u⟩ = 0 if and only if u = 0
A real vector space with an inner product is called a real
inner product space.
2
Euclidean Inner Product on Rn
If u = (u1, u2, …, un) and v = (v1, v2, …, vn) are vectors in
Rn, then the formula
⟨v, u⟩ = u · v = u1v1 + u2v2 + … + unvn
defines ⟨v, u⟩ to be the Euclidean inner product on Rn.
Inner products can be used to define norm and distance in
a general inner product space.
3
Theorem 6.1.1
If u and v are vectors in a real inner product space V, and if k
is a scalar, then
||v|| ≧ 0 with equality if and only if v = 0
||kv|| = |k| ||v||
d(u,v) = d(v,u)
d(u,v) ≧ 0 with equality if and only if u = v
4
Euclidean Inner Product vs. Inner
Product
The Euclidean inner product is the most important inner
product on Rn. However, there are various applications in
which it is desirable to modify the Euclidean inner
product by weighting its terms differently.
More precisely, if w1,w2,…,wn are positive real numbers,
and if u=(u1,u2,…,un) and v=(v1,v2,…,vn) are vectors in
Rn, then it can be shown
which is called weighted Euclidean inner product with
weights w1,w2,…,wn …
5
Weighted Euclidean Inner Product
Let u = (u1, u2) and v = (v1, v2) be vectors in R2. Verify that the
weighted Euclidean inner product ⟨u, v⟩ = 3u1v1 + 2u2v2
satisfies the four product axioms.
Solution:
Note first that if u and v are interchanged in this equation, the right side
remains the same. Therefore, ⟨u, v⟩ = ⟨v, u⟩.
If w = (w1, w2), then
⟨u + v, w⟩ = 3(u1+v1)w1 + 2(u2+v2)w2 =
(3u1w1 + 2u2w2) + (3v1w1 + 2v2w2)= ⟨u, w⟩ + ⟨v, w⟩
which establishes the second axiom.
6
Weighted Euclidean Inner Product
⟨ku, v⟩ =3(ku1)v1 + 2(ku2)v2 = k(3u1v1 + 2u2v2) = k ⟨u, v⟩
which establishes the third axiom.
⟨v, v⟩ = 3v1v1+2v2v2 = 3v12 + 2v22 .Obviously , ⟨v, v⟩ = 3v12 +
2v22 ≥0 . Furthermore, ⟨v, v⟩ = 3v12 + 2v22 = 0 if and only if
v1 = v2 = 0, That is , if and only if v = (v1,v2)=0. Thus, the
fourth axiom is satisfied.
7
Applications of Weighted Euclidean
Inner Products
Suppose that some physical experiment can produce any
of n possible numerical values x1,x2,…,xn.
We perform m repetitions of the experiment and yield
these values with various frequencies; that is, x1 occurs f1
times, x2 occurs f2 times, and so forth. f1+f2+…+fn=m.
Thus, the arithmetic average, or mean, is
8
Applications of Weighted Euclidean
Inner Products
If we let f=(f1,f2,…,fn), x=(x1,x2,…,xn), w1=w2=…=wn=1/m
Then this equation can be expressed as the weighted inner
product
9
Weighted Euclidean Inner Product
The norm and distance depend on the inner product used.
If the inner product is changed, then the norms and distances
between vectors also change.
For example, for the vectors u = (1,0) and v = (0,1) in R2 with the
Euclidean inner product, we have
u = 1 and d (u, v ) = u − v = (1,−1) = 12 − (−1) 2 = 2
However, if we change to the weighted Euclidean inner product
⟨u, v⟩ = 3u1v1 + 2u2v2 , then we obtain
u = u, u
1/ 2
= (3 ⋅1 ⋅1 + 2 ⋅ 0 ⋅ 0)1/ 2 = 3
d (u, v ) = u − v = (1,−1), (1,−1)
1/ 2
= [3 ⋅1 ⋅1 + 2 ⋅ (−1) ⋅ (−1)]1/ 2 = 5
10
Definition
If V is an inner product space, then the norm (or length)
of a vector u in V is denoted by ||u|| and is defined by
||u|| = ⟨u, u⟩½
The distance between two points (vectors) u and v is
denoted by d(u,v) and is defined by
d(u, v) = ||u – v||
If a vector has norm 1, then we say that it is a unit vector.
11
Unit Circles and Spheres in Inner Product
Space
If V is an inner product space, then the set of points in V
that satisfy
||u|| = 1
is called the unite sphere or sometimes the unit circle in
V. In R2 and R3 these are the points that lie 1 unit away
form the origin.
12
Unit Circles in R2
Sketch the unit circle in an xy-coordinate system in R2 using the
Euclidean inner product ⟨u, v⟩ = u1v1 + u2v2
Sketch the unit circle in an xy-coordinate system in R2 using the
Euclidean inner product ⟨u, v⟩ = 1/9u1v1 + 1/4u2v2
Solution
If u = (x,y), then ||u|| = ⟨u, u⟩½ = (x2 + y2)½, so the equation of the unit
circle is x2 + y2 = 1.
If u = (x,y), then ||u|| = ⟨u, u⟩½ = (1/9x2 + 1/4y2)½, so the equation of the
unit circle is x2 /9 + y2/4 = 1.
13
Inner Products Generated by Matrices
 u1 
 v1 
u 
v 
Let u =  2  and v =  2  be vectors in Rn (expressed as n×1
:
:
 
 
u
 n 
vn 
matrices), and let A be an invertible n×n matrix.
If u · v is the Euclidean inner product on Rn, then the formula
⟨u, v⟩ = Au · Av
defines an inner product; it is called the inner product on Rn
generated by A.
Recalling that the Euclidean inner product u · v can be written
as the matrix product vTu, the above formula can be written in
the alternative form ⟨u, v⟩ = (Av) TAu, or equivalently,
⟨u, v⟩ = vTATAu
14
Inner Product Generated by the Identity
Matrix
The inner product on Rn generated by the n×n identity matrix
is the Euclidean inner product: Let A = I, we have ⟨u, v⟩ = Iu ·
Iv = u · v
The weighted Euclidean inner product ⟨u, v⟩ = 3u1v1 + 2u2v2 is
the inner product on R2 generated by
 3
A=
 0
0 

2 
since
 3
u , v = [v1 v2 ]
0
0  3

2  0
3 0 u1 
0  u1 
[
]
=
v
v
 
1
2 
 u  = 3u1v1 + 2u2 v2
u
0
2
2  2 

 2 
15
Inner Product Generated by the Identity
Matrix
In general, the weighted Euclidean inner product ⟨u,
v⟩ = w1u1v1 + w2u2v2 + … + wnunvn is the inner
product on Rn generated by
16
An Inner Product on M22
If U = 
u1
u3
u2 
 v1
and
V
=
v
u4 
 3
v2 
v4 
are any two 2×2 matrices, then
⟨U, V⟩ = tr(UTV) = tr(VTU) = u1v1 + u2v2 + u3v3 + u4v4
defines an inner product on M22
For example, if
1 2 
 −1 0 
U =
and V = 

3 4 
3
2 
then ⟨U, V⟩ = 1(-1)+2(0) + 3(3) + 4(2) = 16
The norm of a matrix U relative to this inner product is
U =< U , U >1/ 2 = u12 + u22 + u32 + u42
and the unit sphere in this space consists of all 2×2 matrices U
whose entries satisfy the equation ||U|| = 1, which on squaring yields
u12 + u22 + u32 + u42 = 1
17
An Inner Product on P2
If p = a0 + a1x + a2x2 and q = b0 + b1x + b2x2 are any two
vectors in P2, then the following formula defines an inner
product on P2:
⟨p, q⟩ = a0b0 + a1b1 + a2b2
The norm of the polynomial p relative to this inner
product is
1/ 2
2
2
2
p =< p, p > = a0 + a1 + a2
and the unit sphere in this space consists of all
polynomials p in P2 whose coefficients satisfy the
equation || p || = 1, which on squaring yields
a02+ a12 + a22 =1
18
Theorem 6.1.2 (Properties of Inner
Products)
If u, v, and w are vectors in a real inner product space,
and k is any scalar, then:
⟨0, v⟩ = ⟨v, 0⟩ = 0
⟨u, v + w⟩ = ⟨u, v⟩ + ⟨u, w⟩
⟨u, kv⟩ =k ⟨u, v⟩
⟨u – v, w⟩ = ⟨u, w⟩ – ⟨v, w⟩
⟨u, v – w⟩ = ⟨u, v⟩ – ⟨u, w⟩
19
Example
⟨u – 2v, 3u + 4v⟩
= ⟨u, 3u + 4v⟩ – ⟨ 2v, 3u+4v⟩
= ⟨u, 3u⟩ + ⟨u, 4v⟩ – ⟨2v, 3u⟩ – ⟨2v, 4v⟩
= 3 ⟨u, u⟩ + 4 ⟨u, v⟩ – 6 ⟨v, u⟩ – 8 ⟨v, v⟩
= 3 || u ||2 + 4 ⟨u, v⟩ – 6 ⟨u, v⟩ – 8 || v ||2
= 3 || u ||2 – 2 ⟨u, v⟩ – 8 || v ||2
20
Example
We are guaranteed without any further proof that the five
properties given in Theorem 6.1.2 are true for the inner
product on Rn generated by any matrix A.
⟨u, v + w⟩ = (v+w)TATAu
=(vT+wT)ATAu
[Property of transpose]
=(vTATAu) + (wTATAu)
[Property of matrix multiplication]
= ⟨u, v⟩+ ⟨u, w⟩
21
Lecture 20: 6.2
Angle and Orthogonality
Wei-Ta Chu
2011/12/5
Theorem 6.2.1
Theorem 6.2.1 (Cauchy-Schwarz Inequality)
If u and v are vectors in a real inner product space, then
|⟨u, v⟩| ≤ ||u|| ||v||
The inequality can be written in the following two forms
The Cauchy-Schwarz inequality for Rn follows as a special
case of this theorem by taking ⟨u, v⟩ to be the Euclidean
inner product u‧v.
23
Angle Between Vectors
Cauchy-Schwarz inequality can be used to define angles
in general inner product cases.
We define
to be the angle between u and v.
24
Example
Let R4 have the Euclidean inner product. Find the
cosine of the angle θ between the vectors u = (4, 3, 1,
-2) and v = (-2, 1, 2, 3).
25
Theorem 6.2.2
Theorem 6.2.2 (Properties of Length)
If u, v and w are vectors in an inner product space V, and if
k is any scalar, then :
|| u + v || ≤ || u || + || v || (Triangle inequality for vectors)
d(u,v)
d(
, ) ≤ d(
d(u,w)
, ) + d(
d(w,v)
, ) (Triangle inequality for distances)
Proof of (a)
(Property of absolute value)
(Theorem 6.2.1)
26
Orthogonality
Definition
Two vectors u and v in an inner product space are called
orthogonal if ⟨u, v⟩ = 0.
Example (⟨U, V⟩ = tr(UTV) = tr(VTU) = u1v1 + u2v2 + u3v3 + u4v4)
If M22 has the inner project defined previously, then the matrices
1 0
0 2 
U =
and V = 


1
1
0
0




are orthogonal, since ⟨U, V⟩ = 1(0) + 0(2) + 1(0) + 1(0) = 0.
27
Orthogonal Vectors in P2
1
Let P2 have the inner product < p, q >= ∫ p( x)q( x)dx and let p = x and
−1
q = x2.
Then
1

1/ 2
p =< p, p > =  ∫ xxdx 
 −1

1/ 2
1 2 2 
1/ 2
q =< q, q > =  ∫ x x dx 
 −1

1
< p, q >=
1/ 2
1 2 
=  ∫ x dx 
 −1

1/ 2
1 4 
=  ∫ x dx 
 −1

2
3
=
1/ 2
=
2
5
1
3
xx
dx
=
x
∫
∫ dx =0
2
−1
−1
because ⟨p, q⟩ = 0, the vectors p = x and q = x 2 are orthogonal
relative to the given inner product.
28
Theorem 6.2.3 (Generalized Theorem of
Pythagoras)
If u and v are orthogonal vectors in an inner product
space, then
|| u + v ||2 = || u ||2 + || v ||2
Proof
29
Orthogonality
Definition
Let W be a subspace of an inner product space V. A vector u
in V is said to be orthogonal to W if it is orthogonal to every
vector in W, and the set of all vectors in V that are
orthogonal to W is called the orthogonal complement (正交
補餘)) of W.
If V is a plane through the origin of R3 with Euclidean inner
product, then the set of all vectors that are orthogonal to
every vector in V forms the line L through the origin that is
perpendicular to V.
L
V
30
Theorem 6.2.4
Theorem 6.2.4 (Properties of Orthogonal Complements)
If W is a subspace of a finite-dimensional inner product
space V, then:
W⊥ is a subspace of V. (read “W perp”)
The only vector common to W and W⊥ is 0;; that is ,W ∩ W⊥ = 0..
31
Proof of Theorem 6.2.4(a)
Note first that ⟨0, w⟩=0 for every vector w in W, so W ⊥
contains at least the zero vector.
We want to show that the sum of two vectors in W ⊥ is
orthogonal to every vector in W (closed under addition)
and that any scalar multiple of a vector in W ⊥ is
orthogonal to every vector in W (closed under scalar
multiplication).
32
Proof of Theorem 6.2.4(a)
Let u and v be any vector in W ⊥, let k be any scalar, and
let w be any vector in W. Then from the definition of W ⊥,
we have ⟨u, w⟩=0 and ⟨v, w⟩=0.
Using the basic properties of the inner product, we have
⟨u+v, w⟩ = ⟨u, w⟩ + ⟨v, w⟩ = 0 + 0 = 0
⟨ku, w⟩ = k⟨u, w⟩ = k(0) = 0
Which proves that u+v and ku are in W ⊥.
33
Proof of Theorem 6.2.4(b)
The only vector common to W and W⊥ is 0; that is ,W ∩ W⊥ = 0.
If v is common to W and W ⊥, then ⟨v, v⟩=0, which
implies that v=0 by Axiom 4 for inner products.
34
Theorem 6.2.5
Theorem 6.2.5
If W is a subspace of a finite-dimensional inner
product space V, then the orthogonal complement
of W ⊥ is W; that is
(W⊥)⊥ = W
35
Example (Basis for an Orthogonal
Complement)
Let W be the subspace of R6 spanned
by the vectors w1=(1, 3, -2, 0, 2, 0),
w2=(2, 6, -5, -2, 4, -3), w3=(0, 0, 5,
10, 0, 15), w4=(2, 6, 0, 8, 4, 18). Find
a basis for the orthogonal complement
of W.
Solution
Since the row space and null space of A
are orthogonal complements, this
problem reduces to finding a basis for
the null space of A.
The space W spanned by w1, w2, w3,
and w4 is the same as the row space of
the matrix
form a basis for this null space.
36