18.103 Fall 2013
Orthonormal Bases
Consider an inner product space V with inner product hf, gi and norm
kf k2 = hf, f i
Proposition 1 (Continuity) If kun − uk → 0 and kvn − vk → 0 as n → ∞, then
kun k → kuk;
hun , vn i → hu, vi.
Proof. Note first that since kvn − vk → 0,
kvn k ≤ kvn − vk + kvk ≤ M < ∞
for a constant M independent of n. Therefore, as n → ∞,
|hun , vn i − hu, vi| = |hun − u, vn i + hu, vn − vi| ≤ M kun − uk + kukkvn − vk → 0
In particular, if un = vn , then kun k2 = hun , un i → hu, ui = kuk2 . For u and v in V we say that u is perpendicular to v and write u ⊥ v if hu, vi = 0. The
Pythogorean theorem says that if u ⊥ v, then
ku + vk2 = kuk2 + kvk2
(1)
Definition 1 ϕn is called an orthonormal sequence, n = 1, 2, . . . , if hϕn , ϕm i = 0 for n 6= m
and hϕn , ϕn i = kϕn k2 = 1.
Suppose that ϕn is an orthonormal sequence in an inner product space V . The following
four consequences of the Pythagorean theorem (1) were proved in class (and are also in the
text):
If h =
N
X
an ϕn , then
n=1
2
khk =
N
X
1
1
|an |2 .
(2)
If f ∈ V and sN =
N
X
hf, ϕn iϕn , then
n=1
kf k2 = kf − sN k2 + ksN k2
(3)
If VN = span {ϕ1 , ϕ2 , . . . , ϕN }, then
kf − sN k = min kf − gk
g∈VN
If cn = hf, ϕn i, then
2
kf k ≥
∞
X
|cn |2
(best approximation property)
(4)
(Bessel’s inequality).
(5)
n=1
Definition 2 A Hilbert space is defined as a complete inner product space (under the distance d(u, v) = ku − vk).
Theorem 1 Suppose that ϕn is an orthonormal sequence in a Hilbert space H. Let
VN = span {ϕ1 , ϕ2 , . . . , ϕN },
V =
∞
[
VN
N =1
(V is the vector space of finite linear combinations of ϕn .) The following are equivalent.
a) V is dense in H (with respect to the distance d(f, g) = kf − gk),
b) If f ∈ H and hf, ϕn i = 0 for all n, then f = 0.
c) If f ∈ H and sN =
N
X
hf, ϕn iϕn , then ksN − f k → 0 as N → ∞.
n=1
d) If f ∈ H, then
2
kf k =
∞
X
|hf, ϕn i|2
n=1
If the properties of the theorem hold, then {ϕn }∞
n=1 is called an orthonormal basis or complete
orthonormal system for H. (Note that the word “complete” used here does not mean the
same thing as completeness of a metric space.)
Proof. (a) =⇒ (b). Let f satisfy hf, ϕn i = 0, then by taking finite linear combinations,
hf, vi = 0 for all v ∈ V . Choose a sequence vj ∈ V so that kvj − f k → 0 as j → ∞. Then
by Proposition 1 above
0 = hf, vj i → hf, f i =⇒ kf k2 = 0 =⇒ f = 0
2
(b) =⇒ (c). Let f ∈ H and denote cn = hf, ϕn i, sN =
N
X
cn ϕn . By Bessel’s inequality
1
(5),
∞
X
|cn |2 ≤ kf k2 < ∞.
1
Hence, for M < N (using (2))
2
ksN − sM k =
N
X
2
cn ϕ n
=
N
X
|cn |2 → 0 as M, N → ∞.
M +1
M +1
In other words, sN is a Cauchy sequence in H. By completeness of H, there is u ∈ H such
that ksN − uk → 0 as N → ∞. Moreover,
hf − sN , ϕn i = 0 for all N ≥ n.
Taking the limit as N → ∞ with n fixed yields
hf − u, ϕn i = 0 for all n.
Therefore by (b), f − u = 0.
(c) =⇒ (d). Using (3) and (2),
2
2
2
2
kf k = kf − sN k + ksN k = kf − sN k +
N
X
|cn |2 ,
(cn = hf, ϕn i)
1
Take the limit as N → ∞. By (c), kf − sN k2 → 0. Therefore,
2
kf k =
∞
X
|cn |2
1
Finally, for (d) =⇒ (a),
2
2
kf k = kf − sN k +
N
X
|cn |2
1
Take the limit as N → ∞, then by (d) the rightmost term tends to kf k2 so that kf −sN k2 →
0. Since sN ∈ VN ⊂ V , V is dense in H. 3
Proposition 2 Let ϕn be an orthonormal sequence in a Hilbert space H, and
X
X
|an |2 < ∞,
|bn |2 < ∞
then
u=
∞
X
a n ϕn ,
v=
n=1
∞
X
b n ϕn
n=1
are convergent series in H norm and
hu, vi =
∞
X
an bn
(6)
n=1
Proof. Let
uN =
N
X
a n ϕn ;
vN =
N
X
1
bn ϕ n .
1
Then for M < N ,
2
kuN − uM k =
N
X
|an |2 → 0 as M → ∞
M
so that uN is a Cauchy sequence converging to some u ∈ H. Similarly, vN → v in H norm.
Finally,
N
N
N
X
X
X
huN , vN i =
haj ϕj , bk ϕk i =
aj bk hϕj , ϕk i =
aj bj
j,k=1
j,k=1
j=1
since hϕj , fk i = 0 for j 6= k and hfj , fj i = 1. Taking the limit as N → ∞ and using the
continuity property (1), huN , vN i → hu, vi, gives (6). If H is a Hilbert space and {ϕn }∞
n=1 is an orthonormal basis, then every element can be
written
∞
X
f=
an ϕn (series converges in norm)
n=1
The mapping
{an } 7→
X
a n ϕn
n
is a linear isometry from `2 (N) to H that preserves the inner product. The inverse mapping
is
f 7→ {an } = {hf, ϕn i}
4
It is also useful to know that as soon as a linear mapping between Hilbert spaces is an
isometry (preserves norms of vectors) it must also preserve the inner product. Indeed, the
inner product function (of two variables u and v) can be written as a function of the norm
function (of linear combinations of u and v). This is known as polarization:
Polarization Formula.
hu, vi = a1 ku + ivk2 + a2 ku + vk2 + a3 kuk2 + a4 kvk2
with
a1 = i/2,
a2 = 1/2,
a3 = −(1 + i)/2,
a4 = −(i + 1)/2
Proof.
ku + ivk2 = hu + iv, u + ivi
= kuk2 + hiv, ui + hu, ivi + kvk2
= kuk2 + i(hv, ui − hu, vi) + kvk2
Similarly,
ku + vk2 = kuk2 + (hv, ui + hu, vi) + kvk2
Multiplying the first equation by i and adding to the second, we find that
iku + ivk2 + ku + vk2 = (i + 1)kuk2 + 2hu, vi + (i + 1)kvk2
Solving for hu, vi yields (7). 5
(7)
MIT OpenCourseWare
http://ocw.mit.edu
18.103 Fourier Analysis
Fall 2013
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.