Best Approximation: Minimax Theory
Let a function f ∈ C[a, b] where C[a, b] is a Banach space if we define the norm to be
||f || = max |f (x)|
a≤x≤b
An important problem is to approxmate f as well as possible by an element of Πn , where Πn
is the space of all polynomials of degree ≤ n. We define
E = inf ||f − p||
p∈Πn
and ask whether best approximation exists i.e. if there exists and element p∗n ∈ Πn such that
E = ||f − p∗n ||. Now this can also be written as
max |f (x) − p∗n (x)| = min max |f (x) − pn (x)|
pn ∈Πn a≤x≤b
a≤x≤b
Hence the best uniform approximation is also called a minimax approximation.
Theorem: (Existence) Let f is continuous in [a, b]. Then there exists p∗n ∈ Πn that minimizes
||f − pn || among all pn ∈ Πn . This is called minimax approximation since
max |f (x) − p∗n (x)| = min max |f (x) − pn (x)|
pn ∈Πn a≤x≤b
a≤x≤b
Proof: Let η = (η0 , η1 , · · · , ηn ) be an arbitrary point in Rn+1 and pη (x) =
Define
φ(η) = φ(η0 , η1 , · · · , ηn ) = ||f − pη ||.
Pn
i=0
ηi xi ∈ Πn .
We need to show that φ attains a minimum in Rn+1 , i.e., there exits a point η ∗ = (η0∗ , η1∗ , · · · , ηn∗ )
such that
φ(η)
φ(η ∗ ) = min
n+1
η∈R
Clearly φ(η) ≥ 0. Now we prove that φ is continuous. For arbitrary η and δ = (δ0 , δ1 , · · · , δn ) ∈
Rn+1 , we find
|φ(η) − φ(δ)| = | ||f − pη || − ||f − pδ || | ≤ ||pη − pδ ||
n
n
X
X
i
= max |
(ηi − δi )x | ≤
|ηi − δi | max{|η|i , |δ|i } → 0,
a≤x≤b
i=0
i=0
as η → δ. Next we show that φ(η) → ∞ as ||η|| → ∞. To prove the last assertion, we define
ψ : Rn+1 → R by
ψ(η) = ||pη ||
Proceeding as in the case of φ, we can prove that ψ is continuous. Let S n = {η ∈ Rn+1 :
||η|| = 1}, where ||η|| is the Eucledian norm. Now S n is closed and bounded and hence
compact subset of Rn+1 . Clearly, φ(η) > 0 for any η ∈ S n . The continuity of ψ then implies
that
µ = infn ψ(η) > 0.
η∈S
Now let η ∈ Rn+1 with η 6= 0. Set δ = η/||η|| ∈ S n . Then
pη (x) = ||η||
n
X
ηi i
x = ||η||pδ (x)
||η||
i=0
Hence
φ(η) ≥ ||pη || − ||f || = ||η|| ||pδ || − ||f ||
= ||η||ψ(δ) − ||f || ≥ ||η||µ − ||f || → ∞,
whenever ||η|| → ∞.
Let ω = inf η∈Rn+1 φ(η) ≥ 0. Since φ(η) → ∞ as ||η|| → ∞, there exists R such that
φ(η) ≥ ω + 1 if ||η|| > R. Hence, ω = inf ||η||≤R φ(η). Since φ is continuous on the bounded
and closed suset BR = {η ∈ Rn+1 : ||η|| ≤ R}, it attains its minimum over BR and hence over
the entire Rn+1 .
Theorem: Let f ∈ C[a, b] and p = p∗n is the best approximation to f for all polynomial in
Πn . Then there exists at least two distinct points x1 , x2 in [a, b] such that
f (x1 ) − p(x1 ) = −(f (x2 ) − p(x2 )) = ||f − p||
Proof: Let E = ||f − p||. If the assertion is false, then we suppose that f (x1 ) − p(x1 ) = E
for some x1 but e = mina≤x≤b (f (x) − p(x)) > −E. Hence e + E 6= 0 and the polynomial
q = p + (E + e)/2 is an element of Πn . Now we show that q is better approximaton then p.
To show this, let us proceed as follows:
E − (E + e)/2 ≥ f (x) − p(x) − (E + e)/2 ≥ e − (E + e)/2
This implies
−(E − e)/2 ≤ f (x) − q(x) ≤ (E − e)/2 =⇒ ||f − q|| ≤ (E − e)/2 < E = ||f − p||,
which is a contradiction
Notation: Given f ∈ C[a, b] and we say ξ ∈ [a, b] is a (+) point for f if f (ξ) = ||f || and
for (-) point, we should have f (ξ) = −||f ||. A set of distinct poins xi , i = 0, 1, 2, · · · , n where
a ≤ x0 < x1 < · · · < xn ≤ b is called an alternating set for f if the xi are alternately + and points, i.e. |f (xi )| = ||f || and f (xi ) = −f (xi−1 ) for i = 1, 2, · · · , n.
The Chebyshev Alternation Theorem: (no proof) Let f ∈ C[a, b] and f ∈
/ Πn . Then
p ∈ Πn is a best (uniform) approximation iff there is an alternating set for f − p consisting of
n + 2 (or more) points.
Theorem: Let f ∈ C[a, b] and p ∈ Πn is the best approximation. Then p is unique.
Proof: Assume that both p and q of Πn are best approximation. Then ||f − p|| = ||f − q|| = d.
Now r = (p + q)/2 ∈ Πn . Hence,
d ≤ ||f − r|| = ||(f − p)/2 + (f − q)/2|| ≤ ||f − p||/2 + ||f − q||/2 = d
Hence r ∈ Πn is also a best approximation. Thus there exists n+2 points xi , i = 0, 1, · · · , n+1
such that |f (xi ) − r(xi )| = ||f − r|| = d for i = 0, 1, · · · , n + 1. Fix k and assume that
f (xk ) − r(xk ) = d. Then we have
f (xk ) − (p(xk ) + q(xk ))/2 = f (xk ) − r(xk ) = d = ||f − p|| ≥ f (xk ) − p(xk ),
which gives q(xk ) ≤ p(xk ). Similarly
f (xk ) − (p(xk ) + q(xk ))/2 = f (xk ) − r(xk ) = d = ||f − q|| ≥ f (xk ) − q(xk ),
leads to p(xk ) ≤ q(xk ) and thus p(xk ) = q(xk ) for k = 0, 1, 2, · · · , n + 1.
On the other hand, if f (xk ) − r(xk ) = −d, then
f (xk ) − (p(xk ) + q(xk ))/2 = f (xk ) − r(xk ) = −d = −||f − p|| ≤ f (xk ) − p(xk ),
which gives p(xk ) ≤ q(xk ) and a simlar procedure gives p(xk ) ≥ q(xk ). Hence, in this case too
p(xk ) = q(xk ) for k = 0, 1, 2, · · · , n + 1. Clearly p − q ∈ Πn and has n + 2 (distinct) zeros and
hence p − q ≡ 0. This implies p = q.
1
Chebyshev polynomial
The Chebyshev polynomial of degree n are defined as
Tn (x) = cos(n cos−1 x),
n = 0, 1, 2, · · ·
which is same as Tn (x) = cos(nθ) where θ = cos−1 (x). The zeros of Tn are given by xk =
cos((2k + 1)π/2n) for k = 0, 1, 2, · · · , n − 1 and these zeros are real, simple and lie in (−1, 1).
It has n + 1 extrema ξk on [−1, 1] and is given by ξk = cos(kπ/n) or ξk = cos((n − k)π/n)
for k = 0, 1, 2, · · · , n. Now T0 (x) = 1 and T1 (x) = x. Evaluating Tn+1 + Tn−1 we find the
recurrence relation Tn+1 (x) = 2xTn (x) − Tn−1 (x) for n ≥ 1. Also, Tn is polynomial with
leading coefficient 2n−1 and |Tn | ≤ 1 for x ∈ [−1, 1].
Consider n ≥ 1 and find the polynomila p ∈ Πn−1 that best approximates f (x) = xn in
[−1, 1]. Now p(x) = xn − 21−n Tn (x) ∈ Πn−1 and it satisfies
21−n = max |xn − p(x)| ≤ max |xn − q(x)|
|x|≤1
|x|≤1
for any q ∈ Πn .
To show this note that xk = cos((n − k)π/n) satisfies −1 = x0 < x1 < · · · < xn = 1 and
Tn (xk ) = cos((n − k)π) = (−1)n−k . Since |Tn (x)| ≤ 1, we found an alternating set for Tn
consisting of n + 1 points. Now xn − p(x) = 21−n Tn (x) and |xn − p(x)| ≤ 21−n and xn − p(x)
has an alternating set of n + 1 points. Hence, p(x) is the best approximation to xn among
polynomials in Πn−1 .
Corollary 1: 21−n Tn (x) is the monic polynomial of degree n having smallest norm 21−n in
C[−1, 1].
To show this, let p(x) be another monic polynomial of degree n such that |p(x)| < 21−n
for all x ∈ [−1, 1]. Let xk = cos((n − k)π/n), k = 0, 1, 2, · · · n be the points of extrema
for Chebyshev polynomial. Then p(xn ) < 21−n Tn (xn ), p(xn−1 ) > 21−n Tn (xn−1 ), p(xn−2 ) <
21−n Tn (xn−2 ) and so on. Hence the polynomial q = p − 21−n Tn changes sign between each
two extrema of Tn . Hence q has n zeros but degree og q is less than n. Thus we arrive at a
contradiction and hence 21−n Tn is the monic polynomial having lowest max-norm.
Corollary 2: The monic polynomial of degree n having smallest norm in C[a, b] is
(b − a)n
2x − b − a
Tn
2n 2n−1
b−a
To show this let p(x) is the monic polynomial of degree n. Make the transformation
2x = (a + b) + (b − a)t. Now
max |p(x)| = max |p̃(t)|
|t|≤1
a≤x≤b
Since p̃(t) = p((b − a)t/2 + (a + b)/2), the leading coefficeint of p̃ is (b − a)n /2n . Hence,
p̃(t) = (b − a)n /2n p̂(t) where p̂ is a monic polynomial of degree n in [−1, 1] and
max |p(x)| =
a≤x≤b
(b − a)n
max |p̂(t)|,
|t|≤1
2n
Hence, for minimum norm, we must have p̂(t) = 21−n Tn (t) and hence the result.
Let ωn+1 (x) = (x − x0 )(x − xn ) · · · (x − xn ) where xi ∈ [a, b] and distinct. Using the
transformation to t we find
ωn+1 (x) =
(b − a)n+1
(t − t0 )(t − t1 ) · · · (t − tn ),
2n+1
where
ti =
a+b
2
xi −
b−a
b−a
Hence for minimum value, we must have
(t − t0 )(t − t1 ) · · · (t − tn ) = 2−n Tn+1 (t)
and hence ti corresponds to the zeros of Tn+1 (t). Thus,
tk
=
=⇒
2
a+b
cos((2k + 1)π/2(n + 1)) =
xk −
b−a b−a
a+b b−a
2k + 1
xk =
+
cos
π , k = 0, 1, 2, · · · , n
2
2
2(n + 1)
Note that the xi ’s are from right to left in [a, b]. If we want xi ’s to be from left to right, then
we replace xk in the above expression by xn−k . Alternatively, we may also write
2k + 1
a+b a−b
xk =
+
cos
π , k = 0, 1, 2, · · · , n
2
2
2(n + 1)
Also the minimum value of maxa≤x≤b |ωn+1 (x)| in this case becomes (b − a)n+1 /22n+1 .