18.335 Problem Set 2
mean
the
error
an be bounded by
Pn
√
nǫmahine i=1 |xi | .
|f˜(x) − f (x)| = O
Due Monday, 29 September 2008.
(Hint: google random walk...you an just
quote standard results for random walks,
no need to opy the proofs.)
Problem 1: Floating-point
(a) Trefethen, probem 13.2.
(For part , you
(e) Compare
an use Matlab, whih employs IEEE double
preision by default.)
error
bounds
above
to
Here,
we will use an old trik to ompute the
oating-point errors:
(b) A generalization of Trefethen, problem 14.2:
given a funtion
your
numerial experiments in Matlab.
g(x)
ompare the results
omputed in one preision to the exat
that is analyti (i.e.,
results
has a Taylor series) for |x| suiently small,
′
and g (0) 6= 0, show that g(O(ǫ)) = g(0) +
′
g (0)O(ǫ).
In
omputed
partiular,
single()
in
we
a
higher
will
use
Matlab
funtion to aumulate the sum
in single preision,
rather than Matlab's
default double preision.
Problem 2: Addition
preision.
the
|f˜(x) − f (x)|/
Plot the error
P
i |xi | as a funtion of n on a
log-log sale (Matlab's loglog ommand),
This problem is about the oating-point error involved in summing
and explain your observation in terms of
the funtion
your results above.
nP
numbers, i.e. in omputing
n
f (x) = i=1 xi for x ∈ Fn (F being
1
the set of oating-point numbers), where the sum
is done in the most obvious way, in sequene. In
This
pseudoode:
le
is
implemented
loopsum.m,
whih omputes the sum
sum = 0
for i = 1 to n
sum = sum + xi
f (x) = sum
in
the
example
posted on the ourse page,
f (x) =loopsum(x)
via the above algorithm in single preision.
For your numerial experiment, ompute
n
the sum of n random inputs x ∈ [0, 1)
via Matlab's
rand(1,n)
then ompute
For analysis, it is a bit more onvenient to dene
given
n
via
funtion. You an
|f˜(x) − f (x)|/
P
i
|xi |
for a
the proess indutively:
s0
sk
with
=
=
0
sk−1 + xk
f (x) = sn .
for
x = rand(1,n);
err = abs(loopsum(x) - sum(x)) /
sum(abs(x));
0 < k ≤ n,
When we implement this in
f˜(x) = s̃n ,
s̃k = s̃k−1 ⊕xk , with ⊕ denoting (orretly
oating-point, we get the funtion
where
Problem 3: Addition, another way
Qn
(a) Show that f˜(x) = (x1 + x2 )
k=2 (1 + ǫk ) +
Qn
Pn
(1+ǫ
)
x
,
where
the
numbers ǫk
k
k=i
i=3 i
.
satisfy |ǫk | ≤ ǫ
mahine
dierent way.
uniformly
the
randomly
[−ǫmahine , +ǫmahine ].
ǫk
values
distributed
Show
f˜(x)
by
two halves and then summing the halves:
k=i (1+ǫk )
that
In partiular, ompute
sively dividing the set of values to be summed in
Qn


0
f˜(x) = x1

˜
f (x1:⌊n/2⌋ ) ⊕ f˜(x⌊n/2⌋+1:n )
() Show that the error an be bounded as:
Pn
|f˜(x) − f (x)| ≤ nǫmahine i=1 |xi |.
(d) Suppose
Pn
a reursive divide-and-onquer approah, reur-
= 1+δi where |δi | ≤
2
(n − i + 1)ǫmahine + O(ǫmahine
).
(b) Show that
f (x) =
i=1 xi as in problem 2, but this time you will ompute f˜(x) in a
Here you will analyze
rounded) oating-point addition.
are
1 Use
in
enough
ommand
that
n
if
if
values to get a lear result.
n = round(logspae(2,6,100))
arithmially spaed
1
if
n
values from
102
to
n=0
n=1,
n>1
e.g.
the
gives 100 log-
106 .
where
y
⌊y⌋
denotes the greatest integer
≤y
you ompute
(i.e.
rounded down). In exat arithmeti, this om-
putes
f (x)
AB
via the simple 3-loop row-
olumn algorithm? What if you use the optimal ahe-oblivious algorithm from lass?
exatly, but in oating-point arith-
meti this will have very dierent error harateristis than the simple loop-based summation
Problem 4: Stability
in problem 2.
(a) Trefethen, exerise 15.1.
n
(a) For simpliity, assume
is a power of 2
(b) Trefethen, exerise 16.1.
(so that the set of numbers to add divides
evenly in two at eah stage of the reursion).
With an analysis similar to that
of problem 2, prove that |f˜(x) − f (x)| ≤
Pn
ǫmahine log2 (n) i=1 |xi | + O(ǫ2mahine ).
That is, show that the worst-ase error
bound grows logarithmially rather than lin-
early with
n!
(b) If the oating-point rounding errors are randomly distributed as in problem 2, estimate
the average-ase error bound.
() Pete R. Stunt, a Mirosoft employee, omplains, While doing this kind of reursion
may have nie error harateristis in theory, it is ridiulous in the real world beause it will be insanely slowI'm proud
of my eient software and an't aord
to have a funtion-all overhead for every
number I want to add!
Explain to Pete
how to implement a slight variation of this
algorithm with the same logarithmi error
bounds (possibly with a worse onstant fator) but roughly the same performane as a
2
simple loop.
(d) On
a
f˜(x)
the
ourse
le
web
=div2sum(x)
rithm.
page,
div2sum.m
by
I've
that
the
posted
omputes
above
algo-
Modify it to not be horrendously
slow via your suggestion in (), and then
plot
its
errors
funtion of
n
for
random
inputs
as in problem 2.
as
a
Are your
results onsistent with your error estimates
above?
m × m ranB (∈ [0, 1)m×m , uniformly distributed) to form C = AB . If you
look at any given entry cij of C , how quikly
do you expet the errors to grow with m if
(e) Suppose we now multiply two
dom matries
2 In
does
A
and
fat, there is a ommon real-world algorithm that
summation
in
preisely
this
reursive
way:
the
Cooley-Tukey fast Fourier transform.
2