Preconditioning
Preconditioning is a technique that can speed up the convergence of iterative methods. To explain the
idea of preconditioning recall that when solving Ax = b, the convergence rates of conjugate gradient
(and the other methods) depends on the condition number of A. If A is well conditioned then the
methods converge quickly and if A is ill-conditioned the methods converge slowly. Suppose we
know a matrix M such that M-1 is easy to calculate and such that M-1 is approximately A . The M-1*A
should be approximately the identity matrix and should be well conditioned. Therefore if we solve
M-1 Ax = M-1 b rather than A x = b
we should get quick convergence.
To make this practical is important that M-1 be easy to calculate. Matlab supplies a function ilu,
incomplete lu factorization, that can be used to produce M. There are two common uses of ilu:


[M1,M2] = ilu(A) does an LU factorization of L with no fill-in. If any entry in A was initially
zero then the corresponding entry will also be zero in L+U. If Gaussian elimination produces
fill-in (a zero becoming a non-zero during elimination) the fill-in is ignored.
setup.type = 'ilutp'; setup.droptol = 0.01; [M1,M2] = ilu(A,setup); does a LU factoriziation with
some fill-in allowed. The factorization will "drop" or set to zero any entries in the factorization
of A that are less than 0.01 (for the above droptol) times a norm of a column or row of A.
Often ilu with droptol = 0.1, 0.01 or 0.001 can be calculated quickly and produce adequate M
matrices
In either case the corresponding M is M = M1 * M2.
It is most efficient to not calculate M or M-1 explicitly. Rather, whenever one wishes to calculate u = M-1 v
for some vector v, solve Ml * M2 u = v by solving two triangular systems M1 w = v and then M2 u =
w. Solving triangular systems is quick compared to finding M-1. This is Matlab’s approach:
tol = 1.e-8; max_it = 5000;
tic, setup.type = 'ilutp'; setup.droptol = 0.01; [M1,M2] = ilu(A,setup);
[x,flag1,relres1,iter1,resvec1] = pcg(A,b,tol,max_it,M1,M2); time1= toc
If A is gr_30_30 with tol = l.e-8, pcg with preconditioning with the above droptol takes17 iterations and
approximately 0.015 seconds on my computer. In comparison preconditioning without the drop
tolerance option “[M1,M2] = ilu(A);” followed by pcg requires 22 iterations and approximately 0.016
seconds and pcg with no preconditioning takes 51 iterations and approximately 0.017 seconds. In this
case there is some small advantage to using preconditioning. The extra work required by calling ilu is
compensated by the fewer number of iterations required by pcg. The condition number of the
gr_30_30 matrix is about 200, which is modest. In examples where A is more strongly ill-conditioned
the advantage of preconditioning can be greater.
In your project I ask that you find an example where the method that you select does not work well for
the ordinary routine (either gmres, bicg, bicgstab, qmr, cgs or lsqr). In addition to running this
example with the ordinary methods you should try the corresponding routine with
preconditioning. In your report describe whether this helps the convergence of your method. You may
want to try several values of droptol (for example 0.1, 0.01 or 0.001) or no drop tolerance in ilu. Also
you may want to try and discuss in your report the precondition routines on some of your other
examples.