The first thing printed out in the trace file is information from the
Jacobian that is used to set the scale factors for each parameter and
the initial trust region boundary.
Each iteration consists of a Jacobian evaluation and a number of
subsequent "steps", each requiring a function evaluation. Step "0"
gives the parameter values (vector X) and the function value (Fnorm)
at the starting point of the iteration. Subsequent steps then give the
difference parameter vector dX, the function norm evaluated at X+dX,
and three quantities that are used to determine the search direction,
specifically the Levenberg-Marquardt parameter (LMpar), the reduction
ratio (Ratio) and the trust region scaling factor (Trscl).
(1) Levenberg-Marquardt parameter (LMpar). This represents the
admixture of the steepest-descent (gradient) direction with the
Gauss-Newton direction used to determine the step direction dX. A
value of zero corresponds to the Gauss-Newton direction, with no
component along the steepest descent direction.
(2) Ratio: This gives the ratio of the actual reduction of the
function norm at f(X+dX) to the reduction that was predicted from the
curvature matrix. It is a measure of how well the truncated Taylor
series expansion used to find the Gauss-Newton direction approximates
the local topology of the function being minimized. If the actual
reduction is much smaller than the predicted reduction (i.e. if ratio
is a small number) then the function is not well-approximated by a
quadratic equation within the trust region. (Note that negative values
of ratio actually correspond to an increase in residuals at X+dX). In
contrast, ratios near unity indicate that that the function is
well-approximated by a quadratic.
(3) Trust region scaling factor (Trscl). The size of the trust
region is adjusted by the factor Trscl according to the values of
Ratio and LMpar. The strategy is to restrict the trust region when
the function is poorly approximated and to try increasing it when the
function is well-approximated by a quadratic.
The following cases require adjustment of the trust region bound:
(1) ratio is between 0 and 0.25: Trscl=0.5
(2) ratio is less than 0: Trscl=0.5 to 0.1, depending on the
magnitude of the increase in residuals.
(3) ratio is greater than 0.75 (or par=0): Trscl=2.0
There are three cases for which Trscl does not reflect how the trust
region bound was adjusted in a given step. These cases are flagged by
a character following the reported value of Trscl, as follows:
* : Trust region was reduced to the scaled length of the step in
the
first iteration.
s : Trust region was less than the step length on the first
iteration
(the initial estimate of the trust region bound is retained).
x : Trust region bound exceeded ten times the scaled step length,
and
was adjusted to that value before multiplication by Trscl.
An iteration ends when some reduction in the function is observed
(ratio > 0.0001)