Michael Ulbrich Stefan Ulbrich Luı́s N. Vicente
A globally convergent primal-dual interior-point filter
method for nonlinear programming
April 2000. Revised February 2002
Abstract. In this paper, the filter technique of Fletcher and Leyffer (1997) is used to globalize the primaldual interior-point algorithm for nonlinear programming, avoiding the use of merit functions and the updating
of penalty parameters.
The new algorithm decomposes the primal-dual step obtained from the perturbed first-order necessary
conditions into a normal and a tangential step, whose sizes are controlled by a trust-region type parameter.
Each entry in the filter is a pair of coordinates: one resulting from feasibility and centrality, and associated
with the normal step; the other resulting from optimality (complementarity and duality), and related with the
tangential step.
Global convergence to first-order critical points is proved for the new primal-dual interior-point filter
algorithm.
Key words. interior-point methods, primal-dual, filter, global convergence
1. Introduction
In this paper we use the filter technique of Fletcher and Leyffer [13] to globalize the
primal-dual interior-point method for nonlinear optimization. This technique incorporates the concept of nondominance (borrowed from multi-criteria optimization) to build
a filter that is able to reject poor trial iterates and enforce global convergence from arbitrary starting points. The filter replaces the use of merit functions, avoiding therefore
the update of penalty parameters associated with the penalization of the constraints in
merit functions.
Since its first appearance in a 1997 paper by Fletcher and Leyffer [13], the filter
technique has been mostly applied, so far, to SLP (sequential linear programming) and
SQP (sequential quadratic programming) type methods [11, 13, 14]. Global convergence
to first-order critical points has been proved for SLP by Fletcher, Leyffer, and Toint [14]
in 1998 and for SQP by Fletcher, Gould, Leyffer, and Toint [11] in 1999. In the context
of composite SQP for equality constrained optimization, Ulbrich and Ulbrich [21] have
Michael Ulbrich: Zentrum Mathematik, Technische Universität München, 80290 München, Germany, E-mail:
mulbrich@ma.tum.de. Support for this author was provided by CRPC grant CCR–9120008.
Stefan Ulbrich: Zentrum Mathematik, Technische Universität München, 80290 München, Germany, E-mail:
sulbrich@ma.tum.de. Support for this author was provided by CRPC grant CCR–9120008.
Luı́s N. Vicente: Departamento de Matemática, Universidade de Coimbra, 3001-454 Coimbra, Portugal, Email: lvicente@mat.uc.pt. Support for this work was provided by Centro de Matemática da Universidade de
Coimbra, by FCT under grant POCTI/35059/MAT/2000, and by the European Union under grant IST-200026063.
Mathematics Subject Classification (1991): 65K05, 90C06, 90C29, 90C30
2
M. Ulbrich, S. Ulbrich, and L.N. Vicente
also proposed, based on filter ideas, a nonmonotone trust-region algorithm. Recently,
Audet and Dennis [2] presented a pattern search filter method for derivative-free nonlinear programming. The filter idea has proven to be very successful numerically in the
SLP/SQP framework [12], motivating its applications to interior-point methods.
Interior-point methods, although quite well studied for linear and convex programming, are still a very open topic of research in nonlinear programming. One of the issues
is guaranteeing global convergence because there seems to be no ideal merit function.
Several approaches for globalizing interior-point methods using different merit functions have been proposed. See the references [5, 8, 10, 15, 16, 22]. On the other hand, the
local convergence properties of interior-point methods for nonlinear programming are
quite well studied in the literature [6, 7, 10, 18, 23, 28], although difficulties arise when
the limit point does not satisfy strict complementarity or linear independence of the
gradients of the active or binding constraints [17, 20, 24].
The primal-dual interior-point method is based on the application of Newton’s method to a perturbed version of the first-order necessary conditions. The perturbation incorporates the (numerically efficient) notion of centrality, forcing the iterates to stay as
much as possible away from the boundary of the feasible set. Our primal-dual interiorpoint filter method is partially motivated by the SQP-filter algorithm of Fletcher, Gould,
Leyffer, and Toint [11]. We also split the primal-dual step into normal and tangential
steps and use a trust-region parameter to control the size of both steps. The normal step
points towards the quasi-central path, trying to achieve an improvement in feasibility
and centrality. The tangential step is designed to reduce the size of the gradient of the
Lagrangian function (and complementarity). The algorithm incorporates also a restoration phase (proposed in [11, 13, 14] for SLP/SQP-filter) aimed to improve, if necessary,
feasibility and centrality.
This paper is organized as follows. A brief outline of the basic concept of filter
methods is given in Section 2. The primal-dual interior-point framework is presented
in Section 3, where a number of estimates are presented for the composite primal-dual
step (the proofs are postponed to an appendix). The filter mechanism and the primaldual interior-point filter method are described in Section 4. Section 5 contains the proof
of global convergence to first-order critical points. A restoration algorithm is proposed
in Section 6 and some final remarks and open questions are stated in Section 7.
to denote the Euclidean norm of a vector as well as the induced matrix
We use
norm. Given two vectors
and
, we use
to represent the vector
. Finally, we pose the nonlinear programming problem in the
general form
s.t.
(1)
"!# $&%('*)
-+ /. 0+21 . )43
3 ,+#
where
6587#9: and ; 65<7=9>6? are twice continuously differentiable functions on an open set @BA 5 .
2. Basic concept of a filter method
We begin by outlining the main concepts of the filter method by Fletcher and Leyffer
[13], using the form analyzed by Fletcher, Gould, Leyffer, and Toint [11]. This method
is applicable to the general nonlinear programming problem. However, since in our
A globally convergent primal-dual interior-point filter method
3
+C1 .
interior-point approach the nonnegativity constraints
will be handled by the
interior-point step calculation, it is sufficient to sketch the basic algorithmic framework
of the original filter method for the simpler (equality constrained) problem
$&%('*)
(2)
, +# s.t. -+ D. )E3
3
where
5 7=9 and - 5 7#9 ? are twice continuously differentiable
functions on an open set @FA 5 .
The concept of NLP filter methods originates from the observation that the solution
of a nonlinear programming problem like (2) consists of the two competing aims of
minimizing a measure
) of feasibility G=+# , e.g. GH,+# -+ , and of minimizing the
objective function ,+# . Hence, (2) can be seen as a bi-criteria optimization problem
with the additional requirement that G has some priority, since convergence to a feasible
point must be ensured. Instead of combining the two objectives by using a penalty function, Fletcher and Leyffer proposed in [13] to use a filter to build the efficient border of
the bi-criteria optimization problem of minimizing infeasibility and objective function
value. The definition of a filter takes into account the fact that we would like to reduce
both,
and
. A filter is a finite set of tuples
– pairs in this case
– that correspond to a collection of points , with the additional requirement that no
filter entry is dominated by any of the others. Hereby, following [13], a point , or the
corresponding pair
, is said to dominate a point , or the corresponding
pair
, if
)
G=+#
,+#
)
G=+ O N ,+ O +
+O
)
I
G=+#N ,+#
G=+#QPGH,+ O +J
)
,GH+KJLM +KJL
and
)
)
+RP , + O MS
+#O
+
If dominates , the latter is most probably of no real interest. A natural requirement
for a new iterate is, therefore, that it should not be dominated by previous iterates. The
filter serves the purpose of collecting information on selected previous iterates, and
thus provides, in terms of dominance, a selection criteria for new iterates. However,
it is obvious that the acceptance of iterates whenever they are not dominated by the
filter (i.e., by any of the filter entries) does not exclude, for example, a clustering of
iterates at an infeasible point. To avoid the acceptance of pairs that are arbitrarily close
to the efficient border (i.e., to the boundary of the set of all pairs that are dominated by
the filter), acceptability of to the filter is defined in [11] in a more stringent way by
requiring that for all filter entries
it holds that
+
)
, GH+ J M + J TUI
$WVYX#Z
)
)
G=+ J 67[G=+#N ,+ J 67 +]\_^4`bacGH+ J M
where `aDd . eLfhgh is fixed. The original) concept of nondominance is still used to add
a point, or the corresponding pair ,GH+M +#i , to the filter: If + is added to the filter,
then all entries that are dominated by + are removed from the filter.
In order to produce new iterates that are acceptable to the filter, Fletcher, Gould,
Leyffer, and Toint [11] proposed the use of a trust-region framework and the decominto a normal step
and a tangential step . The
position of the step
normal step
is computed yielding linearized feasibility, i.e.
j 5k
jLk j 5kml j"nk
j 5k
. j 5k P<p#qRr&k
-+kh l4o -+kh j 5k <
j"nk
(3)
4
M. Ulbrich, S. Ulbrich, and L.N. Vicente
It is assumed that there exists a constant p ^ . such
j p#5k q/
P84p . Gs"e
k foris aallconstant.
t . The tangential step is in turn computed verifying5
5
o -+ks j nk <. j 5k l j nk P8r&k
)
and providing a fraction of Cauchy decrease for a quadratic model uk of .
Problem (3) is infeasible if r&k is too small in comparison to Gsk . Therefore, it is
required in [11] that
j 5k P8p q r k $&%v' Z ehwp=xr xk \
(4)
with constants p=xy^ . and z<< . "e
. In particular, for small r k , the normal step j 5k
must be tiny compared to r k . If it is not possible to compute a normal step satisfying
(4), a restoration phase is entered with the goal of reducing the infeasibility, measured
by G , as much as needed. Then the full normal step can always be taken, and thus GH,+#k l
jLks |{ }rm~k . Possibly after reducing r&k (and reentering restoration) if necessary), the.
new trial iterate will be acceptable to the filter if all filter entries ,G J J satisfy G J ^ ,
which is ensured by the mechanism of selecting new filter entries. If the filter test is
passed, the decrease properties
) of the full step jLk on the model uk are checked. If
the predicted decrease for is not promising, more precisely if uk+khR7ukK+#k l
jLksR€<p=NGhk~ with a constant) p#‚ . "e
, then the infeasibility is considered to dominate
) in . The new iterate is accepted and +#k is added to the filter.
the possible decrease
Otherwise, the -decrease is required to satisfy the standard trust-region acceptance
)
)
criterion
+kh7 +#k l jLks 14ƒ
u k + k 7u k ,+ k l j k .
with preset ƒd/ eL . If the test fails, r„k is reduced. Otherwise,
) the step is accepted
and r„k is updated as in a standard trust-region algorithm. If the -decrease is met, +#k is
not added to the filter, since G k can be very small in this case and adding + k can enforce
where
that
small trust-region radii to get acceptable points in later iterations.
The interior-point filter method introduced in this paper is inspired by the sketched
SQP filter method of [11]. Essentially, our application of the filter concept to the globalization of interior-point methods has the following features:
– A primal-dual interior-point Newton step forms the basis for the trial step computation. The nonnegativity constraints are handled by a centering mechanism.
– We identify two objectives and , corresponding to feasibility and objective function value, such that an appropriate splitting of the step in “normal” and “tangential”
components guarantees decrease for linearized models of and , respectively.
– An adaptation of the filter framework of [11] for the pair
is proposed.
– The trial steps do not require the recomputation of normal and tangential components when the trust-region radius changes.
G
GL
G G
GKiG … 3. Interior-point framework
We return to the nonlinear programming problem posed in the general form (1).
A globally convergent primal-dual interior-point filter method
5
3.1. Step computation
Primal-dual interior-point methods are based on the idea of applying Newton’s method
to an appropriate perturbation of the first-order necessary optimality conditions (KarushKuhn-Tucker or KKT conditions). For the problem under consideration, the KKT conditions can be written in the form
ˆŒ ?
.
o„†Y‡ +ˆi‰ <
<
-Š + . ‰ <. +1 . ‹‰W1 . (5)
(6)
(7)
(8)
‰BŽ 5 are the Lagrange multipliers, ‡ denotes the Lagrange
)
+ˆi‰ ,+# l -+# ˆm7[+ ‰=
‡
Š
and is the diagonal matrix of order  in which the  -th diagonal element is +=‘ . Under
a constraint qualification the conditions (5)–(8) are necessary for + to be a local solution
where
function
and
of (1).
We now perturb block (7) of the KKT system (5)-(7) and write
š›
Ÿ ’‚x”“ +ˆi‰=•— –™˜ o † ‡ - ,++#iˆ#w‰b /. Š ‰œ7Bz  ž
where z
 ^ . . Throughout, we will work with z  ¡ z , where ¡ . eL is a centering
parameter, and
z +=‰ S
(9)
To abbreviate notation we set
/ ,+ˆw‰b and r ¢rm+wrmˆwr„‰bNS
The primal-dual Newton step r is determined by the Newton equation for the perturbed KKT system, i.e.,
’ £ O x +ˆi‰r 7 ’ £ x”,+ˆw‰bN
š› o ~ ‡ ,+ˆw‰b o -+Q7Q¤ Ÿ š› r¦+ Ÿ š› o † ‡ +ˆi‰ Ÿ †
† -,+#
.
.
rmˆ 7 Š -,+#¡ S
o ¥
. Š
r„‰
‰œ7 z ž
The choice z  /¡ z with centering parameter ¡ . eL and complementarity measure
z according to (9) ensures that the primal-dual Newton direction r is a descent direction for +#‰fL and allows thus a dynamic reduction of z . This choice is frequently used
or, in detail, by
in the context of linear programming and was also used in the nonlinear programming
algorithm of El-Bakry et al. [10].
6
M. Ulbrich, S. Ulbrich, and L.N. Vicente
To adapt the methodology of a filter as outlined in Section 2 to our interior-point
context, we have to specify the two quantities for the filter entries, the first component
corresponding to feasibility and the second corresponding to optimality. Along with the
choice of the filter components, we have to find a corresponding decomposition of the
trial step into a normal step and a tangential step that yields a decrease of the feasibilityand optimality-component, respectively.
To motivate our choice of the components in the filter and the step decomposition,
we rewrite the perturbed KKT-conditions in the form
’ £ x +iˆi‰ š›
. Ÿ ›š o † ‡ +ˆi‰ Ÿ .S
Š -‰œ,7+# z ž l § e¨7 . ¡ —z ž /
(10)
z
©_x ª Z +i‰ 3 -,+# /. Š ‰ z ž«\S
3 Dit .seems natural to let the quasi-central path
Therefore, in terms of our filterZ approach,
-,+# \ in Section 2 and to choose the measure
play the role of the feasible set +
of quasi-centrality
GH -+# l Š ‰‚78+ ‰ifL ž The first term in the middle expression measures the proximity to the quasi-central path.
We recall that the quasi-central path, parameterized by (see [1]), is defined by
as the first component in the filter. In this context it is important to mention that the
central path for nonlinear programming
©_x ¬ Z ,+iˆ#w‰b 3 o„†Y‡ D. =-+ <. Š ‰ z ž«\
parameterized by z , is only guaranteed to exist (for sufficiently small z ) in the neighborhood of a point$WVs+X=Z ˆi‰ that satisfies the second-order sufficient conditions, strict
complementarity (
+=‘ii‰L‘—\_^ . i eh"SS"SN ), and linear independence of the gradients of the active or binding constraints.
The second term in the middle expression of (10) measures complementarity and
criticality. For the second filter component we choose therefore the optimality measure
G … + ‰ fY l o † ‡ ~ S
G
We are aware that this choice certainly allows some room for improvements, since the
fact that we are dealing with a minimization problem is not very well reflected by .
However, given that the investigation of filter methods is still in its beginnings, we think
that our choice of the optimality measure is appropriate for the purpose of this paper.
We believe that our approach is also viable for other choices of , and return to this
issue in Section 7.
With this choice of the filter components it remains to define corresponding tangential and normal components of the trial step. We use the decomposition associated with
the splitting (10). For the normal step
we thus choose
GL
j 5 } rm+ 5 ]r¦ˆ 5 ]rm‰ 5 ›š
Ÿ ’ £ O x i j 5 7 - . + Š ‰œ7[z ž
(11)
A globally convergent primal-dual interior-point filter method
7
j n ¢ rm+ n wrmˆ n ]r„‰ n is given by
š›
Ÿ ’ £ O x j n 7 o„†s‡ . S
§e¨7 ¡ —z ž
whereas our tangential step
r j 5 l j"n
j5
(12)
jn
Note that
. However, it will be crucial that we exploit the flexibility of the
step splitting to introduce different stepsizes for and in our trial step computation.
The adjectives normal and tangential are borrowed from the SQP context [4, 19] but
have here a slightly different flavor. The normal step can be seen as a step towards the
quasi-central path . The tangential step is the sum of a tangential component
© xª
j"n­
š›
Ÿ ’ £ O x ij n­ 7 o„†Y‡ . .
that attempts to reduce o † ‡ , with a predictor component j"n
~
š› . Ÿ ’ £ O x j n 7
.
~
eQ7 ¡ —z ž
that seeks the minimization of z +#‰fL (see, for instance, [27]). Therefore,
the
tangential step aims to reduce the optimality measure GL…b +=‰fY l o„†L‡ ~ .
We introduce r as the positive scalar that primarily controls the length of the step
taken along r , forcing the damped components ® 5 ¢r&j 5 and ®nM}r„ij"n , to satisfy
® 5 ¢r&j 5 P8r¯ ® n }r„ij n P<r¯S
Having these bounds in mind, and requiring explicitly ® n ¢r&TP®5¢r& , we set
$&%('&° r
® 5 ¢r& eh j 5 6± (13)
$&%(' °
$„%v' ° r r
(14)
® n ¢r& ® 5 ¢r&M rj n 6± e« j 5 j n ± S
e for j 5 /. and ® n ¢r& &
$
v
%
”
'
Z
5
Hereby, we use for r²^ . the natural definition
®
}
&
r
® 5 ¢r& for jn /. by using the convention e«]³<\ e .
Let also
¢r& +}r&Mˆ”}r„Ni‰#¢r& < l ® 5 ¢r&j 5 l ® n ¢r&ij n (15)
µ
´
8
jb¢r& j † }r&Mwj
¶}r„Nwj
·¢r&¸ }r&¹7
® 5 ¢r&j 5 l ® n }r„ij n S (16)
Thus,
j«}r& P8g«r¯
and r plays here a role comparable to the trust-region radius.
The scalars ® 5 }r& and ®nN}r& define the step sizes taken along the normal and
tangential directions, respectively, and will be such that positivity and some measure
8
M. Ulbrich, S. Ulbrich, and L.N. Vicente
¢r&
r
®5”¢r&
® n }r&
of centrality of the new iterate
are maintained. However, both
and
depend on , that in turn will be adjusted, not only to meet the purpose of positivity
and centrality, but also to enforce global convergence.
We introduce the notation
Gsº= - ,+# »» Š ‰œ7 #+ ‰ ž »» G ¬ ²
 »»
»»
G
¼
„o †L‡ which allows to rewrite the filter components as
G … +#‰ l o † ‡ ~ S
Š
Since ‰ might not be zero, a point that satisfies GH G
¼
D. and ,+w‰bR1 . ,
might not be a KKT point. The definition of G
, however, guarantees that a point verifying GH GL… D. and +i‰R1 . , is indeed a KKT point.
With the purpose of achieving a reduction on the function GL… , we introduce, at a
given point , the quadratic model
u ¢r&
+=‰ l ,+}r„7[+§‰ l ,‰#¢r&7‰§+ l o„†Y‡ l4o ~½ † ‡ }r&7 ~


+
¢
&
r
#
‰
¢
&
r
8
7
+
¢
&
r
¹
7
+#c¢‰=}r„¹7‰ l o † ‡ lo ~½ † ‡ }r„¾7 ~ 
by adding to the linearization of +=‰fY the squared norm of the linearization of om†s‡ .
To shorten notation we also set
z}r& +¢r&§‰#¢r& S
In order to prevent +}r&Mi‰#¢r&i from approaching the boundary of the positive
orthant too rapidly we will keep the iteration in the neighborhood
¿ À`6wÁ ° 3 +i‰T^ . Š ‰&1` +#‰ ¯G º l G ¼ TP<Á +=‰ ±


^ . . This is a frequently used centrality condition in
with fixed `Žµ . eL and Á
infeasible interior-point methods for linear and convex quadratic programming, cf. [27]
and the references therein, and has also proven its efficiency in the context
nonlin¿ À`6wÁÂof implies
ear programming
[10].
We
will
see
in
the
next
subsection
that
}r„Q ¿ `]Á whenever r . ]rmÃ¹Ä ÅLÆ for a given constant rmÃ¹Ä Åm^ . .
G= G ¬ l G º M
3.2. Step estimates
GL
r
¢r&
Gsº G ¬ "G ¼
¢r&7 The following lemma provides, at
, upper bounds on the values of , , , and
in terms of and of the corresponding values at . It also provides a lower bound
for the decrease produced on the quadratic model by the step
.
u
A globally convergent primal-dual interior-point filter method
9
ÁǺ Á ¬ , and Ád¼ , depending on an upper.
o - and o ~† ½ ‡ , such that, for all rµ^ ,
G º ¢r&QP|e¨7® 5 ¢r&i—G º l Á º r ~ (17)
G ¬ ¢r&QP|e¨7® 5 ¢r&i—G ¬ l Á ¬ r ~ (18)
G ¼ ¢ r&QP|e¨7® n ¢r&§G ¼ l Á ¼ r ~ S
(19)
For r satisfying . €<r²Pr„ÈYÉ , it also holds
(20)
G
¢r&QP ´ e¨7® n ¢r&§eQ7 ¡ ¸ GL…b l Ád…Yr ~ for some positive constant Ád… .
Finally, for any r^ . , we also have
u 67u ¢r&Q1® n ¢r&§eQ7 ¡ —G … NS
(21)
Now we state a result that says that if the current point  +ˆi‰ satisfies the
Š
centrality requirement ‰&1`#z ž , so does the next point }r& ,+}r„Nˆ”¢r&Ni‰#¢r& ,
provided r is sufficiently small. A similar property is also stated for the inequalities
GsºH l G
¼
TP<Á|z and +i‰R^ . .
’
­ P<Ë and, for `yd . "e
and ÁÌ^ . , assume that
Lemma 2. Let £ O x ]Ê
Š ‰&1`z žb0GYº= l G
¼L QPÁ|z¹S
There exists a constant r Ã¹Ä Å such that, if . €<r²P<r Ã¹Ä Å , then
Š ¢r&‰=}r„R14`z¢r&§žb
(22)
(23)
GsºH ¢r&i l G"¼L }r&¨P<Á|z¢r&NS
Furthermore, if +i‰R^ . , then, for all r in . wrmÃ¹Ä ÅLÆ ,
+¢r&Mw‰=}r„iT^ . S
(24)
¿ `]Á implies ¢r&T ¿ À`6wÁ for all rd . wrmÃ¹Ä ÅLÆ .
Thus, G
¼
Lemma 1. There exist positive constants
,
bound for and on the Lipschitz constants of
The proofs of these results are quite technical and are left for an appendix of this
paper.
4. The interior-point filter method
GH G ¬ l GsºH G
G
GL
G
G
Our definition of a filter takes into account the fact that we would like to reduce both
and
. Hence, we choose and
to form a filter entry,
where measures feasibility and measures optimality. We thus replace the objective
function value by the criticality measure . Since we introduced the filter concept
already in Section 2, we give here only formal definitions to set the notations for the
rest of the paper.
GL
10
M. Ulbrich, S. Ulbrich, and L.N. Vicente
O
GH M iGL…H i , is said
GH O MiG
O G … QPG … O M
Definition 1 (Dominance). A point , or the corresponding pair
to dominate a point , or the corresponding pair
, if
HG TPGH O and
or, alternatively, if the following inequality is violated:
$WVYX#Z GH ¹7GH O MGL… ¹7G
O ]\Í^ . S
Definition 2 (Filter). A filter is a finite subset IÎAC6~ consisting of pairs GbÏHiG«… Ï ,
with G«Ï •— –™˜ G ºÏ l G«¬Ï , such that no pair can dominate any of the others.
As pointed out in Section 2, the mere requirement that a new iterate is not dominated
by any of the filter entries is too weak as an acceptance criterion. Instead, we require:
` a / . "eLfsg« be fixed. The point is acceptable to the filter I
,G«ÏHG«…Ï TUI , it holds
$&VsX=Z
G Ï 7GH MG …Ï 7GL…b ]\m^`a¾G Ï S
Definition 3. Let
for all
if,
In the course of the algorithm, we will add selected new points to the filter. This
procedure is done in the following way:
Z to the filter I we mean the following operation:
IŒÐ9ÑI Ó G= N GL…K i]\Y32Ò $&%v'”Z
,G Ï G …Ï TUI
G Ï 7[GH MiG …Ï 7[G … M\¦€ .=Ô S
Remark 1. Therefore, if is added to the filter, all old entries that are dominated by the
ÕÖ
new entry are removed.
Our primal-dual interior-point filter method generates iterates k ­ F kK¢r&khm ×
!
k that
are acceptable to the filter, but not all new iterates k ­ are added to the filter.
!
In general, the primal-dual interior-point filter method imposes a sufficient reduction criterion relating the actual reduction in G
with the reduction predicted by its
model uk :
Ø
km14ƒ
where
Ø GL… k«7[GL… k}r„k«
k •§–™˜ u k k 7[u k k ¢r k and ƒ¯d . "e
is a preset constant.
However, the test of this condition is skipped if
u k k 67[u k k ¢r k ¨€pHGH k ~ Ø pFµ . "e
is a preset constant. In other words, the sufficient reduction critewhere
rion kU1ك is only imposed when the reduction
Ø in the model uk is sufficiently good
compared with GH k«§~ . In the situation where k&€8ƒ and uk kh¹7uk kK¢r&ksi¨€
pHGH kh~ , the new iterate k ! ­ Ú kK¢r&kh is accepted and the previous point k is
Definition 4. By adding
A globally convergent primal-dual interior-point filter method
11
GH kh¨^ .
k
Ø
k
k¦14ƒ u k kh
7‚uk kH¢r&ksiT1pHGH «k §~ Â
k ! ­ k }r&ks
k
G= k r k
r k
G
$„%v'”Z
G= khT^8r&k ` ­ §` ~ r¦Ûk \«
where ` ­ , ` , and Ü are preset positive constants. The restoration algorithm must pro~ k ­ that is not only acceptable to the filter but also satisfies G= k ­ RP
duce$&a%('”
new
Z iterate
!
r&k ` ­ §` ~ r¦Ûk \ . In! this situation, the previous iterate . k is added to the filter (guaranteeing also that this new filter entry satisfies G= ks*^
). In Section 6, we propose a
added to the filter (guaranteeing that this new filter entry satisfies
). This selection criterion of adding points
to the filter prevents us from building up a filter for
which the computation of acceptable points would require too small trust-region radii.
If
and
, the iterate
is not added to the
filter. This situation is the only one where a new iterate
is computed
and the previous one, , is not added to the filter.
is too large compared to
(or to an appropriate power of
), the alIf
gorithm enters a restoration phase with the purpose of reducing . More precisely, a
restoration algorithm is called if
restoration algorithm, based on the primal-dual interior-point framework of this paper,
that verifies the requirements of the restoration phase.
The primal-dual interior-point filter method satisfying the above features is now
is always
presented. Note that step 5 guarantees that the potentially new iterate
acceptable to the filter.
Notation. In the following algorithm, the current iterate in iteration is denoted
by
and the normal and tangential trial steps are denoted by and , respectively.
Further, the step sizes
and
are defined according to (13) and (14), respectively, with
and
. Similarly,
and
are defined by (15)
,
, and
.
and (16), respectively, with
k
k¢r&ks
t
jnk j 5k
® 5k ¢r& ®nk ¢r&
j 5 j 5k j8n "j nk k ¢r& j k ¢r& k j 5hÝ n j 5hk Ý n ® 5hÝ nN¢r& ® 5«k Ý n }r„
Algorithm 1 (Primal-dual interior-point filter method).
3 Œß ¡ 4 . "e
, Þ2 . e
. ` ­ ` ~ ^ . , . €Dܾƒwp[€Âe , . and `a| . eLŠ fhgh . Set
. Choose ,+Hàbi‰
à
Í^
and ˆsà , and determine `< "e
such that à"‰
àW1
I
3 /. choose Áá^ . such that Gsº= àL l G
¼" àYTP8Á|z”à .
`z”à with z”à +#. à ‰
àYfL . Further,
Choose 3 rWà âäã ^
and set t
.
1. Set zk +#k ‰YksfY and compute j 5k and jnk by solving the linear systems (11) and
(12), respectively, with z kzks .
2. Compute r Ok å . ]r&kâæã Æ such that
+#kH¢r&R^ . ‹‰LkK¢r&T^ . Š k¢r&§‰YkK}r&R1`zk¢r&ž for all rµå . ]r Ok Æ
and such that r Ok is not smaller than the largest Þç"rWkâäã , è C. "ehS"SS , having this
property.
J
3. Compute the largest r Ok O Þ r Ok , é <. e«SS"S , such that
GsºH kH¢r Ok O l G
¼
k}r Ok O iTP8Á|zk}r Ok O NS
3
Set r&k r Ok O .
0. Choose
12
M. Ulbrich, S. Ulbrich, and L.N. Vicente
$„%v' Z
k to the
GH k«mPŒr&k ` ­ §` ~ r¦Ûk \
k! ­
k ! ­ ¿ À`6wÁ with zk ! ­ +=k ­ ‰Yk ! ­ fL ;
!
k ! ­ is acceptable$„to%v' the
GH k ! ­ ¨3 Pr&k ! ­ 3 Z ` ­ filter;
§` ~ r¦Ûk ! ­ \ with r&k ! ­ r&k .
Set rWkâäã ­ r&k , t t l e , and go to step 1.
!
5. If k}r„k« is not acceptable to the filter (with k considered in the filter if uk k«N7
3
uk kK }r„khT€<pK G= ks ~ ), then
go to stepØ 11.
6. If uk ks¹7[uk k}r&ks /. , then set k D. . Otherwise, compute
Ø G … k 67[G … k }r k k uk kh67[uk kK¢r&khi S
Ø
7. If k €ƒ and u k k ¹7[u k k }r k R1<pHGH k ~ then go to step 11.
8. If uk ks¹7[uk k}r&ksT€<pHGH ks~ then add k to the filter.
9. Choose rW3kâäã ­ 18r&k .
3
!
3 go to step 1.3
3
10. Set k ­ 3 D k¢r&ks , t 3 t l e , and
!
11. Set k ­ / k , j 5k ­ j 5k , j"nk ­ j"nk , r Ok ­ r„kbfsg . Set t t l e and go
!
!
!
!
to step 3.
In practice, step 2 would be implemented as r Ok Fê k r  Ok , where r  Ok is the largest
Š
value of r such that ,+#kH¢r&Mw‰LkK¢r&1 . and k¢r&§‰Yk}r&1`zk¢r&ž and ê k is
a parameter in ¢Þ"e
to enforce ,+#kK}r&Mi‰Yk}r&m^ . . The adjustment of ê k would be
important to achieve a rapid rate of local convergence. We point out that the calculation
of r&k is split in steps 2 and 3 for good reasons. In fact, in step 2 it is possible to
determine explicitly r Ok (more precisely r  Ok ). However, because of the nonlinearity of
Gsº and G
¼ , that is not the case in step 3, where we know from Lemma 2 that indeed all
sufficiently small r Ok O verify G º k ¢r Ok O l G ¼ k ¢r Ok O i_PBÁ|z k ¢r Ok O but we cannot
4. If
then continue in step 5. Otherwise add
filter and call a restoration algorithm that produces a point
such that:
explicitly determine it.
Z
5. Global convergence to first-order critical points
kb\
For the rest of this paper we assume that
is a sequence of iterates generated
by the primal-dual interior-point filter method (Algorithm 1). We will also impose the
following assumptions.
Assumption 1.
Z
,+#kiˆ«ki‰YkYM\
o - o ~† ½ ‡
,+#kKˆ«kbw‰Lkh
Ëë^ .
t
is bounded.
(A1) The sequence
and
exist and are Lipschitz continuous in an open set
(A2) The derivatives
containing all the iterates
and the line segments
.
(A3) There exists
such that for all it holds
.
’ £ O xsì hk Ê ­ å P8k Ë k l jLk¢r&ks—Æ
Remark 2. (A3) holds in a neighborhood of a regular point ‚í satisfying the secondorder sufficient conditions and strict complementarity, see for instance [10]. Moreover,
conditions ¿ are given in [10] under which (A1) andZ (A3) are ensured if the iterates k
are kept in `]ÁÂ and only the boundedness of +#kb\ is assumed.
A globally convergent primal-dual interior-point filter method
13
As we will see, these assumptions will allow us to prove global convergence to KKT
points. Since KKT points are feasible, we are restricting the analysis to problems that
are not infeasible. It is the uniform boundedness of the inverse of the Jacobian given in
Assumption (A3) that rules out infeasibility.
ÕÖ
The following simple result is a direct consequence of these assumptions and of Lemmas 1 and 2.
Z Z Z
Z
G º k ]\ , G ¬ k M\ , z k \ , and G … k ]\ are bounded.
Á º wÁ ¬ wÁ ¼ wÁ … in Lemma 1 are bounded for all t .
r ù. Ä Å ^ . such that the conditions in steps 2 and 3 are satisfied
for
r Ok wr Ok O 8å wr Ã¹Ä Å Æ . Thus, steps 2 and 3 leave rWkâäã unchanged for . P|rWkâäã P
r Ã¹Ä Å $WVsX=Z r k r kâäã .
j 5k j"nk \ÍP8Ë&}Á l , ~T7[ l e
­ Ý ~"§zk for all t .
Š ‰‚7z ž Pُ ~¨7 ­ Ý ~Nz and §e¨7 ¡ —z ž P4 ­ Ý ~Nz .
For the result iv) we use Z that
Z Given the fact that + k ˆ k i‰ k ]\ is bounded,
the boundedness of the sequence
jb}r k ]\ follows by Lemma 3. (Note that j 5k and jnk are bounded by iv) and ® 5k
and ®nk do not exceed one.)
We start the convergence theory by showing that new iterates are always acceptable
to the filter and that all filter entries ,G«ÏKiG«… Ï obey GbÏ^ . , two facts that require no
analysis and follow directly from the structure of the algorithm.
Lemma 4. If k is added to the filter, then GH khR^ . .
$&%(' filter
Z either in step. 4 or in step 8. In the first case
Proof. An iterate k is added to the
(step 4), we see that GH kh¨^Dr&k
` ­ §` ~ r¦Ûk \¦^ . In the second case (step 8), we
conclude from Lemma 1, (21) that
GH k« ~ ^ p e ,uk kh7uk kK¢r&ksiT1 . S
ÕÖ
Thus, in both cases, GH k R^ . .
Lemma 5. In all iterations t¯1 . , the current iterate k is acceptable to the filter.
Proof. The proof is by induction. Since I is empty in iteration t <. , the initial iterate
à is certainly acceptable to the filter. Now let tU1 . and assume that k is acceptable
to the filter. The iterate k ­ is either generated in step 4, or in step 10, or in step 11.
!
If k ­ is obtained in step 4, then k is added to the filter and a restoration algorithm
!
is called that returns a point k ­ that is acceptable to the filter. If k ­ is obtained in
!
!
step 10, then the new iterate k ­ B k¢r&kh passed the filter acceptance test in step
!
5 (with k considered in the filter in the case where k is added in step 8). Finally, if
k ! ­ results from step 11, then k ! ­ ² k and k is not added to the filter. By the
induction hypothesis, k is acceptable to the filter. Since the filter is not changed, the
same holds true for k ­ D k .
ÕÖ
!
Lemma 3. The following hold:
i) The sequences
ii) The constants
iii) There exists
all
and we have
iv) It holds
14
M. Ulbrich, S. Ulbrich, and L.N. Vicente
The next three lemmas describe a few technical results that are needed to establish
global convergence to first-order critical points. The first of these lemmas provides a
crucial inequality showing that feasibility and centrality at
are of the order of
.
r„~k
Lemma 6. There exists a
k¢r&ks
r ç ^ . such that, if r&kmPr ç in step 5, it holds
GH k ¢ r k RPÙ¢Á º l Á ¬ r ~k S
Proof. If step 5 is reached, then
G= ksTP` ~ r k­ ! Û S
$&.%v' weZ get for all
­
„
$
v
%
'
Z
Thus, by (A3) and (11),
we have j 5k P8Ëî` r k ! Û . In the case j 5k /
~
r k ^ . that ® 5k eh]r k f j 5k \ e by our natural convention e«]³<\ e .
Otherwise, we have
r j 5k k 1 Ëî` e r¦Û S
~ k
We see then that ® 5k
e whenever
ñ
–™˜ ï Ëîe ` S
r„kmP<r ç •—0
~Kð
e for r k P8r ç . But then, by Lemma 1,
Thus, we have in both cases ® 5k
GH kK¢r&k«RPÙ¢ÁǺ l Á ¬ r ~k S
ÕÖ
Remark 3. We stress that Lemma 6 as well as the next two Lemmas 7 and 8 make
assertions on the situation in step 5 of the algorithm. Step 5 is preceded by step 4, and
thus, in step 5 always holds
G= khTP8r&k
$„%v'”Z
(25)
` ­ §` ~ r Ûk \«
since otherwise step 4 calls restoration instead of step 5. As we have already seen in
Lemma 6, (25) makes it possible to show GH kH¢r&ks Ž{ }rm~k for sufficiently small
r k , which would be impossible without (25) holding.
ÕÖ
&r k } r„k« k u k kh7duk k}r„«k î€pH}r„GH k« k«§~
k
k
The next two lemmas deal also with step 5 of the primal-dual interior-point filter method.
They provide sufficient conditions on the value of
for
to be acceptable to
the filter in step 5. In both lemmas we analyze the acceptability of
to the filter
considering that the filter contains
if
, despite
the fact that, in this situation,
will possibly be added to the filter only in step 8.
Firstly we consider the case where
is bounded away from a KKT point and the filter
has a finite number of entries.
A globally convergent primal-dual interior-point filter method
ò
G= ks l G
k«21Úò^ .
. €r k P8r ó t
15
Lemma 7. Suppose that
for all . There exists
depending only on and on the values of the filter entries, such that, if
r&ó8^ .
¢ r&kh is in step 5 acceptable to the filter (with k considered in the filter when
uk kh 7uk kK}r&ksT€8pKG= kh~ ).
Proof. Since . €4` a €/eLfhgm€e , we have from Lemma 4 that
$&%v'
GLa a §e¨7[`aR—G Ï ^ . S
Consider first the case where GH ksT18òMfhg . Then kK¢r&kh is acceptable to the filter
(with k considered in the filter when uk kh7uk kK¢r&ksiT€pHGH k« ~ ) if
$&%v' Z
$„%v'”Z
GH kK¢r&ksiTP ge G
aî"e¨7y`aTòNfsg\œ€
G
aî"e¨7y`aTòNfsg\bS (26)
then
We also know from Lemma 6 that
GH kK}r&ksTP|¢ÁǺ l Á ¬ ir ~k
­ö
­§ö
holds for r&k2Për . Thus, (26) is satisfied for r&kPBrõó ô with rõó ô ^ . depending
ç
only on GLa , ò and, of course, on ÁǺ , Á ¬ , `ba , and r .
ç
Otherwise we have GL… khR18òNfsg . If k is not considered in the filter in step 5, then
­ G a instead of (26), shows that if r k PÙr óô ­§ö
a similar argument, with GH k ¢r k i‚P
~
k ¢r k is also acceptable, with k
then k }r k is acceptable to the filter. Moreover
considered in the filter when u k k 7[u k k ¢r k ¨€pHGH k ~ , if, in addition,
(27)
G
kH¢r&ks¹7[GL… ksT€7T`bacGH k«MS
r&k
In the rest of the proof we show how this bound can be achieved for sufficiently small
. Since step 5 is reached, we know that
G= k TP` ~ r k­ ! Û S
On the other hand, we obtain from GL… ksœ1|òNfsg and Lemma 1 with . €Ùr„kUPÙr ÈLÉ
that
GL… k}r„k«67GL…b khRP|7_§e¨7 ¡ §® nk òNfsg l Ád…Lr ~k S
Hence we need to show that
7_§e¨7 ¡ §® nk òNfhg l Ád…Yr ~k €|7T`a` ~ r k­ ! $&Û S %('Z
e« ø q¹÷—ùì ì ø ø q¹÷—úì ì ø \ , we
Since j 5k and j"nk are bounded by a constant Á÷ and ®nk have for all r&kmP8Á ÷ , that ®nk 18r&khfNÁ ÷ . Thus (27) holds if
¡
¡
Ád…Yr„k l `a` ~ r¦Ûk P §e¨û 7 Á ÷ §ò € eQgh7 Á ÷ §ò ö
ö
which in turn holds for all r&k8PÚrõó ô ~ with rõó ô ~ ^ . depending $&
only
%v' Z on ­ò ö and öof
¡
course on the constants Ád… , `a , ` , Ü , , Á ÷ , and r ÈYÉ . Taking r&ó
rõóô wrõóô ~ \
~
concludes the proof.
ÕÖ
16
M. Ulbrich, S. Ulbrich, and L.N. Vicente
GH k«
r„k
Secondly we look at the case where only the measure of optimality is bounded away
from zero, but where we impose a condition relating
and
.
Lemma 8. Suppose that for given
ò*^ .
$„%v'”Z
G= k R^ r g k ` ­ §` ~ ¢r k fhgh Û \bS
G … k R18ò and
(28)
Then there exists r
Ï ^ . such that, if
. €<r k P8r Ï then ¢r k is in step 5 acceptable to the filter (with k considered in the filter when
uk kh7uk kK}r&ksT€8pKG= kh~ ).
Proof. Since, by Lemma 5, k is acceptable to the filter, k ¢r k is acceptable to the
filter (with k considered in the filter when uk ks¹7[uk k}r&khR€<pHGH kh~ ) if
G= kK}r„khTP4GH kh
and
G … k ¢r k iT€G … k 67y` a G= k NS
(29)
We know from Lemma 6 that, if r k P<r ,
ç
GH k ¢r k RPÙ¢Á º l Á ¬ r ~k S
Hence, GH kH¢r&ks¨PGH k« is ensured by the second inequality in (28) if in addition
$&%('”Z
¢Á º l Á ¬ r k P ge ` ­ §` ~ }r k fsgh Û \«S
(30)
Moreover, when . €r&kõPr ÈYÉ , we have by Lemma 1 and the first inequality in (28)
that
G … k ¢r k ¹7[G … k RP|7Q® nk §e¨7 ¡ §ò l Á … r ~k S
We have pointed out before that ®nk 1r k fNÁÇ÷ for all r k PÁÇ÷ , see the end of the
proof of Lemma 7. So,
¡
GL…b kK¢r&ksi67G
k«RP8r&k ï 7 eTÁ 7 ÷ ò l Ád…Yr„k S
ð­
Since we consider step 5 of the algorithm, we know that GH k«TPÇ` r k ! Û , see Remark
~
3, (25). Hence, we obtain (29) whenever
¡
¡
(31)
Ád…Yr„k l `a` ~ r¦Ûk P §e¨g«7 ÁÇ÷ §ò € eQÁÇ7 ÷ §ò S
$&%v' Z
The requirements . €/r„kWP
r ç wr ÈYÉ \ , (30). and (31) on r„k are obviously satisfied if . €8r&k¦P<r
with some constant r
ÕÖ
Ï
Ï ^ .
Now we are ready to derive asymptotic results. We%v$
appeal
used
%('ý first to a commonly
argument in filter convergence proofs to show that ü
kNþ_ÿGH kh . when infinitely many iterates are added to the filter.
A globally convergent primal-dual interior-point filter method
k
Lemma 9. Beginning from the moment where a point
always contains an entry that dominates .
k
k
k
k
17
k
is added to the filter, the filter
k
k tO ^ t
k
k t O O ^Ct O
Proof. Since
dominates
, the assertion is trivial as long as
remains in the
,
filter. If
is removed from the filter then it is replaced by an iterate
,
that dominates
. Thus, the assertion remains true as long as
stays in the filter.
is removed from the filter then it is replaced by an iterate
,
If
, that
dominates
and thus also dominates
by the transitivity of dominance. Thus, the
result follows inductively.
k
k
Z
k
v% $ k /
‘vüþ_ÿ G= k i . S
th‘\
ÕÖ
Lemma 10. Suppose there are infinitely many points added to the filter. Then there
exists a subsequence
such that is added to the filter and
Zt 3 k
(32)
Proof. Let infinitely many points be added to the filter and set
is added to the filter
\«S
GH khR^ .
tõ
t¯
ò¨^ . GH ksT1ò
t¯
k å GH k 67[` a ò"iGH k Ædå G … k 67y` a ò
iG … k Æ S
We prove that for all t 6
with t¯^
it holds
(33)
k ß S
In fact, at the time
where k is added to the filter, the filter contains an entry that dominates according to Lemma 9. In addition, k is acceptable to the filter
Assume (for deriving a contradiction)
that the assertion is wrong. Since by Lemma
4
for
all
,
we
then
find
with
for
all
.
holds
For
, define the square
by Lemma 5, so that at least one of the following inequalities (34) or (35) holds:
GH k R€GH 7y` a GH RP4G= 6 7[` a ò
(34)
67[` a GH RPG … 67y` a ò
S
or G … k R€G … (35)
In
either case, this implies (33). Therefore, all these Z infinitely many squares k , t8
, with area À`aRòM~ are disjoint. Since the sequence GH khMGL… k«]\ is bounded, we
obtain the desired contradiction.
ÕÖ
Z 4. Note that Lemma 10 asserts (32) only for some particular subsequence
Remark
k ]\ of iterates added to the filter and not for any such subsequence. The reason is
that acceptability of a pair does not imply acceptability of a dominated pair. In fact,
©
© eLfsgKeL . Then © is acceptable to © ­ and dominates © ­ ;
let ­ §eh"e
and
~
~
nevertheless, all points in the set åæe¨7`aîM³
e*7[`aQfsgh© are acceptable to
© , but are not acceptable to © ­ . Therefore, if © d­ å(iseQ7in`thea*filter
and
~
~ enters the filter,
©
then, by dominance, ­ is removed, and this can result in points that are acceptable to
the updated filter, but were not acceptable to the filter before the update.
If required, this effect can be circumvented in several ways. The easiest approach is
to never remove dominated entries from the filter. Then the above proof can be easily
18
M. Ulbrich, S. Ulbrich, and L.N. Vicente
modified to establish that (32) holds for any infinite subsequence of iterates that are
added to the filter. An alternative to derive this stronger result, if one wishes to remove
dominated filter entries, can also be obtained by modifying the filter acceptance test a
little bit, see [9, 15.5]. In fact, if we require
either
G Ï 7[GH ¨^` a G Ï
or
G …Ï 7G … T^4` a GH M then acceptability to a pair implies acceptability to all dominated pairs and it is straightforward to prove that (32) holds for any infinite subsequence of iterates added to the
filter, see [9, Lem. 15.5.2].
ÕÖ
%($%('Ký
%v$%('ý
Our next step is to show that in the case where infinitely many iterates are added to
the filter there exists a subsequence of iterates that converges to a KKT-point. In fact,
our previous result
can be extended to
. Since iterates are added to the filter only if restoration is invoked or in step
8, the sequence
of Lemma 10 must contain either a subsequence where restoration
is invoked, or a subsequence where the iterates are added to the filter in step 8. We start
with the first case.
.
GL… ks D
Z
t«‘§\
ü
kNþ_ÿG= kh Ú.
Lemma 11. Suppose that there exists an infinite sequence
restoration is invoked and for which holds
Z
ü
t ‘\
kNþ_ÿGH hk l
of iterations at which
%v$
ü‘vþ_ÿ G= ki /. S
Z
Z
Then t ‘ \ contains a subsequence t J O \ with
%v$
%($
üJ þ_ÿ G= k /. J ü þ¦ÿ GL… k <. S
%($ restoration
Proof. Let t ‘ be a subsequence where
is invoked for every t ‘ (and thus k Â
. . Since the restoration is invoked it
is added to the filter) such that ü ‘™þ_ÿ GH k w
$&%('Z
must hold
G k R^8r k ` ­ §` ~ r¦Ûk \bS
(36)
Therefore, we have
._ ‘™üþ_%v$ ÿ G k ‘™ü þ_%v$ ÿ r k and thus can find Wà^ . such that r&k„€ŽÞKrmÃ¹Ä Å for all t«‘Í1 à with rmÃ¹Ä Å from
Lemma 3.iii, with Þ¯d . "e
. We show next that r&k ­ Pghr&k for all th‘¹
1 Wà which
Ê%($
(
%
$
(
%
$
then yields
.¦ ‘™üþ_ÿ G k ‘™üþ_ÿ r k ‘™üþ_ÿ r k Ê ­ S
(37)
In fact, r&k €CÞKrmÃ¹Ä Å for th‘„1Wà shows that r&k r kâäã for t«‘m1Wà , since, by
Lemma 3.iii, step 2 and step 3 yield only r&k  × rWkâäã if rWkâäã ^CrmÃ¹Ä Å . But then the
result of step 2 and step 3 would be a radius r&k Í^FÞKrmÃ¹Ä Å , which is not the case for
t‘ 1 à . Thus, we have r k rWkâäã for t ‘ 1 à and conclude that r k ‚1Ùr k Ê ­ fsg
1 à . Thus, (37) holds.
for all t ‘ By (37) and Lemma 3.iii we know that for large enough  step 2 and step 3 do not
1 à with
change rWkâäã ­ and rWkâäã . Thus, we find ­ Ê
(38)
r&k Ê ­ r kâäã Ê ­ ár&k r kâäã for all th‘74eœ
1 ­.
A globally convergent primal-dual interior-point filter method
19
th‘67e1
­
rWkâäã r&k Ê ­
&r k „r k Ê ­ h
t
‘
7<e
&r k r„k Ê ­
k
$„%v'”Z
$&%v'”Z
Gsk ¾P<r&k Ê ­ ` ­ §` ~ r¦Ûk Ê ­ \ &r k ` ­ ` ~ r_Ûk \«
which contradicts (36). Hence, step 5 is reached for all t«‘7ÇeÍ1 ­ and thus in particular
$&%v'”Z
GYk Ê ­ P<r„k Ê ­ ` ­ ` ~ r¦Ûk Ê ­ \«S
For the purpose of deriving a contradiction, suppose that GL…b k w[1 ò^ . for
t«‘œ1 ~ with some sufficiently large ~ 1 ­ . We show next that then there exists
œ
1 ~ such that
(39)
GL… k Ê ­ ¨14òNfsg for all t«‘¹
1 sS
In fact, we have either k C k ­ or k C k ­ }r k ­ . In the first case (39)
Ê
Ê
Ê
is obvious since then G … k ­ G … k &1ëò for all t ‘ 1 . In the case k Ê
k Ê ­ ¢r k Ê ­ it follows from Lemma 1 that for . €<r k Ê ­ P<r„~ÈYÉ
G … k G … k Ê ­ ¢r k Ê ­ ¨P|e¨7® nk Ê ­Y§e¨7 ¡ §G … k Ê ­ l Á … r ~k Ê ­Y
and thus for th‘6
1 ~
ò*P4GL… k iRPGL…b k Ê ­ l Ád…Lr ~k Ê ­ S
We can therefore conclude from (37) that (39) holds for œ
1 ~ large enough.
1 Next, we show that step 7 must be reached for all iterations t ‘ 7e with t ‘ 7eÍ
1 large enough. In fact, let r Ï be the bound of Lemma 8 corresponding
and U
1 such that r k ­ PDr holds for all
to òNfhg instead of ò . By (37) we can find m
t ‘ 7eœ1 . Now assume that step 7 is not reached in iteration t ʑ 7 eœ1Ï . Then step
G k Ê ­ , and, using (38), r k r&kâæã r k Ê ­ fhg .
5 is followed by step 11 and thus G k Therefore, by (36),
$&%v'”Z
G k Ê ­ ^ r&k g Ê ­ ` ­ ` ~ ¢r k Ê ­ fsg« Û \«S
Hence, we obtain from Lemma 8 that k ­ ¢r&k ­ was acceptable to the filter in step
Ê step 5 would not have branched
5, since t«‘=7eÍ
1 ensures r&k Ê ­ P8r Ê Ï . Therefore,
to step 11 as assumed. Hence, step 7 is always reached in all iterations t«‘”74eÍ
1 .
We conclude the proof by showing the existence of 1! such that step 9 is
1 . This assertion leads to a contradiction:
reached for all iterations t ‘ 7&e with t ‘ 7&eœ
by (38) and steps 9, 10 we have r k rWkâäã 1<r k ­ , k < k ­ ¢r k ­ . Thus, we
Ê
Ê
Ê
obtain by Lemma 6 for all r k ­ P8r (which holds by (37) for all  large enough)
ç
Ê
G k GH k Ê ­ ¢r k Ê ­ ¨P|}Á º l Á ¬ ir ~k Ê ­ P¢Á º l Á ¬ ir ~k S
We show next that step 5 is reached in all iterations
. In fact, otherwise
the restoration procedure is called in iteration
. Thus, we have
and
consequently by (38). Since by our assumption the restoration is invoked
in iteration
, by using the outcome of the restoration is an iterate
with
th‘#7Çe
This contradicts (36) and (37).
20
M. Ulbrich, S. Ulbrich, and L.N. Vicente
It remains to show that step 9 is eventually reached in all iterations t«‘¾7Âe with
t«‘c7Âe41" , 1" large enough. We note that by (39) and Lemma 1 for all
t«‘¹
1 u k Ê ­ k Ê ­ 67u k Ê ­ k Ê ­ ¢r k Ê ­ iT14® nk Ê ­ §e¨7 ¡ gò 1<r k Ê ­ §e¨7 ¡ g«ÁÇò ÷ S
Hereby, we use again the fact that ®nk ­ 1Br&k ­ fÁ ÷ if r„k ­ PBÁ ÷ , which holds
Ê
Ê
Ê
by (37) possibly after increasing . On the other hand, we have
#
$ }r ~k ­ MS
u k Ê ­ k Ê ­ M7¨u k Ê ­ k Ê ­ }r k Ø Ê ­ N7¨G … k Ê ­ l G … k Ê ­ }r k Ê ­ # %
Ê
The last two inequalities show that k ­ 9Ìe and hence there exists 1 such
&
Ê
that step 9 is reached in all iterations t«‘”7e with th‘ 74eÍ1 .
As we have already seen, this leads to a contradiction with G … k wRZ 1ò for Z all t ‘ 1
$
since there exists a subsequence t J O \_A
t«‘\ for
~ . The%v$ proof is therefore %vcompleted
which ü J þ_ÿGH k ü J þ¦ÿG
k D. .
ÕÖ
Z
The other situation is where the sequence t ‘ \ of Lemma 10 contains a subse-
quence, where the iterates are added to the filter in step 8. Analogously to the previous
Lemma we have the following result.
Z
Z
t«‘—\ of iterations for which
k
Z
ü ‘™þ_ÿGH ki . . Then th‘\
%v$ t J O \ /.
%($
üJ þ_ÿ G= k J ü ¦þ ÿ LG … k <. S
Z
Proof.%($ Let t ‘ \ we a sequence of iterations such that k is added to the filter in step 8
and ü ‘™þ_ÿ GH k w /. . Suppose now that G … k i¨1ò*^ . for all t ‘ 1 à for some
. . By Lemma 1 and since k is added to the filter in step 8 we have
à 1
® nk §e¨7 ¡ §òîPuk N k w7[uk N k ¢r&k iiR€8pHGH k ~ S
Thus, we obtain ®nk 9 . and consequently r&k _9 . . In particular, ®nk 1Žr„k wfNÁ ÷
for large enough  , and since the restoration procedure is not called, we have G= kTP
` ~ r k­ ! Û and conclude that
r&k NeQ7 ¡ §òNfNÁ ÷ Puk M k w7[uk N k "¢r&k iR€8pÀ` ~ r k­ ! Û ~
ÕÖ
which is a contradiction to r k 9 . .
%v$
Lemma 12. Suppose that there exists an infinite sequence
is added to the filter in step 8 and, in addition,
contains a subsequence
such that
We summarize both situations in the next theorem.
Z
tLJs%v$ \
.
J ü þ_ÿ G= k /
Theorem 1. Suppose that infinitely many iterates are added to the filter. Then there
exists a subsequence
such that
%($
.S
üJ þ¦ÿ G … k <
A globally convergent primal-dual interior-point filter method
%v$
Z ü ‘vþ¦ÿZ GH k i µ. Z t J O \ Z t«‘—\
kk
t JO \ t ‘ \
Z
thԤ\
21
k
Proof. By Lemma 10 there exists a sequence
of iterates such that is added to
the filter and
. As we have already observed there exists either a
subsequence
of
such that are added to the filter in the restoration or a
of
such that are added to the filter in step 8. In the first case
subsequence
the assertion follows from Lemma 11; in the second case from Lemma 12.
ÕÖ
It remains to consider the case where the algorithm runs infinitely but the filter is
left with a finite number of entries.
Theorem 2. Suppose that the algorithm runs infinitely and only finitely many iterates
are added to the filter. Then
%($
. ü %vk$4þ_%vÿ 'ý GL… kh /. S
üNk þ_ÿ GH sk <
Proof. The assumption says that for t|1' , with large enough, no further filter
entry is added. Hence, the filter contains for all t2%
many entries,
1 the same finitely
and the restoration is never invoked. Thus, all new iterates k ­ <
are computed in
×
k
!
step 10. We now show that step 10 is reached infinitely many times.
r k .
GH k R^
u k k 67u k k ¢r k iR€8pHGH k ~
for r&k sufficiently
and step 8 is reached. Otherwise, GH k« Ž. and G
kh_^
. and therefore Ø kÙsmall
1 ƒ for all r&k small enough (we may apply exactly the same
arguments as at the end of the proof of Lemma 12). So, step 10 is always reached after
finitely many reductions of r&k , producing always new iterates.
In fact, step 5 is reached in each iteration, and, by Lemma 7, step 7 is reached after
in step 11. Again, step 8 is reached after finitely many
finitely many reductions of
reductions of
. In fact, if
then clearly
r k
Since no further entry is added to the filter we know, cf. step 8, that in step 10 it
always holds
G … k 7G … k ! ­ R14ƒ”u k k 6Z 7[u k k ¢r k iT14ƒpHGH k ~ S
Since this holds for all successful steps and GL…b sk ]\ is bounded, we conclude that
%($
ükNþ_ÿ GH kh D. S
1 and some òQ^ . . Since the filter entries
Now assume that G … k R1ò*^ . for all t¯
Ø r k P|r ó (cf. Lemma
1 , the test in step 5 is passed whenever
do not change for ty%
7). Also, since GL… ksœ1Ùò_^ . , we obtain as before that k¯1ƒ whenever r&kUPÂr O
%('”Z for rWkâäã PFr¦Ã¹Ä Å steps 2 and
for some r O ^ . . Finally, we know by Lemma 3.iii$&that
3 yield r„k rWkâäã . Hence, we see that r&kõ%
1 ( •— –™˜ r&ósfsg]r O fsgKwÞKrm. Ã¹Ä Å\&^ . for
t¯
1 . Thus, step 10 is reached for all successful steps with r&k¦
1 („^ and we have,
as above,
G … k 67G … k ! ­ R1ƒ”u k k 7[u k k ¢r k iT1ƒe¨7 ¡ ò]® nk
$&%('W° (
1ƒ”§e¨7 ¡ §ò
Á $&÷ VseX±Z where Á ÷ is as before a uniform upper bound on
j"nk j 5k \ . This is again a
contradiction to the boundedness of GL… ks and the proof is complete.
ÕÖ
22
M. Ulbrich, S. Ulbrich, and L.N. Vicente
The main result is obtained by combining Theorems 1 and 2:
Corollary 1. Under Assumption 1, the sequence of iterates
primal-dual interior-point method (Algorithm 1) satisfies
Z
%v$%('ý
.S
ü kNþ_ÿ GH hk l GL… ks <
k\
generated by the
6. A restoration algorithm
In this section we present restoration algorithms that can be used in step 4 of the
primal-dual interior-point filter method (Algorithm 1). The purpose of a restoration
algorithm is to find a point
acceptable to the filter and such that
with
. In a restoration algorithm, it is
therefore desired to decrease the value of
. To achieve this goal
we introduce the function
$&%(' Z
GH k ! ­ T P8r k ! ­
k ! ­ ¿ `]ÁÂ
` ­ §` ~ r¦Ûk ! ­ \ r k ! ­ r k
G= sG º= l G ¬ )
G ~ •— ™– ˜ ge ´ G º ~ l G ¬ ~ ¸ ge - + ~ l Š œ‰ 7[z ž +~ * S
6.1. A Restoration Algorithm based on the KKT-Newton-System
The normal step
j5
G ~ . In fact,
Š ‰œ7z žY ¥ r¦+ 5 l Š r„‰ 5 l - + o -,+# rm+ 5
7_ Š ‰œ7z žY Š ‰Í7z žY7d-+ -,+#MS
7*gsG ~ , and j 5 is, in fact, a descent direction for G ~ . One
computed from (11) is a descent direction for
o G ~ j 5
o G ~ j 5 Š ‰œ7z žL eQ7 ¡ § z ž §e¨7 ¡ —z,”zU7”z /.
that the tangential step (12) yields o G §6jn D. . We summarize these two properties
~
Thus,
can also show using
for future reference:
o G ~ j 5 *7 gsG ~ N
.S
o G ~ j n /
(40)
restoration algorithm that we present works with the step framework ¢r& l The
® 5 ¢r&j 5 l ®nM¢r&j"n , where ® 5 }r& , j 5 , ®nN¢r& , and j"n are given by (13), (11), (14),
and (12), respectively. Several other restoration algorithms are plausible but we chose
the following one because it is consistent with the step calculation of our primal-dual
interior-point filter method.
A globally convergent primal-dual interior-point filter method
23
. %ve
' Z kà 3 8 k r àk 3 r&k é 3 D.
w
¯
Þ
d
&
$
k ! ­ 3
G J = Jk QP/r k ` ­ ` ~ r¦Ûk \ Jk
k J 3 J J
J
nJ
z k ,+ k ‰ k fY
z j 5k Jk iz jJk k r Jk . wr kâäã J Æ
+ Jk ¢r&R^ . ‰ Jk ¢r&T^ . Š kJ ¢r&§‰ kJ }r„¨1`z Jk ¢r&§ž for all rdå . ]r Jk Æ (41)
J
J
and such that r k is not smaller than the largest value Þç"r kâäã - , è <. e«SS"S , having
Algorithm 2 (Restoration algorithm).
0. Choose ,
. Set
,
,
and start with step 4.
1. If
and
is acceptable to the filter then set
and stop restoration.
and compute the steps .- and - by solving the linear
2. Set
systems (11) and (12), respectively, with
.
- such that
3. Compute
this property.
4. If
G ~ Jk ¹7[G ~ Jk ¢r Jk iT1|7, o G ~ Jk j Jk ¢r Jk N
(42)
J
J
J
J
J
J
G º k ¢r k l G ¼ k ¢r k iTP<Á|z k ¢r k M
(43)
J ­ J
J ­ 3 / Jk ¢r Jk , é 3 é l e , and return to step 1.
then choose r kâäã - ! 1<r k , set 3 k !
J ­ J
Otherwise set r k ! r k fsg , é é l e , and repeat step 4.
This restoration algorithm terminates successfully in a finite number of iterations as
we prove in our final theorem. In analogy to Assumption 1 we require
Assumption 2. Assumption 1 holds for
respectively.
k , j k ¢r k , z k
replaced by
Jk , j Jk ¢r Jk , z Jk ,
Theorem 3. Under Assumption 2, the restoration algorithm 2 terminates in a finite
number of iterations.
j 5k J
j nk J
Proof. Let us first consider the well-definedness of the algorithm. Hereby, the only
critical issues are the computation of the trial steps /- and - , which is possible by
Assumption 2 (A3), and the computation of
in step 3, which is possible by Lemma
2. Thus, the algorithm is well-defined.
For the rest of the proof, assume that the restoration algorithm does not terminate
finitely. Let
r Jk
$&%v'
G a a e¨7y` a §G Ï S
` a Ú . "eLfhgh , we have from Lemma 4 that GLa ^ . , and Jk is acceptable
J
J
J
to the
$&%('”filter
Z if G= k 4P g10 G ~ k P GLa¨fhgF€ GLa . This condition and GH k ÇP
r&k ` ­ §` ~ r¦Ûk \ are eventually satisfied if
%($%('Ký
ü J þ_ÿ G ~ Jk <. S
(44)
Since
24
M. Ulbrich, S. Ulbrich, and L.N. Vicente
Hence, if the restoration does not terminate finitely, then there exists an ò‚^ . with
G ~ Jk ‚1ò for all é . We show that this uniform bound will lead to a contradiction. In
fact, from (40), we have
o G ~ Jk j Jk ¢r& ® 5.k - J ¢r&J o G ~ Jk J j 5/k - J l ® nk - J ¢r& o G ~ Jk j nk - J
7*gs® 5/k - ¢ r&—G ~ k NS
Moreover, there exists a constant Á
~ ^ . such that
G ~ Jk ¹7[G ~ Jk ¢r&i 7 o G ~ Jk j Jk ¢r&67Á ~ j Jk }r& ~ J
5k - J ¢r& , implies
which in turn, appealing to ® nk - }r&RP® /
G ~ Jk 67[G ~ Jk }r„iT1/7 o G ~ Jk j Jk ¢r&67ÇghÁ ~ ® .5k - J ¢r& ~ j .5k - J ~ l j nk - J ~ NS
5k - J ¢r& such that
Hence, (42) holds for all ® /
gH§e¨27 ,h§® /5k - J }r„§G ~ Jk T1g«Á ~ ® /5k - J ¢r& ~ j /5k - J ~ l j nk - J ~ M
5k - J ¢r& such that
i.e., for all ® .
$&%(' 5
J
® 5/k - J }r&RP4® 3 5/k - J •— –™˜
eh Á e¨j /5k 67 - J ,h §~ G l ~ j k nk - J ~ 7 S
~
¿From Lemmas 2 and 3.iii, we see finally that (41), (42), and (43) are satisfied for all
r Jk such that
. €<r Jk P $&%(9' 8 r Ã¹Ä Å / ® 3 /5k - J j .5k - J ; : J
showing that all these conditions are satisfied after finitely many reductions of r k in
step 4.
$&VsXZ J J J
Now, if G k T1òQ^ . holds for all é , then, since
j /5k - j nk - \ÍP<Á ÷ ,
~
$„%v' 5 r Ã¹Ä Å J
® /5k - J }r Jk T1 ge
j 5/k - J /® 3 /5k - 7 14®3 ^ .
. , and we conclude that
3 ^
for some ®
G ~ Jk 7G ~ Jk ¢r Jk ¨18<g ,=®3 ò
ÕÖ
which yields a contradiction. Hence, we have (44) and the finite termination is proved.
A globally convergent primal-dual interior-point filter method
25
6.2. Other restoration procedures and the Wächter-Biegler example
The restoration algorithm 2 uses the same step computation as the interior-point filter
method. Global convergence is attained as long as Assumptions 1 and 2 hold. In practice
it may happen that Assumption (A3) is violated, affecting not only Algorithm 1, but also
the restoration algorithm 2. We will briefly discuss here the main issues related with this
situation and leave a more detailed study for a forthcoming paper.
For the purpose of this discussion, let us consider the problem proposed by Wächter
and Biegler in [25], which takes the following form:
$„%v' )
D.
D.
.
† >@?A + s.t. + ~­ 7+ ~ lCB ¯+ ­ 7[+D¨7FE ‹+ ~ i+Gœ1 (45)
<
with B 2 and E‚1 . . The particular form of the (sufficiently smooth) objective is not
relevant since the interesting properties of the problem are generated by the constraints.
For brevity, we set - ­ + +#~­7œ+ l9B and - ,+# + ­ 7‚+ 7HE . The problem is nonde~
~
generate, because o -+# has full rank and its condition number is uniformly bounded
on sets where + ­ is bounded. Nevertheless,
shown in [25] that
3 º it†JI was
$&%v' forZ . every initial
ö
à
.
.
.
K
K
point + 7î³< =
õ ]³4§~ with è º ôô † I ö 1 and B 7 <è Eîk P ­
7 B fhg\ any
method fails to converge if it computes the new primal iterate + ! from the old point
+ k . ]³4§~ by a step of the following form:
+ k ! ­ + k l ¡ ksj k . M³ ~ ¡ km . eÆ}
(46)
k
k k
k
where j satisfies -,+ l4o -,+ j /. S
It is straightforward to see that Algorithm 1, which can be trivially adapted to the
fact that + ­ is not subject to nonnegativity constraints, is contained in this class, as
long as restoration is either of the same type or restoration is not called. In fact we
have run this problem using our Matlab implementation of Algorithm 1, combined
with Algorithm 2 for the restoration, and we have observed bad numerical behavior.
We have seen that the iterates approached a point where Assumption (A3) is violated
and that this was the cause for such bad numerical performance.
One way to overcome such situation is obtained by using a restoration algorithm
that terminates successfully even if assumption (A3) is violated. To illustrate this point,
we have considered a restoration procedure that does not belong to the class (46). This
alternative is based on the optimization problem in the variables and
+ ‰
$&%(' e ´ Š
¬ ž ~ ¸ s.t. +1 . ‰&1 . ~
(47)
,
#
+
œ
‰
7
z
l
g
¬
where z •§ –™˜ zk denotes the value of z when restoration is entered. Analogously, we
¬ + ¬ ˆ ¬ i‰ ¬ =•— –™˜ + k ˆ k i‰ k the iterate for which restoration is entered.
denote by Note that if there exists a strictly feasible point of (45), i.e., a point + such that -,+# /.
¬ Š Ê ­ ž and obtain a global minimizer of (47).
and +<^ . , then we can choose ‰ z
Conversely, if there exists a strictly feasible point then any global minimizer +i‰
Š
¬
satisfies -+ ë. , +w‰^ . , and ‰_7Çz ž F. . Therefore, any solver that is able to
find a global minimizer L +
3 <
‰«3 of (47) for which, in addition,
(48)
Gsº=L+3 ˆ ¬ M‰«3 l G
¼
N+3 ˆ ¬ <‰b3 RP8Á|z ¬
26
M. Ulbrich, S. Ulbrich, and L.N. Vicente
k ! ­ +ˆ ¬ ‰b
¬ ~ ,+o„i‰†L‡ 7++ ˆ ¬ ii‰ ؉¬ ~Ø ~
~
­ ~
holds, provides a valid restoration procedure (with
L3
M3 ). In order to include a device to enforce (48) we tried (among many other possible options) to augment
the objective function in (47) by the penalization term O
. In addition,
it can be advantagous to add the regularization term O
to make the
Hessian of the restoration problem positive definite. Both parameters , are chosen
small, i.e.,
. Since these modifications may lead to non-interior solutions,
P
i.e.,
or
for some , we move the nonnegativity constraints slightly. The
resulting problem is
. P Ø ‘ ‹e
.
+ ‘ B. ‰ ‘ B

$„%v' e ´ Š
Ø Ø ¬
¬
­
~
~
~
+
œ
‰
[
7
z
ž
,
+
i
ˆ
w
b
‰
l
l o †‡
l ~ +i‰7,+ ¬ i‰ ¬ ~ ¸
g
1 (s
s.t. +w‰&
where (&^ . is very small
Ø and should be chosen, e.g., depending on the value of z ¬ . If
Ø
required,
the value of can be adjusted during the minimization process. The value of
­ is kept small, but large~ enough to ensure (48). We applied Bertsekas’ projected New-
ton method to solve this problem in the context of (45). We have numerically observed
that Algorithm 1 using this alternative restoration procedure (adapted to the case where
is unrestricted) was able to successfully solve problem (45).
The use of new alternatives for the restoration procedure, including the one presented above, is subject of ongoing research. Another alternative is for instance the use
.
of the restoration algorithm 2 but with a suitable regularization for the matrix
One should also consider a modification of the Algorithm 1 so that restoration is also
called if
exceeds a prescribed, very large bound
– larger than we expect or wish the constant to be in assumption (A3). If the restoration procedure is able
to find a new point
at which, besides the requirements of step 4 in Algorithm 1,
satisfies (A3) with a reasonable value of
, then the chances of
RQ
satisfying assumption (A3) will be improved so that the interior-point filter method can
continue successfully.
+­
’ £ O x ì khMÊ ­ Ë
­
k
’ £ O xsì k ­ !
!
˜O
’ £O x ËڀëË O
7. Concluding remarks
The filter mechanism has been used for the first time to globalize primal-dual interiorpoint methods. Global convergence to first-order critical points has been proved, and
the main result has been reported in Corollary 1. (Since the appearance of this paper,
two others have also appeared in the topic: [3], [26].)
The combination of interior-point and filter ideas led to a new class of algorithms.
We have already tested our primal-dual interior-point filter method in Matlab for QP
problems and small-scale NLP problems. The results are encouraging but there is still
some work ahead. We are currently working on a fortran 90 implementation of our
primal-dual interior-point filter method and we plan to report numerical results in our
future paper on the topic.
This paper is hopefully a first step in this challenging topic. Several issues need to
be addressed and better understood, and among them we highlight the following there.
A globally convergent primal-dual interior-point filter method
27
The new primal-dual interior-point algorithm used a 2D filter: one dimension for
feasibility and centrality combined and the other for the size of gradient of the Lagrangian (with complementarity added). An open question is the use of 3D filters. In a
3D filter, one could use the first dimension for feasibility, the second for centrality, and
the third for the size of the gradient of the Lagrangian.
As we have mentioned in the introduction, the rate of local convergence is well studied in the literature for primal-dual interior-point methods under the standard assumptions, and even in cases where some of these standard assumptions do not hold. The
open issue in interior-point filter methods is whether the globalization scheme (where
the filter plays an important role) becomes locally inactive to allow fast convergence
rates.
Another interesting topic for future research is the choice of alternatives for the
components used in the filter. As already mentioned, it would be desirable to replace
the optimality measure
by a function that reflects better the goal of minimizing .
Essentially, an appropriate candidate should be a function for which the tangential step
yields a fraction of Cauchy decrease close to the quasi-central path.
)
G
j"n
Appendix
¢r&
The following lemma measures the decrease on complementarity obtained by the new
iterate
and is needed to prove Lemmas 1 and 2.
r ^ . and all  e«S"SSNi it holds

+=‘w¢r&§‰L‘w¢r&TPeQ7d® 5 }r„i—+#‘}‰L‘ l ´ ® 5 }r&67® n ¢r&§eQ7
+=‘w¢r&§‰L‘w¢r&T1eQ7d® 5 }r„i—+#‘}‰L‘ l û ´ ® 5 }r&67® n ¢r&§eQ7
z¢r&TP ´ e¨7d® n }r„§eQ7 ¡ ¸ z l r ~ ‹z¢r&T1 ´ e¨7d®
Lemma 13. For all
¡ ¸z l û r
¡ ¸ zU7 û r
n }r&N§e*7 ¡
~
(49)
~ û (50)
¸ zU7 r ~ S
(51)
j 5 and j n , we have
+ ‘ l ® 5 }r&rm+ 5‘ l ® n }r&rm+ n‘ ,‰ ‘ l ® 5 }r&r„‰ ‘5 l ® n }r&r„‰ ‘n +=‘}‰L‘ l ® 5 }r„,‰L‘ r¦+ 5‘ l +#‘}r„‰ ‘5 l ® n ¢r&,‰L‘}rm+ n‘ l +=‘ r„‰ ‘n l ,® 5 ¢r&rm+ 5‘ l ® n ¢r&rm+ n‘ N,® 5 }r&r„‰ ‘5 l ® n }r„ir„‰ ‘n + ‘ ‰ ‘ 7d® 5 }r„+ ‘ ‰ ‘ 7z67d® n }r„§eQ7 ¡ —z l + ‘ }r&67[+ ‘ ,‰ ‘ ¢r&67d‰ ‘ MS
Proof. By the definition of
+ ‘ ¢r&‰ ‘ ¢r& So, inequalities (49) and (50) follow from this derivation and
+ ‘ ¢r&67+ ‘ #S# ‰ ‘ }r„¹7‰ ‘ # P|}ghr& ~ S
Summing (49) and (50) over all  , dividing the result by  , and using z +#‰fL ,
z¢r& +}r&§‰=}r„wfL , yield (51).
ÕÖ
#
We can now prove Lemmas 1 and 2.
28
M. Ulbrich, S. Ulbrich, and L.N. Vicente
Proof of Lemma 1
G¼
¼
˺
Ë ¼ ^e
Proof. Denote by T an upper bound for , and by
and for
and
, respectively. We will prove Lemma 1 with
o -
o ~† ½ ‡
Lipschitz constants
Á º «g Ë º Á ¬ %U;û V 6
Á ¼ ghË ¼ Á … e l T ¼ Ë ¼ l û Ë ¼ ~ r ~ÈLÉ S
We note that o -,+#§6j † }r„ 7Q® 5 ¢r&-+ , and thus
­
G º }r& -+¢r& ²»»» -,+# l
W à o -+ lFX j † }r& j † }r„RY X »»»
»
»
²»» §e*7® 5 ¢r&i-,+# l W ­ ´ o -,+ lCX j † ¢r&¹7 o -+# ¸ j † }r&ZY X »»
»»
»»
à
­
P|§eQ7d® 5 ¢r&§GYº# l ËTº j † ¢r& ~ W à X Y X o ~† ½ ‡ jb¢r& 7Q®nM}r„ o † ‡ and, as above, we get
­
G ¼ ¢r&QPeQ7® n ¢r&§G ¼ l
W à o ~† ½ ‡ lCX jb¢r&7 o ~† ½ ‡ « jb¢r& Y X which proves (17).
Similarly, we have
which yields (19).
The estimate (18) follows from Lemma 13:
[
§e¨7d® 5 ¢r&§+ ‘ ‰ ‘ l ´ ® 5 ¢r&67d® n }r&N§e*7 ¡ i¸z * l û r ~
\ ´
n ¢r&NeQ7 ¡ i¸"z l û r ~
¨
e
d
7
®
[ eR7® 5 ¢r&+#‘—‰
‘7z l U r ~ S
´ + ‘ ¢r&§‰ ‘ ¢r&67z¢r&¸_P
[
)
Inequality (20) is derived by appealing to Lemma 13 and to the previously established inequality (19):
GL… }r„i z ¢r& l G
¼
¢r& ~
P ´ e¨7d® n }r&N§e*7 ¡ i¸z l û r ~
P ´ e¨7d® n }r&N§e*7 ¡ ¸ G … l
l û ´ e¨û 7® n ¢r&§G ¼ l ghË ¼ r
´ l § e¨7d® n ¢r&§G ¼ ¸ Ë ¼ r
~¸~û
~ l Ë ¼~ r S
Finally, we have
ud 7[ud }r& z 7[z}r„ l ,+¢r&7[+ ,‰#¢r&6 7d‰bwfL
l §e¨78§e¨7d® n }r„i ~ o † ‡ ~ ® n }r&N§e*7 ¡ §z l §e¨7<e¨7® n ¢r& ~ o † ‡ ~
1® n }r&N§e*7 ¡ §G
M
and the proof is therefore completed.
ÕÖ
A globally convergent primal-dual interior-point filter method
29
Proof of Lemma 2
Proof. We will prove Lemma 2 with
° ¡
¡
r Ã¹Ä Å û §e l `”§ ieQË&7[¢Á `” l  }ÁǺ l Á¼ l û Á ÁÂË&}Á l  ± S (52)
1. We first show that (22) holds for all rµd . ]rmÃ¹Ä ÅLÆ with rmÃ¹Ä Å satisfying (52).
From Lemma 13 we obtain
Š ¢r&‰=}r„R1 ´ ` l eQ7y` §® 5 ¢r&67d® n ¢r&NeQ7 ¡ ¸ z žQ7 û r ~ žbS (53)
On the other hand, Lemma 13 also yields
#` z}r&RP`z¯7[`”® n ¢r&§eQ7 ¡ —z l û ` r ~ S
Š
Hence, ¢r&‰=}r„R14`z¢r&§ž holds whenever
û r ~ P §eQ7[` N,® 5 ¢r&7®nM¢r&NeQ7 ¡ i—z S
el `
Since ®nN¢r&TP8® 5 }r„ , a sufficient condition for this inequality to hold is
¡
r ~ P ® 5 }r&û e l §eî` 7y ` —z (54)
which, by (13), is implied by
r²P
which in turn is true if
$„%v'
$&%('5^] ¡ e¨7y` —z ¡ § eQ7[`”§z
û §e l `” û e l ` j 5 7 ¡ § e*7y` —z
¡
û e l ` û §e l `”§ieQË&7[¢Á `” l  7 •§ –™˜ ( ­ zN
­
since j 5 P8Ë&¢Á l  ~¨7 Ý ~"—zdP8Ë&¢Á l  §z .
­
On the other hand, we can also deduce that jn P<Ë&¢Á l  Ý ~§z[P<Ë&¢Á l —z ,
and therefore
$WVYXZ j 5 j n \ÍP<Ë&¢Á l §z¹S
(55)
( •—–™˜
We consider now two possible cases in order to show that (22) holds whenever
$&%('
(56)
¢r¯ (sT
P ( ­ zNS
In the first case r `
P ( we use that, as we have just shown, (22) holds provided r P
to (56). In
( ­ z . Since r>
P ( in this case, the inequality
r: P ( ­ z is Eequivalent
the case (€ r , we know that ® 5 ¢r&
, ¢r&
®
M
n
¢
&
r
e
a (s , and jb¢r&
Š
jba(s . Thus, (22) is the same as a(s‰=b(s‚1/`za(s . One can Šapply the same algebraic
arguments used in the first paragraph of the proof to show that a (s‰=b (sR1`za (s holds
if (mP( ­ ,z . Now, since („€r , we have that (¦P( ­ z is also equivalent to (56).
r²P
5_]
30
M. Ulbrich, S. Ulbrich, and L.N. Vicente
r²då . wrmÃ¹Ä ÅLÆ with rmÃ¹Ä Å according to (52). Let
¡
z ce­ d Ä f •— –™˜ û §e l ` §Ë eQ~7[¢Á ` l  ~ S
­
If z4Pz ced Ä f , then ( ­ z is given by its first expression, and (56) follows directly from
­
­
(55), since by the definition of z ced Ä f for z[Pz ced Ä f with (55) holds
] ¡
(mP<Ë&¢Á
l §z[P 0 Ë ~ ¢Á l  ~ z ce­ d Ä f z û eQe 7yl ` ` — z ( ­ ,zMS
­
If z^4z ced Ä f , then ( ­ ,z is given by its second expression, and (56) holds if
$&%('
¢r¯ (sT
P ( ­ z ( ­ z ce­ d Ä f which is true if
¡
r²P
( ­ z ce­ d Ä f û §e l `”§ieQË&7[¢Á `” l  S
2. We prove now that (23) holds for all rµd . ]rmÃ¹Ä ÅLÆ with rmÃ¹Ä Å satisfying (52).
From Lemma 1 and ®nM¢r&TP8® 5 ¢r& we derive
G
¼L }r„i¨PÙ§e¨7d® n }r&—G
¼L l Á¼Mr ~ Gsº= }r„i¨PÙ§e¨7d® n }r&—Gsº# l Áº«r ~ S
Using GsºH l G
¼
TP8Á|z we get
G º ¢r&i l G ¼ }r&¨PÙ§e¨7d® n ¢r&iÁ|z l ¢Á º l Á ¼ r ~ S
Hence, it remains to show (56) for
On the other hand, by Lemma 13
Á|z}r&R1|e¨7® n ¢r&iÁ|z l ¡ ® n ¢r&Á|z7 û ÁÂr ~ S
Therefore, (23) holds whenever
¢Á º l Á ¼ l û ÁÂr ~ P ¡ ® n ¢r&iÁ|z¹
which, by (14), is implied by
r²P
$&%('
5^g
¡|
¡|
Á $Wz VsX=Z
Á z û û
ÇÁ º l Á¼ l Á ¢ÁǺ l Á ¼ l Á j 5 j n \ 7 which in turn is true, by (55), if
rP
$&%v'
5_g
¡|
Á z û Á º l Á ¼ l Á }Á º l Á ¼
¡Á
l û ÁÂË&}Á l  7 •—™– ˜ ( ~ ,zMS
From now on, this part of the proof follows exactly the same steps of part 1, with
z ce~ d Ä f •§ –™˜ }ÁǺ l Á ¼ l
¡Á
û ÁÂË ~ ¢Á l  ~ A globally convergent primal-dual interior-point filter method
31
z­Ä
¡Á
. €8rP
( ~ z ce~ d Ä f ¢ÁǺ l dÁ ¼ l û ÁÂË&¢Á l  S
3. Finally we prove that (24) holds for all r such that (52) is satisfied. We know
from part 1 that (53) and (54) are verified if rE4 . wrmÃ¹Ä ÅLÆ with rmÃ¹Ä Å satisfying (52).
It follows from (54) that
û r ~ € ´ ® 5 ¢r&¹7d® n }r&N§e*7 ¡ ¸ z
replacing the role of ced f , and
So, from (53), we get
for all
r
Š }r&§‰#¢r&R^` ´ e¨7® 5 ¢r&¸"z ž¦1 . for which (52) is satisfied, and assertion (24) follows trivially.
ÕÖ
Acknowledgments
This research collaboration on the globalization of interior-point methods using the filter technique resulted from conversations held during the First Workshop on Nonlinear
Optimization: ”Interior-Point and Filter Methods”, organized in Coimbra in October
1999.
The authors would also like to acknowledge Matthias Heinkenschloss (Rice University) for his support and insightful comments.
Large parts of this work were done while the authors were visiting the Center for
Research on Parallel Computation and the Department of Computational and Applied
Mathematics at Rice University, which provided a very stimulating research environment. The authors would like to thank John Dennis and Matthias Heinkenschloss for
their hospitality and support.
References
1. M. A RGAEZ AND R. A. TAPIA, On the global convergence of a modified augmented Lagrangian linesearch interior point Newton method for nonlinear programming, Tech. Report TR95–38, Department
of Computational and Applied Mathematics, Rice University, 1995. Revised September 1999.
2. C. AUDET AND J. E. D ENNIS , A pattern search filter method for nonlinear programming without derivatives, Tech. Report TR00–09, Department of Computational and Applied Mathematics, Rice University,
2000.
3. H. Y. B ENSON , D. F. S HANNO , AND R. J. VANDERBEI , Interior-point methods for nonconvex nonlinear programming: Filter methods and merit functions, Tech. Report ORFE-00-06, Operations Research
and Financial Engineering, Princeton University, 2000.
4. P. T. B OGGS , Sequential quadratic programming, in Acta Numerica 1995, A. Iserles, ed., Cambridge
University Press, Cambridge, London, New York, 1995, pp. 1–51.
5. R. H. B YRD , M. E. H RIBAR , AND J. N OCEDAL, An interior point algorithm for large-scale nonlinear
programming, SIAM J. Optim., 9 (1999), pp. 877–900.
6. R. H. B YRD , G. L IU , AND J. N OCEDAL, On the local behavior of an interior point method for nonlinear programming, in Numerical analysis 1997 (Dundee) Pitman Res. Notes Math. Ser., 380, Longman,
Harlow, 1998, pp. 37–56.
7. T. F. C OLEMAN AND Y. L I , On the convergence of interior–reflective Newton methods for nonlinear
minimization subject to bounds, Math. Programming, 67 (1994), pp. 189–224.
32
M. Ulbrich, S. Ulbrich h L. N. Vicente: A primal-dual interior-point filter method
8. A. R. C ONN , N. G OULD , D. O RBAN , AND P. L. T OINT, A primal-dual trust-region algorithm for minimizing a non-convex function subject to general inequality and linear equality constraints, in Nonlinear
optimization and related topics (Erice, 1998), Kluwer Acad. Publ., Dordrecht, 2000, pp. 15–49.
9. A. R. C ONN , N. I. M. G OULD , AND P. L. T OINT, Trust-Region Methods, Society for Industrial and
Applied Mathematics (SIAM), Philadelphia, PA, 2000.
10. A. S. E L -BAKRY, R. A. TAPIA , T. T SUCHIYA , AND Y. Z HANG, On the formulation and theory of the
Newton interior–point method for nonlinear programming, J. Optim. Theory Appl., 89 (1996), pp. 507–
541.
11. R. F LETCHER , N. G OULD , S. L EYFFER , AND P. L. T OINT, Global convergence of trust-region
SQP-filter algorithms for general nonlinear programming, Tech. Report 99/03, Département de
Mathématique, FUNDP, Namur, 1999.
12. R. F LETCHER AND S. L EYFFER, User manual for filterSQP, Tech. Report NA/181, Department of
Mathematics, University of Dundee, 1998.
13.
, Nonlinear programming without a penalty function, Math. Programming, 91 (2002), pp. 239–
269.
14. R. F LETCHER , S. L EYFFER , AND P. L. T OINT, On the global convergence of an SLP-filter algorithm,
Tech. Report 98/13, Département de Mathématique, FUNDP, Namur, 1998.
15. A. F ORSGREN AND P. E. G ILL, Primal-dual interior methods for nonconvex nonlinear programming,
SIAM J. Optim., 8 (1998), pp. 1132–1152.
16. D. M. G AY, M. L. OVERTON , AND M. H. W RIGHT, A primal-dual interior method for nonconvex
nonlinear programming, in Proceedings of the 1996 International Conference on Nonlinear Programming, Beijing, China, Y. Yuan, ed., Kluwer Academic Publishers, Dordrecht, The Netherlands, 1998,
pp. 31–56.
17. M. H EINKENSCHLOSS , M. U LBRICH , AND S. U LBRICH, Superlinear and quadratic convergence of
affine-scaling interior-point Newton methods for problems with simple bounds without strict complementarity assumption, Math. Programming, 86 (1999), pp. 615–635.
18. H. J. M ARTINEZ , Z. PARADA , AND R. A. TAPIA, On the characterization of q-superlinear convergence
of quasi-Newton interior-point methods for nonlinear programming, Boletin de la Sociedad Matematica
Mexicana, 1 (1995), pp. 1–12.
19. J. N OCEDAL AND S. J. W RIGHT, Numerical Optimization, Springer–Verlag, Berlin, Heidelberg, New
York, Tokyo, 1999.
20. P. T SENG, Error bounds and superlinear convergence analysis of some Newton-type methods in optimization, in Nonlinear Optimization and Applications, vol. 2, Kluwer Academic Publishers B. V., 1998.
21. M. U LBRICH AND S. U LBRICH, Nonmonotone trust region methods for nonlinear equality constrained
optimization without a penalty function, (to appear in Math. Programming).
22. R. J. VANDERBEI AND D. F. S HANNO, An interior-point algorithm for nonconvex nonlinear programming, Comput. Optim. and Appl., 13 (1999), pp. 231–252.
23. L. N. V ICENTE, Local convergence of the affine-scaling interior-point algorithm for nonlinear programming, Comput. Optim. and Appl., 17 (2000), pp. 23–35.
24. L. N. V ICENTE AND S. J. W RIGHT, Local convergence of a primal-dual method for degenerate nonlinear programming, (to appear in Comput. Optim. and Appl.).
25. A. W ÄCHTER AND L. T. B IEGLER, Failure of global convergence for a class of interior point methods
for nonlinear programming, Math. Program., 88 (2000), pp. 565–574.
26. A. W ÄCHTER AND L. T. B IEGLER, Global and local convergence of line search filter methods for
nonlinear programming, Tech. Report B-01-09, CAPD, Department of Chemical Engineering, Carnegie
Mellon University, 2001.
27. S. J. W RIGHT, Primal-Dual Interior-Point Methods, SIAM, Philadelphia, 1997.
28. H. YAMASHITA AND H. YABE, Superlinear and quadratic convergence of some primal-dual interior
point methods for constrained optimization, Math. Programming, 75 (1996), pp. 377–397.