Hindawi Publishing Corporation
Journal of Applied Mathematics
Volume 2013, Article ID 976509, 9 pages
http://dx.doi.org/10.1155/2013/976509
Research Article
Global Convergence of a New Nonmonotone Filter Method for
Equality Constrained Optimization
Ke Su, Wei Liu, and Xiaoli Lu
College of Mathematics and Computer Science, Hebei University, Baoding 071002, China
Correspondence should be addressed to Ke Su; pigeonsk@163.com
Received 6 December 2012; Accepted 5 March 2013
Academic Editor: Hadi Nasseri
Copyright © 2013 Ke Su et al. This is an open access article distributed under the Creative Commons Attribution License, which
permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
A new nonmonotone filter trust region method is introduced for solving optimization problems with equality constraints. This
method directly uses the dominated area of the filter as an acceptability criterion for trial points and allows the dominated area
decreasing nonmonotonically. Compared with the filter-type method, our method has more flexible criteria and can avoid Maratos
effect in a certain degree. Under reasonable assumptions, we prove that the given algorithm is globally convergent to a first order
stationary point for all possible choices of the starting point. Numerical tests are presented to show the effectiveness of the proposed
algorithm.
1. Introduction
We analyze an algorithm for solving optimization problems
where a smooth objective function is to be minimized subject
to smooth nonlinear equality constraints. More formally, we
consider the problem,
min
𝑓 (𝑥) ,
s.t. 𝑐𝑖 (𝑥) = 0,
𝑖 ∈ 𝐼 = {1, 2, . . . , 𝑚} ,
(𝑃)
where 𝑥 ∈ 𝑅𝑛 , the functions 𝑓 : 𝑅𝑛 → 𝑅 and 𝑐𝑖 (𝑖 ∈ 𝐼) : 𝑅𝑛 →
𝑅 are all twice continuously differentiable. For convenience,
let 𝑔(𝑥) = ∇𝑓(𝑥), 𝑐(𝑥) = (𝑐1 (𝑥), 𝑐2 (𝑥), . . . , 𝑐𝑚 (𝑥))𝑇 and
𝐴(𝑥) = (∇𝑐1 (𝑥), ∇𝑐2 (𝑥), . . . , ∇𝑐𝑚 (𝑥)), and 𝑓𝑘 refers to 𝑓(𝑥𝑘 ),
𝑐𝑘 to 𝑐(𝑥𝑘 ), 𝑔𝑘 to 𝑔(𝑥𝑘 ) and 𝐴 𝑘 to 𝐴(𝑥𝑘 ), and so forth.
There are many trust region methods for equality constrained nonlinear programming (𝑃), for example, Byrd et
al. [1], Dennis Jr. et al. [2] and Powell and Yuan [3], but
in these works, a penalty or augmented Lagrange function is always used to test the acceptability of the iterates.
However, there are several difficulties associated with the
use of penalty function, and in particular the choice of the
penalty parameter. Hence, in 2002, Fletcher and Leyffer [4]
proposed a class of filter method, which does not require
any penalty parameter and has promising numerical results.
Consequently, filter technique has been employed to many
approaches, for instance, SLP methods [5], SQP methods [6–
8], interior point approaches [9], bundle techniques [10], and
so on.
Filter technique, in fact, exhibits a certain degree of nonmonotonicity. The nonmonotone technique was proposed by
Grippo et al. in 1986 [11] and combined with many other
methods. M. Ulbrich and S. Ulbrich [12] proposed a class
of penalty-function-free nonmonotone trust region methods
for nonlinear equality constrained optimization without filter
technique. Su and Pu [13] introduced a nonmonotone trust
region method which used the nonmonotone technique in
the traditional filter criteria. Su and Yu [14] presented a
nonmonotone method without penalty function or filter.
Gould and Toint [15] directly used the dominated area of the
filter as an acceptability criteria for trial points and obtained
the global convergence properties. We refer the reader [16–18]
for some works about this issue.
Motivated by the ideas and methods above, we propose
a modified nonmonotone filter trust region method for
solving problem (𝑃). Similar to the Byrd-Omojokun class of
algorithms, each step is decomposed into the sum of two
distinct components, a quasi-normal step and a tangential
step. The main contribution of our paper is to employ the
nonmonotone idea to the dominated area of the filter so
that the new and more flexible criteria is given, which is
different from that of Gould and Toint [15] and Su and Pu [13].
2
Journal of Applied Mathematics
Under usual assumptions, we prove that the given algorithm
is globally convergent to first order stationary points.
This paper is organized as follows. In Section 2, we
introduce the fraction of Cauchy decrease and the composite
SQP step. The new nonmonotone filter technique is given in
Section 3. In Section 4, we propose the new nonmonotone
filter method and present the global convergence properties
in Section 5. Some numerical results are reported in the last
section.
𝑑𝑘𝑛 is the solution to the subproblem
min
s.t.
1 󵄩󵄩
󵄩2
󵄩󵄩𝑐 + 𝐴𝑇𝑘 𝑑𝑛 󵄩󵄩󵄩 +
󵄩
2󵄩 𝑘
󵄩󵄩 𝑛 󵄩󵄩
󵄩󵄩𝑑 󵄩󵄩 ≤ Δ 𝑘 ,
min
s.t.
Consider the following unconstraint minimization optimization problem:
(1)
𝑥∈𝑅
𝑛
where 𝑓 : 𝑅 → 𝑅 is a continuously differentiable function.
A trust region algorithm for solving the above problem is an
iterate procedure that computes a trial step as an approximate
solution to the following subproblems:
min
s.t.
1
𝑞 (𝑑) = 𝑔𝑇 𝑑 + 𝑑𝑇 𝐻𝑑,
2
where 𝐻 is the Hessian matrix ∇2 𝑓(𝑥) or an approximate to
it and Δ > 0 is a given trust region radius.
To assure the global convergence, the step is required only
to satisfy a fraction of Cauchy decrease condition. This means
that 𝑑 must predict via the quadratic model function 𝑞(𝑑)
at least as much as a fraction of the decreased given by the
Cauchy step on 𝑞(𝑑); that is, there exists a constant 𝜎 > 0
fixed across all iterations, such that
𝑐𝑝
𝑞 (0) − 𝑞 (𝑑) ≥ 𝜎 (𝑞 (0) − 𝑞 (𝑑 )) ,
(3)
where 𝑑𝑐𝑝 is the steepest descent step for 𝑞(𝑑) inside the trust
region.
Lemma 1. If the trial step 𝑑 satisfies a fraction of Cauchy
decrease condition, then
𝑞 (0) − 𝑞 (𝑑) ≥
󵄩
󵄩󵄩
󵄩∇𝑓 (𝑥)󵄩󵄩󵄩
𝜎 󵄩󵄩
󵄩
}.
󵄩󵄩∇𝑓 (𝑥)󵄩󵄩󵄩 min {Δ, 󵄩
2
‖𝐻‖
(4)
Proof (see Powell [19] for the proof). Now, we turn to explain
the composite SQP step. Given an approximate estimate of
the solution 𝑥𝑘 at 𝑘th iteration, following Dennis Jr. et al. [2]
and M. Ulbrich and S. Ulbrich [12], we obtain the trial step
𝑑𝑘 = 𝑑𝑘𝑛 + 𝑑𝑘𝑡 by computing a quasi-normal step 𝑑𝑘𝑛 and a
tangential step 𝑑𝑘𝑡 . The purpose of the quasi-normal step 𝑑𝑘𝑛
is to improve feasibility. To improve optimality, we seek 𝑑𝑘𝑡
in the tangential space of the linearized constraints in such a
way that it provides sufficient decrease for a quadratic model
of the objective function 𝑓(𝑥). Let 𝑞𝑘 (𝑑) = 𝑔𝑘𝑇 𝑑+(1/2)𝑑𝑇 𝐻𝑘 𝑑,
where 𝐻𝑘 is a symmetric approximation of ∇2 𝑓(𝑥).
𝑞𝑘 (𝑑𝑘𝑛 + 𝑑𝑡 ) ,
𝐴𝑇𝑘 𝑑𝑡 = 0
(6)
󵄩󵄩 𝑡 󵄩󵄩
󵄩󵄩𝑑 󵄩󵄩 ≤ Δ 𝑘 .
󵄩 󵄩
Then we get the current trial step 𝑑𝑘 = 𝑑𝑘𝑛 + 𝑑𝑘𝑡 . Let 𝑑𝑘𝑡 =
𝑡
𝑡
𝑊𝑘 𝑑𝑘 , where 𝑑𝑘 ∈ 𝑅𝑛−𝑚 and 𝑊𝑘 ∈ 𝑅𝑛×(𝑛−𝑚) denote a matrix
whose columns form a basis of the null space of 𝐴𝑇𝑘 . We refer
to [2] for a more detailed discussion of this issue.
In usual way that impose a trust region in stepdecomposition methods, the quasi-normal step 𝑑𝑘𝑛 and the
tangential step 𝑑𝑘𝑡 are required to satisfy
󵄩󵄩 𝑛 󵄩󵄩
󵄩󵄩𝑑 󵄩󵄩 ≤ 𝜅Δ 𝑘 ,
(2)
‖𝑑‖ ≤ Δ,
(5)
where Δ 𝑘 is a trust region radius and 𝐴 𝑘 = ∇𝑐(𝑥𝑘 ) ∈
𝑅𝑛×𝑚 , 𝜉 > 0. In order to improve the value of the objective
function, we solve the following subproblem to get 𝑑𝑘𝑡 :
2. The Fraction of Cauchy Decrease and
the Composite SQP Step
min𝑛 𝑓 (𝑥) ,
𝜉 󵄩󵄩 𝑛 󵄩󵄩2
󵄩𝑑 󵄩 ,
2󵄩 󵄩
󵄩
󵄩󵄩 𝑛
󵄩󵄩𝑑𝑘 + 𝑑𝑡 󵄩󵄩󵄩 ≤ Δ 𝑘 ,
󵄩
󵄩
(7)
where 0 < 𝜅 < 1. Here, to simplify the proof, we only impose
a trust region on ‖ 𝑑𝑛 ‖≤ Δ 𝑘 and ‖ 𝑑𝑡 ‖≤ Δ 𝑘 , which is natural.
Note that 𝑊𝑘𝑇 ∇𝑞𝑘 (𝑑𝑡 ) is the reduced gradient of 𝑞𝑘 in
terms of the representation 𝑑𝑡 = 𝑊𝑘 𝑠 of the tangential step:
∇𝑠 (𝑞𝑘 (𝑊𝑘 𝑠)) = 𝑊𝑘𝑇 ∇𝑞𝑘 (𝑊𝑘 𝑠) = 𝑊𝑘𝑇 ∇𝑞𝑘 (𝑑𝑡 ) .
(8)
Define
𝑔̂ (𝑥) = 𝑊(𝑥)𝑇 𝑔 (𝑥) .
(9)
Then the first order necessary optimality conditions (KarushKuhn-Tucker or KKT conditions) at a local solution 𝑥 ∈ 𝑅𝑛
of problem (𝑃) can be written as
𝑐 (𝑥) = 0,
𝑔̂ (𝑥) = 0.
(10)
3. A New Nonmonotone Filter Technique
In filter method, originally proposed by Fletcher and Leyffer
[4], the acceptability of iterates is determined by comparing
the value of constraint violation and the objective function
with previous iterates collected in a filter. Define the violation
function ℎ(𝑥) by ℎ(𝑥) =‖ 𝑐(𝑥)‖22 , it is easy to see that ℎ(𝑥) = 0
if and only if 𝑥 is a feasible point, so a trial point should reduce
either the value of constraint violation or that of the objective
function 𝑓.
In the process of the algorithm, we need to decide whether
the trial point 𝑥𝑘+ is any better than 𝑥𝑘 as an approximate
solution to the problem (𝑃). If we decide that this is the case,
we say that the iteration 𝑘 is successful and choose 𝑥𝑘+ as
Journal of Applied Mathematics
3
𝑓(𝑥)
𝑓(𝑥)
𝑁𝑊(ℱ𝑘 )
𝑘ℱ
ℱ𝑘
+ 𝑘ℱ
𝑓max
ℱ𝑘
𝑓max
𝒟(ℱ𝑘 )
ℱ
ℱ𝑘
𝑓max
ℱ
𝑘
ℎmin
𝑘
ℎmax
(ℎ𝑘+ , 𝑓𝑘+ )
(ℎ𝑘+ , 𝑓𝑘+ )
𝒫𝑘
𝑓max
ℎ (𝑥)
(ℎ𝑘+ , 𝑓𝑘+ )
𝑆𝑊(ℱ𝑘 )
ℱ
ℱ
𝑘
ℎmin
ℱ𝑘
𝑓min
ℱ𝑘
ℎmax
+ 𝑘ℱ
𝑘
ℎmax
𝒫𝑘
𝑓min
𝑆𝐸(ℱ𝑘 )
ℎ (𝑥)
𝑘ℱ
ℱ𝑘
𝑓min
Figure 1
(ℎ𝑘+ , 𝑓𝑘+ )
the next iterate. Let us denote by S the set of all successful
iterations, that is,
󵄨
(11)
S = {𝑘 󵄨󵄨󵄨𝑥𝑘+1 = 𝑥𝑘+ } .
In traditional filter method, a point 𝑥 is called acceptable
to the filter if and only if
ℎ (𝑥) ≤ 𝛽ℎ𝑗 or 𝑓 (𝑥) ≤ 𝑓𝑗 − 𝛾ℎ𝑗 ,
∀ (ℎ𝑗 , 𝑓𝑗 ) ∈ F,
where 0 < 𝛾 < 𝛽 < 1, F denotes the filter set. Define
󵄨
D (F) = {(ℎ, 𝑓) 󵄨󵄨󵄨󵄨ℎ > ℎ𝑗 and 𝑓 > 𝑓𝑗 , ∃𝑗 ∈ F } .
F
F𝑘
ℎmax
= max ℎ𝑗 ,
F def
𝑓min𝑘 =
F𝑘 def
𝑓max
=
def
𝑗∈F𝑘
min 𝑓𝑗 ,
𝑗∈F𝑘
Consider the trial point 𝑥𝑘+ , if the filter is empty, then define
its contribution to the area of the filter by
𝛼 (𝑥𝑘+ , F𝑘 ) = 𝜅𝐹 2 ,
def
(12)
(13)
A trial point 𝑥𝑘+ is accepted if and only if (ℎ𝑘+ , 𝑓𝑘+ ) ∉ D(F𝑘 ).
Now, similar to the idea of Gould and Toint [15], we give a
new modified nonmonotone filter technique. For any (ℎ, 𝑓)pair, define an area that represents its contribution to the area
of D(F), we hope this contribution is positive; that is, the
area of D(F) is increasing. For convenience, we partition
the right half-plane [0, +∞] × [−∞, +∞] into four different
regions (see Figure 1). Define D(F𝑘 )𝑐 to be the complement
of D(F𝑘 ). Let
𝑘
= min ℎ𝑗 ,
ℎmin
Figure 2
where 𝜅𝐹 > 0 is a constant. If the filter is not empty, then
define the contribution of 𝑥𝑘+ to the area of the filter by four
different formulae.
If (ℎ𝑘+ , 𝑓𝑘+ ) ∈ 𝑆𝑊(F𝑘 ), assume
𝑐
F𝑘
𝛼 (𝑥𝑘+ , F𝑘 ) = area (D(F𝑘 ) ∩ [ℎ𝑘+ , ℎmax
+ 𝜅𝐹 ]
def
F𝑘
× [𝑓𝑘+ , 𝑓max
(17)
+ 𝜅𝐹 ]) .
If (ℎ𝑘+ , 𝑓𝑘+ ) ∈ 𝑁𝑊(F𝑘 ), assume
F
(18)
F
(19)
𝑘
𝛼 (𝑥𝑘+ , F𝑘 ) = 𝜅𝐹 (ℎmin
− ℎ𝑘+ ) .
def
If (ℎ𝑘+ , 𝑓𝑘+ ) ∈ 𝑆𝐸(F𝑘 ), assume
def
𝑗∈F𝑘
(14)
𝛼 (𝑥𝑘+ , F𝑘 ) = 𝜅𝐹 (𝑓min𝑘 − 𝑓𝑘+ ) .
def
max 𝑓𝑗 .
𝑗∈F𝑘
If (ℎ𝑘+ , 𝑓𝑘+ ) ∈ 𝑁𝐸(F𝑘 ) = D(F𝑘 ), assume
These four parts are
def
def
(2) the undominated part of lower left corner of the half
plane:
𝑐
F𝑘
F𝑘
𝑆𝑊 (F𝑘 ) = D(F𝑘 ) ∩ [0, ℎmax
] × [−∞, 𝑓max
],
(15)
P
𝑘
𝛼 (𝑥𝑘+ , F𝑘 ) = −area (D (F𝑘 ) ∩ [ℎ𝑘+ − ℎmin
]
(1) the dominated part of the filter: 𝑁𝐸(F𝑘 ) = D(F𝑘 );
def
(16)
(20)
P
× [𝑓𝑘+ − 𝑓min𝑘 ]) ,
P
def
𝑘
where P𝑘 = {(ℎ, 𝑓) ∈ F𝑘 |ℎ𝑗 < ℎ𝑘+ , 𝑓𝑗 < 𝑓𝑘+ }, and ℎmin
=
P
def
min ℎ𝑗 , 𝑓min𝑘 = min𝑓𝑗 .
(3) the undominated upper left corner: 𝑁𝑊(F𝑘 )
F𝑘
F𝑘
) × (𝑓max
, +∞];
[0, ℎmin
def
=
𝑗∈P𝑘
(4) the undominated lower right corner: 𝑆𝐸(F𝑘 )
F
F𝑘
(ℎmax
, +∞] × [−∞, 𝑓min𝑘 ).
def
Figure 2 illustrate the corresponding areas in the filter.
Horizontally dashed surfaces indicate a positive contribution
and vertically dashed ones a negative contribution. Note that
𝛼(𝑥, F) is a continuous function of (ℎ(𝑥), 𝑓(𝑥)).
=
𝑗∈P𝑘
4
Journal of Applied Mathematics
Next, we should consider the updating of the filter. If
(ℎ𝑘 , 𝑓𝑘 ) ∉ D(F𝑘 ), then
F𝑘+1 ←󳨀 F𝑘 ∪ (ℎ𝑘 , 𝑓𝑘 ) .
is true or not, where 𝜂3 and 𝜇 are all positive constants, if it
is not true, then turn to the feasibility restoration phase and
define
(21)
𝑗
𝑟𝑘
If (ℎ𝑘 , 𝑓𝑘 ) ∈ D(F𝑘 ), then
P
P
𝑘
F𝑘+1 ←󳨀 (F𝑘 \ P𝑘 ) ∪ (ℎmin
, 𝑓𝑘 ) ∪ (ℎ𝑘 , 𝑓min𝑘 ) .
(22)
2
(23)
def
where 𝛼𝑘 = 𝛼(𝑥𝑘+ , F𝑘 ), 𝛾F ∈ (0, 1) is a constant. Under the
nonmonotonic situation, we relax condition (23) to be
}
{
𝑘
}
{
}
{
2
max {𝛼𝑘 , ∑ 𝜆 𝑝(𝑗) 𝛼𝑝(𝑗) } ≥ 𝛾F (ℎ𝑘+ ) ,
}
{
}
{ 𝑗=𝑟(𝑘)+1
𝑗∈U
}
{
(24)
def
where 𝛼𝑝(𝑗) = 𝛼(𝑥𝑗 , F𝑝(𝑗) ), U = {𝑘| filter is updated for
(ℎ𝑘 , 𝑓𝑘 )}, 𝑟(𝑘) ≤ 𝑘 is some reference iteration for U, 𝑟(𝑘) ∈ U,
U ⊆ S, 𝜆 𝑝(𝑗) ∈ [0, 1], ∑𝑘𝑗=𝑟(𝑘)+1, 𝑗∈U 𝜆 𝑝(𝑗) = 1.
Compared to condition (2.21) [15], our condition (24) is
more flexible if 𝛼𝑘 is negative.
According to condition (24), it is possible to accept 𝑥𝑘+
even though it may be dominated. Then 𝑥𝑘+ will be accepted
if either (23) or (24) holds.
4. The New Nonmonotone Filter Trust
Region Algorithm
Our algorithm is based on the usual trust region technique;
define the predict reduction for the function 𝑞𝑘 (𝑥) to be
pred (𝑑𝑘 ) = 𝑞𝑘 (0) − 𝑞𝑘 (𝑑𝑘 )
(28)
A formal description of the algorithm is given as follows.
We now return to the question of deciding whether a trial
point 𝑥𝑘+ is acceptable for the filter or not. We will insist that
this is a necessary condition for the iteration 𝑘 to be successful
in the sense that 𝑥𝑘+1 = 𝑥𝑘+ . If we consider an iterate 𝑥𝑘 , there
+
= 𝑥𝑝(𝑘)+1 =
must exist a predecessor iteration such that 𝑥𝑝(𝑘)
𝑥𝑘 . Under the monotonic situation, a trial point 𝑥𝑘+ would be
accepted whenever it results in an sufficient increase in the
dominated area of the filter, that means 𝑥𝑘+ would be accepted
whenever
𝛼𝑘 ≥ 𝛾F (ℎ𝑘+ ) ,
󵄩󵄩 𝑗 󵄩󵄩2 󵄩󵄩 𝑗
󵄩2
󵄩󵄩𝑐(𝑥𝑘 )󵄩󵄩 − 󵄩󵄩𝑐(𝑥𝑘 + 𝑑𝑘𝑗 )󵄩󵄩󵄩
󵄩
󵄩
󵄩
󵄩
=󵄩
.
󵄩󵄩𝑐(𝑥𝑗 )󵄩󵄩󵄩2 − 󵄩󵄩󵄩𝑐(𝑥𝑗 ) + 𝐴(𝑥𝑗 )𝑇 𝑑𝑗 󵄩󵄩󵄩2
󵄩󵄩 𝑘 󵄩󵄩 󵄩󵄩 𝑘
𝑘
𝑘󵄩
󵄩
(25)
Algorithm A 𝑆𝑡𝑒𝑝 0. Choose an initial point 𝑥0 ∈ 𝑅𝑛 , a
symmetric matrix 𝐻0 ∈ 𝑅𝑛×𝑛 , let Δ 0 > 0, 𝜖𝑡 > 0, 𝜂2 , 𝜂3 ∈
(0, 1), 𝛼1 , 𝛼2 ∈ [0, 1], 𝜂1 > 0, 0 < 𝛾0 < 𝛾1 < 1 ≤ 𝛾2 < 𝛾3 ≤
2, 0 < 𝜉 < 1, 𝛾F ∈ (0, 1), 𝜇 > 0, compute 𝑓(𝑥0 ), ℎ(𝑥0 ), let
𝑘 = 0, F0 = 0.
Step 1. Compute 𝑐𝑘 , 𝐴 𝑘 , 𝑊𝑘 , 𝑓𝑘 , ℎ𝑘 , 𝑔𝑘 , 𝑔̂𝑘 = 𝑊𝑘𝑇 𝑔𝑘 .
Step 2. If ‖ 𝑔̂𝑘 ‖ +ℎ𝑘 ≤ 𝜖𝑡 , stop.
Step 3. Solve the subproblems (5) and (6) to get the quasinormal step 𝑑𝑘𝑛 and the tangential step 𝑑𝑘𝑡 . Let 𝑑𝑘 = 𝑑𝑘𝑛 +
𝑑𝑘𝑡 , 𝑥𝑘+ = 𝑥𝑘 + 𝑑𝑘 .
Step 4. If 𝑟𝑘 < 𝜂1 , let 𝑥𝑘+1 = 𝑥𝑘 , then go to Step 8.
Step 5. If 𝑥𝑘 + 𝑑𝑘 is not acceptable to the filter, go to Step 8.
Otherwise 𝑥𝑘+1 = 𝑥𝑘+ and update the filter according to
(21) and (22), the trust region radius and 𝐻𝑘 , then get the
corresponding ℎ𝑘+1 , 𝑓𝑘+1 .
2+𝛼2
Step 6. If ℎ𝑘+1 ≤ 𝜂3 min{𝜇, 𝛼1 Δ 𝑘
Step 1, otherwise go to Step 7.
}, 𝑘 = 𝑘 + 1 and go to
Step 7. By restoration Algorithm B to get 𝑑𝑘𝑟 , then the trial
point 𝑥𝑘𝑟 = 𝑥𝑘 + 𝑑𝑘𝑟 .
Step 8. Update the trust region radius by Algorithm C, let 𝑘 =
𝑘 + 1 and go to Step 3.
We aim to reduce the value of ℎ(𝑥) in the restoration
Algorithm B, that is to get 𝑐(𝑥𝑘𝑟 ) = 0 by Newton-type method.
Algorithm B 𝑆𝑡𝑒𝑝 0. Let 𝑥𝑘0 = 𝑥𝑘 , Δ0𝑘 = Δ 𝑘 , 𝑗 = 0, 𝜂2 , 𝜂3 ∈
(0, 1), 𝛼1 , 𝛼2 ∈ [0, 1], 𝜇 > 0.
𝑗
F
2+𝛼2
Step 1. If ℎ(𝑥𝑘 ) ≤ 𝜂3 min{ℎ𝑘 𝑘 , 𝛼1 Δ 𝑘
𝑗
by the filter, then 𝑥𝑘𝑟 = 𝑥𝑘 , stop.
𝑗
} and 𝑥𝑘 is acceptable
Step 2. Compute
and the actual reduction
ared (𝑑𝑘 ) = 𝑓 (𝑥𝑘 ) − 𝑓 (𝑥𝑘+ ) .
Moreover, let 𝑟𝑘 = ared(𝑑𝑘 )/pred(𝑑𝑘 ), if there exists a nonzero
constant 𝜂1 such that 𝑟𝑘 ≥ 𝜂1 and condition (23) and (24)
hold, the trial point 𝑥𝑘+ will be called acceptable. Then the next
trial point 𝑥𝑘+1 is obtained, and for its feasibility, we consider
the condition
2+𝛼2
ℎ𝑘+1 ≤ 𝜂3 min {𝜇, 𝛼1 Δ 𝑘
}
min
(26)
(27)
s.t.
𝑗
󵄩2
󵄩󵄩 𝑗
󵄩󵄩𝑐(𝑥𝑘 ) + 𝐴(𝑥𝑘𝑗 )𝑇 𝑑𝑘𝑗 󵄩󵄩󵄩 ,
󵄩
󵄩
󵄩󵄩 𝑗 󵄩󵄩
𝑗
󵄩󵄩󵄩𝑑𝑘 󵄩󵄩󵄩 ≤ Δ 𝑘
(29)
𝑗
to get 𝑑𝑘 , then compute 𝑟𝑘 .
𝑗
𝑗+1
Step 3. If 𝑟𝑘 ≤ 𝜂2 , 𝑥𝑘
to Step 2.
𝑗
𝑗+1
𝑗
= 𝑥𝑘 , Δ 𝑘 = Δ 𝑘 /2, 𝑗 = 𝑗 + 1 and go
Journal of Applied Mathematics
𝑗
𝑗+1
𝑗
5
𝑗
𝑗+1
𝑗
Step 4. If 𝑟𝑘 > 𝜂2 , 𝑥𝑘 = 𝑥𝑘 + 𝑑𝑘 , Δ 𝑘 = 2Δ 𝑘 , 𝑗 = 𝑗 + 1,
𝑗+1
F
compute 𝐴 𝑘 and go to Step 1, where ℎ𝑘 𝑘 = min{min𝑗∈F𝑘
{ℎ𝑗 |ℎ𝑗 > 0}, 𝜇}.
Lemma 2. At the current iterate 𝑥𝑘 , let the trial point component 𝑑𝑘𝑛 actually be normal to the tangential space. Under
the problem assumptions, there exists a constant 𝑘1 > 0
independent of the iterates such that
Algorithm C (updating the trust region radius). Given 𝜂1 >
0, 0 < 𝛾0 < 𝛾1 < 1 ≤ 𝛾2 < 𝛾3 ≤ 2, we have the following.
󵄩 󵄩
󵄩󵄩 𝑛 󵄩󵄩
󵄩󵄩𝑑𝑘 󵄩󵄩 ≤ 𝛼1 󵄩󵄩󵄩𝑐𝑘 󵄩󵄩󵄩 .
(1) If 𝑟𝑘 < 𝜂1 or 𝑥𝑘+ is not acceptable to the filter, Δ 𝑘+1 ∈
[𝛾0 Δ 𝑘 , 𝛾1 Δ 𝑘 ].
(2) If 𝑥𝑘+ is acceptable to the filter but does not satisfiy
condition (27), Δ 𝑘+1 ∈ (Δ 𝑘 , 𝛾2 Δ 𝑘 ).
(3) If 𝑥𝑘+ is acceptable to the filter and satisfies
(27), Δ 𝑘+1 ∈ [𝛾2 Δ 𝑘 , 𝛾3 Δ 𝑘 ].
From the description above and the idea of the algorithm,
we can see that our algorithm is more flexible. Every successful iterate must be any better than the predecessor one in
some degree according to the traditional filter method. But
our algorithm relaxes this demand by using the nonmonotone technique and also avoids Maratos effect in a certain
degree. Moreover, Algorithm C allows a relatively wide choice
of the trust region.
5. The Convergence Properties
Assumption
(A1) The objective function 𝑓 and the constraint functions
𝑐𝑖 (𝑖 ∈ 𝐼 = {1, 2, . . . , 𝑚}) are twice continuously
differentiable.
(A2) For all 𝑘, 𝑥𝑘 , and 𝑥𝑘 + 𝑑𝑘 all remain in a closed,
bounded convex subset Ω ⊂ 𝑅𝑛 .
(A3) The matrix 𝐴(𝑥) = ∇𝑐(𝑥) is nonsingular matrix for all
𝑥 ∈ Ω.
−1
𝑇
(A5) The matrix 𝐻𝑘 is uniformly bounded.
By the assumptions, we can suppose there exist constants
V0 , V1 , V2 , V3 such that ‖ 𝑓(𝑥) ‖≤ V0 , ‖ 𝑔(𝑥) ‖≤ V0 , ‖ 𝑐(𝑥) ‖≤
V0 , ‖ 𝐴(𝑥) ‖≤ V0 , ‖ 𝐴(𝑥)𝑇 𝐴(𝑥)−1 ‖≤ V1 , ‖ 𝑊(𝑥) ‖≤ V2 , ‖
𝐻𝑘 ‖≤ V3 , ‖ 𝑊𝑘𝑇 𝐻𝑘 ‖≤ V3 , ‖ 𝑊𝑘𝑇 𝐻𝑘 𝑊𝑘 ‖≤ V3 .
By (A1) and (A2), it holds
0 ≤ ℎ𝑘 ≤ ℎmax
Lemma 3. Under Assumptions, there exist positive constants
𝑘2 , 𝑘3 , 𝑘4 independent of the iterates such that
󵄩󵄩 󵄩󵄩2 󵄩󵄩󵄩
󵄩 󵄩
󵄩 󵄩
𝑇 𝑛 󵄩2
󵄩󵄩𝑐𝑘 󵄩󵄩 − 󵄩󵄩𝑐𝑘 + 𝐴 𝑘 𝑑𝑘 󵄩󵄩󵄩󵄩 ≥ 𝑘2 󵄩󵄩󵄩𝑐𝑘 󵄩󵄩󵄩 min {𝑘3 󵄩󵄩󵄩𝑐𝑘 󵄩󵄩󵄩 , Δ 𝑘 } ,
𝑞𝑘 (𝑑𝑘𝑛 ) − 𝑞𝑘 (𝑑𝑘 )
≥
𝜎 󵄩󵄩 𝑇
󵄩
󵄩
󵄩
󵄩󵄩𝑊𝑘 ∇𝑞𝑘 (𝑑𝑘𝑛 )󵄩󵄩󵄩 min {𝑘4 󵄩󵄩󵄩𝑊𝑘𝑇 ∇𝑞𝑘 (𝑑𝑘𝑛 )󵄩󵄩󵄩 , Δ 𝑘 } .
󵄩
󵄩
󵄩
󵄩
2
(33)
Proof. The proof is an application of Lemma 1 to the two
subproblems (5) and (6).
Lemma 4. Suppose that Assumptions hold, then restoration
Algorithm B is well defined.
∀𝑘,
(30)
where 𝑓min , ℎmax > 0, hence in the (ℎ, 𝑓)-plane, the (ℎ, 𝑓)pair lies in the area [0, ℎmax ] × [𝑓min , +∞].
From (A1), (A2), and (A3), it exists a constant ] such that
󵄨󵄨
󵄨
2
󵄨󵄨𝑓 (𝑥𝑘 + 𝑑𝑘 ) − 𝑞𝑘 (𝑑𝑘 )󵄨󵄨󵄨 ≤ ]Δ 𝑘 .
Proof. The conclusion is obvious, if ℎ𝑘 → 0. Otherwise it
𝑗
exists 𝜖 > 0 such that for all 𝑗, it holds ℎ𝑘 > 𝜖. Consider the
set
󵄨󵄨
󵄩󵄩 𝑗 󵄩󵄩2 󵄩󵄩
󵄩2
󵄩󵄩𝑐𝑘 󵄩󵄩 − 󵄩󵄩𝑐 (𝑥𝑘𝑗 + 𝑑𝑘𝑗 )󵄩󵄩󵄩
{ 󵄨󵄨󵄨 𝑗
󵄩
󵄩
󵄩
󵄩 > 𝜂 > 0} , (34)
󵄨
𝐾 = {𝑗 󵄨󵄨𝑟𝑘 = 󵄩 󵄩2 󵄩
2
}
𝑗
𝑗
𝑗
𝑗
󵄨󵄨
󵄩󵄩𝑐 󵄩󵄩 − 󵄩󵄩𝑐 + (𝐴 )𝑇 𝑑 󵄩󵄩󵄩2
󵄩󵄩 𝑘 󵄩󵄩 󵄩󵄩 𝑘
𝑘
𝑘󵄩
}
{ 󵄨󵄨
󵄩
𝑗
𝑗
𝑗
(31)
𝑗
where 𝑐𝑘 = 𝑐(𝑥𝑘 ), 𝐴 𝑘 = 𝐴(𝑥𝑘 ). By Lemma 3 and the
definition of ℎ𝑘 , we have
∞
󵄩2
󵄩 𝑗 󵄩2 󵄩󵄩 𝑗
𝑗−1
𝑗
𝑗 𝑇 𝑗󵄩
+∞ > ∑ (ℎ𝑘 − ℎ𝑘 ) ≥ ∑ (󵄩󵄩󵄩󵄩𝑐𝑘 󵄩󵄩󵄩󵄩 − 󵄩󵄩󵄩󵄩𝑐𝑘 + (𝐴 𝑘 ) 𝑑𝑘 󵄩󵄩󵄩󵄩 )
󵄩
󵄩
𝑗=1
𝑗∈𝐾
󵄩 󵄩
󵄩 󵄩 𝑗
≥ 𝜂2 𝑘2 ∑ 󵄩󵄩󵄩𝑐𝑘 󵄩󵄩󵄩 min {𝑘3 󵄩󵄩󵄩𝑐𝑘 󵄩󵄩󵄩 , Δ 𝑘 } .
−1
(A4) The matrices (𝐴(𝑥) 𝐴(𝑥)) , 𝑊(𝑥), (𝑊 𝑊) are
uniformly bounded in Ω, where 𝑊(𝑥) denotes a
matrix whose columns form a basis of the null space
of 𝐴(𝑥)𝑇 .
𝑓min ≤ 𝑓𝑘 ,
Proof. It is similar to the proof of Lemma 2 in [13].
𝑗
In this section, to present a proof of global convergence of
algorithm, we always assume that the following conditions
hold.
𝑇
(32)
𝑗∈𝐾
(35)
𝑗
𝑗
By ℎ𝑘 > 𝜖, it holds Δ 𝑘 → 0 for 𝑗 ∈ 𝐾. From Algorithm B, we
𝑗
can obtain that Δ 𝑘 → 0 for all 𝑗.
On the other side,
𝑇 󵄩
󵄩2
󵄩󵄩 𝑗 󵄩󵄩2 󵄩󵄩
󵄩2 󵄩 󵄩2 󵄩󵄩
󵄩󵄩𝑐𝑘 󵄩󵄩 − 󵄩󵄩𝑐 (𝑥𝑘𝑗 + 𝑑𝑘𝑗 )󵄩󵄩󵄩 = 󵄩󵄩󵄩𝑐𝑘𝑗 󵄩󵄩󵄩 − 󵄩󵄩󵄩𝑐𝑘𝑗 + (𝐴𝑗𝑘 ) 𝑑𝑘𝑗 󵄩󵄩󵄩 + 𝑜 (Δ𝑗𝑘 )
󵄩󵄩
󵄩 󵄩 󵄩
󵄩 󵄩 󵄩󵄩
󵄩
(36)
𝑗
𝑗
for Δ 𝑘 → 0. By the algorithm, the radius Δ 𝑘 should be
𝑗+1
𝑗
𝑗
satisfied Δ 𝑘 > Δ 𝑘 , that is contradicted to Δ 𝑘 → 0. The
proof is complete.
Now, we analyze the impact of the criteria (23) and (24).
Once a trial point is accepted as a new iterate, it must be
provided some improvement, and we formalize this by saying
that iterate 𝑥𝑘 = 𝑥𝑝(𝑘)+1 improves on iterate 𝑥𝑖(𝑘) . That is
6
Journal of Applied Mathematics
the trial point 𝑥𝑘 is accepted at iterate 𝑝(𝑘); it happens under
two situations, one is by the criteria (23), that is,
𝑖 (𝑘) = 𝑝 (𝑘)
if 𝑝 (𝑘) ∉ A
(37)
the other is by the criteria (24), that is,
𝑖 (𝑘) = 𝑝 (𝑘)
if 𝑝 (𝑘) ∉ A.
(38)
Now consider any iterate 𝑥𝑘 , it improved on 𝑥𝑖(𝑘) , which
was itself accepted because it improved on 𝑥𝑖(𝑖(𝑘)) , and so on,
until back to 𝑥0 . Hence we may construct a chain of successful
iterations indexed by C𝑘 = {𝑙1 , 𝑙2 , . . . , 𝑙𝑞 } for each 𝑘, such that
Lemma 6. Suppose that Assumptions hold. If Algorithm A
does not terminate finitely and the filter contains infinite
iterates, then lim𝑘 → ∞ ℎ𝑘 = 0.
Proof. Suppose by contradiction that there exists a constant
𝜖 > 0 and infinite sequence {𝑘𝑖 } ⊆ S such that ℎ𝑘𝑖 ≥ 𝜖 for
all 𝑖. Because there are infinite iterations in the filter, we have
|S| = ∞, then ℎ𝑙𝑞 ≥ 𝜖 for ∀𝑞.
area (D (F𝑘 )) ≥ 𝛾F ⋅ 𝑞 ⋅ 𝜖2 .
(46)
𝑗 = 1, 2, . . . , 𝑞 − 1,
(39)
Then by (31), area(D(F𝑘 )) is upper bounded for each 𝑘. That
max
max
≥ 0 such that area(D(F𝑘 )) ≤ 𝜅F
, so
means it exists 𝜅F
max
2
𝑖 ≤ (𝜅F /𝛾F 𝜖 ). Hence 𝑖 must be finite, it contradicts to the
infinity of {𝑘𝑖 }. The proof is complete.
where 𝑙1 is the smallest index in the chain of successful
iterations.
Lemma 7. Suppose that Assumptions hold and Algorithm A
terminate finitely, then ℎ𝑘 = 0.
Lemma 5. Suppose that Assumptions hold and Algorithm A
does not terminate finitely, apply Algorithm A to the problem
(𝑃), then for all 𝑘 and C𝑘 = {𝑙1 , 𝑙2 , . . . , 𝑙𝑞 }, it holds
Proof. From the Algorithm A and the definition of filter, the
conclusion follows.
𝑥𝑙1 = 𝑥0 ,
𝑥𝑙𝑞 = 𝑥𝑘 ,
𝑥𝑙𝑗 = 𝑥𝑖(𝑙𝑗+1 ) ,
𝑞
area (D (F𝑘 )) ≥ 𝛾F ∑ℎ𝑙2𝑗 .
(40)
𝑗=1
Proof. For ∀𝑙𝑗 ∈ C𝑘 , if 𝑝(𝑙𝑗 ) ∈ A, by (24), 𝑖(𝑙𝑗 ) = 𝑟(𝑝(𝑙𝑗 )) =
𝑙𝑗−1 , then
}
{
𝑙𝑗
}
{
}
{
max {𝛼𝑝(𝑙𝑗 ) , ∑ 𝜆 𝑝(𝑖) 𝛼𝑝(𝑖) } ≥ 𝛾F ℎ𝑙2𝑗 .
}
{
}
{
𝑖=𝑙𝑗−1 +1
𝑖∈U
}
{
𝑙
𝑗
If max{𝛼𝑝(𝑙𝑗 ) , ∑ 𝑖=𝑙
𝑗−1 +1, 𝑖∈U
(41)
Lemma 8. For any trial point 𝑥𝑘+1 ≠ 𝑥𝑘 , there must be one
accepted by the filter.
Lemma 9. Suppose that Assumptions hold, there exists 𝑘5 > 0
independent of the iterates such that
󵄩 󵄩
𝑞𝑘 (0) − 𝑞𝑘 (𝑑𝑘𝑛 ) ≥ −𝑘5 󵄩󵄩󵄩𝑐𝑘 󵄩󵄩󵄩 .
󵄩 󵄩
Proof. By (32), the assumptions and 󵄩󵄩󵄩𝑑𝑘𝑛 󵄩󵄩󵄩 ≤ Δ max , it is obvious
that
1
𝑇
𝑞𝑘 (0) − 𝑞𝑘 (𝑑𝑘𝑛 ) = −𝑔𝑘𝑇 𝑑𝑘𝑛 − (𝑑𝑘𝑛 ) 𝐻𝑘 𝑑𝑘𝑛
2
𝜆 𝑝(𝑖) 𝛼𝑝(𝑖) } = 𝛼𝑝(𝑙𝑗 ) ,
𝛼𝑝(𝑙𝑗 ) ≥ 𝛾F ℎ𝑙2𝑗 .
𝑙
𝑗
𝜆 𝑝(𝑖) 𝛼𝑝(𝑖) }
If max{𝛼𝑝(𝑙𝑗 ) , ∑ 𝑖=𝑙
𝑗−1 +1, 𝑖∈U
𝛼𝑝(𝑖) ,
󵄩 󵄩 󵄩 󵄩 1 󵄩 󵄩󵄩 󵄩
≥ − 󵄩󵄩󵄩𝑑𝑘𝑛 󵄩󵄩󵄩 (󵄩󵄩󵄩𝑔𝑘 󵄩󵄩󵄩 + 󵄩󵄩󵄩𝐻𝑘 󵄩󵄩󵄩 󵄩󵄩󵄩𝑑𝑘𝑛 󵄩󵄩󵄩)
2
󵄩󵄩 󵄩󵄩
≥ −𝑘1 󵄩󵄩𝑐𝑘 󵄩󵄩 (]0 + ]3 Δ max )
(42)
=
𝑙
𝑗
∑ 𝑖=𝑙
𝑗−1 +1, 𝑖∈U
(47)
(48)
󵄩 󵄩
= −𝑘5 󵄩󵄩󵄩𝑐𝑘 󵄩󵄩󵄩 .
𝜆 𝑝(𝑖)
def
The proof is complete.
𝑙𝑗
∑ 𝛼𝑝(𝑖) ≥ 𝛾F ℎ𝑙2𝑗 .
(43)
𝑖=𝑙𝑗−1 +1
𝑖∈U
󵄩 󵄩
Lemma 10. Suppose that Assumptions hold and 󵄩󵄩󵄩𝑔̂𝑘 󵄩󵄩󵄩 ≥ 𝜖𝑡 , if
Δ 𝑘 ≤ min{(
If 𝑝(𝑙𝑗 ) ∉ A, 𝑙𝑗−1 = 𝑝(𝑙𝑗 ). By U ⊆ S, it holds
{𝑙𝑗−1 + 1, . . . , 𝑙𝑗 } ∩ U ⊆ {𝑙𝑗−1 + 1, . . . , 𝑙𝑗 } ∩ S = {𝑙𝑗 } . (44)
(
Then from (23), 𝛼𝑝(𝑖) ≥ 𝛾F ℎ𝑙2𝑗 . It implies (43). Moreover
𝑘
𝑞
𝑖=0
𝑖∈U
𝑗=0
area (D (F𝑘 )) ≥ ∑ 𝛼𝑝(𝑖) = ∑ ( ∑ 𝛼𝑝(𝑖) ) .
𝑖=𝑙𝑗−1 +1
𝑖∈U
Together with (42) and (43), the result follows.
]0
)
𝑘1 ]3 √𝜂3 𝛼1
(
𝑙𝑗
(45)
𝜖𝑡
)
2𝑘1 ]3 √𝜂3 𝛼1
2/(2+𝛼2 )
2/(2+𝛼2 )
2/𝛼2
𝜎𝜖𝑡
)
16𝑘1 ]0 √𝜂3 𝛼1
,
𝑘4 𝜖𝑡
,
2
,
(49)
def
} = 𝛿1
one can deduce
𝑞𝑘 (0) − 𝑞𝑘 (𝑑𝑘 ) ≥
𝜎𝜖𝑡
def
Δ 𝑘 = 𝜎Δ 𝑘 .
8
(50)
Journal of Applied Mathematics
7
Proof. By the assumptions and the definition of 𝑑𝑘𝑛 , it holds
󵄩
󵄩
󵄩󵄩 𝑇
󵄩 󵄩
󵄩
󵄩󵄩𝑊𝑘 ∇𝑞𝑘 (𝑑𝑘𝑛 )󵄩󵄩󵄩 = 󵄩󵄩󵄩𝑊𝑘𝑇 (𝑔𝑘 + 𝐻𝑘 𝑑𝑘𝑛 )󵄩󵄩󵄩 ≥ 󵄩󵄩󵄩󵄩𝑔̂𝑘 󵄩󵄩󵄩󵄩 − 󵄩󵄩󵄩𝑊𝑘𝑇 𝐻𝑘 𝑑𝑘𝑛 󵄩󵄩󵄩
󵄩
󵄩
󵄩
󵄩 󵄩
󵄩
󵄩󵄩 󵄩󵄩
≥ 𝜖𝑡 − ]3 𝑘1 󵄩󵄩𝑐𝑘 󵄩󵄩
𝜖
1+(𝛼 /2)
≥ 𝜖𝑡 − ]3 𝑘1 √𝜂3 𝛼1 Δ 𝑘 2 ≥ 𝑡 .
2
(51)
From Lemma 3, 𝑞𝑘 (𝑑𝑘𝑛 ) − 𝑞𝑘 (𝑑𝑘 ) ≥ (𝜎/4)𝜖𝑡 min{Δ 𝑘 , 𝑘4 𝜖𝑡 /2} ≥
(𝜎𝜖𝑡 /4)Δ 𝑘 . Together with
1
𝑇
𝑞𝑘 (0) − 𝑞𝑘 (𝑑𝑘𝑛 ) = − 𝑔𝑘𝑇 𝑑𝑘𝑛 − (𝑑𝑘𝑛 ) 𝐻𝑘 𝑑𝑘𝑛
2
󵄩 󵄩 󵄩 󵄩 1 󵄩 󵄩 󵄩 󵄩2
≥ − 󵄩󵄩󵄩𝑔𝑘 󵄩󵄩󵄩 󵄩󵄩󵄩𝑑𝑘𝑛 󵄩󵄩󵄩 − 󵄩󵄩󵄩𝐻𝑘 󵄩󵄩󵄩 󵄩󵄩󵄩𝑑𝑘𝑛 󵄩󵄩󵄩
2
1+(𝛼2 /2)
≥ − 𝑘1 ]0 √𝜂3 𝛼1 Δ 𝑘
Theorem 13. Suppose the assumptions hold, there must exist
Δ min > 0 such that for each 𝑘, it holds
Δ 𝑘 ≥ Δ min .
Proof. Let 𝑘1 be large enough such that ℎ𝑘1 ≤ 𝜖𝑡 , it is true by
Lemmas 6 and 7. Suppose by contradiction that the index 𝑗 is
the first one after 𝑘1 , which satisfies
{
(1 − √𝛾F ) ℎ𝐹
)
Δ 𝑗 ≤ 𝛾0 min {𝛿2 , (
𝜂3 𝛼1
{
2+𝛼2
1+(𝛼2 /2)
we know 𝑗 ≥ 𝑘1 + 1, that is 𝑗 − 1 ≥ 𝑘1 . From the Algorithm
and (60), it concludes
Δ 𝑗−1 ≤
(52)
then 𝑞𝑘 (0) − 𝑞𝑘 (𝑑𝑘 ) ≥ −(𝜎𝜖𝑡 /8)Δ 𝑘 + (𝜎𝜖𝑡 /4)Δ 𝑘 = (𝜎𝜖𝑡 /8)Δ 𝑘 =
𝜎Δ 𝑘 . It is the conclusion.
1
Δ ≤ 𝛿3 .
𝛾0 𝑗
(61)
By (60) and (61), (53) can be obtained. In Lemma 11, let 𝑗 − 1
instead of 𝑘, it deduces
𝑟𝑗−1 ≥ 𝜂1 .
(62)
Based on Lemma 12, together with (60), (61), and the algorithm, we can see
Lemma 11. Suppose the conditions of Lemma 10 hold, if
(1 − 𝜂1 ) 𝜎 def
} = 𝛿2
]
} def
, Δ 𝑘1 } = 𝛾0 𝛿3 ,
}
(60)
where ℎ𝐹 = min𝑖∈U ℎ𝑖 is the smallest value of violation
function in filter. Then Δ 𝑗 ≤ 𝛾0 Δ 𝑘1 . By the above analysis,
≥ − 2𝑘1 ]0 √𝜂3 𝛼1 Δ 𝑘
Δ 𝑘 ≤ min {𝛿1 ,
1/(2+𝛼2 )
def
− 𝑘12 ]3 𝜂3 𝛼1 Δ 𝑘
𝜎𝜖
≥ − 𝑡 Δ𝑘
8
(59)
(53)
2+𝛼
+
ℎ𝑗−1
≤ 𝜂3 𝛼1 Δ 𝑗−12 ≤ (1 − √𝛾F ) ℎ𝐹 .
(63)
It can be seen that (53) is true for 𝑗 − 1 ≥ 𝑘, with (55), we can
deduce
then 𝑟𝑘 ≥ 𝜂1 .
Proof. From the definition of 𝑟𝑘 and Lemma 10, together with
(31), we have
󵄨
󵄨
2
󵄨 󵄨󵄨𝑓 (𝑥𝑘 + 𝑑𝑘 ) − 𝑞𝑘 (𝑑𝑘 )󵄨󵄨󵄨 ]Δ 𝑘
󵄨󵄨
≤
≤ 1 − 𝜂1 . (54)
󵄨󵄨𝑟𝑘 − 1󵄨󵄨󵄨 = 󵄨 󵄨󵄨
󵄨
𝜎Δ 𝑘
󵄨󵄨𝑞𝑘 (0) − 𝑞𝑘 (𝑑𝑘 )󵄨󵄨󵄨
It is obvious that 𝑟𝑘 ≥ 𝜂1 .
Lemma 12. Suppose the conditions of Lemmas 10 and 11 hold,
if
−1/(1+𝛼2 )
ℎ𝑘 ≤ (𝜂3 𝛼1 )
(
𝜂1 𝜎
)
√𝛾F
(2+𝛼2 )/(1+𝛼2 )
(55)
then
𝑓 (𝑥𝑘+ ) ≤ 𝑓 (𝑥𝑘 ) − √𝛾F ℎ𝑘 .
2+𝛼2
From the Algorithm, ℎ𝑘 ≤ 𝜂3 𝛼1 Δ 𝑘
𝑓 (𝑥𝑘 ) − 𝑓 (𝑥𝑘+ ) ≥ 𝜂1 𝜎(
Hence 𝑓(𝑥𝑘 ) − 𝑓(𝑥𝑘+ ) ≥ √𝛾F ℎ𝑘 .
(57)
, then
ℎ𝑘 1/(2+𝛼2 )
)
.
𝜂3 𝛼1
(64)
+
That means 𝑥𝑗−1
can be accepted by the filter. From above and
(55), we know Δ 𝑗 ≥ Δ 𝑗−1 . Hence the index 𝑗 is not the first
one after 𝑘1 which satisfied (60), that is a contradiction. So,
for any 𝑘 > 𝑘1 , it holds Δ 𝑘 ≥ 𝛾0 𝛿3 . Define
Δ min = min {Δ 0 , . . . , Δ 𝑘1 , 𝛾0 𝛿3 }
(65)
we can see that
Δ 𝑘 ≥ Δ min
(66)
holds for each 𝑘. The proof is complete.
(56)
Proof. By Lemmas 3, 10, and 11, together with ‖ 𝑔̂𝑘 ‖≥ 𝜖𝑡 , it
holds
𝑓 (𝑥𝑘 ) − 𝑓 (𝑥𝑘+ ) ≥ 𝜂1 (𝑞𝑘 (0) − 𝑞𝑘 (𝑑𝑘 )) ≥ 𝜂1 𝜎Δ 𝑘 .
+
≤ 𝑓𝑗−1 − √𝛾F ℎ𝑗−1 .
𝑓𝑗−1
(58)
Lemma 14. Suppose that Assumptions hold and Algorithm A
󵄩 󵄩
does not terminate finitely, then lim inf 𝑘 → ∞ 󵄩󵄩󵄩𝑔̂𝑘 󵄩󵄩󵄩 = 0.
Proof. Suppose by contradiction that for 𝜖𝑡 , there exists a
constant 𝑘 > 0 such that ‖ 𝑔̂𝑘 ‖≥ 𝜖𝑡 .
󵄩
󵄩
󵄩 󵄩
By Assumption (A3) and (A4), 󵄩󵄩󵄩󵄩𝑊𝑘𝑇 ∇𝑞𝑘 (𝑑𝑘𝑛 )󵄩󵄩󵄩󵄩 ≥ 󵄩󵄩󵄩𝑔̂𝑘 󵄩󵄩󵄩 −
]3 𝑘1 ‖ 𝑐𝑘 ‖. From Lemma 6, we know ℎ𝑘 → 0. Hence there
exists ̃𝑘 > 0 such that
2𝜖𝑡
󵄩󵄩 󵄩󵄩
󵄩󵄩𝑐𝑘 󵄩󵄩 ≤
3]3 𝑘1
(67)
8
Journal of Applied Mathematics
for 𝑘 > ̃𝑘. Then ‖ 𝑊𝑘𝑇 ∇𝑞𝑘 (𝑑𝑘𝑛 ) ‖≥ (1/3) ‖ 𝑔̂𝑘 ‖≥ (1/3)𝜖𝑡 for
def
𝑘 > ̂𝑘 = max{𝑘, ̃𝑘}.
It is obvious that
𝑞𝑘 (0) − 𝑞𝑘 (𝑑𝑘 ) = 𝑞𝑘 (0) − 𝑞𝑘 (𝑑𝑘𝑛 ) + 𝑞𝑘 (𝑑𝑘𝑛 ) − 𝑞𝑘 (𝑑𝑘 ) .
(68)
By the proof of Lemma 9, it holds |𝑞𝑘 (0) − 𝑞𝑘 (𝑑𝑘𝑛 )| ≤
]0 ‖ 𝑑𝑘𝑛 ‖ +(1/2)]3 ‖ 𝑑𝑘𝑛 ‖2 . Together with Lemma 6 and the
definition of 𝑑𝑘 , we have
lim (𝑞𝑘 (0) − 𝑞𝑘 (𝑑𝑘𝑛 )) = 0.
𝑘→∞
(69)
By the Algorithm, we can get
∞
∞
𝑘=0
𝑘=0
+∞ > ∑ (𝑓𝑘 − 𝑓𝑘+1 ) ≥ 𝜂1 ∑ (𝑞𝑘 (0) − 𝑞𝑘 (𝑑𝑘 )) .
(70)
Table 1
Problem
HS6
HS7
HS8
HS9
HS26
HS39
HS40
HS42
HS78
𝑛
2
2
2
2
3
4
4
4
5
𝑚
1
1
2
1
1
2
3
2
3
NF
11
9
7
6
24
15
7
8
6
NG
11
3
4
6
24
9
5
8
6
L’s NF
20
15
—
6
42
26
7
11
8
L’s NG
12
11
—
6
24
19
5
11
7
Proof. By Assumption (A1), there exist a subsequence {𝑘𝑗 }
and 𝑥∗ , such that lim𝑗 → ∞ 𝑥𝑘𝑗 = 𝑥∗ . Together with Assump-
tion (A3) and (A4), it hods lim𝑗 → ∞ 𝑊𝑘𝑇𝑗 𝑔𝑘𝑗 = 0, which means
for large enough 𝑗, 𝑔𝑘𝑗 lies in the space spaned by the columns
of 𝐴𝑇𝑘𝑗 . That is there exists 𝜆 𝑘𝑗 such that
Then
lim (𝑞𝑘 (0) − 𝑞𝑘 (𝑑𝑘 )) = 0.
𝑘→∞
(71)
By (68), (69), and (71), it deduces
lim (𝑞𝑘 (𝑑𝑘𝑛 ) − 𝑞𝑘 (𝑑𝑘 )) = 0.
𝑘→∞
The conclusion follows.
𝑞𝑘 (𝑑𝑘𝑛 ) − 𝑞𝑘 (𝑑𝑘 )
≥
𝜎 󵄩󵄩 𝑇
󵄩
󵄩
󵄩
󵄩󵄩𝑊 ∇𝑞 (𝑑𝑛 )󵄩󵄩󵄩 min {Δ 𝑘 , 𝑘4 󵄩󵄩󵄩𝑊𝑘𝑇 ∇𝑞𝑘 (𝑑𝑘𝑛 )󵄩󵄩󵄩}
󵄩
󵄩 (73)
2󵄩 𝑘 𝑘 𝑘 󵄩
≥
𝜎𝜖𝑡
𝑘𝜖
min {Δ min , 4 𝑡 } > 0,
6
6
Theorem 15. Suppose the assumptions hold, and apply the
algorithm to problem (𝑃), then
(74)
where 𝑔̂𝑘 = 𝑊𝑘𝑇 𝑔𝑘 , 𝑔𝑘 = ∇𝑓(𝑥𝑘 ), 𝑊(𝑥) denotes a matrix
whose columns form a basis of the null space of 𝐴(𝑥)𝑇 .
Proof. If the algorithm terminates finitely, it is obvious that it
holds. Otherwise, by Lemmas 6 and 14, the conclusion also
can be obtained.
Theorem 16. Suppose the assumptions hold, and {𝑥𝑘 } is the
infinite sequence obtained by the algorithm, then there must
exist a subsequence such that
lim 𝑥𝑘𝑗 = 𝑥∗
and 𝑥∗ satisfies the one order KKT condition of (𝑃).
6. Some Numerical Experiments
(1) Updating of 𝐻𝑘 is done by 𝐻𝑘+1 = 𝐻𝑘 +(𝑦𝑘𝑇 𝑦𝑘 /𝑦𝑘𝑇 𝑠𝑘 )−
(𝐻𝑘 𝑠𝑘 𝑠𝑘𝑇 𝐻𝑘 /𝑠𝑘𝑇 𝐻𝑘 𝑠𝑘 ), where 𝑦𝑘 = 𝜃𝑘 𝑦̂𝑘 + (1 − 𝜃𝑘 )𝐻𝑘 𝑠𝑘
1
{
{
𝜃𝑘 = { 0.8𝑠𝑘𝑇 𝐻𝑘 𝑠𝑘
{ 𝑇
𝑇
{ 𝑠𝑘 𝐻𝑘 𝑠𝑘 − 𝑠𝑘 𝑦̂𝑘
𝑠𝑘𝑇 𝑦̂𝑘 ≥ 0.2𝑠𝑘𝑇 𝐻𝑘 𝑠𝑘
(77)
and 𝑦̂𝑘 = 𝑔𝑘+1 − 𝑔𝑘 + (𝐴 𝑘+1 − 𝐴 𝑘 )𝜌𝑘 , 𝑠𝑘 = 𝑥𝑘+1 −
𝑥𝑘 𝜌𝑘 is the multipluser of corresponding quadratic
subproblems.
which contradicts (72). The conclusion follows.
𝑗→∞
(76)
(72)
Based on the assumptions, Lemma 3 and Theorem 13, for 𝑘 >
̂𝑘, it holds
󵄩 󵄩
lim inf (ℎ𝑘 + 󵄩󵄩󵄩𝑔̂𝑘 󵄩󵄩󵄩) = 0,
𝑘→∞
lim 𝑔𝑘𝑗 + 𝐴𝑇𝑘𝑗 𝜆 𝑘𝑗 = 0.
𝑗→∞
(75)
(2) We assume the error toleration is 10−5 .
(3) The algorithm parameters were set as follows: 𝐻0 =
𝐼 ∈ 𝑅𝑛×𝑛 , 𝜂1 = 0.25, 𝜂2 = 0.25, 𝜂3 = 0.1, 𝛼1 = 𝛼2 = 0.5,
𝛾 = 0.02, 𝜌 = 0.5, 𝜉 = 10−6 , 𝛾0 = 0.1, 𝛾1 = 0.5, 𝛾2 = 2,
𝛾𝐹 = 10−4 , Δ 0 = 1. The program is written in Matlab.
The numerical results for the test problems are listed in
Table 1.
In Table 1, the problems are numbered in the same way
as in Schittkowski [20] and Hock and Schittkowski [21]. For
example, “S216” is the problem (216) in Schittkowski [20]
and “HS6” is the problem (6) in Hock and Schittkowski
[21]. NF, NG represent the numbers of function and gradient
calculations and “L’s” is the solution in [22]. The numerical
results show that the our algorithm is more effective than the
L’s for most test examples. Moreover, the higher the level of
nonmonotonic, the better the numerical results. The results
show that the new algorithm is robust and effective, and more
flexible for the acceptance of the the trial iterate.
Journal of Applied Mathematics
Acknowledgments
This research is supported by the National Natural Science
Foundation of China (no. 11101115) and the Natural Science
Foundation of Hebei Province (nos. A2010000191, Y2012021).
References
[1] R. H. Byrd, R. B. Schnabel, and G. A. Shultz, “A trust region
algorithm for nonlinearly constrained optimization,” SIAM
Journal on Numerical Analysis, vol. 24, no. 5, pp. 1152–1170, 1987.
[2] J. E. Dennis Jr., M. El-Alem, and M. C. Maciel, “A global
convergence theory for general trust-region-based algorithms
for equality constrained optimization,” SIAM Journal on Optimization, vol. 7, no. 1, pp. 177–207, 1997.
[3] M. J. D. Powell and Y. Yuan, “A trust region algorithm for
equality constrained optimization,” Mathematical Programming
A, vol. 49, no. 2, pp. 189–211, 1991.
[4] R. Fletcher and S. Leyffer, “Nonlinear programming without a
penalty function,” Mathematical Programming A, vol. 91, no. 2,
pp. 239–269, 2002.
[5] C. M. Chin and R. Fletcher, “On the global convergence of
an SLP-filter algorithm that takes EQP steps,” Mathematical
Programming A, vol. 96, no. 1, pp. 161–177, 2003.
[6] R. Fletcher, N. I. M. Gould, S. Leyffer, P. L. Toint, and A.
Wächter, “Global convergence of a trust-region SQP-filter
algorithm for general nonlinear programming,” SIAM Journal
on Optimization, vol. 13, no. 3, pp. 635–659, 2002.
[7] R. Fletcher, S. Leyffer, and P. L. Toint, “On the global convergence of a filter-SQP algorithm,” SIAM Journal on Optimization,
vol. 13, no. 1, pp. 44–59, 2002.
[8] S. Ulbrich, “On the superlinear local convergence of a filter-SQP
method,” Mathematical Programming B, vol. 100, no. 1, pp. 217–
245, 2004.
[9] M. Ulbrich, S. Ulbrich, and L. N. Vicente, “A globally convergent primal-dual interior-point filter method for nonlinear
programming,” Mathematical Programming A, vol. 100, no. 2,
pp. 379–410, 2004.
[10] R. Fletcher and S. Leyffer, “A bundle filter method for nonsmooth nonlinear optimization,” Tech. Rep. NA/195, Department of Mathematics, University of Dundee, Dundee, Scotland,
December 1999.
[11] L. Grippo, F. Lampariello, and S. Lucidi, “A nonmonotone
line search technique for Newton’s method,” SIAM Journal on
Numerical Analysis, vol. 23, no. 4, pp. 707–716, 1986.
[12] M. Ulbrich and S. Ulbrich, “Non-monotone trust region methods for nonlinear equality constrained optimization without a
penalty function,” Mathematical Programming B, vol. 95, no. 1,
pp. 103–135, 2003.
[13] K. Su and D. Pu, “A nonmonotone filter trust region method for
nonlinear constrained optimization,” Journal of Computational
and Applied Mathematics, vol. 223, no. 1, pp. 230–239, 2009.
[14] K. Su and Z. Yu, “A modified SQP method with nonmonotone
technique and its global convergence,” Computers & Mathematics with Applications, vol. 57, no. 2, pp. 240–247, 2009.
[15] N. I. M. Gould and P. L. Toint, “Global convergence of a nonmonotone trust-region filter algorithm for nonlinear programming,” in Multiscale optimization methods and applications, vol.
82, pp. 125–150, Springer, New York, NY, USA, 2006.
9
[16] Z. Chen and X. Zhang, “A nonmonotone trust-region algorithm
with nonmonotone penalty parameters for constrained optimization,” Journal of Computational and Applied Mathematics,
vol. 172, no. 1, pp. 7–39, 2004.
[17] Z. W. Chen, “A penalty-free-type nonmonotone trust-region
method for nonlinear constrained optimization,” Applied Mathematics and Computation, vol. 173, no. 2, pp. 1014–1046, 2006.
[18] F. A. M. Gomes, M. C. Maciel, and J. M. Martı́nez, “Nonlinear
programming algorithms using trust regions and augmented
Lagrangians with nonmonotone penalty parameters,” Mathematical Programming A, vol. 84, no. 1, pp. 161–200, 1999.
[19] M. J. D. Powell, “Convergence properties of a class of minimization algorithms,” in Nonlinear Programming, pp. 1–27, Academic
Press, New York, NY, USA, 1974.
[20] K. Schittkowski, More Test Examples for Nonlinear Programming
Codes, vol. 282, Springer, Berlin, Germany, 1987.
[21] W. Hock and K. Schittkowski, Test Examples for Nonlinear Programming Codes, vol. 187, Springer, Berlin, Germany, 1981.
[22] M. Lalee, J. Nocedal, and T. Plantenga, “On the implementation
of an algorithm for large-scale equality constrained optimization,” SIAM Journal on Optimization, vol. 8, no. 3, pp. 682–706,
1998.
Advances in
Operations Research
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Advances in
Decision Sciences
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Mathematical Problems
in Engineering
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Journal of
Algebra
Hindawi Publishing Corporation
http://www.hindawi.com
Probability and Statistics
Volume 2014
The Scientific
World Journal
Hindawi Publishing Corporation
http://www.hindawi.com
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
International Journal of
Differential Equations
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Volume 2014
Submit your manuscripts at
http://www.hindawi.com
International Journal of
Advances in
Combinatorics
Hindawi Publishing Corporation
http://www.hindawi.com
Mathematical Physics
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Journal of
Complex Analysis
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
International
Journal of
Mathematics and
Mathematical
Sciences
Journal of
Hindawi Publishing Corporation
http://www.hindawi.com
Stochastic Analysis
Abstract and
Applied Analysis
Hindawi Publishing Corporation
http://www.hindawi.com
Hindawi Publishing Corporation
http://www.hindawi.com
International Journal of
Mathematics
Volume 2014
Volume 2014
Discrete Dynamics in
Nature and Society
Volume 2014
Volume 2014
Journal of
Journal of
Discrete Mathematics
Journal of
Volume 2014
Hindawi Publishing Corporation
http://www.hindawi.com
Applied Mathematics
Journal of
Function Spaces
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Optimization
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014