HD28
.M414
V
DEWEY
ZoSo-
ALFRED
P.
WORKING PAPER
SLOAN SCHOOL OF MANAGEMENT
Projective Transformations for Interior Point
Methods, Part
II:
Analysis of
for finding the
An
Algorithm
Weighted Center
of a Polyhedral
System
Robert M. Freund
Sloan W.P. No. 2050-88
October 1988
MASSACHUSETTS
INSTITUTE OF TECHNOLOGY
50 MEMORIAL DRIVE
CAMBRIDGE, MASSACHUSETTS 02139
MASSACHUSETTS INSTITUTE
OF TECHNOLOGY
NOV
1 3 2000
LIBRARIES
Projective Transformations for Interior Point
Methods, Part
II:
Analysis of
for finding the
An
Algorithm
Weighted Center
of a Polyhedral
System
Robert M. Freund
Sloan W.P. No. 205U-88
October 1988
Projective Transformations for Interior Point Methods, Part
Analysis of
An
II:
Algorithm for finding the
Weighted Center of
a
Polyhedral System
Abstract
In Part
II
of this study, the basic theory of Part
I
applied to the
is
X We
present a
problem of finding the w-center of a polyhedral system
projective transformation algorithm, analagous but more general than
.
X
Karmarkar's algorithm, for finding the w-center of
The algorithm
At each iteration, the algorithm either
improves the objective function (the weighted logarithmic barrier function)
by a fixed amount, or at a linear rate of improvement. This linear rate of
improvement increases to unity, and so the algorithm is superlinearly
convergent. The algorithm also updates an upper bound on the optimal
objective value of the weighted logarithmic barrier function at each iteration.
The direction chosen at each iteration is shown to be positively proportional
to the projected Newton direction. This has two consequences. On the
theoretical side, this broadens a result of Bayer and Lagarias regarding the
connection between projective transformation methods and Newton's
method. In terms of algorithms it means that our algorithm specializes to
Vaidya's algorithm if it is used with a line search, and so we see that Vaidya's
algorithm is superlinearly convergent as well. Finally, we show how to use
the algorithm to construct well-scaled containing and contained ellipsoids
centered at near-optimal solutions to the w-center problem. After a fixed
number of iterations, the current iterate of the algorithm can be used as an
approximate w-center, and one can easily construct well-scaled containing
and contained ellipsoids centered at the current iterate, whose scale factor is of
the same order as for the w-center itself.
.
exhibits superlinear convergence.
Keywords:
analytic center, w-center, projective transformation,
Newton method,
program.
Robert M. Freund, Sloan School of Management, M.I.T.,
50 Memorial Drive, Cambridge, Mass. 02139.
Author:
Abbreviated
ellipsoid, linear
Title:
Projective Transformations, Part
II.
Introduction
L
Part
of this study uses the projective-centering
II
methodologies developed
in Part
I
[
3
,
]
to
and the
improvement
local
develop an algorithm for finding the
A
w-center of a polyhedral system, which
is
to the
the solution x
problem
m
Pw
:
maximize
V w
=
F(x)
- AjX)
In (bj
4
i=l
Ax
subject to
+
s
=b
(1.1)
s>0
Mx = g
m
w
where
= (wj,
.
.
.
,
wm
are positive weights that are normalized to
)
V w
=
1,
4
i=l
and
F(-
is
)
a (weighted) logarithmic barrier function.
generalization of the analytic center problem defined
nonuniform positive weights on
all
by Sonnevend
If
and
X
of
.
is
bounded
To be more
since the solution to
X
.
However,
where
8
],
[
9
],
there exists a point
X
if
[
it is
,
precise,
as in Part
is
I
understood that
hyperplanes.
x e
then
Pw
among
Pw
[
is
11
a
], [
12
]
,
for
This problem has had numerous
constraints.
applications in mathematical programming, see Renegar
Monteiro and Adler
Problem P w
[
10
],
Gonzaga
[
5
],
and
others.
X = {x e
Rn
Ax < b, Mx =
g) for which Ax < b,
A
will have a unique solution x , called the w-center
we should
|
say that
A
x
is
the w-center of
X (A, b, M,
g),
dependent on the particular polyhedral representation of
of this study,
X
we
will refer to
A
x
as the w-center of
X
represents a specific intersection of half-space and
,
For the case
algorithms
known
problem. Vaidya
[
when
all
are identical, there are
4
two other
author that have been developed for the w-center
to this
14
w
weights
has developed an algorithm that constructs the
]
Newton-direction from the current iterate and then performs an inexact
He shows
in this direction.
improvement
that at each iteration, there
improvement
in F(x) or a linear rate of
exhibits linear convergence.
for finding the center, that
is
Censor and Lent
convergent
to
[
2
A
x
,
is
line search
either constant
in F(x),
and so
his algorithm
present a primal-dual algorithm
]
but not necessarily in any strong
sense.
The algorithm developed
in this
paper
is
based on the use of projective
transformation methods to w-center a given point, as in Section rV of Part
iteration the
to a
polyhedron
polyhedron Z
direction
d
is
or unbounded.
X = (x e
Rn
so that the current point
If it is
.
A
bounded,
will then
it
steplength
a
is
The algorithm can be run with
by the use of
iteration, there is either a constant
one
Mx =
x
in F(x).
is
g]
the w-center of
is
used
If
A
Pw
search
is
a determined
The new point x NEW
x +
a d back from Z
improvement
in
F(x)
,
the linear rate of
then determined by
is
to
the steplength
is
is
analytically at each
X
.
At each
or a linear rate of
improvement approaches
The search
positively proportional to
chosen
at
each iteration by a line
search, the algorithm specializes to Vaidya's algorithm (with equal weights),
shows (obliquely)
bounded
then computed.
However, because
direction.
.
produce an (updated) upper bound on
direction determined at each iteration of the algorithm
Newton
Z
first to test if
in the limit, the algorithm exhibits superlinear convergence.
the projected
At each
projectively transformed
is
a steplength
a line search.
projectively transforming the point
improvement
< b,
then determined. This direction
A
the value of F(x)
iteration, or
Ax
|
I.
and
this
that Vaidya's algorithm exhibits superlinear convergence, verifying
a conjecture of
Vaidya
15
[
might exhibit stronger convergence
that his algorithm
]
properties.
As was shown
in Section
c X c E OUT
—
((1
at the
I,
ellipsoid
A
w-center x of
E OUT
X
one can construct
with property that
A
A
x
,
of Part
and an outer
an inner ellipsoid E 1N
E IN
II
the center of
is
and (E OUT - x
,
=
)
—
A
—
- w)/ w)(E
IN
E IN and E OUT
w = min
x), where
[wj]
This ratio
.
(m -
is
1)
when
all
i
A
weights are identical. Thus x
we
algorithm
is
number
construct ellipsoids F IN and FOUT
(FOUT - x
is
(1.75m +
the
same
= (1.75/w +
)
5)
A
x
which
as for
is
is
of iterations
it
,
it
may
)
.
When
all
x
In Section IV,
we prove
Section VI
[
14
]
,
we show
we show
by showing
II
and unboundedness
that the objective value
that the algorithm
the relationship
that the direction
proportional to a projected
fixed
number
of iterations,
,
x "close
one can easily
weights are identical, then
0(1/ w)
this ratio is
,
,
and
this ratio
which
is
presents the projective
.
In Section
III,
we
algorithm are valid.
tests in the
bounds produced by the algorithm
each iteration are valid, and prove that the algorithm
In Section V,
never reach the
c X c F OUT
transformation algorithm for solving the w-center problem P w
tests
Although the
•
organized as follows: Section
prove that the optimality
.
will exhibit a point
with the property that F IN
O(m). In general, the order of
E IN and E Q ut
The paper
,
- x
(F IN
5)
X
of
following sense: at the point
to the w-center, in the
X
w-balanced point of
present converges to the w-center
w-center. However, after a fixed
enough"
in a sense a
is
at
linearly convergent in F(x).
exhibits superlinear convergence.
In
between the algorithm and Vaidya's algorithm
d determined
at
each step
Newton
direction.
one can
easily construct ellipsoids
In SectionVH,
is
positively
we show
that after a
F IN and F OUT
about
(1.75/w + 5) (F IN - x
A
EL
)
Section
.
VEQ contains
study
[
3
]
.
We
We
conventions.
assume
the reader
also will cite
Let the given data (A, b,
X = {x e
w>
X c F OUT
,
and
(F OUT
Rn
|
Ax < b, Mx =
g]
are exactly the
same
familiar with these notation
of the results presented in Part
M,
define the polyhedral system
w e Rm
Let
.
solving the w-center problem
g)
w=
1,
in
)
=
.
I
of
and
I.
be a given vector of weights satisfying
where e =
P w given
X
as in Part
many
that e T
and normalized so
is
- x
closing remarks.
Projective Transformation Algorithm for Finding the w-center of
The notation and conventions used here
this
c
x with the property that F 1N
the current iterate
(1.1).
(1,
.
.
1)
,
.
T
We make
.
Our
interest lies in
the following
assumptions regarding the data:
(2.1a)
The matrix
A
mxn
is
and has rank
n.
(2.1)
The matrix
(2.1b)
M
is
k x n
These assumptions are
and has rank
for convenience,
k.
and
if
A
or
M
lacks full rank, then
one can either eliminate variables or constraints, or one can replace certain matrix
inverse operations with pseudoinverse operations in the analysis.
Exactly in the spirit of Karmarkar's algorithm and consistent with the local
improvement algorithm of Section
solving
Pw
.
polyhedral system
x = g
,
and
of Part
I,
we have
Let the data for the problem be [w, A, b,
vector of weights satisfying
M
V
e
>
X
,
is
x
is
w>0
and
eT
w=
the starting point,
1,
M,
(A, b,
the following algorithm for
g,
M,
which must
the optimality tolerance.
x, e
]
Here
.
w
is
the
g) are the data for the
satisfy
s
=
b-A x>0
and
w = min/w
Let
1
we will
Because
J.
w/(l - w) extensively
use the quantity
i
we
throughout the description of the algorithm and the subsequent analysis,
w/(l - w)
the constant k =
bound on
for convenience.
Pw
the optimal objective value of
Also, in the algorithm, F*
is
define
an upper
.
Algorithm for the W-Center Problem
StepO
w=min{Wi }.
Set
Set k =
(
w/(l - w))
Set F* = +°°
.
.
i
Step
Set
1
b-
A
x
y =
,
AT
_1
S
w
.
(Projective Transformation of constraints)
Step 2
Step 3
s~=
(Compute
direction in
d be the solution
Let
Z
A
=
A-
sy T
space)
to the
problem
(P3)
:
- y Td
maximize
d
subject to
dT
A S^W
S' 1
Ad
< k
Md =
If this
If
y
problem
T d =
0,
is
unbounded, then
stop,
x
solves
Pw
stop.
Pw
is
unbounded.
.
Step 4 (Perform Boundedness Tests and Update Upper Bound).
Set y
If
= (-y T d) /k
y £ l/k,stop.
Problem P w
is
unbounded, and d
then set F* = min {F*, F( x) + y +
2
If
Y <
If
2
Y < .08567 then set F* = min {f*, F( x) + .669kY }
1
Step 5 (Compute Steplength)
Set cc=
1
;
Vl+2 Y
is
Y /(2(l-Y))}
a ray of
X.
)
Set z NEW =
a
x +
Step 6 (Transform back
to original
XNEW =
d.
X
space
)
z new ~ x
X +
1
y^NEW"
+
Step 7 (Stopping Criterion)
Otherwise go
x <- x NEW
Set
Step
to
*>
F* - F( x)
If
.
<
e
,
stop.
1.
Note
This algorithm has the following straightforward explanation.
Pw
is
an instance of the canonical optimization problem
(5.1) of Part
I,
first that
namely
m
P
minimize F
:
x,s
q p
ln(q-p Tx) -
=
(x)
£w
ln(bj
- AjX
t
)
i=1
subject
Ax
to:
+
s
=b
s
>
(2.2)
Mx = g
T
p x <q
where q =1 and p =
,
(0,
.
.
.
,
0)
T
,
and hence
F(x)
= -F
(x)
=
-Fj
(x)
can proceed with the local improvement algorithm presented in Section
Let
x e int
projectively
Z
X
be given,
transform
= {z e
Rn
with the function
|
let
X
s
= b-
=
is
1
,
and
(A - sy T )z < b - Ty T x
z = g(x) =
P
x
y =
let
AT
S
_1
w
.
Thus we
of Part
I.
Then we can
to
X— X
~
x +
1
Problem P
A
V
.
-y
l
transformed to
,
Mz =
=-
(x- x
,
g}
as in (3.2), (3.3)
and
(3.4)
of Part
I.
[l_ y T x
(A - sy T )z +
subject to
z ) _
£ w
t
= b-
sy T x
t
>
]_[_ y T]
Uz
equivalent to
is
Parti.
Note
sy T
Z
w-center of
A
<
,
p replaced -y
,
A
and p replaced by
(2.3)
- yT x
1
problem
q replaced by
1
(5.1) of
- yT x
,
and -y) has the property
that
Lemma
Part
I
x +
3.2
to
replaced
x
the
is
of Part
(5.3)
of
(ii)
A
with
Because
etc.
d given by the solution
then the direction
,
.
(and, of course (1.1)) according to
(2.2)
that (2.3) is also an instance of
A= A-
by
lnt
= g
-y Tz
which
.
d maximizes
I
(with
-y T z
over
ze
{zg Rn |Mz = g,(z- x) T
E IN =
Let y = -y T d
w
- w)/
(1
precisely the quantity
)
a = 1 - 1/\
g(-
),
1
+ 2y
namely x =
.
)
and z NEW =
< G( x
—
h(z) =
^NEW
—
x +
1
+ y
(
F(x NEW ) - F( x) = F
from
Lemma
3.2
(ii)
I
W
S" 1
A(z- x)< w/(l
(note that
)
-
(
x +
a
d.
x +
— x
(
x)
+
z— x
=
=-
yUz-
x
=-
z new ~ x
1#
q - pT x =
I.
This
1).
a=
1
X
V^+2y
)
.
is
-1/\1 + 2y
5.1
if
space using the inverse of
(see (3.4) of Part
I)
,
we obtain
)
as in Step 6 of the algorithm,
(x NEW )
>
(
w/(l -
w
))
and
(l + y -
,
of Part
'
- F^
of Part
- w)}
Then by Corollary
w/(l - w)) (l + y-
Projectively transforming back to
1
x new =
!
defined in Step 4 of the algorithm. Let
y
= G( x + a d
S-
as in (5.6) of Part
as in Step 5 of the algorithm,
G(z NEW
AT
Vl+2y )
,
(2.4)
I,
In particular,
will use y
whenever y
^ -08567, then
F(x NEW ) - F( x) >
is
and we
greater than or equal to a given constant,
we have
(
w
w/(l -
))
(.0033)
(2.5)
we make
Before proceeding with the analysis and verification of the algorithm,
the following remarks.
Remark
Use of
2.1.
Because the projective transformation
line search.
directions relative to
x,
one needs
8 >
x + 8 d feasible.
and
there will be at
)
(3.3) of
(
Part
Because F(x)
most one maximizer of
start the line search
I)
X
preserves
directly.
value of 8 that nearly maximizes F( x + 5 d
to find a
F( x
8 =
with
corresponds to a step of size a =
1
+
ay
- l/*\
1
1
concave over
is strictly
+ 8 d
zr-=
1
Z
g(-
the line search can be performed in the space
Specifically,
could
Steps 5 and 6 of the algorithm can be replaced by a
a line search.
x e
X
,
then
over the feasible range of 8
)
over
)
One
.
where a = l-l/"Vl+2v which
,
d
+ 2y
in the projectively
transformed space
.
Remark
2.2.
Efficient
algorithm of Part
dense matrices
procedure
projected
IIL
is
A
I,
one can compute
=
A-
sy T or
d in Step
Newton
As
3.
in the linear
programming
d without working with the possible very
Q=A
S'^W
deferred to Section VI, where
S" 1
A
we show
.
The discussion
that
d
is
of this
proportional to the
direction.
Optimality Test and Unboundedness Test
In this section,
valid,
Computation of
and
that the
we show
that the optimality test of Step 3 of the algorithm
unboundedness
tests of
Steps 3 and 4 are valid.
is
Proposition
(P3)
unbounded, then P w
is
Suppose
Proof:
r
e
the algorithm terminates in Step 3 because optimization
If
3.1.
Rn
is
unbounded.
that at Step 3, that (P3)
such that
T r = -1
y
Mr
,
problem
=
is
unbounded. That means there
and Ar =
0,
0.
Ar =
But
exists a ray
Ar = sy T r =
implies
m
-
s
<
0.
Thus
X
a ray of
r is
,
and
F( x + 6r) =
V
Wj
In
(
s, (1
+ 6)
)
-> +°°
as
i=l
-»oo.
Proposition
3.2.
the algorithm terminates at Step 3 with y T
If
d =0, then
x solves
IV
Proof:
The solution
-y = 2 pA T
0.
0,
S"
1
W
S"
We also obtain
P =
so that
Thus from
Proposition
3.3.
1
(3
or
(2.1) of
d
A
will satisfy the following optimality conditions:
d -
cFa 7
A
t^M where
,
S' 1
d =
Part
I,
W
solves
the optimization
If
1
A
d = (1/2) (-y T d +
Pw
Then
If
not, then let r
straightforward to
it is
then Ar =
y
= d1 - d2
problem
sy T r
,
and y T r *
T r = y^d 1 - T d 2 =
y
0,
because
is
(P3)
that
S'
1
A
T
d) = (1/2) (-y d) =
wT
S
_1
A
t^M
=
in Step 3 has a solution
with a
Ar =
for otherwise
,
d2
solve the optimization problem.
and Mr =
A
0.
Because
would not have rank
d 1 and d 2 are both optimal
A=An.
solutions,
sy T
However,
which
is
a
contradiction.
Proposition
ray of
X
.
3.4.
If
Algorithm 3 stops
=
d)
unique.
where d 1
show
t^M
i.e.
,
cFa 1 S^W
.
nonzero optimal value, then that solution
Proof:
j^M
y =
In either case,
.
x
S"
p(k -
and
p>
in Step 4, then
Pw
is
unbounded, and
d
is
a
,
10
From Theorem
Proof:
Part
I,
and
Algorithm 3 stops in Step
Z
Because z e
However, we
Az <
s (y
Furthermore
Proposition
has
and
full
Pw
is
Section IV.
,
1
b - sy T x
have y z = y
1
T x-
1)
<
.
d *
,
)
3.2).
rank
a <
a
e [-1,1].
and a >
0,
Z
as defined in (3.2) of
by Theorem
as given
In particular, let
whereby z = x +
ct
= l/(ky).
2.1 of
Then
if
adeZ.
,
also
Thus A(z - x
4,
for all
Z
the w-center of
is
(A - sy T )z < b - sy T x so that
,
syT z +
Az <
x
I,
the inner ellipsoid E IN for
lies in
x+adeZ
Thus
Parti.
d
x +
4.1 of Part
.
1
ay
x +
is
d=y
=b-s
+ b - sy T x
But z - x
i
1
=
(see (2.1) ),
Ad<0
—
y
M
and
d =
A
0,
d*
x -1.
d
,
Thus
so that
would have stopped
so that
0.
1
Ax.
a positive scalar multiple of
for otherwise the algorithm
Next, observe that
vT d
=
x + *ky
Thus
d
is
F( x
a ray of
+
d <
0.
in Step 3 (see
X
d) ->
A
.
+~
Because
as
6 ->
A
<«
unbounded.
Linear Convergence and Improved Optimal Objective Value Bounds
The purpose
of this section
is
to establish the
the algorithm for the w-center problem:
following three results regarding
1
1
Lemma
4.1.
(i)
(Optimal Objective Value Bounds)
if
y<1, then P w has
F( x
)
< F(
xj
a
2
Note
that
Lemma
4.1
validates the
A
x
unique optimal solution
+ y +
A
<
then
F(
x
.08567,
Y
At Step 4 of the algorithm
(1
,
,
and
- y)
->
)
< F(
xj
+ .669 kY^
upper bounding procedure presented
in Step 4 of
the algorithm.
Lemma
(i)
if
(ii)
if
Lemma
Improvement). At Step 6 of the algorithm,
4.2 (Local
y > .08567, F(x NEW )
y < .08567,
4.3 (Linear
> F(
F(x NEW ) -
xj
+ (.0033) k
> .4612 kY2
F( x)
Convergence). At each
.
.
iteration, at least
one of the following
is
true:
(i)
F(xNEW )
(ii)
F( x
> F(
x)
+ (.0033) k
_
A
A
)
In SectionV,
- F(x NEW ) <
we
will
show
.32 (F( x
)
- F(
x converge to
A
x
to zero, thus establishing superlinear
,
x)
),
where
A
x
is
the w-center of
stronger than
Lemma
4.3,
that the constant .32 in
Lemma
4.3
a result that
_
that as the iterates
.
is
convergence.
X
.
namely
(ii)
will
go
1
Note
Lemma
Lemma
that
4.3(i) is a
F(x NEW ) -
Lemma
F( x
KJF
4.1
Lemma
and
(ii)
Lemma
To prove Lemma
4.2(i).
4.1
4.3(11),
and
4.2.
note that
if
4.2(H),
.4612
)
>
=
- F(
)
an immediate consequence of
is
restatement of
< .08567, then from
F( x
4.3
-T7gbby
>
-68,
-
x)
and so
F( x
We
- F(x NEW )
)
A
F( x
—
Z
- F( x
)
=
F(x NEW ) -
-
1
F( x
)
Lemmas
thus need to prove
F( x
~
*
)
4.1
- F(
and
)
^
-
1
.68
=
.32
.
x)
4.2.
We
start
by asserting some elementary
inequalities.
Proposition
(Inequalities)
4.1.
a)
In (1 + x) < x
b)
In (1 + x)
c)
1
+ y -
d)
1
+ y
-
f(h - ln(l +
»2
< x - .378 x 2
V1
+ 2y
-^1 +2y
h)h
>
^
whenever -1 < x <
whenever -1 < x <
VI + 8 -
I
x2
-4612 y2
whenever
(a)
follows from the fact that [h - ln(l + h)
h >
(b)
follows from
\1 +
-
Vl +26
substituting
2
) / 6
9 = .08567
We now
will
(a)
is
by substituting h =
decreasing in 6
.
Vl +29 )/Q 2 \y2
Proof:
-1.
.5
for
.5.
>
]
/
whenever
0<y^
h2
h.
is
0<y<9.
.08567.
decreasing in h for
(c)
follows from the fact that
0.
(d) follows
from
(c)
by
.
prove
Lemma
4.2,
followed by
Lemma
4.1(i)
and Lemma
4.1
(11).
y
2
13
Proof of
Lemma
F(x NEW ) -
(2.4),
Proof of
X
x e
and
,
let
is
(i)
a restatement of inequality
"^
> k (l + y -
F( xj
l+y~Vl+2y
Proposition 4.1(d),
statement
Statement
4.2.:
+ 2y )
6 = .08567.
Let
.
> .4612 y2
Then according
< y < .08567
for
According
(2.5).
to
to
which proves
,
(ii).
Lemma
let
z = g(x)
Z
be given in
x be the current point, and
Let
4.1. (i):
where
g(-
)
(3.2) of
y =
let
AT
S
_1
w
For any
.
the projective transformation given in (3.3) of Part
is
Part
Then P w
I.
equivalent to the problem
is
I,
(2.3):
m
minimize G(z) =
In
(1
- yT x +
y
Tz
V w
-
)
In
tj
t
i=l
(A - sy T )z +
subject to
= b - sy T x
t
>
t
Mz
= g
-yTz <
and by the remarks following
-F}
(x)
G( x
It
.
- y - y2 /
)
x
that
= -G(z)
is
(2.2)
and
thus suffices to
(2 (1
-
y))
(2.3),
show
- yT x
and lemma
that
y<
if
By construction of
.
1
y,
3.2(h) of Part
I,
F(x) =
then G(z) >
1,
from Theorem
4.1 of Part
I
we know
m
Z
the w-center of
.
Thus
for
any z €
Z
,
-
Y w
In
>
tj
4
i=l
m
-
V
Wj In
=
s
G( x
i
) ,
where
t
is
the slack corresponding to z in
Z
.
i=l
We now must show
that
-y T z
because
over
-y T x + y
.
x +
- yT x +
ln(l
d maximizes
ze E OUT
Thus
Proposition 2.5 of Part
,
and so
ln(l
I.
for
y Tz
-y T z
)
over
any ze
- yT x +
>
y
Tz)
-y- y2
/ (2 (1
z € E 1N
Z c E OUT
,
(i)
of
then
y))
.
x +
To
>
Lemma
-y- y2
4.1
.
see this, note
d/k maximizes
T
,-y z < y T x
> ln(l -y)
This proves statement
-
-y T d/k
/ (2
(1
-
y))
=
,
from
14
The proof statement
of
(ii)
Lemma
4.1
very involved, and follows from the
is
following sequence of lemmas:
Lemma
_
s
=b-
Let h >
4.4.
be a given parameter.
A
_
A
x,
and suppose
X
x e
X
,
let
satisfies
x- x) T A T S^W S^AC x-
(
x be the w-center of
Let
x
= p2
)
.
Then
m
V
Wj In
^
- A, x
(b,
h-ln(l + hL,2
±
m
V
-
)
Wj In
h-ln(l +
Proof: First
we
In (b,
- AjX
^
-
)
i=l
r
some k = R k
(n/k
/ p
)
We now
Casel.
T
,
W(rVk
-
Sj
(
)
,-
(2.
wTr
that
Id) of Part
I.
-w T
=
Also,
r
= k and by Proposition
/ p)
^
n
i
^
+
r
wnere
'
i^
i=l
Then note
from
w
X
=
,
S
_1
= n TM(x- x) =
A(x- x)
T Wr =
2
p
2.2 of Part
(p<hVk). Then |rj|<h.
Then
.
I,
~2
2
rf
,
i
=
1,
.
.
.
m
From Proposition
™
.
Thus
w
2^
i=l
T
w'r
p> hVk
|
r
1
4
< p
/
Vk
i
,
=
prove the two cases of the Lemma.
(h-ln(l+h))
Tj
Wj In
i=l
= -S" 1 A(x- x).
for
if
r-
h^kp
m
m
^
Wj
(3
observe that
m
V
h)
-2
i=l
i=l
<hVk
if
*
<
Sj
r-
i>
(h-lnO +h))
n
h
2
Tt
= r'Wr
r»i r
«
(h-lnO+h))
r?
h2
-
P^
.
,
^
i
n M
^ +
4.1(a), ln(l
r
i)
-
+ ^
)
<
1,
.
.
.
,
m.
1
Case
2.
(P
> hp/k
)
Because
.
1,
.
,
.
.
< P
|
r—
Vk h/p
^ w
m. Thus
i
<
)
then
y[k,
r
{
mm
r,
™
=
r
;
Proposition 4.1(a), ln(l +
i
|
h/p
<
and so again by
h,
(h - ln(l + h))
r-
^
Vk h/p
r,
rIn (1 + ryvk
Vk h/p
kh 2 /p 2
i
(h-ln(l+h»
<
)
- ,„ n
,
r
,
,
kh^
r^
,
because
i=l
w Tr
and r T Wr = p 2
=
X
(Vk h/p)
w.
+
In (1
r
t
)
However by
.
= (Vkh/P)
the concavity of the log function,
X
m
w.
£
In (1)
m
£
WjlnaVkh/pXl+rj)
w
X
(1-Vkh/P))=
+
+ rjVkh/P)
In (1
4
i=l
i=l
(h-ln(l +
h))
-5
r,
,
Wj
In (1
+t
)
^
Thus
-
,
kh 2
h
£
^
Vk h/p)
i=l
m
<
(1
i=l
i=l
<
+
In (1 + q)
-
——
(h-ln(l +
-
<
.
h))
_
r-
Vk hp
-2
{
.
i=l
Lemma
4.5.
Q = AT
S
_1
x be the current point in the w-center algorithm,
Let
W
S"
1
A
A
where
,
=
sy T
A-
is
A
in Step 4.
Suppose x
is
the optimal solution to
projective transformation given
If
Y<
1,
and h >
is
)
- F( x
)
by
(3.3) of
(
h"
^
+ h))
P
I,
and
defined as
is
A
z=g(x), where g(- is the
a
_
_ _ A
(z - x
Q(z - x) = P 2
)
that
)
.
2
+ pVky + P 2 ky2 /(2(1- Y))
p <
if
hVk
<<\
'
h2
Proof:
Part
P w and
and y
2,
a given parameter, then
'F(x
defined in Step
A
let
According
to
Lemma
— hVkp
3.2(h) of Part
A
F(x
I,
2
2
p ky /(2(l-Y))
pVky +
+
_
_
)
- F( x
)
= G( x
m
G(z) = ln(l-y T x +
y Tz) -
j w
i=l
-
In (b
;
i
A
z)
;
if
P >
hVk
A
)
- G(z
) ,
where
5
1
A
where
b = b - sy T x
defined in Step 2 of the algorithm, and
is
Noting
.
that
m
G( x
= -F(
)
= -
x)
Wj In Sj, then
j£
i=l
m
_
a
£
G( x)- G(z) = -ln(l+y T (z- x)) +
Wj In
(bj
-
m
_ a
z) t
£
A
i=l
However, because
Lemma
x
Z
the w-center of
is
m
~ A
-Ajz)
Wj ln(bj
X w
-
(defined in
(3.2) of
Part
thus remains
then by
ln
i
s
s
i
to
show
A
_
A
d=
(z
-
(h - ln(l + h))
j-
h
_
yUz- x))< p\ky +
The vector
).
maximizes -y T ( x +
g,
-y T (
y= -y T d/k
e
we must have
over
z)
E IN = {ze R n |Mz =
Z c E om ={z
^Pn
if
(3
>
r-
hVk
p2 ky2
T a
-y)
_
x
-y T ( x + d) >
MhVk
'
^
that -ln(l +
p-
if
P
2(1
because
I),
,
^
_
i=l
i=l
Let
.(4.1)
4.4,
m
It
s,
i=l
( --(h-ln(l +h))
£
In
v?.
P
2
<l/k
,
=
,
g, (z
)
< k)
> -pyVk
.
1.
)
I,
where
Thus
Also,
- x ) T Q(z - x
and so pVk <
.
T
T
y d > py
/p ),andso
T
y d
that vector that
is
as discussed in SectionV of Part
(z- x) T Q(z- x
then
|
Step 3 of the algorithm
in
z e E IN
all
x+ d Vk
Rn Mz
d
d/Vk
However,
.
because
< 1/k)
Thus
yp
from Theorem
Vk
<
y
<1
.
2.1 of Part
I,
This then
implies
-ln(l
ta
+ y d
p-
1
)
<
-ln(l
- Pyvk
j)
<
instance of Proposition 2.5 of Part
+py\k +
p..*>i
—p-^k
,
the last inequality being an
2(1 - y)
I,
with e = -yp
Vk and a =
y
.
6
17
Lemma
Y<
then
(3
Under
4.6.
<
1
the hypothesis of
ln(l + h)
-
2h
- 1/2
V
ln(l
Suppose p > hyfk
F(x)-F( x) < f(y,p)
f(Y,P) = -
+hV 12
,
hVkp
in y for
P>0
4.5,
+
pVkY
+
and 0<y<1-
-y)
Straightforward calculation
if
V(ln(l + h)/h) 2
F
Y=
+ 2pVk
(1
- ln(l + h)/h)
(4-3)
2-pVk
y
is
l
ess
A
optimality of x
.
man
m e above quantity, then
f( y
the expression in (4.3)
is
2 - ln(l + h)/h -
,
which equals
1
-
,
p) <
Thus y must be greater than or equal
Next, borrowing the observation in the proof of
Thus,if
hh
(4.2)
2(1
2-ln(l+h)/h-
,
•
where
h2
reveals that f(y, p) =
,
ln(l1 +
In
r
Then from Lemma
.
(h-ln(l +h))
Note f(Y,P) increases
if
4.5, if
hVk
Proof:
Thus
Lemma
—
lnd +
~r
rp>hVk,then
Lemma
,
contradicting the
to the
expression in
< pVk <
4.5 that
(4.3).
1,
then
greater than or equal to
Vdnd
+h)/h) 2 +
h)
1/2
V
ln(l
2(1 - ln(l + h)/h)
+h)1
~¥^]
/rin(l +
ln(l+h)
- 1/2
-^
yil—
"\[
h
,
h)"|2
+
J
f
I
1
"
lnd + KT
h~
1
Proof of Statement
RHS
of (4.2)
Lemma
4.5,
(ii)
< 5^jk
Let us set h =
:
Thus
= -.3781 p 2 +
and because k <
1
y<
if
Then
.5.
on the
the expression
.08567, then
from
Lemma
4.6
and
and
,
F(x)-F(x) <-.3781p 2 +
f(p)
4.1.
then greater than .08567.
is
(3
Lemma
of
pVky
+ P 2 ky2
Vkyp
and y <
2
P k*r / (2(1 -
+
-
/ (2(1
y))
ky2
.08567, then
y))
The function
.
-
/ (2(1
y))
—
concave. Thus the largest value of
f(p)
is
Let us define
.
given by
P
f(P)
< .3781
,
is
quadratic in
so that f(p)
p,
is
\ky
=
ky2
'
.7562-'
(l-y)
with
<
f(p)
f(
p)
ky2
=
2
ky
5-
—
2ky2
1.5124-
V.
Lemma
.669ky2
for
y<. 08567.
2-f
1.5124-
d-y)
(l-y)
This completes the proof of
<
4.1
Superlinear Convergence
In the previous section,
a linear
convergence rate of
Lemmas
4.4, 4.5,
and
4.6.
arbitrarily close to zero,
we showed
.32,
first
Proposition
present
convergence of the algorithm, with
by choosing the value
In this section
we show
h =
that as
.5
we
and applying
choose h >
and
then the linear convergence rate goes to zero in the
thus showing that the algorithm
We
linear
is
limit,
superlinearly convergent.
some elementary
facts
about three particular functions.
5.1.
Let f(h) =
1
-i^S
Let j(0) = Ll + 6 -
ln(l
- 1/2
Vl+26
J /6
2
+h)l 3"
for
£6 >
ln(l +
hV
for
h >
8
0.
(5.1)
(5.2)
19
,
,
Let p(h) =
lim
h->0
Then
We
f(h)
(h-ln(l +h))
=
lim
,
h>0
for
^2
=
j(0)
.5
,
(
lim p(h) =
h->0
and
0->O
.5
5 3)
-
.
then prove the following three propositions, after which the proof of superlinear
convergence easily follows.
Proposition
5.2.
For any h >
"1
f ( x nfw) - F( X
)
+ f(h) - Vl + 2f(W
Proposition
y < f(h)
,
then
(f(h))^
k\l + y-
in Proposition 4.1(c),
6
f(h) for
if
ky2
>
F(x NEW )-F( x) >
Proof:
Step 6 of the algorithm,
at
0,
5.3.
we
Suppose h >
as in Proposition 5.1.
V1
+ 2Y / from
and
sufficiently small
_
)
Now
substituting
obtain the desired result.
At Step 6 of the algorithm,
a
F(x
(2.4).
- F( x
)
if
and
y<
f(h)
f(h),
and p(h) are defined
then
ky2
<
/f/v
2k(f(h))-2
4 P (h)
,
"(l-f(h))
A
where x
Proof:
and
is
the optimal solution to
Because
Lemma
4.5
f(h) is just the
that
if
F(x)-F( x) <
If
h
is
y <
ky2 /(2(1 -
y))
<
(f(h))
2
expression of the
f(h),
-p(h)[3 2
sufficiently small p(h)
/(2(1
-
Pw
y))
of
(4.2),
we have by Lemma
4.6
then
+ (3VkY +
is
RHS
2
(3
kY2 /(2(l-7)).
approximately
<
(f(h))
2
/(2(1
.5
-
from Proposition
f(h))
is
(5.4)
5.1
and
approximately zero. Thus
the
RHS
of (5.4)
is
quadratic and concave in
p.
Its
maximal value occurs
at
Vk'
P=
kT2
2p(h) -
(1-Y)
and the maximum value of
the
RHS
in (5.4) is therefore
k^
2k?2
4p(h) -
(1-7)
However
y < f(h)
,
so
we have
F(x
)
- F( x
)
<
A
4
Proposition
and
For h >
5.4.
fM
kY2
2k (« h » 2
P (h) " (l-f(h))
sufficiently small,
if
x
the optimal solution to
is
P w then
,
i)
if
y ^ f(h)
ii)
if
y <
then F(x NEW ) - F( x
,
>
(l
f(h)
f(h),
1-
,
(ii)
then
f(h)
+
f(h)
-
>/l
+ 2f(h)
)
k
)
k
(2.4),
since
we know
1
+ 6 -
2k(f(h)) 2
- Vl + 2f(h)
4 P (h) "
(f(h))^
then from
+ f(h) - Vl + 2f(h)
Statement
<
y>
+
'
1
F(x) -F( x)
If
(l
then
f(h),
F(x) - F(x NEW )
Proof:
>
)
that F(x NEW )
^l +29
is
- F( x
)
(1
-
f(h))
> \1 +
y-V 1
+ 2y )k
an increasing function of 9 >
follows directly by combining Propositions 5.2 and
5.3.
We
have
that
0.
if
y
21
(
F(x)- F(x NEW )
F(x)
1-
=
F(x NEW ) - F( x
+ f(h)- Vl + 2f(h)
1
(f(h))'
)
1-
<
F(x) - F( x)
-F( x)
\
1
ky
4p(h) -
2k(f(h))
(l-f(h))
\
Cancelling out ky2 and rearranging yields the desired result.
Lemma
5.1.
Proof:
It
h -»
,
The algorithm
suffices to
show
that as
from Propostion
f(h) -»
Pw
for solving
h -»
0,
5.1.
Then note
+
i
with 6 =
9 ->
f(h),
and
h ->
as
exhibits superlinear convergence in F(x).
then the
e-Vi
5
RHS
As
of (5.5) goes to zero.
that
+ 29
=
j(9)
->
as
.5
9 -»
,
by
9
Proposition
p(h) ->
.5
and
approaches
VI.
The
5.1.
1
f(h)
-
last
—
term of
(5.5)
by Proposition
>
(.5) (4(.5)
-
0)
=
4p(h) - 2k(f(h)) 2 /(l -
is
5.1.
Thus the
f(h))
.
As h
-»
entire expression (5.5)
.
Analysis and Computation of the Improving Direction
In this section,
we show
positively scaled projected
that the direction
Newton
direction.
As
d of Step 3 of the algorithm
a
byproduct of
is
a
this result, the
computation of d can be carried out without solving equations involving the
matrix
Q = A T S^W
S"
1
A, which will typically be extremely dense. Vaidya's
algorithm for the center problem
direction
[
14
and performing an inexact
Vaidya's algorithm
when our
]
corresponds
line search.
algorithm
is
to
computing the Newton
Thus, our algorithm specializes to
implemented with
a line search.
?
Furthermore,
this establishes that
")
Vaidya's algorithm then will exhibit superlinear
convergence.
x be the current iterate of the algorithm,
Let
A
and
and
=
A-
Q = AT
S
sy T as in Steps
_1
W
S'^A
of
F(-
Then
)
at
From
the gradient of
x
is
and 2
(2.1a),
s
of the algorithm,
A
has
full
A
= b-
and
let
x,
and y =
Q = AT
rank, so that
Q
is
S"
T
F(-
)
given by -Q,
at
i.e.,
x
V2
is
given by -y
F( x
,
i.e.,
VF( x
)
W
AT
S"
T
S'Hv
A
,
nonsingular and
Let F(x) be the weighted logarithmic barrier function of
positive definite.
(1.1).
.
1
let
P w given
in
= -y, and the Hessian
= -Q.
)
Thus the projected Newton direction
dN
is
the optimal solution to
maximize -y T d - (1/2) d T Qd
Md
subject to
and the Newton direction
=
dN
together with Lagrange multipliers
Jt
N
is
the unique
solution to
Qd N
Mtk n
-
Md N
Because
Q
= -y
(6.1)
=
has rank n and
dN =
M
-Q _1 y
has rank
+
k,
we
can write the solution to
(6.1) as
Q_1 m t k n
(6.2)
It is
where
jc
our aim
to
n
=
(MQ^M^MQ^y
show
the following
23
Lemma
Then
1
Let d N be the
6.1.
+ yT d N >
by the solution
direction given
(6.1)
or
(6.2).
and
+ y Td N >
1
if
(i)
,
Newton
0,
and d N *
,
d = d N Vk
(^/d N Qd N )
/
the
is
direction of Step 3 of the algorithm.
If
(ii)
1
+ yT d N >
and d N =
0,
,
d = dN =
then
Step 3 of the algorithm, and the current iterate
(iii)
if
1
+ yT d N =
0,
unbounded, and P w
Remark
positive scale of the
Newton
is
Remark
dN
d.
.
Lemma
Thus
shows
6.1 (i)
d
that
is
is
just a positive scale of the
implemented with a
2.1.
Then because
(3.4)
of Part
will
be d N
the
Newton
Wj are
I
.
Newton
and 6
the projective transformations g(x)
preserve directions from
direction.
identical.
direction d N
line search replacing Steps 5
Therefore,
And
when
This
is
Q
Lemma
x,
and
/
y d N Qd N
6.1(i)
as suggested in
h(z)
given by
using a line search, the algorithm
through with or without a line search,
we
one need
.
shows
Remark
[
14
is
],
and
(3.3)
the algorithm direction in the space
precisely Vaidya's algorithm
a
is just
Suppose the algorithm
.
,
d,
X
just searching in
when
all
because the complexity analysis of Sections rv and
superlinear convergence.
is
Rather one need
.
and then compute d = d N Vk
dN
d
that
in order to solve for
Relation of Algorithm to Vaidya's algorithm.
6.2.
.
unbounded.
direction
(6.1) for
Pw
x solves
not solve a system involving the possibly-very-dense matrix
only solve the equations
the direction of
then the optimization problem (P3) of Step 3
Simplified Computation of
6.1.
is
V
weights
carries
see that Vaidya's algorithm exhibits
24
Remark
An
6.3.
Extension of a Theorem of Bayer and Lagarias.
shown
Lagarias have
X
sends the
set of
optimal solutions
infinity.
Then
the
transformed space
that
if
one
is
X
image of
the
is
,
to
First
under
,
performing a
Lemma
.
one can
program
to the linear
to the
line search in the
Newton
hyperplane
direction in the
6.1 is in fact a generalization of this result.
trying to find the center of any polyhedron
the algorithm of Section
II) is
one determines steplengths by a
a positive scale of the
X
(bounded or
Newton
It
states
not), then
method
Thus,
direction.
if
line search of the objective function, then the
method corresponds
projective tranformation
at
line search) in the space
the direction generated at any iteration of the projective transformation
(i.e.,
Bayer and
a projective transformation that
image of Karmarkar's algorithm (with a
Z
]
(unbounded) center of an unbounded
to finding the
Z where Z
Z corresponds
1
problem of minimizing Karmarkar's potential function
projectively transform the
polyhedron
[
the following structural equivalence between Karmarkar's
algorithm for linear programming and Newton's method:
over a polyhedron
In
to
Newton's method with
a line
search.
Another important relationship between directions generated by projective
transformation methods and Newton's method can be found in Gill et
We now
Proposition
Proof:
1
Note
+ y Td N =
We
i
6.1.
that
_
thus must
semi-definite,
y
T
Q -1 y(l
Lemma
prove
If
(dN
from
yTQ^y
show
and
- yT Q -1 y)-
rc
N
(6.2),
by
solve
)
a
[
4
].
sequence of three propositions.
(6.1),
then
1
+ y Td N >
0.
we have
+ y TQ- 1
that
that
,
6.1
al.
MT (MQ- M Tr
yTQ
1
_1
y<
1.
Q = Q - yy T
Therefore y T Q
.
-1
Note
Thus
y <
1,
1
MQ-
that
1
y >
1
Q = AT
< yT Q
_1
- yT Q_1 y
S
_1
QQ-1 y
W
S"
1
.
A
is
positive
= yTQ~ 1 (Q-yy T )Cr 1 y =
completing the proof.
25
Proposition
problem
Proof:
6.2.
(d N
If
(P3) of Step 3
From
Also
(6.3)
,
=
-Q^y
r
T Qr =
whereby
-
From
.
r
T
yTQ-V
Pw
Proposition
6.3. If
(6.4)
is
if
solution.
1
+ yT d N =
0,
then
(6.3)
y
=
T
1
Q_1 y
>
0.
(6.4)
^M^MQ^M 1 )'^: =
we have
(1
-y T Q_1 y)
Thus
0.
= °
satisfies
r
is
0,
from
Mr
=
0,
whereby Mr =
(6.3).
Finally,
T Qr =
r
0.
from
and -yTr >
0,
unbounded. From
unbounded.
(d N
,
7i
N
Q = Q - yyT
d N Qd N = d NTQd N (6.2)
we
)
solve
(6.1),
and
1
+ yT d N >
0,
then either d N =
0,
or
0.
T
y dN =
If
,
(y
Td
2
N)
(6.5)
,
obtain d N T Qd N = - y T d N
d N Qd N = - y Td N -
If
see that
then the optimization
0,
0.
Because
d N Qd N >
we
the optimization problem (P3) of Step 3
3.1,
and from
T
y dN =
1+
and
=
(Q-yy T )r=
Proposition
Proof:
(6.1)
unbounded, and P w has no
we have -y T r = y T Q _1 y
d N Qd N >
solve
)
y^^M^MQ^M^MQ-V =
and
r
is
N
the proof of Proposition 6.1,
1
Let
7t
,
d N Qd N =
(y
,
Td
2
N)
then
.
Since
.
Substituting in (6.5) yields
Q
we must have
is
positive semi-definite,
T
y d N = -1, or
-1, then this contradicts the hypothesis that
T
y dN =
0,
Proof of
Lemma
which implies from
6.1.:
l+y T d N >0and
(6.1)
that
From Proposition
1
T
y dN =
+ yTd N
>
0.
0.
Thus
d N =0.
6.1,
d N *0. Then d = d N Vk
we have
/
1
V d NQ d N
+ yT d N >
'
n =
:r
.
N
Suppose
/ (1
+ yTd N
) ,
and
26
P=
'\/d N
Qd N
/ (2>[k (1
+ yT d N
) )
problem
well-defined (by Proposition 6.3) and
all
M
dQ d = k,
satisfy the optimality conditions
for the optimization
are
,
(P3) of Step 3.
d =
Thus
-y = 2
0,
d
is
MT k
d -
(JQ
>
(5
,
0,
the direction of Step 3 of
the algorithm.
Next suppose
d =
0,
71
= kn
+ yT d N >
suppose
Finally,
the optimization
problem
3.2,
1
+ yT d N =
c
"X,
finite
c E OUT and
points near
c
,
is
at
x solves
-M T N
is
,
x
with the property that F IN
c
"X
problem
is
we
,
see
at
unbounded.
x
A
x
,
may
a
result of this section
X
Theorem
2.1
of Part
,
,
F ]N and
and FOUT =
answers
this
the
I.
there
may
involve irrational data.
F OUT
is
such that
A
will converge to x
II
(P3)
conclude that
of a polyhedral system
with center
algorithm of Section
A
The main
6.2,
an approximate w-center point
E IN and E OUT
,
.
unbounded and P w
A
x
= -y and
7C
Pw
whether one can construct good ellipsoids
= 0(1 / w).
affirmative:
Then
Then from Proposition
termination, and in fact the solution
A
x
0.
E IN =( w/(l - w)) E OUT
iterates of the
natural question
where
dN =
of the special features of the w-center
Although the
be
0.
(P3) of Step 3
fact that there exist ellipsoids
E IN
that
the current iterate
Inner and Outer Ellipsoids
One
and
satisfy the optimality conditions for the optimization
By Proposition
in Step 3.
VII.
1
A
FOUT
c
F IN
not
about
,
question in the
27
Theorem
(Inner and Outer Ellipsoids at an approximate w-center point.) Let
7.1.
be the current iterate of the algorithm,
Then
Step 4 of the algorithm.
F IN = {x €
Rn
Mx
|
=
if
let
s
A
x
and
,
y be as defined in
let
y < .08567, the ellipsoids
_1
(x- x ) T A T S
g,
= b -
x
W
S
_1
A(x-
x
)
wj
<
and
F OUT = Sxe
Rn
satisfy
F IN
Remark
7.1.
Mx
|
c
=
X
c
Note
1/m, and
Remark
7.2.
OUT
FOUT
to
F 1N
this ratio is less
The number
x
)
is
1.75-V/-^-
=
less
1.75m +
than
A
x
which
Let
5.
is
(F IN -
I
If
x
)
Thus the
.
w = (l/m)e,
O(m).
to
produce y < .08567
be the optimal solution
to
Pw
the initial value of x in the algorithm,
bounded.
5
needed
is
is
5,
of iterations of the algorithm
bounded
Pw
+
than 1.75/w +
is
if
5^ w
F,
that (FOUT -
ratio of the scale of
w=
]
g,
12
W
>f- w
-
_1
_1
(x- x) TaT
A' c-lw
S W c-l
S A(x- x) <
we must have
.
Then
if
x°
y < .08567 after at most
(1-w] / f(x)-F(x A
(w)
\
-0033
I
iterations
.
This follows from
Lemma
4.2(i).
If
F*
is
any
A
finite
upper bound on the value of
y < .08567 after
at
F( x
)
produced
at
Step 4 of the algorithm, then
most
(l- w)/F*-F(x
w
The proof of Theorem
c
.0033
7.1
propositions and lemmas.
is
iterations.
a
consequence of the following sequence of
28
Proposition
Mx
satisfy
Then
52
x
e int
which
X, and
w
imply
Mx
e
x
Wj
i=l
have
\J£ <
=
W
= b-
A
52
w
s
x - x) <
S^AC
xjWWS'U
x-
(
let
i
1,
so that
s
Ax
= b-
We first
.
By supposition above
.
i-si>
V.
5 <
Let
g.
intX
;
V
so that
,
and
given,
is
s=b-Ax,
if
m
< 62
X
w
< 52
s
(
,
w<w
Because
.
Sj
>
,
i
=
1,
.
.
.
we
,
i
>
1-5
—
1
i
,
so that
<
1-8
sj
i
Next, note that
m
(
AT S- WS- 1 A( x-
x)T
1
x)
=
£
i
~
,2
j
m
a
SJ-
Si
v
Wi
V
"i
J
- s)
,
i
=
that
S^W
1,
.
4
This shows that
m.
s
-=
obtain
s
x-
x - x) S
.
.
s
S"^
,
>
s
0,
- s)
m, we
J
s
Furthermore,
where 6<1.
must show
u
v<
(
,
Rn
Let x e
x.
.
By supposition,
will
x € int
and (x - x ) T A T S _1
=g
w/(l-5) 2
Proof:
Suppose
7.1.
rsjY
I
S:
~
1
S-
V
1
W;
=l
62
(1-6) 2
^
w
s
iy
.
v.
i
y
x
e int
X.
29
_
Lemma
suppose
x e int
Let
7.1.
Rn
F IN = {xe
xe R n
F OUT =
be given
,
and
X
be the w-center of
x
let
and
,
that
(x- x) T A T
Let
A
X
Mx
|
=
Mx
|
S-
=
!
W
g,
(x
(x- x) T A T
g,
S
_1
A(x-
x
)
- x) T A T S _1
S^W
S
_1
<
52
W
S
w
_1
A(x- x)
some 6<1.
for
A(x- x) < wj and
<
+
(1
5)
Then F IN c
X
Lemma
Proof of
c
7.1.:
in Proposition 7.1,
w-center of
X
,
F OUT
w
\
^
I
~- W +
A/
2
W^~
-8
1
^
.
x be an element of F IN
Let
we have
x e
X
X
c
F IN
so that
,
Then by
.
the
A
x
Because
.
same argument
is
as
the
then
- x ) T AT S' l VJ S- ! A(x - x ) < (1 - w)/w for any
A
A
where
from Theorem 2.1 of Part I.
s = b - Ax
A
_
Also, by Proposition 7.1, with x = x , we have
X
xe
(x
(7.1)
,
,
_
A
A
A
A
x-x) TAT S- WS- A( x-x)
< 6 2 w/(l-8) 2
1
1
(
Taking the square-roots of
(7.1)
and
(7.2)
<
-
and noting the
(x- x) TA T S- 1 WS- 1 A(x-
x)
<
N
w
1 =—
w
-
+
5*V
triangle inequality for
^
w
2
for
1-8
Next, note that from the hypothesis of the lemma,
every x e
A
s
-=
-
1
<8
,
and so
,
.
.
.
,
m.
Now let
x e
X
be given, and
let
s
-=
i
s
= b - Ax. Then
(7.3)
.
A
i
1
X
si
s
=
-
we have
norms,
i
7 2 >-
<
<
1
+ 8
30
(x
(l
- x ) TA T
+
6)
S-
]
rt.*VV
£ w
m
W
S^ACx - x
)
=
s
i=l
V °\)
x)
< (1+5) 2
A
A
T
1
1
T
A
(x- x)
S- WS- A(x-
2
:
(s,)
(
w
—
<- w +
-
8
-=
4w
1-8
completing the proof.
Lemma
7.2.
and
a
x
if
Proof: Let
x
If
is
the current iterate of the algorithm
the w-center of
is
A
A
z = g(x
where
)
P <
a
hVk =
a
_
d = z - x
.5^Jk
,
we
x± d
=
g,
2
Lemmas
4.5
and
4.6
that
if
x)
Step 3 of the algorithm and
in
d maximizes -yT z over
T
Q (z .
x) < k}
then -y T ( x + ((3/Vk) d) >
But y = -yT d/k, so ±yT d < pyVk
.
Therefore,
(yTd
2
)
p^k
<
(7.4)
•
Next, note that since Q = Q - yy T , where Q = A T S'^W S" ] A , we have
„A
a
A
_
a
a
_T
_
_
_
.
f a
z - x ) TQ ( z - x ) = (z - x ) T Q ( z - x ) + L (z - x ) T yJ2 = p 2 + (y T d ) 2 <
1
(
A
p
2
+ P 2 kr from
Part
A
(x
^
I,
(7.4)
Also,
.
_
h 2k + h2
(1
_
A
_
(x- x)= (z- x)/(l+y T (z- x)) from
(3.4)
so that
- x ) T Q (x - x
kV
- hky)
:
)
<
(
p
2
+ p 2 k72)/(l + y T d
k(h 2 + h 2 -/2 )
(1
- hy)
:
2
2
)
< (p 2 + p 2 k^)/(l - pVky) 2
w(h 2 + h 2
(l-hy)
:
-/
2
)
.5
y < .08567, then
_
Thus ±y T d < (-p/Vk)yT d
).
.55
4,
_
w.
.
x +
(z-
y < .08567 at Step
a
_
S"'A(x- x ) <
_
given in (3.3) of Part I. Let Q be as given in
A
— t ~ A
—
= ( z - x ) T Q ( z - x ) Then substituting h =
d be the direction defined
Then because
.
|
(
P
if
—
is
_
ze E 1N = (ze R n Mz
-yT
let
and
W
then (x- x ) T A T S _1
obtain from
Let
.
,
g(x)
Step 2 of the algorithm, and
in expression (4.2)
X
—
_
a
<
.55
w.
of
let
3
Proof of Theorem
s
= b-
Ax
.
i=l
^
1-
-==
-
We first show
:
suffices to
It
m
-=
\ Wj fti
7.1.
T
1
-
< w.
show
that
>
s
cX
F IN
that
x e F IN
Because
0.
Therefore, because
w
w
^
i
x e F IN and
Let
.
i
,
=
then
,
1,
let
.
.
,
.
m,
s
i
<
= l,...,m
l,i
,
so that
s,
>
0,
i
F 1N
cX
which shows
that
= l,...,m.
Thus
S:
i
To show
X
that
c F OUT we
,
A
<
x of
w-center
.08567,
then
the
Y
obtain the conclusion that
Lemma
VIII.
A(x - x
1
k/^-=^-+
X c FOUT
satisfy
must
we
in
that
X
S'
- x ) T A T S _1
(1+6) 2
Lemma
W
(x
7.1,
apply
)
< 52
w
,
8 = V^55
where
n1
/
-
^f^~
<
7.2,
-
if
Next, applying 6 = V^55
.
w
;
—
f
1.75A/^-=^ +5"V
,
thus showing
.
Concluding Remarks
Alternative Convergence Constants.
the algorithm that
convergence
to the
we
Lemma 4.3
asserts that at each iterate of
obtain a constant improvement of at least .0033k or a linear
optimal objective value, with convergence constant
.32.
constants .0033 and .32 are derived in Section IV by using the value
h=
Lemmas
h =
4.4, 4.5,
and
4.6.
If
instead of choosing
h =
.5,
one chooses
example, then by parallelling the methodology in Section
with a constant improvement of
convergence constant
.64.
at least
The choice
4,
one obtains
.5
The
in
2, for
Lemma
4.3
.0133k or a linear convergence rate with
of h =
.5
was
fairly arbitrary.
Similar results
can be had by choosing a different value of h to obtain different constants for the
threshold value of y and the relative sizes of FOUT and F IN in
Theorem
7.1.
1
32
Stronger Convergence. Vaidya
algorithm
is
Here,
linearly convergent.
that the algorithm of Section
A
convergence.
II
[
14
has
]
we have
shown
that his
Newton
direction
extended his result and have shown
(and Vaidya's algorithm) exhibit superlinear
natural question for future study
whether one can show the
is
algorithm here to be quadratically convergent.
The behavior
computed. In Step
write y =
and
a
yi x)
of
4,
the parameter y
The value
.
of y( x)
is
_
y( x)
goes
to
Concerning the behavior of
convex,
y(x)
if
Thus
No
y( x)
,
it is
y(x) is
is
A
x
,
natural to ask
decreases at each iteration,
a level set of y(x)
iteration.
x approaches
zero as
computed, and we can
Lemma
4.3,
it is
the optimal solution to
if
the level sets of
yix)
Pw
.
are
The author has demonstrated examples
etc.
not convex, and where
y{x)
increases at a particular
not as well-behaved as one would hope
Finite Termination.
is
then used to derive upper bounds, a step length,
is
_
where
x
as a function of
,
guaranteed improvement in the objective value. From
obvious that
d
In Step 3 of the algorithm, the search direction
y.
As pointed out
for.
in Section VII, the solution
A
x
to the
w-center problem can have irrational components and so the algorithm will not
stop after finitely
algorithm
many
[
1
]
.
may
many
iterations.
Even
This
is
shown by
example
the
In that example, the iterates of a
for the w-center
problem, where
X € {x €
2 -1,
|
the
problem Pw
may
never detect unboundedness, and so
iterations.
R2
if
Xj
x1 <
1,
(Newton) direction defined by each
algorithm will never stop.
,
iterate is
,
the
not stop after finitely
Newton method with
and the
unbounded,
of Section 4 of Bayer
w = (1/3, 1/3, 1/3)
x 2 > oj
is
and Lagarias
a line search are traced
and
x =
starting point is
never a ray of
X
,
(1
/3, 2/3).
and so the
The
References
[
1
Bayer, D. A., and
]
J.C. Lagarias. 1987.
algorithm and Newton's algorithm,
Karmarkar's linear programming
AT&T
Bell Laboratories,
Murray
Hill,
New Jersey.
[2] Censor,
Y.
,
and A.
Lent.
linear equality constraints,
Optimization of 'log x' entropy over
Journal of Control and Optimization 25,
1987.
SIAM
921-933.
Freund, R. 1988. Projective transformations for interior point methods,
[ 3 ]
part I: basic theory and linear programming, M.I.T. Operations Research
Center working paper
OR
179-88.
On
4
P. Gill, W. Murray, M. Saunders, J. Tomlin, and M. Wright. 1986.
projected Newton barrier methods for linear programming and an
equivalence to Karmarkar's projective method, Mathematical Programming
36 183-209.
[
]
Gonzaga, C.C. 1987. An algorithm for solving linear programming
problems in 0(n 3 L) operations. Memorandum UCB/ERL M87/10.
[
5
]
Electronics Research Laboratory, University of California, Berkeley,
California.
[
6
N. Karmarkar. 1984.
]
A new
programming, Combinatorica
[
7
Lagarias, J.C. 1987.
]
4,
polynomial time algorithm for linear
373-395.
The nonlinear geometry of linear programming ITJ.
and Hilbert Geometry. AT&T Bell
Projective Legendre transform coordinates
Laboratories,
Murray
Hill, N.J.
Monteiro, R.C., and I. Adler. 1987. An 0(n 3 L) primal-dual interior point
algorithm for linear programming, Dept. of Industrial Engineering and
Operations Research, University of California, Berkeley.
[
8
]
Monteiro, R.C., and I. Adler. 1987. An 0(n 3 L) interior point algorithm
for convex quadratic programming, Dept. of Industrial Engineering and
Operations Research, University of California, Berkeley.
[
9
[
10
]
]
Renegar,
method,
J.
for linear
A polynomial time algorithm, based on Newton's
programming. Mathematical Programming 40 59-94.
1988.
,
G. Sonnevend. 1985. An 'analytical centre' for polyhedrons and new
classes of global algorithms for linear (smooth, convex) programming,
preprint, Dept. of Numerical Analysis, Institute of Mathematics Eotvos
University, 1088, Budapest, Muzeum Korut, 6-8.
[
11
]
[
12
]
G. Sonnevend. 1985.
A new
method
for solving a set of linear (convex)
and its applications for identification and optimization,
of Numerical Analysis, Institute of Mathematics Eotvos
Dept.
preprint,
University, 1088, Budapest, Muzeum Korut, 6-8.
inequalitites
[ 13 ]
M.J. Todd, and B. Burrell. 1986. An extension of Karmarkar's
algorithm for linear programming using dual variables, Algorithmica
1
409-424.
[
14
]
Vaidya,
P.
1987.
Newton-type method
Laboratories, Murray
[
15
]
Vaidya,
P.
A
locally
well-behaved potential function and
for finding the center of a polytope,
Hill, N.J.
1988.
private communication.
AT&T
Bell
a
simple
MAR
200?
Date Due
Lib-26-67
MIT LIBRARIES
3 9080 02237 3671