A METHOD FOR THE PARAMETRIC CENTER
PROBLEM, WITH A STRICTLY MONOTONE
POLYNOMINAL-TIME ALGORITHM FOR
LINEAR PROGRAMMING
by Robert M. Freund
and
Kok-Choon Tan
WP# 2600
was 2100-89
l
March, 1989
Abstract
Given a system of linear inequalities and equalities Ax b + dt and
Mx = g + ht where the right-hand-sides (RHS) are parametrically deformed
over the scalar t , the parametric center problem is to trace the parametric
family of approximate solutions
(t) to the center problems P(t) , where
P(t)
is the problem:
maximize
, ln (bi + dit - Aix) subject to
Ax <c b + dt
i=l
and Mx = g + ht.
We present an algorithm for tracing the parametric family
of solutions
(t) over the given range t E [ t . At each iterate of the
algorithm, the value of the parameter t is strictly increased and a Newton step
is taken. The sequence of values of t exhibit the following geometric rate of
change: If tk and t k+ ' are two successive values of the parameter t
generated by the algorithm, then either (tk+l - TMIN) (tk - TMIN)(1 + 128m)
or (TMAX
-
tk+1) < (TMAX - t)(1-12 J
where
TMIN
(TMAX) is a lower
(upper) bound on the smallest (largest) value of t for which Ax < b + dt,
Mx = g + ht has a solution. Thus the iterates exhibit either linear growth
away from TMIN or linear convergence toward TMAX , with a rate of change
l
~~1
t
of
128m , where
m
is the number of inequality constraints.
When applied to the linear programming problem, the algorithm is an
O(mL) iteration algorithm for linear programming, that strictly improves the
primal objective value at each iteration, and requires no dual feasible solution (or
even dual feasibility) to start. After O(mL) iterations, the algorithm either
detects primal unboundedness or produces an interior solution that can be
rounded to an optimal solution to the linear program.
Key Words:
Newton step, center, linear program, interior-point algorithm.
Introduction
Given a system of
Ax
b
and k
m
linear inequalities in
equations in
center of the system
Rn
of the form
(A, b, M g)
Rn
of the form
Mx = g,
the (analytic)
is the optimal solution to the convex
program:
P:
maximize
l, n (bi- Aix)
i=l
s.t.
Ax< b
Mx = g
where
(Ai, bi)
respectively.
( x e Rn Ax
system
denotes
ith
row of
A
(See Sonnevend [12, 13] .
b, Mx = g )
(A, b, M g) ,
and
ith
component of
b,
Assuming
is nonemepty and bounded, the center of the
denoted
,
is uniquely defined.
The
computation of points near the center and their properties are important for
interior point algorithms for linear programming and extensions, see
Karmarkar [6] ,Renegar [11], Megiddo [8], Kojima et. al. [7], Vaidya [16],
Monteiro and Adler [10], Mehrotra and Sun [9], Jarre [5], Barnes et. al. [1],
and Todd and Ye [14], among others.
Algorithms for finding the center are
presented in Censor and Lent [2] , Vaidya [15] , and [4]
This study is concerned with the parametric analysis of the family of
centers as the right-hand-side
varies parametrically.
(RHS)
(b, g)
of the system
(A, b, M, g)
We define the parametric center problem
(PCP)
be the problem of tracing the parametric family of optimal solutions
the problems:
1
xt
to
to
III
P(t):
maximize
, In (bi + dit - Ai x)
i=l
s.t.
Ax < b
+
dt
Mx= g + ht
(where
d e Rm
interval
and
te [t, t],
h e Rk
and
are given), as
1t,t
t
varies over a given
are finite or infinite.
algorithm for generating a piecewise-linear function
the property that
x (t)
We present an
x (t)
is an approximation to the center
is an approximate solution to
P(t), as
the approximate path of solutions.
t
is varied.
[ It
-4
x (t),
Rn
with
i.e., x (t)
We refer to
(t) as
(The sense of the approximation and its
properties are defined in Section 3.)
The algorithm starts with an approximate solution
center problem
t = tk
t
P(t)
at
t =t .
At iteration
is chosen as
t = tk+1
( tk+1)
to solve for
.
where
The path
by linear interpolation.
guaranteed increase in
chosen so that either
(TMAX - t
l
)
<
l
tk
> tk
R( t)
t
(tk) .
t
is
The next value of
and a Newton step is performed
is then extended for
t e [tk, tk+ ]
The important feature of the algorithm is the
t
at each iteration.
In particular,
tk+1 can be
(tk+1 - TMIN) > (tk - TMIN)(1 + 12 )
(TMAX - tk)(
1
)
,
where
(upper) bound on the smallest (largest) value of
Ax < b + dt ,
to the
k , the value of
P (tk) is
and the approximate center for
( t)
Mx = g + ht , has a solution.
TMIN (TMAX)
t
is a lower
for which
Thus the iterate values of
demonstrate either geometric growth away from
2
or
TNIIN
or geometric
contraction toward
TMAX, with a rate of change of
1E128m
(where
m
is
the number of inequality constraints).
This algorithm can be applied to solve the linear programming
Suppose we wish to solve the linear problem
problem in a new way.
LP:
maximize
cT x
s.t.
Ax<b
Then we can use the algorithm for
PCP
to solve for the path of centers of
the system
LP(t).
maximize
E
i=l
s.t.
In (bi - Aix)
Ax < b
cx = t
as
t
is increased.
This yields a new "central-trajectory-following"
algorithm for linear programming that differs from other central-trajectory
First, it is a strictly monotone algorithm for linear
methods in two ways.
programming, i.e., an algorithm that strictly increases the objective function
value of the primal at each iteration, unlike other central-trajectory-following
algorithms.
Second, it requires no prior information or bound on the
optimal objective value, and will process a linear program that is unbounded
in the primal objective value (i.e., dual infeasible), unlike other centraltrajectory methods.
However, the complexity of the algorithm is
iterations, as opposed to
(where
O( iM L)
O(mL)
for most other central-trajectory methods
L is the bit-size of the problem instance) and so has an inferior
3
III
complexity bound (by
ff ) .
Perhaps it is the strict monotonicity of the
primal objective value in the algorithm that is responsible for the inferior
complexity bound.
This paper is organized as follows.
In Section 2, we present the main
results regarding the parametric center problem and we present the algorithm
for tracing the approximate parametric path of centers. The remaining three
sections are devoted to proofs of the results of Section 2.
notation and preliminary results.
Section 3 presents
Section 4 contains an analysis of one step
of the algorithm and presents the results on the use of Newton's method.
Section 5 contains the results regarding bounds on feasible values of
are generated by the algorithm.
1i
4
t
that
2.
The Parametric Center Problem
Given a system of
Ax
b
m
and equations
referred to as
linear inequalities in
Rn
of the form
Mx = g , the (analytic) center of the system
(A, b, M, g)
is the optimal solution to the program
m
P:
maximize
ln (bi- Aix)
i=l
s.t.
Ax< b
Mx = g
(see Sonnevend [12, 13] .)
Suppose
is the unique solution to P .
center
as the right-hand-side
varies parametrically.
(RHS)
of the system
(A, b, M, g)
In particular, we are interested in generating the
parametric family of optimal solutions
P'(t):
Our interest lies in tracing the
maximize
xt to the problems
A In (bi + dit - Aix)
i=l
s.t
Ax < b + dt,
(2.1)
Mx + g + ht.
In this section, we present an algorithm for generating a piecewise-linear path
of solutions
and as
t
(t)
such that
(t)
is close to
xt
in a suitable measure,
is varied strictly monotonically over a prespecified range.
First note that there is no loss of generality in assuming that the
equations
Mx = g + ht
are not present.
without loss of generality that
kxk
and is nonsingular.
M = [B, N]
To see this, we can assume
is a
By suitably partitioning
5
kxn
matrix where
A = [C, D]
and
B is
Ill
x = (y,z), we can eliminate the
y
variables to obtain the equivalent
problem
m
P"(:
maximize
+
n (i
dit - Ai z )
i=l
s.t.
where
Az < b
A= D - CB-N,
b = b - CBg ,
straightforward to show that
solves
solves
Zt
and
P"(t)
if and only if
Yt = B 1 (g + ht - Nt)
P'(t) , where
d = d-CB-lh.
.
Itis
t = yt,
Z
We thus can concentrate
on the more convenient problem
P(t):
ln (bi
maximize
+ dit - Aix)
(2.2)
i=l
s.t.
Let X = (x
Suppose
st
=
xt
Ax < b
Rn Ax
b)
is the centerof
b + dt - Axt, andlet
Because
P (t)
+
dt.
and
Xt = (x e Rn Ax < b +dt
Xt, i.e.,
t
solves
P(t).
e
is a convex program,
xt
solves
P (t) if and only if
xt , namely
St = b + dt - At > O0
(2.3a)
eTS t 1 A = 0
(2.3b)
is the vector of ones of appropriate dimension.
We assume that our initial value of
of Xto = {x
Let
St = diag(st).
the Karush-Kuhn-Tucker (K-K-T) conditions are satisfied at
where
.
R I Ax < b}
t
is
t = 0 ,
is inonemnlpty and bounded.
6
and the interior
In this case, it is
straightforward to show that
Xt
will be bounded for all values of t .
T = (t I int Xt *
T
will be an open interval, and it is
) .
Then
Let
straightforward to extend the analysis in Megiddo [8] to show that the path of
parametric centers
r: t -
t , t G T,
Let TMAX = sup (te T) and
is continuous and differentiable.
TMIN = inf (te T . Itis also
t
t
straightforward to show that
if d = Ar
for some
of translations of
Xt'
if and only
r e R n , in which case the sets
Xt
by the translation vector
We therefore assume
throughout this paper that
this case, either
and TMIN = -°
TMAX = +
d
TMIN > -~
tr .
are just a family
does not lie in the column range of
, or
TMAX < +
,
A .
In
or both.
The algorithm presented in this section will trace a piecewise-linear
path
(t)
measure.
that is "close" to the parametric center path
At each iteration the value of
t
least one of two ways, as follows.
in a suitable
is strictly increased; thus the
algorithm is strictly monotone in the parameter
iteration, the magnitude of the increase in
xt
t
Suppose
t .
Furthermore, at each
is bounded (from below) in at
tk
is the current value of
t
Then at the next iteration the algorithm will produce either a finite lower
bound
UB
TMIN
is produced, then
tk+1 - tk
i.e.,
LB
1
128m
or a finite upper bound
tk+' ,
(UB - tk),
(TMAX - t+1) < (1-
the new value of
28m
geometrically, i.e., (tk+ -TMIN)
)(tk
128m
7
If
TMAX,
or both.
If
will satisfy
decreases geometrically,
LB is produced, then t
so that (t -
(tk- LB) ,
(1 +
t,
(TMAX - t)
so that
12m)(TMAX - tk).
will satisfy (tk+ l - tk) > 1
UB >
TMIN)
TMIN)
grows
L
II
At least one of the above two bounds must be satisfied. Note that in either of
the two cases the geometric rate of change of t relative to a bound is at least
1
where m is the number of inequality constraints.
128m '
Given a linear program in the form:
maximize
LP:
x
x
Ax < b
s.t.
we can reformulate
LP
as the following equivalent problem
maximize
x, t
LP:
s.t.
t
Ax < b + Ot
cTx
= O+ t
Thus an algorithm that traces the parametric path of centers to the above
system for strictly increasing values of
algorithm for solving
LP .
t
will be a strictly monotone
The application of the parametric center
problem algorithm to linear programming is presented at the end of this
section, and is an
O(mL)
iteration algorithm for linear programming.
2.1 Properties of the Parametric Center
For
t e (TMIN, TMAX),
St = b + dt - Axt,
space of
that
St A ,
and let
i.e.,
d = St ut + Art
indicator function
f(t)
let
ut
t
be the center of
be t he projection of
Xt , and let
St d
ut = [I - St A (AT St2 A) AT St ] St d .
for some
as
n
rt E R .
f(t) = eT ut ,
8
onto the range
Then note
We now define the path
and note immediately that
'S"1
_S t'
f(t) = eTut = eTSt (d-Art) = eTSt d ,
-1
by (2.3).
since eTSt A = O,
f (t) = e St d .
Therefore an alternative equivalent definition of f (t) is
is the optimal objective value of the
TMAX
follows. It is obvious that
is as
f(t)
The motivation for considering the path indicator function
following linear programming problem:
TMAX = maximize
LP°:
t
x, t
Ax- dt < b
s.t.
Suppose for a given value of
eT St (Ax -b)]
1
i <
Then for
and St = diag(st) .
(x, i) to LP ,
any feasible solution
tS = d
xt be the
Let
f(t) < 0 .
that
^st = b + dt - Axt,
P (t) , let
solution to
t
1 [eT;t
=
+dt)
At-st
(Ax -
= 1 [-m + tf(t)]
f (t)
whereby
t- m
TMAX
f(t)
f(t) > 0
Similarly, if
Now suppose that for a given value of
Let
be the center of the system
xt'
St= diag(b + dt-Axt*) .
(t', t*)
whereby
particular
if both
that
TMIN
TMIN
X
t , say
eTSt A = 0
is the center of the system
and
and
TMAX
TMAX
t ,that
f(t)
f(t*)=O
Ax < b + dt*, and let
and
are finite.
are finite.
If
is bounded from below (above) in
9
eTSt d = 0,
(A, -d)( x, t) < b , from (2.3).
= ((x,t)e Rn+ l I Ax - dt < b)
X
Thus the set
Then
TMIN > t-
isIbounded, and in
We say
X°
T1IN
(T MAX)
t .
is bounded in
t
is finite, we say
Nc)te that if
X°
is
III
bounded in
t,
then
f(t*) = e T S d = 0,
t- ,
the value of
to be on the upper path if f (t) < 0 ,
0.
which yields
is guaranteed to exist.
Returning to the path indicator
f (t)
t
f(t)
defined earlier, we define
xt
and to be on the lower path if
The intuition behind this definition is provided in the next two
propositions.
Proposition 2.1 The path indicator function
decreasing for
t
(TMIN, TMAX)
Proof: The K-K-T conditions of
P(t)
eTSt A = 0
and
Let
t and
respectively.
and
A
unless
require that
At + st
xt be the vector of derivatives of
=
b + dt.
'xt at t ,
differentiating
the
above
expressions
yields
Then differentiating the above expressions yields
t Then
St-2
A O
t
and
t + t = d.
Furthermore, since
then
f (t) = eTut is strictly
f (t) = eT ut = eT ;St d
f'(t) = -t St d = -St
(A it + t) = -t §2St < 0,
st = 0 , in which case
out by the assumption that
d
d = A xt .
But this last possibility is ruled
does not lie in the column range of
10
A .
w
Proposition 2.2 (Upper and Lower Paths)
(i)
and
TMAX
te R
(ii)
f(t ) =
such that
and
,
f (t) > 0
for all
t E (TMIN, t ),
f(t) < 0
for all
t
is finite and
TMAX
all
(iii)
are both finite if and only if there exists
TMIN
(t, TMAX) ;
TMIN =
f(t) < 0
if and only if
--
for
t < TMAX ;
is finite and
TMIN
all
if and only if f(t) > 0 for
TMAX =
t > TMIN.
These three cases are illustrated in Figure 1, (a), (b) and (c).
center of
= ((x, t) IAx- dt < b .
X
and
and so TMAX
TMIN
f(t') =
(i) We have seen earlier that if
Proof:
,
That being the case,
X
are both finite. Conversely, if
TMIN
are both finite, then the center
of X
(x*, t)
,
is the
(t, t)
then
is bounded
TMAX
and
which is the
solution to the problem
maximize
x,t
s.t.
exists uniquely.
and
St
=
.
Ax - dt<b
The K-K-T conditions for
eT St d = 0 ,
diag (st')
, In (bi + dit - Aix)
i=1
i.e.,
f(t )= O,
P° require that
where
st = b + dt* - Ax-
The rest follows from Proposition 2.1.
11
eT St A= 0
and
II
t
T.....
'MAA
rI ,
-
r rt)<
I
f(t) = 0
t
Fc(4a\
AI
I
r
-
MIN
x
Figure 1 (a)
12
_
-
-
V
t
Figure 1 (b)
13
11
,
.
.
i
1
I
t
i
f(t)< 0
i
I
lMIN
Elpx
Figure 1 (c)
14
(ii)
Suppose
is finite and
TMAX
We have seen earlier that
f (t) > 0
implies
f(t) < 0 .
f() = 0
then
But, from (i),if
contradiction.
then
f (I) < 0 .
Thus,
is finite.
TMAX
.
TMIN
is finite.
TMIN
Conversely, if
Consequently, if
Let t < TMAX.
TMIN = -
Thus,
is finite, which is a
f(i) < 0
for some
f(t) < 0 for all
t < TMAX
t,
then
follows from (i).
TMIN = -
The proof of (iii) is similar to (ii).
2.2 Algorithm
PCP
Before presenting the algorithm for the parametric center problem, we
introduce some more notation and the main improvement theorem.
X = {x e Rr- ,Ax < b}
Recall that
and
Xt
(X e R n IAx
=
We assume that
X
is bounded for all
b+dt)}.
is bounded, and so A
t . Let
has full column rank and
Xt
IIv ll denote the Euclidean norm. If M is a
symmetric positive-definite matrix,
wedenote
let
where
I Y-ZIIM =
e intX
(y-z)TM(y - z) . Let
be given, let
= b-Ax,
S = diag() . We say that
X-X Q() < 1/21 .
be thecenterof
X,
and let Q(x) = ATS2 A,
is close to the center of
X
if
The motivation for this criterion of closeness to the
center will be apparent from the following theorem, which also serves as a
basis for the
PCP
(Parametric Center Problem) algorithm.
15
III
Theorem 2.1 (Improvement Theorem).
Let
= b-A3Z,
xxQ
S = diag () ,
, where
<1
Suppose
and
e int X = (x IAx <b)
Q = ATS-2 A .
is the center of the system
l
Suppose
(A,b) .
Define
(projection of S-1 d)
=
[I - §S- A A S- A
r =
(AT2'Ay AT 2d
(translation)
a=
1
801ll11
(step length)
Furthermore, define
AT S-1] -
d
a -
Sa = diag (Sa)
-=
(Newton step)
(AT §a2 A)l AT§ e
(New approximate center)
XNEW = x + q + ar
Then II NEW- Xia
< 1
21
., where
(A, b + ad), and Q = ATSN2EW
In this theorem,
X.
The vectors
so that
x
A where SNEW = diag(b + rd - A NEW).
.
r
are defined and satisfy
The increase in
defined next, and is a function of
reR n, lltll > 0 ,
is the center of the system
is given and is assumed to be close to the center
u and
d = Ai+ S
xa
and so
a
IIu
.
t
from
Because
S§'d = 'IA + I,
t = 0 to
d
Ar
t = a
is
for any
is well-defined. The Newton step
16
of
-
is then
,
defined using the modified slacks
x,
will be suitably close to
Nm
According to the
(a translation vector).
(the Newton step) and
theorem,
is composed, using
zNEW
and
the center of
a,
Xa
This
theorem will be proved in Section 4.
The increase in
is a function of II uiI
t
.
is given as
t = 0
from
However, the fact that
makes good intuitive sense.
1 /si
in the weighted norm with weights
A,
liesveryclosetotherangespaceof
is small. Thenchanging the RHS by
by a r and then changing the
The quantity
1l u
i = 1,...,m . Suppose
d = Si+ Ar,
i.e.,
ad
a is
from the column range of
d
is the minimum (least-square) distance of
A,
which
l
Ideally, we would like this increase to be as large as
possible, to speed algorithmic convergence.
proportional to 1/llU
a =
d
andso IuI
is the same as translating
RHS by only
aS
X
. Then because
Ilul l is small, we can take a big step. If, however, I iUl is large, then
"most of
lies outside of the range space of
d "
RHS of a d
is shifting the shape of
A , so that a change in the
X substantially, not just translating
the polyhedron. Thus, the step we can take will be smaller.
However, as the next theorem indicates, the value of
important bounds on the values of
show that if
a is small, so is
Recall that
f (t),
TMAX
TMAX
or
and TMIN ,
II UiII also gives
and thus will
TMIN
the path indicator function, gives important
information about the boundedness/unboundedness of the path of centers
xt
in both directions, according to Proposition 2.2.
17
111
Theorem 22 (Bounds on
TMhAX
and
TMTN ).
Under the conditions and
definitions of Theorem 2.1,
(i) if
e/lUIl
> 120 ,
then
TMIN
(ji) if
el/IiUll
< -1/20
then
TMAX < UB
then
TMIN > LB -
and
TMAX <
(iii) if letu/ 11 111 < 1/20,
22¥m(m+l) + 1
LB
211 lii ll
22Ym(m+1) + 1
2111 illI
1.6m(m-l) + .6
lull
B -
1.6¥m(m-1) + .6
.
Iill
This theorem will be proved in Section 5.
Note that either (i), (ii), or (iii) must be satisfied, so that either a finite lower
bound on
TMIN
or a finite upper bound on
course of increasing
from
t
t = 0
to
TMAX
t = a,
is produced in the
or both are produced.
Case (i) corresponds to being (approximately) on the lower path at
i.e.,
f(0)> 0 .
path, i.e,.
Case (ii) corresponds to being (approximately) on the upper
f(0) < 0
close to the center of
Case (iii) corresponds to
X
is increased from
f(O)= 0,
is
so that (, 0)
.
Note also that these bounds are
t
t=0,
t = 0
one or both bounds is at least
to
O(m/Iluii)
t = cc = 1/8011 Ill,
1
128m
Thus, even though
the ratio of
a
Therefore repeated increases in
to
t
using the methodology of Theorem 2.1 will result in either geometric growth
18
t-TmTIN
in the quantity
with a growth rate of at least
geometric contraction in the quantity
k)I
(1-
atleast
Prrnacif4an
m
Suppose
tNEW =
'
where
a,
+
I
or
with a contraction rate of
TMAX-t ,
as the next proposition indicates.
(-c.mobrir
Chana
~~~U
V~VII·
~·
· ~~ ~-~· ·
2 .
(1
in
.
With the notation
a
and
~-LRPlaZIh+O
T",,t
nitly
if
XOLD - X ,
v} I
T..
- nnfikA
nr
--
tOLD = 0,
are defined as in Theorem 2.1, then
XNEW
either
(i)
(tNEW-
TMIN)
(1 +
(ii)
(TMAX-tNEW) < 1 -
1
m)( tOLD - TMIN)
or
12
Im) (TMAX - tOLD)
Proof:
Suppose a lower bound, either
or LB , is generated through
Theorem 2.2. Notice that LB < LB if m 2
so in either case,
TmNL
- (1-6 Wm(m- i) + 6) /1912.
(tNEW - TMIN) =
tOLD - TMIN
tNEW
-6
+
-TMIN
I
I
-_ 1_6
Thus
+1 = 1+
_
1
128m ·
A parallel analysis demonstrates (ii) if an upper bound is generated.
m
Now let us return to our initial interest - to trace the parametric center
path
xt
for the program
ranges in an interval
P (t) . Suppose we want to trace the path as
t e It, t] ,
where
19
t > t .
t
Suppose we are given a
III
point
(Such a point can be found by
X.
that lies close to the center of
x
using the algorithm in Vaidya [15] or in [4] .)
The following algorithm, denoted
for Parametric Center
PCP
Problem, is an iterative algorithm that invokes Theorems 2.1 and 2.2. At
The
Step 0, initial lower and upper bounds are set to their extreme values.
initial value of
and the counter
is chosen as' t = ,
t
x is set, as is the
RHS .
F are computed as defined in Theorem 2.1.
which is the increase in
close to the center of
p(tk + a),
(t)
TMIN
-NEw ,
, i.e.,
xNEw
t
Algorithm
In Step 2, the constant
is then computed;
U and
a ,
TMAX
xNEW
is
(tk) = x
, using
In Step-4, the current
are updated, in accordance with Theorem
tk+ l 2 t.
If so, it stops.
If not, it
The output of the algorithm is the piecewise-linear path
t,
namely
t ,
t,
...,
tk , ...
PCP (Parametric Center Problem)
Input:
AE Rmxn,
Step 0
(Initialization)
UB=+oo,
tk .
approximately solves
[tk,tk + a] = [tk, t]
and the incremental values of
x (t)
In Step 1, the values
as endpoints and interpolating.
and
is
In Step 3, a piecewise-linear path
In Step 5, the algorithm checks if
returns to Step 1.
Set
point,
according to Theorem 2.1.
(tk ' l ) = XNEW
bounds on
2.2.
X
is defined in the range
and
t
at the iteration, is defined as in Theorem 2.1.
t
"dose-to-center"
The next
(for the
The current value of
number of iterations) is set equal to zero.
The current value
k
b, d
LB=-Co,
Rm ,
k=O,
t , x0
t ° =t,
20
-=x
,
RHS=b+dt
Step 1
Set
(Projection of d)
= RHS-A
Compute
, S = diag(s).
u = [I - S-1 A(A T -2 A A AT 1]S-1 d
r = (AT - 2 Ar 1 ATS 2 d.
Step 2
(Compute Step Length and Compute New Approximate Center)
a
Set
80 IJUJIl
Sa = RHS+axSi-Ax-, Sa = diag(a);
.
= - (AT
XNEW =
Step 3
AY AT §
e;
x+cXr+q.
(Extend Piecewise-Linear Path)
Set
tk+l = t +a .
For
tk t tk+l ,define
Step 4
2
5t)
+
L a
(XNEW - X) -
.-
(Update Lower and Upper Bounds)
(i) if eT/ltI
> 1/20 ,
then
LB = max{LB,tk-(22imrnm+l+)/(
21
2
lU
I)
'
11
(ii) if eT/l ull
< - 1/20
then UB = min(UB, tk +(22 m (m+) +1)/(21
1 1 1u ) ;
I < 1/20o ,thenLB = max(LB,t -(1 .6m(m1)+ 6)/11l11
(iii) if eT /I
and UB = min (UB, t k +(1.6m (m-1) + .6)/11Ul }).
Step5 If tk+ < ,
set
RHS = RHS+da,
k = k+1 ,
=
NEW,
and go to Step 1.
if t k
+l
t,
STOP.
2.3 Algorithmic Performance
According to Theorem 2.1, if
';
for each
k = 1,2, ...,
x(tk)
x
is close to the center of
will be close to the center of
break points of the piecewise-linear path
i
parameterized center.
Lemma 2.4
x(t)
Xt
X
k
.
,
then
Thus the
will be near the
Furthermore, we will prove in Section 4:
For all values of
t
generated by algorithm
[[x(t)- lZ. k
near thecenter
Xt,
Q = ATS 2 A,
and st = b+dt-Ait),
in the sense that
<
PCP ,
585,
(t) is
where
St = diag(st) .
U
We are now ready to discuss the performance of the algorithm.
According to Proposition 2.3, we obtain at each iteration either a geometric
decrease in the gap
the gap
t - TMIN .
TMAX - t
at each iteration, or a geometric increase in
We thus can measure algorithmic performance
according to the change in
TMAX - t ,
or
t - TMIN ,
on whether we are approximating the upper path
path
(f (t) > 0) ,
or both.
or both, depending
(f (t) < 0) ,
the lower
Suppose that in the course of running algorithm
22
PCP,
that a lower bound on
TMIN
is never generated.
Then all iterates
will satisfy criterion (ii) or (iii) at Step 4, so that all iterates will generate an
upper bound, and all iterates will lie approximately on the upper path.
Lemma 2.5
(Algorithm Performance Based Only on
TMIN = -
or if no iterates of the algorithm
bound on
TMIN ,
then the sequence of
TMAX-tk < (1 1)
In particular, if
Proof:
t values
If
generate a lower
will satisfy
(TMAX) .
t < TMAX,
128m In((TMAX
K =
PCP
TMAX)
-
the algorithm will stop after at most
) / (TMAX - i) ) 1
iterations.
Under the hypothesis of the lemma, the algorithm must satisfy either
r
l
criterion (ii) or (iii) at Step 4. Thus, by Proposition 2.3,
(
128m) (TMAX - tk).
Lemma.
Then
If
128m In ((TMAX-)/ (TMAx-t))l
let K =
Kln(1 l
• -K(
1
)+
ln (TMAX -1)
+ ln (TMAX-t)
< -(in ((TMAX - t) /(TMAX -
Thus,
TMAX-
<
Thus we obtain the geometric decrease of the
t < TMAX,
ln(TMAX-t )
TMAX - tk+1
tk < TMAX-
t,
whereby
stop.
t ))) + n (TMAX-t)
tk > t .
Thus the algorithm will
U
23
ill
Suppose instead that none of the iterates of the algorithm generate an
upper bound at Step 4.
Analogous to Lemma 2.5 we have:
Lemma 2.6 (Algorithm Performance Based Only on TMIN) If TMAX = + °
or if none of the iterates of algorithm
TMAX
at Step 4, then the sequence of
tk - TMIN
1+ .
PCP
generate an upper bound on
t values will satisfy
( TMN)
In particular, the algorithm will stop after
K = r128m in ((t - TMIN)/ (t - TMIN))
iterations.
We next examine the case when the algorithm generates both upper and
lower bounds. We first need the following result, which will be proved in
Section 5.
Lemma 2.7
If criterion (ii) of Step 4 of the algorithm
PCP
is satisfied at
iteration k, then in all subsequent iterations, criteria (ii) or (iii) of Step 4 will
be satisfied.
The significance of Lemma 2.7 is as follows: if at iteration
k
an upper
bound is generated, then an upper bound is generated at every subsequent
iteration.
Lemma 2.8
algorithm
(Algorithm Performance Based on Lower and Upper Bounds) If the
PCP
generates both lower and upper bounds, then there is some
t
24
for all
k > j
j,
the algorithm generates lower bounds only and
the algorithm generates upper bounds, and
Ui)
for all
(ii)
for all k > j,
Furthermore, if
K=
k
such that for
iterate
k :5j
tk-TMIN
TMAX-tk
2
(1128m
(TMAX - tj)
then the algorithm will stop after at most
t < TMAX ,
F256m ln TMAX 2-TMIN
(1 + 12m(- TMIN)
-128m In (t-TMIN) -128m
n (TMAX -
l + 2
iterations.
Proof: The existence of
j
is guaranteed by Lemma 2.7.
The geometric
convergence rates are then a consequence of Proposition 2.3.
suppose
t < TMAX ,
and let K be as defined above.
note that
.5 (In ( - TMIN) + n (TMAX-t))
lnTMAXI- TMIN
2
from the arithmetic-geometric mean inequality.
K
Thus,
128m In (t- TMIN)+ 128m In (TMAX - t)
- 128m In (TMAX - t) - 128m n (t - TMIN)
25
Let
Finally,
t = t,
and
lnt- TMIN
= 128m
-
TMIN
+ 128m1TMAXln
TMAX -
According to Lemma 2.5, with
128m In
TIN
t
replaced by
,
tk
t
after at most
iterations.
- TMIN
Furthermore, according to Lemma 2.6, with
after at most
t
K
t replaced by
tj+ l ,
tk
t
iterations.
.-
2.4 A Strictly Monotone Algorithm for Linear Programming that requires
O(mL)
iterations.
Suppose we wish to solve the problem
LP:
maximize
Ax
s. t.
where
t =
x,
e R n+ l ,
A
b
Rmx(n+1),
and eliminating one of the
x
and
b e Rm .
(n+1)
variables
Upon setting
of
,
LP
is
easily transformed to the form
maximize
x, t
LP:
t
Ax
s. t.
where
x £Rn, A
Rm xn ,
transformation of the data
algorithm
PCP
b E Rm
,
and the data
(A, b, c) .
to trace the path
b+dt
x(t)
Ax < b + dt .
26
We can solve
b)
LP
are a linear
by using the
of center to the parametric problem
x, e R n
Suppose
(x °, t) = (xc, cTx °)
and
path
satisfies the starting criterion of the algorithm
Ix -xtIQ <
namely
A .
Q' =AT(S)
x(t)
for
is a given starting point for which
'-1
21
where
s = b+dtO-Ax ° > 0
Then we can use algorithm
objective value of the linear program
k = 0, ... ,,
LP .
S ° = diag(s ° )
PCP
I[t, TMAX) = [cTxO, z ' ) where z'
t
PCP
to generate the
is the optimal
The sequence of values of
tk ,
will be strictly increasing, according to Theorem 2.1, i.e., the
objective value will be strictly increasing at each iteration.
total number of bits in a binary encoding of
LP .
Let
L
be the
In order to evaluate the
algorithm's complexity, we consider three cases.
Case (i): The linear program is unbounded.
never generate a finite upper bound on
After
k = O(mL)
iterations,
In this case, the algorithm will
TMAX , which equals infinity.
tk > (t -TMIN) (1 +
2 L , and we can conclude that
LP
is unbounded.
Case (ii): The linear program is bounded and
indicator function at
t = to , is negative.
f(to) , the value of the path
This being the case, the algorithm
will always generate upper bounds, and after O (mL)
(z*-tk)
(-t)(1
l
2
)
will exceed
is less than
2L,
iterations,
whereby
(tk)
canbe
rounded to an optimal solution, see Karmarkar [6].
Case (iii): The linear program is bounded and
can show as in case (i) that after
otherwise the
LP
k = O(mL)
would be unboundled.
27
f(to) > O .
In this case, one
iterations, that
f(tk) < O , for
Furthermore, after an additional
k = 0 (mL) iterations, we will obtain via case (ii) that we can round to an
optimal solution.
Thus, after
0 (mL) iterations, we can round to an
optimal solution.
Note that in either of the three cases, that algorithm
process
LP
after O(mL)
PCP
will
(by detecting unboundedness or producing an optimal solution)
iterations.
This algorithm falls into the class of central-trajectory based algorithms,
but is inferior in that the bound of 0 (mL) iterations is worse than the bound
of O(4fiL)
iterations for algorithms such as Renegar [11] or Vaidya [16] that
trace the (weighted) center of the system
Ax
cx
as
b
8
is increased, or to the bound of
O (ff L)
iterations for algorithms
based on barrier penalty methods that trace the solution to
m
maximize
cT
x + e n nsi
i=l
s.t.
Ax+s= b
s>0,
see Monteiro and Adler [10], among others.
All three methods follow the same path in their idealized version.
Yet the latter two obtain convergence in
superior to our algorithm.
O (I¥ L)
iterations, which is
However, these other algorithms do not
guarantee strict improvement in the objective value, (but do guarantee strict
improvement in the duality gap).
In contrast, our algorithm will guarantee
strict improvement in the objective function of
28
t )/128m
('-Cc
at each
iteration.
Perhaps it is the implicit imposition of the strict improvement in
objective value that increases the iteration bound by a factor of
Furthermore, our algorithm does not assume that
Instead, our algorithm will detect unboundedness of
LP
O ( V)
.
is bounded.
LP
directly.
As a final note, note that our algorithm can be used to mimic
Renegar's algorithm [11], tracing the center of
as
PCP
t
is increased.
Ax
b
-cx
-t
Thus, Renegar's set-up is a special case of the problem
we are considering.
However, we see no way to cast problem (2.2) as a
special case of the set-up used by Renegar [11]; we allow all
RHS
values to
vary simultaneously, which is apparently more general than in his work
=
[11].
The remainder of the paper is devoted to proofs of the results
presented in this section.
preliminaries.
algorithm
In Section 3, we present notation and
Section 4 contains an analysis of a single step of the
PCP , and contains proofs of Theorem 2.1 and Lemma 2.4.
Section 5 contains an analysis of bounds generated by the algorithm, and
contains proofs of Theorems 2.2 and Lemma 2.7
r
29
111
3.
Notation and Preliminary Results
In this section we present notation and some preliminary results that
will be used in the proofs of Theorems 2.1 and 2.2 and in subsequent analysis.
3.1
Notation an .d Translations
v , II v
For a vector
matrix
M,
IM
I[
i
denotes the Euclidean norm, and for a
denotes the usual matrix norm, i. e.,
iIMI = sup IIMvIIl/iivI
Note that if
If M
-'
Ilv ll
M
is a diagonal matrix,
lI M ll = max
is a positive definite matrix, the
=
M-norm
mill
of v
is
VTMV.
PM: = I- M(MTM 1 MT
The matrix
denotes the orthogonal projection
matrix which projects onto the null space of
MT . Let Qt (x; u)
the negative of the Hessian of the function
ft (x; u): =
n (bi + tui - Aix) ,
where
u e Rm
is a given vector
i=l
parameter.
(.
Let
(x): = diag(b-Ax)
Then
Qt (x; u)
Q(x)
AT -
=AT),\
-2
and
At(x; u): = diag(b + tu-Ax)
.
hen
(x) A
1I
30
t = 0 , \e denote
denote
Let
xt(u)
denote the center of the system
denote the center of the system
Suppose
observe that
d = u + Ar
Ax < b + tu ,
and let
Ax < b .
for some
At (x; d) = At (x - tr; u)
u
Rm
and
and hence
Thus, the difference between modifying the
re R.
We
Qt (x; d) = Qt (x - tr; u)
RHS
by
d
and by
simply corresponds to a translation of the inequality system by
({x R n I Ax < b+td = x
x
u
tr , in that
R n I Ax < b+tu + tr
The following Lemma is therefore obvious.
Lemma 3.1:
Ci)
Suppose
t (d) =
d = u + Ar
for some
and re R n . Then
(u) + tr ,
(ii) Ilx-xt(d)llqt(Zt(d);d) = II(x-tr)-
(iii) max
ur Rm
t
I Ax < b + td
for some
t(u)IQ(x(u);u) for any
x} = max
xE R n ,
t I Ax b + tu for some
In the sequel, we shall be working with appropriate choices of
and
r
Rn
instead of
d .
Rm ,
u
For the appropriate
where convenient and the context is clear,
welet At(x): = At(X;U)x;,
Rm
As xwe shall see in Section 5, this in fact is
central to the construction of the proof of Theorem 2.2.
u
and
(x = A(x)),
31
Xt = Xt(u) ,
S = b-Ax
x
.
III
1h
S = diag (-s)
and
St = diag(st).
St = b + tu - A
diag (^)
b - AR;,
s
We also will abbreviate
,
Qt(x; u) by Qt(x).
The next Lemma presents some basic inequalities. It is essentially
Proposition 7.2 of [4] with some simple extensions.
Lemma 3.2:
Q(x)
=
xe R n
Suppose
AT S -2 A.
Then for any
satisfies
s = b-A >O
and let
I- XIIQ() < <, 1
xE R n such that
we have
(i)
(ii)
= b-A > 0,
I I A - 1 () A
(iii) IA- ()A
and for any
(iv)
II IIQ()
(R)fl = lS-lslI < 1-81
()ll
=
-S
I
< 1+8
!
VE R n
1 II IIQ(),
1- 8
where
Q () = AT-2 A, and
(V) 11
VIIQ(l
) < (1+s)11
v IIQ() .
3.2
.
Equivalent Measures of Closeness
In this study, we measure how close a point
the system
Ax < b
with the norm
II-XIIQ(X)
32
x
is to the center
x
We shall also make
of
use of a different measure of closeness to
x
that was introduced in [4] , and
we will show a certain equivalence of the two measures.
Define
Q =
m AT-2A, y=y(x) = 1 ATS e
(3.1a)
-
and y = y7()
Note that
Ax
b.
=
/
(m - 1)yTQ-l y
denote the center of the system
ThenQ= Q() . Let
Then y(X) = 0 andso y(x)= 0
In [4], the scalar
= y(x)
is used to measure the closeness of
Lemma 3.3 ([4]).
Let
h > 0
Suppose
(3.lb)
1 yT Q y
Let
to the center
x
x .
Ax < b.
be the center of the system
be a given parameter.
=
(X) < gIngh(1 + h))
C (m'- 1)
where g(a) =
h 2 (1 + )
Then
lx-xlQx
Proof:
Follows from the proof Lemma 7.2 of [4] with weights
(1 - h) 2
w = (llm)e .
U
We shall say that
Corollary 3.1:
x
is approximately centered if y(x) < .0072 .
If y(i) < .0072,
then
33
ill
'iQ(-)
<
1/21
III
Proof:
Let
h = .03
and
(x =
h)
h
h 2 (1 +
we have IIX-XI2Q(). < m
m-1
Substituting for h and
(
Then from Lemma 3.3,
?)
((1-hy)2|
, and noting m
2
gives the desired result.
Therefore, for appropriate values of
h ,
approximately centered implies
is close to the center by our criterion,
that is
1X- XIQ(x)
Lemma 3.4:
the system
Then
< 1/21 .
x
is
h = .03 )
Next, we show the converse implication.
I-xIIQ()
Suppose
(e.g.,
< <
1/2,
where
is the center of
Ax < b
' = y(x)
<
a + 2a
62
where
.1
2(1 - 6)(1 - 26)
Proof:
Let
= b-Ax
II
(^1
=
and
jjX...XjfQ()
= b-Ax
<
From Lemma 2.1 of [ 4], vwith weilghts
m
m
,i
i=1
n s -
,
i=1
nsi<
M
111- 1
1
1-8
From Lemma 3.2,
I X....(XIIF
8
1-6
< 1.
N = (1/m ) e ,
3
(., \here
2(1-ct)
34
a =
8
1-6
<
1 .
.
On the other hand, from (2.4) of [4],
m
m
lni 1=1
Therefore,
Thus,
, lni 2 ( mI )(1 + i=1
a =
y 2<a+
,
where
y = y(x) .
(1+ - - VT-+72y .
a2
2 (1 - a)
7
1+ 2) ,
where
2
a =
82
-)
2(1-a)
.
2(1 - )(1 - 2)
Finally, we present some elementary inequalities.
Lemma 3.5:
Assume
m >2
. Let Q, y and Y be defined as in (3.1) .
I
(i)
For all
(ii)
For all
Proof:
Note that
£>
,
< 1,
9y<
£
implies
(m - 1)yT Q - y
? = (m- 1) TQ y
1 - yT Q-l y
Since
<m
m-1 +2
m- 1
To prove (ii), note that
(i)
(m - 1) yT Q-1 y < £
£ implies
whereby
yT Q- y =
follows immediately.
(mn - 1) T Q-1 y
I-EC
E
implies
m
35
2 < £/(1 - )
i
m-l +92
4.
Analysis of One Iteration of the Algorithm
In this section, we analyze one iteration of the algorithm and prove the
Improvement Theorem (Theorem 2.1) and Lemma 2.4.
if I t
is small then the two centers
x
and
xt
First, we show that
are sufficiently close to
each other with respect to some appropriate norm (so that Newton's method,
when applied, will converge).
Theorem 4.1:
and let
xt
Let
denote the center of the inequality system
u = A-' () u .
Proof:
denote the center of the inequality system
I tl < 1/(76 u)
Suppose
sufficiently small.
note that
=
tIQ(x) < 1/12
The proof makes use of Lemma 3.3 of the previous section. We want
to show that the quantity
y
Ax < b + tu . Let
I -
. Then
Ax < b
AT
y(): = I
A-
Y = 7(x)
for the system
Ax < (b + tu)
We begin by giving expressions for
y
and
() e = 0 (see (2.3)), and so for the system
AT At1 () e
1 AT [At1 ()e
is
Q .
First
Ax < (b + tu),
1()ei
m1 AT At () A- () (t u)
-i
Also,
Q1..=
ATAT
Qt(,)
()(t
= I
)
,
by definition of
AT At- (x) A
36
uG
above.
and so ,
yT Q- y = ml (
()A_ (A T t2()A
) _
A Ta()(t I) -lt
because the eliminated matrix is a projection matrix.
IIlIlt 112
YT Q-1 y(m - 1) <
1-
Thus, from Lemma 3.3 with
Corollary 4.1:
(1 -
5,775 and
y < .0132 .
2)
< ( 1 )2
I
hy) 2
Under the conditions of Theorem 4.1,
I IX
Proof:
1
Hence
h = 1/18
2h 2 (1 +
-X XtQ, .- <
<
I -11II-lul2
t 112
T Q-l y
- Xt II Q, (z
<
1/l
Follows from Lemma 3.2(iv).
.
Furthermore, using Lemma 3.2(ii), the following corollary is
immediate.
Corollarv 4.2:
Let
d
R
2
Suppose
IIx-XIIQ(, < 1/21
be given and define
U: = P AS' d
37
11
denote the center of the inequality system
Let xt
Suppose It <
X -XtiiQt(t-)
< 1/11
Proof:
-
Let
A-1()
tlA - ()SuI
Then
1
u = Su
It I<
u = A() u .
and
Note then that
IIA' (x)sl U < 2llull,
Thus,
80 11 I
< ! 21.
Qt(^Xt <
1
8011u11
20
Hence, from Theorem 4.1, we have
-Xt
and
so that
from Lemma 3.2(ii).
1l
1/76
b+tg-d
'
IIU-
4.1,
Ax
<
1
761 I 11
llx-tl a(-X)< 1/12
and by Corollary
.
1/11
We next use a theorem of Renegar [11] which gives the region and rate
of convergence of Newton's method for our problem.
Theorem 4.2
(Renegar [11]):
xe intXt = ({x Rn
Assume
Ax < b + t
E: = Ix-XtlQt,(x) <I ,1
,here
and
x. is the center of the system Ax < (b + tu) .
38
Let
Qt
t = b+ut-Ax,
ATS-2
Q= ATS
Then
and
St = diag (t)
-t =
- Q
1 ()
AT t1 e, where
A.t
11t-XtlQ,(t)
<
(1+ E2 e
.
We are now ready to prove the Improvement Theorem.
For the
reader's convenience, we restate the theorem before proving it.
Theorem 2.1
Suppose
Define
(Improvement Theorem):
e int X = x l Ax < b)}
U: =
satisfies
II -
I Qc) < 1/21
(projection of g' d) ;
S-1A Sd
I
r: = (AT S 2 A)l AT S- 2 d
a: =
(step length) .
s801ull
s:
Further, define
(translation vector of system) ;
= b + cSu-A,
q: = -(A
T
Sc: = diag(Sa);
A) ' AT S e
S
(Nexw approximate center)
x,Ew: = x + q + c.r
Then l XNE -(X-
IQ (E)V)
-
(Newton step) ;
1
39
Proof: First note that
(dl Q ((d); d) = II~a xa (u) IIQ((u); u)
XNEw -a
Hence, by Lemma 3.2 (iv) (with
where Xa = x+q
and
and
that II Xa - Xa
-
Qa.()
Let
inequality system
Ax < b +ut
W
.
let
u = S,
and
denote the center of the
xt
and let
u,r,
Qt(x) = Qt(x;u) .
For all
I
I I Q(-X)~
Rn ;
76l~lz(
for all
v
12I~i7675
1lviIQ(^)
for all
v E R n ; and
· (I10
(ii) IV
IIQ,
rt') <
(iii)
1/22 , where
.033 < 1/22
as in Theorem 2.1.
NEW
which, by Theorem 4.2, implies
<.1462
Lemma 4.1: Under the conditions of Theorem 2.1, define
te [O,c]
<
=
In Lemma 4.1 (iii) below we
Qa (Xa) = Q (Xa (u); u) .
: = IIX-XalQ()
show that,
,
x = x (d),
IIxcallQ(;a)
it suffices to show that
8 = 1/22 ),
a = Xa (u)
u = Su . Thus by Lemma 3.1,
d = u + A , where
Xt
IIK- x tllQ,(^)
<
.1462
Proof: From Corollary 4.2, iwe have
76
and
IJ|2-
I|Q(;) < 1/11
(4.1)
Thus, by Lemma 3.2(ii) and (iii) ,
(,)) A,(?)I
11A-
11
11~I and
11
an
_ 10 '
j[A.II-:. {\)A(.K)H S
40
(4.2)
IlI A () ~~~t
A () I1= max
i
Also,
bi - Ai
bi + t Ui- Ai
max
1
+
bi - Ai
< 76
t Al()ull
IItA- ()u I
because
I1- 1
76
76
(4.3)
75
from (4.1).
Similarly,
()'At ()II = max bi + tUi- Ai
i
bi -Aix
I,
= max 1+t
ui
i I
bi- Ai x
I
<
Hence,
I VIIQ(X)
77
76
1+11tA
x
= IA- (X)A v
IA
• II)- (R1)
This proves (i).
(4.4)
Qt
( x)
(X)ALt(t) I
A
()
from (4.2) and (4.4)
The proof of (ii) is similar, using (4.2) and (4.3).
show (iii), we have by the triangle inequality,
-'4
41
A vi
Next, to
III
11 3z
-
Xt
>
IIQ,(^)
1
-1 IQ(X) + 11
l I
_ (124t
11 75'
(12
i )(7(2
since
IjX-X-IIQ(-) <-20'21 IIx-0
Xt
x
x
xt
x
0) + 1-
IIQ(x)
< .1462
1
<
from (ii) and (4.1),
I
from Lemma 3.2(iv).
20
.
We are now ready to prove:
Lemma 2.4:
For all values of
Xt ,
near the center of
x (t)- xt Q,((t))
Proof:
Let
C
generated by algorithm
t
PCP ,
x(t) is
in the sense that
.585
a = 1/(801111)
as in Theorem 2.1.
Lemma 3.2(ii), it suffices to show that
Let
te [0, ] . By
IIx(t)- tIIQ() < 0.369
We have, from the proof of Theorem 2.1, and Lemma 4.1(iii),
IiXa
xIQa(a) - I|
x
Hence, IIX- X IQ(x)
IQa(a)
+
I
XC
12I7)
(~ 1 (x--
I Qa(x,)
<
.033 +.1462 < 0.18
I(-
< 2ii'~7i(I
71 Xa - R I (Ux
1I-,(.
11 75 10 76
< 0.222
42
(by Lemma 4.1(ii))
(by Lemma 4.1(i))
Next, observe that
Ix(t) -
IlQ (,) =
= t (a-x)
-(t)-x
Therefore,
LI (a - x) I Q,( ,) < 0.222(t/a)
Thus, by the triangle inequality and Lemma 4.1(iii),
I!5(t)-XtIIQt(,) < Ilx(t)-IllQ()1 ) + I1i -tllQ,(2,) < .222
X
X
x
X~~~~~~~
+ .1462 < .369.
II
5.
Lower and Upper Bounds
In this section, we analyze the upper and/or lower bounds generated by
algorithm
PCP , and we will prove Theorem 2.2 and Lemma 2.7.
Recall that
TMIN:.= inf T
TMIN ·
T = ( t I intX t
,
0
We shall derive upper bounds on
TMAX
and
and lower bounds on
We shall, for convenience, adopt the following notation and assumptions
throughout this section.
The current value of
X is the center of the system
t
is
t =
x - x Q() =
iterate x satisfies s = b-Ax > 0 and
where
TMAX: = supT
Ax < b ,
,
and the current
<
S (s -s
= b-Ax,
1/21
S = diag(s) and
S = diag () . We decompose d into
d = u + AF,
where
= PS-'AS
d and
(AT -2 A)AT- 2 d .
F
By assumption,
u
u =
•
0 .
d
(5.1)
does not lie in the column space of
A
We shall prove Theorem 2.2 and Lemma 2.7 in this section.
begin with three fundamental Lemmas.
Lemma 5.1:
Proof:
and so
eTS
leT S
1
u - eTu
<
I
l/20
u - eT Ie(S
|
-
=e( - S
<
is'
<
44
]ul
R '.r)
·
i
'1
U(s-s
We
The first inequality is the Cauchy-Schwartz inequality and the last follows
from Lemma 3.2 and the assumption that
f() = eT S u
Recall that
If e T i >
IIui/
is the path indicator function at
thenfromLemma 5.1,
20
Therefore, the current center
eTS' u
x is on the lower path and
e T'
then
TMAX
.
>0.
TMIN is
-
m/(eT S u).
is bounded above by
However, we typically cannot deduce the exact value of
- m/(eT -1 u)
eTS' u
< -1uil/20
t=
11/20
eTU11-_
bounded below, as we have seen in Section 2, by the bound
Similarly, if
.
IiX-XjQ(i) < 1/21 .
from the current iterate.
Therefore, we shall later in this section
derive an alternate bound using the next Lemma which is a variant of
Lemma 7.1 of [4].
Lemma 5.2:
Let
be the center of the syst,em
xe intX = (x Ax<b)
the ellipsoid
FOUT
FOUT: =
Then
Proof:
Let x
int X
Suppose
IIX-xIIQ) < 8 < .
Define
by:
xe R n
intX
is given such that
Ax < b .
I IIx-xIIQ()
<( +)Vm(m -1) + 68)
(5.2)
FOUT .
be given and let s = b-Ax . Then from properties of
the center, (see for example [4] , Theorem 2.1), ilx -
45
II Q(~)
<
/im (m-l)
From
III
II x - X I Q(I)< (1 + 8)/m (m-i )
Lemma 3.2(v), we also have
IIx--IIQ(T)
< Ifx-XiIQ(¢)
+
IIX-RIIQ()-< (1 + 8)Vm
(nm-
Thus,
+
.
8
The next Lemma concerns the well-known classical least-square
problem or minimum-norm problem.
Lemma 5.3:
mxn matrix M
Given an
and an
m-vector d ,
for all
scalar t,
IIMx-tdjI
where
t
2
ItIt IPMd
for all x E R n ,
PM = I - M (MT M'MT.
We are now ready to prove Theorem 2.2 and Lemma 2.7.
We shall
prove Theorem 2.2 in two parts.
Proposition 5.1:
(i) if eTI
(ii)
Proof:
Under the definitions and conditions of Theorem 2.2,
[ < -1/20
if eT-/ll
then
> 1/20
(i) Suppose
TMAX
TMIN
then
eT Wllu 1
<-
1/20 .
< UB - (22 Vim+1j+ 1)/21 Iull ,
>
LB
- (22 fr-im+1 + 1) / 21 I 11 .
We first show that
0
is
close to the center of the following extended system with one additional
variable and one additional constraint:
A
x
b'
(5.3)
46
eu S'U
where[ ]e
A=[0
where
0
and
0
=
I 1-
eT S
0 < 0.
.
Note that from the remark following Lemma 5.1,
Therefore,
for some x)
TMAX = SUP {t I Ax < b +dt
t
=
sup t
t
I Ax
t I- x
= sup
t
for some x)
b + ut
for some x}
-1
For system (5.3), define
Then,
A(x,O) =
S O
and
- t
A(x,t): = diag([ ]
I
SO
A , 0) =
-1
Therefore,
x
= le
[eT
X =([x]
1 A, -eT
=
-1 u+
is the center of system (5.3) and
0
lI[] -[ l1
Let
and
0 1
0 1
[eT, 1],& (R",0)
Q(x,t): =
e
Q(X O)
Rn[+1A
=II
xt[ ] <
-'II Q(x) < 1/21
1 I
47
/
and
(x,t)
TA
-- 2tA
.
III
FOUT=
[ :]
E R n+ l
[ X]
Then, from Lemma 5.2,
int'X
Furthermore, because
= A T S-
Q(x, ) =
[
ATs -2 A
-ATS
R
0
<
i5(X,
0)
2m
21
m1 + 1
21
; FOUT
-2
= A T -2
u
]
u , we have
ATS2- A
0o
0
02 +1lu~12
-uT S -2 A 02 + UT S-2u
·
Thus
[tt
0
2(,
0 1 ; Q(i,O)
)
> t 2 11
whereby
II2
Et]
Also, for any
+ t2(0 2 + II II2)
A(x -)112
=1131i
( o0)
0;
t
22
m
iim +1T + 1
21
Vmim+1 + 1
t< t< 22
2111 II
It then follows from Lemma 3.1 that
TMAX = SUP ( t
Ax - tu < b
for some
sup {t
Ax- tu < b
for some x ,Ot < 1)
=
= sup
t
fx
E
intX
for some
48
xI
x
I
(since 0 < 0)
• (22
x}
E FOUT for some
5 max t I [t
(since int X
FOUT).
m (im+)j+ 1)/211ll
The proof of (ii) exactly parallels that of (i).
Proposition 5.2:
(i)
TMAX <
UB
(ii)
TMIN
LB -(
Proof:
S
eTd/ IJuJi
Suppose
I
Then
< 1/20
( 1.6 m.(m-1) + .6 )/
1.6/ m (m
UI I
) + .6)/ [-ll
We shall show that
00] is approximately centered (in the sense of
Section 3.2) for the extended system
[x]
[A, -u]
Let
Section 3.2.
Q, y
Let
for system (5.4).
and y = y(x)
Q,y
forsystem
and y = y(R,O)
Ax
b
be as defined in
be the corresponding parameters
Observe that
= b-A
b -[A, -u]
Therefore, since
(5.4)
u =
u
=
> 0
and
AT S-1
=0 ,
r
Q=4
-AT
L_uT
s-2 [A, -ul
=
49
Q
0
0
IIi2/m
1
11
and
and
y
AT
-u T
m
Therefore,
I
[-eT -d/mj
yT Q-1y + e
T -1
IIX-gXQ()
Recalling that
Y
-I ee
S'
/
2
< 8 = 1/21 , wecompute
conclude from Lemma 3.4 that,
'y
<
.
a+
a = 760
1
and
Next from Lemma 3.5(i),
Thus,
(m
-1))yTQyY
I
+
(760+
Im-)lull)
-
< .0053
2
t
Therefore, by Lemma 3.5(ii),
h = .41
[x:]
< .0735 .
and so
Taking
in Lemma 3.3, we have
]
2r~ <) i
- K
where
xt]
Note that
)
m-
h2(1 + 2) < .36 < (.6)2
, (since
(1 - h) 2
is the center of system (5.4) and
[
X]
Let
q < .0054
X: =([ t]
11~5(XO)
=
IIs-
A(x
and
[A, -u]
50
Q(x,o) =mQ.
- x)-
tu
.
m/(m-1)
2)
FOUT:
=
it]
IIS-'A(x-x)-t
int X
Then by Lemma 5;2,
[:]
e FOUT
+ .6
(m-)
<_. 1.6
By Lemma 5.3, for all
since
> Itl JJUJI
1.6 Im (m-1) + .6 > jj'A(x-R)-fu-jj
Thus
= 0.
ATS-1
Four.
i[
sup (Itl I t
T)
=sup (ltl
[I
< max (Itl [x] e
< (1.6
xl
for some
FOUT
m (m-1) + .6)/lU
x)
for some
e int X
I
U
This completes the proof.
Finally, we
Theorem 2.2 follows immediately from Propositions 5.1 and 5.2.
prove
Lemma 2.7:
iteration
k ,
criterion (ii) of
If
will be satisfied.
Proof:
Suppose
criterion (i)
eT1/ll/jll
< -1/20
in
is satisfied at
algorithm PCP
of
then in all subsequent iterations,
Step 4
that
Step 4
criteria (ii)
iteration k
or
f(tk) = eTS u < 0 .
51
of
It suffices to show
will not be satisfied in all subsequent iterations.
Lemma 5.1, we have
(iii)
By Proposition 2.2,
By
III
I
f (t) < f (tk) < 0
criterion (i)
for all
t > tk .
Therefore, in all subsequent iterations,
will not be satisfied, as it would imply that
t > tk .
,.
52
____11__11_1____11___-.I___--
_
f (t)
0
for some
References
[1]
Barnes, E. R., S. Chopra, and D. L. Jensen (1988). "A polynominal time version
of the affine scaling algorithm." Working paper, Graduate School of
Business Administration, New York University, New York, NY.
[2]
Censor, Y. and A. Lent (1987). "Optimization of 'log x' entropy over linear
equality constraints." SIAM Journal of Control and Optimization 25,
921-933.
[3]
Freund, R. M. (1988). "Projective transformations for interior point methods,
Part I: Basic theory and linear programming." Working paper OR 17988, Operations Research Center, M.I.T., Cambridge, MA.
[4]
Freund, R. M. (1988). "Projective transformations for interior point methods,
Part II: Analysis of an algorithm for finding the weighted center of a
polyhedral system" Working paper OR 180-88, Operations Research
Center, M.I.T., Cambridge, MA.
[5]
Jarre, F. (1988). "On the convergence of the method of centers when applied to
convex quadratic programs." Manuscript, Institut fir Angewandte
Mathamatik und Statistik, Universitit Wiirzburg, Am Hubland, 8700
Wiirzburg, West Germany.
[6]
Karmarkar, N. (1984). "A new polynominal time algorithm for linear
programming." Combinatorica 4, 373-395.
[7]
Kojima, M., S. Mizuno, and A. Yoshise (1987). "A primal-dual interior point
method for linear programming." Research report No. B-193, Dept. of
Information Sciences, Tokyo Institute of Technology, Japan.
[8]
Megiddo, N. (1986). "Pathways to the optimal set in linear programming."
Research report RJ 5295, IBM Almaden Research Center, San Jose, CA.
[9]
Mehotra, S. and J. Sun (1988). "A method of analytic centers for quadratically
constrained convex quadratic programs." Technical report 88-01, Dept.
of Industrial Engineering and Management Sciences, Northwestern
University, Evanston, IL.
[10]
Monteiro, R. C. and I. Adler (1987). "An O(n 3 L) primal-dual interior point
algorithm for linear programming." Report ORC 87-4, Dept. of
Industrial Engineering and Operations Research, University of
California, Berkeley, CA.
[11]
Renegar, J. (1988). "A polynominal-time algorithm, based on Newton's
method, for linear programming." Mathematical Programming
40, 59-94.
III
[12]
Sonnevend, G. (1985). "An 'analytic' center for polyhedrons and new classes
of global algorithms for linear (smooth, convex) programming."
Preprint, Dept. of Numerical Analysis, Institute of Mathematics, E6tv6s
University,; 1088, Budapest, Muzeum K6rut 6-8.
[13]
Sonnevend, G. (1985). "A new method for solving a set of linear (convex)
inequalities and its applications for identification and optimization."
Preprint, Dept. of Numerical Analysis, Institute of Mathematics, E6tv6s
University, 1088, Budapest, Muzeum Kbrut 6-8.
[14]
Todd, M. J. and Y. Ye (1987). "A centered projective algorithm for linear
programming." Technical report No. 763, School of Operations
Research and Industrial Engineering, Cornell University, Ithaca, NY.
[15]
Vaidya, P. (1987). "A locally well-behaved potential function and a simple
Newton-type method for finding the center of a polytope." AT&T Bell
Laboratories, Murray Hill, NJ.
[16]
Vaidya, P. (1987). "An algorithm for linear programming which requires
O(((m+n)n 2 + (m+n) 1 .5 n) L) arithmetic operations." Proceeding of the
Nineteenth ACM Symposium on the Theory of Computing, 29-38.