Dual Gauge Programs,
with Applications to Quadratic Programming and
The Minimum-Norm Problem
by
Robert M. Freund
Working Paper #1726-85
Revised August 1986
Abstract
A gauge function f(.)
positively homogeneous and
is a nonnegative convex function that
satisfies f(O)=O.
are specific instances of a gauge function.
Norms and pseudonorms
This paper presents a
gauge duality theory for a gauge program, which
minimizing the value of a gauge function f(.)
is
is the problem of
over a convex set.
The gauge dual program is also a gauge program, unlike the standard
Lagrange dual.
We present sufficient conditions on f(-)
the existence of optimal
solutions to the gauge
dual, with no duality gap.
qualifications on the constraints.
The
gauge dual
programs,
program is
than the standard dual,
Keywords:
and
independent of any
to the minimum lp norm problem.
shown to provide a smaller duality gap
in a certain sense discussed
in the text.
Gauge function, norm, quadratic program, Lagrange dual,
Running Header:
, , l 11,
its
The theory is applied to a class
duality.
_ll,
program and
These sufficient conditions are
relatively weak and are easy to verify, and are
of convex quadratic
that ensure
Dual Gauge Programs.
Introduction
A gauge
function
function f(.):RnRu{(+
=}
is a nonnegative
convex
that is positively homogeneous and satisfies
f(O)=O.
and pseudonorms are specific instances of a gauge function.
program
is
defined as an optimization
P:
f(.)
is
the form
quadratic
to Mx > b
a gauge
programming fall
function.
Many problems
into this category,
programming,
in mathematical
including strictly convex
linear programming,
problem on a polyhedron (see Luenberger
and the minimum norm
[9]).
Duality for programs
similar to P have been studied by Eisenberg [2],
recently been
generalized by Gwinner
instances where explicit Lagrange
programs
A gauge
minimize f(x)
subject
where
problem of
Norms
like
P can be stated,
[7].
whose work has most
Glassey [6]
has examined
duals of convex homogeneous
without
reference to primal
variables.
This
paper presents a gauge duality theory for gauge programs
that contrasts, but
particular,
is related to,
the gauge dual D of P
the Lagrange dual
of P.
In
is also a gauge program, unlike
its
Lagrange dual.
are
feasible values of the primal and dual objective functions,
z.v >
D.
The gauge duality theory states that if
z and v
then
1, with equality only if z and v are optimal values of P and
This inequality is analagous to the weak duality relationship z
> v for the Lagrange dual.
We present sufficient
that ensure that optimal
solutions
and that z-v=1
solutions.
for these
1
conditions on f(.)
to the dual gauge programs exist
These sufficient conditions are
III
relatively weak and are
easy to verify.
They are independent of any
qualification on the constraints of the gauge program, unlike the
Slater condition
In the case
applicable to
than the
dual
for Lagrange duality, for example.
of quadratic
a class
programming, the theory developed
of quadratic
programs that
class of strictly convex quadratic
is equivalent
program different
is slightly broader
programs.
The gauge
(by a monotone transformation) to a quadratic
from the
Lagrange dual.
Another application of the gauge duality theory is
problem of minimizing the
The gauge dual
certain
is
is shown
lp norm of a vector over a polyhedron.
to provide a smaller duality gap
sense discussed in the text)
hence provides a better
for feasible values of
to the
than the standard dual,
lower bound on the primal
the dual,
(in a
than does
and
objective value,
the standard dual
program.
A final application of the gauge duality theory
programming, where the gauge dual
is different
is to linear
(but equivalent to)
the standard linear programming dual.
In order
1 reviews
to lay the groundwork for the ensuing theory,
basic polarity properties
of gauge functions.
presents the gauge duality theory, which
theorem, and necessary conditions
valid.
Section
Section 2
includes a weak duality
for a strong duality theorem to
be
The duality theory of Section 2 is generalized to gauge
programs with nonlinear constraints
in Section 3.
Section 2 is applied to selected mathematical
Section 4.
programming,
This
section first
The theory of
programming problem in
discusses convex quadratic
followed by a duality analysis of the minimum
problem, for which strictly convex quadratic programming
2
lp norm
is a
special case.
The discussion shows that the duality gap for the
gauge dual is in a certain sense smaller than that of the standard
dual.
Section 4 concludes with an analysis of linear programming in
the context of gauge duality theory.
3
III
1.
Preliminaries
Let R denote
the set
numbers and
real
positively homogeneous
f(ax)
(i.e.,
= af(x)
and pseudonorms are gauge functions.
A
let R=Ru{(+}.
convex, nonnegative,
is
f(-)
f(.):Rn-R is called a gauge if
function
Norms
of
for a > 0),
and f(O)=O.
A gauge function need
not be symmetric and can take on the value +,
unlike a pseudonorm
or a norm.
An example
of a gauge function
f(x)=f(xl,
x2,
(
Note
that f(x)
that
f(x)
= 0
x3)
=
all x =
if 2x1-x2-x3 =
1
(Xl-x3),(xl-x2)12
(+
otherwise.
the plane
is finite only on
for
is
In
(a,a,a).
{xER 3 12xl-x 2 -x
this example,
3
f(.)
= 0),
and
is
symmetric.
For any convex set C c
by C
=
{yeRnlyTx
1 for all
containing the origin,
the polar of C,
Rn,
xC}.
is a gauge
function,
and
is a closed convex set
C
(see Rockafellar
[12],
p.
If
121).
if C is defined by
C =
then f(-)
is defined
= C if and only if C is a closed
and C
convex set containing the origin
f(.)
denoted C,
f(x)
{(xRnl
(1)
< 1)
can be represented by
f(x)
where by convention,
closed function
and only if
(i.e.,
the origin,
function,
({
Furthermore, f(.)
level sets of f(.)
Also,
the function
(2)
> OxEuC).
= +=O.
inf
all of the
C is a closed set.
that contains
closed gauge
we denote
= inf
are closed)
for any closed convex set C
f(.)
defined by
called the closed gauge function
corresponding to C.
4
is a
(2)
is a
if
For any gauge function
=
fo(y)
f(.),
inf
Then f(.)
is a closed gauge.
(ylfo(y)
1),
<
Furthermore,
define its
{v > olyTx
< vf(x) for all x).
If C =
1},
(xlf(x)
whereby f(.) is closed,
if f(.)
is
polar function f(.) by
closed,
since
then fOO(x) =
C
then C
(3)
=
is closed.
f(x),
because C ° O
=
C.
The following summarizes the above statements:
Remark
1 (see Rockafellar
f(.)ofO(.)
[12],
p.
129).
induces a one-to-one symmetric correspondence in the
of all closed gauges on R n .
class
The polarity operation
the origin are polar to
functions are polar
Two closed convex sets containing
each other
if and only if
their gauge
X]
to each other.
We also have:
Remark
2.
If f(.)
is a closed gauge function,
then f(.)
and f(.)
can be written as:
f(x) =
sup yTx
yECO
and
In the case when f(x)
IX{{q,
=
fO(y)
that yTx <
Holder
where 1/p +
=
fO(y)
xHIp,
1/q =
the lp norm,
1
p <
,
then
inequality states
The following generalization of
the
inequality can be stated as:
Remark 3.
f(x) and
If f(.)
fO(y)
by f*(y)
is
a gauge function,
are both finite or that
For a given
function f(.):Rn.R,
= sup {yTx-f(x)).
x
If f(.)
straightforward and demonstrate that
5
ij______l___l__l__I__C_
and
1, and the Holder
= f(x)fO(y).
xUpHUyHq
= sup yTx
xEC
--__I___^_IIIII____.--
then yTx < f(x)fO(y)
{f(x),fO(y)) #
its
(0,}).
conjugate f*(.)
is a gauge
provided
is defined
function, then
it is
III
fo(y)
0 if
f*(y)
If
g(x)
the function
function,
is
=
+= if
a-gauge function
f(x,w):Rn+m-R
< 1
fo(y) >
defined by x E Rn and w
defined by f(x,w)
= g(x)
and
fo(y,z)
where y E Rn and z
go(y)
if
z =
0
+*
if
z
0
=
Rm.
6
is
E
R,
a gauge
then
2.
Dual Gauge Programs
Consider
the following nonlinear program:
P:
to
subject
where f(.)
is
z = f(x)
minimize
Mx > b
The Lagrange dual of P
a closed gauge function.
is
formulated as
{f(x) -
supremum {infimum
>
XT(Mx-b))),
x
to
which can be simplified
-
supremum {bTX
X > O
f*(MTX))},
or
LD:
maximize
v = bTX
subject to
fo(MTX) <
1
X > 0
Dual
programs for classes of programs
developed by Eisenberg
how to construct
problems.
All
[2]
explicit duals
[7].
Glassey
[6] has
shown
for such
problem for which
is convex and positively homogeneous
need not be nonnegative,
to be finite-valued).
P have been
(with no primal variables)
three authors work with a primal
the objective function f(.)
(f(.)
and Gwinner
that include
as in our
primal, but
is restricted
When applied to a gauge program P, however,
the dual programs that each author develops is the program LD above.
function with the polar
If we exchange the objective
constraint in LD,
D:
we obtain a dual gauge program:
minimize
subject
v =
fo(MTX)
to bTX =
1
X > 0
7
gauge
III
Together,
Note
the pair
that both the primal
problems,
and
to primal
appears
(P) and dual
programs,
(D) are minimization
their objective functions
are polar gauge
functions.
variables X are restricted to be nonnegative and correspond
The dual
side
P and D constitute dual gauge
constraints.
in the dual
(RHS) of
in
The constraint matrix M in
the primal
the objective function, and the right-hand
the primal appears
in the equality constraint in the
dual.
The definition of
other
types
constraint
of
of
the gauge dual program D can be extended to
linear constraints
P is Mix
= ()
bi,
in the
then in
ith variable Xi to be unrestricted
standard way.
the dual D,
in sign
(less
If
the
ith
we require
than or equal
the
to
zero.)
Note
that the dual of the dual
write the dual
in
to:
y-MTX = 0
1
X > 0
= f(y).
s with constraints
To see this,
v = g(y,X)
bTX =
where g(y,X)
the primal.
the format:
minimize
subject
is
Then
(i),
(i)
(x)
(ii)
(t)
(iii)
(s).
if we associate the variables x,
(ii),
(iii),
and
is:
minimize
z = gO(x,bt-Mx+s)
subject to:
t = 1
s
>
0.
8
t,
and
the dual of this program
Thus
fOO(x) =
gO(x,w) =
However,
f(x) when w=O, and gO(x,w) =
+.
O.
if w
the last program can be written as
z = f(x)
minimize
Mx-s =
subject to:
b
s >O
is precisely the primal
which
P.
The vector of variables x
if x
said to be feasible
i.e.
satisfy the linear constraints,
(X)
otherwise x
(X)
= +),
(fO(MTX)
If x
infeasible.
is
then x
feasible and f(x)
If P
is
(X)
<
(X)
is essentially
(fo(MTX) < +),
+
infeasible
otherwise P
program;
(D)
0),
= +-
f(x)
If x(X)
infeasible.
is
then x
(X)
is strongly feasible
P
(D)
is an essentially
solution,
has no strongly feasible
(D)
(bTX=l, X >
Mx > b
is feasible but
(X)
for P (D)
is a strongly feasible program.
We have the following preliminary duality result for dual
gauge
programs.
Theorem 1.
Let z
respectively.
and v*
respectively,
then zv >
v*
=
+~.
is essentially infeasible,
i.e.,
z
=
+o.
= O,
then D is essentially
(iii)
If v*
= O,
then P
(iv)
If x and X are feasible solutions
values z and v,
optimal
respectively,
solutions of P and D,
If x and X are
and
(iii)
f(x)
= zv,
and zv =
1, then x and X are
respectively.
by Remark 3.
then we have
This result shows
follow by contradiction from
9
1.
for P and D with objective
strongly feasible for P and D,
1 = XTb < XTMx < fo(MTX)
(ii)
1, and hence z*v* >
i.e.,
If z*
and
(D),
infeasible,
(ii)
(i),
and
strongly feasible for P and D, with objective
values z and v,
PROOF:
(P)
Then:
If x and X are
(i)
be optimal values of
(i).
(iv)
follows
III
from
z.
(i)
is achieved by
bound on z*, and
by noting that 1/v is a lower
[XI
(i)
Assertion
value
of
corresponds to the standard duality result that the
less
is
the max program
Assertions
equals min, then both are optimal.
f(x)
>
0 for any x,
the min
of
if max
(iii)
and
is
Because f(.)
= 0,
the program P
the same way that a standard
in
limit,
program would have a value of -,
i.e.,
If z*
whereby z* > O.
has achieved its absolute lower
infeasible,
(ii)
in the standard theory.
correspond to unbounded cases
a gauge,
to the value
corresponds to the result that
(iv)
program, and assertion
than or equal
and hence the dual program is
= +.
v*
In order to prove a strong duality theorem for the dual gauge
programs
their
the
optimal values,
it is necessary
A convex set S c Rn satisfies
A gauge
both C and C
C =
(xERnlf(x)
the projection property if all
S are closed convex sets,
transformation A:Rn4Rm,
set.
to impose some qualifications on
We proceed as follows.
function f(.).
projections of
and that P and D both attain
that z*v*=l
P and D that asserts
(z
RmIz=Ax for
E
function f(.)
satisfies
i.e.,
if for any
is
some xES)
a closed convex
the projection qualification
satisfy
the projection
property, where C is
< 1).
For notational
convenience,
assumed to be related by relations
linear
(1) and
(2)
f(-)
if
given by
and C will
be
of the previous section,
for the remainder of this paper.
f(-)
satisfies the projection
f(.)
satisfies the projection
Note that by definition,
qualification if and only if
qualification.
If
C and C
are convex and compact,
the projection qualification, and hence
projection qualification
for
1
<
p <
10
.
f(x)
=
then f(-)
satisfies
xI1p satisfies the
Also note
that if
C is a
polyhedron
(bounded or not),
then C
is a polyhedron
Finally, note that
satisfy the projection qualification.
gauge function that satisfies
f(x,w) = g(x) then
f(..)
The projection
the projection
satisfies
and f(-)
and f(.)
if g(-)
qualification, and
the projection qualification.
qualification allows us to prove the following
results which will
be useful in proving the strong duality theorem.
Remark 4.
satisfies the projection
f(x)
is
PROOF:
i.e.
If f(.)
have f(x) =
{ulu
sup yTx
yECO
finite,
Lemma
1.
is
a nonempty closed convex set,
If f(x) is
then f(x) = yTx for some yCO. [X]
If f(.)
satisfies the projection qualification, and X is a
vector yERn such that
The
for a given
> O,
for all xEUC, yTx < 1, and
then there exists a
for all xEX,
yTx > 1.
sets UC and X are convex and have no points in common.
There thus exists a hyperplane
satisfies the
closed.
if
{uIU=yTx for some yECO}.
by the projection qualification.
polyhedron such that UC n X =
PROOF:
= sup
= yTx for some yECO}
a closed interval,
qualification and
= yTx for some yCO.
finite, then f(x)
We
However,
projection
that separates them.
qualification, all
Furthermore, X is polyhedron.
Because
projections
f(.)
of uC are
Consequently, according to
remark 5 in the appendix, there exists a hyperplane that separates
from X and does
that yTx
must be
not meet UC.
< a for all xEUC,
Therefore, there exists
and yTx
a
>
[X]
We can now prove:
11
aa
·--
for all xEX.
positive, whereby by scaling we can presume
completing the proof.
-Ls(B
is a
__
UC
(y,a)E(Rn,R) such
Since x=O EUC, a
it is equal
to
1,
11
Theorem 2.
and
let
Assume that f(.)
z* and v*
satisfies
be the optimal
the projection qualification,
values of P and D,
respectively.
Then:
(i)
If P and D are both
strongly feasible, then
z*v* =1;
and
the optimal values of P and D are achieved for some x*
(ii)
and
*.
P is
essentially infeasible
and v*=O
v*=O;
is
(i.e.
z*=*)
if and only if
achieved for some feasible X*
if P is
infeasible.
(iii)
D is essentially
z*=O;
and
z*=O
infeasible
(i.e.
v*=o)
if and only if
is achieved for some feasible x*
if D is
infeasible.
Let X = {xERnlMx > b}.
PROOF:
(i)
then 0
< z*
< +=
x
f(x)
=
E
X,
Lemma
and 0
/v*.
1, there
< +.
< v*
Assume
exists yRn
so does
convex set,
since yTx
y=MTX,
XTb
fO(MTX)
=
i.e.
(1/v*)C,
1.
x.
and
<
[c,d]
feasible point X*,
dv* < v*,
n X =
and
fr om
x(1/v*)C,
(i).
and d < 1.
there exists X
and
z*
shown in Theorem 1.
> 0 with
Thus P achieves
= f(x*)
Regarding
= l/v*.
its minimum at some
(ii),
For the
the
"if"
"only if"
Then for any finite
, whereby there exists a vector yRn as described in
12
Now
and
a contradiction.
assume that P is essentially infeasible.
UC
(-a,d],
shows that D achieves
establishing
the statement has been
or
is feasible for D,
its minimum at some feasible point x*,
A parallel argument
(1/v*)C n X=¢,
therefore (yTxlxE(l/v*)C) is a closed
satisfy Mx > b,
Now X = X/XTb
for some
But since C satisfies the projection
a closed interval
(1/XTb) fO(y)
Then
that satisfies yTx < 1 for all
> 1 for all x that
>
We must now show that
the contrary.
and yTx > 1 for all feasible
property,
If P and D are strongly feasible,
part o f
part,
> 0,
Lemma
Let X and X be constucted as above.
1.
for D and f(MTX) <
If
1/U,
P is infeasible,
exists X > 0 with MTX
This completes
The proof
Theorem
= 0 and
(iii)
1/u for any U > 0,
i.e.
v*=O.
then by a theorem of the alternative, there
the proof of
of
<
whereby v*
Then X is feasible
bTX = 1.
Furthermore f(MT\) = 0
= v*.
(ii).
parallels that
of (ii).
[X]
2 thus provides a rather weak qualification on f(.)
that
is sufficient to guarantee the existence of primal and dual optimal
solutions, namely that all
projections of C and C
be closed.
Note
that the projection qualification makes no reference to the constraints
of P,
in contrast
to more typical
Slater condition, which is
in Eisenberg
[2]
sufficient
conditions such as the
used directly in Glassey
and Gwinner
[7].
As the proof
Also note that f(.)
+
take on the value
2 is based
theorem
in
> 0,
qualification is
then UC
shown
stemming from an
that
to guarantee the
(i) of Theorem 2 asserts
qualification is sufficient
optimum z* where 0
< z*
The proof
<
in order
,
part
if uC and X are disjoint
optimal
solution when z*=0,
makes no
such assertion when
is
infeasible is
To see that the projection
to guarantee
let f(.)
the
existence of
be defined as in
13
_I
projection
that the projection
consider the following example.
C = {(xl,x2)eR2x2 > x/2) and
_I
The
open separation.
The rather stronger condition that the dual
qualification is not sufficient
can
of Theorem
for a gauge program to attain
(ii)
to be sufficient in this case.
and
"open separation"
can be openly separated from X.
sufficient
Although part
z*=O.
need not be differentiable,
(see the appendix), which states
for some
2 indicates,
the feasible region
the feasible region of P.
essentially on arguments
and indirectly
of Theorem
a constraint qualification is not needed, because
is polyhedral.
6]
_____
(2).
an
Let
Thus
its
III
we have
It
0
xO,
X2=0
+o=
x 1 70,
X2=0
+00
X2
=
f(xl,x2)
that both C and C°
the feasible
Let
< 0
{(Y1,Y2)ER2ly2
=
to derive C
is straightforward
verify
> O
X2
x2/(2x 2 )
satisfy the projection qualification.
region X of P be defined by X =
-2
= X1
which
= 1/(2x2),
/(2x2)
However,
z*=0.
Thus
spirit of
follows.
z*
and v*
(rel
there
to
(xl,x 2 )
for
condition for strong duality in P and D in
is given below.
We
define X =
proceed as
{xERnJAx > b),
<
} and
The dual gauge programs P and D each attain their
optima
<
and z*v*
int Y)
X
n (rel
o).
=
0, bTX > 1).
1,
if
(rel
int Co) f
is precisely Fenchel's
C =
{xERnlf(x)
int X)
n (rel
,
and
intersection of relative
sufficient condition
see Magnanti
int C) #
0.
shown to be equivalent
Lagrange dual,
Let
We have
Note that the condition on the
has been
where
and
is no feasible pair
dual gauge programs P and D,
Given
CO = {yERnlfo(y)
2.
Mx > b},
which goes to zero as x2 goes
the Slater condition
and Y = {yeRnly=ATX,
Lemma
2
2 )=0.
f(xl,x
An alternate sufficient
the
Mx > b,
large,
By choosing x 1 =1 and x2 sufficiently
infinity.
{xER
, and b =
M
f(xl,x2)
2
-Y 1 / 2 ), and to
<
interiors
for strong duality, which
to the Slater condition for the
Unlike the projection qualification,
[10].
this condition can be rather cumbersome to verify
14
in practice.
We will
not prove
consequence of Theorem
of Lemma
proof
Lemma 2 here.
Its proof
2A of the next section,
follows as an immediate
which
is a restatement
2 for a gauge program with nonlinear constraints.
there
indicates,
the intersection
condition of Lemma
As the
2 guarantees
strong duality by guaranteeing proper separation of z*C and X if z
not attained.
The projection qualification,
on the other hand,
gives
us strong duality by guaranteeing open separation of z*C and X if
not attained.
In this sense,
stronger qualification,
separation
valid
because open separation is
Nevertheless,
indeed a "sufficiently"
for all of the applications
a stronger
the projection
weak condition so as to be
in Section 4 of this paper.
15
··_11_1___111________·_____··_·______
z*
the projection qualification is a
than proper separation.
qualification is
is
is
III
3.
An Approach to Dual Gauge Programs with Nonlinear
Constraints
In this section, we show how the gauge duality theory developed
in Section
2 for gauge programs
(with linear constraints) can
conceptually be extended to include
nonlinear constraints.
Consider
the nonlinear gauge program
NGP:
subject
where f(.)
z =
minimize
f(x)
to xEX
is a closed gauge function,
and X
is a closed convex set,
not necessarily a polyhedron.
Corresponding to the gauge function f(.)
fo(.).
is its
polar function
In order to develop a nonlinear gauge dual of NGP, we also use
a duality correspondence for the set X.
X, define X'
closed convex set
by the relation
X'
Following McLinden
=
{yERnlyTx > 1 for all xEX}.
[11],
we will
although this nomenclature
context, X'
For a given
to X'
is not universal.
is the blocker of X
the upper dual set of X.)
refer
in Fulkerson
as the antipolar of X,
(In
a more restrictive
5].
In Ruys
[13],
We define the nonlinear gauge dual
X'
is
of NGP
to be the program
minimize v = fO(y)
NGD:
subject
In the dual,
to
yEX'
the objective
function is
the polar
function, and the dual objective function
primal feasible
dual
region.
is
of the primal gauge
the antipolar of the
In order to characterize when the dual
of the
is the primal, we need to introduce some additional definitions.
For a given nonempty convex set XcRn,
X if for every xEX, x+er E X for all e
16
a vector rERn
> O.
is called a ray of
X is a ray-like set if
every element x of X is also a ray of X.
X an antipolar set,
Lemma 3
Ruys
[13]
is a raylike set.
does not contain the
PROOF:
If X=¢,
then X'
is the
origin,
then X'=Rn,
calls
such an
calls X auerole-reflexive).
(see also McLinden [11],
antipolar X'
(McLinden [11]
p.
176).
For any set X c Rn,
If X is closed,
raylike,
and
= X.
then X"
which
convex,
its
is raylike,
and X"=X.
If X,
intersection of a family of closed halfspaces, and
is closed and convex.
If yX', then eyEX'
for all e
1, and
>
so
so X'
is
raylike.
We now must show that
set that does not contain
if X is a nonempty closed,
the origin,
then X"=X.
yTx > 1 for all yX', whereby xX", and
Then there
exists an element z of X"
Because X is closed and
(z)
strictly separates
with the property
then by
and so
convex,
from X,
Let xeX.
so X c X".
that
Suppose XX".
(y,a) E (Rn,
for all xX, and yTz < a.
yTz > 1=a, a contradiction.
This
Thus a < O.
Also yTz < a
Because OX and X is nonempty, X'
yTx
> 0 for every xX, (y + ey)Tx
Therefore
(y + ey)
(y + ey)Tz >
yTz < a
is
for every e >
1 for all e > 0,
.
and so yTz
> 0,
Because
for every xX.
This
in
turn implies
that
> O,
which contradicts
[X]
3 implies that the
dual of NGD
closed, and X is a closed,
contain the origin.
yX',
is nonempty.
< O.
Lemma
f(-)
EX'
> 0,
for all xX.
yTx > 1 for every xX.
> 1 for any
R),
Because X is raylike,
xX and all e > 1, and hence yTx > 0
Then,
If a
being the case,
yT(ex) > a for all
Let y be any element of X'.
Then
there exists a hyperplane that
rescaling we can assume that a=1.
< O.
raylike
is not contained in X.
and so there exists
that yTx > a
convex,
Of
course,
convex,
if X does
17
is precisely NGP whenever
raylike
set that does not
contain the origin,
then
li
z*=O,
X'-$,
and v*=,
primal and dual,
where
z* and v*
are the optimal values of the
respectively.
The following result is analogous to Theorem 1:
Theorem 1A (Weak Duality).
Let
NGP and NGD,
Then
(i)
respectively.
z* and v*
be optimal
values for
If x and y are strongly feasible for NGP and NGD, with
objective values z and v,
hence
respectively, the zv > 1, and
z*v* > 1.
(ii)
If z*=O,
then NGD is essentially infeasible,
(iii)
If v*=O,
then NGP
(iv)
If x and y are strongly feasible
is
essentially infeasible,
objective values
z and v,
then x and y are
optimal
i.e.
v*=+".
i.e.
z*=+o
for NGP and NGD with
respectively,
and zv =
1,
solutions of NGP and NGD,
respectively.
PROOF:
If x and y are strongly feasible
1 < yTx < fo(y) f(x) =
(ii),
(iii),
and
(iv)
zv,
by Remark 3.
follow from
To see how to obtain the
from NGP and NGD,
X =
let
a raylike
satisfying bTX
Y = (yERnly=MTX for some X > 0
(i),
The
(xeRnjMx > b}
> be for some
Furthermore,
> 1,
satisfying bTX
and
and
> 1}.
X'
=
and define
=
1),
and note that X'
linearly constrained gauge program P
is equivalent to the program
P:
Define X =
> 1) = (ERnMx
raylike and contains Y.
result shows
[X]
set that contains X.
(yERnly=MTX for some X > 0
is
This
then
linearly constrained problems P and D
P be as given.
(xeRnlxEeX for some
Then X is
(i).
for NGP and NGD,
minimize f(x)
subject xEX
18
because even though the feasible region of P contains the feasible
region of P, every point xEX has at least as large an objective
function value as a corresponding point xX.
The nonlinear gauge dual
of P is
D:
minimize fO(MT\)
subject to bTX > 1
>0
However, because f(.) is
homogeneous, we can restrict our .attention
to those X > 0 for which bTX = 1, obtaining the equivalent program:
minimize fo(MTX)
D:
to bTX = 1
subject
>
0
linear gauge dual.
which is the
We have the following strong duality result for the nonlinear
Let E = {xERnlf(x) <
case which is an extension of Lemma 2.
CO =
{yeRnlfO(y)
< o).
Given dual gauge programs NGP and NGD, where f(.) is
Theorem 2A.
closed and X is closed, convex, and raylike,
values of NGP and NGD, respectively.
and (rel
} and
int X') n (rel
int
°
O)
#
*,
let z* and v* be optimal
If (rel int X) n (rel int
) # 0
then z*v* = 1, and each program
attains its optimum.
PROOF:
(rel int
Theorem
Let xo
O).
1A,
e
(rel int X) n (rel int C) and y
(rel int X') n
Then both NGP and NGD are strongly feasible, and by
0
feasible xX.
< z*
<
.
Suppose that z*
Then z*C n X =
separated by a hyperplane H.
is not attained by any
, and so z*C and X can be properly
Thus, there exists yERn and aR such
that yTx > a for all xX, and yTx < a for all XEz*C.
If a > 0,
then we can presume a = 1 by rescaling if necessary.
Then yX'
19
III
and v*
f(y) =
<
0,
Thus
then a
yTxo =
0.
+
x
we can demonstrate
ensures
rel
E
int X,
0 for
all
and so
v*
X.
E
0.
whereby yTxo
+
(1-6)x) > 0, which
Similarly,
0 for all XEX.
xz*C, and hence for all xC.
properly separate X from z*C.
that a > 0,
for NGD.
for every xX, there exists
turn means yTx =
that yTx =
l/z*.
is optimal
E z*C,
Thus yT(6x
X.
is attained
is attained in the primal,
If z*
so
(z*/f(xO))xO
(1-6)x E
This in
< 0.
Thus H does not
xO
Because
6 > 1 such that
implies yTx
C,
rel int
1 and y
=
Then ytxO > 0 because xO
= 0, because OEz*C.
because x
Also,
sup yTx < a/z*
XEZ*C
(/z*)
since vz* > 1, we have z*v* =
However,
If a
sup yTx =
xEC
in
This contradiction
the dual and
then the above proof
z*v* =
1.
is still valid,
long as z*C and X can be properly separated by a hyperplane H.
z*C and X cannot be properly separated,
appendix,
there
exists XE
(rel
int
X)n(rel
int z*C and OEz*C, there exists
xE rel
whereby x
(z*/6)C, and so f(x)
then by Theorem 6 of
can be properly separated, and so v*
6 > 1 such that
is attained in
the
Because
int z*C).
a contradiction.
< z*,
If
6x + (1-6)OEz*C,
Thus z*C and X
the dual,
and
z*v*=1.
A parallel
primal.
argument
establishes that z*
is attained in
the
[X]
The nonlinear gauge duality theory parallels the gauge duality
theory for linear constraints.
there
It
is only natural
then to examine if
is a parallel duality construction that extends the Lagrange-
type dual LD to handle nonlinear constraints.
and does not
contain the origin,
is closed,
NGP can be written as
f(x)
minimize
subject
If X
to
yTx
> g(y)
20
for all y E cone X',
convex,
where cone X'
g(y)
= max
=
{yeRnly = aw for some a > 0 and wX') and
(a > Oy = aw for
The above program
some wX').
suitable format so that Gwinner's dual
[7]
is in a
can be constructed, which
is:
g(y)
maximize
GwD:
subject
yTx < f(x) for all x
to
y E cone X'
Because
be
if f(y)
yTx < f(x) for all x if and only
<
1, program GwD can
transformed into
NLD:
maximize
y,a
a
subject to
fO(ay)
1
yEX'
which we define as the program NLD.
To see that NLD and GwD are
identical, notice that for any feasible solution (y,a) to NLD,
y'
=
(/g(y))y
For every feasible
wEX',
and y =
The
for GwD with identical objective
is feasible
solution y'
(1/d(w))y',
a
=
of
GwD, y'
d(w)
value.
= Bw for some B > 0 and
is feasible for NLD.
dual nonlinear gauge programs NGP and NGD make up a neat
theory in terms of
polar
explicit dual variables
constraints
functions and antipolar sets.
However,
the
y do not directly correspond to primal
(though they do indirectly),
mention of constraints per se
and there is no formal
in either the primal
special
case of the general nonlinear theory is
theory,
instances
of which will be seen
or the dual.
One
of course the linear
in the next
section.
An open
issue regarding nonlinear gauge duality is under what circumstances
can the
nonlinear gauge dual program be written
a finite number of
constraints whose
for every primal constraint?
21
variables
explicitly in terms of
include a multiplier
li
4.
Applications
In this section we explore a number of mathematical
models that correspond to a gauge program,
programming,
problems
involving the
Many of the applications will
minimize
x
subject
where f(-)
that of P.
However,
including convex quadratic
lp norm,
involve programs
and
linear programming.
of the form:
f(Nx+d)
to
is a gauge.
programming
Ax > b
Note that this format does not conform to
it
is equivalent to:
g(x,s)
minimize
x,s
subject to
where g(x,s)
= f(s).
Ax
-Nx + Is
Furthermore,
> b
= d
f(-)
qualification if and only if g(.,.)
satisfies
does.
the projection
The gauge dual
of this
program is:
minimize
g(ATX-NTU,
)
X,U
subject to
+ dT
bTX
X>
But gO(y,t)
= f(t) when y=O,
U
=
and gO(y,t)
last program becomes
minimize
subject to
fO(u)
subject
ATX-NTU
bTX+dTU
to
X>
22
1
0
0
= 0
= 1
= +
for y #
O;
thus this
A.
Convex Quadratic Programming
The standard convex quadratic program
QP:
minimize
subject
(1/2)xTQx
is given by
+ qTx
Ax > b
to
where Q is a symmetric positive semi-definite matrix.
into the form Q = MTM for some square
the matrix Q can be factored
matrix M.
lies
can
If M
is nonsingular
in the row space of
M,
(i.e.,
subject
which
is positive definite),
then q = MTs for some vector sERn,
or
if q
and QP
(1/2)xTMTMx + sTMx
to
Ax > b
is equivalent to the gauge
GP:
minimize
subject
where f(.)
= #'U 2 ,
program
lMx+sU 2
to
Ax > b
and so f(.)
satisfies
Note that QP and GP are equivalent
identical
and their objective
transformation.
the projection qualification.
in that their
constraints are
functions differ by a strictly monotone
In examining the duality properties of QP and GP,
first study the case where Q is
the case when Q is positive
Q is
Q
be written as
minimize
will
Furthermore,
positive definite,
followed by
semi-definite.
Positive Definite
When Q is positive definite the Lagrange dual
bTX -
maximize
X
of QP
is
(1/2)(XTA-qT)Q-1(ATX-q)
(LQP),
subject
to
see Dorn [1].
X
> 0
The guage dual
11(MT)-IATX1{
minimize
of GP,
on the other hand,
is
2
(GD),
subject
to
(bT+qTQ-lAT)X
X > 0
= 1
23
we
II'
to
is equivalent
which
the quadratic program
minimize
XTAQ-1ATX
subject to
(bT+qTQ-1AT)X = 1
X >O
(D).
Note that both LQP and GD are strictly convex quadratic programs.
constraints of LQP consist of the nonnegativity conditions
X > 0, whereas
the single equality constraint (bT+qTQ-1AT)X = 1.
GD also includes
The
The
following theorem shows
the relationship between the gauge dual
programs GD or GD and the Lagrange dual LQP:
Theorem 3.
(i)
is a solution to
If
*
a)
t*
0
dual
LQP,
b)
(ii)
is positive definite,
If Q
t*
GD and
t* = X*TAQ-1ATX*,
if and only if X = X*/t* solves
t
0 if and
only if
\*
then
the Lagrange
infeasible.
= 0 if and only if QP is
is a solution to LQP and t = bTX
If
a)
then
+ qTQ-1ATX, then
= X/t solves
the guage
dual
GD,
b)
t = 0 if and only if QP has a solution x to Ax > b
satisfying Mx + s = O,
i.e.,
if and only if GD is
infeasible.
PROOF:
The proof follows
Tucker conditions
The
for GD and LQP.
direct substitution.
Q
from an examination of the Karush-Kuhntransformations follow from
[X]
is Positive Semi-Definite
We now turn our attention to the broader case,
semi-definite and q lies
some vector sRn.
in the row space
In this case,
of M,
whence q = MTs for
the Lagrange dual
24
when Q is positive
of QP
is
bTX -
maximize
(1/2)xTMTMx
',x
(LQP'),
subject
to
as in Dorn [1].
ATX -
MTMx = MTs
X> O
The gauge dual of GP is
minimize
X,u
null 2
subject to
-ATX + MTu =
bTX
+ sTU
=
0
(GD'),
1
X > 0
which is equivalent to the quadratic program
minimize
uTu
X, u
subject
-ATX + MTU = 0
to
bTX
+
sTU
=
(GD')
1
X > 0
Analogous to Theorem 3, Theorem 4 demonstrates the relationship
between the two different dual quadratic programs LQP' and GD'.
Theorem 4.
If Q is positive semi-definite, Q = MTM and q = MTs for
some s ERn,
then
(i)
(X*,u*)
constitute an optimal solution to the gauge dual
GD' if and only if there exists x*, t* such that
(ii)
(a)
-ATX*+MTU*=O
(b)
bTX*+sTu*=l
(c)
X*
(d)
Ax* > bt*
(e)
U*=Mx*+st*
(f)
X*TAx*=X*Tbt*
> 0
t*=O if and only if QP is infeasible.
only if
t$*O if and
=X*/t*, x=x*/t* constitute a solution to the
Lagrange dual LQP'.
25
III
(iii)
(X,x) constitute an optimal solution to the Lagrange
dual LQP'
if and only if there exists z such that
(a)
X > 0
(b)
AT\-MTM=MTs
(c)
Az > b
(d)
MTMx=MTMz
(e)
XTAz=XTb
XTb+sTs+sTMz=o if and only if QP has a solution Ax > b,
(iv)
Mx+s=O, i.e.
if and only if GD' is infeasible.
XTb+sTs+sTMz~o if and only if (X*,U*) solves the gauge
dual
t*
D'
with multipliers x*,
t*
given by:
= 1/(XTb+sTs+sTMz)
X* = Xt*
U*
=
x*
= zt*.
The conditions
(Mz+s)t*
[X]
(i) and (iii)
of this theorem are simply the Karush-
Kuhn-Tucker conditions, and the transformations in
(ii)
and (iv)
follow from direct substitution.
B.
Programs with the l
If f(x)
=
xllp,
Norm
1 < p <
.
then the polar of f(.)
YUq, where q must satisfy 1/p+l/q = 1.
is
In particular, p=1 or p=
and only if q=* or q=1, respectively.
Consider the
lp norm program:
minimize
xt
IwUp
subject to
Bw+Cz > d
(GPp)
26
f(y) =
if
The gauge dual
of GPp, derived using P and D,
nBTXq
minimize
subject
is
to
CTX
=
0-
dTX
=
1
(GDq)
X > 0
Because f(x)
results
of Theorem
Note
=
Hxlp satisfies the projection qualification,
2 are valid
solution s),
which is a derived equivalent of
i-I
instance of GPp, by setting
C
M
is a more general
When pl, p,
, d =
s
program than QP.
(l/p)
p
w
X,Z
The Lagrange
and p = 2.
the program GPp is equivalent to
minimize
subject
the
(when Q is positive semi-definite and q=MTs has a
can be cast as an
B
Thus GPp
for GPp and GDq.
that the program GP,
quadratic program QP
the
(LPp)
to
dual of
Bw+Cz >
this
d
program
is
maximize XTd-(1/q)UBTn q
q
subject
to
= 0
CT
(LDq)
X> 0
The gauge dual GDq and the Lagrange dual LDq bear a relationship
that generalizes the case of quadratic
4.
This
programming in Theorems 3 and
relationship is demonstrated below.
notation xP,
where xERn,
In the theorem,
denotes the vector whose
(sign xj)(Ixjlp).
27
the
jth component
is
If pl and p,
Theorem 5.
If X*
(i)
t*
then
is an optimal
solution
to the gauge
dual GDq and
= \*T(B)(BTX*)q-1, then
(a)
t'*O if
and only if
=X*/t*
solves the Lagrange
dual LDq.
(b)
t*=O if
and only if GPp
If X is an optimal
(ii)
(a)
XTd~O
is infeasible.
solution to
the Lagrange dual LDq, then
if and only if X*=X/XTd solves the gauge
dual GDq.
(b)
XTd=o
if and only if GPp has an optimal
with value
The proof of
this
0,
Cz > d has a solution.
i.e.
theorem follows
solution
[X]
from examining the Karush-Kuhn-
Tucker conditions and substituting in the transformations as given.
Although
the programs GPp and LPp are equivalent
objective functions differ by a monotone
dual GDq of GPp will yield a better
the
optimal
To see this,
LPp, and
let
let
(w,z)
and
in GPp and LPp,
X = X/(dTX)
h =
dTX -
values of
X and X in
the gauge
lower bound on
the Lagrange dual
LDq for LPp.
to GPp and
be the corresponding objective values of
=
wllp,
' =
(l/p)
P
Uwlp
Let X >
.
(w,z)
0 be any
LDq with a positive objective value, and hence
is a feasible solution to GDq.
(l/q)JJBTx\
larger)
be any strongly feasible solution
namely
feasible solution to
transformation),
(i.e.,
solution to GPp than will
(their
p
p
Let g =
be the corresponding objective
BTXUq, and
function
the programs GDq and LDq, respectively.
Then
1/g and h each represent a lower bound on the optimal primal
objective values for GPp and LPp, and the values
numbers
greater than or equal
the respective dual pairs
g and
/h are
to one that measure the duality gap in
of programs,
28
as a ratio
of the primal
objective
function value
to the dual objective
function value.
We
have:
Lemma
This
3.
Under
the above assumptions,
lemma states
ratio)
<
/h.
that the corresponding duality gaps
(measured as a
is always smaller for the gauge dual GDq than for
Lagrange dual
inasmuch as
monotone
LDq.
Of course,
transformation.
than does
the comparison is somewhat unfair,
Yet the proof below shows
that the gauge
intrinsically better convex inequality
the Lagrange dual LDq.
Proof of Lemma 3:
< XTBw + XTCz
the last
the
the objective functions of GPp and LPp differ by a
dual GDq in a sense uses an
XTd
g
Let w,
=
,
z,
TBw <
, g,
wNUp
h, X,
BTXHq
<
and X be as
(/p)Nw
inequality being an instance of the
arithmetic and the geometric mean.
nonnegative gaps
in
the three
Let
p
+
stated.
Then
(l/q)fBTX\q
inequality between the
s, t,
and u represent the
inequalities above,
respectively.
Then we have
dTX+s+t+u
,g
=
I[w[[p
BTXHq/(dtX)
s+t+u
1 +
=
dTX
dTX
Therefore
s+t+u
g =
Not
1
s+t+u
<
e
the geometric
1 ++
=
-
.
[X]
Note
is
that
the
it
inequalidT-(/)y between
BThe arithmetic mean and
an ives
that
itt
mean that
drives
he rbetween
the result.
29
the arithetic
ean and
i
C.
Linear Programming
As a final
LP:
note, observe
minimize
that the linear programming problem:
cTx
subject to
> b
Ax
can be formulated as a gauge program when
positive.
and C =
In this case,
1)
(xERnlf(x)
to compute C
=
= max
let f(x)
= (xERnlcTx <
yERnly = cv,
v > 0),
the optimal
(cTx,O).
1).
It
value
Then f(-)
of
LP
is
is a gauge
is then straightforward
and
v
if y=cv for some v > 0
+~
else
fo(y) =
The gauge dual
DLP':
of LP then
minimize
X,v
ATX -
bTX
X
is
v
cv
= 0
= 1
>
O
which is equivalent to the standard linear program dual:
DLP:
maximize
X
subject to
bTX
ATX = c
Acknowledgment
I am very grateful to Professor Thomas Magnanti
comments and careful
for his valuable
reading of a draft of this paper.
30
Appendix:
Separation Theorems
Given two convex sets X and Y
in R n , X
is properly separated
from Y by a hyperplane H provided X and Y lie in
the opposite closed
halfspaces bounded by H,
lie in H.
following theorem
and X and Y do not both
of Fenchel
[3]
The
characterizes when X and Y can be
properly separated:
Theorem 6
(Fenchel [3],
nonempty convex sets
see Rockafellar
in Rn.
hyperplane H if and only if
Let X and Y be
X and Y can be properly separated by a
(rel
If X and Y are convex sets
Y by a hyperplane H provided X
bounded by H and Y lies
[12]).
int X)
in Rn, X
(rel int
n
is
=
Y)
.
[X]
openly separated from
lies in one of the open halfspaces
in the other closed halfspace.
Clearly,
if
X is openly separated
from Y,
Klee's
give criteria on X and Y that are sufficient for
results of
[8]
then X and Y are properly separated.
X to be openly separated from Y by some hyperplane H.
Some of these
criteria are given below.
A convex set XcRn
is
is
called evenly convex [4]
provided that X
the intersection of a family of open halfspaces.
A set ZRn
is
called an asymptote of a convex set YcRn provided that Z is an
affine variety, ZnY =
ycRn is
,
and
inf
({z-yH
said to be boundedly polyhedral if
its
bounded polyhedron is a bounded polyhedron.
[8]
is
yY) =
zEZ,
O.
The set
intersection with any
One of Klee's results
the following:
Theorem 7
(Klee [8])
If X and Y are disjoint convex subsets of Rn,
then X can be openly separated from Y if X's
evenly convex,
polyhedral.
projections are all
and Y admits no asymptote and Y is boundedly
[X]
31
________
in
III
Actually,
Klee's results
are much broader
that the stated conditions are maximal
precisely, and
he also gives
than indicated.
He shows
in a sense he defines
five alternative criteria
that guarantee
that X can be openly separated from Y.
Remark 5.
If X and Y are disjoint convex sets and all projections
X are closed,
such
and Y is a polyhedron, then there exists yERn and aER
that yTx < a for
PROOF:
all xX and yTx >
Any closed convex set
for all xY.
is evenly convex,
since any closed
convex set
is equal to the intersection of the family of the closed
halfspaces
that contain it,
halfspace
is
halfspaces.
of X are
of
see Rockafellar
the intersection
Thus,
of an
conditions of Theorem
of X are closed,
If Y is a polyhedron,
polyhedral, and admits no asymptote.
all projections
then Y is
boundedly
Thus X and Y satisfy the
7, whereby the desired
32
and each closed
infinite family of open
if all projections
evenly convex.
[12],
result is obtained.
[X].
References
[1]
W.S. Dorn, "Duality in quadratic
Math. 18, 155-162, 1960.
programming,"
Quart. Appl.
[2]
E. Eisenberg, "Duality in homogeneous programming," Proceedings
of the American Mathematical Society 12, 783-787 (1961).
(3]
W. Fenchel, "Convex cones, sets, and
Princeton University, 1951.
[4]
W. Fenchel, "A remark on convex sets and polarity," Comm.
Math. Univ. Lund. (Medd. Lunds Univ. Math. Sem.) Tome
Supplementaire, 1952, 88-89.
[5]
D.R. Fulkerson, "Blocking and anti-blocking pairs of polyhedra,"
Mathematical Programming 1, 168-194 (1971).
[6]
C.R. Glassey, "Explicit duality for convex homogeneous
programs," Mathematical Programming 10, 176-191 (1976).
(7]
J. Gwinner, "An Extension lemma and homogeneous programming,"
J. Opt. Theory and Applications 47, 321-336 (1985).
[8]
V. Klee, "Maximal separation theorems for convex sets,"
Transactions of the American Mathematical Society, 134
1,
functions,"
lecture notes,
Sem.
1968.
(9]
D.G. Luenberger, Optimization by vector space
Wiley & Sons (New York, 1969).
[10]
T.L. Magnanti, "Fenchel and Lagrange duality are equivalent,"
Mathematical Programming 7, 253-258 (1974).
(11]
L. McLinden, "Symmetric duality for structured convex
programs," Trans. Amer. Math. Soc. 245, 147-181 (1978).
[12] R.T. Rockafellar, Convex analysis,
(Princeton, New Jersey, 1970).
[13]
Princeton
methods, John
University Press
P.H. Ruys and H.M. Weddepohl, "Economic theory and duality,"
in: J. Kriens, ed., Convex analysis and mathematical economics,
Tilburg 1978, Springer (Berlin, 1979).
33