6.252 NONLINEAR PROGRAMMING LECTURE 18: DUALITY THEORY LECTURE OUTLINE • Geometrical Framework for Duality • Lagrange Multipliers • The Dual Problem • Properties of the Dual Function • Consider the problem minimize f (x) subject to x ∈ X, gj (x) ≤ 0, j = 1, . . . , r, assuming −∞ < f ∗ < ∞. • We assume that the problem is feasible and the cost is bounded from below, −∞ < f ∗ = inf x∈X gj (x)≤0, j=1,...,r f (x) < ∞ MIN COMMON POINT/MAX INTERCEPT POINT • Let S be a subset of n : • Min Common Point Problem : Among all points that are common to both S and the nth axis,find the one whose nth component is minimum. • Max Intercept Point Problem : Among all hyperplanes that intersect the nth axis and support the set S from “below”, find the hyperplane for which point of intercept with the nth axis is maximum. Min Common Point S S 0 Max Intercept Point Min Common Point 0 Max Intercept Point (a) (b) GEOMETRICAL DEFINITION OF A L-MULTIPLIER • A vector µ∗ = (µ∗1 , . . . , µ∗r ) is said multiplier for the primal problem if µ ∗j ≥ 0, and to be a Lagrange j = 1, . . . , r, f ∗ = inf L(x, µ∗ ). x∈X w w S = {(g(x),f(x)) | x ∈ X} S = {(g(x),f(x)) | x ∈ X} (µ*,1) (µ*,1) (0,f*) 0 (0,f*) z H = {(z,w) | f* = w + Σ j µ*j zj} (a) 0 Set of pairs (z,w) corresponding to x that minimize L(x,µ*) over X (b) z EXAMPLES: A L-MULTIPLIER EXISTS (µ*,1) (0,1) min f(x) = x1 - x2 S = {(g(x),f(x)) | x ∈ X} (-1,0) s.t. g(x) = x1 + x2 - 1 ≤ 0 x ∈ X = {(x1 ,x2 ) | x1 ≥ 0, x2 ≥ 0} 0 (a) (0,-1) (µ*,1) S = {(g(x),f(x)) | x ∈ X} (-1,0) min f(x) = (1/2) (x12 + x22) s.t. g(x) = x1 - 1 ≤ 0 x ∈ X = R2 0 (b) (µ*,1) (µ*,1) min f(x) = |x1| + x2 (µ*,1) S = {(g(x),f(x)) | x ∈ X} 0 (c) s.t. g(x) = x1 ≤ 0 x ∈ X = {(x1 ,x2 ) | x2 ≥ 0} EXAMPLES: A L-MULTIPLIER DOESN’T EXIST S = {(g(x),f(x)) | x ∈ X} min f(x) = x s.t. g(x) = x2 ≤ 0 x ∈X = R (0,f*) = (0,0) (a) S = {(g(x),f(x)) | x ∈ X} (-1/2,0) min f(x) = - x s.t. g(x) = x - 1/2 ≤ 0 x ∈ X = {0,1} (0,f*) = (0,0) (b) (1/2,-1) Proposition: Let µ∗ be a Lagrange multiplier. Then x∗ is a global minimum of the primal problem if and only if x∗ is feasible and • x∗ = arg min L(x, µ∗ ), x∈X µj∗ gj (x∗ ) = 0, j = 1, . . . , r THE DUAL FUNCTION AND THE DUAL PROBLEM • The dual problem is maximize q(µ) subject to µ ≥ 0, where q is the dual function q(µ) = inf L(x, µ), x∈X ∀ µ ∈ r . • Question: How does the optimal dual value q ∗ = supµ≥0 q(µ) relate to f ∗ ? (µ,1) S = {(g(x),f(x)) | x ∈ X} Optimal Dual Value Support points correspond to minimizers of L(x,µ) over X H = {(z,w) | w + µ'z = b} q(µ) = inf L(x,µ) x ∈X WEAK DUALITY • The domain of q is Dq = µ | q(µ) > −∞ . • q Proposition: The domain Dq is a convex set and is concave over Dq . • Proposition: (Weak Duality Theorem) We have q ∗ ≤ f ∗ . Proof: For all µ ≥ 0, and x ∈ X with g(x) ≤ 0, we have q(µ) = inf L(z, µ) ≤ f (x) + r z∈X µj gj (x) ≤ f (x), j=1 so q ∗ = sup q(µ) ≤ µ≥0 inf x∈X, g(x)≤0 f (x) = f ∗ . DUAL OPTIMAL SOLUTIONS AND L-MULTIPLIERS Proposition: (a) If q∗ = f ∗ , the set of Lagrange multipliers is equal to the set of optimal dual solutions. (b) If q∗ < f ∗ , the set of Lagrange multipliers is empty. • By definition, a vector µ∗ ≥ 0 is a Lagrange multiplier if and only if f ∗ = q(µ∗ ) ≤ q∗ , which by the weak duality theorem, holds if and only if there is no duality gap and µ∗ is a dual optimal solution. Proof: Q.E.D. q(µ) min f(x) = x s.t. g(x) = x2 ≤ 0 x ∈X = R f* = 0 - 1/(4 µ) if µ > 0 q(µ) = min {x + µx2} = - ∞ if µ ≤ 0 x∈R µ { (a) q(µ) min f(x) = - x f* = 0 s.t. g(x) = x - 1/2 ≤ 0 x ∈ X = {0,1} 1 µ q(µ) = min { - x + µ(x - 1/2)} = min{ - µ/2, µ/2 −1} x ∈ {0,1} - 1/2 -1 (b)