Linear Programming
IRIS H.-R. JIANG
3
¨
Linear programming describes a broad class of optimization
tasks in which both the optimization criterion and the
constraints are linear functions.
¨
Linear programming consists of three parts:
¤ A set of decision variables
¤ An objective function:
n maximize or minimize a given linear objective function
¤ A set of constraints:
n satisfy a set of linear inequalities involving these variables
Linear Programming
Iris Hui-Ru Jiang
Fall 2014
Linear programming
Linear Programming
Example: Profit Maximization (1/4)
IRIS H.-R. JIANG
2
¨
Course contents:
¤
¤
¤
¤
¨
Linear programming
Formulation
Duality
The simplex method
IRIS H.-R. JIANG
4
¨
¨
Reading:
¤
¤
Chapter 7 (Dasgupta)
Chapter 29 (Cormen)
¨
¨
Linear programming
A boutique chocolatier has two products:
¤ A (Pyramide): profit $1 per box
¤ B (Nuit): profit $6 per box
Constraints:
¤ The daily demand for these exclusive chocolates is limited to at
most 200 boxes of A and 300 boxes of B
¤ The current workforce can produce a total of at most 400 boxes
of chocolate per day
Decision variables:
¤ x1 = Boxes of A
¤ x2 = Boxes of B
Objective Function:
¤ Maximize profit
Linear programming
Example: Profit Maximization (2/4)
Example: Profit Maximization (4/4)
IRIS H.-R. JIANG
5
¨
¨
¨
IRIS H.-R. JIANG
7
A linear equation in x1 and x2 defines a line in the 2D plane
A linear inequality designates a half-space
The set of all feasible solutions of this linear program is the
intersection of five half-spaces. It is a convex polygon
¨
The Simplex method: hill climbing
¤ George Dantzig, 1947
¤ Starts at a vertex, say (0, 0)
¤ Repeatedly looks for an adjacent vertex (connected by an edge
of the feasible region) of better objective value
¤ Upon reaching a vertex that has no better neighbor, simplex
declares it to be optimal and halts
(100, 300)
Linear programming
Linear programming
Example: Profit Maximization (3/4)
Multipliers?
IRIS H.-R. JIANG
6
¨
IRIS H.-R. JIANG
8
Search for the optimal solution
¤ It is a general rule of linear programs that the optimum is
achieved at a vertex of the feasible region.
max x1 + 6x2
x1 <= 200 (1)
x2 <= 300 (2)
x1 + x2 <= 400 (3)
x1 , x2 >= 0.
¨
Optimal: (x1, x2) = (100, 300); objective value = 1900
¨
Can this answer be checked somehow?
¤ (1) + 6*(2):
x1 + 6x2 <= 2000
¤ 0*(1) + 5*(2) + (3):
x1 + 6x2 <= 1900
¤ The multipliers (0, 5, 1) constitute a certificate of optimality
¤ How would we systematically find the magic multipliers?
(100, 300)
Linear programming
Linear programming
Duality (1/3)
Duality (3/3)
IRIS H.-R. JIANG
9
¨
Multipliers yi’s must be nonnegative
¨
If the left-hand side to look like our objective function, the righthand side is an upper bound on the optimum solution
¨
IRIS H.-R. JIANG
11
max x1 + 6x2
x1 <= 200 (1)
x2 <= 300 (2)
x1 + x2 <= 400 (3)
x1 , x2 >= 0.
¨
Generic form:
¨
Dual theorem: If a linear program has a bounded optimum, then
so does its dual, and the new optimum values coincide.
¤ Max-flow min-cut
We want a tight bound!
Linear programming
Linear programming
Duality (2/3)
The Simplex Algorithm
IRIS H.-R. JIANG
10
¨
IRIS H.-R. JIANG
12
A new LP: finding multipliers that gives the best upper bound
on our original LP
¤ Dual LP
¤ Primal LP
¤
¤
Any feasible value of dual LP is an upper bound on primal LP
If we find a pair of primal and dual feasible values that are equal,
they must be both optimal.
Every constraint specifies an
n-dimensional half-space
Linear programming
Linear programming
Travel along “edges” until no
improvement can be made
Vertex and Neighbors
Example (1/3)
IRIS H.-R. JIANG
13
¨
¨
IRIS H.-R. JIANG
15
Pick a subset of the inequalities. If there is
a unique point that satisfies them with equality,
and this point happens to be feasible,
then it is a vertex
¤ {2, 3, 7} ® A
¤ {4, 6} ® no vertex
Two vertices are neighbors if they have
n - 1 defining inequalities in common
¤ {2, 3, 7} ® A
¤ {1, 3, 7} ® C
Linear programming
Linear programming
The Simplex Algorithm
Example (2/3)
IRIS H.-R. JIANG
14
¨
¨
¨
¨
IRIS H.-R. JIANG
16
On each iteration, simplex has two tasks:
¤ Task 1: Check whether the current vertex is optimal
¤ Task 2: Determine where to move next
Both tasks are easy if the vertex happens to be at the origin
¤ Transform the coordinate system to move vertex u to the origin
Task 1:
¤ The origin is optimal if and only if all ci <= 0
Task 2:
¤ We can move by increasing some xi for which ci > 0
¤ Until we hit some other constraint
Linear programming
Linear programming
Example (3/3)
IRIS H.-R. JIANG
17
Linear programming
Standard Form
IRIS H.-R. JIANG
18
¨
¨
Variants
¤ Either a maximization or a minimization problem
¤ Constraints can be equations and/or inequalities
¤ Variables are restricted to be nonnegative or unrestricted in sign
Standard form
¤ Objective function: minimization
¤ Constraints: equations
¤ Variables: nonnegative
Slack variables
Linear programming