Karmarkar Algorithm
Anup Das, Aissan Dalvandi, Narmada Sambaturu, Seung Min,
and Le Tuan Anh
1
Contents
• Overview
• Projective transformation
• Orthogonal projection
• Complexity analysis
• Transformation to Karmarkar’s canonical form
2
LP Solutions
• Simplex
• Dantzig 1947
• Ellipsoid
• Kachian 1979
• Interior Point
• Affine Method 1967
• Log Barrier Method 1977
• Projective Method 1984
• Narendra Karmarkar form AT&T Bell Labs
3
Linear Programming
General LP
𝑇
Minimize 𝑞 𝑥
Subject to
• 𝑆𝑥 ≥ 𝑡
• 𝑥≥0
•
•
•
•
Karmarkar’s Canonical Form
Minimize 𝑐 𝑇 𝑥
Subject to
• 𝐴𝑥 = 0
• 𝑥𝑖 = 1
• 𝑥≥0
Minimum value of 𝑐 𝑇 𝑥 is 0
Ω ≡ 𝑥 𝐴𝑥 = 0} ≡ 𝑠𝑢𝑏𝑠𝑝𝑎𝑐𝑒 (𝑁𝑢𝑙𝑙 𝑠𝑝𝑎𝑐𝑒 𝑜𝑓 𝐴)
Δ ≡ {𝑥|𝑥 ≥ 0, 𝑥𝑖 = 1} ≡ simplex
π ≡ Ω ∩ Δ ≡ 𝑝𝑜𝑙𝑦𝑡𝑜𝑝𝑒
5
Karmarkar’s Algorithm
• Step 1: Take an initial point 𝑥 (𝑘) , 𝑘 = 0
• Step 2: While 𝑥 (𝑘) − 𝑥 (𝑘−1) ≤ 𝛾 𝑜𝑟 𝑐 𝑇 𝑥
𝑘
≤ 𝜀
• 2.1 Transformation: 𝑇: ∆ → ∆′ such that 𝑥′(𝑘) is center of∆′
• This gives us the LP problem in transformed space
• 2.2 Orthogonal projection and movement: Project 𝑐′ to feasible
region and move 𝑥′(𝑘) in the steepest descent direction.
• This gives us the point 𝑥′(𝑘+1)
• 2.3 Inverse transform: 𝑇 −1 : ∆′ → ∆
• This gives us 𝑥 (𝑘+1)
• 2.4 Use 𝑥 (𝑘+1) as the start point of the next iteration
• Set 𝑘 = 𝑘 + 1
6
Step 2.1
• Transformation: 𝑇: ∆→ ∆′ such that 𝑥′(𝑘) is center of ∆′ - Why?
𝑥 (𝑘)
𝑥′(𝑘)
𝑟
7
Step 2.2
• Projection and movement: Project 𝑐′ to feasible region and
move 𝑥′(𝑘) in the steepest descent direction - Why?
𝑐′
Projection of 𝑐′ to feasible region
8
−𝑐′
Big Picture
𝑥′(𝑘)
𝑥 (𝑘)
𝑥
(𝑘+1)
𝑥′(𝑘+1)
𝑥′′(𝑘+1)
𝑥 (𝑘+2)
𝑥′′(𝑘+2)
9
Karmarkar’s Algorithm
• Step 1: Take an initial point 𝑥 (𝑘) , 𝑘 = 0
• Step 2: While 𝑥 (𝑘) − 𝑥 (𝑘−1) ≤ 𝛾 𝑜𝑟 𝑐 𝑇 𝑥
𝑘
≤ 𝜀
• 2.1 Transformation: 𝑇: ∆→ ∆′ such that 𝑥′(𝑘) is center of ∆′
• This gives us the LP problem in transformed space
• 2.2 Orthogonal projection and movement: Project 𝑐′ to feasible
region and move 𝑥′(𝑘) in the steepest descent direction.
• This gives us the point 𝑥′(𝑘+1)
• 2.3 Inverse transform: 𝑇 −1 : ∆′ → ∆
• This gives us 𝑥 (𝑘+1)
• 2.4 Use 𝑥 (𝑘+1) as the start point of the next iteration
• Set 𝑘 = 𝑘 + 1
11
Projective Transformation
• Transformation: 𝑇: ∆→ ∆′ such that 𝑥′(𝑘) is center of∆′
Transform
𝑥′(𝑘)
𝑥 (𝑘)
(𝑘)
𝑥1
𝑥 (𝑘) =
(𝑘)
𝑥2
⋮
(𝑘)
𝑥𝑛
𝑥′(𝑘)
𝑒
=
𝑛
1
𝑛
1
𝑒
= 𝑛 =
𝑛
⋮
1
𝑛
𝑒 = 111 … 1
𝑇
12
Projective transformation
• We define the transform 𝑥′ = 𝑇𝑥 𝑘 (𝑥) such that
• 𝑥
𝑘
∈ ∆→
𝑒
𝑛
∈ ∆′
• We transform every other point with respect to point 𝑥
𝑘
Mathematically:
(𝑘)
𝑥1
• Defining D = diag (𝑥 (𝑘) ) = ⋮
0
• projective transformation:
• 𝑥 → 𝑥′
⋯
0
⋱
⋮
(𝑘)
⋯ 𝑥2
𝑥′ = 𝑇𝑥 (𝑘) 𝑥 =
• inverse transformation
• 𝑥′ → 𝑥
𝑥 = 𝑇𝑥 (𝑘) 𝑥′ =
𝐷−1 𝑥
𝑒 𝑇 𝐷−1 𝑥
𝐷𝑥′
𝑒 𝑇 𝐷𝑥′
13
Problem transformation:
• projective transformation:
• 𝑥 → 𝑥′
𝑥′ = 𝑇𝑥 (𝑘) 𝑥 =
• inverse transformation
• 𝑥′ → 𝑥
𝑥 = 𝑇𝑥 (𝑘) 𝑥′ =
Minimize 𝒄𝑻 𝒙
s.t.
𝑨𝒙 = 𝟎
𝒆𝑻𝒙 = 𝟏
𝒙 ≥ 𝟎
𝐷−1 𝑥
𝑒 𝑇 𝐷−1 𝑥
𝐷𝑥′
𝑒 𝑇 𝐷𝑥′
𝒄𝑻 𝑫𝒙′
𝒆𝑻 𝑫𝒙′
Transform
Minimize
S.t.
𝑨𝑫𝒙′ = 𝟎
𝒆𝑻𝒙′ = 𝟏
𝒙′ ≥ 𝟎
The new objective function is not a linear function.
14
Problem transformation:
𝒄𝑻 𝑫𝒙′
𝒆𝑻 𝑫𝒙′
Minimize
S.t.
𝑨𝑫𝒙′ = 𝟎
𝒆𝑻𝒙′ = 𝟏
𝒙′ ≥ 𝟎
Convert
Minimize 𝒄′𝑻 𝒙′
S.t.
𝑨′𝒙′ = 𝟎
𝒆𝑻𝒙′ = 𝟏
𝒙′ ≥ 𝟎
15
Karmarkar’s Algorithm
• Step 1: Take an initial point 𝑥 (𝑘) , 𝑘 = 0
• Step 2: While 𝑥 (𝑘) − 𝑥 (𝑘−1) ≤ 𝛾 𝑜𝑟 𝑐 𝑇 𝑥
𝑘
≤ 𝜀
• 2.1 Transformation: 𝑇: ∆→ ∆′ such that 𝑥′(𝑘) is center of ∆′
• This gives us the LP problem in transformed space
• 2.2 Orthogonal projection and movement: Project 𝑐′ to feasible
region and move 𝑥′(𝑘) in the steepest descent direction.
• This gives us the point 𝑥′(𝑘+1)
• 2.3 Inverse transform: 𝑇 −1 : ∆′ → ∆
• This gives us 𝑥 (𝑘+1)
• 2.4 Use 𝑥 (𝑘+1) as the start point of the next iteration
• Set 𝑘 = 𝑘 + 1
16
Orthogonal Projection
Minimize 𝒄′𝑻 𝒙′
S.t.
𝑨′𝒙′ = 𝟎
𝒆𝑻𝒙′ = 𝟏
𝒙′ ≥ 𝟎
• Projecting 𝑐′ onto
• 𝐴′ x ′ = 0 ⇒ projection on ℵ(𝐴′ )
• 𝑒 𝑇 𝑥 ′ = 1 ⇒ normalization
17
Orthogonal Projection
• Given a plane and a point outside the plane, which direction
should we move from the point to guarantee the fastest
descent towards the plane?
• Answer: perpendicular direction
• Consider a plane and a vector 𝑐′
𝑐′
ℵ(𝐴′ )
𝑐𝑃
𝑠
𝐴’
18
Orthogonal Projection
• Let 𝑎1 and 𝑎2 be the basis of the plane (𝐴’)
• The plane is spanned by the column space of 𝐴′ = [𝑎1 𝑎2 ]
• 𝑠 ∈ 𝐴′, so 𝑠 = 𝑦1 𝑎1 + 𝑦2 𝑎2 = 𝐴′ 𝑦
• 𝑐𝑃 = 𝑐′ − 𝑠 = 𝑐′ − 𝐴′ 𝑦
𝑐′
ℵ(𝐴′ )
𝑐𝑃
• 𝑐𝑃 is perpendicular to 𝑎1 and 𝑎2
• 𝐴′𝑇 𝑐′ − 𝐴′ 𝑦 = 0
𝑠
𝐴’
19
Orthogonal Projection
𝑐′
ℵ(𝐴′ )
𝑐𝑃
• 𝐴′𝑇 𝐴′ 𝑦 = 𝐴′𝑇 𝑐′
𝑇
• 𝑦 = (𝐴′ 𝐴′)−1 𝐴′𝑇 𝑐′
• 𝑠=
𝐴′ 𝑦
= 𝐴′
𝑇
𝐴′ 𝐴′
𝑠
−1
𝐴’
𝐴′𝑇 𝑐′ = 𝑃𝑐′
𝑇
• 𝑃 = 𝐴′(𝐴′ 𝐴′)−1 𝐴′𝑇 = projection matrix with respect to 𝐴’
• We need to consider the vector 𝑐𝑃 ∈ ℵ 𝐴′
• Remember 𝐴′𝑇 𝑐𝑃 = 0
• 𝑐𝑃 = 𝑐′ − 𝑠 = 𝐼 − 𝐴′ 𝐴′𝑇 𝐴′
−1
𝐴′𝑇 𝑐 ′ = 𝑃ℵ 𝑐 ′
• 𝑃ℵ is the projection matrix with respect to ℵ(𝐴′)
20
Orthogonal Projection
• What is the direction of 𝑐𝑃 ?
𝑐′
𝑐𝑃
• Steepest descent = −𝑐𝑃
𝑠
𝐴’
• We got the direction of the movement or projected 𝑐’ onto
ℵ(𝐴′ )
• Projection on
𝑒𝑇𝑥′
= 1 ⇒ 𝑐𝑃 =
𝑐𝑃
|𝑐𝑃 |
21
Calculate step size
Minimize 𝒄′𝑻 𝒙′
S.t.
𝑨′𝒙′ = 𝟎
𝒆𝑻𝒙′ = 𝟏
𝒙′ ≥ 𝟎
• r = radius of the incircle =
• 𝑥′(𝑘+1) =
𝑒
𝑛
1
𝑛(𝑛−1)
− 𝛼𝑟𝑐𝑃
22
Movement and inverse
transformation
Transform
𝑥 (𝑘)
𝑥 (𝑘+1)
Inverse
Transform
𝑥′(𝑘)
𝑒
=
𝑛
𝑥′(𝑘+1)
23
Big Picture
𝑥′(𝑘)
𝑥 (𝑘)
𝑥
(𝑘+1)
𝑥′(𝑘+1)
𝑥′′(𝑘+1)
𝑥 (𝑘+2)
𝑥′′(𝑘+2)
24
Matlab Demo
25
Contents
• Overview
• Projective transformation
• Orthogonal projection
• Complexity analysis
• Transformation to Karmarkar’s canonical form
26
Running Time
• Total complexity of iterative algorithm =
(# of iterations) x (operations in each iteration)
• We will prove that the # of iterations = O(nL)
• Operations in each iteration = O(n2.5)
• Therefore running time of Karmarkar’s algorithm = O(n3.5L)
# of iterations
• Let us calculate the change in objective function value at the end of
each iteration
• Objective function 𝑐 𝑇 𝑥 𝑘 changes upon transformation
• Therefore use Karmarkar’s potential function
𝑛
𝑓 𝑥 =
𝑗=1
𝑐𝑇𝑥
ln
𝑥𝑗
(𝑘)
𝑓 𝑥 (𝑘) =
ln 𝑐 𝑇 𝑥 (𝑘) − ln 𝑥𝑗
𝑗
(𝑘)
𝑓 𝑥 (𝑘) = 𝑛 ln 𝑐 𝑇 𝑥 (𝑘) −
ln 𝑥𝑗
𝑗
• The potential function is invariant on transformations (upto some
additive constants)
# of iterations
• Improvement in potential function is
𝑓(𝑥 (𝑘) ) − 𝑓(𝑥 (𝑘−1) )
(𝑘)
= n ln 𝑐 𝑇 𝑥 (𝑘) −
𝑙𝑛 𝑥𝑗
(𝑘−1)
− (𝑛 𝑙𝑛 𝑐 𝑇 𝑥 (𝑘−1) −
𝑗
𝑙𝑛 𝑥𝑗
)
𝑗
• Rearranging,
𝑓 𝑥
𝑘
−𝑓 𝑥
𝑘−1
= 𝑛 𝑙𝑛
𝑐 𝑇 𝑥 (𝑘)
𝑐 𝑇 𝑥 (𝑘−1)
(𝑘)
−
𝑥𝑗
𝑗 𝑙𝑛 (𝑥 (𝑘−1) )
# of iterations
• Since potential function is invariant on transformations, we can use
• 𝑓(𝑥 (𝑘) ) − 𝑓(𝑥 (𝑘−1) ) ≈ 𝑓(𝑥′(𝑘) ) − 𝑓(𝑥′(𝑘−1) )
where𝑥′(𝑘) and 𝑥′(𝑘−1) are points in the transformed space.
• At each step in the transformed space,
• 𝑥′(𝑘) = 𝑥′(𝑘−1) − 𝛼𝑟
• 𝑓 𝑥
𝑘
−𝑓 𝑥
𝑘−1
𝑐𝑝
|𝑐𝑝 |
= 𝑎0 − 𝛼𝑟
≈ 𝑛𝑙𝑛
𝑐𝑝
|𝑐𝑝 |
𝑐 𝑇 𝑎0 −𝛼𝑟
𝑐 𝑇 𝑎0
𝑐𝑝
𝑐𝑝
=
𝑎0 −𝛼𝑟
𝑗 ln(
𝑎0
𝑐𝑝
𝑐𝑝
)
# of iterations
• Simplifying, we get
• 𝑓 𝑥
𝑘
−𝑓 𝑥
𝑘−1
𝑐𝑝
≈ 𝑛𝑙𝑛 1 − 𝛽
𝑗 𝑐𝑗
−
𝑐𝑝𝑗
𝑗 ln(1 − 𝛽 |𝑐 |)
𝑝
Where β = 𝛼𝑟𝑛
• For small 𝑥 ,
ln 1 − 𝑥 ≈ −𝑥
• So suppose both β
𝑓 𝑥
𝑘
−𝑓 𝑥
𝑘−1
𝐜𝐏
𝒋 𝐜𝒋
and β
≈ −𝑛 β
𝐜𝐏𝒋
𝐜𝐏
are small
𝐜𝐏
𝒋 𝐜𝒋
jβ
cPj
cP
≈ −n β
cP
j cj
# of iterations
• It can be shown that
cP
j cj
R≥
≥r >
1
n
• The worst case for
𝑓 𝑥
𝑘
−𝑓 𝑥
𝑘−1
would be if β =
cP
≈ −n β
j cj
1
n
• In this case
𝑓 𝑥
𝑘
−𝑓 𝑥
𝑘−1
≈ −β
(β = α r n)
# of iterations
• Thus we have proved that
𝑘
𝑓 𝑥
−𝑓 𝑥
𝑘−1
≤ −δ
• Thus
𝑓 𝑥
𝑘
−𝑓 𝑥
0
≤ −kδ
where k is the number of iterations to converge
• Plugging in the definition of potential function,
𝑛
𝑗=1 ln
𝑐 𝑇 𝑥 (𝑘)
(𝑘)
𝑥𝑗
−
𝑛
𝑗=1 ln
𝑐 𝑇 𝑥 (0)
(0)
𝑥𝑗
≤ −𝑘𝛿
# of iterations
• Rearranging, we get
𝑙𝑛
•
•
𝑐 𝑇 𝑥 (𝑘)
𝑐 𝑇 𝑥 (𝑘)
𝑐𝑇𝑥
(0)
𝑐 𝑇 𝑥 (𝑘)
𝑐𝑇𝑥
≤ 𝑛 ln 𝑛 − 𝑘𝛿
(0)
𝑐𝑇 𝑥
(0)
𝑘
≤ 𝑒
𝑙𝑛 𝑛−𝑛𝛿
𝑘
≤𝑛𝑒
−𝑛𝛿
# of iterations
• Therefore there exists some constant 𝑘′ such that after
𝑛 𝑞+ln 𝑛
𝛿
𝑘 = 𝑘′(
−𝑘′(
𝑇 (𝑘)
𝑐 𝑥
(0)
𝑐𝑇𝑥
≤𝑛𝑒
𝑐 𝑇 𝑥 (𝑘)
(0)
𝑐 𝑇𝑥
) iterations,
𝑛 𝑞+𝑙𝑛 𝑛
)
𝛿
𝑛𝛿
≤ 𝑒 −𝑞 ≤ 2−𝑞 where 𝑞 = 𝛩(𝐿) , 𝐿 being the number of
bits
2−𝑞 is sufficiently small to cause the algorithm to converge.
• Thus the number of iterations is in O(𝑛𝐿)
Rank-one modification
• The computational effort is dominated by:
𝑐𝑝 = 𝐼 − 𝐴′𝑇 𝐴′𝐴′𝑇 −1 𝐴 𝐷𝑐
𝐴′ = 𝐴𝐷
• Substituting A’ in, we have
𝐴′ 𝐴′𝑇 = 𝐴𝐷 𝐴𝐷 𝑇 = 𝐴𝐷𝐷𝑇 𝐴𝑇 = 𝐴𝐷2 𝐴𝑇
• The only quantity changes from step to step is D (𝐷𝑖𝑖𝑘 = 𝑥𝑖𝑘 )
• Intuition:
• Let 𝐷 and 𝐷′ be 𝑛 × 𝑛 diagonal matrices
• If 𝐷 and 𝐷′ are close the calculation 𝐴𝐷 ′2 𝐴𝑇
𝐴𝐷 2 𝐴𝑇 −1 should be cheap!
−1
given
Method
• Define a diagonal matrix 𝐷′(𝑘) , a “working approximation” to
𝐷 𝑘 in step k such that
•
1
2
≤
′ 𝑘
𝐷𝑖𝑖
𝑘
𝐷𝑖𝑖
2
≤ 2 for 𝑖 = 1,2, … , 𝑛
• We update D’ by the following strategy
• 𝐷 ′ 𝑘+1 = 𝜎 𝑘 𝐷′ 𝑘
later slides)
• For 𝑖 = 1 to 𝑛 do
if
′ 𝑘
𝐷𝑖𝑖
𝑘
𝐷𝑖𝑖
Then let
(We will explain this scaling factor in
2
∉
1
,2
2
𝑘+1
𝐷𝑖𝑖′
=
𝑘+1
𝐷𝑖𝑖
and update 𝐴 𝐷’
𝑘+1
2 𝑇 −1
𝐴
Rank-one modification (cont)
• We have
•
•
•
•
−1 𝑢 𝑀 −1 𝑣 𝑇
𝑀
𝑀 + 𝑢𝑣 𝑇 −1 = 𝑀−1 −
1 + 𝑢𝑇 𝑀−1 𝑣
Given 𝑀−1 , computation of 𝑀 + 𝑢𝑣 𝑇 −1 can be done in 3𝑛2
or 𝑂 𝑛2 steps.
If 𝐷 and 𝐷’ differ only in 𝑖 𝑡ℎ entry then
𝐴𝐷2 𝐴𝑇 = 𝐴𝐷′2 𝐴𝑇 + 𝐷𝑖𝑖2 − 𝐷𝑖𝑖′2 𝑎𝑖 𝑎𝑖𝑇
=> rank one modification.
If 𝐷 and 𝐷’ differ in 𝑙 entries then complexity is 𝑂(𝑛2 𝑙).
Performance Analysis
• In each step: 𝑏 ′ − 𝑎𝑜 ≤ 𝛼𝑟
• Substituting 𝑏’ =
1
𝑟=
and 𝑇(𝑥) =
2
𝑘+1
𝑖
𝑥
𝑖
𝑎0 =
𝑛−1 𝑒
(𝑛(𝑛−1)
𝑥
•
𝑇(𝑥 (𝑘+1) ),
𝑘
𝑖
𝑘+1
𝑗
𝑗
𝑘
𝑥
𝑗
• Then
• Then
𝑥
𝑖
𝑖
𝑥𝑖
−
1
𝑛
≤
𝑘
∆𝑖
𝑘
Let call 𝜎 (𝑘) =
2
𝑘+1
𝑥𝑖 𝜎𝑘
𝛼2
𝑛 𝑛−1
−1
−1
2
≤
𝛽2
≤ 𝛽2
𝑘
Let ∆𝑖 =
𝑥𝑖
1
𝑛
𝑘+1
𝑘
𝑥𝑖 𝜎𝑘
𝑘+1
𝑥𝑗
𝑗
𝑘
𝑥𝑗
𝐷−1 𝑥
,
𝑒 𝑇 𝐷−1 𝑥
Performance analysis - 2
by a factor 𝜎 𝑘
𝐷′ 𝑘+1 = 𝜎 𝑘 𝐷′(𝑘)
• Then for any entry I such that
2
′ 𝑘+1
𝐷𝑖𝑖
1
∉
,2
𝑘+1
2
𝐷
• First we scale 𝐷’
• We reset 𝐷𝑖𝑖′
𝑘+1
𝑘
=
𝑖𝑖
𝑘+1
𝐷𝑖𝑖
• Define discrepancy between 𝐷 𝑘 and 𝐷’ 𝑘 as
′ 𝑘
𝐷
𝑘
𝛿𝑖 = 𝑙𝑛 𝑖𝑖𝑘
𝐷𝑖𝑖
• Then just after an updating operation for index 𝑖
𝑘
𝛿𝑖 = 0
Performance Analysis - 3
• Just before an updating operation for index i
𝛿𝑖
𝑘
≥ 𝑙𝑛 2
Between two successive updates, say at steps k1 and k2
𝛿𝑖
Let call ∆𝛿𝑖
𝑘
= 𝛿𝑖
𝑘2
− 𝛿𝑖
𝑘+1
𝑙𝑛 2 ≤
𝑘1
=
𝑘=𝑘1
𝑘
−𝛿
(𝑘 )
𝛿𝑖 2
𝑘2 −1
[𝛿𝑖
𝑘+1
−𝛿
𝑘
]
Then
= 𝛿𝑖
𝑘2
−𝛿𝑖
𝑘1
=
𝑘2 −1
𝑘=𝑘1
∆𝛿𝑖
𝑘
Assume that no updating operation was performed for index I
∆𝛿𝑖
𝑘
= 𝛿𝑖
𝑘
𝜎 𝑘 𝑥𝑖
𝑙𝑛 𝑘+1
𝑥𝑖
𝑘+1
−𝛿
= −𝑙𝑛∆𝑖
𝑘
′ 𝑘+1
= 𝑙𝑛
𝐷𝑖𝑖
𝑘+1
𝐷𝑖𝑖
′ 𝑘
- 𝑙𝑛
𝐷𝑖𝑖
𝑘
𝐷𝑖𝑖
𝑘
∆𝛿𝑖
𝑘
= 𝑙𝑛∆𝑖
𝑘
′ 𝑘+1
= 𝑙𝑛
𝐷𝑖𝑖
𝑘
𝐷𝑖𝑖
𝑘
×
𝐷𝑖𝑖
′ 𝑘
𝐷𝑖𝑖
=
Performance Analysis - 4
• Let 𝑛𝑖 be the number of updating operations corresponding to index 𝑖 in 𝑚 steps
and 𝑁 = 𝑛𝑖=1 𝑛𝑖 be the total number of updating operations in 𝑚 steps.
• We have
𝑚
𝑙𝑛 2𝑛𝑖 ≤
𝑙𝑛∆𝑖
𝑘=1
𝑛
𝑚
𝑙𝑛 2𝑁 =
𝑖=1
∆𝑖
𝑘
−1
2
𝑘
𝑘=1
≤ 𝛽 2 ⇒ ∆𝑖
𝑙𝑛∆𝑖
𝑘
𝑘
−1 ≤𝛽
𝑖
Because of 𝑥 ≤ 𝛽 < 1 then ln 1 + 𝑥 − 𝑥
⇒ 𝑙𝑛∆𝑖 𝑘 − (∆𝑖 𝑘 − 1) ≤
∆𝑖
𝑖
𝑘
−1
2
𝑥2
≤
2 1−𝛽
2
∆𝑘𝑖 − 1
2 1−𝛽
∆𝑖 𝑘 − 1 ≤ 𝑛𝛽
≤ 𝛽2 ⇒
𝑖
Performance Analysis - 5
• Hence
•
𝑛
𝑖=1
𝑙𝑛∆𝑖
𝑘
𝑛
≤
𝑖=1
∆𝑘𝑖
≤
𝑖
𝑛
𝑖=1
=
𝑙𝑛∆𝑖
𝑘
𝑙𝑛∆𝑖
− ∆𝑖
𝑘
2
𝑘
− ∆𝑖
𝑘
− 1 + (∆𝑖
𝑛
−1
+
𝑖=1
∆𝑖
𝑘
𝑘
−1
−1
𝛽2
+ 𝑛𝛽 ≤
+ 𝑛𝛽
2 1−𝛽
2 1−𝛽
• So
𝑛
𝑖=1
𝑚
𝑘=1
• 𝑁 = 𝑂(𝑚 𝑛)
• Finally 𝑂(𝑛𝐿 𝑛𝑛2 )
𝑙𝑛∆𝑖
𝑘
− 1)
𝛽2
≤𝑚
+ 𝑛𝛽
2 1−𝛽
Contents
• Overview
• Transformation to Karmarkar’s canonical form
• Projective transformation
• Orthogonal projection
• Complexity analysis
44
Transform to canonical form
General LP
𝑇
Minimize 𝑞 𝑥
Subject to
• 𝑆𝑥 ≥ 𝑡
• x≥0
Karmarkar’s Canonical Form
Minimize 𝑐 𝑇 𝑥
Subject to
• 𝐴𝑥 = 0
• 𝑥𝑖 = 1
• x≥0
45
Step 1:Convert LP to a
feasibility problem
• Combine primal and dual problems
Dual
Primal
Minimize 𝑞 𝑇 𝑥
Subject to
Maximize 𝑡 𝑇 𝑢
Subject to
• 𝑆𝑥 ≥ 𝑡
• x≥0
• 𝑆𝑇 𝑢 ≤ 𝑞
• 𝑢 ≥0
Combined
•
•
•
•
𝑞𝑇 𝑥 − 𝑡 𝑇 𝑢 = 0
𝑆𝑥 ≥ 𝑡
𝑆𝑇 𝑢 ≤ 𝑞
x, u ≥ 0
• LP becomes a feasibility problem
46
Step 2: Convert inequality to
equality
• Introduce slack and surplus variable
𝑞𝑇 𝑥 − 𝑡 𝑇 𝑢 = 0
• 𝑆𝑥 − 𝑦 = 𝑡
• 𝑆𝑇 𝑢 + 𝑣 = 𝑞
• x, u, y, v ≥ 0
47
Step 3: Convert feasibility
problem to LP
• Introduce an artificial variable to create an interior starting
point.
• Let 𝑥0 , 𝑦0 , 𝑢0 , 𝑣0 be strictly interior points in the positive
orthant.
• Minimize λ
• Subject to
•
•
•
•
𝑞𝑇 𝑥 − 𝑡 𝑇 𝑢 + (−𝑞𝑇 𝑥0 + 𝑡 𝑇 𝑢0 ) λ = 0
𝑆𝑥 − 𝑦 + (𝑡 − 𝑆𝑥0 + 𝑦0 )λ = t
𝑆 𝑇 𝑢 + 𝑣 + (𝑞 − 𝑆 𝑇 𝑢0 ) λ = 𝑞
𝑥, 𝑦, 𝑢, 𝑣, λ ≥ 0
48
Step 3: Convert feasibility
problem to LP
• Minimize λ
• Subject to
•
•
•
•
𝑞𝑇 𝑥 − 𝑡 𝑇 𝑢 + (−𝑞𝑇 𝑥0 + 𝑡 𝑇 𝑢0 ) λ = 0
𝑆𝑥 − 𝑦 + (𝑡 − 𝑆𝑥0 + 𝑦0 )λ = t
𝑆 𝑇 𝑢 + 𝑣 + (𝑞 − 𝑆 𝑇 𝑢0 ) λ = 𝑞
𝑥, 𝑦, 𝑢, 𝑣, λ ≥ 0
• Change of notation
Minimize 𝑐 𝑇 𝑥
Subject to
• 𝐴𝑥 = 𝑏
• x≥0
49
Step4: Introduce constraint
𝑥′𝑖 = 1
• Let 𝑎 = [𝑎1 , 𝑎2 ,…, 𝑎𝑛 ] be a feasible point in the original LP
• Define a transformation
• 𝑥′𝑖 =
𝑥𝑖
𝑎𝑖
𝑥𝑗
𝑛
𝑗=1𝑎
𝑗
+1
• 𝑥′𝑛+1 = 1 −
for 𝑖 = 1, 2, … , 𝑛
𝑛
𝑖=1 𝑥′𝑖
• Define the inverse transformation
• 𝑥𝑖 =
𝑎𝑖 𝑥′𝑖
𝑥′𝑛+1
for 𝑖 = 1, 2, … , 𝑛
50
Step4: Introduce constraint 𝐴′𝑥 ′ = 0
• Let 𝐴𝑖 denote the 𝑖 𝑡ℎ column of 𝐴
• If 𝑛𝑖=1 𝐴𝑖 𝑥𝑖 = 𝑏
• Then 𝑛𝑖=1 𝐴𝑖 𝑎𝑖 𝑥′𝑖 − 𝑏𝑥′𝑛+1 = 0
• We define the columns of a matrix 𝐴’ by these equations
• 𝐴′𝑖 = 𝑎𝑖 𝐴𝑖 for 𝑖 = 1, 2, … , 𝑛
• 𝐴′𝑛+1 = −𝑏
• Then
𝑛+1
𝑖=1 𝐴′𝑖 𝑥′𝑖
=0
51
Step 5: Get the minimum value
of Canonical form
• Substituting
• We get
𝑎𝑖 𝑥𝑖′
𝑥𝑖 = ′
𝑥𝑛+1
𝑛+1 𝑐 𝑎 𝑥 ′
𝑖 𝑖 𝑖
′
𝑖=1 𝑥𝑛+1
=0
• Define 𝑐′ ∈ 𝑅𝑛+1 by
𝑐𝑖′ = 𝑎𝑖 𝑐𝑖 𝑓𝑜𝑟 𝑖 = 1, … , 𝑛
′
𝑐𝑛+1
=0
• Then 𝑐 𝑇 𝑥 = 0 implies 𝑐 ′𝑇 𝑥 ′ = 0
52
Step 5: Get the minimum value
of Canonical form
• The transformed problem is
Minimize 𝑐 ′𝑇 𝑥 ′
Subject to
• 𝐴′ 𝑥 ′ = 0
• 𝑥′ ≥ 0
•
𝑛+1 ′
𝑖=1 𝑥𝑖
=1
53
References
• Narendra Karmarkar (1984). "A New Polynomial Time
Algorithm for Linear Programming”
• Strang, Gilbert (1 June 1987). "Karmarkar’s algorithm and its
place in applied mathematics". The Mathematical Intelligencer
(New York: Springer) 9 (2): 4–10. doi:10.1007/BF03025891.
ISSN 0343-6993. MR '''883185'‘
• Robert J. Vanderbei; Meketon, Marc, Freedman, Barry (1986).
"A Modification of Karmarkar's Linear Programming
Algorithm". Algorithmica 1: 395–407.
doi:10.1007/BF01840454. ^ Kolata, Gina (1989-03-12). "IDEAS
& TRENDS; Mathematicians Are Troubled by Claims on Their
Recipes". The New York Times.
54
References
• Gill, Philip E.; Murray, Walter, Saunders, Michael A., Tomlin, J.
A. and Wright, Margaret H. (1986). "On projected Newton
barrier methods for linear programming and an equivalence to
Karmarkar’s projective method". Mathematical Programming
36 (2): 183–209. doi:10.1007/BF02592025.
• oi:10.1007/BF02592025. ^ Anthony V. Fiacco; Garth P.
McCormick (1968). Nonlinear Programming: Sequential
Unconstrained Minimization Techniques. New York: Wiley.
ISBN 978-0-471-25810-0. Reprinted by SIAM in 1990 as ISBN
978-0-89871-254-4.
• Andrew Chin (2009). "On Abstraction and Equivalence in
Software Patent Doctrine: A Response to Bessen, Meurer and
Klemens". Journal Of Intellectual Property Law 16: 214–223.
55
Q&A
56