Smoothed Analysis of Multiobjective Optimization
Heiko Röglin
Department of Quantitative Economics
July 2010
DIMAP Summer School
based on joint work with
Rene Beier, Shang-Hua Teng, and Berthold Vöcking
Heiko Röglin
Smoothed Analysis of Multiobjective Optimization
Optimization Problems
Single-criterion Optimization Problem: min f (x ) subject to x ∈ S .
Example:
Shortest Path Problem
1
3
Heiko Röglin
1
1
s
1
9
8
10
9
t
2
Smoothed Analysis of Multiobjective Optimization
Optimization Problems
Single-criterion Optimization Problem: min f (x ) subject to x ∈ S .
Maastricht Aachen
Airport
Example:
Shortest Path Problem
1
1 Amsterdam
9
1
8
1
2
Maastricht
10
University 3
9
Frankfurt
Real-life logistical problems often involve multiple objectives.
(travel time, fare, departure time, etc.)
Heiko Röglin
Smoothed Analysis of Multiobjective Optimization
Optimization Problems
Single-criterion Optimization Problem: min f (x ) subject to x ∈ S .
Maastricht Aachen
Airport
Example:
Shortest Path Problem
1
1 Amsterdam
9
1
8
1
2
Maastricht
10
University 3
9
Frankfurt
Real-life logistical problems often involve multiple objectives.
(travel time, fare, departure time, etc.)
Multiobjective Opt. Problem: min f1 (x ), . . . , min fd (x ) s.t. x ∈ S .
Usually, there is no solution that is simultaneously optimal for all fi .
Question
What can we do algorithmically to support the decision maker?
Heiko Röglin
Smoothed Analysis of Multiobjective Optimization
Pareto-optimal Solutions
Multiobjective Opt. Problem: min w 1 (x ), . . . , min w d (x ) s.t. x ∈ S
x ∈ S dominates y ∈ S ⇐⇒
∀i : w i (x ) ≤ w i (y ) and
∃i : w i (x ) < w i (y )
fare
y
x
travel time
Heiko Röglin
Smoothed Analysis of Multiobjective Optimization
Pareto-optimal Solutions
Multiobjective Opt. Problem: min w 1 (x ), . . . , min w d (x ) s.t. x ∈ S
x ∈ S dominates y ∈ S ⇐⇒
∀i : w i (x ) ≤ w i (y ) and
∃i : w i (x ) < w i (y )
x ∈ S Pareto-optimal ⇐⇒
6 ∃y ∈ S : y dominates x
Heiko Röglin
fare
travel time
Smoothed Analysis of Multiobjective Optimization
Pareto-optimal Solutions
Multiobjective Opt. Problem: min w 1 (x ), . . . , min w d (x ) s.t. x ∈ S
x ∈ S dominates y ∈ S ⇐⇒
∀i : w i (x ) ≤ w i (y ) and
∃i : w i (x ) < w i (y )
fare
x ∈ S Pareto-optimal ⇐⇒
6 ∃y ∈ S : y dominates x
travel time
Often the Pareto curve is generated:
Pareto curve limits options for decision maker.
Monotone functions are optimized by Pareto-optimal solutions,
e.g., λ1 w 1 (x ) + . . . + λd w d (x ) or w 1 (x ) · · · · · w d (x ).
Heiko Röglin
Smoothed Analysis of Multiobjective Optimization
Pareto-optimal Solutions
Multiobjective Opt. Problem: min w 1 (x ), . . . , min w d (x ) s.t. x ∈ S
x ∈ S dominates y ∈ S ⇐⇒
∀i : w i (x ) ≤ w i (y ) and
∃i : w i (x ) < w i (y )
fare
x ∈ S Pareto-optimal ⇐⇒
6 ∃y ∈ S : y dominates x
travel time
Often the Pareto curve is generated:
Pareto curve limits options for decision maker.
Monotone functions are optimized by Pareto-optimal solutions,
e.g., λ1 w 1 (x ) + . . . + λd w d (x ) or w 1 (x ) · · · · · w d (x ).
Central Question
How large is the Pareto curve?
Heiko Röglin
Smoothed Analysis of Multiobjective Optimization
Model
Linear Binary Optimization Problem
set of feasible solutions S ⊆ {0, 1}n
solution x = (x1 , . . . , xn ) ∈ S consists of n binary variables
d linear objective functions:
∀i ∈ {1, . . . , d } : min w i (x ) = w1i x1 + · · · + wni xn
S can encode arbitrary combinatorial structure, e.g., for a given graph,
all paths from s to t, all Hamiltonian cycles, all spanning trees, . . .
Heiko Röglin
Smoothed Analysis of Multiobjective Optimization
Model
Linear Binary Optimization Problem
set of feasible solutions S ⊆ {0, 1}n
solution x = (x1 , . . . , xn ) ∈ S consists of n binary variables
d linear objective functions:
∀i ∈ {1, . . . , d } : min w i (x ) = w1i x1 + · · · + wni xn
S can encode arbitrary combinatorial structure, e.g., for a given graph,
all paths from s to t, all Hamiltonian cycles, all spanning trees, . . .
How large is the Pareto curve?
Exponential in the worst case for almost all problems.
In practice, often few Pareto optimal solutions.
Example: Train Connections
w.r.t. travel time, fare, number of train changes
[Müller-Hannemann, Weihe 2001]
Heiko Röglin
Smoothed Analysis of Multiobjective Optimization
Smoothed Analysis
Smoothed Analysis
S ⊆ {0, 1}n , d objectives: min w i (x ) = w1i x1 + · · · + wni xn
Every coefficient wji is an independent random variable following
a probability density fji : [−1, 1] → [0, φ].
Heiko Röglin
Smoothed Analysis of Multiobjective Optimization
Smoothed Analysis
Smoothed Analysis
S ⊆ {0, 1}n , d objectives: min w i (x ) = w1i x1 + · · · + wni xn
Every coefficient wji is an independent random variable following
a probability density fji : [−1, 1] → [0, φ].
each wji uniformly at random from interval of length 1/φ
wji are Gaussians, adversary specifies means, φ ∼ 1/σ
φ large ≈ worst case
φ small ≈ average case
Heiko Röglin
Smoothed Analysis of Multiobjective Optimization
Smoothed Analysis
Smoothed Analysis
S ⊆ {0, 1}n , d objectives: min w i (x ) = w1i x1 + · · · + wni xn
Every coefficient wji is an independent random variable following
a probability density fji : [−1, 1] → [0, φ].
each wji uniformly at random from interval of length 1/φ
wji are Gaussians, adversary specifies means, φ ∼ 1/σ
φ large ≈ worst case
φ small ≈ average case
Qd (n, φ) = max E number of Pareto-optimal sol. for S and fji
S,fji
Heiko Röglin
Smoothed Analysis of Multiobjective Optimization
Results
Bicriteria Optimization (d = 2):
Beier, Vöcking (STOC 2003)
For any S and any fji , Q2 (n, φ) = O (n4 φ).
There are S and fji such that Q2 (n, φ) = Ω(n2 ).
Heiko Röglin
Smoothed Analysis of Multiobjective Optimization
Results
Bicriteria Optimization (d = 2):
Beier, Vöcking (STOC 2003)
For any S and any fji , Q2 (n, φ) = O (n4 φ).
There are S and fji such that Q2 (n, φ) = Ω(n2 ).
Beier, R., Vöcking (IPCO 2007)
For any S and any fji , Q2 (n, φ) = O (n2 φ).
Extension to integer optimization problems.
Heiko Röglin
Smoothed Analysis of Multiobjective Optimization
Results
Bicriteria Optimization (d = 2):
Beier, Vöcking (STOC 2003)
For any S and any fji , Q2 (n, φ) = O (n4 φ).
There are S and fji such that Q2 (n, φ) = Ω(n2 ).
Beier, R., Vöcking (IPCO 2007)
For any S and any fji , Q2 (n, φ) = O (n2 φ).
Extension to integer optimization problems.
Multiobjective Optimization (d arbitrary constant):
R., Teng (FOCS 2009)
For any S and any fji , Qd (n, φ) = O ((nφ)h(d ) ) for some function h.
h p
For any c ∈ 1,
i
log(n) , Qd (n, φ)c = O ((nφ)c ·h(d ) ).
Heiko Röglin
Smoothed Analysis of Multiobjective Optimization
Generalized Loser Gap
w2
How many
PO x ∈ S with
w 1 (x ) ∈ [t , t + ε]?
t
Heiko Röglin
t+ε
w1
Smoothed Analysis of Multiobjective Optimization
Generalized Loser Gap
w2
How many
PO x ∈ S with
w 1 (x ) ∈ [t , t + ε]?
x?
t+ε
t
2
w1
1
single-criterion problem: min w (x ) s.t. w (x ) ≤ t and x ∈ S
winner: x ? = optimal solution
Heiko Röglin
Smoothed Analysis of Multiobjective Optimization
Generalized Loser Gap
w2
How many
PO x ∈ S with
w 1 (x ) ∈ [t , t + ε]?
x?
L
t+ε
t
2
w1
1
single-criterion problem: min w (x ) s.t. w (x ) ≤ t and x ∈ S
winner: x ? = optimal solution
loser set: L = all solutions x ∈ S with w 2 (x ) < w 2 (x ? )
Heiko Röglin
Smoothed Analysis of Multiobjective Optimization
Generalized Loser Gap
w2
How many
PO x ∈ S with
w 1 (x ) ∈ [t , t + ε]?
Λ2 (t)
x?
L
t
2
Λ1 (t) Λ3 (t)
t+ε
w1
1
single-criterion problem: min w (x ) s.t. w (x ) ≤ t and x ∈ S
winner: x ? = optimal solution
loser set: L = all solutions x ∈ S with w 2 (x ) < w 2 (x ? )
i-th loser gap: Λi (t ) = distance of i-th loser from t
Heiko Röglin
Smoothed Analysis of Multiobjective Optimization
Generalized Loser Gap
w2
How many
PO x ∈ S with
w 1 (x ) ∈ [t , t + ε]?
Λ2 (t)
x?
L
t
2
Λ1 (t) Λ3 (t)
t+ε
w1
1
single-criterion problem: min w (x ) s.t. w (x ) ≤ t and x ∈ S
winner: x ? = optimal solution
loser set: L = all solutions x ∈ S with w 2 (x ) < w 2 (x ? )
i-th loser gap: Λi (t ) = distance of i-th loser from t
Pr ∃` many PO solutions x with w 1 (x ) ∈ [t , t + ε] ≤ Pr Λ` (t ) ≤ ε
Heiko Röglin
Smoothed Analysis of Multiobjective Optimization
Generalized Loser Gap
w2
How many
PO x ∈ S with
w 1 (x ) ∈ [t , t + ε]?
Λ2 (t)
x?
L
t
2
Λ1 (t) Λ3 (t)
t+ε
w1
1
single-criterion problem: min w (x ) s.t. w (x ) ≤ t and x ∈ S
winner: x ? = optimal solution
loser set: L = all solutions x ∈ S with w 2 (x ) < w 2 (x ? )
i-th loser gap: Λi (t ) = distance of i-th loser from t
Pr ∃` many PO solutions x with w 1 (x ) ∈ [t , t + ε] ≤ Pr Λ` (t ) ≤ ε
Heiko Röglin
Smoothed Analysis of Multiobjective Optimization
Generalized Loser Gap
w2
How many
PO x ∈ S with
w 1 (x ) ∈ [t , t + ε]?
Λ2 (t)
x?
L
t
2
Λ1 (t) Λ3 (t)
t+ε
w1
1
single-criterion problem: min w (x ) s.t. w (x ) ≤ t and x ∈ S
winner: x ? = optimal solution
loser set: L = all solutions x ∈ S with w 2 (x ) < w 2 (x ? )
i-th loser gap: Λi (t ) = distance of i-th loser from t
Pr ∃` many PO solutions x with w 1 (x ) ∈ [t , t + ε] ≤ Pr Λ` (t ) ≤ ε
Lemma [Beier, Vöcking (STOC 2004)]
For every ε ≥ 0 and t ∈ R, Pr Λ1 (t ) ≤ ε ≤ nφε.
Heiko Röglin
Smoothed Analysis of Multiobjective Optimization
Generalized Loser Gap
w2
How many
PO x ∈ S with
w 1 (x ) ∈ [t , t + ε]?
Λ2 (t)
x?
L
t
2
Λ1 (t) Λ3 (t)
t+ε
w1
1
single-criterion problem: min w (x ) s.t. w (x ) ≤ t and x ∈ S
winner: x ? = optimal solution
loser set: L = all solutions x ∈ S with w 2 (x ) < w 2 (x ? )
i-th loser gap: Λi (t ) = distance of i-th loser from t
Pr ∃` many PO solutions x with w 1 (x ) ∈ [t , t + ε] ≤ Pr Λ` (t ) ≤ ε
Lemma [R., Teng (FOCS 2009)]
For every ε ≥ 0, z ∈ N, and t ∈ R,
h
Pr Λ2
z −1
i
2
(t ) ≤ ε ≤ 2z +z nz φz εz −1 .
Heiko Röglin
Smoothed Analysis of Multiobjective Optimization
Higher Moments
w2
divide [0, n] into k intervals:
[0, n] = [t0 , t1 ], . . . , [tk −1 , tk ]
t1
Heiko Röglin
t2
t3
t4
t5
Smoothed Analysis of Multiobjective Optimization
t6
w1
Higher Moments
w2
divide [0, n] into k intervals:
[0, n] = [t0 , t1 ], . . . , [tk −1 , tk ]
t1
c
z −1
Pr Q2 (n, φ) ≥ (2
c
t2
t3
t4
t5
· k)
≤ Pr ∃i : [ti , ti +1 ] contains more than 2z −1 PO solutions
2
2n
2z +2z −1 n2z −1 φz
2z −1
(ti ) ≤
≤ Pr ∃i : Λ
≤
z −2
k
Heiko Röglin
k
Smoothed Analysis of Multiobjective Optimization
t6
w1
Higher Moments
w2
divide [0, n] into k intervals:
[0, n] = [t0 , t1 ], . . . , [tk −1 , tk ]
t1
c
z −1
Pr Q2 (n, φ) ≥ (2
c
t2
t3
t4
t5
· k)
≤ Pr ∃i : [ti , ti +1 ] contains more than 2z −1 PO solutions
2
2n
2z +2z −1 n2z −1 φz
2z −1
(ti ) ≤
≤ Pr ∃i : Λ
≤
z −2
k
k
R., Teng (FOCS 2009)
h p
For any S and any fji and every c ∈ 1,
i
log(n) ,
Qd (n, φ)c = (n2 φ)c(1+o(1)) .
Heiko Röglin
Smoothed Analysis of Multiobjective Optimization
t6
w1
Proof Idea for d ≥ 3
How many PO x ∈ S with
w 1 (x ) ∈ [t , t + ε]?
min w 2 (x ), . . . , min w d (x )
s.t. w 1 (x ) ≤ t and x ∈ S
w2
w3
t
Heiko Röglin
w1
Smoothed Analysis of Multiobjective Optimization
Proof Idea for d ≥ 3
How many PO x ∈ S with
w 1 (x ) ∈ [t , t + ε]?
min w 2 (x ), . . . , min w d (x )
s.t. w 1 (x ) ≤ t and x ∈ S
w2
w3
t
w1
d ≥3
P ? = set of PO solutions
d =2
optimal solution x ?
Heiko Röglin
Smoothed Analysis of Multiobjective Optimization
Proof Idea for d ≥ 3
How many PO x ∈ S with
w 1 (x ) ∈ [t , t + ε]?
min w 2 (x ), . . . , min w d (x )
s.t. w 1 (x ) ≤ t and x ∈ S
w2
w3
t
d =2
optimal solution x ?
L = {x ∈ S | w 2 (x ) < w 2 (x ? )}
Heiko Röglin
w1
d ≥3
P ? = set of PO solutions
(x2? , . . . , xd? ) ∈ (P ? )d −1
L = {x ∈ S | ∀i : w i (x ) < w i (xi? )}
Smoothed Analysis of Multiobjective Optimization
Proof Idea for d ≥ 3
How many PO x ∈ S with
w 1 (x ) ∈ [t , t + ε]?
min w 2 (x ), . . . , min w d (x )
s.t. w 1 (x ) ≤ t and x ∈ S
w2
w2 (x?2 )
x?2
w3
t
d =2
optimal solution x ?
L = {x ∈ S | w 2 (x ) < w 2 (x ? )}
Heiko Röglin
w1
d ≥3
P ? = set of PO solutions
(x2? , . . . , xd? ) ∈ (P ? )d −1
L = {x ∈ S | ∀i : w i (x ) < w i (xi? )}
Smoothed Analysis of Multiobjective Optimization
Proof Idea for d ≥ 3
How many PO x ∈ S with
w 1 (x ) ∈ [t , t + ε]?
min w 2 (x ), . . . , min w d (x )
s.t. w 1 (x ) ≤ t and x ∈ S
w2
w2 (x?2 )
x?2
w3 (x?3 )
w3
x?3
L
t
d =2
optimal solution x ?
L = {x ∈ S | w 2 (x ) < w 2 (x ? )}
Heiko Röglin
w
1
d ≥3
P ? = set of PO solutions
(x2? , . . . , xd? ) ∈ (P ? )d −1
L = {x ∈ S | ∀i : w i (x ) < w i (xi? )}
Smoothed Analysis of Multiobjective Optimization
Proof Idea for d ≥ 3
How many PO x ∈ S with
w 1 (x ) ∈ [t , t + ε]?
min w 2 (x ), . . . , min w d (x )
s.t. w 1 (x ) ≤ t and x ∈ S
w2
w2 (x?2 )
x?2
w3 (x?3 )
w3
x?3
d =2
optimal solution x ?
L = {x ∈ S | w 2 (x ) < w 2 (x ? )}
L
Λ1 (t)
t
w
1
d ≥3
P ? = set of PO solutions
(x2? , . . . , xd? ) ∈ (P ? )d −1
L = {x ∈ S | ∀i : w i (x ) < w i (xi? )}
Every (d − 1)-tuple from P defines a loser gap.
Induction: Use bound for Qd −1 (n, φ)d −1 to bound Qd (n, φ).
Heiko Röglin
Smoothed Analysis of Multiobjective Optimization
Proof Idea for d ≥ 3
How many PO x ∈ S with
w 1 (x ) ∈ [t , t + ε]?
min w 2 (x ), . . . , min w d (x )
s.t. w 1 (x ) ≤ t and x ∈ S
w2
w2 (x?2 )
x?2
w3 (x?3 )
w3
x?3
d =2
optimal solution x ?
L = {x ∈ S | w 2 (x ) < w 2 (x ? )}
L
Λ1 (t)
t
w
1
d ≥3
P ? = set of PO solutions
(x2? , . . . , xd? ) ∈ (P ? )d −1
L = {x ∈ S | ∀i : w i (x ) < w i (xi? )}
Every (d − 1)-tuple from P defines a loser gap.
Induction: Use bound for Qd −1 (n, φ)d −1 to bound Qd (n, φ).
Theorem
∀d ∀c ∈ [1,
log(n)]: Qd (n, φ)c = O ((nφ)c ·h(d ) ) for h(d ) = 2d −3 d !.
p
Heiko Röglin
Smoothed Analysis of Multiobjective Optimization
Extensions and Open Questions
Further Results
polynomial bound for Qd (n, φ)c for any constants c and d
⇒ first concentration bounds for |P|
extension to integer case S ⊆ {−m, −m + 1, . . . , m − 1, m}n
polyn. smoothed complexity = expected polynomial running time
Heiko Röglin
Smoothed Analysis of Multiobjective Optimization
Extensions and Open Questions
Further Results
polynomial bound for Qd (n, φ)c for any constants c and d
⇒ first concentration bounds for |P|
extension to integer case S ⊆ {−m, −m + 1, . . . , m − 1, m}n
polyn. smoothed complexity = expected polynomial running time
Open Questions
h(d ) = 2d −3 d !: (nφ)O (d ) possible?
lower bounds?
higher moments, stronger concentration?
Heiko Röglin
Smoothed Analysis of Multiobjective Optimization
Thank you for your attention!
Questions?
Heiko Röglin
Smoothed Analysis of Multiobjective Optimization