Air Traffic Management
(Gestão de Tráfego Aéreo)
MEAer
Optimization and ATM
(updated 6-Dec-2020)
(C) 2010-2020 Rodrigo Ventura, IST
Air traffic in Europe
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Air traffic in Europe
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Air Traffic Flow Management (ATFM)
•
Objectives:
•
to match capacity with demand in air transportation
assure safety and efficiency
avoid congestion and delays
Instruments:
-
ground holding controls
rerouting controls
airborne holding controls
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Ground Holding Problem (GHP)
•
•
•
Problem statement: to assign ground delays to aircraft given airport capacity
Objective: minimize sum of ground and airborne delays (weighted costs)
Inputs:
•
scheduled departure and arrival times
departure, arrival, and enroute capacities
Output:
•
set of flights
amount of delay to impose to each flight
Approach: formulate problem as an optimization problem
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Optimization
• Optimization is widely used methodology in many decision problems
• Examples:
- aircraft performance to maximize fuel efficiency
- airliners crew and aircraft scheduling to minimize cost
- manage financial assets portfolio to maximize profit
- maximize processes efficiency in manufacturing industry
- minimize air traffic delays in overloaded arrival airports
- minimize error while fitting a model to data
- minimize classification error in pattern recognition
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Optimization
•
Bibliography:
(free)
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Optimization problem
•
Variables or unknowns: parameters whose values are being sought
•
Objective: scalar function of the variables to be maximized or minimized
•
examples: cruise FL, departure time, estimator parameters, etc.
examples: efficiency, cost, profit, total delay, estimation error, etc.
max/min reducible to one another ( min{cost} ≡ max{-cost} )
Constraints: restrictions on the values of the variables
-
examples: operational ceiling of aircraft, total budget to invest, etc.
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Mathematical formulation
•
Notation:
•
x is the vector of variables
f is the objective function, mapping x to a real number f(x)
c is a vector of constraints c(x) that x has to satisfy
General optimization problem:
minn f (x)
x2R
subject to
(
ci (x) = 0,
ci (x)  0
where E are the equality constraints, and
I are the inequality constraints.
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
i 2 E,
i 2 I.
Mathematical formulation
•
Example (unconstrained):
min
(x1 ,x2 )2R
(x
1
2
2
2) + (x2
1)
2
where
f(x) = (x1-2)2 + (x2-1)2
x = [x1, x2]T
I = {}
E = {}
x2
x1
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Mathematical formulation
•
min
Example (constrained):
(x1 ,x2 )2R
(x
1
2
2)2 + (x2
where
f(x) = (x1-2)2 + (x2-1)2
x = [x1, x2]T
c = [c1, c2]T
c1(x) = x12 - x2
c2(x) = x1 + x2 - 2
I = {1, 2}
E = {}
1)2
subject to
(
x21 x2  0,
Chapter 1. Introduction 3
x1 + x2  2.
x2
c2
c1
contours of f
feasible
region
x*
x1
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Figure 1.1
Geometrical representation of an optimization problem.
Example: linear regression
•
•
•
•
•
Data: set of pairs of points { (xi, yi) }
Model: yi = m xi + b
Fitting error: ei = yi - m xi - b
Cost function: J = ∑i ei2 = ∑i (yi - m xi - b)2
Optimization problem: min(m,b) J
GTA 2016/2017 — course slides — (C) 2010-2016 Rodrigo Ventura, IST
Example: diet problem
•
•
n different foods, where the i-th food costs ci per unit
•
•
•
each unit of the i-th food contains aji units of the j-th ingredient
min
m basic nutricional ingredients, where each individual must ingest at least bj
units of the j-th ingredient for a healthy diet
let xi be the number of units of the i-th food in the diet
problem: minimize total cost of the diet, that is c1x1 + … + cnxn
subject to minimum nutricional quantities and non-negative amounts xi
x1 ,...,xn
n
X
i=1
c i xi
subject to
(P
xi
n
i=1 aji xi
0,
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
bj ,
j = 1, . . . , m
i = 1, . . . , n
Example: transportation problem
•
•
•
•
•
•
min
one product manufactured by two factories: F1 and F2
twelve retail outlets: R1, …, R12
factory Fi produces ai units per week
outlet Ri sells bi units per week
the cost of shipping one unit from Fi to Rj is cij
problem: how much units xij to ship from factory Fi to outlet Rj in order to
minimize total shipping costs?
X
i,j
cij xij
subject to
8P
>
<Pj xij  ai ,
xij bj ,
i
>
:
xij 0,
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
for all i,
for all j,
for all i and j.
Continuous vs discrete
•
Continuous optimization: all variables are continuous
•
Discrete optimization: all variables are discrete
•
that is, xk ∈ Z or xk ∈ {0, 1} for all k
also known as Integer Programming (IP)
Mixed continuous and discrete variables
•
that is, xk ∈ R for all k
also known as Mixed Integer Programming (MIP)
Example: transportation problem: if units are atomic, xij ∈ Z, that is, an IP
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Constrained vs unconstrained
•
Unconstrained optimization: when there are no constraints
•
easier to solve numerically
Constrained optimization: when there is at least one constraint
-
many constrained optimization problems can be solved using
unconstrained optimization methods iteratively
-
duality methods allows the formulation of a dual optimization problem
which can be used to help solving the original one (also known as the
primal)
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Linear vs nonlinear
•
Linear Programming (LP): when both the objective and the constraints are
linear functions of the variables
•
very common in management science and operations research
very efficient algorithms exist to solve LP problems
the effort of formulating problems as a LP typically pays off
example: the transportation and the diet problems are LP
variant: Integer Linear Programming (ILP), if variables are all integers
variant: Mixed Integer Linear Programming (MILP), if some vars. are integers
Nonlinear programming: when either the objective or a at least one of the
constraints is not a linear function of the variables
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Global vs local optimization
•
Local optimization: when solution given is smaller (bigger) than all other
feasible points in its vicinity
•
most optimization algorithms are local optimizers
for objective functions with multiple local minima (maxima), the solution
is not unique
Global optimization: when solution given corresponds to the absolute best
value of the objective function across all feasible space*
-
under certain conditions, e.g., convexity, there is a unique local solution,
which is therefore also global
* feasible space is the set of all possible variables values satisfying all constraints
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Optimization algorithms
•
Typically numeric optimization algorithms are iterative:
•
starts with an initial guess (cold or warm start)
iteratively improves guess
until a convergence criterion is met
Good optimization algorithms are:
-
robust: with respect classes of problems and initial guess choices
efficient: in terms of computational time and memory usage
accurate: the solution given sufficiently close to the true optimum
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Convexity
•
Convex set: S ⊂ Rn such that a straight line connecting any two points in S lies entirely
inside S, that is
✓x1 + (1
•
✓)x2 2 S
for all 0  ✓  1 and x1 , x2 2 S
Examples:
-
subspaces, half-spaces, and hyperplanes
euclidean balls
probability distributions
semi-definite positive symmetric matrices (A such that A=AT and xTAx≥0 for all x≠0)
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Convexity
•
Examples of operations that preserve convexity of sets
-
Intersection:
Let S1 and S2 be two convex sets, then S1 ⋂ S2 is also convex
-
Affine function:
An affine vector function has the form f(x) = Ax + b where A is a
matrix and b a vector
If S is a convex set, then the set { f(x) | x∈S } is also convex
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Convexity
•
Convex function: if domain is convex and if the linear interpolation between
any two points in the domain is greater or equal than the function itself
f (✓x + (1
✓)y)  ✓f (x) + (1
✓)f (y)
for all 0  ✓  1 and x, y 2 dom f
•
Examples:
-
exponential, power, symmetric of logarithm
norms, max function, linear functions, geometric mean, -log det of matrices
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Convexity
•
First-order condition: function f is convex is and only if
f(y) ≥ f(x) + ∇f(x)T (y - x)
for all x and y
•
Second-order condition: function f is convex if and only if the Hessian ∇2f(x)
is positive semidefinite
-
a matrix A is positive semidefinite if and only if
xT A x ≥ 0
for all x
-
a matrix A is positive semidefinite if and only if all eigenvalues are non-negative
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Convexity
•
Examples of operations that preserve convexity of functions:
-
Nonnegative weighted sum:
f = w1 f1 + w2 f2 + … wm fm
is convex if fi is convex and wi ≥ 0 for i=1, … m
-
Composition with an affine map:
g(x) = f(Ax + b)
is convex if f is convex
-
Pointwise maximum:
f(x) = max{ f1(x), f2(x) }
is convex if f1 and f2 are convex
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Convexity
•
Unconstrained optimization case:
•
if objective function f(x) is convex, then local optimum ≡ global optimum
if f(x) is differentiable, ∇f(x)=0 is necessary and sufficient condition for optimality
most iterative algorithms are guaranteed to converge to the solution
Constrained optimization case:
-
convex programming problem:
•
•
•
-
convex objective function
equality constraints ci(x), i∈E, are linear
inequality constraints, ci(x), i∈I, are concave, i.e., -ci(x) is convex
KKT conditions (see ahead) are necessary and sufficient for optimality
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Descent methods for unconstrained problems
•
General descent step: x(k+1) = x(k) + t(k) Δx(k)
•
•
Δx(k) is the descent direction
9.2
Descent methods
t(k) is the descent step
Exact line search: t = argmins≥0 f(x + s Δx)
f (x + t∆x)
Inexact line search, e.g., fixed or adaptive t
-
backtracking line search:
start with t=1, and reduce t ← 𝛽t while
f (x) + t∇f (x)T ∆x
f (x) + αt∇f (x)T ∆x
t
t=0
t0
f (x + t x) > f (x) + ↵t[rf (x)]T x
Figure 9.1 Backtracking line search. The curve shows f , restricted to the line
over which we search. The lower dashed line shows the linear extrapolation
of f , and the upper dashed line has a slope a factor of α smaller. The
backtracking condition is that f lies below the upper dashed line, i.e., 0 ≤
t ≤ t0 .
where 0<𝛼<0.5 and 0<𝛽<1 are tuning parameters
•
The line search is called backtracking because it starts with unit step size and
then reduces it by the factor β until the stopping condition f (x + t∆x) ≤ f (x) +
αt∇f (x)T ∆x holds. Since ∆x is a descent direction, we have ∇f (x)T ∆x < 0, so
for small enough t we have
Gradient descent: Δx = -∇f(x) with fixed or adaptive t
f (x + t∆x) ≈ f (x) + t∇f (x)T ∆x < f (x) + αt∇f (x)T ∆x,
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
which shows that the backtracking line search eventually terminates. The constant
α can be interpreted as the fraction of the decrease in f predicted by linear extrap-
Descent methods for unconstrained problems
•
Newton’s method:
-
approximate f(x) using second order Taylor expansion:
f (x +
-
x) ⇡ f (x) + [rf (x)]
|
T
1
x+
xT [r2 f (x)] x
{z 2
}
h( x)
h(𝛥x) is convex and easy to minimize:
rh( x) = 0 , rf (x) + [r2 f (x)] x = 0
-
,
x=
[r2 f (x)] 1 rf (x)
Newton’s step: Δx = -[∇2f(x)]-1 ∇f(x) with t=1 or adaptive t
Quasi-Newton methods: uses approximation of [∇2f(x)]-1
Examples: BFGS and L-BFGS are popular methods (used in MATLAB’s fminunc())
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Rosenbrock function:
Example
f (x, y) = (1
x)2 + 100(y
with minimum at f(1,1)=0
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
x2 ) 2
Lagrange multiplier
•
Consider the problem
min 2 f (x, y)
(x,y)2R
s.t.
g(x, y) = 0
<latexit sha1_base64="sWcTU/e0CZCoqAmcfIjW7tFmyTk=">AAACK3icbVDLSgMxFM34rPVVdekmWBQFKTMi6EKh6MZlFVuFppZMmqnBTGZM7kjLMP/jxl9xoQsfuPU/TKdd+DoQOJxzLzfn+LEUBlz3zRkbn5icmi7MFGfn5hcWS0vLDRMlmvE6i2SkL31quBSK10GA5Jex5jT0Jb/wb44H/sUd10ZE6hz6MW+FtKtEIBgFK7VLRxskFKqdbva2+1tEKBJSuPb99Cy72slwkMuYkOIGAd6D1FSgkmFym9AO7ubmodsuld2KmwP/Jd6IlNEItXbpiXQiloRcAZPUmKbnxtBKqQbBJM+KJDE8puyGdnnTUkVDblppnjXD61bp4CDS9inAufp9I6WhMf3Qt5ODKOa3NxD/85oJBPutVKg4Aa7Y8FCQSAwRHhSHO0JzBrJvCWVa2L9idk01ZWDrLdoSvN+R/5LGTsWz/HS3XD0Y1VFAq2gNbSIP7aEqOkE1VEcM3aNH9IJenQfn2Xl3PoajY85oZwX9gPP5BcYXpXw=</latexit>
•
Assuming ∇g(x, y)≠0,
the extremum of f(x, y) along g(x, y)=0
has ∇f(x, y) parallel to ∇g(x, y)
•
•
That is, ∇f(x, y) + λ ∇g(x, y) = 0
Lagrangian function: L(x, y, λ) = f(x, y) + λ g(x, y)
∇(x, y)L(x, y) = ∇f(x, y) + λ ∇g(x, y) = 0
∇λL(x, y) = g(x, y) = 0
where λ is the Lagrange multiplier
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Optimality conditions
•
General constrained optimization problem
minn f (x)
x2R
•
subject to
(
ci (x) = 0,
ci (x)  0
i 2 E,
i 2 I.
Lagrangian:
L(x, , ⌫) = f (x) +
X
i2I
i ci (x) +
X
⌫i ci (x)
i2E
where 𝜆i and 𝜈i are called dual variables or Lagrange multipliers
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
improving the value of the objective function. (ii) The fence
downhill side of the hill, in which case it is both active and is pre
on the hill. We need to distinguish these two cases to determine
optimum. In case (i) you are not at a local optimum, but in cas
Lagrange multiplier tells us whether we are in case (i) or case (ii).
Optimality conditions
•
Lagrangian:
L(x, , ⌫) = f (x) +
X
i2I
•
How does this work? First, we require that all inequality constra
equal form, i.e.
for i=1...m. As always, the gradie
increasing direction. So if a constraint of this form is active, then
is prevented: the constraint prevents gi(x) from getting any big
maximize the objective function: this means we want to move in
direction. Hence if you are at some point and attempting to m
is active, and it's gradient is in the s
i idirection, but an inequality
i i
gradient, then the Lagrange multiplier will be positive, and you w
i2E
local optimum. This concept is sketched in Fig. 20.3.
c (x) +
X
⌫ c (x)
Karush-Kuhn-Tucker (KKT) conditions:
Some grad
lightly ske
the value
lines. As
constraint
by the con
g(x) greate
the object
direction:
equality w
multiplier
ci (x)  0, i 2 I
ci (x) = 0, i 2 E
i
0, i 2 I
i ci (x) = 0, i 2 I
rx L(x, , ⌫) = 0
•
•
max f(x) s.t. g(x)≤30
Figure 20.3: Lagrange multiplier nonnegativity
condition.
KKT conditions are necessary for optimality under some regularity conditions
Practical Optimization: a Gentle Introduction
http://www.sce.carleton.ca/faculty/chinneck/po.html
KKT conditions are necessary and sufficient for optimality for convex problems
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
If the grad
the oppos
Fig. 20.3,
would be
”John W. Ch
Example
•
Consider the problem:
min xT x
x
s.t. Ax = b
•
•
T
T
L(x,
⌫)
=
x
x
+
⌫
(Ax
Lagrangian:
b)
Using the KKT conditions:
rx L(x, ⌫) = 0 , 2x + AT ⌫ = 0
1 T
, x=
A ⌫
2
1
Ax b = 0 ,
AAT ⌫ = b
2
, ⌫ = 2(AAT ) 1 b
•
Solution:
x = AT (AAT ) 1 b
GTA 2016/2017 — course slides — (C) 2010-2016 Rodrigo Ventura, IST
Duality
•
Lagrange dual function: g( , ⌫) = inf L(x, , ⌫)
x
this is a convex function, even if f(x) is not
•
Lagrange dual problem: max g( , ⌫)
,⌫
s.t.
0
this is a convex optimization problem, even if the original problem is not
•
Duality gap: difference between the minimum of the original problem and
the maximum of the dual problem (can be proven to be always positive)
•
•
Strong duality is a condition that implies zero duality gap
Under strong duality, the solution of the dual problem can be used to obtain
the solution to the original problem
GTA 2016/2017 — course slides — (C) 2010-2016 Rodrigo Ventura, IST
Example (revisited)
•
Consider the problem:
min xT x
x
s.t. Ax = b
•
•
T
T
L(x,
⌫)
=
x
x
+
⌫
(Ax
Lagrangian:
b)
Lagrange dual function: g(⌫) = min L(x, ⌫)
x
=
1 T
⌫ AAT ⌫
4
⌫T b
since rx L(x, ⌫) = 0 ) x =
•
Lagrange dual problem: max g(⌫)
⌫
•
1 T
A ⌫
2
solution (dual optimum):
r⌫ g(⌫) = 0 ) ⌫ =
2(AAT ) 1 b
Solution (primal optimum, due to strong duality): x = AT (AAT ) 1 b
GTA 2016/2017 — course slides — (C) 2010-2016 Rodrigo Ventura, IST
Descent methods for constrained problems
•
Consider the problem:
min f (x)
x
s.t. Ax = b
•
Approximating f(x) using a second-order Taylor expansion:
min f (x +
x) ⇡ f (x) + [rf (x)]
s.t. A(x +
x) = b
x
•
1
x+
xT [r2 f (x)] x
2
Lagrangian:
L( x, ⌫) = f (x) + [rf (x)]
•
T
T
1
x+
xT [r2 f (x)] x + ⌫ T [A(x +
2
Applying the KKT conditions

2
r f (x)
A
T
A
0
✓
x
⌫
◆
=
✓
rf (x)
0
◆
we obtain the Newton step 𝛥x while maintaining feasibility
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
x)
b]
Sequential Quadratic Programming (SQP)
•
•
Considering only equality constraints:
Quadratic Program (QP):

<latexit sha1_base64="NYd6kbA9zrT2ANMX4g+gcWeuzb4=">AAACjXicbZHfTtswFMadsA0W2ChwuRtrFYhdUCUIxB/BBOKCXTKJAlKdVY5z0lo4TmSfIKqob8MT7Y63wSlF2kqPZOnT9/knH5+TlEpaDMNnz1/48PHT4tLnYHnly9fV1tr6jS0qI6ArClWYu4RbUFJDFyUquCsN8DxRcJvcXzT57QMYKwt9jaMS4pwPtMyk4OisfuuJKciwxxIYSF3nHI18HAeXdIvunP+5powF506HAQOdvsXMyMEQ42CCbs+gj46hTLkOUj6H+hGczud20gZM5iL9VjvshJOi70U0FW0yrat+6y9LC1HloFEobm0vCkuMa25QCgXjgFUWSi7u+QB6Tmqeg43ryTTHdNM5Kc0K445GOnH/JWqeWzvKE3fT9Tm0s1ljzst6FWaHcS11WSFo8fpQVimKBW1WQ1NpQKAaOcGFka5XKobccIFugc0Qotkvvxc3u51ovxP+3mufnUzHsUS+ke9km0TkgJyRX+SKdInwAi/0jrxjf9Xf90/8n69XfW/KbJD/yr98AQ7FxGA=</latexit>
G
A
Closed form solution:
•
A
0
✓ ◆
x
T
=
✓
d
b
◆
1 T
min x Gx + dT x
x
2
s.t. Ax = b
<latexit sha1_base64="HzBC2ZznyteBEIV7ssT+Ww1FLyo=">AAACJHicbZBNSyNBEIZ7XD/jV1yPXgqDIgjDTFAUd4WIh/UYwaiQiaGn02Mae3qG7hqZMMyP8eJf2cse/GAPXvwtdmIOru4LDQ9vVVFdb5hKYdDzXpyJb5NT0zOzc5X5hcWl5erK93OTZJrxFktkoi9DargUirdQoOSXqeY0DiW/CG+Oh/WLW66NSNQZDlLeiem1EpFgFK3Vrf7YDGKhunlwAEGkKSv8sqiXkF+dwS/IYRt6lnIIgspmgDzHwrjoQglH+WHYrdY81xsJvoI/hhoZq9mtPgW9hGUxV8gkNabteyl2CqpRMMnLSpAZnlJ2Q69526KiMTedYnRkCRvW6UGUaPsUwsj9OFHQ2JhBHNrOmGLffK4Nzf/V2hlG+51CqDRDrtj7oiiTgAkME4Oe0JyhHFigTAv7V2B9arNCm2vFhuB/PvkrnNddf9f1TndqjZ/jOGbJGlknW8Qne6RBTkiTtAgjd+Q3eSCPzr3zx3l2/r63TjjjmVXyj5zXNxDVofM=</latexit>
SQP is an iterative algorithm where:
<latexit sha1_base64="MUcGKzZ0TNqlODPIfwY3nE+v+2s=">AAACeHicbVFNS8MwGE7r9/yaevQSHOKGMNqh6FHwoMcJToW1G2n6VsPSpCapdJT+UC/e/QuezD4OTn0h8PB85A1PoowzbTzv3XGXlldW19Y3aptb2zu79b39By1zRaFHJZfqKSIaOBPQM8xweMoUkDTi8BiNrif64xsozaS4N+MMwpQ8C5YwSoylhvUiaRYtHGiWwiu2eFA2R62qhU9xP8hFDCpShEIZCBJx8sNQDeNwcI8LawwSayn9quxUuLDcP8FBZyF6E+JiWG94bW86+C/w56CB5tMd1j+DWNI8BWEoJ1r3fS8zYUmUYZRDVQtyDRmhI/IMfQsFSUGH5bShCh9bJsaJVPYIg6fsz0RJUq3HaWSdKTEv+rc2If/T+rlJLsOSiSw3IOhsUZJzbCSe1I1jpoAaPraAUMXsWzF9IbYaYz9lYcvkbiMl11XNduP/buIveOi0/fO2d3fWuLqct7SODtERaiIfXaArdIu6qIco+nCWnW1nx/lysXvitmZW15lnDtDCuJ1vZp29jQ==</latexit>
1. approximate cost function
and equality constraints:
f (x) ' f (x
2. solve the resulting QP to get x(k+1)
3. go to (1) until convergence
(k)
) + [rf (x
| {z
d
<latexit sha1_base64="1BD0J+HtRGvnopalAYZPXguLRDE=">AAACdnicbVFdixMxFM2MX2t3das+ChIsxRaxzIjiPq744uMKdnehqUMmvdOGZjJjcmdpCfmhgn/Af+Cjme6AdtcLgZNzcu9JTvJaSYtJ8iOK79y9d//BwcPe4dGjx8f9J0/PbdUYAVNRqcpc5tyCkhqmKFHBZW2Al7mCi3z9qdUvrsBYWemvuK1hXvKlloUUHAOV9a/EaDOmzMoSvlPW6AWY3HABLvDf3Gg99mOfuTe5p6/3ZKagwAllRdg5VnODkisq/F+88czI5QqZCf6YuW6c99lHusn6g2SS7IreBmkHBqSrs6z/iy0q0ZSgUShu7SxNapy71kso8D3WWKi5WPMlzALUvAQ7d7t8PB0GZkGLyoSlke7YfzscL63dluGRw5Ljyt7UWvJ/2qzB4mTupK4bBC2ujYpGUaxoGzZdSAMC1TYALowMd6VixUNkGL5kz6WdjVWlrO+FbNKbSdwG528n6ftJ8uXd4PSkS+mAPCcvyYik5AM5JZ/JGZkSQX5GcXQYHUW/4xfxMH51fTSOup5nZK/i5A9h58CE</latexit>
1 T 2
)] x + x [r f (x(k) )]x
}
| {z }
2
(k)
T
G
@c
c(x) ' c(x ) +
x
| {z } @x x(k)
| {z }
b
GTA 2016/2017 — course slides — (C) 2010-2016 Rodrigo Ventura, IST
(k)
A
Descent methods for constrained problems
•
Exterior point methods:
•
replace constraints with a penalty P(x) to the cost for constraint violation
iterate unconstrained “minimize f(x) + μ P(x)” while bringing μ→∞
Interior point methods:
-
replace constraints with a barrier B(x) on the feasible set frontier
iterate unconstrained “minimize f(x) + B(x)/μ” while bringing μ→∞
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Exterior point method
•
Original constrained optimization problem:
minn f (x)
x2R
•
subject to
ci (x) = 0,
ci (x)  0
i 2 E,
i 2 I.
Replace constraints by quadratic penalty terms:
!
ci (x) = 0
ci (x)  0
•
(
!
Pi (x) = c2i (x)
Pi (x) = [max {0, ci (x)}]
2
This results in the following unconstrained problem, that has to be iteratively
optimized for μ→∞
minn f (x) + µ
x2R
X
Pi (x)
i
GTA 2016/2017 — course slides — (C) 2010-2016 Rodrigo Ventura, IST
Interior point method
•
Original constrained optimization problem:
minn f (x)
x2R
•
subject to
(
ci (x) = 0,
ci (x)  0
Replace inequality constraints by logarithmic barrier terms:
Bi (x) =
log ( ci (x)) ,
and equality ones by quadratic penalty terms:
Pi (x) = c2i (x),
•
i 2 E,
i 2 I.
i2I
i2E
This results in the following constrained problem, that has to be iteratively
optimized for μ→∞
X
1X
minn f (x) +
Bi (x) + µ
Pi (x)
x2R
µ
i2I
GTA 2016/2017 — course slides — (C) 2010-2016 Rodrigo Ventura, IST
i2E
Interior point method
•
•
Problem: how to obtain an initial feasible candidate, such that ci(x)≤0 ?
Solution: phase I method:
-
preliminary stage to find an initial feasible candidate, by solving the
problem:
min n s
s2R,x2R
s.t. ci (x)  s,
ci (x) = 0,
-
i2I
i2E
any x ∈ Rn and s = maxi ci(x) is a feasible start for this problem.
GTA 2016/2017 — course slides — (C) 2010-2016 Rodrigo Ventura, IST
Interior point method
•
Compare stationary point of interior point cost function
"
#
X
1X
r f (x)
log( ci (x)) + µ
c2i (x)
µ
i2I
i2E
X
X
1
=rf (x) +
rci (x) +
2ci (x)µrci (x) = 0
µci (x)
i2I
i2E
with KKT condition
rx L(x, , ⌫) = rf (x) +
•
Note that:
ci (x) < 0,
X
i2I
i > 0,
i rci (x) +
and
X
i2E
⌫i rci (x) = 0
1
! 0,
i ci (x) =
µ
⌫i
ci (x) =
! 0,
2µ
that is, all remaining four KKT conditions are satisfied at the limit
GTA 2016/2017 — course slides — (C) 2010-2016 Rodrigo Ventura, IST
i2I
i2E
Linear programming
•
362
Chapter 13.
The Simplex Method
Linear Programming (LP) is a convex constrained optimization problem:
minn cT x
x2R
c
s.t. A x = b
x
•
•
0
feasible polytope
optimal point x*
Notes:
-
matrix A is m by n where m<n
maximizing cT x is the same as minimizing -cT x
aT x ≤ b
Figure
13.1 A linear program in two dimensions with solution at x .
aT x + s = b and s ≥ 0 (i.e., s is a slack
variable)
∗
+-x14)
- has
+ x- for
unconstrained variables can be eliminated by setting
x=x
with
≥some
0 linear programming proble
methods (see
Chapter
proved toxbe ,faster
but the continued importance of the simplex method is assured for the foreseeable futu
There are extremely efficient methods to solve LPs, e.g., Simplex
LINEAR PROGRAMMING
typical complexity: O(n2m) for m≥n (n variables, m constraints)
GTA 2012/2013 — course slides —
Linear programs have a linear objective function and linear constraints, which
include both equalities and inequalities. The feasible set is a polytope, that is, a con
connected
(C) 2010-2013 Rodrigo Ventura,
ISTset with flat, polygonal faces. The contours of the objective function are pla
Figure 13.1 depicts a linear program in two-dimensional space. Contours of the objec
Linear programming
•
•
Lagrangian:
Lagrange dual function:
g(⌫, ) = min L(x, ⌫, )
x
n
o
T
= min c + AT ⌫
x ⌫T b
x
(
bT ⌫, AT ⌫
+c=0
=
1,
otherwise.
•
Lagrange dual problem:
which is a LP itself
max
L(x, ⌫, ) = cT x + ⌫ T (Ax
,⌫
bT ⌫
b)
T
s.t. AT ⌫ =
0
•
x
Strong duality holds if a feasible solution exists
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
c
Simplex method
•
•
A x = b is underdetermined, since A(m x n) and m < n
•
•
•
Let xB contain the components from x corresponding to the columns in B
•
Fundamental theorem of linear programming:
Let the columns of A be split into [ B | C ]
where B contain m linearly independent column of A
and C the n-m remaining ones
Then, B xB = b, since B spawns Rm
A vector x formed from xB with all other components set to zero is called
a basic feasible solution
-
if there is a feasible solution, then it is a basic feasible solution
if there is an optimal feasible solution, then it is an optimal basic feasible
solution
GTA 2016/2017 — course slides — (C) 2010-2016 Rodrigo Ventura, IST
Simplex method
•
3.3.
T hover
e S i m basic
p l e x Mfeasible
e t h o d 375
The Simplex method is an algorithm that1 iterates
solutions until an optimal one is found
c
0
1
simplex path
2
3
•
•
Guaranteed convergence in a finite amount of steps under mild conditions
Figure 13.3 Simplex iterates for a two-dimensional problem.
Typical computational complexity: O(n2m) for n variables, and m constraints
so by substituting in (13.23) we obtain
GTA 2016/2017 — course slidescT x—
+ (C)
! cT x2010-2016
− (c − s Rodrigo
)x + + c Ventura,
x + ! cT xIST
− s x +.
B
B
q
q
q
q q
B
B
q q
Mixed integer programming
•
A Mixed Integer Program (MIP) is a LP where some of the variables are
constrained to be integer — however, it becomes a NP-hard problem
•
•
A LP with binary variables is a special case of MIP
State-of-the-art MIP solvers are based on a combination of:
-
LP relaxation, that is, consider integer variables as real ones
combinatorial optimization and cutting plane methods, e.g., branch-and-cut
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Ground Holding Problem (GHP)
•
•
•
Problem statement: to assign ground delays to aircraft given airport capacity
Objective: minimize sum of ground and airborne delays (weighted costs)
Inputs:
•
scheduled departure and arrival times
departure, arrival, and enroute capacities
Output:
•
set of flights
amount of delay to impose to each flight
Approach: formulate problem as an optimization problem
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Single Airport GHP (SAGHP)
•
Simplification of GHP taking into account the following assumptions:
•
single arrival airport
no capacity limitations other than this airport
Model:
-
time horizon divided in a discrete set of periods (e.g., 15min each):
t ∈ 1, …,T
-
maximum arrival capacity of Mt flights during time period t
set of flights F, where each f ∈ F is scheduled to arrive at time af ∈ {1, …,T}
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Single Airport GHP (SAGHP)
•
Decision variables:
•
Xft ∈ {0, 1} (binary)
where Xft = 1, if flight f is assigned to arrive at time t
Xft = 0, otherwise
Constraints:
-
each flight is assigned to one and only one time period:
T
X
Xf t = 1
t=af
-
8f 2 F
airport capacity restriction at each time period
X
f 2F : t af
X f t  Mt
8 t 2 {1, . . . , T }
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Single Airport GHP (SAGHP)
flights
time
X11=1
X12=0
X13=0
X14=0
=1
X21=0
X22=0
X23=1
X24=0
=1
≤ M1
≤ M2
≤ M3
≤ M4
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Single Airport GHP (SAGHP)
•
Quantifying delay:
-
ground delay of flight f:
T
X
(t
af ) Xf t
t=af
-
nonlinear ground delay cost of flight f:
T
X
g(t
a f ) Xf t
t=af
example: quadratic cost: g(d)=d2
-
total cost (of objective) function:
X
f 2F
Cf
T
X
t=af
where Cf is a per-flight cost weight (of flight f)
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
g(t
af ) Xf t
Single Airport GHP (SAGHP)
•
Optimization problem:
minimize
subject to
X
Cf
T
X
g(t
f 2F
t=af
T
X
Xf t = 1
t=af
X
f 2F : t af
X f t  Mt
Xf t 2 {0, 1}
•
af ) Xf t
8f 2 F
8 t 2 {1, . . . , T }
8 f 2 F, t 2 {1, . . . , T }
Since both objective function and constraints are linear on variables Xft, this is an IP
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Multi Airport GHP (MAGHP)
•
Extension of SAGHP with:
•
multiple departure and arrival airports
capacity limitations on both departure and arrival rate
couplings between flights
Model:
-
time horizon divided in a discrete set of periods: t ∈ 1, …,T
-
set of flights F, where each f ∈ F is scheduled to depart at time df ∈ {1, …,T}
and arrive at time rf ∈ {1, …,T}
maximum departure and arrival capacities of Dt and Rt flights, respectively,
during time period t
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Multi Airport GHP (MAGHP)
•
•
Decision variables:
-
Utf ∈ {0, 1} and Vtf ∈ {0, 1} (binary)
-
and
where Utf = 1, if flight f is assigned to depart at time t
Utf = 0, otherwise
Vtf = 1, if flight f is assigned to arrive at time t
Vtf = 0, otherwise
Other parameters:
-
slack time Sf after flight f before next flight (couplings)
costs of cGf and cAf per unit of delay time for ground in air holding
respectively
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Multi Airport GHP (MAGHP)
•
Optimization problem:
minimize
where
GTA 2016/2017 — course slides — (C) 2010-2016 Rodrigo Ventura, IST
Multi Airport GHP (MAGHP)
•
Optimization problem:
subject to
where
GTA 2016/2017 — course slides — (C) 2010-2016 Rodrigo Ventura, IST
Optimization solvers
•
Optimization problem may be specified in a text file, including
-
objective function
constraints
variable domains (continuous, integer, or binary)
Solver
problem
specification
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
optimal
solution
CPLEX LP format
•
CPLEX is a well-known LP solver, first released by ILOG in 1988, nowadays
supported by IBM
•
CPLEX file format, supported by many LP/MIP solvers:
-
objective function section
example:
minimize 2 x1 + 3 x2
-
constraints section
example:
subject to
- 4 x1 + x2 <= 0
-
bounds section
example:
bounds
x1 >= 0
-
variable domain section
example:
binary
x2
-
close specification with “end”
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
GLPK solver
•
•
•
GNU Linear Programming Kit — open source LP and MIP solver
•
•
Accepts several file formats, including CPLEX
Website: http://www.gnu.org/software/glpk/
Can be used in various ways: standalone, as a library, from MATLAB (using
glpkmex), from Python, etc.
Example of standalone usage from the command line:
glpsol --lp -o simple.out simple.cplex
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
SCIP solver
•
•
•
•
•
Solving Constraint Integer Programs (SCIP) — open source MIP solver
Handles both linear and non-linear problems
Website: http://scip.zib.de/
Accepts several file formats, including CPLEX and their own ZIMPL format
Example in ZIMPL code:
var x1;
var x2 binary;
minimize cost: 2*x1 + 3*x2;
subto constraint: -4*x1 + x2 <= 0;
subto bounds: x1 >= 0;
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
linprog
MATLAB
Solve linear programming problems
Linear programming solver
•
•
Linear program:
Finds the minimum of a problem specified by
, , ,
Syntax:
,
, and
are vectors, and
and
are matrices.
Syntax
x = linprog(f,A,b)
x = linprog(f,A,b,Aeq,beq)
= linprog(f,A,b)
x = xlinprog(f,A,b,Aeq,beq,lb,ub)
x = xlinprog(f,A,b,Aeq,beq,lb,ub,options)
= linprog(f,A,b,Aeq,beq)
x = linprog(problem)
x = linprog(f,A,b,Aeq,beq,lb,ub)
[x,fval] = linprog(___)
x = linprog(f,A,b,Aeq,beq,lb,ub,options)
[x,fval,exitflag,output]
= linprog(___)
x = linprog(problem)
[x,fval,exitflag,output,lambda]
= linprog(___)
[x,fval] = linprog( ___ )
[x,fval,exitflag,output] = linprog( ___ )
GTA 2016/2017 — course slides — (C) 2010-2016 Rodrigo Ventura, IST
[x,fval,exitflag,output,lambda] = linprog( ___ )
Python + SciPy
•
Linear program:
min cT x
x
s.t. Aub x  bub
Aeq x = beq
x
•
•
0
(by default)
SciPy package: scipy.optimize.linprog
Syntax:
linprog(c, A_ub=None, b_ub=None, A_eq=None, b_eq=None,
bounds=None, method='simplex', callback=None, options=None)
GTA 2016/2017 — course slides — (C) 2010-2016 Rodrigo Ventura, IST
Encoding a SAGHP in CPLEX
•
Example 1: 2 flights, 4 time periods, Mt=1, a1=1, a2=3, Cf=1, g(d)=d2
minimize x12 + 4 x13 + 9 x14 + x24
subject to
x11 + x12 + x13 + x14 = 1
x23 + x24 = 1
x11
<= 1
x12
<= 1
x13 + x23 <= 1
x14 + x24 <= 1
binary
x11
x12
x13
x14
x23
x24
end
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Encoding a SAGHP in CPLEX
•
Example 2: 2 flights, 4 time periods, Mt=1, a1=1, a2=1, Cf=1, g(d)=d2
minimize x12 + 4 x13 + 9 x14 + x22 + 4 x23 + 9 x24
subject to
x11 + x12 + x13 + x14 = 1
x21 + x22 + x23 + x24 = 1
x11 + x21 <= 1
x12 + x22 <= 1
x13 + x23 <= 1
x14 + x24 <= 1
binary
x11
x12
x13
x14
x21
x22
x23
x24
end
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Encoding a SAGHP in CPLEX
•
Example 3: 4 flights, 4 time periods, Mt=1, a1=1, a2=1, a3=1, a4=3, Cf=1, g(d)=d2
minimize x12 + 4 x13 + 9 x14 +
x22 + 4 x23 + 9 x24 +
x32 + 4 x33 + 9 x34 +
x44
subject to
x11 + x12 + x13 + x14 = 1
x21 + x22 + x23 + x24 = 1
x31 + x32 + x33 + x34 = 1
x43 + x44 = 1
x11 + x21 + x31
<= 1
x12 + x22 + x32
<= 1
x13 + x23 + x33 + x43 <= 1
x14 + x24 + x34 + x44 <= 1
binary
[…]
end
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Encoding a SAGHP in CPLEX
•
Example 4: 5 flights, 4 time periods, Mt=1, a1=1, a2=1, a3=1, a4=3, a5=2, Cf=1, g(d)=d2
minimize x12 + 4 x13 + 9 x14 +
x22 + 4 x23 + 9 x24 +
x32 + 4 x33 + 9 x34 +
x44 +
x53 + 4 x54
subject to
x11 + x12 + x13 + x14 = 1
x21 + x22 + x23 + x24 = 1
x31 + x32 + x33 + x34 = 1
x43 + x44 = 1
x52 + x53 + x54 = 1
x11 + x21 + x31
<= 1
x12 + x22 + x32
+ x52 <= 1
x13 + x23 + x33 + x43 + x53 <= 1
x14 + x24 + x34 + x44 + x54 <= 1
binary
[…]
end
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Encoding a random example
•
•
•
•
24h in 10 min periods → T = 144
2 runways, 2 min separation → Mt = 10
1000 flights with scheduled times uniformly distributed → af ~ Uniform(1,T)
Sample problem:
-
3.3MB specification file
72098 variables and 1144 constraints
0.5 seconds (Intel Core 2 Duo @ 2.2GHz)
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
LP relaxation of SAGHP
•
A matrix is totally unimodular if the determinant of every submatrix is
either -1, 0, or 1.
•
If A is totally unimodular and b is an integer vector, then A x ≤ b defines
a polytope with all vertices in integer coordinates
•
Therefore, the LP defined by
minimize cT x
subject to
Ax≤b
x≥0
has an integer solution
•
Given that the SAGHP results in a unimodular LP, the solution of the LP
relaxation coincides with the solution with binary variables!
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
Other problems
•
•
•
•
extend model to include departure, enroute, and arrival capacity restrictions
•
flight planning
(including the trade-off between route length and air navigation charges)
stochastic models to cope with uncertainty in airport capacity
slot assignment to airliners
banking constraints, e.g., to assure transfer in hub airports
(delay banking is a system where delayed aircraft due to flow restrictions obtain
credits, that can be bid against other operators to get higher priority in
congestioned airports)
GTA 2012/2013 — course slides — (C) 2010-2013 Rodrigo Ventura, IST
The End
Thank you!