Convex Optimization
Chapter 1 Introduction
What, Why and How
What is convex optimization
 Why study convex optimization
 How to study convex optimization

What is Convex Optimization?
Mathematical Optimization
Nonlinear Optimization
Convex Optimization
Least-squares
LP
Mathematical Optimization
Convex Optimization
Least-squares
Analytical Solution of Least-squares
f 0 (x) = jjAx ¡ bjj 2 = (Ax ¡ b) > (Ax ¡ b)
2
@f 0 ( x )
@x
= 2A > (Ax ¡ b) = 0
x = (A > A) ¡ 1 A > b
Linear Programming (LP)
Why Study Convex Optimization?
Solving Optimization Problems
Mathematical Optimization
Nonlinear Optimization
Convex Optimization
Least-squares
LP
Mathematical Optimization
Nonlinear Optimization
Convex Optimization
Least-squares
LP
•
•
•
•
Analytical solution
Good algorithms and software
High accuracy and high reliability
Time complexity: C  n 2 k
A mature technology!
Mathematical Optimization
Nonlinear Optimization
Convex Optimization
Least-squares
LP
•
•
•
•
No analytical solution
Algorithms and software
Reliable and efficient
Time complexity: C  n 2 m
Also a mature technology!
Mathematical Optimization
Nonlinear Optimization
Convex Optimization
Least-squares
LP
•
•
•
•
No analytical solution
Algorithms and software
Reliable and efficient
Time complexity (roughly)
 max{ n3 , n 2 m, F}
Almost a mature technology!
Mathematical Optimization
Nonlinear Optimization
Convex Optimization
Least-squares
•
•
•
•
•
LP
Sadly, no effective methods to solve
Only approaches with some compromise
Local optimization: “more art than technology”
Global optimization: greatly compromised efficiency
Help from convex optimization
1) Initialization 2) Heuristics 3) Bounds
Far from a technology! (something to avoid)
Why Study Convex Optimization
If not, ……
there 
is little chance you can solve it.
-- Section 1.3.2, p8, Convex Optimization
How to Study Convex Optimization?
Two Directions

As potential users of convex optimization

As researchers developing convex
programming algorithms
Recognizing least-squares problems

Straightforward: verify



the objective to be a quadratic function
the quadratic form is positive semidefinite
Standard techniques increase flexibility

Weighted least-squares

Regularized least-squares
Recognizing LP problems

Example:
t i = jr i j

Sum of residuals approximation

Chebyshev or minimax approximation
t = maxi ja> x ¡ bi j
i
Recognizing Convex Optimization
Problems
An Example
C 10
P Adding mP linear constraints?????
10 p · 1
m p
8f j 1 ; j 2 ; ¢¢¢; j 10 g
j
k= 1
j=1 j
2
k
Summary
From the book, we expect to learn



To recognize convex optimization problems
To formulate convex optimization problems
To (know what can) solve them!