EMIS 8381/SMU
Nonlinear Programming
Summer 2005
Instructor: Dr. Yanjun Zhang
Syllabus

Week 1 (May 25)
o Overview
o Section 1.2. Mathematical background
o Introduction to SVM

Week 2 (June 1)
o Section 2.1. Necessary and sufficient onditions for minimizers
o Section 2.3. Steepest descent method. Line search method.
o Section 3.1. Newton method
o Section 7.1. Constrained optimization
o Section 9.1. Langrange Multipliers. Kuhn-Tucker conditions.
o Section 9.2. First-order conditions
o Section 9.3. Second-order conditions
o Homework #1 (due in a week)

Week 3 (June 8)
o Section 9.4. Convexity and KT conditions
o Section 9.5. Duality and Wolfe dual
o Application: Support Vector Machine model (SVM)
o Introduction to SVM-light for text classification
o Project Problem: Show the equivalence of two strategies for selecting
working set in SVM-light.
o Project Problem: Understand the QP solvers used in SVM-light
o Section 8.1. Linear programming
o Section 8.2. Simplex method
o Section 8.4. Feasible points for linear constraints
o Homework #2. Discussion of HW #1.

Week 4 (June 15)
o Section 10.1. Quadratic programming (QP)
o Section 10.2. Langrangian methods for QP
o Section 10.6. Dantzig-Wolfe algorithm for QP as an extension of simplex
method
o Introduction to SMO algorithm (Sequential Minimal Optimization)
o Homework #3. Discussion of HW #2.

Week 5 (June 22)
o Section 10.3. Active-set method
o Section 10.5. Least square problem
o
o
o
o
Section 2.4. Conjugate direction method
Section 4.1. Conjugate gradient methods
Application: Conjugate gradient methods for solving SVM
Project Problem: Reading a paper by Linda Kaufman on solving SVM by
conjugate gradient methods.
o Homework #4. Discussion of HW #3.

Week 6 (June 29)
o Discussion of HW #4
o Detailed discussion of the algorithms and implementations of SVM
solvers (SVM-light and SMO).

Week 7 (July 6)
o Problem work-out (1 hour)
o Mid-term exam (2 hours with time extension)

Week 8 (July 13)
o Discussion of mid-term
o Section 12.1. Penalty and barrier methods
o Application: Interior point method for QP
o Project Problem: Reading on interior point method for solving SVM

Week 9 (July 20)
o Nonlinear programming problems in other fields
o financial engineering
o airline’s revenue management
o other applications

Week 10 (June 27)
o Project report: A student may have 30-45 min to present his project. I
can discuss the projects of distance students (if I have the report a few
days ahead).