Teaching a Traditional Optimization
Course in a non-Traditional Manner
James B. Orlin
Mike Metzger
MIT
November 6, 2006
1
Part I: Introduction
Quote of the day.
If you treat individuals as they are they will remain as
they are, but if you treat them as if they were what
they ought to be and could be, they will become what
they ought to be and could be.
– Johann Wolfgang von Goethe.
2
Optimization Methods in Management Science
I’ve scheduled league games
 Models, applications, and
for a major basketball
algorithms for optimization conference. There are an
within Management
incredible number of
constraints, and it is
Science.
very hard to solve.
– finance, manufacturing,
transportation,
timetabling, games,
operations, e-business,
and more.

Linear programming,
integer programming,
network optimization, and
decision analysis.
Mike Trick
3
Syllabus
Topic
Introduction
Applications of Linear
and Non-Linear
Programming
Geometry of LP
Simplex Method 1
Simplex Method 2
Sensitivity Analysis
LP Duality 1
LP Duality 2
Game Theory
Introduction to
Networks
Maximum Flows.
Min Cost Flows
The Network Simplex
Algorithm
IP Models
More IP Models
Branch and Bound
Cutting Planes
Decision Trees 1
Decision Trees 2.
4
Our Goals Today

Lots of pedagogical advances over the years.
– lot of description at INFORMS meetings

Focus on a range of techniques
–
–
–
–
–

visual learning,
active learning
interactive learning
use of technology
and more
End result:
– students like the material better
– more interested in follow on subjects
– superior mastery of the material over students in
previous years
5
Overview of the Structure of the Class

2 Lectures per week per section
(1.5 hours, required)
– 1 small morning section, one large afternoon
section
– MIT students are used to 1 hour per lecture

Required recitation
– weekly quizzes

Nearly weekly homework

2 midterms and a final (and a project)

1 professor, 3 TAs. 150 students
6
Structure of a Lecture

Intro
– quotes
– announcements

First half of class

Mental break

Second half of class

game to review the class material
7
Time line for 15.053
1995: Algorithm based
1996: taught 15.053 as an
Excel-based class
1999: taught 15.053 as an
algorithms based class
2001: introduced
PowerPoint into classes;
Mike Metzger was a
student in the class
8
Time line continued
2004: taught both
sections of 15.053 in
the spring. Mike
Metzger was one of my
TAs. Hired undergrad
graders. TAs helped
develop educational
materials
2005: introduced required
recitations.
2007. dropped required
text and replaced with
our own notes
CLASS
15.053
class
notes
9
Part II: Pedagogy
Imagination is more important than knowledge...
• Albert Einstein
You don't understand anything until you learn it
more than one way.
• Marvin Minsky
10
Overview of Pedagogy

Pedagogy has evolved
from straight lecturing

Techniques adopted
– Engaging Model
Formulations
– Algorithm animation using
PowerPoint
– Use of games in class
– Active learning (talk to
your neighbor)
The Jetson’s diet problem.
Who wants a
piece of
of candy?
piece
TV’s hit quiz show
comes to 15.053
11
A diet problem and its dual

Jane Jetson is trying to design a
diet for George on her home food
dispenser so as to satisfy
nutritional requirements and
minimize cost.
Theme music
Space
Burgers
Saturn
Shakes
Venus
Flies
Requirements
Carbs
1
4
0
9
Fat
4
2
0
8
Protein
4
1
2
12
Cost
(in $100s)
6
6
2
12
Spacely Sprockets has decided to
make food pellets. Mr. Spacely
wants to win over Jane Jetson by
making, carbohydrate, fat, and
protein pellets. How should he price
them to win over Jane and maximize
profit?
Carbs
Fat
Protein
Upper
bound
on Cost
Space Burgers
1
4
4
6
Saturn Shakes
4
2
1
5.5
Venus Flies
0
0
2
2
Choose Prices
13
Discuss with partners

Usual:
– left of the class solves Jane’s primal problem as
best as possible by inspection
– right half of the class solves Spacely’s dual
problem by inspection
– we collect answers from both, and observe that the
primal opt is >= dual opt.

Do you have any other ways of getting students
to understand the intuition behind weak duality?
– (2 minutes)
14
Algorithm Animation

Powerpoint has lots of animation tools that are
useful for illustrating concepts.
Boxes can
appear, and
disappear.
Colors and transitions
can simulate movement.
And one can use real
movement.
And pictures can help.
15
2 Variable Simplex Illustrated
Maximize z = 3 K + 5 S
Start at any feasible corner point.
S
Move to an adjacent corner point with
better objective value.
5
Continue until no adjacent corner point
has a better objective value.
4
3
2
1
1
2
3
4
5
6
K
16
An Alternative Representation
Introducing:
S
LPac Man
5
4
3
2
1
1
2
3
4
5
6
K
17
Algorithm Animation: Simplex Pivoting
z
x1
x2
x3
x4
1
-3
3
-2
2
0
0
=
Non-basic
variable x2
becomes basic.
0
Choose column 2.
0
-3
3
1
0
=
6
0
-4
2
0
1
=
2
z = 0, x1 = 0, x2 = 0, x3 = 6, x4 = 2.
Pivot on the 2.
Basic variable
x4 becomes
non-basic. It
was the basic
variable in
Constraint 2.
Choose constraint 2.
18
Pivoting to obtain a better solution
New Solution: basic
variables z, x2 and x3.
Nonbasics: x1 and x4.
-z
z
x1
x2
x3
x4
1
-1
3
-2
0
0
0
1
=
0
2
0
-3
3
3
0
1
-1.5
0
=
6
3
0
-4
-2
2
1
0
.5
1
=
2
1
19
What is the maximum number of diamonds that can be
packed on a Chinese checkerboard.
class exercise:
how many can you
pack?
Each diamond
covers 4 dots.
20
Here is a best possible
21
For each circle, you can
select at most one
diamond incident to the
circle.
5 slides later
10
19 6 2117
53 84
12
22
An Excellent Integer Program
Let S(i) be the diamonds that are incident with circle
i of the Chinese Checkerboard.
zIP = Max
s.t.


d D
xd
d S ( i )
xd  1
x d  {0,1}
for each circle i
for all d  D
zLP = 27.5
And we found a
solution by hand
with zi = 27
23
Representing the Dual Problem
Assign each circle (or node)
a “weight.”
The weight of a diamond d is
w(d) = the sum of its node
weights.
Class exercise: assigning
¼ to each circle gives a
total weight of 30.25. Can
you do better?
The dual linear program:
choose weights for each
circle so that w(d) ≥ 1 for
each diamond d. Minimize
the weights of the circles.
24
Node
Weight
18
1
1
1/2
24
6
1/3
1/6
0
Each diamond has
weight at least 1.

The number of diamonds is at most 27.
j
w j  27.5
25
Who wants a piece of
CANDY?
TV’s hit quiz show
comes to 15.053
26
Who wants another piece of candy
What is the advantage of using algorithm
animation?
A
C
50-50
It helps students focus
on what is important
It is effective for
students who are
visual thinkers
B
It keeps students
engaged
D
A, B, and C.
27
15.053
Who wants another piece of candy
What is the advantage of using algorithm
animation?
A
C
50-50
B
It is effective for
students who are
visual thinkers
D
A, B, and C.
28
15.053
Who wants another piece of candy
What is the advantage of using algorithm
animation?
A
B
C
D
A, B, and C.
50-50
Next question
29
15.053
Who wants another piece of candy
What is the advantage of using algorithm
animation?
A
B
C
D
A, B, and C.
50-50
Next question
30
15.053
Who wants another piece of candy
Which valid
constraint
What
is the
advantage of 1using algorithm
4
7
should be added to the
animation?
integer programming
formulation of the TSP to
eliminate this solution.
A
 
3
B
3
6
i 1
j 4
x ij  2
C
5
 
6
3
9
i 1
j 4
8
9
x ij  2
D
Both A and B
50-50
2
Neither A nor B
31
15.053
Who wants another piece of candy
Which valid
constraint
What
is the
advantage of 1using algorithm
4
7
should be added to the
animation?
integer programming
formulation of the TSP to
eliminate this solution.
A
2
3
B
C
5
 
6
3
9
i 1
j 4
8
9
x ij  2
D
Both A and B
50-50
32
15.053
Who wants another piece of candy
Which valid
constraint
What
is the
advantage of 1using algorithm
4
7
should be added to the
animation?
integer programming
formulation of the TSP to
eliminate this solution.
A
C
50-50
2
3
B
5
 
6
3
9
i 1
j 4
8
9
x ij  2
D
Last Slide
33
15.053
Who wants another piece of candy
Which valid
constraint
What
is the
advantage of 1using algorithm
4
7
should be added to the
animation?
integer programming
formulation of the TSP to
eliminate this solution.
A
C
50-50
2
3
B
5
 
6
3
9
i 1
j 4
8
9
x ij  2
D
Last Slide
34
15.053
Mental Break: Impossibilities
Fooling around with alternating current is a
waste of time. Nobody will use it, ever.
Thomas Edison
Rail travel at high speed is not possible because
passengers, unable to breathe, would die of
asphyxia.
Dr. Dionysus Lardner, 1793-1859
Inventions have long since reached their limit,
and I see no hope for future improvements.
Julius Frontenus, 10 A. D.
35
Recitations, Tutorials, Homework, Exams, Etc.
The role of the Head TA
36
Structure




Quiz
Statistical Break
Lesson
Summary
37
38
Quiz

4.5
4
3.5
E
3
D
2.5
C
2

Problem 1 (4 points total; 2 points each)
If exactly one of these points is optimal, which
point(s) could it be?
If point B is optimal, must any other of these
point(s) be optimal too? If so, which one(s)?
x

2
G
1.5
B
1
0.5
0
A
0
F
0.5
1
1.5
2
2.5
x1
3
3.5
4
4.5
39
Statistical Break

Break Between Quiz and Lesson
– It takes 5 minutes for them refocus
• Interesting statistics!

Examples:
– Favorite Airline
– Exam Scores
– Have You Heard of Welcome Back Kotter
40
Quiz Grades
20
18
18
16
14
13
12
10
8
6
6
5
4
2
0
0
10
9
8
7
<7
41
Have you
heard of
Back Kotter?
Have you
heard
ofWelcome
Welcome
Back Kotter
Yes.
1010
Yes,
No, 3890
No.
42
Recitation-”The Dual”


Consider the Following Statements
Try to Determine if they Are True or False
– It is possible for an LP to have exactly 2 optimal
solutions
– Two different bases (and thus two different
tableaus) can have the same corresponding basic
feasible solution
– We can have negative z-row coefficients and still
be at an optimal solution.
43
Problem 1: More Simplex Tableau
Suppose while solving a maximization problem we obtain the following tableau, with the
basic variables highlighted in blue:
z
1
0
0
0
x1
a
0
1
0
x2
c
d
-5
e
x3
0
0
0
1
x4
3
4
-1
2
x5
0
1
0
0
rhs
10
5
b
3
Give conditions on the missing values a, b, c, d, e and required to make the each of the
following statements true:
a) The current tableau represents a basic feasible solution in canonical form.
b) The current basic solution is an optimal solution, but the set of optimal solutions
is unbounded.
c) The current basic solution is a degenerate basic feasible solution.
d) The current basic solution is feasible, but the objective function value can be
44
improved by bringing x2 into the basis and pivoting x3 out.
Pivot- Geometry
45
Applications
(Before)
Ebel Mining Company owns two different mines
that produce a given kind of ore. After crushing,
ore is one of three grades: high, medium or low.
Ebel must provide the parent company with at
least 12 tons of high grade, 8 tons of medium
grade, and 24 tons of low-grade ore per week. It
costs $20,000 a day to run the first mine and
$16,000 a day to run the second mine. In one day
each mine produces the following tonnage:
Mine 1
Mine 2
High
6
2
Medium
2
2
Low
4
12
46
Applications
After.
Dear 15.053 Class,
My name is Jessica Simpson. I have been going through some
tough times recently and am having a real problem with one of
my cosmetic lines. The info for the line is on the next page.
Recently though costs are changing based on market demand
in addition to highly fluctuating resource costs. My problem is
this we currently have an LP that we solve to find the optimal
amount to produce of each product. However, every time a
parameter changes, I am always forced to resolve the LP and
this takes too long. I was hoping you guys could find a better
way. Lately I have just been out of it. For example, Nick and I
decided to split our Hummer in half, and now I need to buy a
new one. Ohh yea, about the LP it seems to have been
misplaced when I was moving out of my Malibu house. Please
Help!
- Jessica
47
Interaction


Video clip
More about video and educational technology
later
48
Initial Concerns


Students bored in recitation
Rebellion to required recitation
– Against MIT Grain
49
What We Found

Main Lesson
– By putting out a good product, students didn’t
complain.
– If they are learning, they will be happy
– The Fun Factor helped out
– You can’t please everyone
• But you can please 95%
50
Office Hours

Structure
– Twice per week for two hours each
– Attendance can be as high as 60 Students
– One TA leads each session
• Team teaching is a new approach
– Small vs. large session
– Goal: To help students understand the material on
the homework sets
– Management style
51
Example
According to recent reports in US Weekly, after 12
consecutive nights of partying, Jessica has
decided to leave Nick. Currently they are trying to
determine how to optimally divide some of their
possessions. They hire “you” to try and help
them accomplish this task. To start, you ask them
to assign “value points” to all of the items under
consideration such that they total 100. Their
allocation is shown in the table below.
52
The Use of Excel

The main question
– How to balance Excel Vs. Theory

Students:
– Have disliked Excel
– However, after completing our course often say it
is the most useful tool we taught.
53
The Use of Excel

Staff
– Hard to grade Excel
– Very important for Management students

A new approach
– Take Excel and separate from problem sets and
Use as a case study
– To be tried in spring 2007
54
Our Idea

The Diet Problem as an ongoing example and
case study
–
–
–
–
–
Formulate your problem
Solve using simplex
Take dual and interpret as Grocer’s Problem
Study distribution costs using networks
Look into decision modeling with stochastic
supply
55
Modeling and Abstract modeling

More emphasis on modeling abstract scenarios

Building managerial skills while building on
theory

Framing the problem
Min
S cij xij
s.t.
S aij xj <= bi for all i
x >= 0
56
Exams-Testing the “Why” not “How”

JB was performing the simplex method; however,
at exactly one intermediary step, he made the
following error: instead of pivoting in a variable
with a negative objective row coefficient, he
instead pivoted in a variable a with a positive
objective row coefficient. He chooses the exiting
variable using the usual minimum ratio test.
Assume after this initial error JB continues to
solve the problem using the simplex method and
does not make any additional errors. Also
assume that no basis is degenerate.
57
Testing the “why” not “how”



It is possible that the pivot that JB took will
result in a basic solution that is not feasible.
It is possible that the pivot JB took will result in
an optimal basis.
It is possible that there are fewer simplex pivots
with the error than if the error had not been
made.
58
Exam Management
Testing depth of knowledge without penalizing
students who understand the basics
Our Solution: Short Answer Questions
– Say we have an LP with only one variable.
Which of the following could be the set of all
feasible solutions?
•
i)
A single point, e.g. 2
•
ii)
A line segment, e.g. [2, 4]
•
iii) Two disjoint line segments, e.g, [2, 3]  [6, 7].
•
iv) A ray or infinite half line, e.g. [2, ∞)
59
Suppose we have a primal-dual pair where
each has a finite optimal solution. Say we
remove the ith constraint for the primal.
What can you say about the dual
corresponding to this new problem?
– i)The dual could be infeasible.
– ii)The dual could still have an optimum solution.
– iii)The dual could be unbounded.
60
Use of Technology

Three recent additions to the subject
– Online PowerPoint reviews
– Homework solution podcasts
– Online videos of recitations and reviews
61
Online PowerPoint Reviews

An alternative to standard textbook readings
– Solution: Online Tutorials

Topics:
–
–
–
–
–
–
Abstraction of LP models
Solving systems of equations
Converting an LP to standard form
Degeneracy
Sensitivity analysis using Excel
Integer programming formulations
62
With the Assistance of

Some Friends
63
Degeneracy in Linear Programming
I heard that
today’s
tutorial is all
about Ellen
DeGeneres
Sorry, Tim. But
the topic is just as
interesting. It’s
about degeneracy
in Linear
Programming.
Degeneracy? Students
at MIT shouldn’t learn
about degeneracy.
And I heard that
15.053 students have
already studied
convicts’ sex.
Reverend Jerry Falwell
Ellen DeGeneres
Tim, the turkey
Actually, they
studied convex
sets.
64
What is degeneracy?

As you know, the simplex algorithm starts at a corner point and
moves to an adjacent corner point by increasing the value of a
non-basic variable xs with a negative value in the z-row
(objective function).

Typically, the entering variable xs does increase in value, and
the objective value z improves. But it is possible that that xs
does not increase at all. It will happen when one of the RHS
coefficients is 0.

In this case, the objective value and solution does not change,
but there is an exiting variable. This situation is called
degeneracy.
A basic feasible
z
x1 x2 x3 x4
1
3
-2
0
0
=
2
0
-3
3
1
0
=
6
0
-4
2
0
1
=
2
0
This bfs is degenerate.
solution is called
degenerate if one
of its RHS
coefficients
(excluding the
objective value) is
65
0.
No, Nooz**. This tutorial has many
more slides. Degeneracy adds
complications to the simplex
algorithm. And if you understand
what occurs under degeneracy, you
really understand what is going on
with the simplex algorithm.
Great. I now
know what
degeneracy is.
Now we can move
on to other
matters.
** As you know, “No,
Nooz” is good news.”
Ollie,
the computationally
wise owl.
Nooz, the most
trusted name in fox.
Professor Orlin
apologizes for
this bad pun, but
feels that he
could not resist
the temptation.
Cleaver
66
PodCast’s

MIT Project Launched in Spring 2006
– 15.053 selected as launch course for homework
solutions via podcasts
– Over 85% reported using videos on regular basis
– Turned HW Solutions into tutorials
– Review Session 2 Set the current MIT PodCast
download record in a single night
• Crashed Server!
67
PodCast’s

Show Example
68