Introduction to Constrained Optimization
Doesn’t that sound like something you’d go a long way to avoid?
Q: What is constrained optimization?
A: An umbrella term for the set of algorithms that search for a best solution (optimization) while obeying
limits placed on the solution (constraints).
Q: Does that mean that the transportation model is version of constrained optimization?
A: Yes.
Q: So what are we going to cover now?
A: A more complex algorithm called Linear Programming.
Q: What makes linear programming (LP) more complex than the transportation model?
A: LP can handle more than simple supply and demand constraints. Unfortunately, this makes the
calculations more complex than the transportation algorithm.
Q: Do we have to learn these calculations?
A: No.
Q: Why not?
A: Using what you learned about the transportation model, we can argue by analogy to explain what LP
does.
Q: How does LP work?
A: Let’s break the words down to find out.
Q: Does “programming” mean giving instructions to a computer?
A: No, it is from an old, military term that means “planning.” So “linear programming” means “linear
planning.”
Q: What does “linear” mean?
A: “Linear” refers to the type of equations we will be using.
Q: What is a linear equation?
A: One that follows the form:
Y = mX + b.
Q: Is the following also a linear equation:
3X + 4Y = 240?
A: Yes.
Page 1 of 10
116099766, Page 2 of 10
Q: How do we know?
A: The exponent on the variables is 1 (zero is also allowed, but that is a trivial case). The important point
is that the exponent isn’t 2, or 3, or something higher, and the exponent isn’t less than one, such as ½,
or 1/3, or some other fraction. Also, the variables are added together, not multiplied or divided. So
neither
3X * 4Y = 240
nor
3X/4Y = 240
are linear equations. The first is a quadratic equation, and the second is an asymptotic equation.
Q: Are all business world relationships linear?
A: No, many of them are not.
Q: Why, then, do we use linear equations?
A: Mostly because they are simple (relative to non-linear equations) to solve.
Q: Does linear programming follow the same outline as the transportation model?
A: Yes.
Q: What is the first step in the LP algorithm?
A: Organize the data.
Q: Do we set up a table, as we did in Transportation?
A: No, this time we set up a set of equations. This is called “formulating” or because a formula is another
word for an equation.
Q: Why do we have to set up the equations?
A: The computer cannot understand sentences. Another phrase to describe organizing the data is “putting
it in computer ready form.”
Q: How do you formulate a problem?
A: You should have learned that in an algebra class somewhere. If you don’t remember, see the lecture
notes on “Formulating LP Problems.”
Q: Could you show us an example?
A: Look at the following word problem:
You run a small pottery business, making ceramic bowls ($2.20 profit) and cups ($2.50 profit). Your
only raw material is the clay you use. A cup uses 500 grams of clay, while a bowl uses 1 & 1/4
kilograms. It takes you 25 minutes to shape a bowl on your wheel and 45 minutes to shape a cup. A
bowl must be “fired” for an hour and a cup for 48minutes in your kiln, which can hold 2 items at a
time. You have on hand 40 kilograms of clay and you expect to put in 30 hours at the wheel next week.
116099766, Page 3 of 10
The kiln must be preheated every day, so it is available only 20 hours per week. Set up your
production plan for next week.
The formulation for this problem is:
B  # of bowls to make
C  # of cups to make
Max Z = $2.20 B
s.t.
1250 B
25 B
60 B
B,
+
+
+
+
$2.50
500
45
48
C
C
C
C
C,
<
<
<
>
40,000
1,800
2,400
0
grams of clay
minutes - wheel
minutes – kiln
non-negativity
Q: Is this the solution to this problem?
A: No, this is the formulation. Think of it as setting up the transportation table, before putting in the
initial solution.
Q: Is it possible to make the problem easier to look at and understand?
A: Yes, we could draw a graph of the equations.
Q: How do we graph the equations?
A: Again, you should have learned that in algebra class somewhere. If you don’t remember, see the
lecture notes on “Graphing LP Problems.”
Q: What is the graph for these equations?
A: See Figure 1:
Variable C
90
80
Clay
70
Kiln
60
Wheel
50
40
30
Objective
Function
10
10
20
30
40
Figure 1: LP Graph
50
60
70
80
90
Variable B
116099766, Page 4 of 10
Q: What does this graph show?
A: Every point on the graph is a possible solution to the problem and the shaded area shows all the points
(possible solutions) that satisfy all of the constraints.
Q: How is every point a possible solution?
A: A solution is value for B and a value for C. Every point on the graph has both a value for B (measured
along the horizontal axis) and a value for C (measured along the vertical axis), so every point is a
solution.
Q: How many feasible solutions are there?
A: An infinite number of points lie within the solution space.
Q: How do we find the best one?
A: Since you probably don’t want to test them one at a time, you need to narrow down the search area.
Q: Do we add more constraints?
A: No, we use the objective.
Q: Haven’t we already drawn in the objective?
A: Yes, but to do that we arbitrarily picked a Z-value (total profit) of 100 (see the lecture notes “Graphing
LP Problems” if you don’t remember this).
Q: What would have happened if we picked a different value for Z?
A: We would have drawn a line parallel to the one we drew, but closer in if we picked a smaller Z-value
or further out if we picked a larger Z-value.
Q: What does it mean if the objective function line crosses over the solution space?
A: It means that there are feasible solutions that have a total profit equal to the Z-value we picked, in this
case, Z = 100.
Q: How many feasible solutions have a total profit of $100?
A: An infinite number of solutions, because there are an infinite number of points on the line segment
where the objective function overlaps the solution space. The line segment lies between the two points
where the objective cuts through the border of the solution space.
Q: Does this mean we have an infinite number of optimal solutions?
A: No, because we haven’t found the optimal solution yet.
Q: Why isn’t 100 our optimal objective value?
A: Our objective is to maximize (find the highest value for) profits, so we need to see if we can find any
solution points that would have a higher objective value.
Q: Are there any feasible solutions that have a total profit higher than 100?
116099766, Page 5 of 10
A: To answer that, look at the graph in Figure 1 and lay a pencil, or the edge of a ruler, along the objective
function line. If you can move whatever object you used away from the origin (up and to the right)
and still be over the solution space, then the answer is yes, there are feasible solutions with a total
profit higher than 100. Figure 2 shows a second objective function line, labeled Z´, where there are
feasible solutions with a higher total profit.
90
Variable C
80
Clay
70
Kiln
60
Wheel
50
40
Z´
30
10
10
20
30
40
50
60
70
80
90
Variable B
Figure 2: LP Graph showing Z´
Q: What is the total profit at Z´?
A: We don’t know, but we don’t need to know. It is enough to know that the solutions are feasible and
the total profit has gone up.
Q: Do we have fewer feasible solutions at Z´ than we had at Z?
A: No, we have the exact same number of feasible solution, infinity, because we still have a line segment
crossing the solution space.
Q: Can we push the objective function out again?
A: We can do this again and again and again, each time pushing the objective function a little further out,
so each time the objective value increases a little.
Q: When do we stop pushing the objective function out?
A: When the objective function reaches the upper, right-hand border of the solution space.
Q: Why do we stop there?
116099766, Page 6 of 10
A: If we went any further, we would leave the solution space behind, so there would be no more feasible
solutions.
Q: When the objective function reaches the border, does it still show an infinite number of feasible
solutions?
A: No, we are down to only one feasible solution point.
Q: How is that possible?
A: When two lines for linear equations intersect, it must be at only one point. When two constraint lines
intersect, it creates what is called a “corner point” of the solution space. These corner points stick out
a little from the solution space, as you can see in Figure 3:
90
Variable C
80
70
Corner Point 1
60
50
Optimal
Objective
Function
Corner Point 2 - Optimal
40
30
Corner Point 3
Objective
Function
Corner Point 4
10
20
30
40
50
60
70
80
90B
Variable
Figure 3: Showing the Corner Points
Q: Why does the optimal solution end up on a corner point?
A: Watch the objective function line as it moves away from the origin. As the Z´ line moves out, the line
segment for Z´ that overlaps the solution space gets smaller and smaller.
Q: As the line segment gets smaller, do we have fewer feasible solutions?
A: Oddly enough, no. There are still an infinite number of points on a line segment. A marvelous thing
happens, though, as the objective touches the border at a corner point, where two constraint lines are
meeting. That line segment suddenly shrinks from an infinite number of points to a single point. Since
that single point is on the border, we can’t go out any further, so that single point is the optimal
solution.
116099766, Page 7 of 10
Q: Will pushing the objective out until it touches the border at only one point always find the optimal
solution?
A: It will work with one caveat and one exception, yes.
Q: What is the caveat?
A: The caveat is that you have to have a “maximization” objective. If you have a “minimization”
objective, then you would push the objective function toward the origin until it touches the inner
border at only one point.
Q: What is the exception?
A: If the objective function is parallel to (has the same slope as) a constraint line, then it touches border of
the solution space along one side, a line segment rather than a single point.
Q: Is that a problem?
A: No, because the ends of the line segment on the border are corner points. Since every point on the line
segment has the same objective value, any point is as good as any other. This means the corner points
are as good as any other point on the line segment, so we can focus our attention on the corner points,
no matter how the objective function touches the border.
Q: What is the advantage to focusing on the corner points?
A: They are the only points on the solution space that can actually be identified.
Q: What does that mean?
A: To identify a point on a two-dimensional graph, you need two independent pieces of information about
the point. If you are not directly given the coordinates (a value on the horizontal axis and a value on
the vertical axis), then you have to be given two independent equations. The only points on the
solution space that have two equations intersecting are the corner points.
Q: Can we identify the solution to this problem?
A: Yes.
Q: What do we do first?
A: Identify which two constraints intersect at the optimal solution point, corner point 2.
Q: How can we identify the two constraints?
A: Figure 2 shows that they are the constraints for the kiln and the potter’s wheel.
Q: What is the next step?
A: Copy the two equations for those constraints from the formulation:
25 B
60 B
+
+
45 C
48 C
<
<
1,800 minutes - wheel
2,400 minutes – kiln
Q: What does that do for us?
A: We need to find the coordinates of the point where these two intersect.
116099766, Page 8 of 10
Q: How do we do that?
A: By whatever means you learned in high school. For this pair of equations, the easiest way is to reduce
the coefficients for the variable B to the same value.
Q: How do we do that?
A: Divide the first equation by 5 and the second equation by 12, giving:
5 B
5 B
+
+
9 C
4 C
=
=
360 minutes - wheel
200 minutes – kiln
Q: Why did you switch the inequalities to equal signs?
A: Since we are looking for the intersection point, we know we are on the lines, not below them, so we
can discard the “<” parts of the inequalities
Q: Now what?
A: Subtract the second equation from the first, like this:
5 B
-5 B
+
-
9 C
4 C
5 C
=
=
=
360
- 200
160
Q: How do we finish?
A: Solve for the value of C, then use that value to find the value of B, like this:
5 C
C
5 B
5 B
5 B
B
+
+
=
=
160
32
9 (32) =
288
=
=
=
360
360
72
14.4
Q: So, what is the solution?
A: The optimal solution point is (14.4, 32). By tradition, we list the variable value for the variable on the
horizontal axis first, so this tells us B = 14.4 and C = 32.
Q: Is there anything else we need to know?
A: We need to know the total profit.
Q: How do we calculate the total profit?
A: Use the objective function from the formulation and variable values we just calculated, like this:
Max Z = $2.20 B
Max Z = $2.20 (14.4)
Max Z = $31.68
Max Z = $111.68
+ $2.50
+ $2.50
+ $80.00
C
(32)
116099766, Page 9 of 10
Q: Is that the complete solution?
A: Yes.
Q: What is the conclusion for all this?
A: The optimal solution for an LP problem will always lie on the intersection of two (or more) constraint
lines, i.e. – at a corner point.
Q: How does that help?
A: We can design the algorithm to search only the corner points.
Q: How is that an advantage?
A: There are a lot fewer corner points than there are feasible solution points.
Q: Are there any generalities we can take from this for constrained optimization?
A: Yes. The “constrained” part of the title tells you that all of these algorithms begin by gathering
information about the solution space. The “optimization” part tells you that some criteria is used to
evaluate solutions that satisfy the constraints.
Q: So all constrained optimization algorithms are basically the same?
A: Basically, yes, but in practice, no. All the algorithms are search algorithms, so the follow the pattern
laid out for you in the Transportation model. The types of constraint equations that are allowed (linear
versus non-linear), the types of objectives that are used (linear versus non-linear, single versus
multiple), and the types of solutions that are acceptable (continuous or integer), create an incredible
array of applications.
Q: How many of these algorithms will we cover?
A: We have covered one in detail – the Transportation model. We will cover a second one in less detail,
linear programming. Working from linear programming, we will talk a little about multiple objectives
and integer solutions. We will completely avoid non-linear applications.
Q: What comes next?
A: Understanding the LP algorithm, so we will discuss a bunch of definitions, then work through some
examples using the graphs until you understand how the algorithm works, without ever having to do
the calculations.
Q: Why don’t we do the calculations?
A: There is little reason for you to learn the calculations. They are complex, tedious, and a computer can
do them much faster and more accurately than you can. It is enough if you understand how the
computer conducts its search, so you could explain it to someone else if you had to.
Q: Do you expect us to use LP in our careers?
A: “Expect” might be too strong a word. There is no question that LP could be used in any of your
careers to help you in decision-making. Most of you, however, will not use it.
Q: Why do you say it could help us?
116099766, Page 10 of 10
A: There is not a single industry, service or production, where LP has not been applied. If you would use
it, it would provide an incredible wealth of information, much as we did in the transportation case.
Q: If you are sure we won’t use it, why teach it?
A: I’m not sure you won’t use it, but I suspect you won’t. We teach it because it is a relatively complex
algorithm, so if you can understand what LP is doing, you can probably figure out any other computer
algorithm on you own. Remember, I know you will be working with a computer in your career. I
can’t predict what software you will use, so I try to teach you some generic models. What you learn in
my class will apply to whatever software you use (you’ll just have to trust me on this).