Introduction to Constrained Optimization Doesn’t that sound like something you’d go a long way to avoid? Q: What is constrained optimization? A: An umbrella term for the set of algorithms that search for a best solution (optimization) while obeying limits placed on the solution (constraints). Q: Does that mean that the transportation model is version of constrained optimization? A: Yes. Q: So what are we going to cover now? A: A more complex algorithm called Linear Programming. Q: What makes linear programming (LP) more complex than the transportation model? A: LP can handle more than simple supply and demand constraints. Unfortunately, this makes the calculations more complex than the transportation algorithm. Q: Do we have to learn these calculations? A: No. Q: Why not? A: Using what you learned about the transportation model, we can argue by analogy to explain what LP does. Q: How does LP work? A: Let’s break the words down to find out. Q: Does “programming” mean giving instructions to a computer? A: No, it is from an old, military term that means “planning.” So “linear programming” means “linear planning.” Q: What does “linear” mean? A: “Linear” refers to the type of equations we will be using. Q: What is a linear equation? A: One that follows the form: Y = mX + b. Q: Is the following also a linear equation: 3X + 4Y = 240? A: Yes. Page 1 of 10 116099766, Page 2 of 10 Q: How do we know? A: The exponent on the variables is 1 (zero is also allowed, but that is a trivial case). The important point is that the exponent isn’t 2, or 3, or something higher, and the exponent isn’t less than one, such as ½, or 1/3, or some other fraction. Also, the variables are added together, not multiplied or divided. So neither 3X * 4Y = 240 nor 3X/4Y = 240 are linear equations. The first is a quadratic equation, and the second is an asymptotic equation. Q: Are all business world relationships linear? A: No, many of them are not. Q: Why, then, do we use linear equations? A: Mostly because they are simple (relative to non-linear equations) to solve. Q: Does linear programming follow the same outline as the transportation model? A: Yes. Q: What is the first step in the LP algorithm? A: Organize the data. Q: Do we set up a table, as we did in Transportation? A: No, this time we set up a set of equations. This is called “formulating” or because a formula is another word for an equation. Q: Why do we have to set up the equations? A: The computer cannot understand sentences. Another phrase to describe organizing the data is “putting it in computer ready form.” Q: How do you formulate a problem? A: You should have learned that in an algebra class somewhere. If you don’t remember, see the lecture notes on “Formulating LP Problems.” Q: Could you show us an example? A: Look at the following word problem: You run a small pottery business, making ceramic bowls ($2.20 profit) and cups ($2.50 profit). Your only raw material is the clay you use. A cup uses 500 grams of clay, while a bowl uses 1 & 1/4 kilograms. It takes you 25 minutes to shape a bowl on your wheel and 45 minutes to shape a cup. A bowl must be “fired” for an hour and a cup for 48minutes in your kiln, which can hold 2 items at a time. You have on hand 40 kilograms of clay and you expect to put in 30 hours at the wheel next week. 116099766, Page 3 of 10 The kiln must be preheated every day, so it is available only 20 hours per week. Set up your production plan for next week. The formulation for this problem is: B # of bowls to make C # of cups to make Max Z = $2.20 B s.t. 1250 B 25 B 60 B B, + + + + $2.50 500 45 48 C C C C C, < < < > 40,000 1,800 2,400 0 grams of clay minutes - wheel minutes – kiln non-negativity Q: Is this the solution to this problem? A: No, this is the formulation. Think of it as setting up the transportation table, before putting in the initial solution. Q: Is it possible to make the problem easier to look at and understand? A: Yes, we could draw a graph of the equations. Q: How do we graph the equations? A: Again, you should have learned that in algebra class somewhere. If you don’t remember, see the lecture notes on “Graphing LP Problems.” Q: What is the graph for these equations? A: See Figure 1: Variable C 90 80 Clay 70 Kiln 60 Wheel 50 40 30 Objective Function 10 10 20 30 40 Figure 1: LP Graph 50 60 70 80 90 Variable B 116099766, Page 4 of 10 Q: What does this graph show? A: Every point on the graph is a possible solution to the problem and the shaded area shows all the points (possible solutions) that satisfy all of the constraints. Q: How is every point a possible solution? A: A solution is value for B and a value for C. Every point on the graph has both a value for B (measured along the horizontal axis) and a value for C (measured along the vertical axis), so every point is a solution. Q: How many feasible solutions are there? A: An infinite number of points lie within the solution space. Q: How do we find the best one? A: Since you probably don’t want to test them one at a time, you need to narrow down the search area. Q: Do we add more constraints? A: No, we use the objective. Q: Haven’t we already drawn in the objective? A: Yes, but to do that we arbitrarily picked a Z-value (total profit) of 100 (see the lecture notes “Graphing LP Problems” if you don’t remember this). Q: What would have happened if we picked a different value for Z? A: We would have drawn a line parallel to the one we drew, but closer in if we picked a smaller Z-value or further out if we picked a larger Z-value. Q: What does it mean if the objective function line crosses over the solution space? A: It means that there are feasible solutions that have a total profit equal to the Z-value we picked, in this case, Z = 100. Q: How many feasible solutions have a total profit of $100? A: An infinite number of solutions, because there are an infinite number of points on the line segment where the objective function overlaps the solution space. The line segment lies between the two points where the objective cuts through the border of the solution space. Q: Does this mean we have an infinite number of optimal solutions? A: No, because we haven’t found the optimal solution yet. Q: Why isn’t 100 our optimal objective value? A: Our objective is to maximize (find the highest value for) profits, so we need to see if we can find any solution points that would have a higher objective value. Q: Are there any feasible solutions that have a total profit higher than 100? 116099766, Page 5 of 10 A: To answer that, look at the graph in Figure 1 and lay a pencil, or the edge of a ruler, along the objective function line. If you can move whatever object you used away from the origin (up and to the right) and still be over the solution space, then the answer is yes, there are feasible solutions with a total profit higher than 100. Figure 2 shows a second objective function line, labeled Z´, where there are feasible solutions with a higher total profit. 90 Variable C 80 Clay 70 Kiln 60 Wheel 50 40 Z´ 30 10 10 20 30 40 50 60 70 80 90 Variable B Figure 2: LP Graph showing Z´ Q: What is the total profit at Z´? A: We don’t know, but we don’t need to know. It is enough to know that the solutions are feasible and the total profit has gone up. Q: Do we have fewer feasible solutions at Z´ than we had at Z? A: No, we have the exact same number of feasible solution, infinity, because we still have a line segment crossing the solution space. Q: Can we push the objective function out again? A: We can do this again and again and again, each time pushing the objective function a little further out, so each time the objective value increases a little. Q: When do we stop pushing the objective function out? A: When the objective function reaches the upper, right-hand border of the solution space. Q: Why do we stop there? 116099766, Page 6 of 10 A: If we went any further, we would leave the solution space behind, so there would be no more feasible solutions. Q: When the objective function reaches the border, does it still show an infinite number of feasible solutions? A: No, we are down to only one feasible solution point. Q: How is that possible? A: When two lines for linear equations intersect, it must be at only one point. When two constraint lines intersect, it creates what is called a “corner point” of the solution space. These corner points stick out a little from the solution space, as you can see in Figure 3: 90 Variable C 80 70 Corner Point 1 60 50 Optimal Objective Function Corner Point 2 - Optimal 40 30 Corner Point 3 Objective Function Corner Point 4 10 20 30 40 50 60 70 80 90B Variable Figure 3: Showing the Corner Points Q: Why does the optimal solution end up on a corner point? A: Watch the objective function line as it moves away from the origin. As the Z´ line moves out, the line segment for Z´ that overlaps the solution space gets smaller and smaller. Q: As the line segment gets smaller, do we have fewer feasible solutions? A: Oddly enough, no. There are still an infinite number of points on a line segment. A marvelous thing happens, though, as the objective touches the border at a corner point, where two constraint lines are meeting. That line segment suddenly shrinks from an infinite number of points to a single point. Since that single point is on the border, we can’t go out any further, so that single point is the optimal solution. 116099766, Page 7 of 10 Q: Will pushing the objective out until it touches the border at only one point always find the optimal solution? A: It will work with one caveat and one exception, yes. Q: What is the caveat? A: The caveat is that you have to have a “maximization” objective. If you have a “minimization” objective, then you would push the objective function toward the origin until it touches the inner border at only one point. Q: What is the exception? A: If the objective function is parallel to (has the same slope as) a constraint line, then it touches border of the solution space along one side, a line segment rather than a single point. Q: Is that a problem? A: No, because the ends of the line segment on the border are corner points. Since every point on the line segment has the same objective value, any point is as good as any other. This means the corner points are as good as any other point on the line segment, so we can focus our attention on the corner points, no matter how the objective function touches the border. Q: What is the advantage to focusing on the corner points? A: They are the only points on the solution space that can actually be identified. Q: What does that mean? A: To identify a point on a two-dimensional graph, you need two independent pieces of information about the point. If you are not directly given the coordinates (a value on the horizontal axis and a value on the vertical axis), then you have to be given two independent equations. The only points on the solution space that have two equations intersecting are the corner points. Q: Can we identify the solution to this problem? A: Yes. Q: What do we do first? A: Identify which two constraints intersect at the optimal solution point, corner point 2. Q: How can we identify the two constraints? A: Figure 2 shows that they are the constraints for the kiln and the potter’s wheel. Q: What is the next step? A: Copy the two equations for those constraints from the formulation: 25 B 60 B + + 45 C 48 C < < 1,800 minutes - wheel 2,400 minutes – kiln Q: What does that do for us? A: We need to find the coordinates of the point where these two intersect. 116099766, Page 8 of 10 Q: How do we do that? A: By whatever means you learned in high school. For this pair of equations, the easiest way is to reduce the coefficients for the variable B to the same value. Q: How do we do that? A: Divide the first equation by 5 and the second equation by 12, giving: 5 B 5 B + + 9 C 4 C = = 360 minutes - wheel 200 minutes – kiln Q: Why did you switch the inequalities to equal signs? A: Since we are looking for the intersection point, we know we are on the lines, not below them, so we can discard the “<” parts of the inequalities Q: Now what? A: Subtract the second equation from the first, like this: 5 B -5 B + - 9 C 4 C 5 C = = = 360 - 200 160 Q: How do we finish? A: Solve for the value of C, then use that value to find the value of B, like this: 5 C C 5 B 5 B 5 B B + + = = 160 32 9 (32) = 288 = = = 360 360 72 14.4 Q: So, what is the solution? A: The optimal solution point is (14.4, 32). By tradition, we list the variable value for the variable on the horizontal axis first, so this tells us B = 14.4 and C = 32. Q: Is there anything else we need to know? A: We need to know the total profit. Q: How do we calculate the total profit? A: Use the objective function from the formulation and variable values we just calculated, like this: Max Z = $2.20 B Max Z = $2.20 (14.4) Max Z = $31.68 Max Z = $111.68 + $2.50 + $2.50 + $80.00 C (32) 116099766, Page 9 of 10 Q: Is that the complete solution? A: Yes. Q: What is the conclusion for all this? A: The optimal solution for an LP problem will always lie on the intersection of two (or more) constraint lines, i.e. – at a corner point. Q: How does that help? A: We can design the algorithm to search only the corner points. Q: How is that an advantage? A: There are a lot fewer corner points than there are feasible solution points. Q: Are there any generalities we can take from this for constrained optimization? A: Yes. The “constrained” part of the title tells you that all of these algorithms begin by gathering information about the solution space. The “optimization” part tells you that some criteria is used to evaluate solutions that satisfy the constraints. Q: So all constrained optimization algorithms are basically the same? A: Basically, yes, but in practice, no. All the algorithms are search algorithms, so the follow the pattern laid out for you in the Transportation model. The types of constraint equations that are allowed (linear versus non-linear), the types of objectives that are used (linear versus non-linear, single versus multiple), and the types of solutions that are acceptable (continuous or integer), create an incredible array of applications. Q: How many of these algorithms will we cover? A: We have covered one in detail – the Transportation model. We will cover a second one in less detail, linear programming. Working from linear programming, we will talk a little about multiple objectives and integer solutions. We will completely avoid non-linear applications. Q: What comes next? A: Understanding the LP algorithm, so we will discuss a bunch of definitions, then work through some examples using the graphs until you understand how the algorithm works, without ever having to do the calculations. Q: Why don’t we do the calculations? A: There is little reason for you to learn the calculations. They are complex, tedious, and a computer can do them much faster and more accurately than you can. It is enough if you understand how the computer conducts its search, so you could explain it to someone else if you had to. Q: Do you expect us to use LP in our careers? A: “Expect” might be too strong a word. There is no question that LP could be used in any of your careers to help you in decision-making. Most of you, however, will not use it. Q: Why do you say it could help us? 116099766, Page 10 of 10 A: There is not a single industry, service or production, where LP has not been applied. If you would use it, it would provide an incredible wealth of information, much as we did in the transportation case. Q: If you are sure we won’t use it, why teach it? A: I’m not sure you won’t use it, but I suspect you won’t. We teach it because it is a relatively complex algorithm, so if you can understand what LP is doing, you can probably figure out any other computer algorithm on you own. Remember, I know you will be working with a computer in your career. I can’t predict what software you will use, so I try to teach you some generic models. What you learn in my class will apply to whatever software you use (you’ll just have to trust me on this).