STA3036S Study Guide STA3036S Student August 30, 2025 Chapter 1: Linear Programming: Formulation Key Concepts & Explanations Linear programming problems are a class of programming problems focused on optimizing the use or allocation of scarce resources (like labor, material, machines, and money). The goal is either to minimize cost or maximize profit in the best possible manner. Characteristics of Linear Programming (LP) Problems: Nonnegative decision variables: The quantities you’re trying to determine (e.g., number of units to produce) cannot be negative. A linear objective function: The function you want to maximize or minimize must be a linear combination of the decision variables. This means no variables are multiplied together (no cross products), and they aren’t raised to powers. A set of linear constraints: All restrictions on resources or other conditions must be expressed as linear equations or inequalities involving the decision variables. Why LP is Widely Used: 1. Many real-world problems can be precisely formulated or reasonably approximated as LP models. 2. Efficient techniques exist for solving LP problems, even large ones. 3. LP models allow for sensitivity analysis, which helps understand how changes in input parameters affect the optimal solution. Formulation of LP Models: The Three Basic Steps 1. Identify Decision Variables: Determine the unknown quantities that need to be decided upon and represent them with algebraic symbols (e.g., XA , XB , XC for production rates). Intuitive Explanation: These are the “levers” you can pull to change the outcome. What are you trying to find? 2. Identify Constraints: List all limitations or requirements in the problem and express them as linear equations or inequalities using the decision variables. Intuitive Explanation: These are the “rules” or “boundaries” of the problem. They define what’s possible. 1 Remember the non-negativity constraint (e.g., XA , XB , XC ≥ 0), which states that production quantities or resource allocations cannot be negative. 3. Identify the Objective Function: Express the goal of the problem (e.g., maximize profit, minimize cost) as a linear function of the decision variables. Intuitive Explanation: This is what you’re trying to achieve – your ultimate goal, expressed mathematically. Example 1 Walkthrough (Company Z) Decision Variables: XA , XB , XC (daily production of models A, B, C). Constraints: – Labor: 7XA + 3XB + 6XC ≤ 150 (hours available) – Material: 4XA + 4XB + 5XC ≤ 200 (kg available) – Non-negativity: XA , XB , XC ≥ 0 Objective Function: Maximize Z = 4XA + 2XB + 3XC (total profit) Exam-Oriented Notes Graphical Solution of LP Problems This method is applicable only for LP problems with two decision variables. It’s crucial for understanding core LP concepts. Procedure for Graphical Solution: 1. Formulate the LP model: Define decision variables, objective function, and constraints. 2. Plot the constraints: For each inequality constraint, treat it as an equality to draw a line on a 2D graph. Intuitive Explanation: These lines define the edges of your possible solutions. 3. Identify the feasible region: This is the area on the graph that satisfies all constraints, including non-negativity (typically the first quadrant). Intuitive Explanation: This region represents all possible combinations of your decision variables that don’t violate any rules. A feasible region is always a convex set (a line segment joining any two points in the set is entirely contained within the set). 4. Determine optimal solution: Corner-Point Method: – Identify all corner points (extreme points) of the feasible region. These are points where constraint lines intersect. – Substitute the coordinates of each corner point into the objective function. – For maximization problems, the corner point yielding the highest objective function value is the optimal solution. – For minimization problems, the corner point yielding the lowest objective function value is the optimal solution. Iso-Profit/Iso-Cost Line Method: 2 – Draw an iso-profit line (for maximization) or iso-cost line (for minimization) by setting the objective function to an arbitrary constant value. – Intuitive Explanation: This line shows combinations of variables that give the same profit/cost. – For maximization, move this line parallel to itself in the direction of increasing objective value until it just touches the feasible region at its outermost point. – For minimization, move this line parallel to itself in the direction of decreasing objective value until it just touches the feasible region at its innermost point. – The point(s) where the iso-line last touches the feasible region is the optimal solution. Important LP Characteristics & Definitions for Exams: No Feasible Region: Occurs if the constraints are contradictory, meaning no points satisfy all inequalities. In such cases, no solution can be found. Redundant Constraint: A constraint whose removal would not alter the feasible solution space. It means at least one other constraint is more restrictive. Alternative/Multiple Optimal Solutions: Occurs when the objective function line is parallel to one of the binding constraint lines that forms a boundary of the feasible region. Any point on the segment of that constraint line touching the feasible region represents an optimal solution. The solution will be at least two corner points at the ends of that segment. – In Simplex Tableau: Indicated by at least one non-basic variable with a zero coefficient in the objective function row (optimality row) in the optimal tableau. Binding Constraint: A constraint that forms the optimal corner point of the feasible solution space. If the optimal values of decision variables are substituted into a binding constraint, the left side will exactly equal the right-hand value. Relaxing a binding constraint would lead to an improved solution. – Intuitive Explanation: These are your “bottlenecks” – they limit how much more profit you can make or how much further you can reduce costs. Nonbinding Constraint: A constraint that does not limit the optimal objective function value. If the optimal values of decision variables are substituted, there will be a difference between the left and right sides. Making it less restrictive will not impact the solution. Slack: Occurs only in a “≤” constraint. It’s the amount by which the left side is less than the right side when optimal values are substituted. It represents unused resources. Surplus: Occurs only in a “≥” constraint. It’s the amount by which the left side exceeds the right side when optimal values are substituted. It represents the amount by which a minimum requirement is surpassed. — Chapter 2: Linear Programming: Excel Solutions & Simplex Method Key Concepts & Explanations The Simplex Method The simplex method is an iterative procedure for solving LP problems, especially those with more than two variables where graphical methods are impractical. It works by moving from one corner point (basic 3 feasible solution) of the feasible region to another, systematically improving the objective function value until an optimal solution is found. General Steps of the Simplex Method: 1. Start with an initial basic feasible solution in canonical form. 2. Improve the current solution by finding a new basic feasible solution with a better objective function value. 3. Continue finding better solutions until no further improvement is possible. This final solution is the optimal solution, and the method terminates. LP in Standard Form Before applying the simplex method, any LP model must be converted into standard form: 1. All constraints are converted to equations with non-negative right-hand sides. “≤” constraints: Add a slack variable to the left side (e.g., X1 + 4X2 ≤ 9 becomes X1 + 4X2 + S1 = 9, where S1 ≥ 0). “≥” constraints: Subtract a surplus variable from the left side (e.g., −X1 +5X2 ≥ 6 becomes −X1 + 5X2 − S2 = 6, where S2 ≥ 0). If a right-hand side is negative, multiply the entire equation by −1 (e.g., 2X1 − 4X2 = −8 becomes −2X1 + 4X2 = 8). 2. All variables must be nonnegative. Unrestricted variables (Xi ): Substitute Xi = Xi′ − Xi′′ where Xi′ , Xi′′ ≥ 0. This substitution must be applied throughout the objective function and all constraints. 3. The objective function can be either maximization or minimization. Computational Details of the Simplex Method Step 0: Initial Basic Feasible Solution: In the standard form, determine a starting feasible solution by setting (number of variables - number of constraints) non-basic variables to zero. A common initial approach is to put all slack variables into the basis and set other variables to zero. Step 1: Select an Entering Variable (Optimality Condition): – The entering variable is the non-basic variable that, if increased, would improve the objective function the most. – In a maximization problem, select the non-basic variable with the most positive relative value in the objective function row (often denoted as Cj − Zj or Row). – In a minimization problem, select the non-basic variable with the most negative relative value in the objective function row. – Intuitive Explanation: This chooses the variable that gives you the “most bang for your buck” or reduces cost the most per unit. – If no such variable exists (all values in the objective row are non-positive for max, or nonnegative for min), the current solution is optimal, and you STOP. Step 2: Select a Leaving Variable (Feasibility Condition): – For both max and min problems, the leaving variable is the basic variable with the smallest non-negative ratio. 4 – Calculate the ratio by dividing the coefficient in the “Solution” column by the corresponding coefficient in the column of the entering variable. – Do not calculate a ratio if the denominator is zero or negative. – Intuitive Explanation: This ensures that the new solution remains feasible (doesn’t violate any constraints). The smallest ratio indicates which current basic variable will hit zero first as the entering variable increases. Step 3: Determine the New Basic Solution: Use elementary row operations (like in Gaussian reduction) to make the entering variable basic and the leaving variable non-basic. Then, return to Step 1. Artificial Variables Needed when “≥” or “=” constraints exist in the original LP formulation, as these don’t readily provide an initial basic feasible solution (unlike “≤” constraints which use slack variables). Purpose: To create an initial basic feasible solution in canonical form. Artificial variables are added to “≥” and “=” constraints. Since artificial variables are not part of the original problem, they must be driven to zero in the optimal solution. This is achieved using methods like the Big M simplex method or the TwoPhase simplex method. Special Cases in Simplex Method Application: Alternative Optima: Occurs when the objective function is parallel to a binding constraint. In the optimal tableau, at least one non-basic variable has a zero coefficient in the objective function row. Unbounded Solution: The values of decision variables can be increased indefinitely without violating any constraints, leading to an infinitely improving objective function. In the tableau, this is recognized when all constraint coefficients under the entering variable are negative or zero. Infeasible Solution: No feasible region exists for the problem. In the tableau, this occurs when at least one artificial variable remains in the basis at a positive level (meaning it could not be driven to zero) when optimality is reached. Degenerate Solution: Occurs when there’s a tie for the minimum ratio during the selection of the leaving variable. This can lead to one or more basic variables becoming zero in the next iteration, which might cause “cycling” (revisiting the same solutions). Duality in Linear Programming Every LP problem (the primal) has an associated LP problem called its dual. These two problems offer different perspectives on the same underlying situation and share the same optimal objective function value. Rules for Deriving the Dual Program: 1. Dual Variables from Primal Constraints: For every constraint in the primal problem, there is a corresponding decision variable in the dual problem. 2. Dual Constraints from Primal Variables: For every decision variable in the primal problem, there is a corresponding constraint in the dual problem. 3. Coefficient Mapping: 5 The right-hand side (RHS) of a primal constraint becomes the objective function coefficient for the corresponding dual variable. The objective function coefficient of a primal variable becomes the RHS of the corresponding dual constraint. The constraint coefficients of a primal variable form the left-hand side coefficients for the corresponding dual constraint. The coefficient matrix is transposed. Table for Dual Relationships (Requires Primal in Standard Form): Primal Objective Dual Objective Maximization Maximization Maximization Minimization Minimization Minimization Minimization Minimization Minimization Maximization Maximization Maximization Primal Constraint Type ≤ ≥ = ≥ ≤ = Dual Variable Sign ≥0 ≤0 Unrestricted ≥0 ≤0 Unrestricted Benefits of Duality: The optimal objective function value of the dual equals that of its primal. Sometimes, solving the dual problem can be computationally more efficient than solving the primal, especially if the dual has fewer constraints than variables. The solution of the dual is directly related to sensitivity analysis of the primal. Exam-Oriented Notes Solving LP Problems using Excel Solver Excel Solver uses the simplex algorithm and is an essential tool for solving LP problems with many variables. Step-by-step Excel Solver Setup: 1. Organize your Spreadsheet: Inputs: Enter all numerical data (coefficients, RHS values) directly into separate cells. Do not use formulas for inputs. Changing Cells (Decision Variables): Designate a set of cells (e.g., B17:E17 for production levels) that will represent your decision variables (x1 , x2 , x3 , x4 ). Enter initial “guess” values (even zeros) here; Solver will change them. Formulas for Resources Used / LHS of Constraints: In adjacent cells, create formulas that calculate the left-hand side (LHS) of your constraints using SUMPRODUCT or similar functions, multiplying inputs by the changing cells. Target Cell (Objective Function): Create a cell that contains the formula for your objective function (e.g., total profit), again using SUMPRODUCT with inputs and changing cells. 2. Invoke Solver: Go to Data > Solver. 6 3. Fill in Solver Parameters Dialog Box: Set Objective: Select your Target Cell. Choose “Max” or “Min” as appropriate. By Changing Variable Cells: Select the range of your Changing Cells (decision variables). Subject to the Constraints: Click “Add” to input constraints. – Resource constraints: Select the “Resources Used” range, choose the inequality (≤, = , or ≥), and select the “Available Resources” range (RHS values). – Sales/demand constraints: Select the “Produced” range, choose ≤, and select the “Max Sales” range. – Non-negativity: Either add Produced >= 0 as a constraint or, more simply, check the “Make Unconstrained Variables Non-Negative” box in Solver Options. Select a Solving Method: Choose “Simplex LP” from the dropdown menu (this is crucial for linear programming models). 4. Solve: Click “Solve”. 5. Interpret Results: Solver will find the optimal solution and display a message. You can choose to keep the Solver solution. Interpreting Solver Output / Sensitivity Analysis Sensitivity analysis is critical for understanding how changes in model parameters affect the optimal solution. Excel Solver provides reports (Answer, Sensitivity, Limits) that are heavily tested. Adjustable Cells (Decision Variables) Final Value: The optimal quantity to produce/use for each decision variable. Objective Coefficient: The current coefficient of the variable in the objective function. Reduced Cost: – For a non-basic variable (Final Value = 0): It’s the amount by which its objective function coefficient would have to improve (increase for maximization, decrease for minimization) before it would be profitable to produce/use that variable (i.e., before it enters the solution at a positive level). – Example: If a variable has a reduced cost of −0.2 in a maximization problem, its profit coefficient would need to increase by at least $0.20 for it to become part of the optimal solution. – For a basic variable at its bound (e.g., upper bound for sales): It represents the change in the optimal objective function value that would result from a one-unit change in that bound. – Example: If “Frame 1” (max problem) is at its upper bound of 1000 and has a reduced cost of 2, increasing the upper bound to 1001 would increase profit by R2 (for small changes). Allowable Increase/Decrease (Range of Optimality): – This is the range of values for an objective function coefficient within which the optimal values of the decision variables will not change. The current solution’s optimal mix remains valid, though the total objective value will change. – Intuitive Explanation: How much can a profit/cost factor change before you need to rethink your production plan entirely? 7 Constraints Final Value: The total amount of a resource used or the value of the LHS of the constraint at the optimal solution. R.H. Side (Constraint RHS): The original right-hand side value of the constraint (e.g., available hours, material). Shadow Price: – The marginal value that indicates the amount by which the objective function value would change if there were a one-unit increase in the right-hand side (RHS) of that constraint. – For a binding constraint (Final Value = RHS): The shadow price indicates how much profit would increase (or cost decrease) if one more unit of that resource were available. It represents the maximum amount you’d be willing to pay for one additional unit of that resource. – For a nonbinding constraint (Final Value ̸= RHS): The shadow price is zero. This means adding or removing a unit of that resource will have no immediate impact on the objective function value, as you already have slack/surplus. Allowable Increase/Decrease (Range of Feasibility): – This is the range of values for the right-hand side (RHS) of a constraint within which the shadow price remains constant. Outside this range, the shadow price may change, and the impact on the objective function needs re-evaluation (i.e., re-running Solver). – Intuitive Explanation: How much can the resource availability change before the value of an extra unit of that resource changes? Chapter 3: Integer Programming Key Concepts & Explanations Introduction to Integer Programming (IP) In many real-world problems, decision variables must take on integer values (e.g., you can’t produce 0.7 of an aircraft). IP models allow for this integrality restriction, which significantly increases their modeling power beyond simple production quantities. IP can model logical requirements, fixed costs, sequencing, and scheduling. Types of Integer Programming Problems: Pure Integer Programming Problem: All decision variables are required to be integers. Mixed Integer Programming Problem: Some variables are restricted to be integers, while others are not. 0-1 (Binary) Integer Programming Problem: Integer variables are restricted to be either 0 or 1. These are extremely useful for modeling “yes/no” or “either/or” decisions (e.g., invest or not invest). Computational Difficulty IP problems are significantly harder to solve than LP problems. Even small problems can be computationally challenging, with solution times increasing exponentially with the number of variables. 8 Modeling with Integer Variables (using 0-1 variables) 0-1 variables are commonly used for: Yes/No Decisions: Xj = 1 if investment j is made, 0 otherwise. Conditional Constraints / Logical Requirements: – “If investment 2 is made, investment 4 must also be made”: X2 − X4 ≤ 0 (or X2 ≤ X4 ). – “If investment 1 is made, investment 3 cannot be made”: X1 + X3 ≤ 1. Relationship to Linear Programming (Linear Relaxation - LR) Given an Integer Program (IP), its linear relaxation (LR) is formed by simply dropping the integrality restrictions (i.e., allowing variables to be continuous). Bounds: – For minimization IPs, the optimal objective value of the LR is less than or equal to the optimal objective value of the IP. – For maximization IPs, the optimal objective value of the LR is greater than or equal to the optimal objective value of the IP. Feasibility: If the LR is infeasible, then the IP is also infeasible. Optimal Integer Solution: If the LR is optimized by integer variables, then that solution is also feasible and optimal for the IP. Rounding: Rounding the optimal solution of the LR does not guarantee an optimal (or even feasible) solution for the IP. Solving Integer Programs: Branch and Bound Method The Branch and Bound method is the most effective technique for solving IP problems, used by Excel Solver. It systematically explores possible integer solutions by dividing the problem into smaller subproblems. Branch and Bound Algorithm Steps: 1. Solve the Linear Relaxation (LR): Solve the original IP problem as a standard LP (ignoring integer restrictions). If the solution is entirely integer, then it’s the optimal solution for the IP. STOP. If the solution contains fractional variables (non-integer values), proceed to branching. 2. Branching: Choose one of the fractional variables (e.g., if X2 = 2.353). Create two new subproblems (branches) by adding new constraints based on the fractional variable. For X2 = 2.353, create Subproblem 1 with X2 ≤ 2 and Subproblem 2 with X2 ≥ 3. Intuitive Explanation: You’re forcing the fractional variable to be integer, creating two mutually exclusive paths that cover all possibilities. 9 3. Bounding: Keep track of the best lower bound (for maximization) or best upper bound (for minimization) found so far. This is the objective value of the best feasible integer solution found. The optimal LR solution from Step 1 provides the initial best upper bound (for maximization) or best lower bound (for minimization) for the entire IP. 4. Fathoming (Termination of Subproblems): A subproblem is terminated (no further branching is needed) if any of the following conditions occur: Integrality: The linear relaxation of the subproblem yields a solution where all decision variables are integer. If this integer solution is better than the current best known integer solution, it becomes the new best bound. Infeasibility: The linear relaxation of the subproblem has no feasible solution. Bounding Argument: The optimal objective value of the subproblem’s LR is worse than the current best known integer solution (lower than the best lower bound for maximization, or higher than the best upper bound for minimization). Intuitive Explanation: If a subproblem can’t possibly lead to a better integer solution than one you’ve already found, there’s no point exploring it further. 5. Selection and Repetition: Choose an active subproblem (one that has not been fathomed). For maximization, choose the active subproblem with the highest objective function value (most promising) to branch from next. For minimization, choose the active subproblem with the lowest objective function value. Repeat steps 1-4 until there are no active subproblems left. The best integer solution found is the overall optimal solution for the IP. Final Recap Section: Cheat Sheet Chapter 1: Linear Programming Formulation Definition: Optimizing resource allocation (min cost/max profit) under linear constraints. Characteristics: Nonnegative variables, linear objective, linear constraints. Formulation Steps: 1. Decision Variables: Identify unknowns (e.g., X1 , X2 ) 2. Constraints: Express limitations as linear inequalities/equations (e.g., aX1 + bX2 ≤ C; X1 , X2 ≥ 0) 3. Objective Function: Express goal as a linear function (e.g., Max/Min Z = cX1 + dX2 ) Graphical Solution (2 Variables): – Plot constraint lines. – Identify feasible region (satisfies all constraints, is a convex set). – Find optimal solution at a corner point (extreme point) by evaluating objective at all corner points or using an iso-profit/iso-cost line. Key LP Terms: – Redundant Constraint: Removal doesn’t change feasible region. 10 – Alternative Optima: Objective function parallel to binding constraint. – Binding Constraint: Met exactly at optimality (LHS = RHS); limits objective. – Nonbinding Constraint: Not met exactly (LHS ̸= RHS); has slack/surplus. – Slack: Unused resources in “≤” constraint. – Surplus: Exceeds minimum in “≥” constraint. Chapter 2: Linear Programming: Excel & Simplex Simplex Method: Iterative procedure moving between corner points to optimize. Standard Form Conversion: – ≤ constraint: Add slack (S ≥ 0) to LHS to make =. – ≥ constraint: Subtract surplus (S ≥ 0) from LHS to make =. – = constraint: No slack/surplus needed initially (may need artificial variable later). – Negative RHS: Multiply by −1. – Unrestricted Variable Xi : Substitute Xi = Xi′ − Xi′′ (Xi′ , Xi′′ ≥ 0). Simplex Tableau - Iteration Steps: 1. Initial BFS: All slack/artificial variables in basis, others 0. 2. Entering Variable (Optimality Condition): – Max: Most positive relative value in Objective Row (Cj − Zj ). – Min: Most negative relative value in Objective Row (Cj − Zj ). – STOP if no improvement possible. 3. Leaving Variable (Feasibility Condition): – Smallest non-negative ratio: (Solution column value) / (Entering variable column value). – Ignore zero/negative denominators. 4. New BFS: Perform row operations, return to Step 2. Artificial Variables: Added to ≥ or = constraints to get an initial BFS. Must be driven to zero. Duality: Primal and Dual problems have the same optimal objective value. – Rules for Dual Derivation: 1. Primal constraint ↔ Dual variable. 2. Primal variable ↔ Dual constraint. 3. Transposed coefficients and objective/RHS exchange. Excel Solver Setup: – Inputs, Changing Cells, Target Cell (Objective). – Constraints: LHS (SUMPRODUCT) vs RHS (Available). – “Make Unconstrained Variables Non-Negative” checked. – Solving Method: “Simplex LP”. Sensitivity Analysis (Excel Output): – Adjustable Cells (Variables): * Reduced Cost: Cost/profit impact if a non-basic variable enters solution, or if a basic variable at its bound is forced to change. * Allowable Increase/Decrease (Range of Optimality): Range for objective coefficient where current optimal decision variables don’t change. – Constraints: * Shadow Price: Change in objective value for a one-unit increase in RHS of a constraint. Zero for nonbinding constraints. Max willingness to pay for resource. * Allowable Increase/Decrease (Range of Feasibility): Range for RHS where shadow price remains constant. 11 Chapter 3: Integer Programming Purpose: Variables must be integers (e.g., number of items). Types: Pure IP (all integer), Mixed IP (some integer), 0-1 IP (binary choice). Modeling with 0-1 Variables: Yes/no decisions, logical conditions (e.g., X1 ≤ X2 implies if X2 = 0 then X1 = 0). Linear Relaxation (LR): IP with integrality restrictions dropped. – LR provides a bound: LRMax ≥ IPMax ; LRMin ≤ IPMin . – Rounding LR solution does not guarantee optimal or even feasible IP solution. Branch and Bound Algorithm: 1. Solve LR: If integer, optimal. Else, branch. 2. Branching: Choose a fractional variable (e.g., X1 = 3.7) and create two subproblems (e.g., X1 ≤ 3 and X1 ≥ 4). 3. Bounding: Maintain a “best bound” (e.g., highest integer solution for maximization). 4. Fathoming (Termination): A subproblem terminates if: – Integrality: All variables are integer. – Infeasibility: No feasible solution. – Bounding: Its LR objective is worse than the current best integer solution found. 5. Repeat: Select active subproblem (highest objective for max) and branch until no active subproblems. The best integer solution found is optimal. 12
0
You can add this document to your study collection(s)
Sign in Available only to authorized usersYou can add this document to your saved list
Sign in Available only to authorized users(For complaints, use another form )