Decision-making and Optimization Part 1 János D. Pintér MSIS Department, RBS Course Overview* • * Please consult the syllabus for a more detailed outline • Main topic: the quantitative analysis of business problems that require decision making in a stochastic (partially uncertain, randomly changing or varying) environment • This course extends and complements the concepts and tools studied in the core course Operations Management • We will discuss problem-specific modeling and solution approaches to a range of stochastic decision problems • We will primarily use Excel as the model development and solution platform, also referring to other modeling and solver environments that may become useful alternatives to Excel • Free student versions of prominent modeling and solver environments (e.g., AMPL, GAMS, LINGO MPL,…) are available: these can become useful in your independent studies and work career 2 Course Resources • Textbook: Frederick Hillier and Gerald Lieberman, Introduction to Operations Research 11th Edition, McGraw Hill, 2021 • This textbook discusses all topics covered by the course – and much more • The book also covers the essentials of probability theory and spreadsheet-based modeling, and it includes a collection of Excel model examples • Lecture notes and recorded commentary will be largely based on this textbook, with added notes • All presentation materials and examples will be made available on the course site • Three freely available calculus and algebra lecture notes are also posted for your perusal: these works provide useful mathematical background information 3 Learning Objectives • Learn about optimization modeling techniques to formulate various decision problems • Select appropriate models to analyze specific problem-types • Formulate optimization models expressed in mathematical notation • Translate mathematical optimization models to a computer-based modeling environment • Understand the choice of appropriate solver engines and their expected performance • Analyze and interpret the solution of optimization problems in their real-world context • Recognize existing and potential applications of optimization 4 Lecture Topics • Business Analytics / Management Science / Operations Research (BA/MS/OR) concepts and tools serve to develop and to solve optimization models, in order to find high quality (formally optimized) solutions to quantitative decision problems arising in business, engineering and the sciences • As a prelude to the main topic of the course – decision-making under uncertainty – in this lecture we present simple illustrative deterministic optimization problems and highlight some real-world applications • These examples are followed by a generic optimization model, followed by model type specifications and further examples, providing conceptual clarity for our further work • Reading: Chapter 1 of the textbook, as well as Appendixes 1 to 4 5 An Illustrative Optimization Problem • We start with a very simple, but realistic engineering optimization example • Goal: design a cylindrical container that holds a given volume of (sufficiently liquid or porous) food, at minimal material cost • This simple problem will motivate the discussion of several key points regarding optimization model essentials • We want to develop a corresponding optimization framework (model) • To do this, we need to determine the following model components: • decision variables with bounds to determine the container shape • constraints that must be satisfied by all feasible (admissible) design options • objective function to express our primary goal • input data (given external model parameters) • Discussion: please suggest appropriate model components 6 Food Container Design: Model Development • Decision variables: the shape of a cylindrical container is determined by its radius r [cm] and height h [cm] • Observe that one can – and always should – give reasonable lower and upper bounds related to the variables r and h • Constraint(s): the container must hold a given volume of food • Objective: minimize the total material costs of the container • Input data: always provide and document appropriate units [.] Amount (volume) of food to contain: V [cm3] Unit price of bottom, top and side: PB, PT, PS [$/cm2] Note: here we do not consider production costs, only materials • How would YOU formulate now the model? Please think a little, before proceeding to the next slide 7 Food Container Design: Model Formulation • Decision variables: container radius r and height h • Objective: minimize total material cost minimize PB(r2π) + PT(r2π) + PS(2hrπ) • Constraints: The container must hold a given volume of food r2πh = V [a general functional constraint] • Lower and upper bounds on r and h: rl ≤ r ≤ ru [bound constraints on r] hl ≤ h ≤ hu [bound constraints on h] • Discussion Are the variable bounds important? Are tighter bounds better for search? Could we write constraint r2πh ≥ V, instead of r2πh = V? Is this a better model form? It depends… • Can YOU solve this problem now? How would YOU proceed? 8 Food Container Design: Solution 1 • Eliminate variable h (or eliminate r; YOU can try which one works better…) r2πh = V hence h = V/(r2π), an explicit function of r • Find r that minimizes the (now one-variable) cost function f(r) = PB(r2π) + PT(r2π) + PS(2rhπ) = (PB + PT)(r2π) + PS(2rπ(V/(πr2)) = (PB + PT)(r2π) + 2 PS(V/r) 9 Food Container Design: Solution 2 • Minimization of f(r) is done by finding the solution of f’(r) = 0 • In this simple model, this directly leads to the optimal solution the core math background is highlighted on the next slide f(r) = (PB + PT)(r2π) + 2 PS(V/r) f’(r) = 2(PB + PT)r π – 2(PS·V)/r2 = 0 → r3π = (PS·V) / (PB + PT)] Therefore, r = [ 1/π (PS·V) / (PB + PT) ]1/3 and h = V/(π r2) • These general formulas can be used for all valid input parameter settings Always check your results on some examples, to make sure that they seem reasonable Example 1: let PB = PT = PS = 0.01 [$/cm2], V = 1000 [cm3] Then r = [V/(2π)]1/3 ~ 5.4193 [cm] and h = V/(πr2) ~ 10.8385 [cm] Example 2: V = 100 [cm3], r = [V/(2π)]1/3 ~ 2.5154 [cm], h = V/(πr2) ~ 5.0308 [cm] 10 Food Container Design: Discussion 1 • How do we know that we found the optimal solution indeed? f(r) Hint: think of the cases if r → 0 or r → ; then f(r) → Unimodality (in fact, convexity) of f(r) can be proven, see figure: therefore, we found the unique optimal solution The key concept of convexity will be discussed later on • Discussion • What happens if the required volume V changes? • What happens if the bottom, or top, or side of the container becomes more expensive? • See next slide for answers r 11 Food Container Design: Discussion 2 • If YOU take a good look at the cans in the supermarket, then YOU will notice a lot of similar shapes • What happens if the required volume V changes? • If all cost data are in the same proportion as before, then the shape of the container remains the same, since we just changed the unit chosen to express the volume YOU can verify this point (check the examples on slide 9) • What happens if the bottom, or the top, or the side of the can becomes more expensive compared to the other cost factors? • For more expensive top and/or bottom, the container becomes “slimmer”… • For more expensive side, the container becomes “chubbier”… • Figure credits: https://studiousguy.com/cylinder-examples/ 12 Food Container Design: Concluding Notes • Observe that formulating and solving such a simple optimization model is already not quite trivial Several important technical points had to be considered: modeling, data requirements, solution method, optimality verification, ex-post analysis • YOU can certainly think of far more complicated decision problems within the scope of BA/MS/OR applications: a substantial range of real-world examples will be discussed later on • Except in certain simple cases, the direct analytical (provable) solution of optimization problems is not possible • Therefore, it is essential to use at least a calculator, or much better: a computer-based modeling environment and solver engine(s) to tackle optimization problems numerically • The solvers in a modeling environment are based on an implementation of specific algorithms 13 Desktop Computer Box Design 1 • In our next (still very simple) example, we would not be able find the solution analytically: hence, we need to use a modeling environment with numerical solution capabilities • Goal: We want to design a desktop computer box • Model outline: Decision variables, with bounds: box depth, width, height Objective function: minimize the material cost of the computer box [$] Feasible region: defined by design requirements listed below Surface should be at least 4000 [cm2] Volume should be at least 15000 [cm3] Footprint should be at most 600 [cm2] Box should not be too narrow (expressed by the rate of depth and width) Box should not be too high (expressed by the rate of width and height) Input parameters: unit cost factors [$/cm2] 14 Desktop Computer Box Design 2 AMPL Model* # Box bottom, side, top cost parameters [$/cm^2] param cb := 0.025; param cs := 0.015; param ct := 0.02; # Decision variables and bound constraints [cm] var d >= 10 <=50; # depth (bounds set by developer) var w >= 10 <=50; # width (set by developer) var h >= 10 <=50; # height (set by developer) # Cost function minimize Cost: cb*d*w + ct*d*w + 2*cs*(w*h + d*h); subject to # Design requirements Surface: 2*(h*d + h*w + d*w) >= 4000; # Convex Volume: h*d*w >= 15000; # Convex Footprint: d*w <= 600; # Nonconvex Not_too_narrow: d >= 0.4*w; # Linear Convex Not_too_high: h <= 1.5*w; # Linear Convex *Based on a LINGO model: modifications and AMPL code shown by JDP We will discuss an Excel model implementation later on Model function features will be discussed later 15 Desktop Computer Box Design 3 Feasible Region The simple-looking constraint set of this model leads to a nonconvex feasible set, see several views of the feasible set: this model feature could trick local scope solvers… more on this point follows later on 16 Desktop Computer Box Design 4 AMPL Model Results Cost = 65.64478465 option solver lgo; # Cost = 65.64478465 option lgo_options 'opmode = 0'; # local solver # option solver minos; # Cost = 65.64478465 # option solver snopt; # Cost = 66.07061488 solve; # Set display precision option display_precision 10; # Display results display Cost; # Objective function display _varname, _var; # Decision variables display _conname, _con.slack; # Constraints # Send results to output text file display Cost > BD.txt; display _varname, _var >> BD.txt; display _conname, _con.slack >> BD.txt; : _varname _var := 1 d 14.16159721 2 w 26.57320152 3 h 39.85980228; : 1 2 3 4 5 _conname _con.slack := Surface -8.185452316e-12 Volume -1.455191523e-10 Footprint 223.6810235 Not_too_narrow 3.532316602 Not_too_high 0; Conclusion: The optimized PC box has dimensions as shown above (with the given material cost parameters) The total cost of the PC box is $65.64 The data used are illustrative… we did not consider production costs, only materials 17 Optimization Problems Arise “Everywhere”… • • • • • • • • Discussion: a few examples of real-world decisions Assignment of tasks to people Facility location Investment planning Healthcare delivery Production planning Transportation logistics YOU can add your own examples • A key aspect of the decision problems discussed in this course is substantial inherent uncertainty If YOU think about the examples listed above (and great many others), then YOU will realize that the majority of real-world problems could have substantial uncertainty factors that will influence the results of decisions 18 Optimization: Theory, Models, Algorithms, Software • Models are idealized representations of reality • Optimization models, algorithms and software are essential BA/MS/OR tools applied to the analysis, design, operation and improvement of the systems studied – as illustrated by the very simple illustrative examples presented so far • History: Following upon classical research by Newton, Euler, Gauss, Cauchy, Lagrange, and others, optimization models and algorithms have been developed intensively since the 1940s • Consult Chapter 1 of the course textbook for an overview of the origins and nature of Operations Research, and its relation to Business Analytics (BA encompasses OR) • For courseware and key math background, consult also Appendixes 1 to 4 of the book: software resources, convexity, classical optimization, vectors and matrices • We will use Excel as the modeling platform in this course; other software mentioned in the textbook can be used at your discretion (such as AMPL, GAMS, LINGO, MPL, and others) 19 Optimization Theory, Models, Algorithms, Software • Optimization theory helps to analyze the proposed decision model structure, thereby assisting the choice of a suitable solution algorithm(s) • Algorithm is a finite sequence of instructions to accomplish a given computational task • Optimization algorithms are aimed at finding the best solution of a certain type (class) of optimization problem • Optimization algorithms are implemented as computer programs, in order to make them readily applicable to support decision-making • The age of fast, inexpensive personal computers (since the 1980s) greatly facilitates the use of modern optimization tools by academics, students, researchers, and practitioners 20 Optimization Paradigm 1 • Based on the illustrative examples seen so far, one can formalize the general model framework • Decision variables: the key unknown factors of the problem studied that we could change, in order to find the best possible solution (“best possible” needs to be defined carefully) • Constraints: physical, logical, resource, financial, legal, and other restrictions that define the feasible (admissible, acceptable) values of the decision variables • Objective function: examples are total cost, total net profit, completion time, service quality, and so on; a function of the decision variables that is either minimized or maximized depending on the context • Technical note: the word optimization is frequently used interchangeably with the term (mathematical) programming 21 Optimization Paradigm 2 • Discussion YOU can directly associate the general optimization paradigm with the examples presented so far: Container design problem Desktop computer box design problem Decision variables? Constraints? Objective function? • YOU can think of many real-world examples where optimization can be – or should be – applied to improve decisions, with significant benefits: recall the examples listed earlier 22 Decision Problems and Optimization • Optimization models serve to formulate decision problems mathematically Model components Notation Decision variables x = (x1,…,xn)Rn Bound constraints xl ≤ x ≤ xu Constraint functions g(x) = (g1(x),…, gm(x)) g: Rn→Rm Objective function f(x): Rn→R • Examples seen so far (check details if needed) Food container design: n = 2 decision variables and bound constraints, m = 1 general functional constraint Computer box design: n = 3 decision variables with bounds, m = 5 general functional constraints 23 Generic Optimization Model Concise model form minimize f(x) can be abbreviated as min f(x) subject to the constraints can be abbreviated as s.t. (subject to) g(x) ≤ 0 relation ≤ interpreted component-wise xl ≤ x ≤ xu relations ≤ interpreted component-wise 24 Optimization Model, in More Details min f(x) n variables, scalar output g1(x) ≤ 0 , g2(x) ≤ 0 , … gm(x) ≤ 0 m general function constraints xl1 ≤ x1 ≤ xu1 , xl2 ≤ x2 ≤ xu2 , … xln ≤ xn ≤ xun n bound constraints Technical notes Formal equivalence of maximization and minimization problems: min f(x) = - max (-f(x)) Equivalent forms of general inequality and equality constraints: g(x) ≤ 0 and g(x) ≥ 0 g(x) = 0 In many practical optimization applications, the model parameters n and m are large, leading to substantial size model instances 25 Optimization Model Classes (Categories) • The generic optimization model can be further expanded and specified, depending on the application context (i.e., the problem to solve) • Linear (continuous) / integer / mixed integer(-continuous) / nonlinear (continuous, integer, mixed) / stochastic (continuous, integer, mixed) and other model types can be defined, depending on the specific assumed or provable properties of the functions f and g • Next, we will highlight the model classes listed above, with illustrative examples • Please let me know whether these model categories are familiar, or if you need further details • The course textbook discusses all model types mentioned above as well as several others, including all model types discussed in this course 26 Linear Optimization (Linear Programming, LP) An Example • A factory produces a range of items (by assumption, in sufficiently large quantities) • Each unit of a product has certain resource requirements, e.g., in terms of manpower, materials, and machines • The available resources for production are limited • Objective: find the best combination of the products that can be produced at the factory under the resource constraints, maximizing net profits (in the considered time period) • If we can assume that all underlying model functions (objective and constraints) are linear, and the decision variables are all continuous, then we can formulate an LP model • LP models can be solved efficiently, used across a broad range of real-world applications 27 Integer Linear Optimization (Programming, ILP) An Example • A bank (at a certain branch location) has certain staffing requirements during its work hours • These requirements are either met by permanent staff (working 8-hour shifts), or by part-time staff (working 4-hour shifts) • Existing data are used to estimate staffing needs during banking hours (e.g., 9 am to 5 pm) • We want to find the staff schedule that meets the daytime-specific staffing requirements, at a minimal total cost (considering only salary costs) • We assume that a staff member either works or not within any given banking hour (i.e., no prorated staffing is possible): similar “yes/no” assumptions lead to ILP models • ILP models are harder to solve than LPs, used across a broad range of real-world applications 28 Mixed Integer-(Continuous) Linear Programming (MILP) • Frequently, there is a need to use both integer and continuous variables, in order to develop more realistic models • An example: new site selection for production capacity expansion at a manufacturing company, in conjunction with implied logistics related decisions: which site(s) to choose, and what capacity will be needed at the chosen site(s)? How site selection influences the shipment of goods from production to destinations? • In general, the introduction of integer variables leads to far more difficult models, than models with continuous variables (assuming the same model structure) • MILP models provide substantial flexibility, used across a broad range of real-world applications 29 Nonlinear Optimization (Programming, NLP) • “Definition”: NLP models have at least one nonlinear (objective or constraint) function component • Depending on the model structure, NLP problems can be relatively easy or very difficult to handle • The food container design and computer box design problems are simple examples of NLP models • NLP is used across a broad range of real-world applications • As we will see later, many of the stochastic models introduced in this course possess an underlying nonlinear structure 30 Stochastic Optimization (Programming, SP) • In SP models, decision outcomes depend also on random (uncertain, unknown, varying) factors • SP problems and suitable approaches to solving these will be the main topic of this course • SP problems often lead to technically more involved and difficult models, since one must combine probabilistic modeling concepts with optimization methodology • SP is used across a broad range of real-world applications • Examples: production design under uncertainty, investment portfolio selection under uncertainty, queuing and other service systems, inventory management, etc. • These and many other topics will be discussed in this course 31 Thank you for your attention! Questions and comments welcome jpinter@business.rutgers.edu 32