Decision-making and Optimization
Part 1
János D. Pintér
MSIS Department, RBS
Course Overview*
•
* Please consult the syllabus for a more detailed outline
•
Main topic: the quantitative analysis of business problems that require decision making in a
stochastic (partially uncertain, randomly changing or varying) environment
•
This course extends and complements the concepts and tools studied in the core course
Operations Management
•
We will discuss problem-specific modeling and solution approaches to a range of stochastic
decision problems
•
We will primarily use Excel as the model development and solution platform, also referring to
other modeling and solver environments that may become useful alternatives to Excel
•
Free student versions of prominent modeling and solver environments (e.g., AMPL, GAMS, LINGO
MPL,…) are available: these can become useful in your independent studies and work career
2
Course Resources
•
Textbook: Frederick Hillier and Gerald Lieberman, Introduction to Operations Research
11th Edition, McGraw Hill, 2021
•
This textbook discusses all topics covered by the course – and much more
•
The book also covers the essentials of probability theory and spreadsheet-based modeling,
and it includes a collection of Excel model examples
•
Lecture notes and recorded commentary will be largely based on this textbook, with added notes
•
All presentation materials and examples will be made available on the course site
•
Three freely available calculus and algebra lecture notes are also posted for your perusal: these
works provide useful mathematical background information
3
Learning Objectives
•
Learn about optimization modeling techniques to formulate various decision problems
•
Select appropriate models to analyze specific problem-types
•
Formulate optimization models expressed in mathematical notation
•
Translate mathematical optimization models to a computer-based modeling environment
•
Understand the choice of appropriate solver engines and their expected performance
•
Analyze and interpret the solution of optimization problems in their real-world context
•
Recognize existing and potential applications of optimization
4
Lecture Topics
• Business Analytics / Management Science / Operations Research (BA/MS/OR) concepts and tools
serve to develop and to solve optimization models, in order to find high quality (formally optimized)
solutions to quantitative decision problems arising in business, engineering and the sciences
• As a prelude to the main topic of the course – decision-making under uncertainty – in this lecture
we present simple illustrative deterministic optimization problems and highlight some real-world
applications
• These examples are followed by a generic optimization model, followed by model type
specifications and further examples, providing conceptual clarity for our further work
• Reading: Chapter 1 of the textbook, as well as Appendixes 1 to 4
5
An Illustrative Optimization Problem
• We start with a very simple, but realistic engineering optimization example
• Goal: design a cylindrical container that holds a given volume of
(sufficiently liquid or porous) food, at minimal material cost
• This simple problem will motivate the discussion of several key points
regarding optimization model essentials
• We want to develop a corresponding optimization framework (model)
• To do this, we need to determine the following model components:
• decision variables with bounds to determine the container shape
• constraints that must be satisfied by all feasible (admissible) design options
• objective function to express our primary goal
• input data (given external model parameters)
• Discussion: please suggest appropriate model components
6
Food Container Design: Model Development
• Decision variables: the shape of a cylindrical container is determined by its
radius r [cm] and height h [cm]
• Observe that one can – and always should – give reasonable lower and
upper bounds related to the variables r and h
• Constraint(s): the container must hold a given volume of food
• Objective: minimize the total material costs of the container
• Input data: always provide and document appropriate units [.]
Amount (volume) of food to contain: V [cm3]
Unit price of bottom, top and side: PB, PT, PS [$/cm2]
Note: here we do not consider production costs, only materials
• How would YOU formulate now the model?
Please think a little, before proceeding to the next slide
7
Food Container Design: Model Formulation
• Decision variables: container radius r and height h
• Objective: minimize total material cost
minimize PB(r2π) + PT(r2π) + PS(2hrπ)
• Constraints:
The container must hold a given volume of food
r2πh = V
[a general functional constraint]
• Lower and upper bounds on r and h:
rl ≤ r ≤ ru
[bound constraints on r]
hl ≤ h ≤ hu
[bound constraints on h]
• Discussion
Are the variable bounds important? Are tighter bounds better for search?
Could we write constraint r2πh ≥ V, instead of r2πh = V? Is this a better model form? It depends…
• Can YOU solve this problem now? How would YOU proceed?
8
Food Container Design: Solution 1
• Eliminate variable h (or eliminate r; YOU can try which one works better…)
r2πh = V hence h = V/(r2π), an explicit function of r
• Find r that minimizes the (now one-variable) cost function
f(r) = PB(r2π) + PT(r2π) + PS(2rhπ) =
(PB + PT)(r2π) + PS(2rπ(V/(πr2)) =
(PB + PT)(r2π) + 2 PS(V/r)
9
Food Container Design: Solution 2
• Minimization of f(r) is done by finding the solution of f’(r) = 0
• In this simple model, this directly leads to the optimal solution
the core math background is highlighted on the next slide
f(r) = (PB + PT)(r2π) + 2 PS(V/r)
f’(r) = 2(PB + PT)r π – 2(PS·V)/r2 = 0 → r3π = (PS·V) / (PB + PT)]
Therefore, r = [ 1/π (PS·V) / (PB + PT) ]1/3 and h = V/(π r2)
• These general formulas can be used for all valid input parameter settings
Always check your results on some examples, to make sure that they seem reasonable
Example 1: let PB = PT = PS = 0.01 [$/cm2], V = 1000 [cm3]
Then r = [V/(2π)]1/3 ~ 5.4193 [cm] and h = V/(πr2) ~ 10.8385 [cm]
Example 2: V = 100 [cm3], r = [V/(2π)]1/3 ~ 2.5154 [cm], h = V/(πr2) ~ 5.0308 [cm]
10
Food Container Design: Discussion 1
• How do we know that we found the optimal solution indeed?
f(r)
Hint: think of the cases if r → 0 or r → ; then f(r) → 
Unimodality (in fact, convexity) of f(r) can be proven,
see figure: therefore, we found the unique optimal solution
The key concept of convexity will be discussed later on
• Discussion
• What happens if the required volume V changes?
• What happens if the bottom, or top, or side of the container
becomes more expensive?
• See next slide for answers
r
11
Food Container Design: Discussion 2
• If YOU take a good look at the cans in the supermarket, then
YOU will notice a lot of similar shapes
• What happens if the required volume V changes?
• If all cost data are in the same proportion as before, then the shape of the container remains the
same, since we just changed the unit chosen to express the volume
YOU can verify this point (check the examples on slide 9)
• What happens if the bottom, or the top, or the side of the can becomes more expensive compared
to the other cost factors?
• For more expensive top and/or bottom, the container becomes “slimmer”…
• For more expensive side, the container becomes “chubbier”…
• Figure credits: https://studiousguy.com/cylinder-examples/
12
Food Container Design: Concluding Notes
• Observe that formulating and solving such a simple optimization model is already not quite trivial
Several important technical points had to be considered: modeling, data requirements, solution
method, optimality verification, ex-post analysis
• YOU can certainly think of far more complicated decision problems within the scope of BA/MS/OR
applications: a substantial range of real-world examples will be discussed later on
• Except in certain simple cases, the direct analytical (provable) solution of optimization problems is
not possible
• Therefore, it is essential to use at least a calculator, or much better: a computer-based modeling
environment and solver engine(s) to tackle optimization problems numerically
• The solvers in a modeling environment are based on an implementation of specific algorithms
13
Desktop Computer Box Design 1
• In our next (still very simple) example, we would not be able find the
solution analytically: hence, we need to use a modeling environment
with numerical solution capabilities
• Goal: We want to design a desktop computer box
• Model outline:
Decision variables, with bounds: box depth, width, height
Objective function: minimize the material cost of the computer box [$]
Feasible region: defined by design requirements listed below
Surface should be at least 4000 [cm2]
Volume should be at least 15000 [cm3]
Footprint should be at most 600 [cm2]
Box should not be too narrow (expressed by the rate of depth and width)
Box should not be too high (expressed by the rate of width and height)
Input parameters: unit cost factors [$/cm2]
14
Desktop Computer Box Design 2
AMPL Model*
# Box bottom, side, top cost parameters [$/cm^2]
param cb := 0.025;
param cs := 0.015;
param ct := 0.02;
# Decision variables and bound constraints [cm]
var d >= 10 <=50; # depth (bounds set by developer)
var w >= 10 <=50; # width (set by developer)
var h >= 10 <=50; # height (set by developer)
# Cost function
minimize Cost: cb*d*w + ct*d*w + 2*cs*(w*h + d*h);
subject to
# Design requirements
Surface:
2*(h*d + h*w + d*w) >= 4000; # Convex
Volume:
h*d*w >= 15000; # Convex
Footprint:
d*w <= 600;
# Nonconvex
Not_too_narrow:
d >= 0.4*w; # Linear Convex
Not_too_high:
h <= 1.5*w; # Linear Convex
*Based on a LINGO model:
modifications and AMPL code
shown by JDP
We will discuss an Excel model
implementation later on
Model function features will be
discussed later
15
Desktop Computer Box Design 3
Feasible Region
The simple-looking constraint set of this model leads to a nonconvex feasible set, see several views of
the feasible set: this model feature could trick local scope solvers… more on this point follows later on
16
Desktop Computer Box Design 4
AMPL Model Results
Cost = 65.64478465
option solver lgo; # Cost = 65.64478465
option lgo_options 'opmode = 0'; # local solver
# option solver minos; # Cost = 65.64478465
# option solver snopt; # Cost = 66.07061488
solve;
# Set display precision
option display_precision 10;
# Display results
display Cost; # Objective function
display _varname, _var; # Decision variables
display _conname, _con.slack; # Constraints
# Send results to output text file
display Cost > BD.txt;
display _varname, _var >> BD.txt;
display _conname, _con.slack >> BD.txt;
: _varname _var
:=
1 d
14.16159721
2 w
26.57320152
3 h
39.85980228;
:
1
2
3
4
5
_conname
_con.slack
:=
Surface
-8.185452316e-12
Volume
-1.455191523e-10
Footprint
223.6810235
Not_too_narrow 3.532316602
Not_too_high
0;
Conclusion:
The optimized PC box has dimensions as shown
above (with the given material cost parameters)
The total cost of the PC box is $65.64
The data used are illustrative… we did not
consider production costs, only materials
17
Optimization Problems Arise “Everywhere”…
•
•
•
•
•
•
•
•
Discussion: a few examples of real-world decisions
Assignment of tasks to people
Facility location
Investment planning
Healthcare delivery
Production planning
Transportation logistics
YOU can add your own examples
• A key aspect of the decision problems discussed in this course is substantial inherent uncertainty
If YOU think about the examples listed above (and great many others), then YOU will realize that
the majority of real-world problems could have substantial uncertainty factors that will influence
the results of decisions
18
Optimization: Theory, Models, Algorithms, Software
• Models are idealized representations of reality
• Optimization models, algorithms and software are essential BA/MS/OR tools applied to the
analysis, design, operation and improvement of the systems studied – as illustrated by the very
simple illustrative examples presented so far
• History: Following upon classical research by Newton, Euler, Gauss, Cauchy, Lagrange, and others,
optimization models and algorithms have been developed intensively since the 1940s
• Consult Chapter 1 of the course textbook for an overview of the origins and nature of Operations
Research, and its relation to Business Analytics (BA encompasses OR)
• For courseware and key math background, consult also Appendixes 1 to 4 of the book: software
resources, convexity, classical optimization, vectors and matrices
• We will use Excel as the modeling platform in this course; other software mentioned in the
textbook can be used at your discretion (such as AMPL, GAMS, LINGO, MPL, and others)
19
Optimization Theory, Models, Algorithms, Software
• Optimization theory helps to analyze the proposed decision model structure, thereby assisting
the choice of a suitable solution algorithm(s)
• Algorithm is a finite sequence of instructions to accomplish a given computational task
• Optimization algorithms are aimed at finding the best solution of a certain type (class) of
optimization problem
• Optimization algorithms are implemented as computer programs, in order to make them readily
applicable to support decision-making
• The age of fast, inexpensive personal computers (since the 1980s) greatly facilitates the use of
modern optimization tools by academics, students, researchers, and practitioners
20
Optimization Paradigm 1
• Based on the illustrative examples seen so far, one can formalize the general model framework
• Decision variables: the key unknown factors of the problem studied that we could change,
in order to find the best possible solution (“best possible” needs to be defined carefully)
• Constraints: physical, logical, resource, financial, legal, and other restrictions that define the
feasible (admissible, acceptable) values of the decision variables
• Objective function: examples are total cost, total net profit, completion time, service quality, and
so on; a function of the decision variables that is either minimized or maximized depending on
the context
• Technical note: the word optimization is frequently used interchangeably with the term
(mathematical) programming
21
Optimization Paradigm 2
• Discussion
YOU can directly associate the general optimization paradigm with the examples presented so far:
Container design problem
Desktop computer box design problem
Decision variables? Constraints? Objective function?
• YOU can think of many real-world examples where optimization can be – or should be – applied to
improve decisions, with significant benefits: recall the examples listed earlier
22
Decision Problems and Optimization
• Optimization models serve to formulate decision problems mathematically
Model components
Notation
Decision variables
x = (x1,…,xn)Rn
Bound constraints
xl ≤ x ≤ xu
Constraint functions
g(x) = (g1(x),…, gm(x)) g: Rn→Rm
Objective function
f(x): Rn→R
• Examples seen so far (check details if needed)
Food container design: n = 2 decision variables and bound constraints, m = 1 general functional constraint
Computer box design: n = 3 decision variables with bounds, m = 5 general functional constraints
23
Generic Optimization Model
Concise model form
minimize f(x)
can be abbreviated as min f(x)
subject to the constraints
can be abbreviated as s.t. (subject to)
g(x) ≤ 0
relation ≤ interpreted component-wise
xl ≤ x ≤ xu
relations ≤ interpreted component-wise
24
Optimization Model, in More Details
min f(x)
n variables, scalar output
g1(x) ≤ 0 , g2(x) ≤ 0 , … gm(x) ≤ 0
m general function constraints
xl1 ≤ x1 ≤ xu1 , xl2 ≤ x2 ≤ xu2 , … xln ≤ xn ≤ xun
n bound constraints
Technical notes
Formal equivalence of maximization and minimization problems: min f(x) = - max (-f(x))
Equivalent forms of general inequality and equality constraints: g(x) ≤ 0 and g(x) ≥ 0  g(x) = 0
In many practical optimization applications, the model parameters n and m are large, leading to
substantial size model instances
25
Optimization Model Classes (Categories)
• The generic optimization model can be further expanded and specified, depending on the application
context (i.e., the problem to solve)
• Linear (continuous) / integer / mixed integer(-continuous) / nonlinear (continuous, integer, mixed) /
stochastic (continuous, integer, mixed) and other model types can be defined, depending on the
specific assumed or provable properties of the functions f and g
• Next, we will highlight the model classes listed above, with illustrative examples
• Please let me know whether these model categories are familiar, or if you need further details
• The course textbook discusses all model types mentioned above as well as several others, including all
model types discussed in this course
26
Linear Optimization (Linear Programming, LP)
An Example
• A factory produces a range of items (by assumption, in sufficiently large quantities)
• Each unit of a product has certain resource requirements, e.g., in terms of manpower, materials, and
machines
• The available resources for production are limited
• Objective: find the best combination of the products that can be produced at the factory under the
resource constraints, maximizing net profits (in the considered time period)
• If we can assume that all underlying model functions (objective and constraints) are linear, and the
decision variables are all continuous, then we can formulate an LP model
• LP models can be solved efficiently, used across a broad range of real-world applications
27
Integer Linear Optimization (Programming, ILP)
An Example
• A bank (at a certain branch location) has certain staffing requirements during its work hours
• These requirements are either met by permanent staff (working 8-hour shifts), or by part-time staff
(working 4-hour shifts)
• Existing data are used to estimate staffing needs during banking hours (e.g., 9 am to 5 pm)
• We want to find the staff schedule that meets the daytime-specific staffing requirements, at a minimal
total cost (considering only salary costs)
• We assume that a staff member either works or not within any given banking hour (i.e., no prorated
staffing is possible): similar “yes/no” assumptions lead to ILP models
• ILP models are harder to solve than LPs, used across a broad range of real-world applications
28
Mixed Integer-(Continuous) Linear Programming (MILP)
• Frequently, there is a need to use both integer and continuous variables, in order to develop more
realistic models
• An example: new site selection for production capacity expansion at a manufacturing company, in
conjunction with implied logistics related decisions: which site(s) to choose, and what capacity will be
needed at the chosen site(s)? How site selection influences the shipment of goods from production to
destinations?
• In general, the introduction of integer variables leads to far more difficult models, than models with
continuous variables (assuming the same model structure)
• MILP models provide substantial flexibility, used across a broad range of real-world applications
29
Nonlinear Optimization (Programming, NLP)
• “Definition”: NLP models have at least one nonlinear (objective or constraint) function component
• Depending on the model structure, NLP problems can be relatively easy or very difficult to handle
• The food container design and computer box design problems are simple examples of NLP models
• NLP is used across a broad range of real-world applications
• As we will see later, many of the stochastic models introduced in this course possess an underlying
nonlinear structure
30
Stochastic Optimization (Programming, SP)
• In SP models, decision outcomes depend also on random (uncertain, unknown, varying) factors
• SP problems and suitable approaches to solving these will be the main topic of this course
• SP problems often lead to technically more involved and difficult models, since one must combine
probabilistic modeling concepts with optimization methodology
• SP is used across a broad range of real-world applications
• Examples: production design under uncertainty, investment portfolio selection under uncertainty,
queuing and other service systems, inventory management, etc.
• These and many other topics will be discussed in this course
31
Thank you for your attention!
Questions and comments welcome
jpinter@business.rutgers.edu
32