Deterministic Operations Research
OR 6205
Module 1: Introduction
to Operations Research
and Linear Programming
Objectives
Describe OR & LP as they relate to decision
making in organizations
! Use the Gauss-Jordan method to solve a
system of simultaneous linear equations in
algebraic and matrix form
!
2
Lesson 01: History of OR and LP
Operations Research (OR)
Have you ever been asked to make a decision
for your organization to optimize a process or
procedure?
! OR is an entire field devoted to understanding
how to make these decisions.
! OR deals with the application of scientific
methods to decision making, especially to the
allocation of resources.
!
4
Operations Research Cont.
OR uses analytical and numerical techniques
to develop and manipulate mathematical and
computer models of organizational systems
composed of people, machines, and
procedures.
! OR draws upon ideas from many diverse
areas, such as, engineering, management,
mathematics and psychology to contribute to
a wide variety of application domain.
!
5
Organization
Application
Dutch Ministry of Infrastructure
National Water management policy development, adding new facilities, operating and costs
and the Environment
procedures
(Netherlands Rijkswaterstaat)
Year
Yearly savings
1985 $15 millions
Monsanto Corp.
Production's operations optimization to obey goals with a minimum cost
1985 $2 millions
Weyerhaeuser Co.
Cutting trees optimization to maximize wood products production
1986 $15 millions
Electrobras/CEPAL Brasil
United Airlines
Optimal allocation of hydraulic and thermic resources in the national energy generation system
1986 $43 millions
Shifts at book offices and airports scheduling to accomplish with the customer needs at minimal cost 1986 $6 millions
CITGO Petroleum Corp.
Optimization of refinement, offer, distribution and commercialization of products operations
1987 $70 millions
Santos, Ltd., Australia
Capital investment optimizing to produce natural gas along 25 years in Australia
1987 $3 millions
Electric Power Research
Institute
San Francisco Police
Department
Administration of oil and coal inventories for the electric service with the intention of balancing
inventory costs and risks of remaining
1989 $59 millions
Optimization programming and assignment of Patrol's officers with a computed system
1989 $11 millions
Texaco Inc.
Optimizing the mixing of ingredients available in order to obtain fuels which met with the quality
requirements and sales
1989 $30 millions
IBM
Integration of a national network of spare parts inventory to improve support service
1990
U.S. Military Airlift Command
Rapidity in the airplanes, crew, load and passengers coordination to drive the evacuation by air in
the "Desert Storm" project in the Middle Orient
1992 Victory
American Airlines
Design of a pricing, overbooking and flight coordination structure system to enhance benefits
1992
Yellow Freight System, Inc.
Optimizing the design of the national transport network and the scheduling of shipping routes in the
1992 $17.3 millions
U.S.
New Haven Health Dept.
Design of an effective program of needles change to combat the AIDS contagion
1993 33% less of contagions
AT&T
Development of a computer system to design call centers to guide customers
1993 $750 millions
Delta Airlines
Maximizing profits from the allocation of aircraft types in 2.500 national flights in the U.S.
Restructuring of the whole supply chain among suppliers, plants, distribution centers, potential sites
and market areas
Selection and optimum programming of mass projects to obey with future energy needs of the
country
Optimal restructuring of the size and form of the South African National Defence Force and his
weapons system
1994 $100 millions
Digital Equipment Corp.
China
South African National
Defence Force (SANDF)
Procter & Gamble
Taco Bell
Hewlett-Packard
$20 millions + $250 millions in
minor inventory
$500 millions of additional
revenue
1995 $800 millions
1995 $425 millions
1997 $1.100 millions
Redesign of the North American production and distribution system to reduce costs and to improve
1997 $200 millions
the incoming rapidity to the market
Optimum employees programming to provide the service to desired clients with a minimum cost
Redesign of security inventories' size and location at printer production line to obey the production
goals
1998 $13 millions
$280 millions of additional
1998
revenue
6
Source: http://www.phpsimplex.com/en/real_cases.htm
Operations Research Cont.
OR is closely related to several other fields in
the "Decision Sciences:" Applied Mathematics,
Computer Science, Systems Engineering,
Industrial Engineering and Economics.
! OR with machine learning
!
7
Brief History of OR
Research on military Operations problems by
the British government during world war II led
to the development of OR
• OR led to the development of methods for
effectively utilizing new defense technologies,
such as the radar
• The air chief marshal directed the project for
the Royal Air Force, 1938
•
8
Brief History of OR Cont.
This first OR project contributed to the first
allied victory, the Battle of Britain, 1940
• Rapid expansion of OR in the USA began in
1942, again as military OR
• After the war, interest spread from the
military to other government organizations
and to the industry.
•
9
OR Professional Organizations
In 1952, the OR Society of America (ORSA)
was founded by Professor Philip Morse of MIT
• In 1995, ORSA merged with The Institute of
Management Sciences (TIMS) to form
INFORMS
• There are more than 50 societies in other
countries, all under the umbrella of the
International Federation of Operational
Research Societies.
•
10
OR Application Areas
Manufacturing
! Transportation
! Communication
! Construction
! Health Care
! Banking
!
11
Theories and Methods Based on OR
•
•
•
•
•
•
•
Linear programming
Network analysis
Dynamic programming
Markov Chains
Queuing theory
Forecasting
Inventory theory
12
Theories and Methods Based on OR
•
•
•
•
•
Game theory
Integer and nonlinear programming
Scheduling
Simulation
Multiple criteria decision making
13
Network analysis
It refers to some interesting LP problems that
have network structure.
! Examples:
!
q Transportation problem,
q Assignment problem,
q Shortest path problem,
q Maximum flow problem and
q Generalized network flow problem.
14
Dynamic programming
!
!
!
Different from LP
Method that is used to make a sequence of
interrelated decisions.
Example, the multi-period production planning
problem: we know the customer demand each
month and we want to determine the production
level each month that will minimize total cost,
including production setup cost, variable production
costs, and inventory carrying costs. The decisions are
interrelated.
15
Types of Decisions
Decisions are usually classified with respect to
their frequency of occurrence as:
• Strategic decisions are performed every 5 to
10 years. Example: locating a plant or a
warehouse. The setup cost is so high that we
cannot afford to open a plant every month or
year.
• Operating decisions are performed routinely.
16
Example of Operating Decisions
The delivery of products from a warehouse to
customers.
! Every day, there are orders of different
customers to be delivered to different
locations by a fleet of trucks.
! The routing decisions of the trucks (customer
sequencing) have to be made every day.
!
17
Linear Programming (LP)
First OR area, developed in the 1940's
! Most common application of LP: allocation of
limited resources to competing activities in
the optimal way
! LP stands for the linearity of the functions
used (linear), and the planning of activities
(programming)
!
18
Linear Programming Contd.
!
!
!
LP uses a mathematical model of the system: Activity
levels - decision variables that are used to define the
measure of performance (objective function);
Restrictions of the problem - functions in equality or
inequality form (constraints).
Tremendous impact of LP is mostly due to the
development of the digital computer and an efficient
computational procedure called simplex (Dantzig 1947).
New (interior point) methods were developed (N.
Karmarkar, Bell Laboratories, 1984) which can solve large
size problems.
19
LP Applications
•
Allocating production facilities to products
o Activities: Products to be produced this week
o Resources: Labor, machines & raw materials
o Activity Levels: No. of units of each product to make
•
Selection of shipping patterns: Distributing products
from 3 warehouses to 4 customers
o Activities: Each pair of warehouse-customer
o Resources: Amount of product that is available at each
warehouse
o Activity Levels: Amount of products to be shipped from
each warehouse to each customer
20
LP Applications Cont.
•
Allocating limited funds to investment opportunities
o Activities: Possible securities to buy
o Resources: Capital to be invested
o Activity Levels: Money to invest in each of the securities
•
Agricultural planning: Farm business
o Activities: Crops (wheat, corn, etc.) & types of livestock to
grow this coming year
o Resources: Operating budget, equipment, labor, & water
o Activity Levels: Amount of crops & livestock to grow
21
Lesson 02: OR modeling
An Introduction to Modeling
Operations research (management science)
is a scientific approach to decision making that
seeks to best design and operate a system,
usually under conditions requiring the
allocation of scarce resources.
Term coined during WW II when leaders asked
scientists and engineers to analyze several military
problems.
A system is an organization of interdependent
components that work together to accomplish
the goal of the system.
2
The scientific approach to decision making
requires the use of one or more mathematical
models.
A mathematical model is a mathematical
representation of the actual situation that may
be used to make better decisions or clarify the
situation.
3
Example 1: A Modeling Example
Eli Daisy produces the drug Wozac in batches
by heating a chemical mixture in a pressurized
container.
Each time a batch is produced, a different
amount of Wozac is produced.
The amount produced is the process yield (measured
in pounds).
Daisy is interested in understanding the factors
that influence the yield of Wozac production
process.
Describe a model-building process for this
situation.
4
Example 1: Solution
Daisy is first interested in determining the
factors that influence the process yield.
This is a descriptive model since it describes the
behavior of the actual yield as a function of various
factors.
Daisy might determine that the following
factors influence yield:
Container volume in liters (V)
Container pressure in milliliters (P)
Container pressure in degrees centigrade (T)
Chemical composition of the processed mixture
5
Ex. 1: Solution continued
Letting A, B, and C be the percentage of the
mixture made up of chemical A, B, and C, then
Daisy might find , for example, that: 2
2
Yield = 300 + 0.8V +0.01P + 0.06T + 0.001T*P - 0.01T
11.7A + 9.4B + 16.4C + 19A*B + 11.4A*C 9.6B*C
0.001P +
To determine the relationship the yield of the
process would have to measured for many
different combinations of the factors.
Knowledge of this equation would enable Daisy
to describe the yield of the production process
once volume, pressure, temperature, and
chemical composition were known.
6
Prescriptive models
e c be beha
f
an organization that will enable it to best meet
its goals.
Components of this model include:
objective function(s)
decision variables
constraints
An optimization model seeks to find values
of the decision variables that optimize
(maximize or minimize) an objective function
among the set of all values for the decision
variables that satisfy the given constraints.
7
Ex. 1: Solution continued
The Daisy example seeks to maximize the yield
for the production process.
In most models, there will be a function we
wish to maximize or minimize. This function is
ca ed he
de objective function.
To maximize the process yield we need to find
the values of V, P, T, A, B, and C that make the
yield equation (below) as large as possible.
Yield = 300 + 0.8V +0.01P + 0.06T + 0.001T*P - 0.01T2
+ 11.7A + 9.4B + 16.4C + 19A*B + 11.4A*C
0.001P2
9.6B*C
8
 In many situations, an organization may have
more than one objective.
For example, in assigning students to the two high
schools in Bloomington, Indiana, the Monroe County
School Board stated that the assignment of students
involve the following objectives:
equalize the number of students at the two high
schools
minimize the average distance students travel to
school
have a diverse student body at both high schools
9
Variables whose values are under our control
and influence system performance are called
decision variables.
In the Daisy example, V, P, T, A, B, and C are decision
variables.
In most situations, only certain values of the
decision variables are possible.
For example, certain volume, pressure, and
temperature conditions might be unsafe. Also, A, B,
and C must be nonnegative numbers that sum to
one.
These restrictions on the decision variable
values are called constraints.
10
Ex. 1: Solution continued
Suppose the Daisy example has the following
constraints:
Volume must be between 1 and 5 liters
Pressure must be between 200 and 400 milliliters
Temperature must be between 100 and 200 degrees
centigrade
Mixture must be made up entirely of A, B, and C
For the drug to perform properly, only half the
mixture at most can be product A.
11
Ex. 1: Solution continued
Mathematically, these constraints can be
expressed:
V
5
A≥0
V
1
B≥0
P ≤ 400
C≥0
P ≥ 200
A + B + C = 1.0
T ≤ 200
A ≤ 0.5
T ≥ 100
12
Ex. 1: Solution continued
The Complete Daisy Optimization Model
Letting z represent the value of the objection function
(the yield), the entire optimization model may be
written as:
Maximize z = 300 + 0.8V +0.01P + 0.06T + 0.001T*P - 0.01T2
0.001P2 + 11.7A + 9.4B + 16.4C + 19A*B + 11.4A*C
Subject to (s.t.)
V
5
T ≤ 200
A≥0
V
1
T ≥ 100
B≥0
P ≤ 400
A + B + C = 1.0
C≥0
P ≥ 200
A ≤ 0.5
9.6B*C
13
Ex. 1: Solution continued
Any specification of the decision variables that
a f e a he
de c
a
ad
be
in the feasible region.
For example, V = 2, P = 300, T = 150, A = 0.4, B =
0.3 and C = 0.3 is in the feasible region.
 An optimal solution to an optimization model:
any point in the feasible region that optimizes (in
this case maximizes) the objective function.
 It can be determined that the optimal
solution to its model is V = 5, P = 200, T = 100, A =
0.294, B = 0, C = 0.706, and z = 209.384.
14
A static model is one in which the decision
variables do not involve sequences of decisions
over multiple periods.
A dynamic model is a model in which the
decision variables do involve sequences of
decisions over multiple periods.
A linear model is one in which decision
variables appear in the objective function and
in the constraints of an optimization model.
The Daisy example is a nonlinear model. In
general, nonlinear models are much harder to
solve.
17
If one or more of the decision variables must
be integer, then we say that an optimization
model is an integer model.
If all the decision variables are free to assume
fractional values, then an optimization model is
a noninteger model.
The Daisy example is a noninteger example since
volume, pressure, temperature, and percentage
composition are all decision variables which may
assume fractional values.
Integer models are much harder to solve then
noninteger models.
18
A deterministic model is a model in which for
any value of the decision variables the value of
the objective function and whether or not the
constraints are satisfied is known with
certainty. If this is not the case, then we have
a stochastic model.
If we view the Daisy example as a deterministic
model, then we are making the assumption that for
given values of V, P, T, A, B, and C the process yield
will always be the same.
Since this is unlikely, the objective function can be
viewed as the average yield of the process for given
decision variable values.
19
The Seven-Step Model-Building
Process
1. Formulate the Problem
Define the problem.
Specify objectives.
Determine parts of the organization to be studied.
2. Observe the System
Determines parameters affecting the problem.
Collect data to estimate values of the parameters.
3. Formulate a Mathematical Model of the
Problem
20
4. Verify the Model and Use the Model for
Prediction
Does the model yield results for values of decision
variables not used to develop the model?
What eventualities might cause the model to become
invalid?
5. Select a Suitable Alternative
Given a model and a set of alternative solutions,
determine which solution best meets the
organizations objectives.
21
6. Present the Results and Conclusion(s) of
the Study to the Organization
Present the results to the decision maker(s)
If necessary, prepare several alternative solutions
and permit the organization to chose the one that
best meets their needs.
Any non-a
a f he
d
ec
e da
may have stemmed from an incorrect problem
definition or failure to involve the decision maker(s)
from the start of the project.
In such a case, return to step 1, 2, or 3.
22
7. Implement and Evaluate
Recommendations
Assist in implementing the recommendations.
Monitor and dynamically update the system as the
environment and parameters change to ensure that
recommendations enable the organization to meet its
goals.
23
Lesson 03: Linear Algebra for
Operations Research
Gaussian Elimination
!
!
There are cases, however, where a system of
equations has no solution (inconsistent system of
equations), or it may have an infinite no. of solutions.
The no. of solutions depends on the rank of the
matrices.
In LP, the resulting systems of equations most often
have an infinite number of solutions and we try to
identify the optimal solution.
23
Gauss-Jordan Method
!
!
LP problems are converted to systems of
simultaneous linear equations and then solved by
the simplex algorithm.
At each iteration, simplex solves for a new
variable, a system of n simultaneous linear
equations in n unknowns (variables), using the
Gauss Jordan method.
24
Gaussian Elimination- Elimination
of variables
!
As an example, we solve the following
system of equations.
• Step 1: we solve the first equation for x1 as
follows.
25
Gaussian Elimination
• Step 2: Substitute x1 in other equations.
Variable x1 has been eliminated from the
equations.
• Step 3: Apply same idea to the remaining
equations. We have
26
Gaussian Elimination
• Step 4: Substitute x2 in other equations.
Variable x2 has been eliminated from the
equations.
• Step 5: Finally, eliminate x3 and we have
and substitute in other other equation.
In Sum, the solution is
27
Gaussian EliminationTransformation of equations
!
Step 1: Look at the same example, we
transform the equations as follows.
28
Gaussian EliminationTransformation of equations
!
Step 2: We subtract the proper multiplies of
the 2nd equation from later equations.
29
Gaussian EliminationTransformation of equations
!
Step 3: We perform the following.
• Now, we have a triangular liner system
that is easy to solve.
30
Gaussian Elimination as Triangular
Factorization
• A matrix view of Gaussian elimination
• Observe: E1(Ax)=(E1A)x=E1b
31
Gaussian Elimination as Triangular
Factorization
• The resulting equations of E1(Ax)=(E1A)x=E1b
are exactly the same transformation we just
did in step 1.
• The resulting equations of (E2E1A)x=(E2E1)b
are the equations we had in step 2 above.
32
Gaussian Elimination as Triangular
Factorization
• Finally, the resulting equations of
(E3E2E1A)x=(E3E2E1A)b are the equations we
had in step 3 above.
• Now, the (E3E2E1A)x=(E3E2E1A)b is exactly the
easy-to-solve triangular system we had before.
33
Gaussian Elimination as Triangular
Factorization
• The LU factorization: the matrix multiplication
above can be generalized as follows.
U= (E3E2E1)A=VA, where
34
Gaussian Elimination as Triangular
Factorization
• Interpretation: Gaussian elimination can be
considered as finding a lower triangular matrix
V that, when multiplied left to A, gives desired
matrix U.
• Alternatively, U=VA can be rearranged as A=LU
where L is lower triangular, and U is the upper
triangular matrix and A=LU is called a matrix
factorization.
• In particular, L is just the matrix of multipliers
that we used in elimination process.
35
Gaussian Elimination as Triangular
Factorization
• Use LU factorization: Given A and b, we solve
Ax=b for x.
• We solve (LU)x=b. Firstly, we let L(Ux)=Lv=b for
v. Then, we solve Ux=v for x. Both of these
systems are easy to solve.
• Revisit the example:
36
Gaussian Elimination as Triangular
Factorization
• Revisit the example: continue solve for x
• We just applied forward and backward
substitution to solve for v and x above.
• How does LU factorization help in solving
system of equation?- Consider now bà b’
37
Gaussian Elimination as Triangular
Factorization
• Consider b=(3,1,0,0)à b’=(-4,1,0,0) and
others remain unchanged. Since we
already know L and U, we just repeat
38
Gaussian Elimination with Pivoting
• Consider Ax=b below.
• First step results in the equations below. We can
not continue further as the diagonal element is
0.
39
Gaussian Elimination with Pivoting
• We rearrange rows and/or columns as follows.
After which, it is trivial to solve for x=(1,1,2).
• Algorithm:
40
Gaussian Elimination
• Non-square linear equations: the Gaussian
elimination can be extended to m linear
equations with n variables where m<>n.
• Example 1:
41
Gaussian Elimination
• Example 1 continue:
• Resulting equations
42
Gaussian Elimination
• Example 2:
• Resulting equations
43
Exercise
To practice the Gaussian Elimination method, solve the
following system of three simultaneous linear
equations in three unknowns.
2x1 + 3x2 + x3 = 11
-3x2 + 2x3 = 5
x1 + x2 + x3 = 7
a) Solve in algebraic form and specify in each iteration
the pivot column, pivot row and pivot element
b) Solve in matrix form, by inverting manually the
appropriate matrix and postmultiply by the RHS vector.
44