Algorithms of Linear Programming
Marks Speight and Taylor Bishop
Abstract
This paper examines the improvements of linear
programming time complexity and proves the importance
of optimization in applied mathematics. The time efficiency
of searching known constraints and maximizing the value
of minimal inputs within the feasible region of a maximal
value has increased from the convex hull and brute force to
the geometric and exponential, and the simplex pivots with
a linear complexity. The convex hull’s obstacles of finding
all of the extreme points with brute force is overcome with
its linear time of finding the high points. The geometric
method is useful because the slope never changes as the
system of linear equations solves for the maximal value
which reduces the area of the feasible region of possible
maximal value points until its value is found. Yet, the
method limits at two decision variables. The simplex
method involves hundreds and thousands of problem
constraints. Invented by George Danztig in 1947, the
simplex algorithm is the most common method of solving
maximization problems. The conclusion of this paper result
prove the importance of the method by demonstrating its
capabilities in a simple maximization problem of interest.
Key Words: Traveling Salesman Problem, Dynamic
Programming, Branch and Bound, Convex Hull,
Minimum Spanning Tree
paper, the coefficients of the objective function and the
constraint equation must be identified. Linear
programming assumes the values to be constant over time.
Many times during a study of a problem the factors of the
constraints will change, but for the sake of linear
programming and the methods described in this paper, the
variables will remain certain. In each of these methods:
convex hull, geometric and simplex, the basic steps of
formulating the linear programming model will identify the
parameters of the decision of inputs in linear algebraic
functions and matrices, identify the constraints which
restrict the decision of inputs in the form of linear function
inequalities, and determine the objective which is to be
optimized and the maximal value is identified.
2. Linear Programming Problems
Problem of interest:
maximize 3x + 5y
subject to x + 3y ≤ 6
x+y≤4
x ≥ 0, y ≥ 0
machine
M
N
Widget a
1
1
Widget b
3
1
Hours
6
4
1. Introduction
The linear programming problem takes inequalities
relating to a situation and finds the best value under those
conditions. Linear programming is an important part of
optimization in real life problems. The algebraic steps of
solving this problem work with a simple two variable
problem, though typical systems can have dozens of
hundreds of variables. Real life application extends the
model considering added constraints. The time
complexities discussed in this paper focus on the efficiency
of finding the boundary of the feasible region of the
constraints.
Examples of this problem include operational research,
a branch of applied mathematics giving advice which
perform the objective: minimization of production costs
and maximization of profit. Limitations of linear
programming include business problems which require the
form of a quadratic equation. Before the linear program
method is applied in the maximization problem in this
The first method used to solve this problem is the convex
hull which constructs a hull of extreme points within the
feasible region bound by the linear inequalities. The hull
represents the smallest polygon of intersecting constraint
sets. The algorithmic clockwise linear search of the outer
bound of these points produces the extreme points which
when substituted into the maximizing equation 3x + 5y =
z, where x is the number of Widget a produced and y is the
number of Widget b, the point (3, 1) of the hull produces
the maximal value of 14 and the plant will benefit from
producing three Widget a’s and one Widget b, the greatest
value of the four extreme points.
Brute force finds the extreme points and is a
limitation of the convex hull. The feasible region contains
a line segment connecting two points, pi and pj if all other
points of the set lie on the same side of the straight line
through these two points. The test repeats this test for every
pair of points and produces a list of convex points.
The geometric method is another graphical method of
finding the feasible region. The intersection points of
constraint lines are solved by a system of linear equations.
The sign of the constraints define the quadrant of the
Cartesian plane of which the lines intersect, and calculating
the value of the maximal value is possible by simply
shifting the slope of a test z value of the maximizing
equation up or down, i.e. 3x + 5y = 20 is obviously not in
the feasible region. Shifting the value will eventually find
the maximal value of 14.
maximize 3x + 5y = P
subject to
x + 3y ≤ 6
x+y≤4
x ≥ 0, y ≥ 0
The method converts the system of inequalities
(constraints) to equations using slack variables, sets the
objective function equal to zero and creates a tableau with
active variables labeled. The pivot is the column with the
most negative number on the left side of the bottom row (if
all the numbers on the bottom row are positive, the search
stops). The constant column entry is divided by the positive
entries of the pivot column and the pivot row is the smallest
ratio of those values. The pivot is the entry is the positive
number in the pivot column and pivot row.
1x + 3y + 1u + 0v = 6
1x + 1y + 0u + 1v = 4
-3x – 5y + 0u + 0v = 0
u = 6, v = 4
u
x
1
y
3
u
1
v
0
v
1
1
0
1
4
-3
-5
0
0
0
0
1
-
1
0
0
0
2
u
The time complexity of geometric is O(n2); and, similar
to brute force, it is an inefficient method of finding the
feasible region of problems with extreme points which
increase exponentially with the number of constraint
equations.
During WWI, George Dantzig recognized the
limitations of the previous algorithms and discovered a
new way of handling the constraints of linear programming
problems which is known as optimization. The brute force
search of extreme points presents a problem for linear
programming because of the size of the problem. The
simplex method inspects a fraction of the extreme points
and finds the maximal value. Like the convex hull and
geometric methods, this method finds the extreme points in
the feasible region and checking if the value of the object
if function is improved by pivoting to adjacent extreme
points. If not, the search stops, if so, continue searching
points until the optimal solution is found.
3
1
v
3
4
x
y
3
1
0
0
0
0
0
3
1
6
2
3
2
1
3
5
10
3
1
3
-
2
(constant)
2
1
2
2
1
1
2
1
14
x = 3, y = 1
Select the pivot which is the positive entry in the pivot
column and pivot row and identify the new active variable.
Perform row operations to make the remaining elements in
the pivot column equal to zero and repeat the process by
identifying the most negative entry in the last row. The
process is repeated by identifying the most negative entry
in the last row. Once the left side of the last row is all
positive, the solution is the right most entry in the row.
Geometric
3. Analysis
The convex hull’s brute force worst case run time is O(n3)
of initially sorting the n points of the sets. The best time of
searching the clockwise extreme points is O(n). This
proves to be an inefficient algorithm. The geometric
method of linear programming is also limited in that it is
only useful for problems involving two decision variables
and cannot be used for three or more variables. The
geometric method saves time from the convex hull method
because instead of searching exhaustively for extreme
points and then searching the positive and negative sides of
a plane, the algorithm uses one slope which it changes to
suit the possible input solutions of the feasible region. The
obvious disadvantages of the geometric search is the sign
check in the Cartesian plane which still leaves our results
with an O(n2) runtime. While our problem did not require
a method which searched thousands of variables and
problems constraints, the steps of the problem identified
the benefit of the pivot in saving time checking for
maximal points within the bounded region of the extreme
points. Given the n decision variables in our problem, the
pivots converge in O(n) operations with O(n) pivots, thus
the name linear programming. The method exploits the
extreme points by taking advantage of the geometry of the
problem, visiting vertices of a feasible set and checking the
optimality of each visited vertex. The pivot operations
become expensive without the advantages of cutting the
plane which is demonstrated with the advancement of the
geometric method incorporated into the simplex algorithm.
4. Results
Convex Hull
maximize 3x + 5y
subject to x + 3y ≤ 6
x+y≤4
x ≥ 0, y ≥ 0
The extreme points are (0,2), (0,0), (3,2), (4,0).
x=0 y=2
(0,2) = 10
3(0) + 5(2) = 10
x=0 y=0
3(0) + 5(0) = 0
(0,0) = 0
x=3 y=1
3(3) + 5(1) = 14
(3,1) = 14
x=4 y=0
(4,0) = 12
3(4) + 5(0) = 12
Minimizes 3x + 5y with a maximal value of 14.
Simplex
Simplex Matrix Example:
Maximize:3x+5y
Subject to: x+3y<=6
x+y<=4
x,y>=0
X
1
1
-3
Y
3
1
-5
1/3 R1
X
1/3
2/3
-4/3
3/2R2
X
0
1
0
Max=14
X=3
Y=1
S1
1
0
0
Y
1
0
0
Y
1
0
0
S2
0
1
0
M
0
0
1
R2-1(R1)
S1
1/3
-1/3
5/3
R1-3R2
S1
½
-1/2
1
C
6
4
0
S2
0
1
0
R3+5(R1)
M
0
0
1
C
2
2
10
S2
-1/2
3/2
2
R3+(4/3R)2
M
0
0
1
C
1
3
14
5. Conclusion
The solution to our problem is best solved by the
simplex method due to its reduction capabilities
which forge the idea of reducing known solutions
to linear problems and producing the concept of
optimization. The method can be difficult to
notice mistakes when compared to the graphical
method. The simplex method also has the
advantage of addressing problems with more
than two decision variables. Like the geometric
method, the simplex method starts with a guess.
There are volumes of data of the management
decisions of which the method provides
optimization techniques including ways to route
shipments for transportation firms, a way of
pricing products and the petroleum industry
explores, blends and schedules distribution with
the algorithm.
6. References
Levin, Anany. Introduction to the Design & Analysis of
Algorithms. 3 ed. England: Pearson, 2012. Print.