Travelling Salesperson Problem
MEI Conference 2009: D2
D2 Networks
7
A
B
Route Inspection Problems (Chinese Postman)
7
(3) A
B (3)
C
3
4
2
Identify the odd vertices in the network
Consider all the routes joining pairs of odd vertices and select the
shortest
•
Find the sum of the weights on all the edges
• Shortest distance is the sum of all weights plus the extra that must
be travelled
•
Find a tour which repeats the edges found in step 2.
AD
BC
6
3
9
Choose to repeat AC and BD
Length of all the edges is 24
Length of minimum tour 24 + 6 = 30
Possible tour
ACDBCADBA
A
7
2
-
A
B
C
D
B
7
3
4
C
2
3
-
complete matrix
D
4
2
-
A
B
C
D
A
7
2
4
B
7
3
4
1 4
Use nearest
neighbour
algorithm
starting at A
Odd vertices A, B, C, D
2
4
6
D
Upper Bounds: Nearest Neighbour Algorithm
D (3)
•
•
AC
BD
2
distance matrix
(3) C
7
2
9
4
3
2
AB
CD
3
2
A
B
C
D
A
7
2
4
C
2
3
2
2
B
7
3
4
C
2
3
2
D
4
4
2
-
A complete matrix
is required for
NNA
3
D
4
4
2
-
For a full
solution, NNA
should be
repeated
starting at each
vertex in turn
upper bound length 15
Lower Bounds
•
•
•
Delete a vertex and the edges incident on it
Find a minimum spanning tree for the remaining network
Reconnect the deleted vertex by the two shortest edges
e.g. Delete B
7
B
minimum connector
2+2=4
reconnect B (BC, BD) 3 + 4 = 7
A
4
3
Lower bound
4 + 7 = 11
2
C
2
Repeat for all vertices.
Greatest lower bound is the
best lower bound
D
Find a tour such that least upper bound > tour > greatest lower bound
Floyd’s Algorithm
B
Example
7
There are circumstances in which we require all shortest paths in a
network. Clearly we could achieve this by applying Dijkstra n times one for each starting vertex. This would have complexity 0(n3). An
alternative cubic algorithm is Floyd’s algorithm which achieves the
same ends.
•
•
A “distance” (or weight) matrix. This gives the “length” of each
arc in the network, and is symmetric unless we are dealing with a
digraph. The distance from a vertex to itself is shown as 0.
Where there is no direct connection (i.e. no edge) the distance is
shown as α.
A route matrix. The initial version of this has a column of 1’s
followed by a column of 2’s, etc.
The Algorithm
3
A
The method successively modifies the following two matrices -
1
3
C
3
D
Initial Distance Matrix (D0)
destination
A B C D
A ∞ 7 3 5
start B 7 ∞ 3 ∞
C 3 3 ∞ ∞
D 5 ∞ ∞ ∞
E ∞ 1 6 3
E
∞
1
6
3
∞
Step 2
Compare every element in the chosen row or column with the
sum of the corresponding entries in the chosen row and
column.
These numbers represent
distances by single stage
routes. The highlighting is ready
for the first iteration.
Step 3
If the calculated sum is smaller than the element then
replace the element by the sum.
D(A)
Via A
Step 5 Repeat steps 1 to 4 until all rows/columns have been
chosen in step 1.
Step 6
The distance matrix will now contain the shortest distances.
The route matrix shows the next vertex “en route” from one
to another along a shortest path. It can be used to construct
complete shortest paths.
start
Initial Route Matrix
destination
A B C D
A A B C D
B A B C D
C A B C D
D A B C D
E A B C D
start
Choose the first row and column, and highlight all elements
in the chosen row and column. Note the iteration (via A)
If the element is replaced then we need to update the
corresponding entry of the route matrix. The value to
replace it with is found in the route matrix from the previous
iteration - in the same row as the entry it is replacing, but in
the column given by the iteration.
E
5
Step 1
Step 4
6
(R0)
E
E
E
E
E
E
These letters represent the first vertex
on corresponding single stage routes,
so the first vertex encountered on the
single arc route from E to D is D
R(A)
A
B
C
D
E
destination
A B C
∞
7
3
7 14 3
3
3
6
5 12 8
∞
1
6
D
5
12
8
10
3
E
∞
1
6
3
∞
start
A
B
C
D
E
destination
A B C
A B C
A A C
A B A
A A A
A B C
D
D
A
A
A
D
E
E
E
E
E
E
Look at the highlighted row and column: 7 + 5 = 12
(B)
D
Via B
start
A
B
C
D
E
destination
A
B C
14
7
3
7
14 3
3
3
6
5
12 8
8
1
4
1 + 3 = 4; 4 < 6
R(B)
D
5
12
8
10
3
E
8
1
4
3
2
start
destination
A B C
A B B C
B A A C
C A B A
D A A A
E B B B
D
D
A
A
A
D
E
B
E
B
E
B
The route matrix shows that the first
vertex on the route from E to A is vertex B
D(C)
Via C
start
A
B
C
D
E
destination
A B C
6
6
3
6
6
3
3
3
6
5 11 8
7
1
4
R(C)
D
5
11
8
10
3
E
7
1
4
3
2
A
B
C
D
E
start
This shows that there is a path of
length 7 from vertex E to vertex A
D(D)
Via D
start
A
B
C
D
E
destination
A B C
6
6
3
6
6
3
3
3
6
5 11 8
7
1
4
A
B
C
D
E
destination
A B C
6 6 3
6 2 3
3 3 6
5 4 7
7 1 4
D(E)
Via E
start
E
7
1
4
3
2
A
B
C
D
E
start
R(E)
D
5
4
7
6
3
E
7
1
4
3
2
D2: Linear Programming
D
D
C
A
A
D
E
C
E
B
E
B
This entry came from row
5, column 3 in the previous
route matrix
R(D)
D
5
11
8
10
3
destination
A B C
C C C
C C C
A B A
A A A
B B B
start
A
B
C
D
E
Introduction
Many problems in Decision Maths are to do with finding the best solution
to a problem, given that the solution must satisfy a number of
constraints. Linear programming is, perhaps, the most obvious example
of this type of problem. Linear programming was developed during the
Second World War to solve logistical problems. The term LINEAR refers to
the functions defining the problem which must be linear i.e. the output is
proportional to the input. The simplex algorithm was developed by
Dantzig in the 1940s in order to solve particular types of Linear
Programming problems.
The Syllabus
destination
A B C
C C C
C C C
A B A
A A A
B B B
destination
A B C
C C C
C E C
A B A
A E E
B B B
D
D
E
B
E
D
D
D
C
A
A
D
E
C
E
B
E
B
E
C
E
B
E
B
The final matrix D(E) shows all of the shortest distances, so the shortest
distance from vertex C to vertex D is 7.
The R(E) matrix shows that the first vertex on the route is vertex B. Now
use the R(E) matrix again to find the first vertex on the shortest route from
vertex B to vertex D. This is vertex E, so the route is now seen to be C, B,
E, .....
Finally the first vertex on the shortest route from vertex E to vertex D is
vertex D. So the final route is C, B, E, D.
The Simplex Algorithm: Be able to solve simple maximisation problems
with < constraints and with two or more variables. Use the tabular form
of the algorithm
Geometric interpretation: Be able to identify tableaux (initial,
intermediate and final) with points, particularly in the case of problems
involving two or three variables. Show alternative feasible points and
their costs.
Greater than ( > ) inequalities: Understand the use of two stage
simplex, and of the big M methods to construct an initial feasible solution
to problems involving > constraints, including the possibility that there is
no feasible solution
Equality constraints: Understand how to model an equality constraint
using a pair of inequality constraints.
Getting started: Formulation
Translating a real life problem into a linear programming problem is called
formulating the problem and is an example of mathematical modelling.
Each problem must have clearly defined variables, an objective function
and is subject to certain constraints
Slack Variables
In order to enable problems to be converted into a format that can be
dealt with by computer, slack (or surplus) variables are introduced to
change the constraint inequalities into equalities. Each vertex of the
feasible region would then be defined by the intersection of two lines
where the variables equal zero.
The Simplex Method
The Simplex Method starts at one vertex and systematically moves round
all the vertices, increasing the objective function as it goes, until it
reaches the one with the optimal solution. This is easy to visualise on a 2
dimensional problem, but can be generalised to include more variables.
Once there are more than two variables, a graphical approach is no
longer appropriate, so we use the simplex tableau, a tabular form of the
algorithm which uses row reduction (remember Gaussian elimination?) to
solve the problem.
Adaptations of the algorithm
Integer programming: In many practical situations, the solution to a
problem must take an integer value. This situation is much more difficult
to guarantee a correct solution, but since the IP solution is almost always
close to the LP solution a heuristic approach can be used.
Greater than Inequalities: these lead to problems which involve
minimisation. In 2 dimensions these problems like this can be solved
graphically in the same way as maximisation problems. There are also
adaptations of the Simplex algorithm to deal with this type of problem.
Negative values: The simplex algorithm requires all variables to be nonnegative. If a variable, x say, has a negative value, it will be replaced in
the formulation by x1 - x2 where x1 < x2.
Other types of problems: Simplex can be adapted to solve a range of
problems, such as shortest path (minimisation), maximum flow
(maximisation) and game theory (maximin).
Formulation
Define Variables
(x, y, z or x1, x2, x3, ....) which are
usually restricted to positive values
Define Objective Function
(Maximise/Minimise an expression in
terms of variables)
Define Constraints (inequalities)
restrictions on variable values
Our example:
Let x be ........
Let y be ........
Maximise x+y
Subject to
2x + y ≤ 16
2x + 3y ≤ 24
y≤ 6
x, y ≥ 0
Note on finding pivot column (step 2)
Can choose any variable in objective row with negative coefficient, but
usual to pick largest magnitude, and priority is to increase original
rather than slack variables (NB for two stage simplex will sometimes
have minimise objective - if so then pick positive and decrease).
Note on ratio test (step 3)
Divide each R.H.S. value by the corresponding element in the pivot
column - ignore negative ratios and division by zero - choose row
giving smallest and mark pivot column element as pivot.
Worked example
P
1
0
0
0
x
-1
2
2
0
y
-1
1
3
1
s1
0
1
0
0
s2
0
0
1
0
s3
0
0
0
1
RHS
0
16
24
6
R1
R2
R3
R4
1
0
0
0
-1
2
2
0
0
0
0
1
0
1
0
0
0
0
1
0
1
-1
-3
1
6
10
6
6
+R4
-R4
-3R4
R4/1
1
0
0
0
0
0
1
0
0
0
0
1
0
1
0
0
½
-1
½
0
-½
2
-3/2
1
9
4
3
6
+R3
-2R3
R7/2
same
1
0
0
0
0
0
1
0
0
0
0
1
¼
½
¾
-½
¼
-½
-¼
½
0
1
0
0
10
2
6
4
+½R2
R10/2
+1½R2
-R2
Simplex Algorithm (Tableau)
Simplex: Two Stage and Big M methods
1.
2.
3.
4.
5.
When?
6.
7.
Represent the problem in a tableau
Use the objective row to find the pivot column
Use the ratio test to find the pivot element
Divide through the pivot row by the pivot element
Add/subtract multiples of the transformed pivot row to/from the
other rows to create zeros in the pivot column
Repeat until no negatives in objective row
Set basic variables (with columns containing a 1 and 0s) to
corresponding RS, and non-basic variables to zero.
For ≥ constraints
Minimisation problems
Two Stage Simplex
Simplex relies on (0,0) being a feasible solution. If it is not, then we
introduce a set of artificial variables a1, a2, etc. to all ≥ constraints (to
ensure that none of the variables have a negative value) and a new
objective function Q = a1 + a2 + …
The artificial variables effectively move us from (0,0) into the feasible
region.
We now need surplus (as opposed to slack) variables which are
subtracted from the constraints.
In a similar problem to our earlier one, the constraints could be
The big M method
Using the same problem as for Two Stage Simplex, we have
Maximise x+y
Subject to
2x + y + s1 = 16
2x + 3y + s2 = 24
x - s3 + a1 = 1
y - s4 + a2 = 1
Maximise x+y
Subject to
2x + y ≤ 16
2x + 3y ≤ 24
x, y ≥ 1
Firstly we need to add the
slack variables to the ≤ constraints and subtract the surplus variables
from the ≤ constraints. Since clearly x=y=0 is not a feasible solution
we add artificial variables and form a supplementary problem giving
new constraints: x - s3 + a1 = 1 and y - s4 + a2 = 1
The additional objective Q = a1 + a2 is written in terms of the other
variables and rearranged Q + x + y - s3 - s4 = 2. We then aim to
minimise Q = a1 + a2 to ensure that we are in the feasible region
before completing the Simplex in the normal way
Q
1
0
0
0
0
0
P
0
1
0
0
0
0
x
1
-1
2
2
1
0
0
0
0
0
Q
1
0
0
0
0
0
0
1
0
0
0
0
P
0
1
0
0
0
0
0
0
0
0
1
0
x
0
0
0
0
1
0
1
0
y
1
-1
1
3
0
1
s1
0
0
1
0
0
0
s2
0
0
0
1
0
0
s3
-1
0
0
0
-1
0
s4
-1
0
0
0
0
-1
a1
0
0
0
0
1
0
a2
v
0
2
0
0
0 16
0 24
0
1
1
1
1
-1
1
3
0
0
0
1
0
0
0
s1
0
0
1
0
0
0
0
0
0
1
0
0
s2
0
0
0
1
0
0
0
-1
2
2
-1
0
s3
0
-1
2
2
-1
0
-1
0
0
0
0
-1
s4
0
-1
1
3
0
-1
-1
1
-2
-2
1
0
a1
-1
1
-2
-2
1
0
0
0
0
0
0
1
a2
-1
1
-1
-3
0
1
1
y
0
0
0
0
0
1
1
1
14
22
1
1
v
0
2
13
19
1
1
Since the artificial variables are now 0, these, and the supplementary
objective can be removed and the remaining tableau solved (2
iterations) to give the solution as before.
Our aim is to maximise P=x + y - M(a1 + a2)
where M is an arbitrarily large value and hence to achieve our
maximisation we require a1 and a2 to be zero.
Re-writing without the artificial variables:
P - x - y + M(1 - x + s3 + 1 - y + s4)=0
⇒
P - (1+M)x + (1+M)y + Ms3 + Ms4 = -2M
P
x
y
1 -(1+M) -(1+M)
0
2
1
0
2
3
0
1
0
0
0
1
s1
0
1
0
0
0
s2
0
0
1
0
0
s3
M
0
0
-1
0
s4
M
0
0
0
-1
a1
0
0
0
1
0
a2
0
0
0
0
1
RHS
-2M
16
24
1
1
P
1
0
0
0
0
x
y
0 -(1+M)
0
1
0
3
1
0
0
1
s1
0
1
0
0
0
s2
0
0
1
0
0
s3
-1
2
2
-1
0
s4
a1
M 1+M
0
-2
0
-2
0
1
-1
0
a2
0
0
0
0
1
RHS
1-M
14
22
1
1
P
1
0
0
0
0
x
0
0
0
1
0
s1
0
1
0
0
0
s2
0
0
1
0
0
s3
-1
2
2
-1
0
s4
a1
a2
-1 1+M 1+M
1
-2
-1
3
-2
-3
0
1
0
-1
0
1
RHS
2
13
19
1
1
y
0
0
0
0
1
Our artificial variables are no-basic (zero) and the M only appears in
their columns. They can therefore be deleted and the remaining
tableau is as before for further iteration.
D2: Logic and Boolean Algebra
Boolean variables: p, q
Truth tables
NOT
AND
p
1
1
0
0
p
1
1
0
0
~p
0
0
1
1
IMPLIES
p
1
1
0
0
p∧q
1
0
0
0
q
1
0
1
0
p
1
1
0
0
p
1
1
0
0
EQUIVALENCE
p⇐q
1
1
0
1
q
1
0
1
0
p∨q
1
1
1
0
q
1
0
1
0
IMPLIED BY
p⇒q
1
0
1
1
q
1
0
1
0
OR
p
1
1
0
0
q
1
0
1
0
p⇔q
1
0
0
1
a∨b
1
1
1
0
b
1
0
1
0
~(a∧b)
0
0
0
1
RHS
a
1
1
0
0
b
1
0
1
0
~a
0
0
1
1
~b
0
1
0
1
~a∧~b
0
0
0
1
e.g Prove a∧(b∨c) ⇔ (a∧b)∨(a∧c)
LHS
a b
1 1
1 1
1 0
1 0
0 1
0 1
0 0
0 0
c
1
0
1
0
1
0
1
0
b∨c
1
1
1
0
1
1
1
0
a∧(b∨c)
1
1
1
0
0
0
0
0
e.g. Simplify
Complement laws
Double complement
Indempotent laws
Identity laws
Dominance laws
Commutative laws
Distributive laws
Absorption law
DeMorgan's laws
p∨(~p∧q)
= (p∨~p)∧(p∨q)
= 1∧(p∨q)
= p∨q
e.g. Prove ~ (a∨b) = ~a∧~b
LHS
a
1
1
0
0
p∨~p = 1
p∧~p = 0
~(~p) = p
p∨p = p
p∧p = p
p∨0 = p
p∧1 = p
p∨1 = 1
p∧0 = 0
p∨q = q∨p
p∧q = q∧p
p∨(q∧r) = (p∨q)∧(p∨r)
p∧(q∨r) = (p∧q)∨(q∧r)
p∧(p∨q) = p
~(p∨q) = ~p∧~q
~(p∧q) = ~p∨~q
RHS
a b
1 1
1 1
1 0
1 0
0 1
0 1
0 0
0 0
c
1
0
1
0
1
0
1
0
a∧b
1
1
0
0
0
0
0
0
a∧c
1
0
1
0
0
0
0
0
(a∧b)∨(a∧c)
1
1
1
0
0
0
0
0
Laws of Boolean algebra
The most obvious way to simplify Boolean expressions is to manipulate
them in the same way as you would manipulate algebraic expressions.
A set of rules for is needed for this and these are shown in the table
Circuits
a
b
c
this circuit can be written as
a∨(b∧c)
Logic Gates
Students need to be able to apply their knowledge of truth tables and
circuits to simple circuits involving logic gates: NOT, AND, OR, NAND,
NOR.
D2: Decision Analysis
Terminology
Decision trees can be drawn in situations where the outcome of
events is uncertain.
EMV (expected monetary value) can be calculated for various
courses of action to assist decision-making in situations where
probabilities can be assigned to the outcomes.
The decision tree has three types of nodes
Decision node
Chance node
Payoff node
Example
The organiser of a tennis tournament has to decide whether to take out
insurance against bad weather. For a premium of £1250, she will receive
a payout of £10000 if it rains on the day of the tournament. The
organiser calculates that if it rains, the losses will total £13000 but if it is
fine they will make £9500 profit. The weather forecast estimates that
there is a 25% chance of rain on the day of the tournament.
What would you advise the organiser to do?
10000 – 1250 – 13000
insurance – premium – losses
(–4350 X 0.25) +
(8300 X 0.75)
-£4250
rain (0.25)
£5162.50
insure (-£1250)
fine (0.75)
£5162.50
+£6500
select the
larger
figure
£8300
7500 – 1000
profit - insurance
don’t
insure
rain (0.25)
-£13000
£3875
fine (0.75)
(–13000 X 0.25) +
(9500 X 0.75)
Solution
Advise the organiser to take out insurance
EMV £5162.50
£9500
Teaching D2 using the online resources
Chapter 1: Linear programming, the simplex method
•
•
•
•
•
•
•
•
This chapter is quite challenging for students. It can be difficult to teach and off-putting for students.
It is worth taking some time to make sure that they can interpret tableaux at every stage of the working –
including giving values of the slack/surplus variables.
Don’t be tempted to take too long on this chapter – it can easily become “death by simplex”. Students are
expected to have a good understanding of what the method is doing but are not going to be asked to
perform long and complicated simplex tableaux in the exam.
When starting out on basic simplex and again when doing 2 stage simplex and big M method, it is worth
tackling a reasonably complex 2 variable problem and working the simple alongside a graph. This gives
students a much clearer picture of how the simplex algorithm works and what it is actually doing. This can
make tacking 3 or 4 variable problems less intimidating.
Don’t forget to subtract the surplus variables when tackling 2 stage simplex.
Some find it difficult to subtract multiples of M and hence make algebraic errors in the big M method
Students may be asked to do an ordinary simplex but it s unlikely that they will be asked to do a full 2-stage
or big M
Two stage simplex has come up on the exam more often that big M
Chapter 2: Networks
•
•
•
•
•
•
•
•
•
•
Route inspection is the topic which students seem to find easiest so this could be tackled first.
TSP uses the MST algorithms learned in D1so make sure the students are secure in this first.
It is important to distinguish between TSP and RI and which to use for what type of problem.
When teaching TSP, make sure students understand the distinction between the classical problem, which
uses a complete network, and the practical problem, which almost certainly doesn’t.
When finding a lower bound for TSP, we use the method where a vertex is deleted. The resulting lower
bound DOES NOT give a tour, except in exceptional circumstances.
Students don’t understand why it is the GREATEST lower bound and LEAST upper bound.
Floyd’s algorithm is straightforward but takes a bit of time to do thoroughly.
When applying Floyd’s algorithm students can easily get lost using the distance and the route matrices side
by side.
Draw attention to the way to change the route matrix
Use colours – highlighter pens are indispensable when doing Floyd’s algorithm.
Chapter 3: Decision analysis
•
•
•
•
•
Decision analysis is a dynamic tool. The power of decision analysis is not in the numbers at the decision
node, it is the ability to change the utilities and probabilities and see how this affects the decision node.
When drawing a decision tree you can use increase in wealth (profit) or total wealth but it is important to
be consistent.
EMV can be calculated in the same way as expectation in statistics. You do not need to have drawn a
decision tree in order to calculate the EMV for a situation.
When considering utility always use total wealth, since utility is a function of wealth.
The utility function is specific to a person or situation and is given for each problem. It can be loosely
described as a measure of “happiness”
Chapter 4: Logic and Boolean algebra
•
•
•
•
•
Students usually prefer to use truth tables to prove logical propositions. They need to know, or be able to
work out, the truth tables.
They do need to know how to use the laws of Boolean algebra as well – it can be specified on the exam
paper.
Students must be able to distinguish between a combinatorial circuit and a switching circuit.
They can be asked to produce tables of outputs for circuits
They are not required to memorise the half adder.
D2 Online resources
Teaching D2 using the online resources
Every chapter contains
• A glossary of terms
• An end of chapter assessment – the answers to this are in the teachers resources
Every section within the chapter contains
• A study plan
• Crucial points that students need to be aware of when learning the material
• Notes that explain the topic and more examples than are in the textbook.
• Worked solutions to some of the questions in the textbook.
A multiple choice test which can printed off and used in class or be done online by registered students. They
will receive instant feedback.
The additional resources are being added to all the time. Below is a list of some of the things currently available
Topic
section
Linear
The simplex method
Programming
Networks
Decision
analysis
Logic
about
The initial simplex tableau,
Manipulating systems of equations,
The simplex method
Problems involving ≥
Terminology, Post-optimal analysis,
constraints,
Two-stage simplex, The big M method
Problem solving
Equalities and practicalities
Route inspection
Euler Cycles. Solving route inspection
problems
Travelling salesperson Hamiltonian cycles. The travelling
salesperson problem, The practical
problem and the classical problem,
Upper and lower bounds, Tour
building
Floyd’s algorithm
Drawing an activity network,
identifying a critical path, earliest and
latest event times, float
Decision analysis
Construct and interpret Decision
trees. Calculate EMV. Understand
Utility functions and use utility to
compare alternatives
Propositional calculus Be able to understand and form
and truth tables
compound propositions using the
correct symbols. Use truth tables to
analyse propositions
Boolean algebra
Manipulate Boolean expressions and
use the laws of Boolean algebra
Switching circuits
Be able to model compound
propositions with simple switching and
combinatorial circuits.
Additional resources
Interactive excel
Interactive excel
Power point
Power point
Power point