City Logistics Seminar at University of Illinois, 17th March 2008
Vehicle routing and scheduling with soft
time windows and the uncertainty of
travel times
Eiichi Taniguchi, Naoki Ando and Ali Gul Qureshi
Kyoto University
www.citylogistics.org
Outline
1 Introduction to city logistics
2 Exact solution for Vehicle Routing and
scheduling Problem (VRP) with soft time
windows using column generation
3 Vehicle Routing and scheduling Problem
with Time Windows (VRPTW) and the
uncertainty of travel times
1 Introduction to city logistics
Challenging issues (1)
• Competition
• Efficient logistics systems --- Just In Time
transport systems
• Freight carriers --- better services with
lower costs
• Shippers --- designated time windows
Challenging issues (2)
• Increase in urban
freight transport
– Congestion
– Negative
environmental
impacts
– Crashes
– Energy
consumption
Noise
Crash
Air pollution
Vibration
4
Congestion in urban areas
(20th century)
Trade-off
Efficient freight
transport
systems
Environment
friendly
systems
Efficiency
(21st century)
and
environment Efficient and environment friendly
freight transport systems
(function of
city logistics)
City Logistics
20th century
• Any major reduction in environmental
impact does not seem possible without
putting the logistics innovations
themselves into reverse (J. Cooper, 1991)
21st century
• ICT (Information and Communication
Technology), e-commerce (B2B, B2C)
• Development and deployment of ITS
(Intelligent Transport Systems)
• SCM (Supply Chain Management)
ERP (Enterprise Resource Planning)
CRP (Continuous Replenishment
programme)
• Outsourcing of freight transport---3PL
What is City Logistics?
• City logistics is the process for totally
optimising the logistics and transport
activities by private companies with the
support of advanced information systems
in urban areas considering the traffic
environment, its congestion, safety and
energy savings within the framework of a
market economy (Taniguchi et al. 2001)
Characteristics of City Logistics
• Total optimisation taking into account
environment, congestion, safety, energy
etc.
• Free activities of companies supported by
public sector through deregulation
• Full utilisation of advanced information
techniques including ITS and GIS
• Mindset of Co-opetition
Visions for city logistics
• We need visions for city logistics to
establish efficient and environmentally
friendly urban logistics systems through
the process of city logistics
Sustainability
Mobility
Global competitiveness
Efficiency　Environment friendliness　Congestion alleviation　Security　Structure
of visions
for city
logistics
Safety　Energy conservation　Labour force　Liveability
Two driving forces to promote
city logistics schemes
• Innovative technology (ICT and ITS)
• Corporate Social Responsibility (CSR)
2 Exact solution for Vehicle
Routing and scheduling
Problem (VRP) with soft time
windows using column
generation
Introduction
Truck based urban freight movement causes many problems
• Problems such as
Accidents
On street parking
Environmental Problems like generation of NOx, SPM and CO2,
etc.
• Time window adds further pressure on freight operators and their
various treatments can produce different results
• City Logistics deals with the measures to alleviate these problems
• Measures such as
Route optimization
Ideal location of logistic terminals
Controlling load factors
Cooperative Delivery Systems
etc.
Vehicle Routing
Shippers
Freight Carriers
Cost
Cost: less vehicles, less travel
Reliability
Idling: less waiting time
Reliability
Green Image
City Logistics
Administrators
Congestion
Environment
On-street Parking
Residents and
Customers
Faster Deliveries
Congestion
Environment
Safety
On-street Parking
VRPTW
[ 4 – 7 pm]
7
[ 10 – 12 am]
8
10
[ 2 – 3 pm]
4
[6 – 7 pm]
[ 10 – 11 am]
2
[ 1 – 5 pm]
Depot
k = 1, 2, … K
5
6
[ 10 – 11 am]
9
[ 3 – 4 pm]
Vehicle Routing Problem with
Time Win d ow s (VRPTW) is
defined as to find the minimum
cost routes for k vehicles to
service all the clients.
3
[ 7 – 8 pm]
Constraints: A vehicle can not
serve more clients than its
capacity. Delivery at each client
m u s t b e w it h i n s o m e p r e defined time windows.
VRPTW Variants
Hard Time Windows
Delivery is not possible outside
the specified Time Windows
(VRPHTW).
∞
Penalty
cost
Soft Time Windows
Delivery is possible outside
the specified Time Windows
with penalties (VRPSTW).
∞
Penalty
cost
[
]
ai
bi
Time
Hard Time Windows (HTW)
In exact solution techniques,
waiting is allowed at no cost
Time
[
]
]
ai
bi
bi’
Similarly allowing waiting
without cost in Soft Time
Windows results in:
Semi Soft Time Windows (SSTW)
Previous Research
Exact solution techniques were used for VRPHTW whereas heuristic
techniques were used for soft time windows variants such as
VRPSSTW
Most of the research is based on Solomon’s bench mark instances
Drawbacks
VRPHTW is important but it does not have the practicality and to
some extent the economic considerations.
Fair analysis was not possible since approx. solution of VRPSSTW
obtained by heuristics were under optimized as compared to exact
solution of VRPHTW
Rough environment analysis (total emissions) was possible based
on bench mark problems which only provide geographical locations.
A single value of speed was assumed though the generation of
emissions which depends on speed (i.e. traffic characteristics)
Objectives
Develop an exact solution approach for VRPSSTW
Using exact solution techniques for both VRPHTW and VRPSSTW, to
compare their relative characteristics with respect to
Cost of Delivery
Waiting Time
Environment Impacts
NOx
CO2
SPM
Using some practical logistics problems based on real road network
so that above mentioned parameters can be calculated on the basis
of actual traffic characteristics : Travel time, Travel Speed
A detailed link based environmental comparison identifying
emission intensities on each link produced by the VRPHTW and
VRPSSTW solutions
VRPTW Formulation
min   cij X ijk
(1)
kK ( i , j )A
subject to
  X ijk  1,
i C
(2)
 qi  X ijk  Q,  k  K
(3)
 X 1kj  1,
(4)
kK
iC
jV
jV
kK
jV
 X ihk   X hjk  0,
jV
 hC
kK
(5)
jV
k
X
 i1  1,
iV
kK


Sik  tij  S kj  1  X ijk M ijk ,
ai  Sik  bi ,
X ijk  {0, 1},
 i V ,
 i, j   A
(6)
 i, j   A
kK
kK
kK
(7)
(8)
(9)
Kohl et al., 1999.
Exact Solution Technique: Column Generation
Column generation or Dantzig-Wolfe decomposition, decomposes
the VRPTW problem (1 – 9) (NP-hard) into:
* Elementary shortest path problem with resource constraints
(ESPPRC) (3 – 9). (Sub-Problem)
* Set partitioning problem (Master Problem).
Past Research: VRPHTW
New Algorithm: VRPSSTW
New subproblem : Elementary shortest path problem with resource
constraints and late arrival penalties (ESPPRCLAP). New Labeling
Algorithm is developed.
c’ij = cij ,
cij + cl (sj - bj)
if sj ≤ bj
if sj > bj
Penalty
cost
Late Arrival Penalty
∞
[
ai
bi’
]
bi
]
Time
Exact Solution Technique: Column Generation
Sub Problem
ESPPRCLAP
Master Problem
Set Partitioning LP
min  c pYp
pP
subject to
 aipYp  1,
No
Feasible routes of
negative reduced cost
i  C
pP
Yp  {0, 1}
p  P
Yes
Integer Solution
No
Branch
&
Bound
End
Yes
Master Problem
Optimize, Prices (πi)
Reduce Cost
cij - πi
Upper and Lower Bound
VRPSSTW Solution
3400
Col. Gen. stops with integer
Optimum Solution
3350
Obj. function value
3300
3250
3200
3150
Col. Gen. stops but solution is not
integer: Branching on xij
3100
3050
3000
Col. Gen. stops but solution is not integer
Branching on Number of Vehicles
2950
2900
0
20
40
60
80
Col. Gen. Iteration
Upper Bound
Lower Bound
100
120
Solomon Test Instances
R101-Type
Customer Location : Random
90
80
y-coordinate
70
60
50
40
30
20
10
0
0
20
40
x-coordinate
60
80
Solomon Test Instances
RC101-Type
Customer Location: Random + Clusters
90
80
y-coodinate
70
60
50
40
30
20
10
0
0
20
40
60
x-coodinate
80
100
10
125
R
10
150
R
10
225
R
10
250
R
10
325
R
10
350
R
10
425
R
10
525
R
10
550
R
C
10
125
R
C
10
150
R
C
10
525
R
C
10
550
R
cost
Cost Comparison
12000
VRPSSTW
VRPHTW
10000
8000
6000
4000
2000
0
10
125
R
10
150
R
10
225
R
10
250
R
10
325
R
10
350
R
10
425
R
10
525
R
10
550
R
C
10
125
R
C
10
150
R
C
10
525
R
C
10
550
R
time
Waiting Time Comparison
450
400
VRPSSTW
VRPHTW
350
300
250
200
150
100
50
0
Detailed Comparison
Instance
Network
increase
Decrease in
cost
Decrease in
waiting
time
%
%
%
Vehicle
saved
Labels
ratio
Time
ratio
R101-25
19.20
-10.97
-11.61
1
2.71
5.67
R101-50
16.13
-20.16
-77.90
3
4.78
7.03
R102-25
5.10
-13.27
-3.89
1
1.83
6.21
R102-50
6.05
-23.92
-66.49
3
9.16
46.21
R103-25
1.34
-17.40
-37.40
1
8.95
11.58
R103-50
9.48
-19.54
-33.94
2
14.21
43.68
R104-25
0.35
0.00
0.00
0
10.54
40.64
R105-25
12.67
-14.82
-17.75
1
4.69
6.54
R105-50
14.53
-9.17
-54.19
1
7.16
7.28
RC101-25
10.51
-22.37
-81.84
1
2.63
0.56
RC101-50
15.02
-7.82
-50.85
1
3.85
1.97
RC105-25
12.16
-22.23
-68.67
1
3.06
5.87
RC105-50
12.09
-7.34
-75.73
1
4.32
4.32
Practical Test Instance
Test instance on Tokyo Road
Network TD1_39_djk
Depot
Contains a single depot and 38
customers’ locations of a chain
of convenience store
Routes
VRPHTW Case
VRPSSTW Case
Comparisons between VRPHTW and VRPSSTW
Penalty
cost
∞
Penalty
cost
[
ai
Parameter
]
bi
VRPHTW
∞
[
ai
bi’
Time
707
]
Time
Total Cost
Cost
Total
VRPSSTW
60000
Network Size
]
bi
16.3 %
782
No. of
Subproblems
81
Cols. added
to LP (Paths)
278
Labels per
7014
113
2348
47227
subproblem
Computation
Time
Cost(yen)
50000
40000
30000
20000
10000
0
106.52
520.88
VRPSSTW
VRPHTW
Travel+Vehicle cost Penalty cost
Comparisons of Delivery Time and Waiting Time
Waiting Time
Delivery Time
10.7 %
83.4 %
140
120
200
150
Time (sec.)
Time (sec.)
250
100
50
100
80
60
40
0
20
VRPSSTW
VRPHTW
0
VRPSSTW
VRPHTW
1
(VRPHTW)
(VRPSSTW)
9%
40%
60%
91%
Waiting Time
Delivery Time
Waiting Time
Delivery Time
Comparisons of Total Emissions
24 %
19.3 %
140
30000
120
25000
80
60
40
20000
15000
10000
5000
20
0
VRPSSTW
0
VRPSSTW
1
VRPHTW
18.3 %
SPM (gm)
CO2 (gm)
100
1
VRPHTW
12
11
10
9
8
7
6
5
4
3
2
1
0
VRPSSTW
1
VRPHTW
Comparisons of Average Emissions per Used Link
22.4 %
100
80
CO2 (gm)
0.4
0.3
0.2
60
40
20
0.1
0
VRPSSTW
17.6 %
SPM (gm)
0.5
SPM (ave.)
CO2 (ave.)
NOx (ave.)
NOx (gm)
NOx (gm)
SPM (total)
CO2 (total)
NOx (total)
1
VRPHTW
0
VRPSSTW
1
VRPHTW
0.045
0.04
0.035
0.03
0.025
0.02
0.015
0.01
0.005
0
VRPSSTW
16.7 %
1
VRPHTW
Comparisons of Max. Emissions per Used Link
∞
Penalty
cost
[
ai
]
bi
71.8 %
CO2 (gm)
NOx (gm)
4
2
0
VRPSSTW
VRPHTW
1
Time
0.8
71.5 %
0.7
0.6
1200
1000
800
600
400
200
0
VRPSSTW
]
SPM (max.)
1800
1600
1400
10
6
]
bi
CO2 (max.)
72.4 %
8
[
ai
bi’
Time
NOx (max.)
12
∞
SPM (gm)
Penalty
cost
0.5
0.4
0.3
0.2
0.1
1
VRPHTW
0
VRPSSTW
1
VRPHTW
Distributions of Used Links (NOx)
NOX (VRPSSTW)
7%
1%
1%
15%
0 0.2
0.2 0.4
0.4 0.8
46%
0.8 1.2
1.2 1.6
30%
>1.6
NOX (VRPHTW)
5%
2%
3%
13%
0 0.2
0.2 0.4
0.4 0.8
52%
25%
0.8 1.2
1.2 1.6
>1.6
Distributions of Used Links (NOx)
> 1.6
1.2 – 1.6
0.8 – 1.2
0.4 – 0.8
0.2 – 0.4
0 – 0.2
Nox Scale
(gm)
VRPSSTW
VRPHTW
Distributions of Used Links (CO2)
CO2 (VRPSSTW)
13%
4%
1%
0%
0 50
50 100
100 200
51%
200 300
300 400
31%
>400
CO2 (VRPHTW)
3% 1% 2%
14%
0 50
50 100
100 200
200 300
25%
55%
300 400
>400
Distributions of Used Links (CO2)
> 400
300 – 400
200 – 300
100 – 200
50 – 100
0 – 50
CO2 Scale
(gm)
VRPSSTW
VRPHTW
Distributions of Used Links (SPM)
SPM (VRPSSTW)
6%
1%
1%
16%
0 0.02
44%
0.02 0.04
0.04 0.08
0.08 0.12
0.12 0.16
>0.16
32%
SPM (VRPHTW)
3%
2%
3%
16%
0 0.02
0.02 0.04
0.04 0.08
49%
0.08 0.12
0.12 0.16
27%
>0.16
Distributions of Used Links (SPM)
> 0.16
0.12 – 0.16
0.08 – 0.12
0.04 – 0.08
0.02 – 0.04
0 – 0.02
SPM Scale
(gm)
VRPSSTW
VRPHTW
Conclusions and Future Work
Exact solution techniques is developed for VRPSSTW and it is used
along with Practical logistics instance to compare the relative
characteristics of VRPSSTW and VRPHTW.
• VRPSSTW resulted in lower cost as compared to VRPHTW
• Waiting time was significantly reduced in VRPSSTW as compared to
VRPHTW
• Total emissions and average emission intensities per used link were
found less in VRPSSTW as compared to VRPHTW
• Maximum emissions intensities on any used link were significantly
less in VRPSSTW
• VRPSSTW resulted in fewer links in the maximum range category in
link distribution as per emissions
Conclusions and Future Work
• As computation time is high exact solution technique is only
recommended for smaller practical instances at the moment.
Future Work
• At the moment fixed travel times are used, it is desirable to extend
the work to incorporate dynamic traffic conditions.
• Increase in the network size which can be handled
• Full soft time windows
3 Vehicle Routing and
scheduling Problem with Time
Windows (VRPTW) and the
uncertainty of travel times
Introduction
• Urban traffic congestion in Japanese cities
• Freight transport has large influence on
traffic conditions and the environment
• ICT & ITS allow us to obtain traffic
information
• Concept of “City Logistics” becomes more
important
• Better routing for pickup – delivery trucks
can contribute to the improvement of traffic
flow
VICS (Vehicle Information Communication
System)
VICS is a part of ITS
Now 78,000+ links on service in Japan
Information are updated every 5 minutes
and historical data is accumulated
Historical data is available
Historical data of VICS
Sample Link(Route1:6760m:Business Day)
Hour
0H
3H
6H
9H
12 H
38M
35M
32M
29M
26M
23M
20M
17M
14M
11M
8M
5M
15 H
Same Link(Business Day：8o'clock~9o'clock)
Frequency
200
180
160
140
120
100
80
18 H
60
40
21 H
20
0
5M
11M
Travel times (Min.)
Travel times on real network
17M
23M
29M
35M
Travel Times (Min.)
41M
47M
Objectives
• Develop VRPTW-P with ants routing
model to incorporate the variable travel
times
• Confirm the effect of using route learning
for VRPTW-P by the field experiment in
terms of costs and environmental impacts
Literature survey
• Vehicle Routing Problems with Time Windows (VRPTW):
Solomon, 1987; Russell, 1995; Bramel et al., 1996; Taniguchi et al., 1998
• Stochastic Vehicle Routing and scheduling Problems:
Jaillet and Odoni, 1988; Powell et al., 1995; Gendreau et al, 1996 Swihart and
Papastavrou, 1999; Secomandi, 2000
• Routing and scheduling with variable travel times:
Laporte et al., 1992; Malandraki and Daskin, 1992; Taniguchi et al., 2000b;
Kenyon and Morton, 2003
• Probabilistic Vehicle Routing and Scheduling Problem with Time
Windows MODEL
Taniguchi et al. (2001a)
VRPTW-P
• VRPTW-P (probabilistic vehicle routing and scheduling
with time window-probabilistic) uses travel times
distribution of each link as variable travel times.
Objective Function
Minimise
fixed cost
operation cost
early arrival and
delay penalty
Early arrival and delay penalty
Penalty (yen)
c d , n (i )
1
ce , n ( i )
Probability of arrival time
Penalty of early arrival and
delay (yen)
e
×
＝
1 t ns ( i )
t n(i )
Arrival time
Arrival time
Arrival time
Parameters of Genetic
Algorithms
VRPTW are solved with GA (Genetic Algorithm)
Number of individuals = 300
Number of generations = 1,000
Number of elite individuals = 30
Crossover rate = 0.8
Mutation rate = 0.02
VRPTW-P with route learning
• Visiting orders of customers are
determined with travel time distributions
• Route choices were determined by
shortest path method with mean value of
link travel times
• Effect of Ants routing (route learning) for
VRPTW-P was examined
Algorithms of Machine Learning
Feedback
Training
Data
No
Yes
No
Unsupervised
Learning
Yes
－
Reinforcement
Learning
Supervised
Learning
Supervised learning rely on having error signals for the
system’s output
Reinforced learning is appropriate when teacher is not
available
Framework of Reinforcement
Learning
• Learn the policy to maximise reword
acquisition
Agent
State
St
Action
At
Reword
Rt
Environment
Agent learns the suitable policy through trial and error
Ants routing algorithms
• Ants routing is proposed by Subramanian
et al. (1997)
• Routing rules are treated as random
variables.
• Only backward exploration is used for
updating routing table
Several linls
Single link
Z2
Z1
S
D
Move
X
Y
Present node Y
Destination node S
Update
Probability table will be updated when the ant generated at S
moves from X to Y
Updated probability table of Y expresses the moving probability
from Y to X where destination is S
Next
destination
Probability
Ｘ
0.15
Ｙ
0.5
：
：
Ｚi
0.1
Normalize
Update probability table by ants
routing
p  k / f (t )
k :learing rate f(t) :travel time
PY ( S , X )  p
PY ( S , X ) 
1  p
PY ( S , Z )
PY ( S , Z ) 
1  p
Z  neighbour of Y
Field experiments
• 17-24 November 2006, Central Osaka
Japan
• Travel time information: VICS
• Number of customers: 24
• Number of trucks: 2 at each case
• Comparison VRPTW-P with route
learning by ants routing (VRPTWPA),VRPTW-P and VRPTW-F with
shortest path by mean value of travel
times
Field Experiments
Network (All VICS links)
225nodes, 789links
# of Customers: 24
With Time Windows
# of Trucks: 2
Route choice:
Shortest Path
(Mean value of LTT)
Route Learning
(Ants Routing）
VICS Data
01 October 2004 –
16 November (30days)
VRP
VRPTW-P(SP・RL),
VRPTW-F(SP)
Depot
Experiment Date:
17-24 November 2006
(5days)
VRPTW-F
Diagram of 21 November
Delay even in the former half, large delay in the latter half
11/21 VRPTW-F Truck 1
11/21 VRPTW-F Truck 2
120
120
Delay
80
Distance（km）
Distance（km）
100
Delay
100
60
40
80
60
40
20
20
0
0
8
9
10
11
12
13
Hours
14
15
Time Window
16
17
8
10:00:00
11:30:00
11:00:00
14:00:00
14:00:00
15:00:00
Depot 8:30:00
17:00:00
9
10
11
12
13
Hours
14
15
16
17
VRPTW-P
Diagram of 21 November
Large delay in the latter half
11/21 VRPTW-P Truck 2
11/21 VRPTW-P Truck 1
120
120
Delay
100
80
Distance（km）
Distance（ｋｍ）
100
60
40
20
Delay
80
60
40
20
0
0
8
9
10
11
12
13
Hours
14
15
Time Window
16
17
8
10:00:00
11:30:00
11:00:00
14:00:00
14:00:00
15:00:00
Depot 8:30:00
17:00:00
9
10
11
12
13
Hours
14
15
16
17
VRPTW-P + Ants Routing
Diagram of 21 November
Small delays
11/21 VRPTW-PA Truck1
11/21 VRPTW-PA Truck 2
100
100
80
Delay
60
40
Delay
20
Distance（km）
120
Distance（ｋｍ）
120
Delay
80
60
40
20
0
0
8
9
10
11
12
13
Hours
14
15
Time Window
16
17
8
10:00:00
11:30:00
11:00:00
14:00:00
14:00:00
15:00:00
Depot 8:30:00
17:00:00
9
10
11
12
13
Hours
14
15
16
17
23/28
Results (Running time)
Total time (Average)
0:00:00
Early Arrival
Delay
Operation
21:00:00
18:00:00
15:00:00
12:00:00
9:00:00
6:00:00
3:00:00
0:00:00
VRPTW-F
VRPTW-P
VRPTW-PA
24/28
Results (Costs)
(Unit:Yen)
November
VRPTW-F
Early Arrival
62,523
VRPTW-P
43,094
VRPTW-PA
Expected
38,373
21
22
24
Mean
(%)
0
0
0
0
0
38,930
50,637
45,938
51,046
52,246
47,759
8,887
9,469
9,409
9,567
9,407
9,348
68,652
80,940
76,182
81,448
82,488
77,942
78
17
75
0
135
61
29,335
36,141
30,092
36,188
23,821
31,115
9,749
9,924
9,634
10,009
9,201
9,703
59,996
66,918
60,636
67,032
53,992
61,715
97
0
0
35
162
59
Delay
6,120
10,161
11,035
9,295
9,725
9,267
Operation
8,625
9,115
8,992
9,102
8,939
8,955
35,677
40,111
40,862
39,267
39,661
39,116
Operation
Total
Early Arrival
Delay
Expected
18
0
Delay
Expected
17
Operation
Total
Early Arrival
Total
100
79.2
50.2
Comparison with conventional
planner
•
•
•
•
Plan was made by a planner of freight company
Compared on expected costs
Use 3 trucks, no delay, large early arrival
Operation cost and total cost are larger than that of
VRPTW-PA
(Unit:Yen)
Fixed cost
Operation
cost
Delay
Penalty
Early arrival
penalty
Total cost
(%)
VRPTW-F
20,835
11,004
30,408
276
62,523
129.9
VRPTW-P
20,835
10,760
11,305
194
43,094
89.5
VRPTW-PA
20,835
11,593
5,725
219
38,373
79.7
Planner
31,253
14,518
0
2,361
48,132
100.0
Results (negative impacts on the
environment)
Total distance and mean vehicle speed of each case
Total Distance
(km)
Mean Speed
(km/h)
VRPTW-F
193.70
17.43
VRPTW-P
187.90
16.29
VRPTW-PA
199.30
18.72
Negative impacts on the environment
CO2(g)
NOx(g) SPM(g)
VRPTW-F
70.06
73.52
15.05
VRPTW-P
69.46
72.49
14.77
VRPTW-PA
69.65
73.33
15.15
27/28
Conclusions
• VRPTW-P with ants routing model is
useful for planning better routing
incorporating the uncertainty of travel
times
• VRPTW-P with ants routing model can
reduce total costs compared with the other
VRPTW model with shortest path method
• Negative impacts on the environment of
VRPTW-PA are same as the others
Summary
• Vehicle routing and scheduling problem
with time window model is a basic model
for evaluating city logistics schemes
• Exact solution approach in VRPSSTW and
incorporating the uncertainty of travel
times in VRPTW was presented including
the practical case studies
Thank you for your attention.