Metaheuristics: GRASP
FEUP | PDEEC | Decision Support
Group 1:
Clara Gouveia
Daniel Oliveira
Fabrício Sperandio
Filipe Sousa [Presenter]
January 3rd, 2011
Metaheuristics: GRASP
Outline
FEUP | PDEEC | Decision Support





GRASP Overview
Construction Phase
Local Search Phase
GRASP Example
GRASP Variations
 3D BPP Definition
 2D BPP Definition
 Hybrid GRASP/VND for BPP
 Preprocess Phase
 Construction Phase
 Improvement Phase
 Improvement Procedures
 Combined Strategies
 Diversification Phase
 Computational Results
 Comparison with other Algorithms
 Conclusions
2
Metaheuristics: GRASP
FEUP | PDEEC | Decision Support
GRASP Overview
G
R
A
S
P
reedy
andomized
daptive
earch
rocedure
3
Metaheuristics: GRASP
GRASP Overview
GRASP is a metaheuristic:
 A metaheuristic is a method that works with local improvement procedures and
other higher level strategies to create processes capable of escaping from local
optima, and performing a robust search of a solution space.
GRASP was first described in 1989:
FEUP | PDEEC | Decision Support
 Published by Thomas A. Feo and Mauricio G. C. Resende.
 In this first publication GRASP was applied to the set covering problem.
References:
M. Gendreau, and J. Potvin. Handbook of Metaheuristics. 2nd edition, Springer, 2010.
T. A. Feo, and M. G. C. Resende. A Probabilistic Heuristic for a Computationally Difficult Set Covering Problem. Operations Research Letters, no. 8, pp. 67-71, 1989.
4
Metaheuristics: GRASP
GRASP Overview
GRASP can be divided in two phases:
 Construction: where a feasible solution is built.
 Local Search: from the solution obtained a neighborhood is built and the search
is then performed.
FEUP | PDEEC | Decision Support
For N iterations
Construction Phase
Local Search Phase
Improved Solution
References:
T. A. Feo, and M. G. C. Resende. Greedy Randomized Adaptive Search Procedures. Journal of Global Optimization, no. 6, pp. 109-133, 1995.
M. G. C. Resende. Greedy Randomized Adaptive Search Procedures (GRASP). AT&T Labs Research Technical Report, 98.41.1, Dec. 1998.
M. G. C. Resende, and Celso C. Ribeiro. Greedy Randomized Search Procedures. AT&T Labs Research Technical Report TD-53RSJY, version 2, Aug. 2002
Best Solution
5
Metaheuristics: GRASP
GRASP Overview
GRASP is easy to implement:
 Few parameters need to be set and tuned.
 Effort can be transferred to the implementation of efficient data structures.
GRASP is also easily implemented in parallel.
For N iterations
Local Search Phase
Improved Solution
compare
FEUP | PDEEC | Decision Support
Construction Phase
Construction Phase
Local Search Phase
Best Solution
Improved Solution
References:
T. A. Feo, and M. G. C. Resende. Greedy Randomized Adaptive Search Procedures. Journal of Global Optimization, no. 6, pp. 109-133, 1995.
M. G. C. Resende. Greedy Randomized Adaptive Search Procedures (GRASP). AT&T Labs Research Technical Report, 98.41.1, Dec. 1998.
M. G. C. Resende, and Celso C. Ribeiro. Greedy Randomized Search Procedures. AT&T Labs Research Technical Report TD-53RSJY, version 2, Aug. 2002
6
Metaheuristics: GRASP
Construction Phase
The RCL (Restricted Candidate List) is built based on the α parameter:
 α is variable between 0 and 1:
 0 makes the construction too random.
 1 makes the construction too greedy.
 If β is the best free element and Σ represents the free elements, RCL is
composed by:
 RCL U {Σ ≥ α.β}
Repeat until element list is empty
FEUP | PDEEC | Decision Support
Initialize elements
Build RCL
Update Candidate
List
Randomly choose
an element
Update Solution
References:
T. A. Feo, and M. G. C. Resende. Greedy Randomized Adaptive Search Procedures. Journal of Global Optimization, no. 6, pp. 109-133, 1995.
M. G. C. Resende. Greedy Randomized Adaptive Search Procedures (GRASP). AT&T Labs Research Technical Report, 98.41.1, Dec. 1998.
M. G. C. Resende, and Celso C. Ribeiro. Greedy Randomized Search Procedures. AT&T Labs Research Technical Report TD-53RSJY, version 2, Aug. 2002
Contructed Solution
7
Metaheuristics: GRASP
Construction Phase
The randomness of GRASP is present when an element is picked by chance
from the RCL.
After the element is added to the current solution the cost of each free element
is updated:
 This is the adaptive part of the GRASP metaheuristic.
Repeat until element list is empty
FEUP | PDEEC | Decision Support
Initialize elements
Build RCL
Update Candidate
List
Randomly choose
an element
Update Solution
References:
T. A. Feo, and M. G. C. Resende. Greedy Randomized Adaptive Search Procedures. Journal of Global Optimization, no. 6, pp. 109-133, 1995.
M. G. C. Resende. Greedy Randomized Adaptive Search Procedures (GRASP). AT&T Labs Research Technical Report, 98.41.1, Dec. 1998.
M. G. C. Resende, and Celso C. Ribeiro. Greedy Randomized Search Procedures. AT&T Labs Research Technical Report TD-53RSJY, version 2, Aug. 2002
Contructed Solution
8
Metaheuristics: GRASP
Local Search Phase
The Create Neighborhood can be implement in several different ways:
 This is problem dependent.
 The stopping criteria varies with the implemented method.
Repeat until stopping criteria is satisfied
FEUP | PDEEC | Decision Support
Contructed Solution
Create
Neighborhood
Compare with
Existing Best
Improved Solution
Select Best Solution
References:
T. A. Feo, and M. G. C. Resende. Greedy Randomized Adaptive Search Procedures. Journal of Global Optimization, no. 6, pp. 109-133, 1995.
M. G. C. Resende. Greedy Randomized Adaptive Search Procedures (GRASP). AT&T Labs Research Technical Report, 98.41.1, Dec. 1998.
F. Parreño, R. Alvarez-Valdes, J. F. Oliveira, and J. M. Tamarit. A hybrid GRASP/VND algorithm for two- and three-dimensional bin packing. Annals of Operations Research, vol. 179,
no. 1, pp. 203-220, Oct. 2008.
9
Metaheuristics: GRASP
GRASP Example
 Set Covering Problem:
 Having α = 40% implies a RCL = {P1, P4, P5, P6, P7}.
 Randomly selecting P5 translates into:
 Covering elements 3, 4, and 5.
 The next GRASP step consists in the candidate list update.
P1
P2
P3
P4
P5
P6
P7
P8
FEUP | PDEEC | Decision Support
X
X
X
X
X
2
1
1
X
X
X
X
X
X
X
X
3
3
3
1
X
2
X
3
X
4
5
2
1
References:
T. A. Feo, and M. G. C. Resende. Greedy Randomized Adaptive Search Procedures. Journal of Global Optimization, no. 6, pp. 109-133, 1995.
10
Metaheuristics: GRASP
GRASP Example
Set Covering Problem:
 RCL = {P3, P4, P6, P7}.
 Randomly choosing P3 leaves P6 as the only option:
 Translates into solution {P5, P3, P6}.
P1
P2
P3
P4
P5
P6
P7
P8
FEUP | PDEEC | Decision Support
X
X
X
1
X
2
3
4
5
0
0
1
1
0
1
1
0
References:
T. A. Feo, and M. G. C. Resende. Greedy Randomized Adaptive Search Procedures. Journal of Global Optimization, no. 6, pp. 109-133, 1995.
11
Metaheuristics: GRASP
GRASP Example
Set Covering Problem:
 Once again, having α = 40% implies a RCL = {P1, P4, P5, P6, P7}.
 However, if P6 had been randomly chosen, and then P4 we would have reached
an optimal solution:
 {P6, P4}.
P1
P2
P3
P4
P5
P6
P7
P8
FEUP | PDEEC | Decision Support
X
X
X
X
X
2
1
1
X
X
X
X
X
X
X
X
3
3
3
1
X
2
X
3
X
4
5
2
1
References:
T. A. Feo, and M. G. C. Resende. Greedy Randomized Adaptive Search Procedures. Journal of Global Optimization, no. 6, pp. 109-133, 1995.
12
Metaheuristics: GRASP
GRASP Variations
Reactive GRASP:
 RCL size is adjusted according to the quality of the solutions previously
found.
GRASP using GA (Genetic Algorithm) methodology :
 Introduces a mutation in the local search phase.
FEUP | PDEEC | Decision Support
GRASP with cost perturbation:
 The cost associated with an element is modified in some way to cause a
perturbation in the greedy function.
Other variations exist...
References:
T. A. Feo, and M. G. C. Resende. Greedy Randomized Adaptive Search Procedures. Journal of Global Optimization, no. 6, pp. 109-133, 1995.
M. G. C. Resende, and Celso C. Ribeiro. Greedy Randomized Search Procedures. AT&T Labs Research Technical Report TD-53RSJY, version 2, Aug. 2002
13
Metaheuristics: GRASP
Outline
FEUP | PDEEC | Decision Support





GRASP Overview
Construction Phase
Local Search Phase
GRASP Example
GRASP Variations
 3D BPP Definition
 2D BPP Definition
 Hybrid GRASP/VND for BPP
 Preprocess Phase
 Construction Phase
 Improvement Phase
 Improvement Procedures
 Combined Strategies
 Diversification Phase
 Computational Results
 Comparison with other Algorithms
 Conclusions
14
Metaheuristics: GRASP
Part Two: Paper Presentation
A hybrid GRASP/VND algorithm for two- and three-dimensional bin packing.
F. Parreño, R. Alvarez-Valdes, J.F. Oliveira, and J.M. Tamarit
FEUP | PDEEC | Decision Support
Annals of Operations Research
Volume 179, Number 1, Pages 203-220, 25 October 2008.
15
Metaheuristics: GRASP
3D BPP Definition
The three-dimensional bin packing problem (3BP):
 NP-hard problem.
 Useful for industrial applications (loading cargo into pallets, containers or
vehicles, or packaging design).
Assumptions:
FEUP | PDEEC | Decision Support
 Known dimensions:
 Bin (W,H,D) .
 Boxes (wi,hi, di ), (i = 1, . . . , n).
 wi ≤ W, hi ≤ H, and di ≤ D.
 The items cannot be rotated.
16
Metaheuristics: GRASP
2D BPP Definition
The two-dimensional bin packing problem (2BP):
 Special case of 3BP where di =D , i=1, . . . , n.
Assumptions:
FEUP | PDEEC | Decision Support
 Known dimensions:
 Bin (W,H).
 Boxes (wi,hi), (i = 1, . . . , n).
 wi ≤ W, and hi ≤ H.
 The items cannot be rotated.
17
Metaheuristics: GRASP
Hybrid GRASP/VND for BPP
Preprocess:
k=1; f=[]
B={b1,...,bn}
 Simplify the problem.
Inputs:
 Items to pack.
Preprocess
Constructive Phase:
 Develop a feasible solution.
 Number of iterations.
B={b1,...,bm}
S={E}
 Alpha.
Improvement Phase:
 Improves the solution.
FEUP | PDEEC | Decision Support
Constructive
Phase
k=k+1
Target: n-1
YES
NO
NO
𝑩=∅
or
𝑺=∅
YES
𝑽≥
𝑽𝒕𝒐𝒕𝒂𝒍
𝑽𝒕𝒐𝒕𝒂𝒍 −
(𝒏𝒃𝒆𝒔𝒕 − 𝟏)
YES
Improvement
Phase
NO
𝑽′ > 𝑽
f=[]
18
Metaheuristics: GRASP
Hybrid GRASP/VND for BPP
Constructive Phase
Vectors with boxes left to pack:
(0)
Initialization
 B={b1,...,bm}.
Set of empty maximal spaces:
 S={E}.
FEUP | PDEEC | Decision Support
(1)
Choose
Maximal Space – S*
(2)
Choose
Boxes to Pack
(3)
Update
List S
𝐵 = ∅ and 𝑆 = ∅
19
Metaheuristics: GRASP
Hybrid GRASP/VND for BPP
Constructive Phase: Step 1
(0)
Initialization
Choose Maximal Space:
 Maximal Space (S*): maximal space in S closer to a corner.
(1)
Choose
Maximal Space – S*
 The volume can be used as a tie breaker.
Objective:
FEUP | PDEEC | Decision Support
(2)
Choose
Boxes to Pack
 First fill the corners.
 Then the sides.
 Later the inner space.
(3)
Update List S
20
Metaheuristics: GRASP
Hybrid GRASP/VND for BPP
Constructive Phase: Step 2
(0)
Initialization
Choose Boxes to Pack:
 Boxes fitting in S* can be packed:
(1)
Choose
Maximal Space – S*
 Individually.
 Layer: several boxes with the same dimensions.
 Using one of the following criteria:
FEUP | PDEEC | Decision Support
(2)
Choose
Boxes to Pack
(3)
Update List S
 Best volume: order by the box that produces the largest
increase in the bin volume.
 Best fit: order by the box which fits best in the maximal space.
 The box to be moved is selected randomly among the
(1-α)*100% blocks.
 α is the RCL GRASP parameter.
21
Metaheuristics: GRASP
Hybrid GRASP/VND for BPP
Constructive Phase: Step 2
(0)
Initialization
α parameter:
(1)
Choose
Maximal Space – S*
FEUP | PDEEC | Decision Support
(2)
Choose
Boxes to Pack
 Responsible for the random selection of the boxes to pack.
 The authors used the reactive-GRASP principle to determine
α:
 α is initially chosen randomly from the discrete set {0.1,
0.2,...,0.9}.
 After some iterations the probability distribution of α is
adjusted taking into account the relative quality of the
solutions.
 This incorporates a learning mechanism in GRASP, which
is memory less in its constructive phase.
(3)
Update List S
22
Metaheuristics: GRASP
Hybrid GRASP/VND for BPP
Constructive Phase: Step 3
(0)
Initialization
(1)
Choose
Maximal Space – S*
FEUP | PDEEC | Decision Support
(2)
Choose
Boxes to Pack
(3)
Update List S
Update List S:
 Update S:
 If the box intercepts other maximal space.
 If the box does not fit exactly in the space S*.
 If the unpacked boxes are larger than the remaining
maximal spaces.
 Update B:
 The placed boxes must be removed from the list.
 If 𝑆 ≠ ∅ and 𝐵 ≠ ∅ then return to step 1:
 Otherwise the constructive phase ends with n bins used.
 For the next GRASP iteration, the target of the constructive
phase is n-1.
23
Metaheuristics: GRASP
Hybrid GRASP/VND for BPP
Improvement Phase
Order Solution
by Non-increasing
Volume
Initial Procedure
Improvement
Procedures
Procedure 1
Repacking
FEUP | PDEEC | Decision Support
Procedures
Procedure 3
Best Fit
Objects
Combined
Procedure 2
All Moves
1→ 4
Procedure 4
Local Search
Best Volume
VND
(N1, N2, N3, N4)
VNDseq
(N1, N2, N3, N4)
24
Metaheuristics: GRASP
Procedure 1:
Improvement Phase
Improvement Procedures
 Eliminates the last k% items in the solution.
 k is a random value between 30 and 90%.
Procedure 2:
 Removes the last k% pieces packed from each bin, whose occupied volume is
less than the average.
FEUP | PDEEC | Decision Support
Procedure 3:
 Select all the bins in which the occupied volume is lower than the overall
average occupancy.
 Split each bin into two parts randomly selecting how to cut: vertically or
horizontally.
 Choose randomly one side of the bin: [up/down left/right]:
 Remove all boxes that are mostly in the selected side.
25
Metaheuristics: GRASP
Improvement Phase
Improvement Procedures
Procedure 4 – Local Search:
 Neighborhood: consists in all pairs of bins in which at least one have an
occupancy bellow the average.
FEUP | PDEEC | Decision Support
 Search method:
 For each pair in the neighborhood unpack all the items.
 The first box to pack must be one of the boxes that was not packed (initial solution).
 The moves are chosen considering one of the following criteria:
 First Improve: select the first pair which improves the current solution.
 Best Improve: Examine all the neighborhood and select the best improvement.
 The move improves the solution if the total volume of boxes packed into the bins
increases.
26
Metaheuristics: GRASP
Improvement Phase
All moves:
Combined Strategies
 The improvement procedures are applied: 1→ 4.
VND (Variable Neighborhood Descent):
 Neighborhood (N1 to N4): generated by the four improvement procedures.
 Start with the solution given by the constructive heuristic (x):
FEUP | PDEEC | Decision Support
1. set p ← 1
2. while p <= 4
a) Exploration of the neighborhood to find the best x’ in the neighbor Np(x).
b) Move if x’ is better then x. Return to p ← 1. Otherwise set p ← p +1.
VNDseq (Sequential Variable Neighborhood Descent):
 Similar to VND but:
 In step 2.a) instead of setting p=1, the algorithm proceeds sequentially to the (p+1)th.
27
Metaheuristics: GRASP
Computational Results
Computation and Data Sets:
FEUP | PDEEC | Decision Support
 The algorithm was coded in C++ and run on a standard laptop.
 2D and 3D data sets were used in order to test the proposed algorithm:
 For the 3D simulation, a standard benchmark generated by Martello et al
was used. This data set was also used by Faroe et al.
 For the 2D simulation, more than one data set was used. To compare with
already published results of other heuristics.
28
Metaheuristics: GRASP
Computational Results
Experiments
Choosing the best strategy:
 Constructive Phase:
 Number of corners to consider:
 [1, 2, 3, 4] for the 2D BPP.
 [1, 2, 3, 4, 8] for the 3D BPP.
 Deterministic or random constructive phase.
FEUP | PDEEC | Decision Support
 Improvement Phase:
 Compare the performance of the four improvement methods individually.
 Compare the performance of the combined improvement methods.
 Comparison with other algorithms to solve 2D and 3D BPP.
29
Metaheuristics: GRASP
Computational Results
Constructive
2D
3D
Overall
Deterministic
Random
7635
10148
17783
7492
9877
17369
The results obtained for each
dataset showed that using the
randomized constructive algorithm
proved to be better than the
determinist.
Applying the constructive phase with randomization:
FEUP | PDEEC | Decision Support
 Each improvement methods were tested with both objective functions: Best
Volume and Best Fit.
 At each iteration one of the four strategies to choose the corner is randomly
selected.
 The algorithm run for 5000 iterations.
30
Metaheuristics: GRASP
Computational Results
Improvements
Best Volume
Best Fit
1
2
3
4
1
2
3
4
2D
7492
7492
7494
7478
7492
7484
7494
7476
3D
9876
9867
9875
9820
9877
9865
9871
9828
Overall 17368 17359 17369 17298 17369 17349 17365 17304
2D:
FEUP | PDEEC | Decision Support
 Method 4 was the best independently of the objective function used.
 Method 2 with best fit approach also produced good results.
3D
 For both objective functions, method 4 performed better followed by method 2.
31
Metaheuristics: GRASP
Computational Results
Combined Improvement
The diversification algorithm is applied
after
500
iterations
without
improvement in the following 100
iterations .
The analysis of the three strategies for
the combination of improvements
shows:
FEUP | PDEEC | Decision Support
 2D/3D:
 “All moves” is the worst strategy.
 Small difference among the others.
The method chosen for the final implementation was:
 VNDseq + Diversification.
32
Metaheuristics: GRASP
Comparison with other Algorithms
2D statistical analysis show:
FEUP | PDEEC | Decision Support
 GRASP is better than TS3 and it favors GRASP against the others algorithms.
3D statistical analysis show:
 GRASP and SCH are significantly better than TS3, GLS, and HBP.
GRASP and SCH achieved optimal solutions:
 Indicates a good performance of the proposed algorithm.
33
Metaheuristics: GRASP
Conclusions
Regarding the GRASP algorithm:
 Grasp consists in a constructive phase followed by a local search procedure.
 It is easy to implement due to the reduced number of parameters
 Parameter α defines the randomness of Grasp
 Basic version of GRASP does not have memory. It can be combined with other
procedures such as reactive GRASP that adjust α.
FEUP | PDEEC | Decision Support
Hybrid GRASP/VND for BPP:
 Combination of GRASP with VND allowed the algorithm to obtain high quality
solutions.
 The quality of the solutions were similar to well known algorithms to solve 2D
and 3D BPP.
 Reinforces the fact that GRASP is flexible and could be adapted easily to
accommodate constraints or other conditions.
34
Metaheuristics: GRASP
FEUP | PDEEC | Decision Support
Thank you for your attention!!!
Questions?