Combining Column Generation and Genetic Algorithms
Geraldo Ribeiro Filho 1, Luiz Antonio Nogueira Lorena2
1
UMC/INPE - Av Francisco Rodrigues Filho, 399
08773-380 - Mogi das Cruzes – SP Brazil
Phone: 55-11-4791-3743
geraldo@lac.inpe.br
2
LAC/INPE - Caixa Postal 515
12.201-970 São José dos Campos – SP Brazil
Phone: 55-12-345-6553
lorena@lac.inpe.br
Abstract. This work describes a combined use of a Constructive Genetic Algorithm (CGA) and
the column generation process to approximately solve graph coloring problems. Column
generation is a powerful tool for solving large scale linear programming problems. Problems
may arise for which the columns are not known in advance and a complete enumeration is not
an option. The proposed combined method is divided in two phases. The constructive phase
builds an initial pool of columns using the CGA, each column representing an independent set
of vertices. The second phase solves by column generation the set covering formulation of the
graph coloring problem. The CGA is also used to determine incoming columns solving
approximately weighted independent set problems. Computational experiments show the
combined process applied to some graph coloring instances from the literature.
1 Introduction
Column generation is a powerful tool for solving large scale linear programming problems. Problems may
arise for which the columns are not known in advance and a complete enumeration of all columns is not an
option, or the problem is rewritten using Dantzig-Wolfe decomposition (the columns correspond to all
extreme points of a certain constraint set) [5]. Column generation is a natural choice in several applications,
such as the well-known cutting-stock problem, vehicle routing and crew scheduling [2, 8, 9, 25].
Lorena and Furtado [20] recently proposed the Constructive Genetic Algorithm (CGA) approach. A typical
CGA uses not only problem solutions (structures), but also solution parts, known as schemata. The
constructive algorithm works with an initial population formed only by schemata, which evolves to a
population of structures representing good solutions to the problem at issue. The CGA works with a dynamic
population and uses heuristics on schema and structure representations [20]. It was successfully applied to
timetabling [24], linear gate assignment in VLSI design [22], cellular manufacturing systems design [23] and
location problems [1].
Let G = (V,E) be an undirected graph. A k-coloring of G is a partition of V into k subsets Ci, i = 1,…,k,
such that no adjacent vertices belong to the same subset. The Graph-Coloring Problem is to find k-coloring of
G with k as small as possible. This optimal value of k corresponds to the so called chromatic number of G. It
is well known that this problem is NP-hard [13], and heuristics must be used for large graphs. Each vertex
subset is an independent vertex set, and the coloring problem could be seen as a clustering problem to form
independent vertex sets.
Graph coloring is a very studied problem [7] and efficient algorithms have been developed. Applications
appear in scheduling (like timetabling) [6, 19], frequency assignment [12] and register allocation [3]. The use
of metaheuristics has produced the best results for a large class of graph instances. Johnson et al. [17] applied
Simulated Annealing. Costa and Hertz [4] have applied Ant Colony. Friden et al. [11] and Hertz and de Werra
[15] applied Tabu Search and Fleurent and Ferland [10] applied Hybrid Genetic Algorithm with aggressive
local search.
A column generation approach to graph coloring was early studied by Mehrotra and Trick [21] to provide
lower bounds to the chromatic number. The column generation was embedded on a tree search and solves
successfully small to moderate size problems. In this paper this approach is revisited. The tree search is not
considered, but only the liner programming (LP) relaxation of a set covering formulation of the problem. This
LP is solved by column generation combined with a CGA. The proposed combined method is divided in two
phases. The constructive phase builds an initial pool of columns using the CGA. Each column forms an
independent set of vertices. The second phase solves by column generation the set covering formulation. The
CGA is also used to determine incoming columns solving approximately weighted independent set problems.
Computational experiments show the combined process applied to some graph coloring instances from the
literature.
This paper is organized as follows. Section 2 reviews the Mehrotra and Trick’s approach to graph coloring.
The CGA application to graph coloring is presented in the section 3, while section 4 details the combined use
of CGA and column generation. Computational results are presented in the last section.
2 Mehrotra and Trick’s column generation for graph coloring
The use of column generation for coloring problems was first proposed by Mehrotra and Trick [21]. In a
classical fashion of column generation, the algorithm iterates between a column generation sub-problem and a
restricted master problem. Solving the master problem yields a certain dual solution, which is used in the subproblem to determine whether there is any column that might be an incoming column.
The master problem (MP) is
Min
∑x
j∈S
j
∑a
Subject to
j∈S
ij
x j ≥ 1, i = 1,..., V ,
x j ∈ {0,1} .
Where S = {S1 , S 2 ,..., S m } , S i ⊂ V , is the set of all maximal independent sets of G and [ aij ]nxm , is a
matrix with aij = 1 if i ∈ S j , and aij = 0 otherwise.
Initial pool
of columns
λi , i ∈V
MPLP
no
WMIP
New
Columns?
stop
yes
Fig. 1. The column generation process.
The master problem MP is a set-covering problem with a large number of (generally) unknown columns
that are generated when necessary. A linear programming relaxation of MP (called MPLP ) will be solved by
column generation. An initial pool of columns must be given to form the initial MPLP , that is optimally
solved and returns the dual variables λi , i ∈ V . New columns are elected to compose the MPLP if they return
bounds larger than 1 on the following weighted maximum independent set problem (WMIP)
Max
∑λ z
i∈V
i i
Subject to zi + z j ≤ 1, ∀(i , j) ∈ E
z i ∈ {0,1}, ∀i ∈ V .
Problem MPLP is resolved and the process continues until no more incoming columns. Figure 1 resumes
the column generation process.
3 The CGA application
The CGA is applied to compose the initial pool of columns, to give an upper bound to the chromatic number
and to approximately solve problem WMIP. The CGA application to graph coloring is resumed in the
following (see [20] for a general view of the CGA and [24] for a recent CGA application to school
timetabling).
The representation of schemata and structures uses a string of |V| positions and three symbols (or labels).
These symbols are: the “do not care” symbol (‘#’), indicating the vertices which are not assigned to any
cluster; a symbol to indicate the vertex is a “seed” to form a cluster (‘1’); and a third symbol indicating the
vertices assigned to some cluster (‘0’). The number of seed vertices is exactly the number of colors being
used, or clusters being formed. The objective is to find clusters composed only by independent vertices.
The vertex-to-cluster assignment uses an adaptation of a heuristic known as Recursive Large First (RLF)
[19]. This can be better understood using an example. Suppose we are looking for a 3-coloring for the
following graph with ten vertices (Figure 2):
2
10
3
1
8
9
4
7
5
6
Fig. 2. An example graph (RLF adapted).
Consider the following sets: Ci = {vertices in the i-th cluster}, Ui = {vertices adjacent to any vertex in Ci},
Vsch = {schema vertices}, and Vi = Vsch – Ui. Let the n-th schema in the population be sn =
(#,1,0,1,#,0,0,#,1,0), where: 1 = seed vertex, 0 = vertex to be assigned and # = vertex not to be considered.
The initial configuration is then C1 = {2}; C2 = {4}; C3 = {9}; V1 = {3,6,10}; V2 = {6,10}; V3 = {3,6,7,10};
U1 = {7}; U2 = {3,7} and U3 = { }.
Now, take the vertex v in Vi, i=1,2,3 with the largest degree in Ui, i=1,2,3 and assign v to Ci. Then, update
the sets Ci, Vi, and Ui. The new configuration will be C1 = {2,10}; C2 = {4}; C3 = {9}; V1 = {3,6}; V2 = {6};
V3 = {3,6,7}; U1 = {7}; U2 = {3,7} and U3 = { }.
The process continues until all Vi sets are empty. Considering a non-independent vertex pair inside a
cluster as a conflict, at this point there are no conflicts inside the clusters or the last vertices remained in any
Vi set were assigned to the clusters aiming the smallest possible number of conflicts. The final configuration
is C1 = {2,3,10}; C2 = {4,6} and C3 = {7,9}.
Let Χ be the set of all structures and schemata that can be generated by this 0-1-# string representation
The initial population is created with |V| schemata randomly generated, with 20% of labels ‘0’, exactly k
labels ‘1’ and the remaining labels ‘#’. The initial population size, and the number of labels ‘0’ are CGA
parameters that can be adjusted.
The evolution process is guided by a variable α ≥ 0 (also called time) and the expression
g(s n ) - f(sn ) ≥ d.g max - α .d[gmax - g(s n)]
(1)
that represents a condition for elimination of a schema or structure sn from the current population. Where
[(C
k
g (s n ) = ∑
p =1
)
]
− 1 . C np / 2 (total number of edges if
np
k
complete graphs of sizes C np ) ,
f ( s n ) = g (s n ) − ∑ E (C np ) (number of edges actually linking vertices on the sets Cnp ), k is a pre-fixed
k
p =1
number of colors, Cnp is the set of vertices receiving the color p on schema sn (the notation C np is the
number of vertices in set Cnp ), and E (Cnp ) is the set of edges with both terminal vertices in Cnp . The
(
)
expression C np − 1 . C np / 2 gives the number of edges of a complete graph with
C np vertices. Two
 ( V / k  − 1)⋅ V / k 
parameters are given a priori: the upper bound for coloring problems g max = mult ⋅ k ⋅  

2


, and real number 0 < d ≤ 1. To obtain gmax, first is considered to divide the number of vertices |V| in k sets
with approximately the same number of elements ( the expression V / k  gives the large integer smaller than
|V| / k ), then the same procedure used for g(sn) is applied, where the positive integer mult is considered to
certify that gmax > Max g ( s n ) . The factor mult is empirically determined and instance dependent.
sn ∈X
The parameter α starts from zero and is increased in small time intervals which amount is also empirically
determined and instance dependent. According to expression (1), schemata or structures with small gaps [g(sn
) - f(sn )] and/or higher g(sn) values, are conserved for long time in population and have more chance
participate of the recombination and mutation process. When f(sn) = g(sn) the k sets Cnp are independent
sets.
The recombination process is applied at each time interval (a value of α). The population is kept in a nondecreasing order, according to the following key: ∆ ( sn ) = (1 + d n ) / ( V − n# ) , where dn = [g ( sn ) − f ( sn ) ] / g ( sn ) ,
n# is the number of labels ‘#’ in sn. Schemata with small n# and/or presenting small dn are considered better
and appear in first order positions. The method used for selection takes a first schema from the |V| first
positions in the population (base schema) and a second schema from the whole population (guide schema).
Before recombination, the first schema is complemented to generate a structure representing a feasible
solution, i.e., all labels ‘#’ are replaced by ‘0’. This structure suffers mutation and is compared to the best
solution found so far (which is kept throughout the process). The recombination merges information from
both selected schemata, but preserves the number of labels ‘1’ (number of colors) in the new generated
schema or structure. The recombination is best described as (to be executed in this order):
Recombination
if sbase(j) = sguide (j) then snew(j) ← sbase(j)
if sguide (j)=# then snew(j) ← sbase(j)
if sbase(j) = # or 0 and sguide (j)=1 then
snew(j) ← 1 and snew(i) ← 0 for some snew(i)=1
if sbase(j) = 1 and sguide (j)=0 then
snew(j) ← 0 and snew(i) ← 1 for some snew(i)=0
At each generation, V
| | new individuals are created by recombination. If a new individual does not
represent a feasible solution (is a schema), it is inserted into the population; otherwise it suffers mutation and
is compared to the best solution found so far. The population interval from which the base schema is take and
the number of new individuals generated are parameters of the CGA and can be adjusted.
For an original cluster configuration, the mutation process can be described as: 1) For each cluster, move
the seed to the vertex with the largest degree in the cluster, re-assign all vertices using the RLF approach and
count the conflicts, save the best move found for all clusters; 2) If the best cluster configuration found in step
1 is better (less conflicts) than the original, replace the original by this new one and return to step 1, otherwise
stop.
4 Combing the CGA and column generation
The CGA resumed above is applied in two phases of the column generation process. First to define an initial
pool of columns, and then to approximately solve the problem WMIP.
Initially, it is set a number of colors, and if the CGA find this specified coloring, this number is reduced,
until no more improvement is found. A number of independent sets found during the last CGA application are
stored to compose the first pool of columns. The optimal solution of MPLP may give a lower bound for the
chromatic number, and the dual variables are saved to be used on the problem WMIP.
In the sequence, the same CGA is used to approximately solve WMIP, setting the last used number of
colors minus one, and storing the independent sets found. The new independent sets with bounds greater than
one on the problem WMIP are appended to the previous pool of columns and the MPLP is resolved. The
process continues until no more columns are found to be added to the MPLP .
A lower bound for the chromatic number is obtained at each iteration applying the Farley´s bound [9],
given by v( MPLP ) / v(WMIP) , where v(.) is the optimal solution of the corresponding problem. These values
change at each process iteration.
5 Computational tests
Computational tests were made with several instances taken from different groups found in the literature:
“book” graphs, “games” graphs, “miles” graphs, “register” graphs, “Mycielski” graphs and “Queen” graphs
[21].
The table 1 bellow gives initial computational results. The table contains the number of instances, average
numbers of vertices and edges, conflicts remaining after the CGA application for each instance group and
computer times for a C language code running on a Pentium II - 266 MHz machine. All the experiments were
made with three runs for each instance, and all of them for the known optimal number of colors (chromatic
number).
Table 1. CGA computational results
Group
Instances
Vertices
Edges
CGA
Times (s)
Books
4
94.7
363.5
0
1.9
Games
1
120
638
0
3.0
Miles
3
128
1223.3
0
17.4
Register
4
204.8
3848.6
0
508.9
Mycielski
3
111
1117
0
3.3
Queen
5
51
753.2
0.5
1021.5
The quality of the results can be easily seen, especially for the queen graphs, considered harder to solve.
The problem called Queen99 was the only one for which the chromatic number was not reached by the CGA.
Laguna and Martí [18] reported computational results for the same instances (excluding the Queen class)
applying a GRASP algorithm, and compare the results with other known heuristics. Table 2 shows the
comparison made and includes an extra column referring to the CGA application to the same instances. The
instances “book”, “games” and “miles” are grouped and named as SGB, and a larger number of instances are
investigated. Columns show the number of instances, average numbers of vertices and edges, the average
chromatic numbers, and the averaged minimum number of colors resulted after the application of CGA and
the following heuristics: GRASP [18]; TABUCOL [15] and Simulated Annealing (SA) [17]. Times are not
reported as the computational tests were performed on distinct machines. The CGA reproduced all the GRASP
best results and can be conjectured as an efficient heuristic.
Table 2. CGA comparison with other heuristics
Group
Instances
Vertices
Edges
Colors
TABUCOL SA
GRASP
CGA
Mycielski
5
73.4
688.4
6.0
6.0
6.0
6.0
6.0
Register
14
362.1
7608.1
37.4
57.1
40.1
37.4
37.4
SGB
10
113.9
2835.2
22.6
24.1
26.0
22.7
22.7
Finally we have proceed the tests with the column generation approach on the Queen99 instance. Table 3
shows the results.
Table 3. Column generation process for Queen99
Iteration Number
of colors
Best CGA number
MPLP bound
Farley´s bound
Time(sec.)
of conflicts
0
10
2
9.226
8.359
295
1
9
10
9.059
8.155
283
2
8
25
9.007
8.542
246
3
7
47
9.000
-
177
4
6
75
-
-
121
The times in table 3 correspond to the CGA application. The CPLEX 6.5 software package [16] used to
solve the MPLP problem was very fast, and their times are not reported. The best solution found by the CGA
was 11 colors. Considering the best MPLP and Farley´s bounds, it can be conjectured that the smallest
number of colors for Queen99 is 9, 10 or 11 (actually the number is 10). As the number of colors decreases,
the number of conflicts in CGA solutions increases, but the new columns improve the MPLP bounds. The
Farley´s bound have an oscillating behavior due to the fact that the problem WMIP was not exactly solved.
We have used a SUN-ULTRA30 and the CGA main parameters was iteration limit =20, α increase interval =
0.01, d = 0.15 and mult = 2.0.
6 Conclusion
The graph-coloring problem is an important classical and very studied problem. This paper described the
combined use of the Constructive Genetic Algorithm (CGA) as an auxiliary tool for the column generation
process. The initial pool of columns and the extra columns generated using the CGA is an original
contribution. The process gives upper and lower bounds for the chromatic number, that can be useful on tree
search algorithms.
The CGA approach considers the coloring problem as a clustering problem, introducing some new aspects,
such as the direct evaluation of schemata and the use of a known graph coloring heuristic as a part of the
representation.
The initial computational tests on instances available in literature have shown good results for the CGA
application when compared to other approaches and have opened a view to the integration of evolutionary
algorithms and well accepted mathematical programming approaches, like the column generation process.
Acknowledgments : The second author acknowledges Conselho Nacional de Desenvolvimento Científico e
Tecnológico - CNPq (proc. 300837/89-5) and Fundação para o Amparo a Pesquisa no Estado de S. Paulo FAPESP (proc. 99/06954-7) for partial financial support.
References
1. Arakaki, R. G. I. and Lorena, L. A. N., “A Constructive Genetic Algorithm for the Maximal Covering
Location Problem,” in Proceedings of MIC 2001: 4th Metaheuristics International Conference, Porto,
Portugal, 2001, pp 13-17.
2. Barnhart, C.; Johnson, E.L.; Nemhauser, G.L.; Savelsbergh, M.W.P. and Vance, P.H., “Branch-andPrice: Column Generation for Solving Huge Integer Programs,” Operations Research vol. 46, pp. 316329, 1998.
3. Briggs, P.; Cooper, K.; Kennedy, K. and Torczon, L., “Coloring heuristics for register allocation,” in
Proceedings of ASCM Conference on Program Language Design and Implementation, 1989, pp.275-284.
4. Costa, D. and .Hertz, A., “Ants can color graphs,” Journal of the Operational Society vol. 47, pp. 1-11,
1996.
5. Dantzig, G.B. and Wolfe, P., “Decomposition principle for linear programs,” Operations Research vol. 8,
pp. 101-111, 1960.
6. de Werra, D., “An introduction to timetabling,” European Journal of Operational Research vol.19, pp.
151-162, 1985.
7. de Werra, D., “Heuristics for Graph Coloring,” Computational Graph Theory, pp.191-208, 1990.
8. Desrochers, M. and Soumis, F., “A Column Generation Approach to the Urban Transit Crew Scheduling
Problem,” Transportation Science vol. 23, pp. 1-13, 1989.
9. Farley, A., “A note on bounding a class of linear programming problems, including cutting stock
problems, “ Operations Research vol. 38, pp. 922-923, 1990.
10. Fleurent, C. and Ferland J. A., “Genetic and Hybrid Algorithms for Graph Coloring,” Annals of
Operations Research vol. 63, pp. 437-464, 1996.
11. Friden, C.; Hertz, A. and de Werra, D., “ STABULUS: A technique for finding stable sets in large graphs
with tabu search,” Computing vol. 42, pp. 35-44, 1989.
12. Gamst, A. , “Some lower bounds for a class of frequency assignment problems,” IEEE Transactions of
Vehicular Technology vol. 35, pp.8-14, 1986.
13. Garey, M. R. and Johnson, D. S. Computers and Intractability: a Guide to the Theory of NPCompleteness, W. H. Freemann: San Francisco, 1978.
14. Gilmore, P.C. and Gomory, R.E. , “A linear programming approach to the cutting stock problem,”
Operations Research vol. 9, pp. 849-859, 1961.
15. Hertz A. and de Werra, D. , “Using tabu search techniques for graph coloring,” Computing vol. 39, pp.
345-351, 1987.
16. ILOG Inc. CPLEX 6.5, 1999.
17. Johnson, D. S.; Aragon, C. R. ; McGeoch, L. A. and Schevon, C. ,”Optimization by simulated annealing:
An experimental evaluation; part II, graph coloring and number partitioning,” Operations Research vol.
39, pp.378-406, 1991.
18. Laguna, M. nad Martí, R. Computational Optimization and Applications – to appear.
19. Leighton, F. T. , “A graph coloring algorithm for large scheduling problems,” Journal of Research of the
National Bureau of Standards vol. 84, pp. 489-506, 1979.
20. Lorena, L.A.N. and Furtado, J.C., “Constructive genetic algorithm for clustering problems,” Evolutionary
Computation vol. 8, pp. 309-327, 2001.
21. Mehrotra, A and Trick, M., “A column generation approach for graph coloring,” Available from
http://mat.gsia.cmu.edu/color.ps, 1995.
22. Oliveira, A. C. M. and Lorena, L. A. N., “A Constructive Genetic Algorithm for the Linear Gate
Assignment Problem”,
in Proceedings of the Genetic and Evolutionary Computation Conference,
GECCO, San Francisco, USA, 2001, p. 756.
23. Ribeiro Filho, G. and Lorena, L. A. N., “A Constructive Evolutionary Approach to the Machine-Part Cell
Formation Problem,” Buildings Competencies for International Manufacturing - Perpectives for
Developing Countries, pp. 340-348, 2000.
24. Ribeiro Filho, G.; Lorena, L. A. N. ,”A Constructive Evolutionary Approach to School Timetabling,”
Lecture Notes in Computer Science vol. 2037, pp. 130-139 , 2001.
25. Vance, P.H.; Barnhart, C.; Johnson, E.L. and Nemhauser, G.L. ”Solving Binary Cutting Stock Problems
by Column Generation and Branch-and-Bound,” Computational Optimization and Applications vol. 3,
pp. 111-130., 1994.