Applied Intelligence 20, 179–194, 2004 c 2004 Kluwer Academic Publishers. Manufactured in The United States. CLSS: An Intelligent Crane Lorry Scheduling System HON WAI CHUN AND REBECCA Y. M. WONG Department of Computer Science, City University of Hong Kong, Tat Chee Avenue, Kowloon, Hong Kong SAR andy.chun@cityu.edu.hk csymwong@cityu.edu.hk Abstract. Companies that provide crane-lorry services are faced with the daily need to perform vehicle and driver allocation and scheduling. Many companies still do this manually due to the lack of suitable technologies. This manual approach is both time consuming and inaccurate and most probably will not lead to an optimized plan that can reduce operational costs. In this paper, we describe the design of a system called “Crane Lorry Scheduling System” (CLSS) that we have developed for the largest crane lorry company in Hong Kong. A crane lorry company is a company that provides lorries with different types of mounted crane equipment and drivers to service different types of moving and lifting jobs. CLSS is a Web-based application that streamlines communication with customers, subcontractors and employees/lorry drivers. We modeled the lorry-assignment problem as a constraint-satisfaction problem (CSP) algorithm, which we call the “Crane Lorry Optimizing Engine” (CLOE). CLOE was designed to be easily customizable to match the needs and requirements of different crane lorry companies. We experimented with two versions of CLOE, regular CLOE that finds “best” solutions and X-CLOE that finds “optimal” solutions. Results from our tests show that CLOE is faster and generates better quality plans than the manual approach. Keywords: constraint satisfaction problem, CSP, constraint programming, vehicle scheduling, optimization 1. Introduction The main purpose of our research is to investigate how constraint-satisfaction techniques might be applied to solve a crane-lorry scheduling problem. The main research issues included the modeling of the problem as a constraint-satisfaction problem (CSP), the representation of domain knowledge as CSP constraints, the generation of “feasible,” “best” and “optimal” solutions, and the satisfaction of potentially contradicting objectives. This research was performed using data from the largest crane-lorry company in Hong Kong. Besides this company, there are roughly 30 other crane-lorry companies and hundreds of self-employed drivers that operate their own crane-lorry vehicles. On a daily basis, crane-lorry companies assign vehicles and drivers to service customer orders. Practically all companies here currently perform this manually. A human planner must have a rich knowledge of the problem domain as well as a good memory of historical patterns. In this paper, we describe how we encapsulated the knowl- edge of these human planners and modeled the problem as a Constraint Satisfaction Problem (CSP); the knowledge is represented as CSP constraints and search heuristics. Background on Crane-Lorry Companies A crane-lorry combines the mobility of a truck with the lifting ability of a crane, which is usually permanently mounted on the truck. These vehicles are especially popular here due to the large number of major capital projects, the narrow and winding roads, and the small operating spaces on many construction sites. There can be a wide variety of crane to lorry combinations. The exact combination to be dispatched depends greatly on the job requirements, such as the size and weight of the payload, the placement of the payload and the geography of the worksite. Cranes can be installed either at the front (behind the cab), center or rear of a truck. Crane trucks with crane 180 Figure 1. Chun and Wong Side and top view of truck with crane mounted in front. mounted in the front can carry larger payloads, but are usually considered less flexible or versatile because the cab of the truck blocks part of the crane’s path of movement (see Fig. 1). One the other hand, crane trucks with the crane mounted at the rear (R-type lorry) are considered more versatile as crane movement can cover a much wider area (see Fig. 2). Rear-mounted cranes are, however, limited by the size, weight and length of the payload that they can carry. In addition to the positioning of the crane, the size of the crane also affects versatility and capacity. Larger payloads require larger cranes, which in turn require larger trucks, for example construction material or even shipping containers. The lifting capacities of the cranes may range from a few to 60 or more tons. In Hong Kong, all cranes are assumed to have a 2-meter outreach. Since there are many different types of cranes, each with different features and operational procedures that can be substantially different from each other, crane Figure 2. Side and top view of truck with crane mounted in rear. operators must be adequately trained on the particular equipment he/she is operating. In most cases, the crane operator is also the lorry driver. For safety reasons, crane-lorry companies usually keep the crane-lorry to driver/operator assignment fixed for a period of time to minimize chances of operational errors. This assignment consistency gives the operator time to get familiar with the crane-lorry equipment and improves safety and operational efficiency. This assignment is usually only changed when a new crane-lorry is purchased or an old one is decommissioned; or when there is change in staffing or if the regular driver is not available. Moreover, although drivers have a set number of “days off” each month, their working days are not fixed. That is, they are assumed available whenever a customer order is assigned to them. Since the driver assignment schedule changes on an ad hoc basis, our algorithm does not generate the driver schedule. Instead, it assumes this schedule is a manually maintained table within the system. CLSS: Crane Lorry Scheduling System The Manual Scheduling Process The scheduling and allocation of crane-lorries to customer orders is the most critical daily task. The schedule must first of all be accurate—assigning the “right” vehicle-equipment combination to satisfy each customer order. It must also be optimizing—assigning the crane-lorry with just the right level of capabilities. If not, some jobs might need to be subcontracted out due to lack of suitably equipped vehicles. This not only means lost revenue, but even worse, lost customers that might not return. Another factor affecting revenue is the operational cost of assigning a suitable lorry with necessary capability to fit the customer’s budget requirement so as to avoid assigning an over-equipped lorry which will cost more than the customer is willing to pay and unduly increases the operational cost to the company. Another consideration is the traveling distance to the customer location; all factors being equal, the crane-lorry closest to the customer will be assigned to reduce operational costs. The priority of the customer and his/her preferences are the most important factors to be considered in generating the daily schedule. Other considerations include the features and capabilities of the lorry and its mounted crane, driver experience, past order history, and geographical location. To automate the entire scheduling process, our CLOE algorithms encapsulate these considerations as CSP constraints and search heuristics. Search heuristics are used to improve scheduling performance and quality. Quality is measured in terms of how well the schedule matches customer preferences, maximizes lorry-crane utilization and reduces cost. Since each crane-lorry company may have different business objectives and operational constraints, CLOE was designed to be easily customizable to fit the needs of different companies. Most of the service orders arrive by the end of each business day. At that time, the human planner will begin to construct the schedule for the following day. The planner must complete the schedule as soon as possible in order to determine which orders, if any, must be subcontracted, and then to contact subcontractors to reserve the required capacity. Finding suitable subcontractors that have the appropriate equipment available and confirming the subcontract orders can be a very time consuming process. The sooner a schedule is produced, the better the chance of finding a suitable subcontractor. As time passes, it becomes more and more difficult to find suitable crane-lorries, as other compa- 181 nies will have contracted them already. This implies that lorries will have to be leased from a more expensive company or a lesser quality or larger lorry must be used to satisfy customer needs. When the scheduling process is performed manually, there is always a chance of mistakes because of the large amount of data and large number of constraints and criteria that must all be considered simultaneously and under tight time constraints. Therefore, one of the key objectives in the design of our algorithm is to reduce the scheduling time while improving the quality of schedules. Under the same circumstances and conditions, a human planner can take up to an hour or more, depending on experience, to complete a schedule whereas CLOE can produce a schedule within seconds. It takes several months for a new planner to fully understand all the scheduling criteria and constraints in order to produce an error-free schedule; even the most experienced planner may make mistakes and cannot guarantee an optimal schedule can be produced. There is no way a human planner can thoroughly consider driver experience or historical data when performing scheduling. Only the most experienced planner can keep all the historical details in his/her head when constructing a schedule. Since all relevant constraints and all historical data are considered by CLOE, the system guarantees no mistakes will ever be made and the quality of schedule is best if not optimal. The measurement of how “good” a schedule is may differ from company to company. Our design allows companies to dynamically adjust CLOE to employ the heuristics that best reflect the company’s business objectives. For example, the primary objectives of the company we did this research on are to minimize operational cost and at the same time maximize customer satisfaction. Obviously, the second factor is very important in any service-oriented business because customer satisfaction is a critical part of business referrals and repeat business, both of which directly affect the company’s bottom line. In the following sections, the architecture of CLSS and the detail of the CLOE algorithms will be presented. 2. Architecture The software architecture (see Fig. 3) can be divided into four key modules according to the key business processes—Sales Module, Scheduling Module, Subcontracting Module, and Dispatching Module. Figure 3 182 Figure 3. Chun and Wong Software architecture. outlines the general architecture and operation of each module. Sales module responsible for maintaining customer and order information. It receives customer orders, handles customer queries and provides order tracking. Customer requests and orders come in throughout the day. The sales staff will process customers’ enquiries and provide product information, such as which types of lorries and cranes are available, and their respective capabilities and differences. After an order is finalized, it is passed to the “Scheduling Module” that assigns a suitable crane-lorry to each order. Scheduling module responsible for resource allocation and contains the CLOE algorithm. Scheduling is performed at a predetermined time each day. Allocation is done according to customer needs, driver experience, and other business and operational rules and considerations. In addition, the “Scheduling Module” will decide whether subcontractors are needed and which orders are to be contracted out. If there is an excess supply of crane-lorries, the planner will also determine which lorries are on “stand-by” for possible non-reserved orders. The scheduling process produces the following results: (1) which crane-lorries are still available, (2) which orders need to be subcontracted out, and (3) which crane-lorry is assigned to which order. The scheduling results are then passed to the Sales Module, the Subcontracting Module, and the Dispatching Module respectively for further processing. Dispatching module responsible for dispatching orders to drivers and for receiving confirmation from the drivers on their job status. Dispatching is done using mobile wireless devices. In the evening, after the Scheduling Module has completed the vehicle assignment, the Dispatching Module dispatches the next day’s jobs to the drivers. As jobs are mainly local, they are usually completed within the same day or overnight. Subcontracting module subcontracts out any excess orders it cannot handle to other companies. From the Scheduling Module, it receives a daily list of orders that cannot be fulfilled with internal company resources. The company then tries to subcontract these orders out to business partners. There is usually a pre-negotiated pricelist for different types of services. Consequently, all the subcontractors need to do is simply to confirm whether they have the capacity to handle the excess order that has been passed to them. If not, the order is sent to the next subcontractor in the pool. The key purpose behind subcontracting is to minimize revenue loss while maximizing quality of service. This means keeping the more important or larger jobs, especially those that offer long-term work, within the company itself. In this paper, we will focus mainly on the Scheduling Module that contains the CLOE algorithms. In this CLSS: Crane Lorry Scheduling System following Section, we will describe the knowledge encoded in and used by CLOE for scheduling. 3. Business Rules and Constraints The business logic, business objectives and operational constraints will impact how a scheduling algorithm is designed. The first piece of information used during scheduling is the details of the customer order. Each order is represented by a list of attributes, such as: • Date and time vehicle is needed • Duration of job—half day, whole day or overnight • Required crane-lorry capacity, including the size and type of lorry • Location of job • Lorry/driver preference if any • Contact information, etc. During the scheduling process, a human scheduler will not only need to consider these basic attributes, but also how to maximize customer satisfaction while minimizing costs. As these factors closely affect each another, it is important to achieve a balance between them. Customer Satisfaction The meaning of customer satisfaction can be different for different companies. CLOE is designed to be customizable and allows measures to be easily adjusted to suit the needs of different companies. For this company, customer satisfaction is defined as a combination of customer preference, driver performance and driver experience. For instance, a customer may prefer using a particular equipment or driver that it has utilized before. Assigning a driver with more experience can also increase customer satisfaction. Driver experience is defined as the number of times a driver has been assigned to a crane-lorry combination for a particular type of job before. The driver’s job performance is generated from historical statistics on customer’s after-service feedback. The net effect this will have on CLOE is that it will provide some regularity and consistency in vehicle-driver assignments over time, which this company has found to be highly desirable among its customers. An additional benefit is that drivers will be more familiar with their customers and their operations, which in many cases improves job satisfaction for the drivers. 183 Operational Cost Two key factors directly affecting the operational cost are the resource utilization and transportation costs. Maximizing resource utilization ensures resources are not underused. This also increases the likelihood that the company can satisfy more orders itself and minimize the need for subcontracting. Even if subcontracting is needed, the company should try to satisfy the larger orders and only subcontract out smaller, lowermargin orders to other companies. Besides subcontracting, assigning a crane-lorry with just the right capability is another way to reduce operational costs, because the size of the lorry is directly proportional to the operational cost of maintaining and operating it. Allocating crane-lorries that are closer to the customer sites will reduce transportation costs. This is of course assuming that the lorries have the same capability and degree of potential customer satisfaction as defined previously. Rules and Objectives The following is a list of the more important business knowledge extracted during business analysis. They are implemented either as CSP constraints (indicated by a “C” in the identifier), demons (“D”) or search heuristics (“H”). Implementation details will be described in later sections of this paper. • [C1] No overlap constraint: each lorry should only be assigned to one order at a time. • [C2] Fulfill workload constraint: the lorry assigned to a job must be adequately equipped for the job. • [D1] Minimize number of subcontract orders constraint: when subcontracting, minimize the number of orders given to subcontractors. • [D2] Minimize subcontract order size constraint: when subcontracting, subcontract orders that require smaller-sized lorries to reduce operational costs (considered by the “Subcontracting Module”). • [H1] Maximize driver experience and customer satisfaction: the most experienced available lorry-driver should be assigned to the most important customer and if the customer has a special preference for a particular lorry, try to satisfy the customer’s preference whenever possible. • [H2] Maximize utilization: if it is necessary to assign an over-equipped lorry in order to fill all remaining customer orders, the lorry with the closest level 184 Chun and Wong of capacity should be assigned without frustrating another order. In another words, the total tonnage required to carry out all customer requests should be minimized. • [H3] Minimize travel distance: where possible, assign the closest available lorry to the worksite. Underutilization Guidelines As we mentioned before, the customer may explicitly request a particular equipment/truck or driver when ordering. In some cases, this will cause the equipment to be underutilized and will lower its overall utilization rate and increase operational cost. On the other hand, fulfilling customer requests will improve customer satisfaction. In order to achieve a balance between lowering operational costs and improving customer satisfaction, the company has these “Underutilization Guidelines:” • Underutilization must not increase the number of subcontract orders nor the order size • Underutilization must not cause another order to be unsatisfiable Furthermore, the “importance” or priority of the customer will also be taken into consideration before a decision is made. In any case, the size-utilization1 of a crane-lorry should not be lower than 40% due to operational considerations; larger crane-lorry with less carrying capacity actually operates less efficiently. 4. Related Research There are many different areas of optimization and scheduling-related research on vehicles. For example, there is the vehicle routing problem (VRP), vehicle scheduling problem (VSP), automated guided vehicles (AGV), etc. However, there are only a few research are related to what we call the vehicle assignment problem (VAP), which this paper is about. Vehicle scheduling, in general, have played an important role in operations research (OR) ever since works from Dantzig and Ramser [3] and Clarke and Wright [4] have been published. More recently, researchers have been experimenting with a wider variety of techniques, including mathematical/linear programming (LP) [34, 37], constraint programming (CP) [5], genetic algorithms (GA) [6] as well as swamp/ant algorithms (AA) [38, 39] in solving these types of scheduling problems. LP represents the vehicle scheduling problem as a set of linear equations that can be solved using mathematical programming tools like CPLEX [36]. CP represents these problems in terms of variables and constraints and uses arc consistency techniques to improve search efficiency. GA, on the other hand, represents a vehicle schedule as a chromosome and uses survival-of-the-fitness techniques to iteratively evolve a solution. Similar to GA and borrowing from ants in nature, AA also iteratively evolves a solution. AA uses pheromone trail laying techniques to record past evaluation/heuristic function values as artificial pheromone. The vehicle routing problem (VRP) is basically a type of traveling salesman problem (TSP) where we need to schedule vehicles, instead of salesmen, to visit a set of customers to deliver and/or pickup goods. Amongst other things, there are constraints on the vehicles, customers, drivers, goods and time. The objective is to minimize cost, which is usually measured in terms of the number of vehicles and/or distance traveled and time spent. Shaw [7] used CP techniques combined with local searches in a way similar to the shuffling technique used in job-shop scheduling to solve the vehicle routing problem. On the other hand, Ochi et al. [8], used parallel GA combined with a scattered search and a decomposition-into-petal procedure to solve a similar problem. Interestingly, Barnier and Brisset [9] offer another approach to solve VRP by combining both CP and GA techniques. This combined approach was shown to perform well even for large search spaces. CP is used to compute feasible solutions in a subspace of the search space, while GA is used on the space formed by the subspaces to perform the optimization. There are also different variations to the vehicle routing problem. For example, Frizzell and Giffin [10] use a heuristic approach to solve an extended VRP with split delivery, allowing delivery to a customer to be split between two or more vehicles, and a timeframe when delivery must be made. Shang and Cuff [11] also used a heuristic algorithm to solve pickup and delivery problems within a Health Maintenance Organization (HMO). In this problem, vehicles are used to transport patient records, equipment and supplies. The objective was to minimize vehicle expense, tardiness, and travel time. Their research showed that their heuristic algorithm performed better at meeting these objectives than the manual approach. CLSS: Crane Lorry Scheduling System Since a wide variety of approaches have been tested on the vehicle routing problem, formal systems are being developed to help formalize and compare these approaches. For example, Desrochers et al. [12] outlined a formal system that can be used to define models and algorithms for vehicle routing and scheduling. Related to the vehicle routing problem is the vehiclescheduling problem (VSP) [13]. In simple terms, VSP is the problem of assigning vehicles to timetabled trips such that each trip is performed by one vehicle, a set of constraints is satisfied, and a cost function is minimized. The objectives usually include minimizing the number of vehicles and different operational costs, such as gas, labor, and deadheading trips. For example, bus scheduling is a typical vehicle-scheduling problem. Many real-life factors must be considered in solving VSP, such as the number of depots, travel time at different locations and times of the day, labor regulations that govern work hours of drivers, the types of vehicles available, refueling needs, etc. For example, Park and Song [14, 15] used heuristic algorithms combined with a simple estimation model for time varying travel times to solve VSP. A different set of problems related with vehicles and their scheduling is the problem of automated guided vehicles (AGV). AGV are used in many different environments such as factories, container terminals, and warehouses to transport goods and materials from one location to another. For example, Shawik [16] described research work in a flexible manufacturing system (FMS) and its related AGV scheduling problem using a heuristic approach. Ulusoy et al. [17] solves a similar problem for scheduling AGV within a FMS, by using genetic algorithms. Shawik, Ulusoy et al., combined manufacturing machine scheduling with AGV scheduling. Other researchers focused just on AGV scheduling. For example, Wu [18] worked on a restricted AGV scheduling problem using problem decomposition combined with a local search. Huang and Hallam [19] used a totally different approach of distributed AI (DAI) and negotiation among software agents to perform conflict resolution within the AGV scheduling problem. The work performed in the crane-lorry allocation is related to these research areas but with some differences. We call this the vehicle assignment problem (VAP). It is a specialized combination of the vehicle routing problem (VRP) and vehicle-scheduling problem (VSP). In our crane-lorry assignment problem, we need to satisfy pickup and delivery jobs by assigning 185 vehicle-driver pairs to each job. Unlike assumptions made by many VRPs and VSPs, our vehicles are not homogeneous and may contain different types of equipment. The routing is simplified, as the vehicles are highly specialized and unique and can only perform certain types of jobs. Hence, the vehicles usually perform only one or two jobs a day. The key problem to be solved in VAP is not to compute the most efficient route or schedule, but to find the most appropriate assignment of the equipment to each jobs while maximizing utilization and customer satisfaction. The “quality” of the assignment plays an important role as this can directly impact customer satisfaction. Hence VAP can be modeled as a resource assignment problem. Gill [20], in his SunRay V system, solved a related VAP for freight-forwarders also using CP techniques. Sunray V’s scheduling algorithm determines how many jobs the company can handle, which jobs to subcontract, and assigns trucks or towheads to different jobs. The key objective is to minimize subcontracted jobs, meet all job deadlines and minimize inter-job idle time. However, in the case of Sunray V, the towheads are generally homogeneous, unlike the crane-lorries in this problem. On the other hand, it is not necessary to schedule the jobs as the customer dictates when the lorry is needed. For the present problem, customer satisfaction and compatibility of driver’s experience with the job assignment is more important than minimizing travel time. Before actual scheduling, the SunRay V algorithm identifies the best pairings of tasks to minimize the number of paid-trips before the actual scheduling. The paid-trip also captures the transport and non-transport related execution times and associated auxiliary trips. There are also other preferences and considerations that must be taken into account in the actual connection selection procedure. The SunRay V system divides the scheduling into two parts. The first part produces a predictive schedule and identifies the most appropriate trip for sub-contracting if resources are insufficient. In the second part, the schedule produces an online schedule, which assigns each towhead with one trip after the executing or completing the current trip. Table 1 shows the differences between the SunRay V container-trucking problem and our cranelorry scheduling problem. Due to the high variation in the execution time and connection-time between each trip, the SunRay V requires an online scheduling module and must divide the schedule into parts. In addition, it pairs tasks before 186 Chun and Wong Table 1. Differences between the SunRay V container trucking and the crane-lorry scheduling problems. SunRay V scheduling Crane-lorry scheduling The scheduling module is divided into two consecutive parts. Produces the schedule and makes the sub-contracting decision simultaneously. Many factors affect the duration of the job and the connection-time between jobs. The customer specifies the required time (usually half a day, whole day or mid-night) when making the order. A Towhead may be assigned more than one trip a day and the assignment depends on the currently executing or completing trips. Each lorry is assigned one order per day (if it is a full day order). If two half-day orders suit the same lorry it will be assigned more than one order. Usually a lorry will not be assigned more than two orders on the same day. There is a high variability in job execution time, response time and communication time with the customer, which can cause frequent rescheduling. Customer preference and driver working experience are important considerations during the scheduling process and are the primary factors determining the schedule quality. Online schedule Offline schedule the actual scheduling in order to minimize the number of trips. In the crane-lorry scheduling problem, fewer time constraints need to be considered than in SunRay V, and the most important consideration is whether a cost effective schedule can be produced while satisfying customer preference. Another related research is Duncan’s Schedule-IT [21] application. Schedule-IT also uses a CP algorithm to allocate drivers and vehicles for the delivery of customer orders. Like SunRay V and CLOE, excess work is subcontracted out to other companies. There are a number of constraints concerning the capabilities of the resources (vehicles and drivers) and limitations on working hours. Schedule-IT uses a driver-driven algorithm that schedules all the jobs of one driver before moving on to the next. In this way, the scheduler partitions the problem into sub-problems. In the case at hand, all the orders are solved in one global set. To meet the objective of maximizing the degree of match between driver experience and the work order, the global approach was determined to yield better solutions. Table 2 shows the differences between the ScheduleIT and the crane-lorry scheduling system. Table 2. Differences between the Schedule-IT and the crane-lorry scheduling problems. Schedule-IT Completes one driver’s schedule after another, sequentially. The search space is split into separable sub-problems. A driver may be assigned different vehicles. JSolver scheduling module All orders are solved in one global set. The search space is not split into sub-problems. Each driver is responsible for one lorry; usually, no changes are made. 5. Model and Implementation Our CLOE scheduling algorithms were implemented using a Java constraint-programming language called JSolver [1, 2]. JSolver extends the object-oriented programming paradigm of Java with constraint-based declarative programming. The basic underlying mechanisms include an arc consistency checking algorithm, constraint propagation, and backtracking. It allows the development of scheduling and resource management systems in Java and deployment of these systems over the Web or within an Enterprise Java Bean (EJB) application server. JSolver has been successfully used to solve different types of engineering problems as well as e-trade negotiation problems [22]. We modeled the crane-lorry assignment problem as a constraint-satisfaction problem (CSP) [23–27]. In general, any scheduling or resource allocation problems can be formulated as a constraint-satisfaction problem (CSP), which involves the assignment of values to variables, subjected to a set of constraints. We have successfully used CSP algorithms for other transportationrelated resource allocation problem before [28–30]. CSP can be defined as consisting of a finite set of n variables v1 , v2 , . . . , vn , a set of domains d1 , d2 , . . . , dn , and a set of constraint relations c1 , c2 , . . . , cm . Each di defines a finite set of values (or solutions) that variable vi may be assigned. A constraint c j specifies the consistent or inconsistent choices among variables and is defined as a subset of the Cartesian product: c j ⊆ d1 × d2 × · · · × dn . The goal of a CSP algorithm is to find one tuple from d1 × d2 × · · · × dn such that n assignments of values to variables satisfy all constraints simultaneously. CSP algorithms increase the efficiency of the search CLSS: Crane Lorry Scheduling System by looking ahead, and actively using the constraints to prune the search space, thus minimizing backtracking. When the lorry-assignment problem is formulated as a CSP, each variable or unknown represents the assignment of a lorry to an order. The domain of each variable, that is, the possible orders a lorry may be assigned to, will initially contain the set of all orders for the next day. The CSP constraints are restrictions on how these lorries can be assigned based on business and operational rules and objectives. The CLOE Algorithm The CLOE algorithm is a recursive search algorithm that iteratively assigns a value to a variable. We experimented with two models—a lorry-centric model that used variables to represent crane-lorries, and an order-centric model that used variables to represent customer orders. In this paper, we only documented the lorry-centric model as their performances were similar. For the lorry-centric model, at each recursion level, CLOE selects the next “best” unassigned crane-lorry and either assigns the “best” order to it or marks it as “unassignable,” while satisfying all the predefined constraints. A crane-lorry is “unassignable” if it is under-equipped to handle any of the outstanding orders. On the other hand, assignment can still be made if the crane-lorry is over-equipped. Over-equipped cranelorries should only be assigned, however, if there are no other options left, in order to avoid under-utilization or, even worse, making another order unfulfillable due to a lack of suitable equipment. We use search heuristics to define what “best” unassigned crane-lorry means, as well as what “best” order to assign means. These heuristics guide the search as well as improve the quality of the resulting schedule. Further use of an arc-consistency algorithm is made to perform constraint-propagation and domain reduction in order to further improve search performance. The following is the pseudo-code for the CLOE algorithm using notations similar to those used by Tsang [23]: [Z ] finite set of variables (the set of all unassigned crane-lorries) [D] finite set of all variable domains (domain contains all the orders for the day) [C] finite set of constraints (restrictions on how lorries can be assigned) 187 PROCEDURE CLOE (Z, D, C) BEGIN Solution ← NIL; D ← Propagate (Z, D, C, 0); IF (no domain in D is empty) THEN solution ← Search (Z, 0, D , C); END The CLOE procedure first calls the Propagate procedure to perform domain reduction on posted constraints. (The last parameter to Propagate is the variable binding that triggered this propagation. This is null for the initial constraint posting.) This pre-search arc-consistency check will improve performance by removing all “obvious” infeasible choices from the domains of the variables. Infeasible choices are defined through constraints contained in the set of constraints “C” that governs how lorries can be assigned. This set contains instances for several types of constraints. At this pre-search stage of the algorithm, the constraint (see Section 4) that has the largest impact is: • [C2] Fulfill Workload Constraint: the lorry assigned to job must be adequately equipped for the job There is one “Fulfill Workload Constraint” per variable. The effect of posting each “Fulfill Workload Constraint” is that the domain of that variable will be guaranteed to contain only orders it can fulfill. This is done through domain reduction and is performed by the Propagate procedure prior to calling the Search. All orders that the lorry is not equipped to fulfill are removed from its domain. If the problem is “overconstrained,” some of the resulting domains will be empty. When this happens, there is no solution and the algorithm exits. Otherwise, after the initial search space has been reduced, CLOE simply initializes the set of committed labels, i.e., the partial solution set, (second parameter to the Search procedure) to nil and starts the actual search. The Search procedure is the recursive algorithm that performs the actual search and is defined below. The following notations are used: x: an unassigned variable (an unassigned crane-lorry) Dx: the domain of variable x (the set of all orders that the lorry can fulfill) x, v: assignment of value v to variable x (the assignment of an order to a lorry) U : set of unlabelled variables (set of unassigned cranelorries) 188 Chun and Wong CL: set of committed labels (set of assigned cranelorries) C: finite set of constraints (restrictions on how lorries can be assigned) We first evaluate the number and size of the order to be subcontracted. After an order is assigned to a lorry, we can check whether the remaining order and unassigned lorry will result in increase the number and PROCEDURE Search (U, CL, D, C) BEGIN fail ← FALSE; IF (U = {}) THEN RETURN (CL) //all lorries have been assigned orders ELSE BEGIN x ← SelectLorry (U ); //find next “best” lorry to schedule REPEAT v ← SelectOrder (Dx); //find next “best” order to assign Dx ← Delete (v, Dx); //delete selected order from Dx IF (AddValueDemon = pass) //if CL + {x, v} violate no constraints THEN BEGIN D ← Propagate (U − {x}, D, C, x, v); // constraint propagation If (no domain in D is empty) THEN BEGIN // recursively search result ← Search (U − {x}, D, CL + {x, v}, D , C); IF (result ! = NIL) THEN RETURN (result); END ELSE fail ← TRUE; // domain in D is empty END ELSE fail ← TRUE; // new compound label violates constraints UNTIL (Dx = {} / fail ! = TRUE); RETURN (NIL); END END The main tasks for Search are: (1) to select the next best unassigned lorry using the SelectLorry procedure; (2) assign the next best order with SeletOrder; (3) after a value is assigned to the variable the AddValueDemon will execute to check whether the new compound label violates any constraint; and finally (4) to Propagate constraints. SelectLorry defines the heuristic for selecting the next “best unassigned crane-lorry.” SelectOrder defines the heuristic for selecting the next “best order to assign.” AddValueDemon is executed just after a value is assigned to a variable in order to check those relational constraints. The following relational constraints defined in Section 4 are implemented as demons and processed in AddValueDemon: • [D1] Minimize Number of Subcontract Order Constraint: minimize the number of orders given to subcontractors • [D2] Minimize Subcontract Order Size Constraint: when subcontracting, subcontract orders that require smaller-sized lorries to reduce operational costs size of order that need to be subcontracted to an outside company. If the constraint has been violated, the algorithm will backtrack to a previous choice point. Propagate is the constraint propagation algorithm that performs domain reduction. It is a type of lookahead strategy [23, 31] to detect constraint violations. The key constraints (see Section 4) that are propagated during the search are instances of: • [C1] No Overlap Constraint: each lorry should only be assigned to one order at a time There is one “No Overlap Constraint” per variable, which represents a lorry-to-order assignment. The effect of posting each “No Overlap Constraint” is that whenever an order has been assigned a lorry, all the domains of the remaining unassigned lorries will have their domains reduced by removing this order, as the order will no longer be a valid choice. Whenever any domain gets reduced to an empty set or when a legal lorry-order assignment cannot be found, CLSS: Crane Lorry Scheduling System the Search algorithm returns NIL. This forces the algorithm to backtrack to a previous level and continue the search at the previous choice points. The Search Heuristics The above describes how CLOE implements “hard” constraints of the crane-lorry assignment problem. Hard constraints are requirements that must never be violated. With just hard constraints, CLOE will be able to generate “feasible” solutions. CLOE uses search heuristics to further improve the quality of the search to produce “best” solutions. Best solutions are those that try to satisfy other “soft” requirements such as customer preferences. In CLOE, soft constraints are implemented in the search heuristics SelectLorry and SelectOrder. 189 received from the customer on how “satisfied” they were with the service provided. We found that these heuristics performed equally well if schedule quality was defined in terms of “costs” only. However, if we consider “customer satisfaction”, the Experience Heuristic performed significantly better [32]. CLOE implements these heuristics as a weighted sum with the weights adjustable for different companies. The SelectCrane Procedure The SelectCrane procedure defines the heuristic for what the “best” order to assign to a crane-lorry. The heuristic used by CLOE is: The SelectLorry Procedure • Maximize Utilization Heuristic: Always select the largest orders a lorry can handle and from these the order that it has most experience in handling. The SelectLorry procedure defines the heuristic for what the next “best” unassigned crane-lorry means. Many different domain-specific heuristics [32] were experimented with in order to find the one that produces the best improvement in the quality of the generated schedule. These include the Versatility Heuristic, the Size Heuristic and the Experience Heuristic: The definition of “largeness” may vary from company to company and usually means the orders that have the most stringent requirements on lorry capabilities. This may be in terms of maximum gross vehicle weight, maximum capacity, size of the lorry, or combinations of these factors. In the company we performed this work on, we define “largeness” as a weighted sum of all these factors. • Versatile Heuristic: By selecting the most “versatile” lorry first and assigning the best order to it, we try to maximize customer satisfaction by assigning the “most preferred” equipment first and also maximize equipment utilization. “Versatility” is defined in terms of the normalized maximum ton-meters of the crane, normalized maximum gross vehicle weight of the lorry, normalized payload area of the vehicle, mounting location of the crane and company-specific weighting factors. • Size Heuristic: The Size Heuristic is a simplified form of the Versatility Heuristic and focuses just on the workload handling capability of the lorry. “Size” is defined to be the normalized maximum gross vehicle weight (GVW) of the lorry. • Experience Heuristic: This heuristic is based on the historical statistics of the crane-lorry-driver combination in performing similar tasks (potential jobs in the variable domain) in the past and the feedback The Objective Function An extended version of the CLOE algorithm, called X-CLOE, was also implemented to experiment with generating solutions that optimized an objective function. X-CLOE is similar to CLOE with the addition of “branch and bound” optimization. Multi-criterion decision modeling is used to formulate this objective function, TF, which returns a number that reflects the overall “goodness” of the schedule. This goodness measure includes driver experience, customer satisfaction, size utilization and the travel distance. The definition of this function may differ from company to company and depends on their respective business strategies. TF is the total sum of the goodness measure, Fcxj , for each assignment in the schedule. For this company, Fcxj is defined as: Fcxj = (w1 ∗ lc ∗ Pcxj ) + (w2 ∗ Scj ) + (w3 ∗ Dcj ) (1) 190 Chun and Wong where Fcxj : the “goodness” measure of assigning crane-lorry c and driver x to job j lc : the “importance” level or priority of a customer Pcxj : customer feedback historical statistics on performance of crane-lorry-driver for job j; range: 0..100, Pcxj = 100 if customer highly recommends the combination and Pcxj = 0 if no preference. Scj : size utilization if crane-lorry c is assigned to job j; range: 0..100, Scj = 100 if resource c will become fully utilized. Dcj : the normalized travel distance between home location of crane-lorry c to location of job j; range 0..100, Dcj = 0 if the crane-lorry is right next to the worksite. wi : weighting factors that vary from company to company. Higher values for w1 increases customer satisfaction, while w2 and w3 improves utilization and reduces costs. After each solution, a new TF value is computed and compared with the previous using a “branch and bound” technique [33]. This way, X-CLOE incrementally finds the next best solution that optimizes TF. Depending on the number of orders, the search space can be quite large. Instead of waiting of an “optimal” solution, X-CLOE also implements a timer mechanism to return the best “near optimal” solution so far up to the pre-set time limit. For all the X-CLOE experiments in this paper, we set the timer to return a near optimal solution to 20 seconds. The CLOE algorithms can be customized for different crane-lorry companies simply by adjusting the weights used in Eq. (1) to reflect a company’s unique business objectives and operational needs. Alternatively, a totally different objective function with a different formula can be “plugged in” to replace the default one used by CLOE. 6. Test Results The following are results from our tests on the CLOE and X-CLOE algorithms using seven sets of actual data reflecting typical orders received by the project company. Historical data, used to determine “experience,” was compiled using the actual schedule from the prior two weeks as well as customer feedback on driver performance. The results from testing these two algorithms were then compared according to two key criteria: “cost” and “quality”. “Cost” is measured in terms of distance traveled and the total tonnage required to fulfill all customer requests. These criteria represent the primary operational costs of the company. “Quality” is measured in terms of customer satisfaction and this translates to a historical preference-level. The Cost Factor Operational cost is one of the factors we used to compare CLOE with the manual approach. Operational cost is directly proportional to the total tonnage and the traveling distance. The “total tonnage” used is measured in terms of the number and type of vehicles that were allocated. Table 3 shows the number and types of vehicles used in the manual schedule. The vehicle type is important, as larger vehicles require more resources to operate. For simplicity, the vehicle type is represented as the maximum lifting capacity (payload weight) of the mounted crane. Crane outreach is normalized to 2 meters. For example, “25T” indicates that a vehicle has a crane that can lift a payload of 25 tons with an outreach of 2 meters. The “total tonnage” is pretty much dictated by the customer orders and room for improvement is limited. We used this measure purely to verify the accuracy of the CLOE algorithms. As Table 4 and Chart 1 illustrate, both CLOE and X-CLOE performed equally well, if not better than, the manual approach in terms of “total tonnage.” CLOE and X-CLOE performed better than the manual schedule in Test 5. After comparing results of the manual and CLOE generated schedules for Test 5, the analysis indicated that the manual schedule assigned a larger crane-lorry to one of the orders, which resulted in the increase in total tonnage used. This is one example of how CLOE can prevent human errors in allocation. Table 3. Number of vehicles used for each set of test data (manual approach). Manual Test 1 Test 2 Test 3 Test 4 Test 5 Test 6 Test 7 12T 0 0 0 0 0 0 0 15T 0 1 2 2 2 2 0 17T 1 3 4 2 2 4 4 20T 0 3 3 1 3 3 3 25T 2 5 6 3 6 5 6 35T 2 2 2 1 0 2 1 55T 0 0 0 1 0 1 0 CLSS: Crane Lorry Scheduling System Chart 1. Percentage reduction in terms of “total tonnage” on schedule generated by CLOE and X-CLOE for each of the test case. “Travelling distance” is another way to measure cost. Table 5 shows the total distance between the parking location of the lorry and the job location. The percentage cost reduction from CLOE is shown in Chart 2. Chart 2 shows that the CLOE performed better on Test 3, and equally well on Tests 1, 2 and 4 compared with the manual schedule. However, CLOE did slightly Table 4. tonnage. 191 Percentage reduction in operation cost in terms of total Test 1 Test 2 Test 3 Test 4 Test 5 Test6 Test 7 Manual 137 321 378 249 294 408 313 CLOE 137 321 378 249 281 408 313 X-CLOE 137 321 378 249 284 408 313 %Reduction— CLOE 0 0 0 0 4.42 0 0 %Reduction— X-CLOE 0 0 0 0 3.4 0 0 Table 5. The total distance and the corresponding percentage reduction in each test case. Distance Test 1 Test 2 Test 3 Test 4 Test 5 Test 6 Test 7 Manual 185 430 465 340 365 645 445 CLOE 185 430 440 340 385 655 495 X-CLOE 185 395 440 340 370 650 445 %Improve— CLOE 0 0 5.38 0 −5.48 −1.55 %Improve— X-CLOE 0 8.14 5.38 0 −1.37 −0.78 −11.24 0 worse on Tests 5, 6 and 7. Compared with manual schedules, X-CLOE performed better on Tests 2 and 3, and equally well on Tests 1, 4 and 7. Although XCLOE performed slightly poorer, around 1%, on Tests 5 and 6, it did obtained an overall 11.3% cost reduction in terms of distance. In all our test cases, the X-CLOE performed better than the CLOE in reducing the overall distance traveled. The Quality Factor The second key criterion in our evaluation of the CLOE scheduling algorithms is whether they can generate “quality” assignments, which is measured in terms of degree of “customer satisfaction.” This is measured by consolidating historical statistical data and customer feedback and reflects driver experience and customer preference. This “preference-level” measure is the parameter Pcxj defined in our objective function (Eq. (1) in the previous section). Table 6 shows the Pcxj values for the CLOE algorithms compared with the manual approach. Chart 3 shows the percentage improvements. As Chart 3 shows, CLOE algorithms performed better than manual approach, except for Tests 3 and 5. After comparing results between manually and CLOE-generated schedules for Test 3, we found that the manual schedule incorrectly allocated a highly over-equipped crane-lorry to one of the orders (a 25T crane-lorry was assigned to an order requiring an 8T crane-lorry) resulting in only 32% size utilization. Underutilization improves the preference score by increases cost. CLOE ensures utilization is always 192 Chun and Wong Chart 2. Percentage cost reduction in terms of distance between the location of the crane-lorry and the worksite. Chart 3. Percentage improvement in quality (in terms of preference level) for each of the algorithm compared with manual approach. maximized. Overall, CLOE obtained 8% improvement in customer satisfaction. In all our test cases, X-CLOE performed better than CLOE and obtained an overall 12.6% improvement. Table 6. Total preference-level and the percentage improvement for each of algorithm compared with manual approach. Preference Test 1 Test 2 Test 3 Test 4 Test 5 Test 6 Test 7 Manual 130 510 536.8 480 CLOE 130 510 530 480 454.94 464.57 420 450 480 450 X-CLOE 130 510 530.4 480 470.4 480 450 %Improve (CLOE) 0 0 −1.19 0 −1.086 3.32 7.14 %Improve (X-CLOE) 0 0 −1.19 0 3.31 3.32 7.14 By combining the percentage improvements in cost and quality factors, see Table 7, X-CLOE performs better than the manual approach and regular CLOE in most of the test cases. On the surface, these percentage improvements may seem small. However, there are numerous other significant benefits from CLOE that are hard to quantify. For example, long-term business benefits Table 7. Overall % improvement of cost and quality factors. Test 1 Test 2 Test 3 Test 4 Test 5 Test 6 Test 7 %Improvement— CLOE %Improvement— X-CLOE 0 0 0 4.19 0 8.14 4.19 0 −2.146 1.77 −4.1 5.34 2.54 7.14 CLSS: Crane Lorry Scheduling System from better overall customer satisfaction cannot be underestimated. There is also the operational benefit for having a precious 1 to 2 hour lead time over competitors in acquiring necessary vehicles from business partners—time saved by using CLOE. This lead time can make a big difference in being able to satisfy all customer orders as well as getting a better price from subcontractors. Expertise in crane-lorry scheduling requires years of experience in the same company. Unfortunately, this experience does not map well to another company as each company’s equipment and team, as well as business requirements and objectives can be very different. With increasing consolidation in this industry, companies must rely on automated systems such as CLSS and CLOE as human expertise does not scale well for this problem. 7. Conclusion The quality of service provided and the degree of customer satisfaction are important differentiators for service-oriented companies such as crane-lorry leasing companies. Schedules for operations must be created with these crucial business objectives in mind. In this paper, we presented a CLOE scheduling algorithm that achieves these objectives by capturing domain knowledge in the crane-lorry leasing industry and representing them as constraints and heuristics in a CSP algorithm. The X-CLOE algorithm then extends the original CLOE with optimizing branch-and-bound to incrementally search for better solutions. Overview, the CLOE algorithms produce more cost effective schedules than manual approaches. Our tests show that by considering the experience of the crane-lorry-driver combination, traveling distance and the size utilization, the CLOE algorithms were able to generate schedules that were equal or better than those produced by human planners and in greatly reduced time—within seconds. Although this type of constraint-based scheduling is rarely used in the vehicle leasing industry, we hope that this work can excite interest in this area by demonstrating the business values for improving customer satisfaction and lowering operational costs. Acknowledgment The work described in this paper was substantially supported by a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China (Project No. 9040517, CityU 1109/00E). This work 193 was also partially supported by a grant from City University of Hong Kong (Project No. 7001286). Note 1. Size utilization = size of the order ∗ 100/size of the lorry. Example: the size utilization for a 35T lorry to carry out a 25T order is equal to 71%. References 1. H.W. Chun, “Constraint programming in Java with JSolver,” in PACLP 99, London, April 1999. 2. http://www.ilog.com 3. G.B. Dantzig and J.H. Ramser, “The truck dispatching problem,” Management Science, vol. 6, pp. 80–91, 1959. 4. G. Clarke and J.W. Wright, “Scheduling of vehicles from a central depot to a number of delivery points,” Operations Research, vol. 12, pp. 568–581, 1964. 5. J.-F. Puget, “Object-oriented constraint programming for transportation problems,” in ILOG Solver Collected Papers, ILOG A, France, 1994. 6. J.H. Holland, Adaptation in Natural and Artificial Systems, University of Michigan Press, Ann Arbor, 1976. 7. P. Shaw, “Using constraint programming and local search methods to solve vehicle routing problems,” in CP’98, edited by M. Maher and J.-F. Puget, vol. LNCS 1520, pp. 417–431, 1998. 8. L.S. Ochi, D.S. Vianna, L.M.A. Drummond, and A.O. Victor, “A parallel evolutionary algorithm for the vehicle routing problem with heterogeneous fleet,” Future Generation Computer Systems, vol. 14, pp. 285–292, 1998. 9. N. Barnier and P. Brisset, “Optimization by hybridization of a genetic algorithm with constraint satisfaction techniques,” Evolutionary Computation Proceedings, pp. 645–649, 1998. 10. P.W. Frizzell and J.W. Giffin, “The split delivery vehicle scheduling problem with time windows and grid network distances,” Computers Operations Research, vol. 22, no. 6, pp. 655–667, 1995. 11. J.S. Shang and C.K. Cuff, “Multicriteria pickup and delivery problem with transfer opportunity,” Computers & Industrial Engineering, vol. 30, no. 4, pp. 631–645, 1996. 12. M. Desrochers, C.V. Jones, J.K. Lenstra, M.W.P. Savelsbergh, and L. Stougie, “Towards a model and algorithm management system for vehicle routing and scheduling problems,” Decision Support Systems, vol. 25, pp. 109–133, 1999. 13. F. Baita, R. Pesenti, W. Ukovich, and D. Favaretto, “A comparison of different solution approaches to the vehicle scheduling problem in a practical case,” Computers & Operations Research, vol. 27, pp. 1249–1269, 2000. 14. Y.B. Park and S.H. Song, “Vehicle scheduling problems with time-varying speed,” Computers & Industrial Engineering, vol. 33, nos. 3/4, pp. 853–856, 1997. 15. Y.B. Park, “A solution of the bicriteria vehicle scheduling problem with time and area-dependent travel speeds,” Computers & Industrial Engineering, vol. 38, pp. 173–187, 2000. 16. T. Sawik, “A multilevel machine and vehicle scheduling in a flexible manufacturing system,” Mathematical Computer Modeling, vol. 23, no. 7, pp. 45–57, 1996. 194 Chun and Wong 17. G. Ulusoy, F.S.-Serifoglu, and U. Bilge, “A genetic algorithm approach to the simultaneous scheduling of machines and automated guided vehicles,” Computers Operations Research, vol. 24, no. 4, pp. 335–352, 1997. 18. X.B. Wu, “The application of analytic process of resource in an AGV scheduling,” Computers & Industrial Engineering, vol. 35, no. 1/2, pp. 169–172, 1998. 19. X. Huang and J. Hallam, “Spring-based negotiation for conflict resolution in AGV scheduling,” IEEE International Conference on Systems, Man and Cybernetics, vol. 1, pp. 789–794, 1995. 20. A.J. Gill, “SUNRAY V—An intelligent container trucking management system,” in Proceedings of the First ILOG SOLVER and ILOG SCHEDULE International User’s Meeting, 1995. 21. Tim Duncan, “Schedule-IT: An intelligent vehicle scheduling,” in Proceedings of the First ILOG SOLVER and ILOG SCHEDULE International User’s Meeting, 1995. 22. http://www.cmis.csiro.au/aai/ 23. E. Tsang, Foundations of Constraint Satisfaction, Department of Computer Science, University of Essex, Colchester, Essex, U.K. 24. J. Cohen, Constraint Logic Programming, Communications of the ACM, vol. 33, no. 7, pp. 52–68, 1990. 25. G.L. Steele Jr., The Definition and Implementation of a Computer Programming Language Based on Constraints, Ph.D. Thesis, MIT, 1980. 26. V. Kumar, “Algorithms for constraint satisfaction problems: A survey,” Al Magazine, vol. 13, no. 1, pp. 32–44, 1992. 27. P. Van Hentenryck, Constraint Satisfaction in Logic Programming, MIT Press, 1989. 28. H.W. Chun, “Scheduling as a multi-dimensional placement problem,” Engineering Applications of Artificial Intelligence, vol. 9, no. 3, pp. 261–274, 1996. 29. H.W. Chun and R.W.T. Mak, “Intelligent resource simulation for an airport check-in counter allocation system,” IEEE Transactions on Systems, Man, and Cybernetics—Part C: Applications and Reviews, vol. 29, no. 3, pp. 325–335, 1999. 30. H.W. Chun, S. Chan, F. Tsang, and D. Yeung, “Stand allocation system (SAS)—A constraint-based system developed with software components,” Al Magazine, vol. 21, no. 4, pp. 63–74, 2000. 31. D. Frost and R. Dechter, “Look-ahead value ordering for constraint satisfaction problems,” in Proceeding of the Fourteenth International Joint Conference on Artificial Intelligence, 1995, vol. 1. 32. R.Y.M. Wong and A.H.W. Chun, “An intelligent crane-lorry scheduling system using constraint satisfaction,” in Proceedings of 4th World Multiconference on Systemics, Cybernetics and Informatics (SCI2000), Orlando, Florida, July 23–26, 2000. 33. P.H. Winston, Artificial Intelligence, 3rd edition. AddisonWesley Pub. Co., 1992. 34. H. Paul Williams, Model Building in Mathematical Programming, 4th edition, John Wiley & Sons, 1999. 35. K. Marriott and P.J. Stuckey, Programming with Constraints: An Introduction, MIT Press, 1998. 36. CPLEX, http://www.ilog.com/products/cplex/ 37. W.L. Winston, et al., Introduction to Mathematical Programming: Applications and Algorithms, Wadsworth Publishing Company, 1997. 38. J. Kennedy, et al., Swarm Intelligence, Morgan Kaufmann Publishers, 2001. 39. E. Bonabeau, M. Dorigo, and G. Theraulaz, Swarm Intelligence: From Natural to Artificial Systems, Oxford University Press, 1999.