To: Dr. Michael Branicky
From: Gary Doran
Re: Particle Swarm Algorithms for Business Optimization
Date: February 12, 2016
Executive Summary
I propose to research the application of particle swarm algorithms to solving Mixed Integer
Programming (MIP) and related business optimization problems, with an eye to utilizing the parallelism of
modern processing architectures.
Without computers in general, and specifically without theoretical research into efficient problemsolving and optimization algorithms, modern businesses could not operate. For every business that produces,
ships, or stores goods, there exists the need to schedule production lines, load vehicles for product transport,
and allocate warehouse space, just to name a few examples. Particle Swarm, or simply Swarm, Algorithms
have proven to be a valuable optimization technique for a variety of problems. Swarm Algorithms extend the
concept of genetic algorithms by facilitating interactions between agents, and were originally designed to
mimic the seemingly intelligent behavior of swarms of animals in nature, such as flocking geese or insect
colonies. Literature shows both that swarm algorithms can be applied to optimization in discrete domains (i.e.
to MIP and related problems) and that they can be readily parallelized, providing evidence that swarm
algorithm research might address a significant business need.
Due to the possibly profitable nature of the research, there are many stakeholders. First, Case
Western Reserve would own rights to any patentable material that may result from the research.
Furthermore, specialized business software companies such as ILOG and SAP would have a stake in the
research because they would profit from more efficient algorithms. The software companies’ interest is
demonstrated by their own research into improving current optimization techniques. ILOG and others’ stake
in the proposed research might be leveraged as a funding opportunity. Working through Case Western
Reserve’s Technology Transfer Office, a “Sponsored Research Agreement” would grant companies like ILOG
the opportunity to negotiate with the university regarding intellectual property rights to the fruits of the
research, giving them an incentive to financially support our endeavor. Lastly, typical businesses who are the
clientele of ILOG and others are stakeholders in the research due to the possible efficiency increase in their
own operations. Thus, the multiplicity of stakeholders presents numerous funding opportunities.
Having presented the case for the proposed investigation’s necessity and merit, I ask you to please
permit me to conduct this promising research.
Problem Statement
Without computers in general, and specifically without theoretical research into efficient problemsolving and optimization algorithms, modern businesses could not operate. For every business that produces,
ships, or stores goods, there exists the need to schedule production lines, load vehicles for product transport,
and allocate warehouse space, just to name a few examples. Each of these problems requires a complex
process of optimization that can only be accomplished by computer systems. Many of the optimization
techniques and algorithms that already exist are implemented by specialized software companies like SAP and
ILOG. However, both these companies and those in academia continue to research new techniques to
improve the efficiency and quality of results. A particularly active area of investigation concerns Mixed Integer
Programming (MIP). ILOG researchers have attempted to employ non-local search techniques such as genetic
algorithms to MIP problems as recently as 2007, demonstrating their interest in improving performance.
Furthermore, the more multiprocessor technology becomes accessible technologically and financially, the
greater the need for algorithms designed specifically to exploit parallelism to replace standard singleprocessor techniques. Therefore, I propose to research the application of particle swarm algorithms to solving
MIP and related business optimization problems, with an eye to utilizing the parallelism of modern processing
architectures. Such research would not only advance knowledge in the field of optimization, but would be
valuable to companies like ILOG and to their clientele as well.
Research Description
Although a relatively recent development in the field of optimization algorithms, Particle Swarm, or
simply Swarm, Algorithms have been shown to be a valuable optimization technique for a variety of problems.
Swarm Algorithms extend the concept of genetic algorithms by facilitating interactions between agents. The
hope is that, much like human interaction through culture, a sharing of information between agents will more
quickly produce a solution than keeping individual agents separate. In principle, the justification that swarm
algorithms work is an appeal to the seemingly intelligent behavior of swarms of animals in nature, such as
flocking geese or insect colonies. Like genetic algorithms, swarm algorithms can be applied to a wide range of
problems. Although originally developed to optimize solutions to continuous-valued problems, some research
suggests that swarm algorithms are also useful for problem-solving in discrete domains, which makes particle
swarm optimization efficient for solving MIP and related business problems. Furthermore, a swarm can be
split up into several sub-swarms that are each run concurrently on a multiprocessor system. Accordingly, the
nature of swarm algorithms creates an excellent opportunity for fruitful research into the application of
swarm techniques for solving problems in business and manufacturing optimization. I propose research to
leverage these aspects of swarm algorithms that give them a unique advantage in business optimization over
other local-search optimization techniques such as genetic algorithms or gradient search.
A Brief Literature Review
The technique of swarm optimization first appeared in a 1995 paper by James Kennedy and Russell
Eberhart (1942). The swarm algorithm was originally inspired by the seemingly intelligent behavior of groups
of animals in herds, schools, flocks and colonies (1942-1943). The method uses a swarm of “particles”
travelling through a space of problem solutions. Particles, “remember” both the location of the best solution
they have found, and the best solution found by any member in the group. The trajectories of the particles
are randomly influenced by the positions of best solutions (1944). Originally, the authors used swarm
algorithms to optimize weights in Neural Networks, producing better results than existing techniques such as
back-propagation, but suggested possible applications to other optimization problems in general (1947-1948).
As discussed, businesses must find optimal solutions to the problems of producing, shipping and
storing goods. In particular, many of these problems can be generalized to Mixed Integer Programming (MIP),
whose solutions exist in a discrete, rather than continuous, search space. Companies such as ILOG, which
create specialized computer systems for business and industrial optimization, have researched techniques for
solving such problems. Current techniques employed for MIP problems, such as those explored in a 2003
technical paper by ILOG’s Emilie Danna, Edward Rothberg and Claude La Pape, rely on relaxing MIP problems
to a continuous space, then optimizing using local search techniques (2).
Among the local search techniques applied to MIP problems are genetic algorithms, which were
specifically explored in a 2007 paper by ILOG’s Edward Rothberg (534). Rothberg found that the search space
around already-known good solutions tended to be filled with better, more optimal solutions (541).
Rothberg’s paper is promising for the proposed swarm algorithm research for several reasons. First, it has
been known since the 1995 Kennedy and Eberhart paper that swarm optimization performs well, and often
better, on problems optimized by genetic algorithms (1947). Thus, swarm algorithms can be expected to
produce optimal results for MIP problems more efficiently than current techniques. Secondly, the recentness
of the 2007 Rothberg paper demonstrates the current business need (of companies like ILOG) for faster and
more efficient optimization techniques for MIP. Swarm algorithm research is the perfect opportunity to
produce efficient solutions for this business need.
Since the introduction of particle swarm algorithms in the mid-1990s, several modifications have made
them particularly well-suited for applications in business optimization. The first of which, discussed in a 1997
paper by Kennedy and Eberhart, is the modification of swarm algorithms to directly search discrete spaces
(4104). The paper demonstrates a contrasting and complementary approach to that of Danna, Rothberg, and
La Pape, who attempted to relax discrete MIP problems into continuous spaces (2). Kennedy and Eberhart
specifically mention scheduling optimization as an application of the discrete particle swarm, and their paper
provides strong evidence for the case that swarm algorithms are applicable to business optimization problems
(4104, 4107).
Another modification to particle swarm algorithms makes them amenable to current advances in
multiprocessor technology. Namely, a 2004 paper by Schutte et al. explored parallelizing the swarm
algorithm. The opportunity for parallelization, as noted by the authors, stems partly from the decrease in
price and the increase in performance of multiprocessor computer clusters and network technology (2). The
two major advantages offered by parallelism, which were demonstrated in the paper, are the increase in
speed (solutions can be found in a matter of hours rather than a matter of days or weeks), and the increased
robustness of solutions that is a result from larger possible swarm “population” sizes on multiprocessor
systems (1-2, 11-12). Multiprocessor implementation of swarm optimization is made possible by the nature of
the algorithm, since sub-groups of the swarm can be evaluated concurrently across different processors (5).
Because not all optimization techniques can be so easily parallelized, the opportunity offers a distinct
advantage of swarm algorithms over other techniques.
Recent Thesis work in 2005 by Michael Kovacina at Case Western Reserve was aimed at developing a
generalized platform for testing various swarm optimization techniques. The SWarm Experimentation and
Evaluation Platform, called SWEEP, uses genetic algorithms to evolve intelligent swarm behavior (1). For his
Thesis project, Kovancina applied the SWEEP platform to tracking chemical clouds using unmanned air vehicles
(27). However, the SWEEP platform can be leveraged for furthering swarm algorithm research as it applies to
business optimization techniques for companies such as ILOG.
This review presents the creation and development of swarm algorithms (and the business
optimization problems they might address). Specifically, the need for swarm algorithm research for business
optimization has been demonstrated, along with several ways in which particle swarm optimization can
address the needs of companies such as ILOG. Reviewing the literature on swarm algorithms, the case for the
research’s necessity and applicability is apparent.
Research Plan
The proposed research can be completed in 10 weeks, and would ideally be conducted over a summer
session. What follows is a brief description of the research plan, as well as a schedule for the major sections of
the plan.
The first week would involve acquiring and setting up software necessary to perform the research.
According to its website, ILOG offers its CPLEX programming environment at a discounted rate to academic
institutions for the purposes of performing research. CPLEX is specialized for optimization programming, and
already contains implementations of other algorithms that can be used as a benchmark during testing. The
Java programming language will be used for the implementation of the particle swarm optimizer, particularly
for the purposes of controlling multithreading. The second week will consist of compiling particle swarm
research to design an implementation aimed specifically at optimizing MIP. The SWEEP platform will be used
to create the initial swarm models and to perform a preliminary evaluation of parameters. The following
three weeks will be dedicated to the implementation, testing and debugging of the swarm algorithm. Weeks
six and seven will be used to compare the performance of the swarm algorithm on single-processor
architectures to comparable MIP optimization algorithms in CPLEX. Weeks eight and nine will be used to
extend testing to multiprocessor architectures, measuring the efficiency of parallelism. The final week will be
needed to compile the results into a paper for publication. An overview of the research schedule can be found
in Table 1.
Weeks Deliverable(s)
1
Software and System Setup
Compile Swarm & MIP Research
2
Design Algorithm
3-5
Implementation and Testing of Swarm Algorithm
6-7
Single Processor Testing and Comparison
8-9
Multi-Processor Testing and Comparison
10
Compile Results into Paper
Table 1: Research Schedule
Personal Qualifications and Goals
Both educational and experiential factors qualify me to perform the proposed research. Being an
Associate Professor for the Computer Science department, you understand the scope and extent to which I
have been exposed to appropriate technical material in my academic career. Having personally been the
Professor of my Introduction to Programming and Artificial Intelligence classes, you know that I am dedicated
to producing not just adequate, but exceptional and timely work in completing assignments. From Artificial
Intelligence, you have examples of my work in which I implement algorithms related to the proposed research,
namely genetic and grid search algorithms for optimization. I have completed the Operating Systems class,
demonstrating my ability to produce the multithreaded programs and appropriate data-sharing mechanisms
necessary for this research.
Internship experience has qualified me to relate the theoretical aspects of optimization algorithms to
real-world business needs. Working for the Demand Planning department of McCormick, the international
spice company, I was exposed to a plethora of optimization issues and currently implemented solutions.
McCormick uses the SAP business software, utilizing the APO (Advanced Planner and Optimizer) component to
both forecast sales and optimize production and shipping. I created automated reports in Microsoft Excel
(using Visual Basic for Applications) which took data extracted from APO, calculated appropriate metrics, and
allowed business professionals to review and modify line schedules as needed. My work required an
understanding of the optimization problems that businesses face, and it provided experience that will
promote the success of the proposed research.
My motivation for conducting swarm algorithm research is twofold. First is my desire to further my
academic career and explore possible research topics for a Master’s Thesis. Secondly, business optimization is
a possible career path, and this research would provide considerable experience in the field. My personal
goals for the proposed research demonstrate that I have a stake in its success.
Outcomes and Audiences
The first and most straightforward anticipated outcome of the research is an advantage in both the
speed and robustness of swarm algorithms in comparison to conventional local search techniques employed
to solve MIP problems. Because these outcomes are addressed thoroughly in the Literature Review, the focus
here will be less apparent results of the research.
Some outcomes involve stakeholders in the research that exist outside of the university. Namely,
companies such as ILOG or SAP may be interested in the results of the research if swarm algorithms prove to
be an effective method for business optimization. Business interest is demonstrated by active research in
improving MIP optimization. Because ILOG and others are expending financial resources to conduct research,
one assumes that they expect to profit from more efficient algorithms. Continuing along this line of reasoning,
that companies like ILOG stand to profit from improved optimization techniques suggests that their clientele
also have a financial stake in the proposed research. Therefore, a chain of stakeholders, whose potential
involvement is described below, is revealed by closely examining the real-world applications of the research.
Given the potential to profit from swarm algorithm research, there are obvious intellectual property
issues that will be relevant. The Technology Transfer Office at Case Western Reserve has standardized
intellectual property policies listed on their website. According to university policy, intellectual property rights
for algorithms and software are held by the university unless agreements are made with industries that
sponsor research. Thus, the university as a whole is also a stakeholder in the proposed research.
Stakeholder Involvement
Considering the many audiences for particle swarm optimization, it is clear that there exists a complex
set of interactions between them. The involvement of several stake-holding parties is discussed here, and
Figure 1 is provided to visually describe the interactions between parties.
Case Western Reserve
Tech Transfer
Funding
SAP
Research
Proposal
Professor
Branicky
ILOG
Gary Doran
Figure 1: Stakeholder Interaction
Clientele
McCormick
I expect to perform much of the research independently with some input and feedback from you as a
faculty research advisor. Therefore, you need not worry that taking on the proposed research will burden
your schedule. Regarding funding, the budget for this project should be minimal. Between software licenses
and computing time on a multiprocessor cluster, total expenses should be on the order of several hundred
dollars. If even this small budget seems too expensive, I suggest leveraging your credibility to propose particle
swarm research to companies such as ILOG for funding.
ILOG, for example, states on its website that it “supports the academic community's efforts to expand
optimization.” One way in which ILOG demonstrates this support is through discounted software for academic
institutions. However, considering the potentially profitable nature of the research, ILOG may be willing to
sponsor particle swarm optimization for MIP. To maximize the effectiveness of the proposal, we must utilize
the Technology Transfer Office to facilitate a relationship with ILOG. The Technology Transfer Office has on
their website forms for “Sponsored Research Agreements,” which set guidelines for intellectual property
ownership for industry-sponsored research. Should patentable material be produced, the company (e.g. ILOG)
can pay for it and have the right to negotiate with the Case Western Reserve for ownership of rights to the
patent. Though it might seem that companies like ILOG (or their clientele) would be reluctant to sponsor
academic research, there clearly is an incentive for them to take a small financial risk for the potential
ownership of improved MIP algorithms.
Conclusion
The case for particle swarm algorithms for solving MIP and other business-related optimization
problems has been presented. As the literature demonstrates, there are real-world needs that can be
addressed through specialized implementations of swarm algorithms, which are both inherently more efficient
and multi-processor-friendly than alternative techniques. I have shown both a specific plan for the proposed
research, and outlined my qualifications for conducting it. Various stakeholders in the research have been
identified, namely Case Western Reserve, software companies like ILOG, and the businesses that they service.
Intellectual property issues have been addressed, suggesting that ILOG and others have an incentive to fund
the proposed research. Please permit me to conduct this promising research.
Bibliography
Danna, E., Rothberg, E., and Le Pape, C. (2005). Exploring relaxation induced neighborhoods to improve MIP
solutions. Mathematical Programming, 102(1), 71-91.
ILOG, an IBM Company. (2009). Academic Sales Program. In ILOG Academic Program: Support for
research, teaching and consulting. Retrieved April 1, 2009, from ILOG, an IBM Company Web site:
http://www.ilog.com/products/optimization/academic/.
Kennedy, J. and Eberhart, R. (1995). Particle swarm optimization. IEEE International Conference on Neural
Networks, 1995. Proceedings, 4, 1942-1948.
Kennedy, J. and Eberhart, R. (1997). A discrete binary version of the particle swarm algorithm. IEEE
International Conference on Systems, Man, and Cybernetics, 1997, 5, 4104-4108.
Kovacina, M. A. (2005). Swarm algorithms: simulation and generation. Unpublished master's thesis. Case
Western Reserve University, Cleveland, OH. Retrieved 13 Mar. 2009
http://dora.cwru.edu/msb/pubs/makMS.pdf.
Rothberg, E. (2007). An evolutionary algorithm for polishing Mixed Integer Programming Solutions. INFORMS
Journal On Computing, 19(4), 534-541.
Schutte, J. F., Reinbolt, J. A., Fregly, B. J., Haftka, R. T., and George, A. D. (2004). Parallel global optimization
with the particle swarm algorithm. International Journal for Numerical Methods in Engineering, 61(13),
2296-2315.
Technology Transfer Office. (2006). Forms. In Technology Transfer. Retrieved April 1, 2009, from
Case Western Reserve University Web site: http://ora.ra.cwru.edu/techtransfer/pages/forms.htm.