CS420 Project VI
Experimentation with Particle Swarm Optimization
Alexander Saites
25 April 2012
Alexander Saites
1
Introduction
In this project, I implemented a particle swarm optimization algorithm in Matlab. The program takes in
the number of epochs and particles, inertia, and the cognitive and social parameters as parameters and
uses them to simulate PSO. To test the implementation, I used the following two formulas with the
following definitions:
=
max +max /2
= (
− 20) +
−7
= (
+ 20) +
+7
Problem 1:
,
= 100 ∗ (1 −
)
Problem 2:
,
= 9 ∗ max(0,10 −
) + 10 ∗ 1 −
+ 70 ∗ 1 −
I then spent time playing with the parameters, running them on the two problems, attempting to draw
conclusions about how the parameters affect the performance of the algorithm. To measure this
performance, I graphed the total error of the population (defined as the sum of the distances between
each particle and the global maximum of the problem) along with the total number of particles to reach
a distance of 0.001 from the global maximum, each graphed over time. I also plotted out the particles
positions during each epoch to measure the performance qualitatively, and to have something to watch.
Below, I begin by showing the results for the provided values (100 epochs, 20 particles, inertia of .8,
cognition and social value of 2). After this, I present a discussion of how the different values affect
performance, and finally, I provide values that tend to result in the particles reaching the global
optimum for problem 2.
Graphs and Analysis
This first set of graphs show the results for each problem using the initial set of values:
Alexander Saites
2
Figure 1: Final state of the board
Figure 2: Error
Alexander Saites
3
For these parameters, none of the particles stabilize within .001 of the global maximum. Despite this,
the error graph shows that the particles stabilized nearby.
For problem two, the particles stabilize at the local maximum:
Figure 3: Board for Problem 2
Figure 4: The particles stabilized at the local maximum
Alexander Saites
4
The dip in the error followed by the steep rise is due to the particles moving first toward the global
maximum, but then moving to the local maximum.
After running these problems with the first set of parameters, I began experimenting with each of them
one at a time.
Number of Particles
I started by experimenting with the total number of particles to see what effect they had on the
problems. First, I tried using 200 particles:
Figure 5: Problem 1, with 200 particles
Alexander Saites
5
Figure 6: Error for Problem 1, 200 Particles
Figure 7: Problem 2, 200 Particles
Alexander Saites
6
Figure 8: Error for Problem 2, 200 Particles
With more particles, the noise in the graphs is reduced, but given the other values, increasing the
particle number seems to have little effect.
I next tried reducing the particle number to 5:
Figure 9: Problem 1 with 5 particles
Alexander Saites
7
Figure 10: Error for problem 1 with 5 particles
Figure 11: Problem 2 with 5 particles
Alexander Saites
8
Figure 12: Problem 2 with 5 Particles
Here we see the noise in the error increase, as there are only a few particles flying around. I suspected
the inertia of the particles had quite a bit to do with this, so I experimented with it next.
Inertia
I started by dropping the inertia to .1.
Figure 13: Problem 1 with .1 inertia
Alexander Saites
9
Figure 14: Error on problem 1 with .1 inertia
Figure 15: Particles reaching global maximum for Problem 1, .1 inertia
Here we see that all particles reach the global maximum after just 40 epochs. This makes sense, as the
lowered inertia allows the particles to reach the maximum and stay there without bouncing around so
Alexander Saites
10
much. On the other hand, the inertia forces a certain level of exploration, so I do not expect problem 2
to perform as well, since the particles are likely to reach the local maximum and stop.
Figure 16: Problem 2 with low inertia
Figure 17: Error for Problem 2 with low inertia
We can see from the error graph that this is precisely what happens.
Alexander Saites
11
Cognition and Social
Next, I tested the effect of changing the cognition and social parameter. My expectation is that
increasing the cognition above the social will cause particles to mostly stay still, especially if the
maximum velocity remains low (in my experiments, I used a value of 2). If social is zero, particles should
stay in the spot they begin. If cognition is zero, performance should be highly dependent upon particle
starting positions.
First, I lowered the cognition factor to zero while keeping the social factor at 2:
Figure 18: Final board using a low cognition
Alexander Saites
12
Figure 19: Error for low cognition
Figure 20: Low cognition with Problem 2
Alexander Saites
13
Figure 21: Error for Problem 2 with low cognition
Both of these problems show that with a cognition factor lower than the social factor, the particles
oscillate around their final resting space (in problem 2, they still tend to stay around the local
maximum).
Next, I tried a cognition factor of 2 with a social of 0.
Figure 22: Zero social for Problem 1
Alexander Saites
14
Figure 23: Zero social for Problem 2
Here we see that the particles stay where they begin, as there is no social pull causing them to move.
Next, I tried high cognition with a low, but nonzero, social factor:
Figure 24: Particles converged perfectly with a low, but nonzero, social factor
Alexander Saites
15
Figure 25: Particles fully converge at the end of the epochs in problem 1
Here we see that in Problem 1 all the particles were able to converge slowly on the global maximum.
This makes sense, as the particles are able to move gradually toward what everyone else is pointing at,
but will put more interest into their own best location. Since there is only one maximum and the
problem is simple, they slowly move toward the maximum and stay once they arrive.
In the second problem, the same pattern emerges, but they are unable to converge on the global
optimum, and instead prefer the local. However, they move toward this location in the same way the
particles in problem one move toward the global maximum.
Lastly, I tried lowering the cognition factor to .1 with the social factor at 4. In this scenario, the particles
move toward the socially optimal point, but this causes them to oscillate much more around the global
maximum. However, putting this emphasis on social helps encourage the particles in the second
problem to move toward the global maximum, performing far better than in the past:
Alexander Saites
16
Figure 26: With more focus on the social factor, the particles are more likely to converge on the global maximum
Figure 27: The error in this scenario is much better than before
Problem 2 Convergence
Finally, I focused on getting problem two to converge more consistently. By emphasizing the social
factor, as before, and lowering the inertia, I was able to get the second problem to converge with more
than 80% consistency.