Swarm behaviour and
traffic simulations
Using stigmergy to solve algorithmic problems,
predict and improve vehicle traffic
Overview (1)



Swarms in nature
Social insects and Stigmergy
Ant algorithms and application examples:




Foraging in ants
Using foraging behaviour to solve the TSP
Labour division among social insects
Mailmen using adaptive task allocation model
Overview (2)




Traffic simulation by cellular automata
Adopting the stigmergic process
Prediction of driver behaviour
Traffic infrastructure optimization



Drivers as ant agents
Signal lights as social insects
Other real world applications
Swarms in nature
What is a swarm ???
Pictures of swarms (1)
Pictures of swarms (2)
Pictures of swarms (3)
Characteristics of swarms

Aggregation of animals with similar size and often
similar orientation

Interaction of animals leads to new intelligent forms
of behaviour that are not inherited in the individuals

E.g. insects, birds, fish, bacteria
The main actor of the presentation
Social insects and stigmergy

Social insect societies are distributed systems with
highly structered social organization

They can accomplish complex tasks that far exceed
the individuals abilities

Here: focus on stigmergy as important means of
indirect communication paradigm
Stigmergy

Originally defined by Grassé:
Stimulation of workers by the performance they have
achieved

Method of indirect communication in a self-organizing
emergent system where its individual parts communicate with
each other by modifying their local environment

Here: Pheromones  diffusing chemical substance
Stigmergy example

Example: termites buildung nest
pillars with soil pellets

Stimulus  response

Autocatalytic process
Stigmergy behaviour of ants
Stigmergy behaviour in ants and their transfer to
computer algorithms:

Foraging and the TSP

Labour Division and adaptive task allocation
Foraging in ants

Foraging means searching for food

Ants manage to find the shortest path between their
nest and a food source

Achieved through trail-laying and trail–following
behaviour with pheromones
 Stigmergy
Foraging example

Two paths with
different lengths

Ants follow way with
most pheromones

Autocatalytic process
leads to „differential
length effect“
The Travelling Salesman Problem

Consists of a set of given cities

Goal is to visit all cities in a closed
loop of shortest length

Every city must be visited only
once!

E.g. 15 biggest cities of Germany
TSP represented by graph theory
TSP defined more generally by graph theory:
 Graphs consist of vertices V and edges E
 Cities are vertices, edges are connections between cities
 In the TSP each city is connected to each other!
 Each edge has a certain length

Example: 4 cities A,B,C,D
represented by vertices;
6 connnections with lengths
represented by edges
Ants solving the TSP



Artificial ants exploring the TSP graph
Artificial pheromones added by ants after completion of a
complete loop proportional to 1/length of route
Probabilistic transition rule for ant k to next city j:




City j visited?
Length of edge gives desirability measure ηij = 1/length
Amount of pheromones τij(t) on edge (i, j)
Evaporation of pheromones over time lets system forget bad
information
Labour division in ants

Fundamental in social insects: Division of
reproductive castes from worker castes

Further divisions:




subcastes of workers  specialists
subcastes of age and morphology
again dividing subcastes into behavioural castes
Plasticity: Workers switch tasks in response to
internal and external pertubations
Labour division model

Based on idea of response threshold

Stimulus exceeds individuals response threshold
 individual engages in task performance

Stimulus plays role of Stigmergy here
(can be pheromones, amount of encounters, …)
Extended labour division model
More realistic:
Extend previous model by threshold varying in time.
 if an individual performs a certain task its
threshold related to this task decreases
 the thresholds related to all other tasks, not
performed meanwhile, are increasing
Example: Express mail retrieval (1)

Group of mailmen has to
pick up letters in a city

Goal: Allocation of mailmen to appearing demands should be
optimal  realized with adaptive task allocation model

Each mailmen i reacts with a certain probability p to arising
demands, depending on:



response threshold Ө related to area j with demand
the distance d to the area with demand
the intensity s of the demand  stimulus
Example: Express mail retrieval (2)

Figure (a): demand of a certain area over time

At t = 2000 the mailman that is specialized on this area gets
removed

Figure (b): response threshold of another mailmen that is
reacting to the loss of the specialist
Traffic simulations
Why would one do that?

to predict drivers behaviour in order to adjust dynamic
traffic signs, or propose alternative routes in navigation
devices or radio

to improve traffic infrastructure and traffic light plans in
big, complicated traffic networks like cities
Cellular automata traffic models (1)

Two major approaches on traffic simulation:


Fluid-dynamical, with continuous traffic
Discretized cellular automata model
 macroscopic
 microscopic

Focus in presentation: discretized cellular automata models

Discretized:


Street is diveded into fixed sites, cars have integer velocities
Each site can be occupied by a car or can be empty
x meters
e.g. one lane traffic
car 1
site n
site n+1
car 2
site n +2
site n +3
site n + 4
Cellular automata traffic models (2)
Assuming a simple one lane model:

One update of the system consists of the following
steps performed with each car in parallel:




Acceleration or slowing down:
Depending on maximal speed and distance to next car
Randomization:
To contribute human behaviour and external influences
Car motion:
The advancment of sites, according to the speed
Model shows nontrivial and realistic behaviour
Adopting Stigmergy

Cars adopt the pheromone laying and sniffing behaviour of
ants

Leads to very realistic and dynamic system

Reduces communication between cars to local information
creation and retrieval  stigmergy

Computational costs can be reduced for collision checking

Still, information about traffic signs and other environmental
signals are non-local
 „cellular automata ant cars“
How cars behave like ants




Each car leaves and sniffs pheromones on the road
Pheromones fade over time, like with ants
Faster cars leave longer trails then slower cars (a)
Additional pheromone dropping necessary for




stopped cars (b)
quick deceleration
lane changing (c)
like using signal and
brake lights
(a)
(b)
(c)
Traffic prediction



Measurement devices like cameras determine the vehicles
entering an area
Implementing foraging behaviour of ants leads to realistic
system of interacting drivers
 drivers follow other drivers and try to escape jams
Various types of driver support:



Adjustment of dynamic traffic signs to avoid congestions
Knowledge of growth of traffic jams allows to give reasonable
redirections
Through use of foraging behaviour alternative routes can be given
more effectively, cars spread more
Optimizing traffic light plans


Microscopic traffic model by individual cars with
individual aims
1st Approach:



Cars as agents facilitate change of light plans by voting
Evolutionary process improves overall light plans
2nd Approach:


Groups of ligths at an intersection behave like social
insects
Adaptive task allocation is responsible for running plans
Cars voting for traffic ligths

Each car keeps track of two variables:


total driving time dtot
total waiting time wtot
 waiting measure:

Statistics give information about overall fitness
 overall waiting measure:

Cars that are stopped at a light vote for it

Lights with many votes are more probable to be mutated
 quicker adaption in the evolutionary process
Evolutionary process


Probabilistic mutations of traffic light timings
Mutation changes length of correlated light phases at
an intersection (e.g. N–S and E-W)
mutate
simulation 1
simulation 2
mutate
choose fittest
....
simulation 3


Simulation of each branch of a new generation
Survival of the fittest  with least waiting time of
cars
Traffic Simulation under SuRJE
SuRJE: Swarms Under R&J using Evolution
 Design environment to build, test and optimize traffic
scenarios
 Uses swarm based approach and
evolutionary adaption
 Features:




Enables to build multi-lane road maps
Car seeding areas define the
input and output of cars
Initial lights settings for starting point
Evolutionary adaption parameters can be set
Simulation example in SuRJE


Example network: „Looptown“ (a)
Figure (b) shows the decrease of overall waiting time
over generations
Traffic lights as social insects




Traffic lights are implemented as social insects
 all lights at an intersection form one insect
Insect has to perform one traffic light plan out of
several for its intersection
Traffic can be modeled by any microscopic traffic
simulator
Cars emit pheromones when crossing and waiting at
an intersection to provide stimulus
Use of adaptive task allocation

Adaptive task allocation model is applied to insects


light-plans are chosen through stimulus – response strategies
communication of intersections only by Stigmergy

Each insect/intersection has individual thresholds related to
available light-plan
Stimulus for light-plan j provided by cars:

Reinforcement learning is used to specialize intersections:



Threshold j of intersection i:
Learning coefficient:
Example of stimulus evaluation

4 way intersection with two light-plans:


N
1st plan gives priority to North-South leading lanes
2nd plan has priority for West-East leading lanes
W

Traffic situation:



E
Many cars are driving on West – East direction
Few cars on North-South direction
S
Initially plan 1 is driven:


Big amount of pheromones for W-E (many, waiting cars)
Small amount for N-S (few, almost not waiting cars)
 evaluation of stimulus yields higher value for 2nd plan!
Traffic simulation - Recapitulation

Traffic prediction:



Traffic light timing improvment by cars as agents:




Simulation into future by using swarm based approach
Giving the driver useful information about choosing the best route
Cars are modeled with swarm based approach
Improvement of traffic light plans through evolution
Good for simple traffic lights with static timing program
Dynamic traffic light adoption through social insects:



Cars are modeled by any microscopic traffic simulation
Intersections choose their light plan through adaptive task allocation
Good for traffic lights with sensors for counting cars
Examples for real world applications

Scheduling Problems, e.g. subway, train

Vehicle Routing, e.g. bus, taxi

Connection-oriented network routing, e.g. internet, TCP/IP

Connection-less network routing, e.g. bluetooth, infrared

Optical networks routing
Thank you for your attention!