Adaptive Robotics
COM2110
Autumn Semester 2008
Lecturer: Amanda Sharkey
1
So far …
Lect 1: what is a robot? Brief history of robotics
Early robots, Shakey and GOFAI, Behaviour-based
robotics
 Mechanisms and robot control (and biological
inspiration)
 Lect 2: Grey Walter, Brooks and Subsumption
Architecture.
 Lect 3: Adaptation and learning
 Lect 4: Artificial Neural Nets and Learning
 Lect 5: Evolutionary Robotics


2
“Robots in the news”
Honda – new wearable assisted walking gadget
Designed to support bodyweight, reduce stress on the knees and
help people get up steps and stay in crouching positions.
To be used by workers in auto factories
To be tested next month with assembly-line workers
Based on technology developed for their Asimo robot
3
Phoenix: NASA Martian probe
Has come to the end of its mission
 Not enough light to recharge batteries, and winter
 Has been on Mars for 5 months – sent back 25000
images

4
 Due by Monday 17th Nov at 11 am

Write an essay (1500-2500 words) on one of the following
topics. You should use the lectures as a starting point, but also
research the topic yourself. Plan your answer. Include a
reference section, with the references cited in full.
1. Identify the main characteristics of Behaviour-based robotics,
and contrast the approach to that of “Good old-fashioned AI”.
2. To what extent did Grey Walter’s robots, Elsie and Elmer, differ
from robots that preceded, or followed them.
3. Explain how the concepts of “emergence” and “embodiment”
are related to recent developments in robotics and artificial
intelligence.
5
Collective Robotics
Swarm Robotics
6
Collective robotics
 Why invest in collections of robots, why not build
a reliable individual robot?
-
Task difficult (or impossible) for one robot
Can be performed better by many
Redundancy – task more likely to be completed
Simplicity – many cheaper robots instead of one
expensive one.
7
What kinds of collections?
Possibilities range from
- remote controlled robots
- centrally controlled robots
- completely autonomous robots
8
Cao, Fukunaga and Kahn (1997) Cooperative mobile robotics: antecedents and directions.
Autonomous Robots, 4,1, 7-27.
9
 Advantages of robot collectives shown in
Environmental exploration
Materials transport
Coordinated sensing – collective cooperates to
provide maximal sensor coverage of moving target.
Robot soccer
Search and Rescue
10
Swarm Robotics
 Taking a swarm intelligence approach to robotics
11
Swarm intelligence
Swarm intelligence is “any attempt to design
algorithms or distributed problem-solving devices
inspired by the collective behaviour of social
insect colonies and other animal societies”
Bonabeau, Dorigo and Theraulaz (1999)
12
13
Natural swarms
 Decentralised – no-one in control
 Individuals are simple and autonomous
 Local communication and control
 Cooperative behaviours emerge through self-
organisation
e.g. repairing damage to nest, foraging for food,
caring for brood
14
15

Self-organisation
Organisation increases in complexity, without external
guidance
Self-organising systems often display emergent properties
“self-organisation is a set of dynamical mechanisms whereby structures appear
at the global level of a system from interactions among its lower-level
components. The rules specifying the interactions among the system’s constituent
units are executed on the basis of purely local information, without reference to
the global pattern, which is an emergent property of the system rather than a
property imposed upon the system by an external ordering influence”
(Bonabeau, Dorigo and Theraulaz, 1999)
16
 Emergence
An emergent property, e.g. pattern formation, from
more basic constituents
An emergent behaviour can appear as a result of the
interaction of components of the system
E.g. flocking, or organisation of ant colony
17
 Real life example of self-organised behaviour in
humans
Emergence of paths across grassy area
Most popular paths are reinforced
 Counter –example e.g. a team of carpenters
building a house….not self-organised.
18
Swarm robotics
Inspired by self-organisation of social insects
 Using local methods of control and communication

Local control: autonomous operation
Local communication: avoids bottlenecks
Scalable – new robots can be added, or fail without need for
recalibration
Simplicity – cheap, expendable robots
Self-organisation
 Decentralisation

19
Disadvantages of centralised
control and communication.
 Central control: failure of controller implies failure
of whole system
 Robot to robot communication becomes very
complex as number of robots increases.
 Communication bottlenecks
 Adding new robots means changing the
communication and control system
20
Applications of swarm approach
Some tasks are particularly suited to group of expendable
simple robots
e.g. - cleaning up toxic waste
- exploring an unknown planet
- pushing large objects
- surveillance and other military applications
21
What issues are investigated?

Weak AI questions:
E.g. how can complex behaviour, such as cooperation, emerge as a result of
interactions between simple agents and their environment?
– Biological modelling – better understanding of social insects for
example.
- Biological inspiration – emulating behaviour and capabilities of
biological systems
22
Cooperation and communication
 Examples of communication in cooperative
systems:
 Increasing sophistication….
Bacteria
Ants
Wolves
Non-human primates
Humans
23
Bacteria
 Live in colonies
 Explicit chemical signals mediate their ability to
cooperate.
 E.g. Mycobacteria assemble into multicellular
structures known as fruiting bodies.
 Bacteria emit and react to chemical signals
24
Ants
 Also termites, bees and wasps
 Display cooperative behaviour
e.g. pheromone trails to food source
Chance variations that result in shorter trail are
reinforced at faster rate.
Can find optimal shortest path
Stigmergic communication.
25
Wolves
 Territory marking through repeated urination on
objects on periphery of territory
 Also more sophisticated communication directed
at particular individuals
Specific postures and vocalisations
26
Non-human primates
 Sophisticated cooperative behaviour
Higher primates can represent the internal goals,
plans, dispositions and intentions of others, and
to construct collaborative plans jointly through
acting socially.
27
Humans
 Many forms of communications – including
written and spoken language
 Many forms of cooperation, from basic altruism
to cooperative relationships where we exchange
resources for mutual benefit
28
 Focus of interest here:
 Emergent cooperation
e.g. social insects: ants, bees, wasps, termites
Stigmergic communication: one of the
mechanisms that underlies cooperation.
29
Swarm robotics
Biologically inspired by social insects
- emergent complex behaviour from simple agents
 Swarm Intelligence Principles:
Autonomous control
Simple agents (debateable – swarms of helicopters?)
Expendable, fast and flexible responses
Local communication
Scalable
Decentralised

Use and exploration of stigmergy
30
Mystery: cooperative behaviour
when insects seem to work alone
31
32
33
individual insect responds to changes in
environment created by itself or others
 Grassé (1959) – stigmergy
- Indirect social interaction via the environment
-
34
E.g. Termite nest building
 Building arches
 Termites make mudballs, which they deposit at random.
Chemical trace added to each ball
 Termites prefer to drop mudballs where trace is
strongest.
 Columns begin to form
 Deposit more on side nearest to next column –
eventually leads to formation of arch.

35
Example paper: Holland and
Melhuish (1999)
 Holland, O., and Melhuish, C., (1999) Stigmergy,
self-organisation and sorting in collective
robotics. Artificial Life, 5, 173-202.
36
37
 Example of ant brood sorting
“The eggs are arranged in a pile next to a pile of larvae
and a further pile of cocoons, or else the three
categories are placed in entirely different parts of the
nest…if you tip the contents of a nest out onto a
surface, very rapidly the workers will gather the
brood into a place of shelter and then sort it into a
different pile as before (Deneubourg, et al, 1991)
38

Franks and Sendova-Franks (1992)
Brood sorting of Leptothorax unifasciatus
- brood items sorted into concentric rings of progressively more
widely spaced brood items at different stages of development.
39
Use of simulations
Deneubourg et al (1991) “The dynamics of collective
sorting: Robot-like ants and ant-like robots”
Showed agents could use stigmergy to cluster scattered objects of a
single type, and to sort objects of two different types
For sorting – agents needed short-term memory to sense local density of
different types of brood items and to know the type of brood item they
were carrying.
But – a simpler solution can be found with physical agents – greater
exploitation of real world physics.
40
41
Holland and Melhuish experiments:
 Small U-bot robots, with infrared sensors, and gripper
designed to sense, grip, retain, and release frisbees.
 When robot moves forward, frisbee remains in gripper
 When robot reverses, frisbee left behind, unless pin
extended to keep it in place
 When 2 or more frisbees pushed into, this triggers
microswitch in gripper – not triggered when pushing or
bumping into 1 frisbee.

42
43
44
Exp 1: how many U-bots in arena without too many
collisions
 Exp 2: Simple rule set
Rule 1: if (gripper pressed and object ahead) then make
random turn away from object -> ie turn away from
boundary
Rule 2: if (gripper pressed and no object ahead) then
reverse small distance (dropping the frisbee) and make
random turn left or right -> ie has encountered another
frisbee.
Rule 3: go forward

45
 44 frisbees placed across the arena
 10 robots released.
 Frisbees gradually collected in small clusters –
after 8 hours 25 mins, a cluster of 40 frisbees
formed.
 Frisbees taken from intermediate clusters if
struck at an angle without triggering gripper
46
47





Experiment 5: sorting and pull-back algorithm
Plain yellow frisbees and black and white ring frisbees
Pin-dropping mechanism applied to plains
Rule 1: if (gripper pressed and object ahead) then make random
turn away from object
Rule 2: if (gripper pressed and no object ahead) then
If plain lower pin and reverse for pullback distance
raise pin
reverse small distance (dropping frisbee)
make random turn left or right
Rule 3: go forward
48
- now if robot is pushing a plain frisbee and hits another,
or if not pushing frisbee and collides with another plain
in a cluster, the plain will be dragged backwards and
dropped away from contact point.
 Result (after 7h 35 m): central core of 17 ring frisbees
with 11 plains and 4 rings round outside.
 I.e. annular sorting, based on simple mechanism
 - Example of seemingly complex behaviour (sorting)
emerging from the application of simple rules.

49
50
51
What do these experiments
show?
Apparently co-operative behaviour, with no central
control, and no direct communication.
 Segregation and crude annular sorting can be achieved
by system of simple (reactive) mobile robots
 - robots can only sense the type of object they are
carrying
 - they have no capacity for spatial orientation or
memory

52
 Some elements of these mechanisms found in
social insects
 E.g. Leptothorax building behaviour: possible
use of increased resistance to pushing a building
block forward against other building blocks as a
cue to drop it
 “the ants drop their granule if they meet sufficient
resistance” (Franks et al, 1992)
53
Also mechanism like pull-back mechanism
“workers individually carry granules into the nest.
They walk head
first towards the cluster of their nest-mates who are already
installed in the nest, forming a fairly tight group. After coming
close to the group of ants, the builder then turns through 180
degrees to face outwards from the nest. The worker then
actively pushes the granule it is carrying into other granules
already in the nest, or after a short time, if no other granules are
encountered, it simply drops its load” (Franks et al, 1992)
54
Summary of Holland and Melhuish paper:
 A simpler solution obtained in the robotic experiments
where the physics of the environment can be exploited,
than in abstract computer simulations
 Simple behavioural rule set – no capacity for spatial
orientation or memory, but robots able to achieve
effective clustering and sorting
 Example of stigmergy – indirect communication via the
environment.

55
 Sorting and clustering accomplished here by
robots with no memory, and no understanding of
their task.
 Does this mean that ants also have no memory
or understanding of their tasks?
56


Kube, C.R. and Bonabeau, E. (2000) Cooperative transport by
ants and robots. Robotics and Autonomous Systems, 30, 85101.
Current interest in robotics is result of
Relative failure of classical AI program. Swarm-based robotics, and idea
that group of robots can perform tasks without explicit representations of
environment and other robots
Mobile robots becoming cheaper and more efficient
Artificial life and emphasis on emergent behaviour – increasing
awareness of biological systems
57
 “As the reader will perhaps be disappointed by the simplicity of
the tasks performed by state-of-the-art robotic systems such as
the one presented in this paper, let us remind her or him that it is
in the perspective of miniaturisation that swarm-based robotics
becomes meaningful … understanding the nature of coordination
in groups of simple agents is a first step towards implementing
useful multirobot systems”
(Kube and Bonabeau, 2000)
58
 Cooperative transport in robots, and cooperative
prey retrieval in social insects
 E.g Moffett (1988) a group of 100 ants
Pheidologeton diversus could transport a 10 cm
earthworm weight 1.92g (more than 5000 x 0.3
mg minor worker)
 Single ants carry burdens at most 5 times their
body weight
59
60
Questions about cooperative
prey retrieval in social insects
Is there an advantage to group transport over solitary
transport?
 When and how does an ant know it cannot carry an
item alone?
 How are nest mates recruited?
 How do several ants cooperate and coordinate their
actions to transport an item?
 How do ants ensure the right number of ants help?
 How does a group of ants handle deadlocks?

61
 Group vs solitary transport
Moffet (1988) transport efficiency per ant
(product of burden weight by transport velocity
divided by no. of carriers) increases with group
size up to a maximum for groups of 8-10, and
then declines
Switching from solitary transport
resistance to transport
62
 Recruitment of nestmates
Holldobler et al (1978) African weave ant
Aphaenogaster species
Short range recruitment – releasing poison gland
secretion in the air when prey discovered. Ants
recruited from 2m distance
Long range recruitment – chemical trail of poison
gland recruitment laid from prey to nest.
63
Coordination in collective transport – not well
understood. Movement of one ant engaged in group
transport modifies the stimuli perceived by other group
members (stigmergy)
 Number of ants – an increasing feature of how difficult
(weight and resistance) it is to carry the prey
 Deadlock and stagnation recovery: ants show realigning
and repositioning behaviours

64
 Robot task: cooperative box pushing
Previous versions – centralised planning and
conflict resolution, with explicit communication
between robots
Kube and Zhang (1994) directed box pushing by
robots
Applied in simulation first, then robots
65
Non-directed box pushing

Physical robots: 2 behaviours
AVOID (left and right obstacle sensor mapped to left and right
wheel motors)
GOAL (left and right sensors mapped to right and left wheel
motors causing robots to turn towards brightly lit box)
Controllers allowed robots to locate box, converge and push
it
But stagnation could arise
How do ants recover from stagnation?
66
 Cooperative prey retrieval
 Sudd (1960) strategies to combat stagnation
observed in ants cooperatively retrieving prey
Realignment of body without releasing grasp
If that fails, grasp released, and ants reposition
Same strategies used for robot box pushing
67

Comparison of strategies
1.
2.
3.
4.
No stagnation recovery
Realignment only
Reposition only
Realignment and reposition
Performance improved with strategies
For small group strategy (2) best, for large group (3) is
better, and (4) is best.
68
Directed box-pushing
 Now 3 phases
Finding the box
Moving towards the box
If oriented with respect to goal, pushing box
Box detection simplified by placing bright light on
box
Goal detection simplified by shining spotlight
69
70
Directed box-pushing
 Phase 1: robots execute FIND-BOX and MOVE-TO-BOX
 Phase 2: robots incorrectly positioned for pushing move counter
clockwise round perimeter
caused by cycling through FIND-BOX, MOVE-TO-BOX, and
PUSH-TO-GOAL when contact is made. ?SEE-GOAL indicates
robot on wrong side for pushing, and REPOSITION behaviour
until empty position found.
Phase 3: push to goal – robots continuously monitor ?SEE-GOAL.
If robot cannot see goal it will reposition.

71
 Varying the number of robots
 More robots = more interference
72
What do these experiments
show?




Coordinated group effort is possible without use of direct communication or
robot differentiation
- ants not always efficient – eg ants can take 10 minutes to begin moving
object.
Model makes testable predictions about stagnation recovery mechanism to be
expected depending on ecological conditions and prey size
E.g. adding mechanisms for stagnation recovery increases retrieval time, and
probability of success
Where little competition, should find more stagnation recovery
mechanisms
Where strong competition, should find less stagnation recovery
73
Research questions in Swarm
Robotics
Self-organised methods for task allocation to ensure
that enough robots are allocated to a task
Collective decision making
Communication – local methods to detect when
needs of group have changed
Control and coordination of heterogeneous groups
Incorporation of some learning and recognition –
e.g. of landmarks
74
What have we looked at in this
lecture?

Idea of collective robotics
Possible reasons for using several robots
Swarm intelligence and swarm robotics
Self-organisation and emergence
Possible applications

Cooperation and communication
Forms of communication
– Stigmergic indirect communication

Example papers:
Holland and Melhuish (1999) and sorting
Kube and Bonabeau (2000) and box pushing
75