Advanced Mobile Robotic
The City College of the City University
of New York
Introduction to AI Robotics
Instructor: Prof. Jizhong Xiao
Presenters: Diana Acevedo
Sefton Bennett
Clara Nieto-Wire
Jorge O. Peche
Marios Timotheou
March 13, 2006
Reactive Paradigm



Biological foundations of the reaction
paradigm
The reactive paradigm
Designing a reactive implementation
Biological foundations of
the reaction paradigm
Chapter 3
Why??????


Animals  Open World
AGENT  Self-Contained and Independent
Interacts with the world to
make changes or sense what is
happening.
Humans-Animals-Robots = AGENT
Marr’s Computational Theory
Level 1
What are we trying to represent?
Level 2
How Represent it?
Level 3
How is it implemented?
Computational Theory Levels
1.
2.
3.
Existent proof of what can/should be done.
Agents share commonality of purpose or
functionality.
“What”=Inputs, outputs and transformations.
Creating Flow Charts of black boxes. Agents
exhibit common processes.
How to implement the processes. Agents
may have little or not commonality in their
implementations.
BEHAVIOR
SENSORY
INPUT
BEHAVIOR
PATTERN OF
MOTOR ACTIONS
Mapping of sensory inputs to a pattern of
motor actions which then are used to
achieve a task.
Pattern  Actions and sequences that are
always the same.
Types of Behavior

Reflexive
: Stimulus-Response.

Reactive
: Learn-Consolidate

Conscious : Deliberative
Reactive Paradigm  Reflexive Behavior
Reflexive Behavior
NO CONGNITION

Reflexes
: TRESPONSE = TSTIMULUS

Taxes
: Response  Movement

Fixed-Action Patterns:
TRESPONSE > TSTIMULUS
…Ways to Acquire Behaviors




Innate
Sequence of innate
behaviors
Innate with
memory
Learn
Innate Releaser Mechanisms
IRM  Activates
the Behavior
SENSORY
INPUT


BEHAVIOR
PATTERN OF
MOTOR ACTIONS
Releaser: Control Signal
(internal or external).
Behaviors can be implicitly chained
together by their releasers.
Concurrent Behaviors



Equilibrium 
Behaviors balance each
other.
Dominance of one 
Winner takes all.
Cancellation 
Behaviors cancel each
other.
PERCEPTION
WORLD
Acts &
Modifies
World
COGNITIVE
ACTIVITY

Samples, Finds
Potential Actions
Directs what
to look for
Cognitive activities
(Feedback and Feed forward control)
• Planning
• Reacting

Two uses of perception
• Release a behavior
• Guide a behavior
PERCEPTION
OF
ENVIROMENT
Gibson’s Ecological Approach
 Affordances : Perceivable potentialities
of the environment for an action.
• Directly Perceivable.
• Sensing does not need/require memory or
interpretation (optic flow).
 Structural models attempt to describe an
object in terms of physical components.
Neisser: Two perceptual systems


Direct Perception:
Gibsonian or ecological
track of the brain.
Structures low in the
brain that evolved
earlier. (Affordances)
Recognition: More
recent perceptual
track in the brain.
Schema Theory
Schema  Knowledge of how to act
and/or perceive.
Computational process to
accomplish the activity.
• schema instantation is specific to a
situation, equivalent to an instance in
OOP.
The Reactive Paradigm
Chapter 4
Objectives

Define what the reactive paradigm is in terms of :
• i) the three primitives SENSE, PLAN, and ACT
• ii) sensing organization.


List and discuss the characteristics and connotations of a reactive
robotic system.
Describe the two dominant methods for combining behaviors in a
reactive architecture.
• i) subsumption
• ii) potential field summation.

Evaluate these two reactive architectures in terms of:
•
•
•
•
i) support of modularity
ii) niche targetability
iii) ease of portability to other domains
iv) robustness

Be able to program a behavior using a p.field methodology.

Be able to construct a new p.field from primitive p.fields, and sum
the p.fields to generate and emergent behavior.
Hierarchical Paradigm
is characterized by having a
horizontal decomposition.
Reactive Paradigm
is characterized by having a
vertical decomposition.
If anything happens to an advanced
behavior, the lower behaviors would
still operate.

All reactive systems are composed of
behaviors.

Attributes of Reactive Paradigm

Behaviors.

SENSE-ACT organization.

Behavior specific (local) sensing.
Attributes of Reactive Paradigm
BEHAVIOR
(Ethological def.) a direct mapping of sensory inputs to a pattern of
motor actions that are then used to achieve a task.
(Mathematically) a transfer function , transforming sensory inputs into
actuator commands.
Schema: a way of expressing the basic unit of activity. It is consists of the knowledge of how to
act or perceive, and the algorithm by which it uses to accomplish the activity.
(like a generic template for doing some activity).
Attributes of Reactive Paradigm
SENSE-ACT Organization (S-A)
The SENSE and ACT
components are tightly
coupled into behaviors
and all robotic
activities emerge as
the result of these
behaviors operating
either in sequence or
concurrently. (NO
PLAN component).
The S-A organization does not specify how the
behaviors are coordinated.
What happens when multiple
behaviors are active
simultaneously?
Attributes of Reactive Paradigm
Behavior-Specific (local) Sensing
•Sensing is local
•Sensors can be
shared
•Sensors can be
fused locally by a
behavior
Behaviors favor the use of affordances (direct perceptions). The behavior does not rely on any
central representation built up from all sensors. The sensing portion of the behavior is
nearly instantaneous and action is very rapid.
Connotations of Reactive Behaviors:

A reactive robotic system executes rapidly.
Behaviors can be implemented directly in hardware as
circuits or with low computational complexity.

Reactive robotic systems have no memory
Reactive behaviors are limited to pure “stimulus-response
reflexes”
BEHAVIORS ARE CONTROLLED BY WHAT IS HAPPENING IN THE WORLD,
DUPLICATING THE SPIRIT OF INNATE RELEASING MECHANISMS, rather
than by the program storing and remembering what the robot did last.
Characteristics of Reactive Behaviors:

Robots are situated agents (integral part of the world)
operating in an ecological niche (goals of robot, world where it
operates, and how it perceives the world).


Behaviors serve as the building blocks for robotic
actions. And the overall behavior of the robot is
emergent.
Only local, behavior-specific sensing is permitted.
(Any sensing which does require representation is
expressed in ego-centric coordinates).

These systems inherently follow good software design
principles.

Animals models of behavior are often cited as a basis
for these systems or a particular behavior.
Advantages of programming by
behavior:
Behaviors support good software
engineering principles through:



decomposition
modularity and
incremental testing.
Representative Architectures

Potential Field Methodologies

Subsumption Architecture
Provide mechanisms for:


Determine what happens when multiple behaviors are
active at the same time
Triggering behaviors
Subsumption Architecture
Subsumption refers to how behaviors are combined.
In many applications behaviors are embedded directly in the
hardware or on small microprocessors.
Subsumption Philosophy




Modules should be grouped into
layers of competence
Modules in a higher lever can
override or subsume behaviors in the
next lower level
• Suppression: substitute input
going to a module
• Inhibit: turn off output from a
module
No internal state in the sense of a
local, persistent representation
similar to a world model.
Architecture should be taskable:
accomplished by a higher level
turning on/off lower layers
EXAMPLE
Subsumption Architecture
•Level 0 Avoid collisions
Vector representation of sonar readings
Level 0 recast as primitive
behaviors
Robot-centric
polar plot of 8
sonar range
readings
Polar plot unrolled into a plot
Subsumption Architecture
EXAMPLE
LEVEL 1: WANDER
Level 1: Wander
Level 0 : Avoid Collisions
•Inhibition
•Suppression
Level 1 recast as primitive behaviors
EXAMPLE
LEVEL 2: Follow Corridors
Level 2: Follow Corridors
Level 1: Wander
Level 0 : Avoid Collisions
Subsumption Architecture
Subsumption Summary






Subsumption groups schema-like modules into layers of
competence, or abstract behaviors.
Higher layers may subsume and inhibit behaviors in lower
layers, but behaviors in lower layers are never rewritten or
replaced.
The design of layers and component behaviors is difficult. It
is more of an art than a science. (True for all reactive
architectures)
Behaviors are released by the presence of stimulus in the
environment. (No PLAN component)
The releaser is almost always the percept for guiding the
motor schema.
Perception is ego-centric and distributed. (sonar plot was
relative to the robot, available to any behavior which
needed it, and the output from perceptual schemas can be
shared with other layers)
Potential Fields:
Ron Arkin
From http://www.cc.gatech.edu/aimosaic/faculty/arkin
From http://www.cc.gatech.edu/aimosaic/robot-lab/MRLhome.html
Potential Fields Philosophy



The motor schema (what the robot should do) component of a
behavior can be expressed with a potential field’s methodology
• A potential field can be a “primitive” or constructed from
primitives which are summed together
• The output of behaviors are combined using vector summation
From each behavior, the robot “feels” a vector or force
• Magnitude = force, strength of stimulus, or velocity
• Direction
We visualize the “force” as a field, where every point in space
represents the vector that it would feel if it were at that point
Potential fields are continuous
- Even with a small element at a point can be associated with a vector
How an obstacle will exert a field on the robot and make it run
away?
If the robot is close to the obstacle it is inside the potential field and will feel
a force that makes it want to face directly away from the obstacle and move
away.
If the robot is not within that range then it just sits there because there is no
force on it.
Example: Run Away via Repulsion
5 Primitive Potential Fields
a. Uniform
b. Perpendicular
c. Attraction
d. Repulsion
e. Tangential
Common fields in behaviors

Uniform
• Move in a particular direction, corridor following

Repulsion
• Runaway (obstacle avoidance)

Attraction
• Move to goal

Perpendicular
• Corridor following

Tangential
• Move through door, docking (in combination with other
fields)

random
• do you think this is a potential field?
Magnitude Profiles
- The length of the arrows gets smaller closer to the object and the way
the magnitude of the vectors in the field change is called the magnitude
profile.
- Magnitude profiles solve the problem of constant magnitude, which
allows the designer to represent reflexivity (response is proportional to
the strength of the stimulus).
- The main motivation for magnitude profiles is to fine-tune the behavior.
Linear Drop off
If a robot is far away from the object it will turn and move quickly towards
it, then slows up to keep from hitting the object.
Mathematically, the rate at which the magnitude of the vectors drops off
can be plotted as a straight line.
Exponential Drop off
To have the robot react more the closer it is to the object (constant
magnitude). Stronger reaction but more of a taper.
The drop off is proportional to the square of the distance for every unit of
distance away from the object, the force on the robot drops in half.
Potential Fields and Perceptions
 Potential Fields are ego-centric because robots perception is ego-centric.
 This makes programming easy. The visualization of the entire field may
appear to indicate that the robot and the objects are in a fixed, absolute
coordinate system but they are not.
The robot computes the effect of the potential field, usually as a straight line,
at every update, with no memory of where it was previously or where the
robot has moved.
Combining Fields for Emergent Behavior
goal
obstacle
obstacle
If robot were dropped anywhere on this grid,
it would want to move to goal and avoid obstacle:
Behavior 1: MOVE2GOAL(attraction field)
Behavior 2: RUNAWAY (repulsive field)
The output of each independent behavior is a vector,
the 2 vectors are summed to produce emergent behavior
Fields and Their Combination
Note: inrepulsive
this example,
can
the
Note: In this example,
field onlyrobot
extends
forsense
2 meters;
meters
away
the robot runsgoal
awayfrom
only if10
obstacle
within
2 meters
Path Taken
Robot only feels
vectors for this
point when it (if)
reaches that point



If robot started at this location, it would take the
following path
It would only “feel” the vector for the location, then
move accordingly, “feel” the next vector, move, etc.
Pfield visualization allows us to see the vectors at all
points, but robot never computes the “field of vectors”
Update rates, Holonomicity and Local Minima
1. Distance between updates are different (length of the arrows)
Note between t3 and t4, the robot actually goes further without turning
and has to turn back as to go to goal. A smoother path would have a
faster update.
2. Robots turn in any direction but have to stop and errors develop due
to the contact between the wheels and the surface.
3. The vectors only have head and no body, this means the magnitude
is 0.0 and that if the robot reaches to that spot it will stop and not move.
Example of using one behavior per sensor
How does the robot see a wall without
reasoning or intermediate representations?

Perceptual schema “connects the dots”,
returns relative orientation
PS:
Find-wall
Sonars
orientation
MS: Perp.
S
MS: Uniform
Sum all 8 ranges internally
Return the single largest distance and direction
OK, But why isn’t that a representation of
a wall?

It’s not really reasoning that it’s a wall, rather it
is reacting to the stimulus which happens to be
smoothed (common in neighboring neurons)
Pfields compared to subsumption
Level 0: Runaway
Note: multiple instances of
a behavior vs. 1:
Could just have 1
Instance of RUN AWAY,
Which picks nearest reading;
Doesn’t matter, but this
Allows addition of another
Sonar without changing the
RUN AWAY behavior
Level 1: Wander
Wander is
Uniform, but
Changes direction
aperiodically
Level 2: Follow Corridor
Follow-corridor
Example: follow-corridor or followsidewalk
Perpendicular
Uniform
Note use of
Magnitude profiles:
Perpendicular decreases
Combined
Pfields


Advantages
• Easy to visualize
• Easy to build up software libraries
• Fields can be parameterized
• Combination mechanism is fixed, tweaked with
gains
Disadvantages
• Local minima problem (sum to magnitude=0)
• Jerky motion
Example: Docking Behavior
Orientation, ratio of
pixel counts 
tangent vector
Total count 
attraction vector
•Arkin and Murphy, 1990, Questa,
Grossmann, Sandini, 1995, Tse and
Luo, 1998, Vandorpe, Xu, Van
Brussel, 1995. Roth, Schilling, 1998,
Santos-Victor, Sandini, 1997
Pfields Summary





Reactive Paradigm: SA, sensing is local
• Solves the Open World problem by emulating
biology
• Eliminates the frame problem by not using any
global or persistent representation
• Perception is direct, ego-centric, and
distributed
Two architectural styles are: subsumption and
pfields
Behaviors in pfield methodologies are a tight
coupling of sensing to acting; modules are
mapped to schemas conceptually
Potential fields and subsumption are logically
equivalent but different implementations
Pfield problems include
• local minima (ways around this)
• jerky motion
Designing a Reactive
Implementation
Chapter 5
Objective



Use schema theory to design and program
behavior using object-oriented
programming principle.
Design a complete behavioral system,
including coordinating and controlling
multiple concurrent behaviors.
Describe the two methods for assembling
primitive behaviors into abstract
behaviors: finite state machines and
scripts.
Schema and Behavior Classes
Schema
Coordinate Control Progarm
Peceptual Schema
Motor Schema
Behavior
Behavior
Coordinate Control Progarm
Peceptual Schema
Motor Schema
Behavior
Emergent Behavior



The reactive movement is characterized by
robotics running a very small set of behavior
which are combined internally to produce an
overall emergent behavior.
The key components of a reactive
architecture are the behavior plus the
mechanism for merging the output of the
concurrent behaviors.
Series of Behaviors are operating in
recognizable sequence such as Pick Up the
Trash Robot
Behavior as Object in OOP



Schema is well suited for transferring
theoretical concept in Object-Oriented
Programming (OOP).
Schema theory is used as a bridge between
concept in biological intelligence and robotic
(reactivity and affordance).
A schema as a programming object would be
a class: Coordinate Control Program,
Behavior, Motor Schema, and Perceptual
Schema.
Behavior as Object in OOP



Perceptual Schema takes sensor input and
transforms it into data structure called a
percept; it is linked to the sensor.
Motor Schema transform the percept into
a vector or other form of representing an
action; it is linked to the robot’s actuators
Behaviors are composed of at least one
Perceptual Schema and one Motor
Schema, these schemas act as the
methods for Behavior classes.
Primitive and Abstract Behaviors





Primitive behavior is composed of only one
perceptual schema and one motor schema,
and there is no need to have any
coordinated control program.
Monolithic
Simple mapping
Simple method (no multiple methods or objects.
Abstract behavior is composed of multiples
perceptual schema and motor schema, and
there are farther removed from the sensor
and actuator than the primitive behavior.
Primitive and Abstract Behaviors
Follow Corridor
Vector Sumation
Motor Schema 1:
Perpendicular
Motor Schema 2:
Uniform
Perceptual Schema 1:
Find-walls
Follow Corridor
Vector Summation of Behaviors
Behavior 1: follow_wall
lelt
Behavior 2: follow_wall
right
Motor Schema: follow-wall
(which-side)
Perceptual Schema: find-wall
(polar plot)
Unmanned Robotics Competition




Describe the task: specify what the robot
has to do to be successful.
Describe the robot: determine the basic
physical abilities of the robot and any
limitation.
Describe the environment: determine the
situatedness of the robot and identifies
perceptual opportunity for the behavior
(Perceptual Schema and its functionality).
Describe how the robot should act in
response to its environment:Identify the
set of one or more candidate primitive
behavior
Unmanned Robotics Competition



Refine each behavior: concentrate on the
design of each individual behavior (reactive
organization system and activities).
Test each behavior independently.
Test with other behavior: Perform
integration testing, where the behaviors
are combined. This includes testing in
actual environment.
Step in designing a reactive
behavior system
Describe the task
Describe the robot
Specification
& Analysis:
ecological
niche
Describe the environment
Describe how the robot should
act in response to its environment
Design
Implement & refine each behavior
Test each behavior independently
Test behaviors together
Implementation
& unit testing
System
Testing
Finite State Automata (FSA)










Finite State Automata (FSA)
A set of popular mechanism for specifying what a
program should be doing at a given time.
Can be represented as a state diagram or a table
M = { K, Σ, δ, s, F }
M : finite state machine
K : set of all states q ε K
Σ : set of all inputs σ ε Σ
δ : transitions between states
s : starting state
F: terminated ( final ) state
Finite State Automata (FSA)
q
σ
δ
start
range far
follow line
start
range near
move ahead
follow-line
range near
move ahead
follow-line
range far
follow line
follow-line
see finish line
victory dance
move-ahead
time remaining
move ahead
move-ahead
time elapsed
follow line
move-ahead
see finish line
victory dance
Finite State automata (FSA)
Relationship between FSA and behaviors
FSA Behavioral analog
K
All the behaviors for the task
Σ
The releasers for each behavior in K
δ
The function that computers the new
state
The behavior the robot starts in when
turned on
The behavior(s) the robot must reach
to terminate
s
F
Finite State automata (FSA)
Pick Up the Trash FSA
Finite State automata (FSA)
Pick Up the Trash FSA
Finite State automata (FSA)

Pick Up the Trash FSA
Finite State automata (FSA)

Advantages
• Formal mechanism
• Have to fill in the table, so a way of spotting
unforeseen effects
• Can be implemented in a number of ways

Disadvantages
• Hard to keep up with implicit behaviors

Ex. Avoid obstacle is often running throughout ALL
states, gets tedious to express it formally
• Tend to add explicit variable for transitions,
rather than rely on emergent behavior from
environment
Abstract Behaviors
Template abstract behaviors
- pick-up-trash (trash-color, trash can color, size-trash
can)
- Different initial conditions
Imprinting
- Detect best value for sensing. ( red can )
Exceptions
- Substitute behavior under certain contentions
(different values of red, raster scan)
Scripts



Originally used in natural language processing.
Think of the robot and the tasks in terms of a screenplay.
Causal chain.
Script
Behavior Analog
Examples
Goal
Task
Find victims in rubble
Places
Environment,
applicability or
“taskability” for new
tasks
Collapsed buildings
Actors
Behaviors
explore(), dance(), avoid(),
crawl, move2void,
move2victim
Props, cues
Percepts
Voids: Dark, concave
Victims: heat, motion, color
Causal Chain
Sequence of
behavior
Explore, dance, move2void,
crawl, move2victim,
dropRadio
Subscripts
Exception handling
If lose communications,
return home
Pick up the trash example:
Scripts
for each update…
\\ look for props and cues
first: cans, trash cans,
gripper
rstatus=extract_color(red,
rcx, rSize);
if(rStatus==TRUE)
SEE_RED=TRUE;
else
SEE_RED=FALSE;
bStatus=extract_color(blue,bc
x,bSize);
if(bStatus==TRUE){
SEE_BLUE=TRUE;
NO_BLUE=FALSE;
} else {
SEE_BLUE=FALSe;
NO_BLUE=TRUE;
}
AT_BLUE=looming(size,bSize)
;
gStatus=gripper_status();
if(gStatus==TRUE) {
FULL=TRUE; EMPTY=FALSE;
} else {
FULL=FALSE; EMPTY=TRUE;
}
\\ index into the correct step in the
causal chain
if(EMPTY) {
if(SEE_RED) {
move_to_goal(red);
} else {
wander();
}
} else {
grab_trash();
if(NO_BLUE){
wander();
} else if (AT_BLUE){
drop_trash();
} else if (SEE_BLUE) {
move_to_goal(blue);
}
}
Scripts

Advantages
• A more storyboard like way of thinking about
the behaviors
• If-then, switch style of programming like FSA
• Since a Script is “in” a behavior, other
behaviors such as avoid can be running
concurrently without having to appear “in” the
script
• Exception handling is a big plus in Real Life

Disadvantages
• Can be a bit of overkill for simple sequences,
especially with C++