EE690
Design of
Embodied Intelligence
Janusz Starzyk
EECS, Ohio University
EE141
Intelligence
AI’s holy grail
From Pattie Maes MIT Media Lab

“…Perhaps the last frontier of science – its ultimate
challenge- is to understand the biological basis of
consciousness and the mental process by which we
perceive, act, learn and remember..” from Principles of Neural
Science by E. R. Kandel et al.
E. R. Kandel won Nobel Price in 2000 for his work on physiological
basis of memory storage in neurons. His family roots are in Kolomyje
– so he once told “as with all bright people, my roots are in Poland”

“…The question of intelligence is the last great terrestrial
frontier of science...” from Jeff Hawkins On Intelligence. Jeff
Hawkins founded the Redwood Neuroscience Institute devoted to brain
research. He co-founded Palm Computing and Handspring Inc.
EE141
Outline
Traditional Artificial Intelligence
 Embodied Intelligence (EI)
 Challenges of EI

 We need to know how
 We need means to implement it
 We need resources to build and sustain its
operation

Promises of EI
 To economy
 To society
EE141
Intelligence
Mainstream Science on Intelligence
December 13, 1994:
An Editorial With 52 Signatories,
History, and Bibliography by Linda
S. Gottfredson, University of
Delaware
From http://www.indiana.edu/~intell/map.shtml
EE141
Intelligence is a very general
mental capability that,
among other things, involves
the ability to reason, plan,
solve problems, think
abstractly, comprehend
complex ideas, learn quickly
and learn from experience.
Various Definitions of Intelligence
 The American Heritage Dictionary:
 The capacity to acquire and apply knowledge.
 The faculty of thought and reason.
 Webster Dictionary:
 The act or state of knowing; the exercise of the understanding.
 The capacity to know or understand; readiness of
comprehension;
 Wikipedia – The Free Encyclopedia:
 The capacity to reason, plan, solve problems, think abstractly,
comprehend ideas and language, and learn.
 The classical behavioral/biologists:
 The ability to adapt to new conditions and to successfully cope
with life situations.
 Dr. C. George Boeree, professor in the Psychology Department at
Shippensburg University:
 A person's capacity to (1) acquire knowledge (i.e. learn and
understand), (2) apply knowledge (solve problems), and (3)
engage in abstract reasoning.
 Stanford University Professor of Computer Science Dr. John McCarthy, a
pioneer in AI:
 The computational part of the ability to achieve goals in the world.
EE141
Traditional AI

Abstract intelligence
Embodied Intelligence

 attempt to simulate
“highest” human faculties:
 knowledge is implicit in the
fact that we have a body
– language, discursive
reason, mathematics,
abstract problem solving

Environment model
 Condition for problem
solving in abstract way
 “brain in a vat”
EE141
Embodiment
– embodiment is a
foundation for brain
development

Intelligence develops
through interaction with
environment
 Situated in a specific
environment
 Environment is its best
model
Design principles of intelligent systems
from Rolf Pfeifer “Understanding of Intelligence”, 1999








EE141
Interaction with
complex environment
cheap design
ecological balance
redundancy principle
asynchronous
parallel, loosely
coupled processes
sensory-motor
coordination
value principle
Agent
Drawing by Ciarán O’Leary- Dublin Institute of Technology
Embodied Intelligence
 Definition
 Embodied Intelligence (EI) is a mechanism that learns
how to survive in a hostile environment
EE141
– Mechanism: biological, mechanical or virtual agent
with embodied sensors and actuators
– EI acts on environment and perceives its actions
– Environment hostility is persistent and stimulates EI to act
– Hostility: direct aggression, pain, anxiety or scarcity of resources
– EI learns so it must have associative self-organizing memory
– Knowledge is acquired by EI
Embodiment of Mind
Definition

Embodiment of a mind
is a mechanism
under the control of the
intelligence core
that contains
sensors and actuators
connected to the core through
communication channels.
EE141
Embodiment
Sensors
Intelligence
core
Actuators
Environment
Embodiment of Mind




Necessary for development of
intelligence
Hosts brain’s interfaces that
interact with environment
Not necessarily constant or in
the form of a physical body
Boundary transforms modifying
brain’s self-determination
EE141
Embodiment
Sensors
Intelligence
core
Actuators
Environment
Embodiment of Mind




Brain learns own body’s dynamic
Self-awareness results from
identification with own embodiment
Embodiment can be extended by
using tools and machines
Successful operation depends on
correct perception of environment
and own embodiment
EE141
Motor cortex
Somatosensory cortex
Sensory associative
cortex
Pars
opercularis
Visual associative
cortex
Broca’s
area
Visual
cortex
Primary
Auditory cortex
Wernicke’s
area
Brain Organization
EE141
While we learn
its functions
can we emulate
its operation?
How can we design intelligence?
 We
need to know how
 We need means to
implement it
 We need resources to
build and sustain its
operation
EE141
Requirements for Embodied Intelligence

State oriented
 Learns spatio-temporal patterns
 Situated in time and space

Learning
 Perpetual learning
 Screening for novelty

Value driven
 Goal creation
 Competing goals

Emergence
 Artificial evolution
 Self-organization
EE141
EI Interaction with Environment
Agent Architecture
Reason
Short-term Memory
Perceive
Act
RETRIEVAL
LEARNING
Long-term Memory
INPUT
OUTPUT
Task
Environment
Simulation or
Real-World System
EE141
From
Randolph M. Jones, P : www.soartech.com
Challenges of Embodied Intelligence

Development of sensory interfaces




Active vision
Speech processing
Tactile, smell, taste, temperature, pressure sensing
Additional sensing
– Infrared, radar, lidar, ultrasound, GPS, etc.
– Can too many senses be less useful?

Development of reinforcement interfaces
 Energy, temperature, pressure, acceleration level
 Teacher input

Development of motor interfaces
 Arms, legs, fingers, eye movement
EE141
Challenges of Embodied Intelligence (cont.)

Finding algorithmic solutions for
 Association, memory, sequence learning,
invariance building, representation, anticipation,
value learning (pain reduction), goal creation,
planning

Development of circuits for neural computing
 Determine organization of artificial minicolumn
 Self-organized hierarchy of minicolumns for
sensing and motor control
 Self-organization of goal creation pathway
EE141
Human Intelligence - Uniform Structure



V. Mountcastle argues that all
regions of the brain perform the
same computational algorithm
Groups of neurons (minicolumns)
have the same structure and are
connected in a pseudorandom way
Minicolumns organized in
macrocolumns
VB Mountcastle (2003). Introduction [to a special issue of
Cerebral Cortex on columns]. Cerebral Cortex, 13, 2-4.
EE141
Selective Processing in Minicolumns
Sensory inputs are represented by gradually
more abstract features in the sensory hierarchy
 Use “sameness principle” of the observed
objects to detect and learn feature invariance
 Learn to store temporal sequences
 Use random wiring to preselect sensory
features
 Use feedback for input prediction and screen
input information for novelty
 Use redundant structures of sparsely connected
processing elements

EE141
Interactions between Minicolumns
Pleasure
Positive
Reinforcement
Pain
Value
center
Learn
Sensory
processing
Direct
Negative
Reinforcement
Control
Motor
processing
Associate
Sensory
Inputs
EE141
Motor
Outputs
Functions of Neurons in Minicolumns

Sensory neurons are responsible for representing environment





receive inputs from sensors or sensory neurons on lower level
represent the environment
receive feedback input from motor and higher level sensory neurons
help to activate motor and reinforcement neurons
Motor neurons are responsible for actions and skills
 are activated by reinforcement and sensory neurons
 activate actuators or provide an input to lower level motor neurons
 provide planning inputs to sensory neurons

Reinforcement neurons are responsible for building the value
system, goal creation, learning, and exploration




EE141
receive inputs from reinforcement neurons on the lower level
receive inputs from sensory neurons
provide inputs to motor neurons
initiate learning and force exploration
Selective Sensory-Motor Pathways




Hierarchical sensory and motor pathways
Branching off from more general to more specific
Easy to expand to higher levels
Gradual loss of interconnect plasticity towards inputs
inputs
EE141
internal
representations
Selective Sensory-Motor Pathways
Increasing connection’s plasticity
Sensory pathway
Sensory neurons
in a minicolumn
…………
106 neurons
10 neurons
104 neurons
EE141
Activation pathway
10 neurons
1011 neurons
…
Environment
…
…...
Organization of Sensory-motor Pathways
General characteristics:
Function:
 Hierarchical structure
 Minicolumn processing
 Spatial and temporal association
via interconnections and dual
neurons
 Feedback connections
 Selective adaptation
 Invariant representation
 Anticipation
 Screening for novelty
 Value learning
... …
... …
... …
EE141
Sensory-motor Coordination
Goal creation
&Value system
Sensor path
Motor path
D
…
…
R
…
… A
R: representation
E: expectation
A: association
D: direction
P: planning
EE141
Environment
’ adaptability
Increasing connection’s
E
Goal Driven Behavior





EE141
Goal driven behavior is one of the
required elements of intelligence
Perceptions and actions are activated
selectively to serve the machine’s
objectives
In the existing EI models, the goal is
defined by designers and is given to the
learning agent
Humans and animals create their own
goals
The goal creation may be one of the most
important elements of EI mechanism
Goal Creation





EE141
Goals must be built and understood in a similar way to
building perceptions
More complex goals can be understood only if
representations for these goals are build
Goal creation should result from EI interaction with
environment, by perceiving successes or failures of its
actions
Goal creation ability is essential
for developing intelligence
We will create goals based on
simple structures interacting with
sensory and motor pathways
Pain-center and Goal Creation





Simple Mechanism
Dual pain level
Pain increase
Creates hierarchy of
(-)
+
values
(-)
(+)
Leads to formulation of
complex goals
(+)
Reinforcement
(+)
(-)
neurotransmitter:
Pain level
Pain decrease
• Pain increase
• Pain decrease
Excitation
Forces exploration
EE141
Sensor
Environment
Motor
Abstract Goal Creation
 The goal is to reduce
the primitive pain
level
 Abstract goals are
created to satisfy
the primitive goals
Sensory pathway
(perception, sense)
Motor pathway
(action, reaction)
refrigerator
Open
-
+
food”becomes a
“
sensory input to
abstract pain center
Abstract pain
(Delayed memory of pain)
Food
Eat
-
Association
Inhibition
Reinforcement
Connection
Planning
Expectation
EE141
Level II
Level I
+
Dual pain
Pain
Primitive
Level
Stomach
The Three Pathways Combined



Goal creation, sensory
and motor pathways
interact on different
hierarchy levels
Pain driven goal creation
sets goal priorities
The tree pathways
emerge together from
interaction with the
environment
Pain tree I
Pain tree II
Motor pathway
Sensory pathway
Pain center to motor
Sensor to motor
Sensor to pain center
EE141
How can we design intelligence?
We need to know how
 We need means to
implement it






EE141
Hardware self-organization
Sparsity of interconnect
Functionality requirements
EI Design Efforts
We need resources to build
and sustain its operation
We need means to implement it

What is needed is a self-organizing hardware





Saves design work
Increases hardware reliability
Improves production yield
Lowers design cost
Basic mechanism for self-organized hardware
is pruning of random interconnections
 Nature does it

Activity triggered “growth” of new neurons
 Spare neurons used in the later stages of learning
EE141
We need means to implement it

Brain is self-organizing and sparse
Human Brain
at Birth
EE141
6 Years Old
14 Years Old
Rethinking the Brain, Families and Work Institute, Rima Shore, 1997.
Synaptic Density over the Lifespan
Thompson, R. A., & Nelson, C. A. (2001). Developmental science and the media: Early brain
development. American Psychologist, 56(1), 5-15.
EE141
Sparse Connectivity
The brain is sparsely connected.
A neuron in cortex may have on the order of
100,000 synapses. There are more than 1010
neurons in the brain. Fractional connectivity is
very low: 0.001%.
Implications:
Connections are expensive biologically.
 Short local connections are cheaper than
long ones.

EE141
Hierarchical Self-organizing
Learning Memory (HSOLM)


Feature extraction – using WTA
Generalization – using overlapping regions
winner
h+1
…
…
h
…
…
s
…
…
…
…
r-lower level region (single WTA in each)
EE141
Hierarchical Self-organizing Learning Memory



3-layer R-net in hierarchical structure
Reduced connection density
The winner pathway replaces the WTA
 Established using feed-back and local competition
winner
h+1
s2
…
…
…
…
number of output links
from the lower level nodes
s1
lo  3 nh1 (i  1, 2, ...nh (ns1, ns 2 ))
h
…
Input pattern
EE141
Self-Organizing Learning Arrays SOLAR
EE141
Wiring in SOLAR
Initial wiring and final wiring selection for credit card
approval problem
EE141
Associative SOLAR
EE141
Sequence Learning Mechanism
Multiple winner
Detection Neuron
Excitation
link
miss
IR
n1
mit
mom
n2
n3
1 2 3 4
1 2 3 4
1 2 3 4
1 2 3 4
1 2 3 4
s
m
i
t
o
MUX
MUX
MUX
MUX
MUX
TP
LFN
LN
Preduction
Neuron
Prediction
Matching
Neuron
EE141
PCN
From
ESN
Software or hardware?
Software





Sequential
Error prone
Require programming
Low cost
Well developed
programming methods
EE141
Hardware





Concurrent
Robust
Require design
Significant cost
Hardware prototypes
hard to build
Future software/hardware capabilities
11
10
10
10
g
alo
n
A
SI
L
V
9
Number of neurons
10
(F
ch
a
ro
pp
a
are
w
d
r
Ha
8
10
7
10
re
tw a
Sof
6
10
io
u lat
m
i
S
Human
brain
complexity
A)
G
P
d)
ase
b
C
n (P
5
10
4
10
2005
2010
2015
2020
2025
Year
EE141
2030
2035
2040
How can we design intelligence?
 We
need to know how
 We need means to
implement it
 We need resources to
build and sustain its
operation
EE141
Major Intelligent Systems Design Efforts

GOMS - Goals, Operators, Methods, and Selection Rules
 Proposed by Card, Moran, and Newell (1983), for modeling and describing
human task performance

SOAR - State, Operator, Application, Result
 Based on the book by Newell, A. (1990), Unified Theories of Cognition.

ACT-R (Adaptive Control of Thought - Rational )
 Described by Anderson, J. (1993), Rules of the Mind

Artificial General Intelligence – Adaptive A.I. Inc.
 http://adaptiveai.com/

Hierarchical Temporal Memory – Numenta Inc.
http://www.numenta.com/
3DANN
“Brain On Silicon” will not be
just a science fiction!
EE141
Irvine Sensors Corporation (Costa Mesa, CA)
EI Design Efforts

GOMS - Goals, Operators, Methods,
and Selection Rules





Inspired by psychology of immediate action
Explicit hierarchical task decompositions
Explicit pairing of goals with tasks
Belief, Goal and Plan representation and
selection
Primary Engineering Activities




Cognitive task analysis
Design of goal hierarchy
Definition of plans and selection rules
Definition of primitive operators
and resource constraints
Proposed by Card, Moran, and Newell (1983),
for modeling and describing human task performance
EE141
From slide by Dan Glaser
EI Design Efforts

SOAR - State, Operator, Application, Result
 Problem space and physical symbols
 Minimal set of rules to support intelligent behavior
 Belief, Goal and Plan representation and selection

Primary Engineering Activities




Cognitive task analysis
Identification of primitive operations
Design of belief and goal hierarchies
Design of situation interpretation hierarchies
 Interactions between knowledge units

http://sitemaker.umich.edu/soar
Newell, A. 1990. Unified Theories of Cognition.
Cambridge, Massachusetts: Harvard University Press.
EE141
EI Design Efforts

ACT-R (Adaptive Control of Thought - Rational ):
 a cognitive architecture inspired by psychological models of
memory, skills, and learning
 Optimization-oriented integrated memory, action, and learning
 Procedural knowledge units with explicit retrieval
 Uses if then production rules
 Belief, Goal and Plan representation and selection

Primary Engineering Activities
 Cognitive task analysis
 Identification of procedural and
declarative knowledge
 Design of semantic associations and
retrievable goal hierarchies
 Identification of primitive operations
with logical and associative context
EE141
Anderson, John. Rules of the Mind,
Lawrence Erlbaum Associates, Hillsdale, NJ (1993).
EI Design Efforts

Hierarchical Temporal Memory – Numenta Inc.







Primary Engineering Activities





Uses hierarchy of spatio-temporal associations
Discover causes in the world
Learns complex goal-oriented behavior
Makes predictions while learning
Combines unsupervised and supervised learning to make associations
Aims at exceeding human level cognition
Cognitive structure analysis
Pattern learning in hierarchical memories
Temporal sequence learning mechanism
Goal creation
Die Stack
http://www.numenta.com/
Irvine Sensors Corporation (Costa Mesa, CA)
EE141
EI Design Efforts

Artificial General Intelligence – Adaptive A.I. Inc.








Primary Engineering Activities





Has general cognitive ability
Acquires knowledge and skills
Uses bidirectional, real-time interaction
Selects information and uses an adaptive attention
Unsupervised and self-supervised learning
Adaptive and self-organizing data structure
Processes dynamic patterns
Explicitly engineering functionality
Conceptual design
General proof of concept
Animal level cognition
3DANN
http://adaptiveai.com/
“Brain On Silicon” will not be
just a science fiction!
EE141
Irvine Sensors Corporation (Costa Mesa, CA)
How can we design intelligence?
 We
need to know how
 We need means to
implement it
 We need resources to
build and sustain its
operation
EE141
Resources – Evolution of Electronics
EE141
From Ray Kurzwail, The Singularity Summit at Stanford, May 13, 2006
EE141
By Gordon E. Moore
EE141
Clock Speed (doubles every 2.7 years)
EE141
From Ray Kurzwail, The Singularity Summit at Stanford, May 13, 2006
Doubling (or Halving) times








EE141
Dynamic RAM Memory “Half Pitch” Feature Size
Dynamic RAM Memory (bits per dollar)
Average Transistor Price
5.4 years
1.5 years
1.6 years
Microprocessor Cost per Transistor Cycle
Total Bits Shipped
Processor Performance in MIPS
Transistors in Intel Microprocessors
Microprocessor Clock Speed
1.1 years
1.1 years
1.8 years
2.0 years
2.7 years
From Ray Kurzwail, The Singularity Summit at Stanford, May 13, 2006
Why should we care?
EE141
Source: SEMATECH
Design Productivity Gap  Low-Value Designs?
Percent of die area that must be occupied by memory to
maintain SOC design productivity
100%
80%
60%
% Area Memory
40%
% Area Reused
Logic
20%
% Area New Logic
19
99
20
02
20
05
20
08
20
11
20
14
0%
Source = Japanese system-LSI industry
EE141
Biomimetics and Bio-inspired Systems
Mission Complexity
Impact on Space Transportation, Space Science and Earth Science
2002
2010
2020
2030
Embryonics Self Assembled Array
Space Transportation
Biologically inspired
aero-space systems
Sensor Web
Brain-like
computing
Extremophiles
Mars in situ
life detector
Skin and Bone
Self healing structure
and thermal protection
systems
EE141
Biological nanopore
low resolution
Artificial nanopore
high resolution
DNA
Computing
Biological Mimicking
Nanosystems
Revolutions in electronics and computing
will allow reconfigurable, autonomous,
"thinking" spacecraft
Universal Joint
http://ipt.arc.nasa.gov/nanotechnology.html
NanoEngineer-1 is an
Open Source (GPL)
project sponsored by
Nanorex, Inc.
It is an interactive 3D
nanomechanical
engineering program
Planetary Gear
EE141 Nanorex Inc.
From
EE141
From Ray Kurzwail, The Singularity Summit at Stanford, May 13, 2006
Promises of embodied intelligence

To society
 Advanced use of technology
– Robots
– Tutors
– Intelligent gadgets
 Society of minds
– Superhuman intelligence
– Progress in science
– Solution to societies’ ills

To industry
 Technological development
 New markets
 Economical growth
EE141
ISAC, a Two-Armed Humanoid Robot
Vanderbilt University
Transhumanist
values








Nothing wrong about “tampering with nature”
Individual choice in the use of enhancement technologies;
Peace, international cooperation, anti-proliferation of WMDs
Improving understanding (research and public debate; critical thinking;
open-mindedness; scientific progress; open discussion of the future)
Getting smarter (individually; collectively; develop machine intelligence)
Pragmatism; engineering and entrepreneur-spirit; “can-do” attitude
Diversity (species, race, religious creed, sexual orientation, life style, etc.)
Saving lives (life extension, anti-aging research, and cryonics)
EE141
Copyright Institute for Ethics and Emerging Technologies 2004-2005
Sounds like science fiction


If you’re trying to look far
ahead, and what you see
seems like science fiction,
it might be wrong.
But if it doesn’t seem like
science fiction, it’s
definitely wrong.
From presentation by Feresight Institute
EE141
EE141
Embodied Artificial Intelligence
Based on:
[1] E. R. Kandel et al. Principles of Neural Science,
McGraw-Hill/Appleton & Lange; 4 edition, 2000.
[2] F. Inda, R. Pfeifer, L. Steels, Y. Kuniyoshi, “Embodied Artificial
Intelligence,” International seminar, Germany, July 2003.
[3] R. Chrisley, “Embodied artificial intelligence, ” Artificial
Intelligence, vol. 149, pp.131-150, 2003.
[4] R. Pfeifer and C. Scheier, Understanding Intelligence, MIT
Press, Cambridge, MA, 1999.
[5] R. A. Brooks, “Intelligence without reason,” In Proc. IJCAI-91.
(1991) 569-595 .
[6] R. A. Brooks, Flesh and Machines: How Robots Will Change Us,
(Pantheon, 2002).
[7] R. Kurzweil The Age of Spiritual Machines: When Computers
Exceed Human Intelligence, (Penguin, 2000).
EE141