The SNePS Approach to
Cognitive Robotics
Stuart C. Shapiro
Department of Computer Science and Engineering
and Center for Cognitive Science
University at Buffalo
shapiro@cse.buffalo.edu
S.C. Shapiro
Outline
•
•
•
•
•
•
•
•
Introduction
Intensional Representation & Propositions
SNePS Connectives and Quantifiers
SNeRE Acting Constructs
Example Plans
Representation and Use of Indexicals
A Personal Sense of Time
Summary
S.C. Shapiro
Goal
• A computational cognitive agent that can:
–
–
–
–
–
–
–
S.C. Shapiro
Understand and communicate in English;
Discuss specific, generic, and “rule-like” information;
Reason;
Discuss acts and plans;
Sense;
Act;
Remember and report what it has sensed and done.
Embodied Cassie
• A computational cognitive agent
– Embodied in hardware
– or Software-Simulated
– Based on SNePS and GLAIR.
S.C. Shapiro
SNePS
• Knowledge Representation and Reasoning
– Propositions as Terms
• SNIP: SNePS Inference Package
– Specialized connectives and quantifiers
• SNeBR: SNePS Belief Revision
• SNeRE: SNePS Rational Engine
• Interface Languages
– SNePSUL: Lisp-Like
– SNePSLOG: Logic-Like
– GATN for Fragments of English.
S.C. Shapiro
GLAIR Architecture
Grounded Layered Architecture with Integrated Reasoning
Knowledge Level
SNePS
Perceptuo-Motor Level
Sensory-Actuator Level
NL
Percepts
S.C. Shapiro
Action
Interaction with Cassie
English
(Statement, Question, Command)
(New Belief)
[SNePS]
(Current) Set of Beliefs
[SNePS]
Reasoning
Clarification Dialogue
GATN Parser
Looking in World
Answer
Actions (Updated) Set
of Beliefs
[SNIP]
[SNeRE]
[SNePS]
GATN
Generator
Reasoning
English sentence expressing
new belief answering question reporting actions
S.C. Shapiro
Cassie, the BlocksWorld Robot
S.C. Shapiro
Cassie, the FEVAHR
S.C. Shapiro
FEVAHR/Cassie in the Lab
S.C. Shapiro
FEVAHRWorld Simulation
S.C. Shapiro
UXO Remediation Cassie
Corner flag
Field
UXO
Drop-off zone
NonUXO object
Battery
meter
Corner flag
Recharging
Station
S.C. Shapiro
Corner flag
Cassie
Safe zone
Crystal Space Environment
S.C. Shapiro
Outline
•
•
•
•
•
•
•
•
Introduction
Intensional Representation & Propositions
SNePS Connectives and Quantifiers
SNeRE Acting Constructs
Example Plans
Representation and Use of Indexicals
A Personal Sense of Time
Summary
S.C. Shapiro
Entities, Terms, Symbols, Objects
• Cassie’s mental entity: a person named Bill
• SNePS term: B5
• Object in world:
S.C. Shapiro
Intensional Representation
Intensional entities are distinct
even if coreferential.
“The morning star is the evening star.”
“George IV wondered if Scott was the author of
Waverly.”
S.C. Shapiro
McCarthy’s Telephone Number
Problem
Mary's telephone number is Mike's telephone number.
I understand that Mike's telephone number is Mary's
telephone number.
Pat knew Mike's telephone number.
I understand that Pat knew Mike's telephone number.
Pat dialed Mike's telephone number.
I understand that Pat dialed Mike's telephone number.
S.C. Shapiro
Answering the Telephone
Number Problem
Did Pat dial Mary's telephone number?
Yes, Pat dialed Mary's telephone number.
Did Pat know Mary's telephone number?
I don't know.
S.C. Shapiro
Representing Propositions
Propositions must be
first-class entities of the domain
Represented by terms.
S.C. Shapiro
Discussing Propositions
That Bill is sweet is Mary's favorite proposition.
I understand that Mary's favorite proposition is
that Bill is sweet.
Mike believes Mary's favorite proposition.
I understand that Mike believes that Bill is sweet.
S.C. Shapiro
Outline
•
•
•
•
•
•
•
•
Introduction
Intensional Representation & Propositions
SNePS Connectives and Quantifiers
SNeRE Acting Constructs
Example Plans
Representation and Use of Indexicals
A Personal Sense of Time
Summary
S.C. Shapiro
Logic for NLU &
Commonsense Reasoning
Either Pat is a man or Pat is a woman or Pat is a robot.
I understand that Pat is a robot or Pat is a woman
or Pat is a man.
Pat is a woman.
I understand that Pat is a woman.
What is Pat?
Pat is a woman and Pat is not a robot
and Pat is not a man.
S.C. Shapiro
Representation in FOPL?
Man(Pat)  Woman(Pat)  Robot(Pat)
S.C. Shapiro
Representation in FOPL?
Man(Pat)  Woman(Pat)  Robot(Pat)
but don’t want inclusive or
S.C. Shapiro
Representation in FOPL?
Man(Pat)  Woman(Pat)  Robot(Pat)
but don’t want inclusive or
Man(Pat) +Woman(Pat) +Robot(Pat)
T
T
T
F
T
So don’t want exclusive or either
S.C. Shapiro
andor
andor(i, j){P1, ..., Pn}
True iff at least i, and at most j of the Pi are True
S.C. Shapiro
thresh
thresh(i, j){P1, ..., Pn}
True iff either fewer than i,
or more than j
of the Pi are True
Note: thresh(i, j) ~andor(i, j)
S.C. Shapiro
or-entailment
{P1, ..., Pn} v=> {Q1, ..., Qn}
True iff
S.C. Shapiro
for all i, j
Pi  Qj
and-entailment
{P1, ..., Pn} &=> {Q1, ..., Qn}
True iff
for all j
P1 &…& Pn  Qj
S.C. Shapiro
Numerical entailment
{P1, ..., Pn} i=> {Q1, ..., Qn}
True iff
for all j
andor(i, n){P1, …, Pn }  Qj
S.C. Shapiro
Universal Quantifier
all(ū)({R1(ū),..., Rn(ū)}
&=> {C1(ū),..., Cm(ū)})
Every ā that satisfies
R1(ū)&…& Rn(ū)
also satisfies
C1(ū),..., Cm(ū)})
S.C. Shapiro
Numerical Quantifiers
nexists(i,j,k)(x)
({P1(x),..., Pn(x)}: {Q(x)})}
There are k individuals that satisfy
P1(x) ... Pn(x)
and, of them, at least i and at most j also satisfy
Q(x)
S.C. Shapiro
Outline
•
•
•
•
•
•
•
•
Introduction
Intensional Representation & Propositions
SNePS Connectives and Quantifiers
SNeRE Acting Constructs
Example Plans
Representation and Use of Indexicals
A Personal Sense of Time
Summary
S.C. Shapiro
MENTAL ACTS
• Believe(proposition)
• Disbelieve(proposition)
S.C. Shapiro
Act Selection
• Do-One({act1 ... actn})
• Snif(if(condition, act),
...
if(condition, act)
[else(act)])
S.C. Shapiro
Act Iteration
• Do-All({act1 ... actn})
• Sniterate(if(condition, act),
...
if(condition, act),
[else(act)])
• Snsequence(act1, ..., actn)
• Cascade(act1, ..., actn)
• P-Do-All({act1, ..., act2})
S.C. Shapiro
Entity Iteration
WithSome(var, suchthat,
do, [else])
WithAll(var, suchthat,
do, [else])
WithSome+(var, suchthat,
do, [else])
WithNew(vars, thatare, suchthat,
do, [else])
S.C. Shapiro
Proposition/Act Transformers
•
•
•
•
•
•
•
•
Achieve(proposition)
ActPlan(act, plan)
GoalPlan(proposition, act)
Precondition(act, proposition)
Effect(act, proposition)
WhenDo(proposition, act)
WheneverDo(proposition, act)
IfDo(proposition, act)
S.C. Shapiro
Outline
•
•
•
•
•
•
•
•
Introduction
Intensional Representation & Propositions
SNePS Connectives and Quantifiers
SNeRE Acting Constructs
Example Plans
Representation and Use of Indexicals
A Personal Sense of Time
Summary
S.C. Shapiro
Conditional Plans
If a block is on a support then a plan to achieve
that the support is clear is to pick up the block
and then put the block on the table.
all(x, y)
({Block(x), Support(y), On(x, y)}
&=> {GoalPlan(Clear(y),
Snsequence(Pickup(x),
Put(x, Table)))})
(STRIPS-like representation: No times)
S.C. Shapiro
Use of Conditional Plan
GoalPlan(Clear(B),
Snsequence(Pickup(A),
Put(A, Table)))
Remember (cache) derived propositions.
S.C. Shapiro
Use of Conditional Plan
GoalPlan(Clear(B),
Snsequence(Pickup(A),
Put(A, Table)))???
SNeBR to the rescue!
S.C. Shapiro
A FEVAHR Acting Rule
all(a, o) ({Agent(a), Thing(o)}
&=> {Precondition(Follow(a, o), Near(a, o)),
GoalPlan(Near(a, o), Goto(a, o)),
Precondition(Goto(a, o), Lookat(a, o)),
ActPlan(Lookat(a, o), Find(a, o))})
Uses a temporal model.
S.C. Shapiro
Acting According to the Rule
S.C. Shapiro
Acting According to the Rule
Follow a red robot.
I found a red robot.
I am looking at a red robot.
S.C. Shapiro
Acting According to the Rule
Follow a red robot.
I found a red robot.
I am looking at a red robot.
I went to a red robot.
I am near a red robot.
I am following a red robot.
S.C. Shapiro
A Plan for Blowing up UXOs
all(a)(Agent(a) =>
ActPlan(Blowup(a, UXOs),
Act(a,
Cascade(SearchforUxo(a),
WithSome+(obj,
Near(a, obj),
WithNew({ch ex},
{Charge(ch), Explosion(ex)},
Possess(a, ch),
Cascade(Place(a, ch, obj),
Hide(a),
Waitfor(a, ex),
SearchforUxo(a))),
goto(a, SafeZone))))))
S.C. Shapiro
Outline
•
•
•
•
•
•
•
•
Introduction
Intensional Representation & Propositions
SNePS Connectives and Quantifiers
SNeRE Acting Constructs
Example Plans
Representation and Use of Indexicals
A Personal Sense of Time
Summary
S.C. Shapiro
Representation and Use of Indexicals
• Words whose meanings are determined by
occasion of use
• E.g. I, you, now, then, here, there
• Deictic Center <*I, *YOU, *NOW>
• *I: SNePS term representing Cassie
• *YOU: person Cassie is talking with
• *NOW: current time.
S.C. Shapiro
Analysis of Indexicals
(in input)
•
•
•
•
S.C. Shapiro
First person pronouns: *YOU
Second person pronouns: *I
“here”: location of *YOU
Present/Past relative to *NOW.
Generation of Indexicals
• *I: First person pronouns
• *YOU: Second person pronouns
• *NOW: used to determine tense and aspect.
S.C. Shapiro
Use of Indexicals 1
Come here.
S.C. Shapiro
Use of Indexicals 2
Come here.
I came to you, Stu.
I am near you.
S.C. Shapiro
Use of Indexicals 3
Who am I?
Your name is ‘Stu’
and you are a person.
Who have you talked to?
I am talking to you.
Talk to Bill.
I am talking to you, Bill.
Come here.
S.C. Shapiro
Use of Indexicals 4
Come here.
I found you.
I am looking at you.
S.C. Shapiro
Use of Indexicals 5
Come here.
I found you.
I am looking at you.
I came to you.
I am near you.
S.C. Shapiro
Use of Indexicals 6
Who am I?
Your name is ‘Bill’
and you are a person.
Who are you?
I am the FEVAHR
and my name is ‘Cassie’.
Who have you talked to?
I talked to Stu
and I am talking to you.
S.C. Shapiro
Outline
•
•
•
•
•
•
•
•
Introduction
Intensional Representation & Propositions
SNePS Connectives and Quantifiers
SNeRE Acting Constructs
Example Plans
Representation and Use of Indexicals
A Personal Sense of Time
Summary
S.C. Shapiro
A Personal Sense of Time
• *NOW contains SNePS term representing
current time.
• *NOW moves when Cassie acts or
perceives a change of state.
S.C. Shapiro
Representation of Time
before
!
after
before
!
after
!
B1
B6
I
lex
find
S.C. Shapiro
NOW
Movement of Time
t1
NOW
S.C. Shapiro
before
!
after
t2
NOW
before
!
after
t3
NOW
Performing a Punctual Act
t1
before
!
after
t2
before
!
after
t3
time
!
event
NOW
S.C. Shapiro
NOW
Performing a Durative Act
t1
before
!
after
t2
!
time
t3
!
event
NOW
NOW
S.C. Shapiro
The Pacemaker
• PML process periodically increments
variable COUNT.
• *COUNT = some PML integer.
• Reset to 0 when NOW moves.
• Provides bodily “feel” of passing time.
S.C. Shapiro
Quantizing Time
Cannot conceptualize fine distinctions in time
intervals.
So quantize, e.g. into half orders of magnitude
(Hobbs, 2000).
S.C. Shapiro
Movement of Time with
Pacemaker
!
time
t1
before
!
duration
after
q
t2
KL
PML
hom
NOW
COUNT
n
0
S.C. Shapiro
The Problem of the Fleeting Now
How can you reason about “now”
if it never stands still?
S.C. Shapiro
Fleeting Now Example 1
12:15:00:
12:15:02:
12:17:00:
12:19:00:
“Is John having lunch now?”
Agent walks to John’s office.
Agent sees John at his desk, eating.
Agent reports “yes”.
Appropriate granularity.
S.C. Shapiro
Fleeting Now Example 2
12:15:00: “Is John having lunch now?”
Agent knows John is at home without a phone.
Agent contemplates driving to John’s home.
Don’t bother---inappropriate granularity.
S.C. Shapiro
The Vagueness of “now”
I’m now giving a talk.
I’m now supervising PhD students.
I’m now visiting Paris.
I’m now living in Buffalo.
The agent is now walking to John’s office.
The agent is now seeing if John is eating lunch.
Multiple now’s at different granularities.
S.C. Shapiro
NOW-MTF
Maximal Temporal Frame based on *NOW
NOW
Semi-lattice of times, all of which contain *NOW,
any of which could be meant by “now”
Finite---only conceptualized times of conceptualized states
S.C. Shapiro
Moving NOW with MTF
NOW
S.C. Shapiro
Typical Durations
“If the walk light is on now, cross the street.”
Relevant duration is typical duration of walk lights.
“Is John having lunch now?”
Relevant duration is typical duration of lunch.
Use quantized typical durations when updating
NOW-MTFs.
S.C. Shapiro
Using Appropriate Granularity
Lunch time
NOW
Yes!
S.C. Shapiro
Outline
•
•
•
•
•
•
•
•
Introduction
Intensional Representation & Propositions
SNePS Connectives and Quantifiers
SNeRE Acting Constructs
Example Plans
Representation and Use of Indexicals
A Personal Sense of Time
Summary
S.C. Shapiro
Goal
• A computational cognitive agent/robot
• That can communicate in natural
language.
S.C. Shapiro
Intensional Representation
& Propositions
•
•
•
•
SNePS terms represent mental entities.
May assert that two entities are coreferential.
Relations/acts may be declared transparent.
Propositions are first-class entities.
S.C. Shapiro
SNePS Connectives and
Quantifiers
• Designed logical connectives
and rules of inference
More appropriate for NLU and
Commonsense reasoning
than in standard FOPC.
S.C. Shapiro
SNeRE Acting Constructs
• Separate, but Coordinated
Syntax and Semantics
For Acting and for Reasoning
S.C. Shapiro
Representation and Use
of Indexicals
• Use of Deictic Center for parser to interpret
indexicals as current referents
• And for generator to generate indexicals
from current referents.
S.C. Shapiro
A Personal Sense of Time
• *NOW is current time.
• Updated when Cassie acts
or perceives a change of state.
• Points into MTF to support vagueness
of “now”.
S.C. Shapiro
For More Information
•
•
•
•
•
•
Personnel
Manual
Tutorial
Bibliography
ftp’able SNePS source code
etc.
• http://www.cse.buffalo.edu/sneps/
S.C. Shapiro