A Soar’s Eye View of ACT-R
John Laird
24th Soar Workshop
June 11, 2004
Soar / ACT-R Comparison
• What changes relative to ACT-R would significantly alter Soar?
• Not just extensions (activation, RL, EpMem) but fundamental changes.
• What changes relative to Soar would significantly alter ACT-R?
Soar
Soar
Soar
ACT-R
ACT-R
ACT-R
2
Obvious Similarities
Input/Output
Short-term memories
Long-term memories
Sequential control
Goal Structures
Learning
Soar 9
Buffers & Async.
Graph Structure
Activation
Production Rules
Episodes
Operator
State stack
Chunking
Reinforcement
Episodes
ACT-R 5
Buffers & Async.
Chunks in buffers
Base Activation
Production Rules
Declarative Memory
Rule Utilities
Chunk Associations
Production
Goal & Declarative Memory
Production Composition
Utility Learning
Chunks -> Decl. Memory
Goal & Chunk Association
Base Activation
3
Short-term Memories
ACT-R
Soar
• Chunks (flat structures) in buffers
• Unbounded graph structure
•
•
•
•
• Multi-valued attributes: sets
• Decision on ^operator of state
• I-support and o-support
• Explicitly represent state
• Short-term identifiers
• Long-term identifiers for each chunk
• Provides hierarchical structure
• Generated each time retrieved
• Values can be long-term symbols
state
One chunk/buffer
Chunk types with fixed slots
Goal, Declarative Memory, Perception
All persistent until replaced/modified
state
visualization
goal
#3
declarative
memory
perception
#45
#45
red #3 ‘x’ #9
4
Implications for Soar
• Unbounded working memory
• No easy way to move subset of short-term memory to
long-term memory piece by piece
• Can’t maintain connections between objects without longterm memory symbols
• Makes it possible to determine results automatically
• Supports automatic removal of irrelevant data
state
state
5
Implications for ACT-R
• Bounded representation
• Long-term memory symbols allow dynamic encapsulation
• Can learn to test only chunk id instead of substructure
• Flat representation
• Hard to represent sets
• Requires “unpacking” of object symbols to access features
• But can learn rules that access symbols directly
• How can it recognize structured objects from perception?
• (Blending?)
• Unitary object representation primacy (vs. independent features)
• All features are equally important (activation is object based)
• Chunk types are architecturally meaningful
goal
#3
declarative
memory
perception
#45
6
Implications for ACT-R II
• Persistence
• Easy to have inconsistent beliefs
• Consistency always competes with other reasoning
• Working Memory = retrieved LTM Declarative Memory
(Changes in working memory change declarative memory)
• No memory of old values in chunks
• Difficult to maintain independent copies of same object
• Hypothetical reasoning
goal
#3
declarative
memory
perception
#45
7
Fundamental Issue:
Long-Term Object Identity
• Architectural (ACT-R) vs. Knowledge-based (Soar)
• Connecting to perception
• Connecting to other long-term memories
• Copying structures
8
Decision Making
•
•
•
•
•
Generate features
Generate alternatives
Compare & rate alternatives
Select
Apply
Soar
Parallel rules
Parallel rules
Parallel rules
Architecture
Parallel rules
ACT-R
Sequential rules
Match rule conditions
Rule utility
Architecture
Rule actions
Dimensions for comparison:
• Simple metrics
• # of reasoning steps required
• # of sequential rule firings
• # knowledge units (rules) required
– ACT-R often trades off chunks + interpretation + learning for rules.
• Capabilities
•
•
•
•
Expressibility
Use context
Open to meta-reasoning
Modification through learning
9
Execution Steps
•
•
•
•
•
Generate features (F)
Generate options (O)
Compare & rate options (C)
Select
Apply (A)
• # of rule firings
• # of sequential steps
Soar
ACT-R
Parallel rules
Parallel rules
Parallel rules
Architecture
Parallel rules
Sequential rules
Match rule conditions
Rule utility
Architecture
Rule actions
F+ O + C+A
1
F+1
F+1
• This is complicated by declarative memory retrievals in ACT-R – but they are not
really procedural knowledge directly involved in decision making, although they
are sometimes involved indirectly.
10
Propose and Apply Knowledge Units
• For a single O that can be selected in S Situations and has A was of Applying:
• Soar: O + A rules
O: Independent
Proposals
A: Independent
Applications
Op
• ACT-R: O * A rules
11
Selection Knowledge Units
• In Soar, independent numeric indifferent rules combine values for decision
• Allows linear combinations of desirability
Qi
•Architecture
Qj
Qk
Architecture
Q
• In ACT-R, only a single utility value is associated with each rule
• No run time combination
• Conflates legality (proposal) and desirability
• Must have separate rule for each unique context application pair
12
Expressibility
• Soar allows “open decisions”
• Which knowledge contributes is determine at run time
• Does not require pre-compilation of important features.
• Separates knowledge about “can” do an action from “should”
• Makes easy to express and add knowledge to modify method
• Symbolic preferences
• Possibility of one-shot learning for decision making
• Can be told not to do an action (and overcome statistical)
• Can learn to not do an action
13
Use Run-time Context
• Generate alternatives
Soar
ACT-R
Yes – rules
Yes – rule conditions
• Compare/rate alternatives Yes – rules
No – rule utility
• Select
Architecture Architecture
• Apply
Yes – rules
No – rule action
14
Meta-Reasoning
• Soar has tie impasses & subgoals
• Can detect when knowledge is uncertain/incomplete
• Can use arbitrary reasoning to analyze and make decision
• Including look-ahead planning with hypothetical states
• Can return results that modify the decision
• Learning can directly modify decision
• ACT-R
• Difficult to detect uncertainty a & reason about decision
• Could create impasse when utilities are close or uncertain
• Difficult to modify decision without experience
• How could other reasoning change a production rule selection?
15
Predictions!
• ACT-R
• Something to deal with meta-cognition
• Detecting uncertainty and deliberate reasoning to deal with it (and the
learning).
• Planning
• Integration of emotion/pain/pleasure for learning
• Episodic memory
• Soar
• Long-term declarative memory & architectural declarative
learning
• Some one will build ASCOT-ARR!
• ACT-R memory structure with Soar operators
16
Gold and Coal
• Goal: Having alternative architectures
• Provides inspiration for architectural modification
• Provides comparison
• Forces us to examine arbitrary decisions
• Coal: Most comparisons to date are:
•
•
•
•
Informal (such as this)
Not theory directed (AMBER)
Confound programming & architecture
Not exactly same task
17