The Newell Test for a Theory of Mind
• Anderson, John R., & Lebiere, Christian
(forthcoming), “The Newell Test for a Theory of
Mind”, Behavioral & Brain Sciences
• Newell, Allen (1980), “Physical Symbol
Systems”, Cognitive Science 4: 135-183.
• Newell, Allen (1990), Unified Theories of
Cognition (Cambridge, MA: Harvard University
Press).
The Newell Test
• Based on lists of criteria that cognitive theories
should satisfy.
• Anderson & Lebiere:
– Classical connectionism:
• Newell-test strengths:
– Addresses empirical phenomena
– Especially in language, cognitive development
• NT weaknesses:
– Fails to acknowledge symbolic level of thought
– ACT-R (Anderson’s own theory):
• NT strengths:
– Includes symbolic & subsymbolic components
– Both are tightly integrated
• NT weaknesses:
– Lacks intensive analyses of some NT criteria
Newell’s Criteria: Summary
• 12 functional constraints on cognitive architecture
– First 9:
• Things needed “to implement human intellectual capacity”
– Last 3:
• “Constraints on how these functions are to be achieved”
• There are others, not on the list:
– E.g., a system could satisfy all 12, but “not correspond
to the human mind”
– So: could add others (but won’t):
• Behavioral adequacy:
– cognitive theory should correspond to details of human cognition
• Be capable of practical application (e.g., to education, therapy)
• There is a grading scheme!
Newell’s 12 Criteria
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
Flexible behavior
Real-time performance
Adaptive behavior
Vast KB
Dynamic behavior
Knowledge integration
Natural language
Consciousness
Learning
Development
Evolution
Brain
necessary
for
cognition
constraints
1. Flexible Behavior
• I.e., computational universality
• Grading:
– If theory is well-specified, then s/b easy to
determine whether computationally universal
– Doesn’t require that:
• people find everything equally easy
• human performance be error free
2. Real-Time Performance
• Theory should explain how humans can
satisfy criterion 1 in real time
• Grading:
– If theory has well-specified constraints on
speed of its processes, then s/b easy to
determine whether it can achieve real time for
specific cases of cognition
3. Adaptive Behavior
• Human computations are relevant to their needs
– How do basic processes of the architecture serve useful
functions?
– How is the whole system put together?
• Does its overall computation meet human needs?
• Grading:
– If theory is completely specified, then s/b able to
determine if its behavior would be functional in real
world
4. Vast Knowledge Base
• Distinguishes human cognition from expert
systems
– Problems:
• Not all K is equally reliable or relevant
• Relevance can change with time
• Storage/retrieval problems
• Grading:
– Determine how performance changes with scale of KB
– If theory is well specified, then s/b able to formally
analyze
– Size may affect performance
• Cf. learning names of 20 students vs. 200 students
5. Dynamic Behavior
• Precondition for survival:
– World can change in unexpected & uncontrollable ways
– Human actions can have unexpected effects
– Requires theories of perception, action, & cognition
• BUT: NOT(reaction is all there is to cognition)
• But also: NOT(cognition can ignore external world)
• Grading:
– Test system in an uncontrolled environment
6. Knowledge Integration
• I.e., achieving the capability to “combine
symbols intellectually” in order to make
inferences about the external world
• Grading:
– Can the theory deal with inference, induction,
metaphor, analogy
7. Natural Language
• Complete theory of mind must be able to
communicate in NL
– Newell: language depends on symbol
manipulation (! )
• Grading:
– Read a passage & answer questions about it.
• Requires parsing, comprehension, inference, use of
prior knowledge
• Conversational ability is nice, but not necessary
8. Consciousness
• Important for a full theory of human cognition
– [what about a full theory of cognition, simpliciter?]
• Grading:
– Ability to produce subliminal perception, implicit
learning & memory, metacognitive processes
– & show how they are functional aspects of human
cognition
9. Learning
• “Theory of cognition must account for
humans’ ability to acquire their
competencies”
• Grading:
– Semantic memory, episodic memory, skills,
priming, conditioning
• Based on Squire’s (1992) theory of memory in the
hippocampus
10. Development (constraint #1)
• Human cognition is constrained to unfold as
the organism grows & responds to
experience.
• Grading:
– Can the theory account for specific cases of
developmental progression?
11. Evolution (constraint #2)
• Evolutionary constraint:
– Human cognitive abilities evolved
• Comparative constraint:
– Differences between human and other-mammalian
cognition
• E.g., cognitive plasticity, language
• What’s unique to human cognition?
• Grading:
– How does the theory relate to these constraints?
12. Brain (constraint #3)
• Degree to which data about brain
functioning constrains the cognitive theory.
• Grading:
– Enumeration:
• Mapping of parts of the cognitive architecture onto
brain structures
– Proof:
• Brain computations match cog-arch computations
Grading
• BEST:
criterion on which theory has done the best
• Better:
• mixed:
4 criteria on which the theory has done better
2 criteria on which theory has most mixed
record
4 criteria on which the theory has done worse
criterion on which theory has done the worst
• worse:
•
worst:
Scoreboard
Conn’ism
mixed
ACT-R
Better
worse
BEST
SNePS
Mixed
Mixed
Adaptive
Vast KB
Dynamic
Kn. Integration
Better
worse
Better
mixed
Better
mixed
Worse
Better
Worse
Best
Nat. Lang.
Conscious
Learning
Better
worse
worse
worse
Better
Better
Better
Better
Better
Development
Evolution
Better
worse
worst
worst
Worse
Worst
Brain
BEST
worse
worse
Criterion
Flexible
Real-time
worse
mixed
Comparative Ranking
Grade
Connec’ism
ACT-R
BEST
BRAIN
REAL-TIME
Better
Learning
Learning
Adaptive
Adaptive
Development
Dynamic
Nat’l. Lang.
Flexible
flexible
k. integration
dynamic
vast k.b.
real-time
nat’l. lang.
vast k.b.
development
k. integration
brain
consciousness
consciousness
evolution
evolution
mixed
worse
worst
SNePS
KN. INTEG
Nat’l Lang
Conscious.
Vast KB
Learning
Flexible
Real Time
Dynamic
Adaptive
Developmt
Brain