1
The Mechanism of Thought
Confabulation Theory
Robert Hecht-Nielsen
University of California, San Diego
Confabulation Neuroscience Laboratory
r@ucsd.edu
http://r.ucsd.edu
Filmed at UCSD Atkinson Hall
Thursday 10 August 2006
•
This Kate Mark film was made in Atkinson Hall of the University of California, San Diego on
Thursday 10 August 2006 using video facilities of the California Institute of Telecommunications
and Information Technology. Special thanks to Doug Ramsey and Alexander Matthews of
CALIT2 for filming and editing this presentation.
•
The film is designed to be used in three ways:
1. As courseware for an introductory one-quarter or one-semester graduate or advanced
undergraduate course on confabulation theory.
2. As a vehicle for efficient professional self-study.
3. For use in connection with a concentrated two-day 16-class-hour introductory short-course
on confabulation theory.
•
For suggestions on use of this film, see the book Preface.
•
These presentation viewcell notes provide commentary and expansion upon topics briefly
discussed in the film. In many cases, the material provided in these notes does not appear
elsewhere in the book.
2
Overview
► Motivating Example
► Confabulation Theory Overview
– Cortical Representation of the Objects of the Mental World
– Acquisition and Storage of Cognitive Knowledge
– Confabulation
– The Origin of Behavior
– Confabulation Mathematics
► Confabulation Neuroscience
► Simulating Confabulation on a Computer
– A Single Confabulation
– Multiconfabulation
► Practical Applications
–
ChancellorTM Project
Chancellor is a trademark of Fair Isaac Corporation.
• After a brief motivating example, this presentation sketches confabulation theory
• Confabulation neuroscience (study of the implementation of the functional elements of
confabulation theory by neurons in the human cerebral cortex, thalamus, and other brain nuclei)
is briefly discussed next.
• The presentation then turns to computer simulations of confabulation. First, single confabulations
are carried out. Then, ensembles of temporally-overlapping, dynamically interacting, converging
confabulations (multiconfabulations) are considered.
• Finally, prospects for practical applications of confabulation theory are discussed.
Add a Plausible Next Sentence to Two Given News Story
Context Sentences
Several other centenarians at Maria
Manor had talked about trying to live
until 2000, but only Wegner made it.
3
Her niece said that Wegner had always been
a character – former glove model , buyer for
Macy's, owner of Lydia's Smart Gifts
downtown during the 1950s and '60s – and
that she was determined to see 2000 .
second context
sentence
first context
sentence
confabulation
architecture
(exposed to millions of news
stories, NO: algorithms, rules,
Bayesian networks, etc.)
plausible next sentence
generated by the architecture
She was born in the Bronx Borough of New York City.
© 2006 Fair Isaac Corporation.
Context sentences from the Detroit Free Press
• As with a human child, this ‘purple box’ has no software (other than the simulations of the brain
structures used in its construction), no algorithms, no rules, no ontologies, no n-grams, no fuzzy
logic, etc.
• The box’s knowledge is based entirely on noting meaningful pairwise word and phrase cooccurrences in text examples (consecutive sentence triples from well-written English
paragraphs) that it has been exposed to. Each of these is converted into a knowledge link.
• Consider five questions regarding the capabilities demonstrated by this purple box:
1. What must the box know about the crafting of English sentences?
2. How extensive must its knowledge of the world be?
3. What percentage of the American population could outperform this box (say, with a blind
jury grading the generated sentences)?
4. How much more capability, if any, must be added to the box before you would declare that
the purple box possesses a type of “true intelligence”?
5. What do you think the reaction of the attendees at the 1956 Dartmouth University meeting
(where the field of AI was formally founded) would have been to this purple box?
Sentence Generation Confabulation Architecture
4
confabulation
architecture
(exposed to
millions of news
stories, NO:
algorithms, rules,
ontologies,
Bayesian
networks, etc.)
First Context
Sentence
Second Context
Sentence
Third Sentence
and
Generated Sentence
Michelle strengthened from a Category 2 to a Category 4 storm Saturday, with winds reaching
140 mph, but it was expected to weaken before it reached Florida.
The storm or its effects could strike the Keys and South Florida tonight or early Monday, said
Krissy Williams, a meteorologist at the National Hurricane Center in Miami.
Forecasters warned residents to evacuate their homes as a precaution.
© 2006 Fair Isaac Corporation.
Context sentences from the Detroit Free Press
• Note that the same confabulation architecture used to carry out the learning process is later
used to carry out sentence generation. In general, this is how humans acquire skills. While
there are sometimes ways around this limitation (e.g., reasoning can sometimes be used –
particularly when augmented by paper & pen or Microsoft Word – to carry out novel types of
cognition); most human mental capabilities are of this “monkey-see / monkey-do” type. This
points up the enormous value of the accumulated heritage of skills that the human species has
created (many of which, such as minor languages, folk medicines, obscure juggling / acrobatic /
“magic tricks”, and social traditions / rituals, have already been lost).
• The thought processes used to generate confabulation architecture outputs (represented
here by the swirling red arrow – see Chapter 6) are usually simple and of a standardized type –
making learning of these relatively easy. The examples created by experts are indispensable.
• The confabulation architecture used here possesses over 10 billion items of knowledge. It is
this vast amount of knowledge – and the fact that confabulation architectures automatically
apply large quantities of relevant knowledge in parallel during thinking – that help account for
the enormous power and flexibility of thought. Another key factor is the interoperability of the
pairwise symbol co-occurrence knowledge used in confabulation (see Chapter 3).
Q: Why Do Confabulation Architectures Work?
A: An Underlying Mathematical Principle
5
Theorem The Fundamental Theorem of Cognition : Given
non-exceptional assumed facts α, β, γ, and δ, and expectation
element ε, then the following exact relationship holds between
cogency p(αβγδ|ε) and the confabulation product p(α|ε) · p(β|ε)
· p(γ|ε) · p(δ|ε):
[p(αβγδ|ε)]4 = [p(αβγδε)/p(αε)]
neurons in cerebral
cortex and thalamus
· [p(αβγδε)/p(βε)]
rapidly and accurately
cogency: the
· [p(αβγδε)/p(γε)]
compute this quantity
universal
·
[p(αβγδε)/p(δε)]
decisionmaking
criterion of
· [p(α|ε) · p(β|ε) · p(γ|ε) · p(δ|ε)] .
█
cognition – an ideal
but uncomputable
quantity
≈ C · [p(α|ε) · p(β|ε) · p(γ|ε) · p(δ|ε)] .
• Confabulation architectures operate on a mathematical principle which was needlessly
overlooked for decades for interesting sociological reasons (see Chapter 3).
• Confabulation theory hypothesizes that the mathematical principle underlying cognition is
cogency maximization.
• Confabulation theory posits that brains explicitly evolved to exploit a mathematical identity (the
Fundamental Theorem of Cognition – see Chapters 3 and 4); which allows the (inherently
slow) neurons within human thalamocortical modules (and their neuroanatomical functional
analogs in other animals) to approximately carry out cogency maximization at blazing speed, via
massively parallel processing.
• Unlike past theories of brain function, confabulation theory is a falsifiable scientific theory. It is
either roughly correct, or it is wrong. So far, the theory seems consistent with the known facts of
neuroscience.
Overview
6
► Motivating Example
► Confabulation Theory Overview
– Cortical Representation of the Objects of the Mental World
– Acquisition and Storage of Cognitive Knowledge
– Confabulation
– The Origin of Behavior
– Confabulation Mathematics
► Confabulation Neuroscience
► Simulating Confabulation on a Computer
– A Single Confabulation
– Multiconfabulation
► Practical Applications
–
Chancellor Project
• An overview of confabulation theory is now presented.
• Confabulation theory has four key elements. These are examined first (see Chapter 1 for an
overview of these elements).
• After the key elements of the theory have been presented, the mathematics underlying
confabulation theory is briefly discussed (see Chapters 3 and 4 for more details).
• This overview of the theory covers a lot of ground and introduces four complicated and
unfamiliar constructs (the key elements of confabulation theory). Chapters 3, 5 and 8 provide an
expanded discussion of this material from the standpoint of confabulation neuroscience.
• Unlike ordinary neuroscience – wherein most textbooks are in their second or higher editions
and where the material presented has undergone many years of intensive scrutiny and validation
– confabulation theory is wild new stuff. Almost nothing in this book has undergone careful
scrutiny, replication, and criticism. Not enough time has elapsed. Welcome to the cutting edge.
• The main premise of this book is that the world wants the information contained herein to be
made available as rapidly as possible; even if it is in an unpolished state. The sense is that there
exists a large audience that is eager to obtain, evaluate, and build upon, these discoveries.
7
Confabulation Theory Summary
+
+
+
1
2
3
4
+5
6
7
8
9
10
11
.
.
.
+
+126,007
126,008
+
I(4)
I(9)
I(126,007)
‘answer’
• Confabulation theory is concrete and specific. The theory hypothesizes that human cerebral cortex
(viewed by many as the pinnacle of the brain’s evolutionary hierarchy of function) is comprised of
about 4,000 separate ‘dumb’ information processors (termed thalamocortical modules) that
operate (exactly like a muscle) only when they receive an explicit external thought command –
commands which originate from the same mechanisms which control muscles. Movement and
thinking are siblings. This view conflicts starkly with a currently popular notion that cortex is a vast
interwoven mish-mash of interacting, continuously free-running, information processing activities. In
essence, cerebral cortex is comprised of thousands of discrete, deliberately controlled muscles of
thought.
• Confabulation theory hypothesizes that each thalamocortical module’s function is centered around a
collection of thousands of symbols (each represented by a collection of about 60 special neurons),
and the view that there is one and only one information processing operation used in cognition: a
trivial type of highly parallel ‘sort’ operation among symbols.
• Despite apparent dissonance with some neuroscience opinion, confabulation theory seems
compatible with both the facts of neuroscience and with many existing conceptual models of neuron
function. See Chapters 5 and 8 for details.
8
Confabulation Theory Summary
thalamocortical
module
+
+
+
thought
command
signal
1
2
3
4
+5
6
7
8
9
10
11
.
.
.
+
+126,007
126,008
+
I(4)
I(9)
► Representation of the
Objects of the Mental World
► Acquisition and Storage of
Knowledge
► A Universal Information
Processing Operation:
Confabulation
► Each Confabulation
Conclusion Triggers a
Behavior
I(126,007)
confabulation
conclusion
• Here the four key elements of confabulation theory are listed.
• The consistency of these elements with the known facts of neuronal function is noted.
• Section 1.2 provides an introductory discussion of each of these four functions.
Each Thalamocortical Module is Responsible for
Describing One Attribute, Which Objects
in the Mental Universe May Possess
cortical patch
(about 45 mm2)
9
A
a thalamocortical module
(one of about 4,000)
B
cerebral cortex
reciprocal
axonal
interconnections
thalamus
uniquely paired
first-order
thalamic zone
CONFABULATION THEORY KEY ELEMENT 1
• Each thalamocortical module includes a small patch of cerebral cortex, a small zone of first order
thalamus (see Section 1.2.1 and Chapters 3, 5 and 8), and the reciprocal axonal connections
which link them.
• There is strong neuroscience evidence of many kinds supporting the existence of thalamocortical
modules.
• Thalamocortical modules are largely disjoint from one another and, together, their cortical
patches tile all of cerebral cortex.
• The average human brain possesses roughly 4,000 thalamocortical modules. The cortical patch
of the average module occupies roughly 45 mm2 of cortical area (out of a total of roughly 180,000
mm2).
• Each thalamocortical module possesses roughly 4 million neurons, of which a special subset of
about 400,000 neurons (perhaps located in cortical layers II and III) are used to represent
symbols. These symbol-representing neurons are the colored dots illustrated in inset A.
• While in this presentation the cortical patches of thalamocortical modules will be illustrated as
convex planar areas (inset A), in the brain, they are probably actually irregularly shaped, as
shown in inset B.
Each Thalamocortical Module is Responsible for
Describing One Attribute, Which Objects
in the Mental Universe May Possess
Symbol 1
cortical patch
of module
cerebral cortex
thalamocortical
module
1
2
3
4
5
6
7
8
9
10
11
.
.
.
Symbol 2
CONFABULATION THEORY KEY ELEMENT 1
10
Symbol 126,008
126,007
126,008
• Each module represents a single attribute that an object (sensory, language, abstract,
movement, thought, plan, etc.) of the mental universe may possess (see Section 1.2.1).
• Each module implements a large set of discrete symbols (this one has 126,008); each
represented by a collection of roughly 60 neurons of the symbol-representing neuron population.
• If an attribute of an object is to be described at a particular moment; typically one symbol of that
module is activated (‘primary’ modules such as those in V1 and M1 typically express multiple
symbols). Symbols are mostly formed in childhood.
• Symbols are the stable terms of reference that must exist if knowledge is to be accumulated
and used over long periods of time.
• The red box at the lower left illustrates two additional graphical representations for a
thalamocortical module: a dashed oval with a linear list of symbols inside it (for use when
individual symbols need to be referred to) and a simple square (for use where the internal details
are not being referred to). Sometimes, a number is placed within a such a square to indicate how
many symbols are currently being considered as viable conclusions (this is explained further
below).
An Individual Knowledge Link Unidirectionally
Connects a Source Symbol to a Target Symbol
symbol
(neuron
collection)
representing word
apple
thalamocortical module
representing words
describing mental
world objects
Knowledge links are formed
between meaningfully
co-occurring symbols,
essentially as postulated by
Hebb
CONFABULATION THEORY KEY ELEMENT 2
unidirectional
neuron collection-to-neuron
collection knowledge link
11
symbol
(neuron collection)
representing color red
Cerebral Cortex
thalamocortical module
representing colors
describing mental world
objects
• Confabulation theory postulates that all cognitive knowledge takes the form illustrated here: a
pairwise unidirectional axonal knowledge link from one symbol (termed the source symbol of
the knowledge link) to another (termed the target symbol of the knowledge link). Section 1.2.2.
• The average adult human is postulated to possess billions of knowledge links.
• Knowledge links are formed (assuming the required axons are genetically provided) whenever
two symbols meaningfully co-occur.
• Here, a knowledge link is formed because a symbol representing the name attribute of a
particular object (a red apple being viewed) and a symbol representing the visual color of that
same object meaningfully co-occur.
• Knowledge links typically form temporarily every time two symbols are co-active for the first time
and those symbols are used – and thus co-occur again – a moment later.
• If, during the 100 hours or so after a knowledge link is temporarily formed, this link is deemed to
have been associated with a drive or goal state reduction (the hippocampus is responsible for
keeping track of this) – then the link is solidified and made permanent.
• Solidified knowledge links can last for many decades – particularly if they are sometimes used.
A Mental World Object is its Collection of
Attribute Descriptors
12
symbol representing
apple skin texture
symbol
representing color red
symbol representing
word apple
The Average Human
Possesses
Billions of Items of Knowledge
Cerebral Cortex
symbol
representing apple
odor and taste
CONFABULATION THEORY KEY ELEMENT 2
symbol representing
apple chewing
motor behavior
• As illustrated here, a mental world object (here, a red apple) usually has many (but rarely all) of
its attributes simultaneously being described by excited symbols. See Section 1.2.2.
• All pairs of these symbols for which the required axons exist will be connected with knowledge
links in both directions. This illustrates one reason why the number of knowledge links is so large.
• Symbols describing attributes of different objects that frequently appear together also become
linked pairwise.
• Astoundingly, confabulation theory hypothesizes that this simple type of knowledge IS THE
ONLY TYPE OF COGNITIVE KNOWLEDGE THAT EXISTS! All aspects of cognition (seeing,
hearing, planning, reasoning, language understanding and generation, triggering of movement
and thought processes, etc., etc.) are carried out using nothing but these knowledge links.
• Because (generally) all of the attribute descriptor symbols of a mental world object are connected
pairwise by knowledge links; von der Malsburg’s famous “binding problem” does not apply to
confabulation theory. Relevant sets of attribute descriptor symbols are ‘pre-bound’.
• Notice that knowledge links often bridge between representations having radically different
characters (visual, motor, somatosensory, linguistic, etc.). Knowledge links are interoperable.
Two-Stage Abeles Synfire Chain Knowledge Link
Implementation
transponder
neurons
13
these synapses
are strengthened
target
module
symbol λ
Cerebral
Cortex
source
module
symbol ψ
CONFABULATION THEORY KEY ELEMENT 2
• Although human cerebral cortex has hundreds of trillions of axon collaterals (and terminal
synapses), it is still very unlikely that the neurons which represent the source symbol of a
knowledge link will synapse directly with the neurons which represent the target symbol of that
link.
• However, if the outputs of the source symbol neurons can be sufficiently strong and
synchronized, an intermediate population of transponder neurons can become highly excited by
a symbol being expressed by a source module.
• This set of highly excited transponder neurons is much larger in number than the 60 neurons of
the symbol itself (e.g., a crude human cortex simulation indicates that each symbol might have
900 transponder neurons – see Chapter 3).
• The 900 transponder neurons can then complete the communication of the knowledge link by
sending signals to a sufficiently large subset of the neurons which represent the target symbol.
• This kind of multi-step signaling process was vaguely proposed by Hebb in 1949. But it was
Abeles in 1991 who provided the first detailed study of such multi-stage neuronal-population-toneuronal-population signaling structures – which he dubbed “synfire chains”. See Chapters 3, 5
and 8.
Confabulation is a Winners-Take-All
Competition Between the Symbols of a Module Based upon
14
each Symbol’s Summed Input Excitation
β
α
ε
γ
δ
+
+
+
CONFABULATION THEORY KEY ELEMENT 3
1
2
3
4
+5
6
7
8
9
10
11
.
.
.
+
+126,007
126,008
+
I(4)
I(9)
I(126,007)
• Confabulation theory contends that cognition employs only one information processing operation:
a simple winners-take-all competition called confabulation. See Section 1.2.3.
• The symbol which, at the time of the confabulation, is receiving the highest level of summed
knowledge link excitation ‘wins’ the competition and is declared the confabulation’s conclusion.
• All of the 60 neurons which represent the conclusion symbol become activated and all other
symbol-representing neurons of that module are not activated.
• An activated symbol can, for a brief time, transmit excitation through knowledge links for which it
is the source symbol (assuming that the knowledge bases in which those knowledge links reside
are currently enabled). For example, here, symbols α, β, γ, and δ are the conclusions of
confabulations which recently occurred on the four modules depicted on the left.
• Assuming not too much time has elapsed (a few tens of seconds), the last confabulation
conclusion of a module can be reactivated by simply commanding a new confabulation on that
module with no external knowledge link inputs. This capability is termed working memory.
• In effect, cognition proceeds forward throughout the day; using knowledge links from the
conclusions of past confabulations to feed new confabulations.
Each Module Receives a Thought Command Input,
Which Causes the Module to Implement Confabulation
15
externally
supplied
thought
command
signal
t 0 + 80 ms
t0
t 0 + 40 ms
CONFABULATION THEORY KEY ELEMENT 3
Confabulation is a
fast, parallel,
‘winners-take-all’
competition between
the module’s symbols
based upon their
summed knowledge
link input excitations
• The challenge of implementing confabulation is that potentially huge numbers of partially excited
symbols must somehow compete with one another over a very short period of time. In the end,
the neurons that represent the winning symbol become activated and the other symbolrepresenting neurons are not. See Section 1.2.3 and Chapters 3, 5, and 8.
• Here, in the upper left illustration, four symbols (represented by red, blue, green, and black
neurons, respectively) have, at the starting time of the confabulation t0, some of their neurons
excited by the action of incoming knowledge links from the assumed facts being used. 40
milliseconds later, as illustrated in the middle illustration, the red neurons (representing the
symbol which is receiving the most knowledge link excitation) are rapidly becoming activated and
the neurons of the ‘losing’ blue, green, and black symbols are rapidly being shut down.
• At the end of the confabulation, illustrated at the upper right 80 milliseconds after the start, all the
red neurons (those representing the conclusion symbol) are maximally activated and all the other
symbol-representing neurons are not activated.
• Given the slow dynamics of individual neurons, for this competition process to be finished in 80
milliseconds, there must be massively parallel interactions (e.g., within-symbol mutual excitation
and between-symbol mutual inhibition) between the neurons of different symbols.
Thalamocortical Modules Function as the Muscles of
Thought – when Deliberately Commanded, they
Implement Confabulation
a knowledge base
(one of tens of
thousands)
16
a thalamocortical module
(one of thousands)
the muscles of thought
module control signals
(akin to motorneuron outputs)
• Confabulation theory views cerebral cortex and thalamus as implementing 4,000 discrete,
individually controlled, processing modules. Unidirectionally connecting genetically-selected pairs
of these modules are, individually enabled, 40,000 knowledge bases.
• A thought process is a properly timed and coordinated sequence of deliberate ‘contractions’
(confabulations) of modules (and enablements of the required knowledge bases).
• Thalamocortical modules thus function as the muscles of thought.
• The sequences of contractions and enablements used in thought processes are learned, stored,
and recalled in exactly the same manner as are the postural goals of movement processes.
• Movement and thought are siblings.
• See Section 1.2.3.
17
The Conclusion-Action Principle:
The Origin of Behavior
Whenever a confabulation yields a conclusion, associated action commands are
immediately issued. Action commands cause motor and/or thought processes (many
per second).
thalamocortical
module
thought
command
signal
1
2
3
4
5
6
7
8
9
10
11
.
.
.
126,007
126,008
skill
knowledge
9
action command outputs
(these proceed to subcortical
motor and thought nuclei)
confabulation
conclusion
CONFABULATION THEORY KEY ELEMENT 4
• Confabulation theory hypothesizes that all non-reflexive and non-autonomic behavior arises from
the action commands which are issued each time a module confabulation yields a conclusion.
• Each symbol of each module has a set of action commands associated from it. These
associations are termed skill knowledge. See Section 1.2.4.
• Skill knowledge is established by subcortical structures (e.g., the basal ganglia), but it is actually
stored in cerebral cortex.
• Whenever a symbol becomes activated, its associated action commands are instantly launched.
This is the theory’s Conclusion → Action Principle.
• Action commands can cause behaviors to occur (sometimes instantly, sometimes after a delay).
• Thought and movement processes are stored in the form of hierarchically-organized
spatiotemporal action symbol sequences. For flexibility, most individual sequences are of short
duration.
• The expression of such an action symbol sequence causes launch of an associated action
command sequence. Such sequences can result in movement and thought.
• Skill knowledge is not formally part of cognition. It comes into play only when thinking is over.
18
The Mathematics of Confabulation
Given four assumed fact symbols α, β, γ, and δ (each being expressed on its own
separate module), confabulation theory proposes that confabulation finds that
conclusion symbol ε having maximum cogency p(αβγδ|ε) (where juxtaposition
indicates Boolean AND)
CONFABULATION THEORY MATHEMATICS
β
α
ε
γ
δ
Confabulation produces that conclusion which, if assumed true, is most supportive
of the probability of the assumed facts being true
• What makes confabulation powerful? In particular, what makes it a general-purpose
decisionmaking procedure? Confabulation theory hypothesizes that, because of a particular
mathematical fact (called The Fundamental Theorem of Cognition), the conclusion reached by
a confabulation approximately maximizes cogency.
• A conclusion which maximizes cogency is the “optimal answer” in the sense that it is, among all
of the possible conclusions (the symbols of the confabulating module), the one that is maximally
supportive of the probability of truth of the set of assumed facts being used in the confabulation.
Confabulation theory hypothesizes that cogency maximization is the universal principle upon
which all animal cognition is based.
• Thus, confabulation is a universal information processing procedure that can be applied in a wide
variety of situations to discover the ‘best’ conclusion.
In a ‘Logical’ Information Environment,
Confabulation Produces Logical Conclusions
19
Theorem 1: In an Aristotelian logic information
environment, if αβγδ ⇒ ε uniquely then ε uniquely
maximizes cogency p(αβγδ|ε).
Thus, when doing mathematics or playing chess,
confabulation will produce logical answers.
CONFABULATION THEORY MATHEMATICS
• When the assumed facts, knowledge links, and possible conclusions involved in a confabulation
come from a ‘logical’ information environment, this theorem shows that cogency maximization will
yield the same conclusion as logical implication (assuming the conclusion is unique).
• Thus, when Aristotelian logic applies, confabulation yields logical conclusions of the assumed
facts.
• When Aristotelian logic does not apply (e.g., when you are parking your car), confabulation still
works: it yields a conclusion which approximately maximizes cogency.
• Clearly, this is the fundamental origin of the notion of ‘fuzziness.’ When possible, confabulation
yields logical results. When logic does not apply, confabulation yields a ‘best
approximation’ to logic.
• An interesting research question is to see if the mathematics of cogency maximization yields
conclusions which formally obey something like the mathematics of fuzzy theory.
• What is already known (see Chapter 3) is that, in general, cogency maximization does not yield
“Bayesian” (maximum a posteriori probability) conclusions. The widespread assumption that
cognition must be Bayesian probably delayed the discovery of confabulation theory for decades.
The Mathematics of
Additive Knowledge Combination
20
Theorem 2 The Fundamental Theorem of Cognition:
Given non-exceptional assumed facts α, β, γ, and δ, and
expectation element ε, then the following exact relationship holds
between cogency p(αβγδ|ε) and the confabulation product
p(α|ε) · p(β|ε) · p(γ|ε) · p(δ|ε):
[p(αβγδ|ε)]4 = [p(αβγδε)/p(αε)]
· [p(αβγδε)/p(βε)]
· [p(αβγδε)/p(γε)]
· [p(αβγδε)/p(δε)]
· [p(α|ε) · p(β|ε) · p(γ|ε) · p(δ|ε)] . █
≈ C · [p(α|ε) · p(β|ε) · p(γ|ε) · p(δ|ε)] .
CONFABULATION THEORY MATHEMATICS
• Confabulation theory postulates that neurological evolution discovered neuroanatomical designs
hundreds of millions of years ago that would cause the terms in green to be approximately
constant for all highest-cogency conclusions ε.
• These highly-conserved neurophysiological designs allow confabulation product maximization
(maximization of the quantity in purple) to approximate cogency maximization.
• Confabulation theory hypothesizes that confabulation product maximization can be carried out
accurately and rapidly by the neurons within a thalamocortical module utilizing input excitations
delivered by knowledge links.
• Each knowledge link has a ‘weight’ which approximately implements a logarithmic function of the
antecedent support probability p(σ|τ) for its source and target symbols σ and τ, respectively.
• Since the logarithm of a product is the sum of the logarithms of its terms; the sum of the
excitations being delivered to a symbol by knowledge links is thus related to a logarithmic
function of the confabulation product (see notes for next viewcell).
• To keep the exposition simple, assumed fact symbols are herein taken to have unit outputs.
• See Chapters 3, 4, 6, and 7 for more details.
The Mathematics of
Additive Knowledge Combination
21
Thus, to maximize cogency p(αβγδ|λ), we can instead find that
target symbol λ which maximizes:
I(λ) ≡ [loga(p(α|λ)/p0) + b]
+ [loga(p(β|λ)/p0) + b]
+ [loga(p(γ|λ)/p0) + b]
+ [loga(p(δ|λ)/p0) + b] .
Note: if a = 1.5849 and b = 30, then the required “synapse strengths” are
between 30 and 50 – a small dynamic range.
unstrengthened
final knowledge link
vestigial synapse (99%?)
strength: 1
strengthened
final knowledge link
synapse (1%?)
strength range: 30 - 50
incoming axon from
transponder neuron
incoming axon from
transponder neuron
CONFABULATION THEORY MATHEMATICS
A
target symbol
neuron dendrite
B
• By hypothesis (see previous viewcell), we can maximize cogency by maximizing the
confabulation product. Taking logarithms and adding a positive constant b to each term gives us
the input excitation I(λ) for symbol λ. Maximizing I(λ) is clearly mathematically equivalent to
maximizing the confabulation product. See Chapters 3, 5, and 8.
• Each term in the sum I(λ) corresponds to one knowledge link.
• Note that (as with the values cited in this viewcell) constants a and b can be chosen so that each
individual logarithm has a value that is greater than or equal to b. This means that the value of
I(λ) ‘counts’ the number of incoming knowledge links. For example, a symbol λ with three
knowledge link inputs will always have an input intensity I(λ) that is smaller than a symbol with
four knowledge link inputs. This ensures that only symbols having the maximum number of
knowledge link inputs can win the confabulation competition.
• The thought control input signal can be manipulated so that only symbols with a particular
minimum number of knowledge link inputs can win (see Chapter 7). If there are no such symbols,
then the confabulation does not yield a winning symbol (this is termed a null conclusion).
• In brains, this mathematics implies that solidified knowledge link synapses are all very strong.
Confabulation Characteristics
22
► Confabulation is a fast, neuronally implementable, decisionmaking operation for finding the ‘best’ (maximum cogency)
conclusion to a universally-applicable type of probabilistic
‘question.’
► Confabulation is postulated to be the underlying mechanism of all
aspects of animal cognition (seeing, hearing, movement and
thought process origination, planning, reasoning, language, etc.,
etc.).
► Multiconfabulations (multiple, temporally overlapping, mutually
dynamically interacting, confabulations) are the norm in cognition.
CONFABULATION CHARACTERISTICS
• Confabulation theory hypothesizes that the universal basis for cognitive information processing is,
given knowledge link inputs from a particular set of assumed facts (each the conclusion of a
previous confabulation), finding that conclusion which has the maximum cogency.
• While neuronal implementation of cogency maximization is only approximate, this underlying
mathematics still works, because of a number of favorable factors. [e.g.: In some cognitive arenas,
confabulations often have many possible candidate conclusions with very similar near-maximum
levels of cogency – making it unimportant exactly which of these is selected. Confabulations in
certain other cognitive arenas generally have only one conclusion that receives the maximum
number of knowledge links – making it unnecessary for links to even have weightings., etc.]
• As will be seen in the experiments below, a key aspect of cognition is the ubiquity of thought
processes in which multiple contemporaneous, mutually interacting, confabulations
(multiconfabulations) take place. Multiconfabulation is much more powerful than a single
confabulation because it allows massive parallel application of relevant knowledge.
• Another key strength of confabulation is the interoperability of knowledge links. A knowledge link
from a visual attribute symbol to a language attribute symbol has the same effect as one from
another language attribute symbol. This makes symbols and knowledge a universal language.
Overview
23
► Motivating Example
► Confabulation Theory Overview
– Cortical Representation of the Objects of the Mental World
– Acquisition and Storage of Cognitive Knowledge
– Confabulation
– The Origin of Behavior
– Confabulation Mathematics
► Confabulation Neuroscience
► Simulating Confabulation on a Computer
– A Single Confabulation
– Multiconfabulation
► Practical Applications
–
Chancellor Project
• In this section of the presentation, human confabulation neuroscience research (study of the
implementation of the functional elements of confabulation theory by neurons in the human
cerebral cortex, thalamus, and other brain nuclei) is briefly overviewed.
• Unfortunately, neuroscience knowledge has advanced slowly, and surprisingly little is actually
known for sure about how neurons in the cerebral cortex and thalamus function and interact.
• One of the hopes of confabulation neuroscience is that having the concrete description of overall
cortical and thalamic function provided by confabulation theory will provide an intellectual
framework for learning more about function at the neuron level.
• At the moment, confabulation neuroscience research is focused on creating an up-to-date,
comprehensive compilation of recent high-quality primary neuroscience studies for human
cerebral cortex and thalamus; with an eye towards explaining the four key functional elements of
confabulation theory at the neuron level.
• Another key methodology is to look across species – particularly higher mammals. Since all
these species have considerable cognitive capability, they all presumably implement the
elements of confabulation theory. These comparisons may be useful in spotting conserved
functional designs that underlie implementation of the key elements of confabulation theory.
24
Vertebrate Brain Archetype
Trout
Telencephalon
(cerebral cortex,
hippocampus,
amygdala, etc.)
Diencephalon
(thalamus,
hypothalamus,
subthalamus,
retina, etc.)
Raven
Human
Rhombencephalon
(cerebellum,
locus coeruleus, etc.)
Mesencephalon
(superior colliculus,
inferior colliculus,
oculomotor nucleus,
ARAS, etc.)
Spinal Cord
(ascending fascicles,
descending fascicles,
gray matter, etc.)
G. F. Striedter (2005) Principles of Brain Evolution, Sunderland, MA: Sinauer Associates.
• An important conclusion that is emerging from neuroethology (comparative studies of the
nervous systems of different species) is that all vertebrate species (with a small number of
strange exceptions) seem to have the same basic brain design; as shown here.
• Since all vertebrates seem to have some cognitive capabilities, it is reasonable to suppose that
the key functional elements of confabulation theory are present in every specific embodiment of
this vertebrate brain design archetype.
• This presumed unity of cognitive function makes perfect sense from an evolutionary standpoint;
since the success of vertebrates is uniformly strongly attributable to their cognitive power.
• Notwithstanding that our most recent common ancestor lived perhaps half a billion years ago,
trout, ravens, and humans each share in the triumph and legacy of the original cognizing species.
• Out running in the morning I talk at the ravens and bluejays; thinking that they might like to land
on my shoulder, share a distant-cousin kiss, and maybe reminisce about grandma and grandpa
from the Triassic. No takers yet.
25
Cortex and Thalamus Implement Cognition
• Today, confabulation theory is confined to explaining The Mechanism of Thought (i.e.,
cognition), hence the title of this book.
• This mechanism is implemented entirely within cerebral cortex and thalamus.
• However, cognition interacts extensively with, and depends critically upon, many other brain nuclei
and processes.
• It is important to realize that these ‘external’ interactions are, with the exception of some aspects
of the Conclusion → Action Principle, totally ignored in this book.
• This viewcell and the next two are intended to make the point that these external interactions are a
crucially important collection of topics that, using confabulation theory as a base, can probably
now be extensively investigated.
Today’s Confabulation Architectures Must Be Controlled
by Contrived External Thought Commands
26
thought
controller
• The thought command signals that cause thalamocortical modules to carry out confabulation, and
which enable knowledge bases, are provided by subcortical mechanisms and structures that are
not discussed in this book.
• Thus, when we wish to conduct experiments with computer-simulated thalamocortical modules
and knowledge bases, we have to create some sort of ‘external thought controller’ as a crude
substitute for these many missing brain nuclei.
• Fortunately, for the confabulation architectures (ensembles of thalamocortical modules and
knowledge bases) discussed in this book, the thought control procedures are relatively simple;
making it possible to get interesting results using a crude external thought controller.
• From a neurotechnology standpoint, such a makeshift approach is not necessarily all that bad. For
example, aviation flourished for decades using heavy, unreliable, low power output, piston engines
before the widespread arrival of turbine engines; and electronics was a large industry for decades
using vacuum tubes before the widespread arrival of transistors.
Full Brain Function Requires Many Additional Nuclei:
There is Much More to Learn
27
caudate nucleus
putamen
globus pallidus
subthalamus
locus coeruleus
substantia nigra
red nucleus
pons
cerebellum
hippocampus
amygdala
centromedian TN
etc.
• While it may be possible for practical applications of confabulation theory to get by for a few years
with use of contrived thought controllers, this approach is fundamentally limited in its capabilities.
• The advancement of both neuroscience and neurotechnology demand investigation of the
extensive interaction of the cognitive processes of cerebral cortex and thalamus with the
information processing functions carried out by the brain nuclei listed on the right side of this
viewcell as examples (and many others).
• Research of this sweeping multinuclear type, extending across many brain nuclei in an integrated
fashion, is not common in today’s neuroscience. New social norms, methodological approaches,
and talents will have to be developed to enable such research. This will take time.
• Surprisingly, even though the detailed functioning of the other nuclei of the human brain remains
mysterious, it is possible to hypothesize the general information processing function of some of
them. For example, the striatum (including the caudate nucleus, putamen, and globus pallidus)
may function as an adaptive switch (connecting action command outputs from thalamocortical
modules to specific behavior evaluation devices) and a suggested behavior evaluator (if a
particular suggested behavior has previously been associated with a successful reduction in a
currently active drive or goal state, it is sent back to cortex for execution).
UCSD Confabulation Neuroscience Laboratory
28
► The detailed neuronal implementation of thalamocortical module functions
is not known.
► Research focus: How the neurons of human cerebral cortex / thalamus
implement the functional elements of confabulation theory (symbols,
knowledge links, confabulation, behavioral triggering).
► Methodology: Collection of relevant high-quality neuroscience research
findings. Unified graphical representation of collected findings. Iterative
development, evaluation, and improvement of biologically realistic
computer models of functional neuronal structures based on the graphically
illustrated findings.
► Strong interaction with the La Jolla neuroscience community.
► Lab sponsor: Office of Naval Research
► Graduate Student Researcher: Soren Solari
• The UCSD Confabulation Neuroscience Laboratory is developing a Confabulation Neuroscience
Knowledge Environment resource tailored to the needs of neuron-level modeling of cortical and
thalamic cognitive function.
29
Thalamocortical Module Neuroanatomical Models
I2/3
I2/3
I2/3
P1/2/3
P1/2/3
P1/2/3
I4
I4
I4
L4
P4
P4
P4
I5
L5
Lower laminar
differentiation
Higher laminar
differentiation
A1/V1
L1
L2
L3
I5
I5
P5
P5
L6
P5
P6
P6
I6
P6
I6
I6
Basal
Ganglia
Basal
Ganglia
BG
TRN
Thalamus
TRN
Parvalbumin
(FO,SP)
Thalamus
TRN
Calbindin
(HO,NSP)
Thalamus
Calbindin
(HO,NSP)
Thalamus
Senses
L=Layer
excitatory
Inhibitory
Graphical Neuroanatomical Representation courtesy of Soren Solari, 2006
• Graphical representations (such as this early prototype) play a dominant role in the CNKE; as do
extensive notes with relevant literature citations.
UCSD Confabulation Neuroscience Course
30
► ECE-270A/B/C Neurocomputing is a year-long (three quarter
sequence) course covering Confabulation Neuroscience Modeling.
► ECE-270 participants build, evaluate, and present to the class, models of
thalamocortical modules, knowledge links, behavioral triggering, skill
learning, and reasoning capability acquisition.
► ECE-270A concentrates on learning Confabulation Theory and
experimenting with Thalamocortical Module Models 1, 2, 3, and 4.
► ECE-270B concentrates on experimenting with Thalamocortical Module
Models 5 and 6 and Knowledge Link Models A, B, and C.
► ECE-270C concentrates on Behavioral Triggering Model Alpha and
experiments with Skill Learning and Reasoning Capability acquisition.
► ECE-270 is the development platform for a possible future
Confabulation Neuroscience textbook.
• ECE-270 is focused on building, running, and evaluating mathematical neuronal models of the key
elements of confabulation theory.
• Course participants participate in expanded exploration of existing models and in the development
of new models.
Overview
31
► Motivating Example
► Confabulation Theory Overview
– Cortical Representation of the Objects of the Mental World
– Acquisition and Storage of Cognitive Knowledge
– Confabulation
– The Origin of Behavior
– Confabulation Mathematics
► Confabulation Neuroscience
► Simulating Confabulation on a Computer
– A Single Confabulation
– Multiconfabulation
► Practical Applications
–
Chancellor Project
• By itself, confabulation theory does not seem particularly compelling. What makes these
particular four key elements any better than others which might have been chosen?
• This section of the presentation answers this question by demonstrating the practical
effectiveness, and compelling characteristics, of confabulation theory applications.
Computer Simulated Confabulation:
Adding a Next Word to a Text String
β
α
ε
γ
δ
A COMPUTER SIMULATED SINGLE CONFABULATION
32
• First, single confabulation experiments are carried out.
• For practical convenience, a problem within the cognitive domain of language is considered:
adding a next word to a given string of up to four English words.
• Confabulation is used to select the next word.
• The experiments presented in this section all utilize language attribute thalamocortical modules.
However, experience with sound and visual attribute representation modules indicates that these
also work. The final section of the presentation below briefly discusses a couple of confabulation
architectures for sound processing. See Chapter 7 for more discussion of sound and visual
processing.
• Extensive successful experimentation with hierarchical learning, storage, and recall of
spatiotemporal sequences of symbols has also been carried out. This work (most of it carried out
as homework projects in past incarnations of UCSD ECE-270B/C) has included demonstration of
smooth context-change-driven switching between alternative sequences during recall and
learning of spatiotemporal sequences from past examples of expert behavior (e.g., records of
checkers games). None of this work has yet been published and it is not mentioned in this book.
Computer Simulated Confabulation:
Adding a Next Word to a Text String
33
(unified, blockbuster, comprehensive, definitive, coordinated, protein)
a
of
?
________
ge
knowled
link
(one of
millions)
ε
…
…
…
…
…
Assumed
Facts → α
β
γ
δ
Knowledge
Bases
A COMPUTER SIMULATED SINGLE CONFABULATION
lack
5 modules, 4 knowledge bases,
5,251,335 knowledge links
for
• This re-representation of the diagram of the previous viewcell shows the four knowledge bases
used in this example (the four red curved arrows). One red knowledge link is also shown.
• Using strings of five contiguous words drawn from within millions of proper English text
sentences; a total of 5,251,335 knowledge links were created (see Chapters 4 and 7) by noting
pairwise word symbol co-occurrences.
• Here, a string of four test words, for lack of a, have been entered into the system and
represented as conclusion symbols (black dots) on the first four simulated thalamocortical
modules. These symbols are the assumed facts that will be used in the upcoming confabulation.
• All of the knowledge links connecting each assumed fact symbol to symbols of the fifth module
(the one on the far right) are used to deliver input excitation to the symbols of module five. These
excitations are summed for each module five symbol λ; yielding that symbol’s total input
excitation intensity I(λ).
• Confabulation results in activation of symbol ε, having the highest input intensity value I. This is
the confabulation conclusion. All other symbols are inactive.
• As shown, the conclusion symbol ε represents the word unified (the runners-up are also shown).
Computer Simulated Confabulation:
Adding a Next Word to a Text String
34
W1
W2
she
W3
W4
W5
could determine
1. whether
2. exactly
3. if
4. why
5. how
6. precisely
A COMPUTER SIMULATED SINGLE CONFABULATION
4 knowledge bases
confabulation
• This is re-representation of the confabulation architecture of the previous two viewcells using a
different set of graphical conventions.
• These graphical conventions will be used henceforth in the description of additional confabulation
architectures.
• Notice that the five symbols which ended up having the first highest, second highest, third
highest, etc. input intensities are all perfectly acceptable next words. This illustrates one of the
non-obvious strengths of confabulation from a biological implementation perspective: the
numerical accuracy and precision of the knowledge link strengths can be low, with no major
effect on the quality of the conclusion – since even if the top conclusions were interchanged as a
result of inaccurate calculation implementations, the final answer would still be excellent.
Computer Simulated Confabulation:
Adding a Next Word to a Text String
35
word string
she could determine
she could determine whether
added word
A COMPUTER SIMULATED SINGLE CONFABULATION
• Here is yet another graphical representation of the operation of the confabulation architecture of
the previous three viewcells.
• Note that in the previous viewcell the confabulation architecture was illustrated inside a ‘purple
box’ – of which the ‘front panel’ had been removed.
• The depiction here shows this purple box with the front panel in place.
• This graphical depiction is typically used when the overall function of a confabulation architecture
is being emphasized.
Computer Simulated Confabulation:
Adding a Next Word to a Text String: Results
36
► she could determine (whether, exactly, if, why, how, precisely) 8
► if it was not (immediately, clear, enough, true, properly, stupid) >999
► earthquake activity was [centered]
► a lack of (urgency, oxygen, understanding, confidence, communication, enthusiasm)
407
► regardless of expected [outcome, length]
► cars drove down a (lane, freeway, highway, dirt, taxi, tying) 9
► driving west on interstate [highway, freeway]
► snow fell in (freezing, montana, portions, northwestern, northeastern) 11
► tune card fly bold [ ]
► threats of terrorist [attacks, retaliation, strikes, violence]
► the machine (tools, tool, guns, gun, operator, shop) 33
► children can learn [lessons, math, english]
► students can learn [lessons, math, english]
A COMPUTER SIMULATED
SINGLE CONFABULATION
► college students can learn [math]
► knowledge of historical [facts, subjects, styles]
► her responsibility for taking [sole, matters]
• Here are results of single confabulation experiments using the confabulation architecture
illustrated on the previous four viewcells. See Chapters 4 and 7 for full details.
• When six or fewer symbols emerged as finalists in a confabulation, the words they represent are
shown within brackets.
• When more than six symbols emerged as finalists, the first words represented by the first six are
shown within parentheses; followed by the total number of finalists.
• Note that nonsense word string tune card fly bold yields no finalists (a null conclusion). This is a
confabulation architecture’s way of saying I DON’T KNOW.
Sentence Continuation Multiconfabulation Experiment
(add 4 words to a starter)
starter
37
confabulation architecture
I was very
I was very nervous about my ability …
A) Case 1 – No previous sentence context supplied
sentence continuation
The forward missed the penalty shot.
context sentence
starter
I was very
I was very upset with his team's …
B) Case 2 – Previous sentence context supplied
sentence continuation
COMPUTER SIMULATED MULTICONFABULATION
• Now, multiconfabulation experiments are carried out. See Chapter 6 for full details.
• In these experiments, the goal is to add four words (jointly referred to as a continuation) to a
given starting word string (called the starter) that begins a new sentence.
• In some experiments (Case 2 – at bottom in the Viewcell), a previous context sentence is
provided.
• In other experiments (Case 1 – at top in the Viewcell), no context sentence is provided.
• In all Cases, the goal is to create a sentence continuation that is sensible in terms of both the
given starter and the given context sentence (if one is provided).
• A key performance issue is to evaluate the influence that the context sentence (if present) has on
the continuation.
• Note that in the examples shown here (which establishes the color coding that will be used to
display experimental results later in the presentation); the continuation changes appropriately
when a context sentence is added.
• The contents of this sentence continuation ‘purple box’ are now described.
38
Sentence Continuation
Confabulation Architecture
82 modules, 1071 knowledge bases,
≈ 2 Billion knowledge links
multiple
contemporaneous,
mutually interacting,
confabulations
applying millions of
items of knowledge in
parallel to yield a final
consensus of
conclusions
thalamocortical module
COMPUTER SIMULATED MULTICONFABULATION
• Here, the confabulation architecture inside the ‘purple box’ of the previous viewcell is elaborated.
For complete details, see Chapter 6.
• Recall that the confabulation architecture used to add one word to a word string via a single
confabulation (described in viewcells 32 thru 36) had 5 modules and 4 knowledge bases (with a
combined total of about 5 million knowledge links).
• The confabulation architecture depicted here, which employs multiconfabulation, has 82 modules
and 1,071 knowledge bases (with a combined total of over 2 billion knowledge links). The
knowledge links were formed by exposing the system to about 70 million examples of pairs of
immediately consecutive well-crafted English sentences and linking co-occurring symbol pairs.
• The ‘zeroth-order’ design characteristic of the human cerebral cortex is a vast number of
synapses (roughly hundreds of trillions) but only a large number of neurons (roughly tens of
billions). This lopsided proportion of knowledge links to modules is clearly shared by
confabulation architectures.
• Exactly as neuroscientists have been guessing for decades, confabulation theory hypothesizes
that cognitive processing is based upon a very simple information processing operation
(confabulation) employing a large complement of neuronal interconnections.
39
Sentence Continuation Thought Process
knowledge
base
sentence
meaning
content
summary
modules
phrase
modules
word
modules
Context Sentence Module Grouping
S
P1
W1
P2 P3
W2
W3
P4
P5 P6
W4
W5
W6
P7
W7
Continuation Sentence Module Grouping
• Here is further enlargement of the purple box’s sentence continuation confabulation architecture.
• The swirling red arrow illustrates the modules that will, together, be confabulated.
• By ensuring that the ever-shrinking sets of conclusions being considered in each of the
confabulating modules are consistent with all of the external assumed facts (the starter and,
when supplied, the context sentence), AND WITH EACH OTHER, the final set of conclusions will
form a confabulation consensus.
• An astounding aspect of this multiconfabulation process is that by carrying out this ‘mutual
consultation’ process repeatedly (illustrated here by the recursive swirling red arrow), the set of
conclusions which end up having mutual knowledge link connections in ‘loops’ that ‘close’ is very
small.
• As with the single confabulation experiments previously described, there are typically a number
of symbol sets (confabulation consensuses) which could emerge in the end.
• During the initial stages of multiconfabulation swirling, huge numbers of knowledge links are
utilized in parallel. All of these knowledge links are relevant, in the sense that each originates
from a viable candidate conclusion symbol. Convergence then proceeds rapidly.
40
Sentence Continuation Thought Process
context sentence
The football quarterback fumbled the snap .
Shortly thereafter he ADD FOUR WORDS HERE
starter
SENTENCE CONTINUATION THOUGHT PROCESS
CONCRETE EXAMPLE
• To illustrate the details of multiconfabulation in this architecture, the processing of the specific
concrete set of assumed facts (starter and context sentence) shown here is now examined stepby-step over many viewcells.
• The multiconfabulation procedure employed here is undoubtedly far from optimal. For one thing,
the knowledge links which are applied at each stage are not necessarily optimally chosen. For
another, there are probably many steps that could be consolidated; with an overall benefit in
speed and performance.
• Regardless of these shortfalls, this exhaustive example illustrates many key points about
confabulation architectures and multiconfabulation. If confabulation theory is correct, then this
example illustrates the inner workings of animal thought and explains why thought is so
powerful, flexible and fast.
• For complete details of these experiments, see Chapter 6.
41
Sentence Continuation Thought Process
previous (context)
sentence meaning
content representation
S
P1
P2
P3
P4
P5
P6
P7
W1
W2
W3
W4
W5
W6
W7
The football quarterback fumbled the snap .
context sentence
THOUGHT PROCESS STEP 1
• The first Step of the thought process (the same thought process is used no matter what assumed
facts are provided) is entry of the context sentence (here: The football quarterback fumbled
the snap.). First, the symbol for the word The is activated in the leftmost word module of the
context sentence module grouping. Next, the word football is activated in the second word
module of the context sentence module grouping. And so on. [Note that only the first seven word
and phrase modules of each row of each grouping are illustrated here. Each of these rows
actually has 20 modules. See Chapter 6 for complete details.]
• Once all of the words of the context sentence have been entered, the knowledge bases
proceeding to the phrase modules from the word modules are used to parse the sentence into
phrases and words (see Chapter 7 for more details on sentence parsing). This leaves the
sentence re-represented on the second row of modules of the context sentence grouping.
• Finally, the sentence is re-represented again using the context sentence meaning content
representation module. Knowledge links from the excited symbols of this representation then
excite symbols on the continuation sentence meaning content representation module S. These
symbols on S are used in many subsequent steps (all shown below) to deliver meaning content
from the context sentence to the phrase modules involved in the multiconfabulation.
42
Sentence Continuation Thought Process
S
P1
P2
P3
P4
P5
P6
P7
W5
W6
W7
Shortly thereafter he
starter
W1
W2
W3
W4
The football quarterback fumbled the snap .
THOUGHT PROCESS STEP 2
• In Step 2 of the thought process, the knowledge links from the activated symbols of the phrase
modules representing the starter Shortly thereafter he and the knowledge links from the excited
symbols of the summary module S provide excitation to the symbols of the first phrase module,
P4, of the multiconfabulation.
• The 1,000 most excited symbols of P4 are allowed to remain excited. All other symbols are shut
off permanently for the duration of the multiconfabulation.
• This set of 1,000 candidate conclusions on P4 is termed an expectation (see Chapters 6 and
7). As multiconfabulation proceeds, this expectation is reduced in size. Eventually, only one
symbol, the P4 module conclusion, is left. This convergence process is completed on P4 before
P5, P6, and P7 converge.
• This number 1,000 is arbitrary – it is designed to try to ensure that the set of candidate
conclusions contains all symbols which could possibly end up becoming the final conclusion (i.e.,
all symbols which are compatible with the starter and the context sentence).
• Notice how, in this situation, knowledge links act as constraints on the set of reasonable
candidate conclusions. Only those conclusions which receive knowledge links are considered.
43
Sentence Continuation Thought Process
S
Shortly
thereafter
he
P1
P2
P3
P4
1
1
1
1000
Note: Each excited symbol
has an average of about 200
knowledge links emanating
from it; per knowledge base.
W4
P5
P6
P7
W5
W6
W7
THOUGHT PROCESS STEP 2
• This viewcell is a ‘zoomed in’ graphical re-representation of Step 2 of the thought process.
• The graphical convention used here will be used to illustrate all of the remaining thought process
Steps.
• Note that modules P1, P2, and P3 each have the number 1 shown within them. This means that
only one symbol – representing, in this case, a single word (Shortly, thereafter, and he in this
case, respectively) – is activated on each of these modules.
• Module P4 has the number 1000 shown within it because this is the current size of its candidate
conclusion expectation.
• Note that the knowledge bases used to deliver excitation to P4 from the symbols representing the
starter and from the symbols on S representing the context sentence are illustrated using arrows.
Each individual knowledge base (arrow) typically has millions of individual knowledge links. As
with the human cerebral cortex; cognition is dominated by axons and synapses; with neurons
playing a relatively minor role – at least hardware percentage wise.
44
Sentence Continuation Thought Process
P1
P2
P3
P4
1
1
1
1000
P5
1,000 items of knowledge
being used
P6
P7
≈ 600,000 items of knowledge
being used
≈ 150,000 items of knowledge
being used
758
3000
W4
W5
W6
W7
THOUGHT PROCESS STEP 3
• This viewcell shows the first step of the swirling involved in this thought process. For complete
details (and there are many which are not covered in this presentation) see Chapter 6.
• First, the 1,000 candidate conclusions of the P4 expectation send excitation to W4 via their
knowledge links. In other words, the knowledge base from P4 to W4 is enabled. Each P4 symbol
sends one knowledge link to W4 – namely, to the first word of its word or phrase.
• On W4, a total of 758 symbols (the unique first words of the 1,000 P4 symbols) receive excitation
– thus defining the initial W4 expectation.
• In general, each expectation symbol will have an average of about 200 knowledge links
emanating from it in each knowledge base. Thus, when the knowledge base from W4 to W5 is
enabled, about 150,000 knowledge links will be used.
• On W5, an expectation consisting of the 3,000 most excited symbols is formed. These symbols
then send excitation to P4 through about 600,000 knowledge links.
• In effect, multiconfabulation thought processes involve the parallel application of huge
numbers of relevant knowledge links. This massively parallel relevant knowledge
application is one of the key strengths of multiconfabulation.
45
Sentence Continuation Thought Process
P1
P2
P3
P4
1
1
1
17
P5
14
3000
W4
W5
P6
P7
W6
W7
THOUGHT PROCESS STEP 4
• Step 4 of the thought process involves a second swirl around this loop.
• Notice that the roughly 600,000 knowledge links coming back to module P4 at the end of the
Step 2 swirl only delivered significant excitation to 17 P4 symbols. Thus, the P4 expectation has
now been reduced to 17 symbols from its starting size of 1,000. These 17 P4 symbols only have
14 unique first words – thus contracting the W4 expectation from 758 symbols to 14.
• This second swirl does not reduce the P4 expectation further; and so the thought process moves
on to the next part of the architecture (if the P4 expectation had been reduced further, a third
swirl would have been launched instead).
• Thought process Step 4 thus ends with only 14 candidates for the first word of the continuation.
This illustrates how rapidly thought processes tend to converge – another subtle advantage of
the confabulation theory design.
• In many cases, any of the remaining 14 words would be a good choice for the first word of the
continuation. This illustrates why, in many instances, having weighted links (as opposed to binary
links – yes-link or no-link) is not necessary: even though the mathematical theory requires
weighting. After not-mutually-supported symbols are eliminated, only good candidates remain.
46
Sentence Continuation Thought Process
P1
P2
P3
P4
1
1
1
17
P5
P6
P7
W6
W7
2
14
3000
W4
W5
THOUGHT PROCESS STEPS 3 - 4
• Graphically, Steps 3 and 4 can be represented as shown here: a swirl of two cycles around the
depicted knowledge bases.
• This graphical convention will be employed further below.
47
Sentence Continuation Thought Process
P1
P2
P3
P4
P5
1
1
1
17
1820
14
3000
W4
W5
P6
P7
W6
W7
THOUGHT PROCESS STEP 5
• Thought process Step 5 involves forming an initial expectation on P5 based upon knowledge link
inputs from the expectations of P4 and S.
• Only 1820 symbols of P5 received sufficient excitation to be considered candidate conclusions.
48
Sentence Continuation Thought Process
P1
P2
P3
P4
P5
P6
1
1
1
17
197
14
77
3000
W4
W5
W6
P7
2
W7
THOUGHT PROCESS STEPS 6-7
• Thought process Steps 6 and 7 involve the two-cycle swirl shown here.
• During this swirl, the P5 expectation was reduced to 197 symbols from the initial 1820.
• Also, the W5 expectation is reduced to 77 symbols from an initial 3,000.
• A new expectation of 3,000 symbols on W6 emerges from this swirl.
• To spell it out: swirling is a massively parallel and extremely fast operation which, among other
things, effortlessly carries out what could be called constraint satisfaction. The astounding
thing is that this works only because of the use of additive knowledge combination – which is
only possible because of The Fundamental Theorem of Cognition and confabulation theory’s
hypothesis of evolutionary adaptation to its approximation.
• By now, you may be starting to understand the amazing power of the key elements of
confabulation theory. They fit together perfectly and yield the highly favorable characteristics
demonstrated here. But we don’t stop here – things get even better as our exploration of the
practical application of confabulation theory continues!
49
Sentence Continuation Thought Process
P1
P2
P3
P4
P5
P6
1
1
1
3
197
14
77
3000
W4
W5
W6
P7
W7
THOUGHT PROCESS STEP 8
• In Step 8, the knowledge base from W6 to P4 is enabled.
• Only 3 of the 17 remaining symbols of the P4 expectation receive significant excitation.
• Thus, the P4 expectation is reduced to 3 symbols. Thus, there are only three words that remain
as contenders to being the first word of the continuation.
50
Sentence Continuation Thought Process
P1
P2
P3
P4
P5
P6
1
1
1
3
197
3
77
3000
W4
W5
W6
P7
W7
THOUGHT PROCESS STEP 9
• In Step 9 a ‘test swirl’ of this previously used loop is carried out to see if the P4 expectation
shrinks further (sometimes it does; but, most of the time, as in this example, it does not).
51
Sentence Continuation Thought Process
P1
P2
P3
P4
P5
P6
1
1
1
3
130
3
77
3000
W4
W5
W6
P7
W7
THOUGHT PROCESS STEP 10
• Step 10 enables the P4 to P5 knowledge base.
• Only 130 of the 197 symbols in the P5 expectation receive knowledge links; so the P5
expectation is reduced to these 130 symbols.
52
Sentence Continuation Thought Process
P1
P2
P3
P4
P5
P6
1
1
1
3
130
3
52
3000
W4
W5
W6
P7
W7
THOUGHT PROCESS STEP 11
• Step 11 does another swirl around the second loop.
• The W5 expectation is reduced to 52 symbols as a result.
• Thus, at this point, the second word of the continuation can only be one of 52 choices.
• Because the P5 expectation does not shrink, only one swirl cycle is completed.
• At this point, you are starting to see that shrinking the expectations is happening progressively as
the multiconfabulation proceeds.
• In a single confabulation, this list shrinkage occurs in one step: the symbol with the highest input
excitation is activated as the conclusion.
• However, here, the expectation lists contract slowly. Each module is like a muscle participating
in an orchestrated, coordinated, movement. As the thought process proceeds, the enablements
of the proper knowledge bases and the contractions of the proper modules are carried out. This
is why the thalamocortical modules are termed the muscles of thought.
53
Sentence Continuation Thought Process
P1
P2
P3
P4
P5
P6
1
1
1
3
130
3
52
3000
W4
W5
W6
P7
W7
THOUGHT PROCESS STEP 12
• Step 12 again attempts to contract the P4 expectation; but the W6 expectation has not changed
since Step 8 and so nothing happens.
54
Sentence Continuation Thought Process
P1
P2
P3
P4
P5
P6
1
1
1
3
130
328
3
52
3000
W4
W5
W6
P7
W7
THOUGHT PROCESS STEP 13
• Step 13 creates the initial expectation on P6.
55
Sentence Continuation Thought Process
P1
P2
P3
P4
P5
P6
P7
1
1
1
3
130
111
3
52
98
3000
W4
W5
W6
W7
2
THOUGHT PROCESS STEPS 14-15
• Steps 14 and 15 are two swirls of this loop.
• These swirls result in a P6 expectation having 111 symbols – of which there are 98 unique first
words (the expectation on W6).
56
Sentence Continuation Thought Process
P1
P2
P3
P4
P5
P6
P7
1
1
1
3
130
111
3
52
98
3000
W4
W5
W6
W7
THOUGHT PROCESS STEP 16
• A first attempt to use the new W7 expectation to reduce the P4 expectation fails.
57
Sentence Continuation Thought Process
Shortly
thereafter
he
fumbled
P1
P2
P3
P4
P5
P6
1
1
1
1
130
111
3
52
98
3000
W4
W5
W6
W7
P7
THOUGHT PROCESS STEP 17
• After failing to get P4 to converge on the first three swirls and the two attempted expectation
feedbacks from W6 and the one from W7; Step 17 is to simultaneously enable all of the
knowledge bases from W4, W5, W6 and W7 to P4 and complete the P4 confabulation to the
single most excited conclusion.
• This process yields the P4 symbol for the word fumbled. This is the first word of the continuation.
58
Sentence Continuation Thought Process
Shortly
P1
thereafter
P2
he
P3
fumbled
P4
P5
P6
P7
P8
W4
W5
W6
W7
W8
3
SUMMARY OF THOUGHT PROCESS STEPS 3 – 17
• Here, a superposition of thought process Steps 3 through 17 are shown. An interesting question
is: “To what degree could these separate steps be carried out in parallel?”
• This possibility of parallelization is not discussed in this book; but examining the individual Steps
involved makes it clear that it may be possible to replace individual Steps 3 through 17 with the
three ‘grand swirls’ illustrated here.
59
Sentence Continuation Thought Process
thereafter
he
fumbled
P2
P3
P4
P5
P6
P7
P8
1
1
1
0
0
0
0
Move Frame of Reference Over One Word
and
Repeat Same Thought Process Steps
0
0
0
0
W5
W6
W7
W8
THOUGHT PROCESS STEP 17
• At the end of Step 17, module P4 has converged. So the involvement of P4 and W4 in the
multiconfabulation process is over.
• At this point, the thought process, in effect, ‘moves over one module’, and then repeats the same
Steps again to create the P5 conclusion.
60
Sentence Continuation Thought Process
P2
P3
P4
P5
P6
P7
P8
1
1
1
71
0
0
0
0
0
0
0
W5
W6
W7
W8
THOUGHT PROCESS STEP 18
• Step 18 builds a fresh P5 expectation based upon excitation from knowledge links originating
from the symbols of P2, P3, P4, and S.
• Note that all of the previously constructed expectations from previous steps have been
discarded.
• The resulting P5 expectation has 71 symbols.
61
Sentence Continuation Thought Process
P2
P3
P4
P5
P6
P7
P8
1
1
1
17
0
0
0
9
3000
0
0
W5
W6
W7
W8
2
THOUGHT PROCESS STEPS 19 - 20
• Two swirls (Steps 19 and 20) reduce the P5 expectation from 71 symbols to 17 (representing
words and/or phrases with 9 unique first words – the expectation of W5).
• The fact that this P5 expectation contains 17 symbols (as with the P4 expectation after Step 3) is
just a coincidence.
62
Sentence Continuation Thought Process
P2
P3
P4
P5
P6
P7
P8
1
1
1
17
29
0
0
9
3000
0
0
W5
W6
W7
W8
THOUGHT PROCESS STEP 21
• Step 21 creates a new P6 expectation with 29 symbols.
63
Sentence Continuation Thought Process
P2
P3
P4
P5
P6
P7
P8
1
1
1
17
8
0
0
9
7
3000
0
W5
W6
W7
W8
2
THOUGHT PROCESS STEPS 22-23
• Two swirls (thought process Steps 22 and 23) end up reducing the P6 expectation from 29
symbols to 8.
64
Sentence Continuation Thought Process
P2
P3
P4
P5
P6
P7
P8
1
1
1
9
8
0
0
9
7
3000
0
W5
W6
W7
W8
THOUGHT PROCESS STEP 24
• Enablement of the knowledge base from W7 to P5 and further contraction of P5 reduces the P5
expectation from 17 symbols to 9.
• Note the slow contraction of the P5 expectation as the multiconfabulation proceeds towards
completion of construction of the sentence continuation.
65
Sentence Continuation Thought Process
P2
P3
P4
P5
P6
P7
P8
1
1
1
9
8
0
0
6
3000
3000
0
W5
W6
W7
W8
THOUGHT PROCESS STEP 25
• Another swirl of the P5 loop has no effect on the P5 expectation; so a second swirl does not take
place.
66
Sentence Continuation Thought Process
P2
P3
P4
P5
P6
P7
P8
1
1
1
9
5
0
0
6
3000
3000
0
W5
W6
W7
W8
THOUGHT PROCESS STEP 26
• Application of the contracted P5 expectation and input from S reduces the P6 expectation from 8
symbols to 5.
67
Sentence Continuation Thought Process
P2
P3
P4
P5
P6
P7
P8
1
1
1
9
5
0
0
6
4
3000
0
W5
W6
W7
W8
THOUGHT PROCESS STEP 27
• The swirl of Step 27 has no benefit (in terms of contracting the P6 expectation further).
68
Sentence Continuation Thought Process
P2
P3
P4
P5
P6
P7
P8
1
1
1
9
5
0
0
6
4
3000
0
W5
W6
W7
W8
THOUGHT PROCESS STEP 28
• Step 28 has no effect.
69
Sentence Continuation Thought Process
P2
P3
P4
P5
P6
P7
P8
1
1
1
9
5
63
0
6
4
3000
0
W5
W6
W7
W8
THOUGHT PROCESS STEP 29
• Step 29 produces a new P7 expectation with 63 symbols.
70
Sentence Continuation Thought Process
P2
P3
P4
P5
P6
P7
P8
1
1
1
9
5
20
0
6
4
15
3000
W5
W6
W7
W8
2
THOUGHT PROCESS STEPS 30-31
• Steps 30 and 31 – both P7 swirls – yield a reduction in the P7 expectation from 63 symbols to 20.
Of these 20 symbols representing words and/or phrases, there are 15 unique first words (the
expectation on W7).
71
Sentence Continuation Thought Process
P2
P3
P4
P5
P6
P7
P8
1
1
1
9
5
20
0
6
4
15
3000
W5
W6
W7
W8
THOUGHT PROCESS STEP 32
• Step 32, enablement of the knowledge base from W8 to P5, does not lead to a reduction in the
P5 expectation.
72
Sentence Continuation Thought Process
thereafter
he
fumbled
in the
end zone
P2
P3
P4
P5
P6
P7
P8
1
1
1
1
5
20
0
6
4
15
3000
W5
W6
W7
W8
THOUGHT PROCESS STEP 33
• Finally, Step 33 enables the four knowledge bases from W5, W6, W7, and W8 to P5 and the P5
confabulation is concluded.
• The winning P5 symbol represents the phrase: in the end zone.
• This completes the continuation: fumbled in the end zone.
• Notice that at the end of both the P4 and the P5 convergence processes that it became
necessary to ‘force’ selection of the final confabulation conclusion. This is usually because all of
the symbols on the expectation at that point are excellent choices. This again illustrates the
robustness of confabulation – which translates into robust biological implementability.
73
Sentence Continuation Thought Process
shortly
thereafter
he
fumbled
in the
end zone
P1
P2
P3
P4
P5
P6
P7
P8
W4
W5
W6
W7
W8
6 swirls
SUMMARY OF THOUGHT PROCESS STEPS 3 – 33
• This diagram summarizes the entire thought process.
• If carried out with more parallelism, the Steps 3 thru 33 of the thought process might be reduced
to 6 swirls.
74
Sentence Continuation Thought Process
The football quarterback fumbled the snap .
Shortly thereafter he fumbled in the end zone …
FINAL ANSWER
• Here is the final confabulation consensus (with the ‘extra’ word obtained, zone, shown in pink).
75
Sentence Continuation Thought Process
NO context sentence
The New York ADD FOUR WORDS HERE
starter
SENTENCE CONTINUATION THOUGHT PROCESS
CONCRETE EXAMPLE #2
• Above, in the first concrete example presented, “re-entrant” (to expand, to whole modules, a
term coined by Gerald Edelman with regard to individual neuron populations) swirling was
emphasized as one of the most powerful components of thinking. This is because swirling allows
vast numbers of relevant knowledge links to be applied in parallel to ensure constraint
satisfaction among the ever-narrowing sets of candidate conclusions (the expectations).
• Now, two more concrete examples are considered to illustrate another of the most powerful
components of thinking: additive knowledge combination.
• Together, swirling and additive knowledge combination, along with the underlying mathematics of
confabulation, give thought an enormous ability to generalize to radically novel (but sensible)
combinations of familiar objects and to freely mix arbitrary kinds of established knowledge.
• Concrete Example #2 is shown in this viewcell. It is just a starter without a context sentence.
• Concrete Example #3 (shown later) uses the same starter; but it includes a context sentence.
• The only difference between these examples is the presence of a context sentence in Example
#3. The focus of attention is answering the question of why the continuation switches when the
context sentence is added. The answer is that the extra available knowledge is essentially just
added in, altering the symbols in the expectations of the affected modules.
76
Sentence Continuation Thought Process
S
P1
starter
The
P2
P3
New
W1
W2
W3
P4
P5
P6
P7
W6
W7
York
W4
W5
THOUGHT PROCESS STEP 2
CONCRETE EXAMPLE #2
• The same thought process is applied here to Concrete Example #2 as was used in the first
example.
• However, since there is no context sentence this time, there are no knowledge links from
symbols of module S to deliver excitation to the symbols of P4 (or any of the other phrase
modules later in the thought process).
77
Sentence Continuation Thought Process
The
New
York
P1
P2
P3
Times’ computer
model
collapses
P4
P5
P6
P7
P8
W4
W5
W6
W7
W8
6 swirls
SUMMARY OF THOUGHT PROCESS STEPS
• After the entire thought process has been applied, the continuation shown in green results.
78
Sentence Continuation Thought Process
The New York Times' computer model collapses …
FINAL ANSWER
CONCRETE EXAMPLE #2
• So, here is the final answer to Concrete Example #2: Times’ computer model collapses
• Note the correct use of the apostrophe in the plural possessive. This purple box knows grammar!
79
Sentence Continuation Thought Process
Context sentence
Stocks proved to be a wise investment .
The New York ADD FOUR WORDS HERE
Same starter
SENTENCE CONTINUATION THOUGHT PROCESS
CONCRETE EXAMPLE #3
• Now the same starter is used; but this time with a context sentence.
80
Sentence Continuation Thought Process
additive
knowledge
combination
previous (context)
sentence meaning
content representation
S
P1
starter
The
P2
P3
New
W1
W2
W3
P4
P5
P6
P7
W6
W7
York
W4
W5
Stocks proved to be a wise investment .
context sentence
THOUGHT PROCESS STEP 2
CONCRETE EXAMPLE #3
• Here is the first difference that occurs: now, the P4 expectation is based on both the starter and
on the context sentence. In other words, the P4 symbols which respond are those which are
excited by the combined influence of the starter and the context sentence (for exact details see
Chapter 6).
81
Sentence Continuation Thought Process
The
New
York
markets
traded
lower
yesterday
P1
P2
P3
P4
P5
P6
P7
P8
W4
W5
W6
W7
W8
6 swirls
SUMMARY OF THOUGHT PROCESS STEPS
• The thought process proceeds as before; except with P5, P6, P7, and P8 all being influenced by
the context sentence; as shown with P4 in the last viewcell.
82
Sentence Continuation Thought Process
Stocks proved to be a wise investment .
The New York markets traded lower yesterday …
FINAL ANSWER
CONCRETE EXAMPLE #3
• Now, the continuation changes; reflecting a confabulation consensus that was influenced by both
the starter and the context sentence.
• This illustrates another powerful aspect of thinking: roughly, the ability to combine together
arbitrary items of available knowledge.
• In short, whatever knowledge is available can generally just be ‘added in’.
• There are many additional issues and considerations surrounding additive knowledge
combination; but Concrete Examples #2 and #3 do an excellent job of illustrating how powerful
this confabulation capability is.
Sentence Continuation
Experimental Results
83
The New York Times' computer model collapses …
Stocks proved to be a wise investment .
The New York markets traded lower yesterday …
Downtown events were interfering with local traffic .
The New York City Center area where …
Coastal homes were damaged by tropical storms .
The New York City Emergency Service System …
Medical patients tried to see their doctors .
The New York University Medical Association reported …
When the United Center Party leader urged …
MULTICONFABULATION
The car assembly lines halted due to labor strikes .
When the United Auto Workers union representation …
The price of oil in the Middle East escalated yesterday .
When the United Arab Emirates bought the …
• This and the next two viewcells present more examples of sentence continuation experimental
results. For the complete, unabridged, listing of the results of our experiments see Appendix B of
Chapter 6.
• The first and second lines present the continuations for Concrete Example #2 and Concrete
Example #3; which were already described above. These illustrate the text color coding used in
presenting these experimental results.
• Consider what this system must “know” about “reading” and “writing” English – and about the
world in general – in order to be able to reliably construct continuations of this quality.
Sentence Continuation
Experimental Results
84
But the Roman Empire disintegrated during the …
She learned the history of the saints .
But the Roman Catholic population aged 44 …
She studied art history and classical architecture .
But the Roman Catholic church buildings dating …
I was very nervous about my ability …
Democratic citizens voted for their party's candidate .
I was very concerned that they chose …
Restaurant diners ate meals that were served .
I was very hungry while knowing he …
MULTICONFABULATION
In spite of yesterday's agreement among analysts …
The Mets were not expected to win .
In spite of the pitching performance of …
• Here are some additional examples.
• Studying these examples, some people have an epiphany. They suddenly submit to the
realization that this experiment is revealing multiple deep insights into the fundamental nature of
thinking.
• One deep insight is that, since there is no time, or capability, for iterative reasoning (a type of
thinking that is beyond the scope of this book and which requires many other subcortical brain
structures to carry out), this kind of intelligence is a real-time performance. Excellent
continuations reflect both exposure to many expertly-crafted examples AND to a powerful
information processing approach with spectacular generalization capability and flexibility.
• Another deep insight is that knowledge is useful for both rejecting non-viable conclusions (a
constraint satisfaction-style function) AND for encouraging viable conclusions that are more
strongly supported by the remaining candidate conclusions in other module’s expectations. This
is why swirling is so important – it causes the final consensus of conclusions to be compatible.
• Missing knowledge links enforce constraints. Existent knowledge links confer encouragement
to symbols.
Sentence Continuation
Experimental Results
85
The President was certain to be reelected .
In spite of his statements toward the …
She had no clue about the answer .
In spite of her experience and her …
It meant that customers could do away …
The stock market had fallen consistently .
It meant that stocks could rebound later …
I was not able to solve the problem .
It meant that we couldn't do much …
MULTICONFABULATION
The company laid off half its staff .
It meant that if employees were through …
The salesman sold men's and women's shoes .
It meant that sales costs for increases …
• Yet another deep insight about thought is that, as the swirling initially proceeds, many symbols
with similar meanings (in the context being considered) often have ‘downstream’ knowledge
links to the same symbols. Because of this property, expectation symbols with low excitation
levels -- but similar meanings, in context, to a highly excited expectation symbol – often
themselves become highly excited when swirling closes the loop. This is a crucially important
aspect of thought, in that it causes an early fluffing of a few initially highly-excited symbols into a
larger collection of newly-promoted symbols of closely-related meaning (what fuzzy theorists
might term fuzzification). Fluffing provides ‘raw materials’ (relevant symbols) that are then
whittled away to a consistent set of final conclusions as confabulation convergence proceeds.
• Notice that swirling and fluffing are aspects of thought process design – not strictly part of the
basic mechanism of thought (the four key functional elements of confabulation theory).
• It seems likely that as human thought processes are studied in detail (which will surely be
possible in a few years, as brain activity imaging techniques improve in both spatial and temporal
resolution), many additional thought process design tricks, beyond swirling and fluffing are sure
to be learned. To get at the details of thought process design will surely require thousands of
additional researchers and many hundreds of high-capability real-time brain activity imagers.
86
Sentence Continuation Experiment Collaborators
Chapter 6 authors; photographed in San Diego on 15 February 2006 by Matthias
Blume. Left to right: Kate Mark, Robert Hecht-Nielsen, Luke Barrington,
Andrew Smith, Robert W. Means, and Syrus C. Nemat-Nasser.
• Here is the team that did the sentence continuation experiments. This was a once-in-a-lifetime
shared adventure. Thanks to Fair Isaac Corporation for generous support of this research.
Sentence Continuation Experiment Lessons
87
► Multiconfabulation allows massively parallel application of
relevant knowledge
► Grammar and syntax are clearly emergent properties of the
mechanism of thought, confabulation – essentially as
hypothesized by Miller, by Lenneberg, and by Chomsky
► Additive knowledge combination endows thought with
enormous flexibility – whatever constraints or
considerations need to brought to bear are simply ‘added in’
by enabling the appropriate knowledge bases
► If there is no viable conclusion; confabulation yields a null
output. Being able to say “I don’t know” is enormously
powerful
► Confabulation exhibits phenomenal generalization – novel,
but sensible combinations of familiar elements can almost
always be dealt with effectively
CONFABULATION THEORY CHARACTERISTICS
• Grammar, syntax, and many other linguistic concepts turn out (as long proposed by some
linguists) to be nothing but by-products (emergent properties) of the mechanism of thought (i.e.,
confabulation). Grammar, syntax, etc. are fictions. Like phlogiston (a hypothetical fluid
substance within matter once believed to be the explanation for heat), they do not actually exist.
Confabulation Theory Characteristics
88
► In Confabulation Architectures there are NO:
– Algorithms
– Software routines (beyond the simulations of the functional elements)
– Rules
– Ontologies
– Priors
– Bayesian networks, etc.
► Conclusion: Thinking is starkly alien. The cerebral cortex and
thalamus are comprised of roughly 4,000 separate ‘digital
processors,’ interconnected pairwise by roughly 40,000 analog
knowledge bases (together containing tens of billions of individual
items of knowledge). Like movements, thought processes are
stored and recalled coordinated parallel ensembles of swirling
analog processor ‘contraction’ commands.
CONFABULATION THEORY CHARACTERISTICS
• Obviously, the legacy of computer information processing will form the basis for the construction
of artificial brains. However, going forward, the amount to be done vastly outweighs what was
accomplished in this predecessor technology era.
• Reorienting the world of information processing will not be painless. As consumers of information
processing technology begin to demand truly intelligent systems; hundreds of thousands of new
enterprises will blossom to create the supply. Revolutionary developments will eclipse earlier
revolutionary developments decade after decade, for centuries to come.
• It seems likely that the artificial brain business, taken as a whole (including all business activities
derived therefrom), will, by, say 2020, be at least 100 times larger than today’s computer
industry.
• The economic product (work output) of intelligent machines (producing prodigiously with minimal
consumption) will quickly obsolete economics itself.
• A child of 2100 will have no concept of a relationship between work and material well-being.
89
Confabulation Theory Summary
► Confabulation Theory:
– Each of 4,000 human cortical modules describes one object
attribute
– Each module has thousands of symbols to describe its attribute
– A knowledge link forms between each pair of meaningfully cooccurring symbols
– Additive target module symbol knowledge link input excitation
combination
– Confabulation: winner-take-all selection of most-excited target
module symbol
– Simple, fast, mutual-consultation multiconfabulation convergence
– Immediate triggering of the action commands linked from the
confabulation conclusion
CONFABULATION THEORY CHARACTERISTICS
• Confabulation theory specifies the key functional elements of cognition and the origin of behavior.
• These key elements essentially specify the ‘processing hardware functions’ used in cognition.
This is akin to a description of a computer and its function.
• However, as we have already seen in the sentence continuation example, the manners in which
these key elements are used by thought processes are, just as with software, where the
enormous power lies.
• Evolving new neuronal information processing hardware is difficult – which probably accounts for
the extremely strong conservation of the basic vertebrate brain design (presumably including the
key elements of confabulation theory) over millions of evolved species and hundreds of millions
of years.
• Evolving new thought processes is easy – particularly for species such as humans where these
can be conveyed culturally via “monkey-see / monkey-do” learning (see Chapters 6 and 7) and
then rapidly refined, built upon and passed down as a cultural legacy.
Overview
90
► Motivating Example
► Confabulation Theory Overview
– Cortical Representation of the Objects of the Mental World
– Acquisition and Storage of Cognitive Knowledge
– Confabulation
– The Origin of Behavior
– Confabulation Mathematics
► Confabulation Neuroscience
► Simulating Confabulation on a Computer
– A Single Confabulation
– Multiconfabulation
► Practical Applications
–
Chancellor Project
• In this final section of the presentation, prospects for practical applications of confabulation
theory are discussed.
Chancellor
91
– Cat Food for Zeus
Chancellor
© 2006 Fair Isaac Corporation.
• The Cat Food for Zeus vignette is revisited.
• Given the preceding discussion; it is now clearer that building a product of this type may be
possible in the near term.
• “Canned Thought Process Controllers”, along the lines of those demonstrated in this
presentation, may be sufficient for construction of many high-value applications.
Chancellor Project Roadmap
92
1. Conversational Response Generation
1A. Meaning content representation modules
1B. Response generation thought processes
1C. Transition from proper text English to colloquial spoken English
2. Consumer Service Task Management
2A. Task partition learning and subtask completion state detection
2B. Subtask execution management
3. Conversational Banter Mode
3A. Superb human conversationalist / raconteur performance capture
4. Speech Understanding
4A. Transition to colloquial spoken English language
4B. Cocktail party front end and speaker recognition
4C. New speaker learning and login tools
5. Superb Human Consumer Service Agent Performance Capture
5A. Implement instrumented pilot-application call center
5B. Train task management subsystem
6. Chancellor Prototype
6A. Integrate Confabulation Architecture
6B. Build, test ↔ improve, and demonstrate Chancellor prototype
© 2006 Fair Isaac Corporation.
• Capturing, and then effectively applying, the details of expert human performance with an
‘adaptive system’ is a theme that goes back decades. Ten years ago, an automobile (developed
at Carnegie-Mellon University) controlled by a multilayer perceptron neural network trained on
examples of the performance of a skilled human driver, autonomously drove on public freeways
all the way from Washington, DC to San Diego, CA without incident.
• Here, the extraordinarily more powerful generalization capabilities of confabulation architectures
are exploited to build multiple, initially isolated, subsystems. These are later interconnected with
knowledge bases and exposed to further learning examples in order to build total-system
capability.
• This hypothetical ChancellorTM system has both a ‘serious’ task-oriented customer service
component and a light-hearted conversational banter component.
• An ultra-high-accuracy speech understanding front end (see Chapter 7 for more details) carries
out both separation of the user’s voice sounds from other sounds in the single-microphone
soundstream, and segmentation and word disambiguation of the separated speaker
soundstream. Current technical approaches to speech processing do not effectively address
either of these essential functions.
Chancellor System
93
Speech Synthesizer
Response Generation
Banter
Task Execution
Action Selection
Drives and Goals
Task Planning
Action Control
Task Selection
Legend
Confabulation Architecture Element
Language Module
Preprocessing
Speech Understanding
Knowledge Base
Action Command Base
© 2006 Fair Isaac Corporation.
• This somewhat whimsical, and certainly incomplete, block diagram illustrates how a
confabulation architecture using “canned thought control” might look.
• Each tan box is a confabulation architecture (like the one used for sentence continuation)
typically consisting of hundreds of thalamocortical modules and thousands of knowledge bases.
• Different tan boxes are interconnected with knowledge bases. The overall architecture would
typically not get formed until the individual boxes are working well in isolation.
• Early versions of the tan boxes with the orange borders have already been built.
• This block diagram is perhaps also useful as an indicator of what the future of the information
processing industry may look like. Imagine a product design meeting at which ordinary
thoughtcrafters (touchy-feely artistic types, many with zero technical education) discuss
possibly imposing a reduction of fluffing in the speech input language module in order to speed
conversational response – hopefully with no reduction in performance.
• The key players in the future of information processing are likely to be much like the best aircraft
and spacecraft designers (e.g., the Wright brothers, Werner von Braun, Sergey Korolev, Kelly
Johnson, and Burt Rutan): polymath genius types who can craft all the disparate pieces so they
fit together and function well as a whole system.
Plausible Next Sentence Generation Architecture
First Context Sentence
Second Context Sentence
94
Third Sentence
and
Generated Sentence
• Now, with the background obtained from the detailed discussion of the sentence continuation
confabulation architecture, the plausible next sentence generation confabulation architecture
(which was exhibited as the motivating example at the beginning of the presentation) is briefly
revisited.
• As seen here, this architecture (which, as usual for confabulation architectures, resides within a
purple box) has three groupings of modules: one for each of the three consecutive English
sentences that are supplied to it during each episode of learning.
• As learning progresses (see Chapters 6 and 7 for a more detailed description of how learning
works for the sentence continuation architecture – this one is similar), the thousands of
knowledge bases are progressively filled out via multiple passes through the learning examples.
• Once learning has ended, the knowledge bases, with billions of knowledge links, are frozen.
• Then, pairs of consecutive context sentences from fresh news stories are entered into the first
two module groupings and a thought process carried out on the third grouping then generates a
plausible third sentence.
• Examples of generated third sentences are provided below, using the above color scheme.
Sentence Generation: Add a Plausible Next Sentence to
two previous Context Sentences
Several other centenarians at Maria
Manor had talked about trying to live
until 2000, but only Wegner made it.
95
Her niece said that Wegner had always been
a character – former glove model , buyer for
Macy's, owner of Lydia's Smart Gifts
downtown during the 1950s and '60s – and
that she was determined to see 2000 .
second context
sentence
first context
sentence
confabulation
architecture
plausible next
sentence
She was born in the Bronx Borough of New York City.
Chancellor Project
© 2006 Fair Isaac Corporation.
Context sentences from the Detroit Free Press
CONVERSATIONAL RESPONSE GENERATION
• Here again is the example considered at the beginning of the presentation.
• There is obviously similarity between the manner in which this confabulation architecture
acquires its capabilities (exposure to well-crafted example sentence triples) and the manner in
which a human child acquires language skills. See Chapters 6 and 7 for more discussion.
• Unlike today’s information processing systems, no software, algorithms, rules, ontologies,
grammars, etc. are used.
• Notwithstanding the spectacular capabilities of such architectures, it is important to emphasize
the limitations of this approach. For one thing, there cannot be any reasoning. Thus, applications
of these primitive initial capabilities must be crafted to live within the rather strong limitations of
creating responses that constitute one, or perhaps a small number, of sentences – probably
addressing only one issue at a time.
• The neurotechnology challenge is to take the capabilities that can be implemented using canned
thought process control and successfully use them to build high-value applications.
• Clearly, an exciting era now begins for both neuroscience and neurotechnology.
Plausible Next Sentence
Experimental Results I
96
Seeing us in a desperate situation, the Lahore airport authorities switched on the runway lights
and allowed us to land with barely one to two minutes of fuel left in the aircraft, he said.
At Lahore, Pakistani authorities denied Saran's request to accept wounded passengers and
women and children, but they refueled the plane.
Airport authorities said they were not consulted beforehand.
Michelle strengthened from a Category 2 to a Category 4 storm Saturday, with winds reaching
140 mph, but it was expected to weaken before it reached Florida.
The storm or its effects could strike the Keys and South Florida tonight or early Monday, said
Krissy Williams, a meteorologist at the National Hurricane Center in Miami.
Forecasters warned residents to evacuate their homes as a precaution.
But the constant air and artillery attacks that precede the advance of Russian troops have left
civilians trapped in southern mountain villages, afraid to venture under the bombs and shells
raining on the roads, Chechen officials and civilians said.
Residents of the capital Grozny who had fled the city in hopes of escaping to Georgia, which
borders Chechnya to the south, have been stuck in the villages of Itum-Kale, 50 miles south of
Grozny, and Shatoi, 35 miles south of Grozny.
Russian forces pounded the strongholds in the breakaway republic.
© 2006 Fair Isaac Corporation.
Context sentences from the Detroit Free Press
Chancellor Project
CONVERSATIONAL RESPONSE GENERATION
• Again, you are invited to consider the degree to which this system is “intelligent”.
• As you carefully read each example, consider what this ‘purple box’ must “know” about reading
and interpreting English sentences, about the objects in the world in general, and about the
construction of English sentences, in order to craft its output sentence.
Plausible Next Sentence
Experimental Results II
97
A total of 22 defendants were convicted after the five-month trial of possessing explosives and
plotting terrorist acts, but all were acquitted on charges that they were linked to the Al Qaeda
terrorist network.
Jordanian authorities now have a second chance on the Hijazi case.
The defendants are accused of conspiring with the outlawed rebel group.
The doctrine is frank about Russia's economic weaknesses, calling for efforts to strengthen the
economy in order for the country to remain a major power.
It acknowledges that it is in Russia's interest to maintain its economic links to the outside
world and there is no suggestion that it intends to abandon free market principles.
President Boris Yeltsin has expressed his willingness to compromise.
Investigators say one man who got his license through a fixed test was Ricardo Guzman, the
driver of a truck involved in a 1994 wreck in Wisconsin that killed six children in a burning
minivan.
Prosecutors say Bauer, now retired, hastily shut down the probe of the accident and blocked
other investigations that might have embarrassed Ryan.
The driver fled the scene after the collision.
© 2006 Fair Isaac Corporation.
Context sentences from the Detroit Free Press
Chancellor Project
CONVERSATIONAL RESPONSE GENERATION
• Some of these examples may seem out of date. This is because the material used for learning
was mostly from the 1990s.
• The context sentences are more recent.
Plausible Next Sentence
Experimental Results III
98
The National Corn Growers Association says Gore is likely to have an ear of corn following him
too if EPA sides with California officials, who oppose using ethanol.
Ten days before the Iowa caucuses, Gore was more than 20 points ahead of Bradley in
various Iowa presidential polls.
Gore's aides said they would not have any problems.
The incident threatens relations between the Americans and Kosovo civilians, whom the
peacekeepers were sent to protect after the 78-day NATO bombing campaign.
We don't want them here to give us security if they are going to do this, said Muharram
Samakova, a neighbor of the girl's family.
NATO has struck a military airfield near Pale.
Now, I must admit that I'm not so sure the Palestinians really wanted to reach a framework
agreement, Eran said Tuesday.
Eran wondered aloud whether the Palestinian strategy might be to negotiate as much land as
possible in the remaining transfers, then declare statehood unilaterally – as the Palestinians
have threatened to do before when talks bog down.
Netanyahu said the Palestinians would be barred from jobs in Israel.
© 2006 Fair Isaac Corporation.
Context sentences from the Detroit Free Press
Chancellor Project
CONVERSATIONAL RESPONSE GENERATION
• People are often struck by the “human look and feel” of these examples.
• This instructs us that applications which are based upon mimicking a particular skilled human
performance that does not involve reasoning (e.g., immediate, ‘reflexive’, conversational
response) might be possible using this “canned thought control” approach.
• Preliminary experiments with a primitive conversational response system at Fair Isaac
V
!
Corporation (the acations ™ system – which converses with users via typed English text
about their favorite vacation trips) indicate that this technical approach may work for building
conversationally interactive systems.
Plausible Next Sentence
Experimental Results IV
99
The shortage has been attributed to rapid expansion of the prison system, low pay, a
booming economy that makes the prospect of spending the day guarding convicts less
attractive, and the risks of dealing with inmates who seem to be getting meaner and more
violent.
Prison officials are scrambling to keep penitentiaries staffed, recruiting at schools and from the
Internet.
Prison officials are still debating what they have to do.
Outside investigators announced the conclusions Tuesday as NASA's top scientist confirmed
that the agency will cancel plans to launch a robot spacecraft in 2001 on a mission to land on
Mars and indefinitely postpone all future launches to Mars, with one exception: a 2001
mission.
With only its aging Mars Global Surveyor in orbit around Mars, the agency is reassessing its
entire approach to the exploration of the planet after losing all four of its spacecraft bound for
Mars last year – a package totaling $360 million.
Mars Global Surveyor will be mapping out the planet.
Chancellor Project
© 2006 Fair Isaac Corporation.
Context sentences from the Detroit Free Press
CONVERSATIONAL RESPONSE GENERATION
• Clearly, whatever “intelligence” this purple box possesses is the product of using a huge number
of knowledge links between co-occurring symbols.
• Confabulation theory contends that this peculiar type of symbol co-occurrence knowledge is the
only cognitive knowledge animals possess.
• Neurophilosophers will now have a lot of work to do in re-interpreting human and animal nature.
• One (surely overly harsh) interpretation of the confabulation theory insights is that humans are
not really ‘operationally’ intelligent at all: we are merely actors that can give a convincing
performance of being intelligent. Only when we ‘cheat’, by applying reasoning over a very long
span of time to an externally collected and graphically expressed body of data (such as in a
Ph.D. dissertation or a published book), can we achieve ‘genuinely’ intelligent output.
• Another interpretation is that humans are like honeybees: as a collective ‘colony’ we have
created a vast ‘hive’ with huge, temporally durable, ‘honeycombs’ of refined knowledge, data, and
performance skills. Individually, humans vary enormously in their possession and mastery of the
contents of the ‘honeycombs’. It is the accumulated mass of these ‘honeycombs’ that so
dramatically distinguishes and ennobles humans in comparison with all other Earth animals. It is
definitely time to comprehensively archive and safeguard the entire human legacy.
Plausible Next Sentence
Experimental Results V
100
However, despite his acquittal by the Senate, Clinton still faces a continuing investigation by
Independent Counsel Robert, who has said he has hired additional prosecutors and is
considering whether to indict Clinton after he leaves office.
Clinton said that I wouldn't be surprised by anything that happens but I'm not interested in
being pardoned.
Starr is investigating the Clintons' Whitewater affair in Arkansas.
In one violent showdown in front of the Treasury Building a block from the White House, a
few hundred demonstrators charged a barricade and faced a counter-assault from police
swinging billy clubs and squirting pepper spray.
Closer to the IMF building police discharged a canister of bright green ammonia gas to
disperse a crowd surrounding a police bus on G Street, near George Washington University.
The protesters hurled stones at the riot shields.
He started his goodbyes with a morning audience with Queen Elizabeth II at Buckingham
Palace, sharing coffee, tea, cookies and his desire for a golf rematch with her son, Prince
Andrew.
The visit came after Clinton made the rounds through Ireland and Northern Ireland to offer
support for the flagging peace process there.
The two leaders also discussed bilateral cooperation in various fields.
© 2006 Fair Isaac Corporation.
Context sentences from the Detroit Free Press
Chancellor Project
CONVERSATIONAL RESPONSE GENERATION
• Consider, in detail, the content of the generated sentences of the examples presented in these
slides.
• Note the logical consistency of the statements made with the context provided. Bridging from
the context to this output content goes vastly beyond simple ‘word and phrase associations’.
• This illustrates another emergent property of confabulation theory: multiconfabulation conclusions
automatically inherit the strongly causal logical structure of the physical world.
• In his pioneering investigations into causality, Judea Pearl made the assumption that the proper
conclusion to seek in any situation is the one with maximum a posteriori probability (see Chapter
3). Computer experiments shows that confabulation architectures provide multiconfabulation
consensuses that are inherently mutually ‘logically consistent’ with one another. This logical
consistency arises both because of the mutual constraint satisfaction inherent in
multiconfabulation and because of the intrinsically logical character of maximal cogency
conclusions.
• As confabulation architectures are augmented with additional brain machinery (basal ganglia,
cerebellum, pons, substantia nigra, subthalamus, limbic system, etc.), many new fundamental
issues will surely arise. Great adventure lies ahead.
101
SPEECH UNDERSTANDING
Speech Understanding Confabulation Architecture
© 2006 Fair Isaac Corporation.
• The computer confabulation experiments presented above were all involved with thalamocortical
modules aimed at representing language objects (words, phrases, sentences, etc.).
• Here, a module representing short-time-duration speech sounds is shown in action. At the top is
the raw incoming soundstream (plot of differential sound pressure vs. time). This is essentially
the signal impinging on the ‘eardrum’ of the system.
• Beneath the raw soundstream, the time-varying responses of the intermediate representations
(features) used in this system are shown. These are crudely analogous to the outputs of the
‘brainstem auditory nuclei’ of the system.
• The black rectangle at the right of the frame shows a primary sound input module with 10,592
symbols. Each significantly excited symbol is shown as a dot within this rectangle.
• Note that, as with all primary modules (sensory, action, etc.), multiple symbols are typically being
expressed by the module at each moment.
• As you watch the video, note that each symbol is expressed for only a brief moment. The
duration of each symbol is not fixed: it keeps responding as long as the soundstream
components that excite it stay active.
• The sound symbols are connected with symbols in other modules using knowledge links.
Speechstream Segmentation
© 2006 Fair Isaac Corporation.
102
SPEECH UNDERSTANDING
• Here, symbol changes occurring on a ‘higher-level’ sound module (not shown) are used to
effectively segment the incoming soundstream in time (as shown by the vertical lines in the
lower sound feature flow).
• Note that the segmentation boundaries includes points in time that are almost always close to the
actual word boundaries (as shown by the vertical lines in the upper sound feature flow).
• Via knowledge links connecting sound modules with modules of a language system, this speech
understanding system is able to apply short-term and long-term context (established by the
previous content of the current conversation and by past conversations with this same system
user) to attentional focusing (at the primary sound representation module level) on only the
soundstream components associated with the attended speaker (with those components
associated with other sound sources being ignored) and to disambiguation of each word
spoken by the attended speaker.
• Chapter 7 provides many details about how a speech understanding confabulation architecture
can carry out speaker sound separation and word disambiguation to potentially achieve ultrahigh accuracy speech understanding.
Chancellor
103
– Cat Food for Zeus
Chancellor
© 2006 Fair Isaac Corporation.
• The Cat Food for Zeus vignette is revisited.
• Given the preceding discussion; it is now clearer that building a product of this type may be
possible in the near term.
• “Canned Thought Process Controllers”, along the lines of those demonstrated in this
presentation, may be sufficient for construction of many high-value applications.
GhostWriter TM – A Giant Step Beyond Word Processing
© 2006 Fair Isaac Corporation.
104
GhostWriter is a trademark of Fair Isaac Corporation.
• Here, the idea is that text generation might be controlled via interactive voice commands.
• First, the overall genre of the document to be produced is verbally selected by the user from a
huge menu (the system has undergone a many-stage learning process that includes examples of
hundreds of millions of documents from hundreds of genres).
• Once the user has selected the document type and mode of address, the system writes the
document outline under iterative conversationally interactive direction from the user.
• After the outline is finished, the user verbally guides the system in defining the tone and topical
approach of each lowest-level section as it writes.
• This hypothetical product illustrates that many other sorts of functionalities (beyond
conversational customer service) may be possible.
• A world without keyboards and mice will quickly create a demand for a single personal
conversational interface that gains vast experience interacting with that person. This way, the
accumulated history of interaction, preferences, history, and so much else connected with that
individual, can be accumulated (and held confidential). The hypothetical Chancellor product is
envisioned as fulfilling this secure universal personal conversational interface function for all
‘back-end’ applications.
Chancellor – Shopping for a Handbag at EarthMall TM
© 2006 Fair Isaac Corporation.
105
EarthMall is a trademark of Fair Isaac Corporation.
• With the astounding virtual worlds already produced by the computer game industry, it is not
much of a leap to imagine a universal Earth-wide shopping center with hundreds of thousands of
‘virtual stores’. Retailers, Business-to-Business enterprises, Governments, NGOs, and so much
more, will be the tenants at EarthMallTM.
• Shopping will be an interactive experience via a personal navigable avatar (either featureless,
and therefore anonymous, or with a person’s face), with virtual hands, that can be navigated
around EarthMall.
• You can meet your pals and attend a movie together at a theater, or go shopping together.
• Here, a shopper is examining handbags at a retail outlet via a projection from the Chancellor unit
on the table. Her avatar’s virtual hands (maneuvered by the Chancellor unit via interpretation of
the shopper’s own hand movements; as visualized with its built-in camera) can pick up the
mathematical model of the bag and check out its features.
• Sales clerks (both human avatars and automated) are always available to help with shopping.
• With EarthMall, the need for physical stores may largely disappear.
• This example illustrates the potential social impact of confabulation theory.
Fair Isaac Chancellor Project
106
► For more conversational machine vignettes, visit:
fairisaac.com/chancellor
© 2006 Fair Isaac Corporation.
• More hypothetical vignettes are available at this website (as of Summer 2006).
Summary
► Motivational Example
► Confabulation Theory Overview
– Cortical Representation of the Objects of the Mental World
– Acquisition and Storage of Cognitive Knowledge
– Confabulation
– The Origin of Behavior
– Confabulation Mathematics
► Confabulation Neuroscience
► Confabulation on a Computer
– A Single Confabulation
– Multiconfabulation
► Practical Applications
–
Chancellor Project
• That’s it!
• Hope you enjoyed this presentation.
107
Research Sponsors
The support of this research by:
Fair Isaac Corporation (NYSE: FIC)
Office of Naval Research
is gratefully acknowledged.
• Our sponsors’ support is greatly appreciated.
108
Domestic Cat Zeus Hecht-Nielsen
109
Official Portrait
(shown with his favorite toy – a purple string)
Tranquility Base Here
Prowling the Track
Rolling Behavior on the Track
(Hecht-Nielsen Compound – Park Side)
Awaiting Ingress
(Hecht-Nielsen Compound – House Side)
Breakfast Time
• These photos were taken by the author on 19 March 2004.
© 2006 Robert Hecht-Nielsen. All rights reserved.
Co-Investigator, Behavioral Observation Subject, Muse
An Indispensable Contributor to the Confabulation Discovery
Portrait
2 February 2006
Prowling the Track
Penthouse Spiral Stairway
Sure Footed
Track to Driveway Transition
Garden Observation Post
© 2006 Robert Hecht-Nielsen. All rights reserved.
Zeus Hecht-Nielsen wearing his confabulation theory commemorative medallion
110
• These photos were taken by the author on 2 February 2006.
• At the end of the video presentation, following this viewcell, a brief excerpt of a Zeus movie is
shown.
• For more details on Zeus Hecht-Nielsen, see Chapters 3, 5, and 8.