Mental Functioning and the
Ontology of Language
Barry Smith
October 1, 2012
with thanks to Janna Hastings
1
Basic Formal Ontology
BFO:Entity
BFO
BFO:Continuant
BFO:Independent
Continuant
BFO:Dependent
Continuant
BFO:Occurrent
BFO:Process
BFO:Disposition
2
Basic Formal Ontology
and Mental Functioning Ontology (MFO)
BFO:Entity
BFO
BFO:Continuant
BFO:Independent
Continuant
BFO:Occurrent
BFO:Dependent
Continuant
MFO
BFO:Process
Bodily Process
Organism
BFO:Disposition
BFO:Quality
Mental Functioning
Related Anatomical
Structure
Cognitive
Representation
Mental Process
Behaviour
inducing state
Affective
Representation
3
Basic Formal Ontology
and Mental Functioning Ontology (MFO)
BFO:Entity
BFO
BFO:Continuant
BFO:Independent
Continuant
BFO:Occurrent
BFO:Dependent
Continuant
MFO
BFO:Process
Bodily Process
Organism
BFO:Disposition
BFO:Quality
Mental Functioning
Related Anatomical
Structure
Cognitive
Representation
Mental Process
Behaviour
inducing state
Affective
Representation
4
Functions vs. Functionings
Continuants vs. Occurrents
BFO:Entity
BFO
BFO:Continuant
BFO:Independent
Continuant
Organism
BFO:Occurrent
BFO:Dependent
Continuant
MFO
BFO:Process
Bodily Process
BFO:Disposition
BFO:Quality
Cognitive
Representation
Mental Process
Mental
Function
Mental
Functioning
5
Aboutness (‘Intentionality’)
BFO:Entity
BFO
BFO:Continuant
BFO:Independent
Continuant
Organism
BFO:Occurrent
MFO
BFO:Process
BFO:Dependent
Continuant
Bodily Process
BFO:Disposition
BFO:Quality
Cognitive
Representation
Mental
Function
Mental Process
Mental
Functioning
does all mental functioning involve cognitive
representation (aboutness)?
what is aboutness?
6
Mental Functioning Ontology (MF)
7
Linguistic Functioning Ontology
(1. Speech and hearing)
BFO:Entity
BFO
BFO:Continuant
BFO:Independent
Continuant
BFO:Occurrent
BFO:Dependent
Continuant
MFO
BFO:Process
Bodily Process
BFO:Disposition
BFO:Quality
Linguistic
competence
Behaviour
inducing state
Speech competence of a
population
= a [spoken] language
Speech competence of
an individual
Cognitive
Representation
Speech
process
Speechmediated
cognitive
representation
Hearing
(registering)
8
process
8
Linguistic Functioning Ontology
(2. Reading and writing)
BFO:Entity
BFO
BFO:Continuant
BFO:Independent
Continuant
BFO:Occurrent
BFO:Dependent
Continuant
MFO
BFO:Process
Bodily Process
BFO:Disposition
BFO:Quality
Linguistic
competence
Behaviour
inducing state
Written linguistic
competence of a
population
= a [written] language
Written linguistic
competence of an
individual
Cognitive
Representation
Writing
process
Writtenlanguagemediated
cognitive
representation
Reading
(registering)
9
process
9
Linguistic Functioning Ontology
(the whole thing)
BFO:Entity
BFO
BFO:Continuant
BFO:Independent
Continuant
BFO:Occurrent
BFO:Dependent
Continuant
MFO
BFO:Process
Bodily Process
BFO:Disposition
BFO:Quality
Linguistic
competence
Behaviour
inducing state
Linguistic competence
of a population
= a language
Linguistic competence
of an individual
Cognitive
Representation
Writing
Languagemediated
cognitive
representation
Speaking
Reading
10
10
How to build MF±Language
First idea:
Along what dimensions human beings
can depart from normal (‘canonical’)
mental and linguistic functioning?
11
http://bioportal.bioontology.org/ontologies/45578/
from WHO | World Health Organization
12
For ICD mental function is a subtype of
body function
ICD definition of ‘Body functions’:
Body functions are the physiological
functions of body systems (including
psychological functions).
Impairments are problems in body
function or structure as a significant
deviation or loss.
13
http://bioportal.bioontology.org/ontologies/45578/
14
http://bioportal.bioontology.org/ontologies/45578/
15
Definition of ‘Mental Functions’
Definitions: 'This chapter is about the
functions of the brain: both global
mental functions, such as
consciousness, energy and drive, and
specific mental functions, such as
memory, language and calculation
mental functions.'
16
17
18
Quality ≠ Function
19
20
21
Dreaming?
22
http://apps.who.int/classifications/icd10/browse/2010/en#/F44.8
23
http://bioportal.bioontology.org/ontologies/45578/
24
http://bioportal.bioontology.org/ontologies/45578/
25
26
Ontological traffic rule:
always use singular nouns
27
28
29
Mental Functioning Ontology (MF)
30
Mental Functioning Ontology (MF)
31
Definitions: 'Specific mental functions
of focusing on an external stimulus or
internal experience for the required
period of time.'
32
33
Mental Functioning Ontology (MF)
brainin
endocrine
gland
34
35
Mental Functioning Ontology (MF)
36
Mental Functioning Ontology (MF)
brainin
olfactory
system
37
38
Mental Functioning Ontology (MF)
brainin
olfactory
system
39
Again, confusion of function and quality
40
41
Integrative language functions
Definitions: 'Mental functions that organize semantic
and symbolic meaning, grammatical structure and
ideas for the production of messages in spoken,
written or other forms of language.'
42
WHO ICF Pro
Performs an amazingly difficult task of unifying
public data across the planet (Versions in 5
different languages, plus Youth Version)
Serves billing task in US
http://apps.who.int/classifications/icfbrowser/
43
http://cirrie.buffalo.edu/icf/
44
WHO ICF Con
Use of ‘other’ and other features make it is not
future proof (will not evolve gracefully with
advances in neuroscience
Poor or absent definitions will mean
inconsistent annotation of data
Lack of logical definitions would mean poor
support for computational reasoning with data
that is annotated
45
Strategy for MF Ontology
• Ensure that the ontology is developed in
conformity with basic biology ontologies such
as the Gene Ontology
• Provide human-readable definitions for each
term to ensure consistent annotation
• Provide logical definitions to support
reasoning with data
46
Examples from
MF-Emotion Ontology
http://bioportal.bioontology.org/ontologies/49078/
47
‘emotion
process’
48
‘physiological response to emotion’
need link to a physiology ontology
49
Mental Functioning is Neural Functioning:
Towards a Unified Ontology of
Mind, Brain, and Behavior
Gwen A. Frishkoff
Department of Psychology
Georgia State University
NeuroInformatics Center
University of Oregon
Outline of Talk
• What is a mental process?
– A view from cognitive psychology
– The Mind–Brain problem and three proposed
solutions (ontology views)
• A neurophsysiological framework for
understanding mental processes
– Levels of brain, levels of mind
– What are mental representations “about”?
(Proposed solution to problems of subjectivity, aboutness)
What is a Mental Process?
A view from cognitive psychology
Sensation,
Perception
Short-term memory
Cognitive control
Motor control,
Action
Long-term Memory
Habits & Skills
How do we know any of this?
That is, where did the components of the
standard model come from?
The mind as a black box
•
•
X
Mental processes cannot be observed.*
They must be inferred based on what we can observe.
What can we observe?...
*We can revise this assumption later (if Mind = Brain)
What we can observe… and How
• Physical processes in body  Behavior
(response type, accuracy, reaction time)
•
Physiological processes in brain 
Neural activity and correlates of neural
activity (blood flow to brain regions)
A schematic of Helmholtz’s apparatus
for measuring the time course of
muscle contraction and the
propagation velocity of the nerve
impulse. Source: Bennett, 1999.
CogPO!
A 256-channel electrode “net” that is used
to measure brain electrical activity (EEG)
“A mental process is a neural process.”
ARGUMENTS IN FAVOR
• Avoids Mind-Body dualism
• More precise than other two
solutions
• Gives ready framework for
comparative neurophysiology &
comparative cognition
• Knowledge of brain structure &
function informs understanding
of mental function (and
dysfunction)
Mental Functioning Ontology (MF)
brainin
endocrine
gland
57
brain
endocrine
gland
ENVIRONMENT
Mental Functioning Ontology (MF)
58
brain
endocrine
gland
ENVIRONMENT
Aboutness
59
Levels of brain, levels of mind
Representation, monitoring and
control of bodily interface to
external environment
(“real world”)
Representation, monitoring and
control of internal environment
(“self”)
Mesulam, 1990
61
Levels of brain, levels of mind
Representation, monitoring and
control of bodily interface to
external environment
(“real world”)
Note use of “sneer” quotes
– “real world”, “self”
Representation, monitoring and
control of internal environment
(“self”)
Mesulam, 1990
62
Mental representations: What are they “about”?
Sensoy-motor maps in the brain
Peripheral (sensory-motor) parts of the body
are “mapped” to (represented by) an orderly
set of discrete regions within sensory and
motor cortex.
Maps of the internal milieux
The Hypothalamic-Pituitary-Adrenal (HPA) Axis
monitors and controls internal bodily functions, such
as blood circulation, breathing, digestion, stress, and
arousal.
Shimon Edelman’s
Riddle of Representation
two humans, a monkey, and a robot
are looking at a piece of cheese;
what is common to the representational
processes in their visual systems?
64
Answer:
The cheese, of course
65
The real cheese
66
Perception of
internal
(bodily)
environment
(“self”)
Perception of
external
environment/
sensory input
(“real world”)
69
Mental processes, states, and representations
or objects or processes
inside the body
or objects or processes
inside the body
objects and processes inside and
outside the body play a role here too
all of these together
form the environment
external targets
internal and external features causally
relevant to perception, nociception, etc.
the arrow of aboutness
external
targets
internal and external features causally
relevant to perception, nociception, etc.
Brentano-Husserl approach to
intentionality
76
intentionality can be mediated by
language
“food”
77
Brentano, Husserl, Chisholm Searle:
the primacy of the intentional
linguistic expressions have meanings, because there
are mental experiences which have aboutness
Roderick M. Chisholm, “The Primacy of the
Intentional”, Synthese, 61, 1984, 89-109
78
the primacy of language (Sellars …): mental
experiences are about objects because words
have meaning
meaning
79
to understand the aboutness of the mental,
study the semantics of language (model theory)
meaning
80
language comes later than mental
aboutness
81
What is a Mental Process?
A view from cognitive psychology
Sensation,
Perception
Short-term memory
Cognitive control
Motor control,
Action
Long-term Memory
Habits & Skills
All of this is present before there is language
Thesis: aboutness is a primitive relation
between a mental process and a target
external to that mental process
Problems for this thesis:
1. mismatch
2. non-existence
1. mismatch of content to target
“poison”
• the apple is poisonous
• the apple is not poisonous
two phenomenologically identical mental
experiences
84
1. mismatch of content to target
“poison”
• the apple is poisonous
• the apple is not poisonous
two neurologically identical mental
experiences
85
2. there is no target
“unicorn”
86
Information artifacts, too, involve
aboutness, and the same 3 kinds of
mismatch
BFO:Continuant
BFO:Independent
Continuant
BFO:Dependent
Continuant
can be copied
BFO:Generically
Dependent
Continuant
concretized in
a bearer
Information
Content Entity
is about something
(anything)
$64,000 problem of providing a
coherent account of intentionality
Neurology-based solution: we are never
directed towards real objects in any case, but
only to “real objects”
Let us find an easier, neutral, route to building
an ontology which does not rest on finding a
solution to this problem
88
Ontological traffic rule:
to build an ontology of the types of
entities in a complex domain, focus on
the canonical instances
89
Canonical fear
fear
subtype
canonical
fear
EMOTION COMPONENT
Action tendency
Subjective emotional feeling
Behavioural response
Characteristic appraisal
CHARACTERISTIC FOR FEAR
Fight-or-flight
Negative, tense, powerless
Characteristic fearful facial
expression
Something (some real thing) in
my environment is dangerous
to me
90
Canonical and non-canonical fear
Canonical fear gives rise to action tendencies
that are conformant to a perceived danger
Phobias = dispositions giving rise to noncanonical fear, e.g. laridaphobia
Another case involving non-canonical fear:
people taking pleasure in watching horror films
91
Canonical pain & variants
PCT: pain with concordant tissue damage: the
patient experiences pain of the evolutionarily most
basic sort = pain in response to concordant tissue
damage
Variant pain
PNT: pain with peripheral trauma but discordant
(elevated) relative to tissue damage: there is
peripheral trauma, but the patient is experiencing
pain of an intensity that is discordant therewith;
NN: neuropathic nociception: no peripheral
trauma, but the patient is experiencing pain in
result of a neuropathic disorder in the nociceptive
system.
92
93
Pain-related phenomena without pain
PBWP: pain behavior without pain: there is a
cry or report of pain, but no pain is being
experienced (a fact which may or may not be
detectable by an external observer)
TWP: Tissue-damage without pain: tissue
damage normally of the sort to cause pain does
not activate the pain system.
94
Pain Ontology (PN) branch of MF-EM
Lying
about pain
95
Canonical pain
pain
canonical
pain
EMOTION COMPONENT
CHARACTERISTIC FOR PAIN
Action tendency
Withdrawal
Subjective emotional feeling
Negative, tense, powerless
Behavioural response
Characteristic painful facial
expression
Characteristic appraisal
Something is dangerous to me
97
How shall we structure the MF
ontology?
100
simple object-presenting acts vs.
judgments, evaluations, …
mental process
content
(putative) target
content of presentation
presenting act
object of presentation
“apple”
judgment-content
judging act
“the apple over there is
ripe”
evaluating act
emotional act
appraisal
…
“it is good that the apple
over there is ripe”
state of affairs
objective, fact
?
101
Successful intentionality
mental process
content
presenting act
content of presentation
(putative) target
object of presentation
target
present
target
absent
“apple”
• target present = you are in physical contact with target
• successful intentionality
102
Successful intentionality
mental process
content
(putative) target
presenting act
content of
presentation
object of presentation
target
present
+
evidence
target
Absent
+
evidence
“apple”
–
evidence
• target present = with direct evidence
• target absent = with indirect evidence, with no evidence
at all
103
relational acts
• include also cases of unconscious awareness,
e.g. of the chair that you are sitting on
104
Veridical intentionality
mental process
content
presenting act
content of presentation
(putative) target
object of presentation
“apple”
target
present
target
absent
object
exists
object does
not exist
ordinary perception
105
Veridical intentionality
mental process
content
presenting act
content of presentation
(putative) target
object of presentation
“apple”
target
present
target
absent
object
exists
object does
not exist
veridical thinking about
106
Non-veridical intentionality
mental process
content
(putative) target
presenting act
content of presentation
object of presentation
target
present
target
absent
“apple”
object
exists
object does
not exist
non-veridical thinking about (error,
hallucination, imagination, …)
107
Non-veridical intentionality
mental process
content
presenting act
content of presentation
(putative) target
object of presentation
“apple”
target
present
target
absent
object
exists
object does
not exist
error, hallucination = the presenting
process is dependent on an
underlying false belief
108
Non-veridical intentionality
mental process
content
presenting act
content of presentation
(putative) target
object of presentation
“apple”
target
present
target
absent
object
exists
object does
not exist
thinking about Macbeth = the
presenting process is not dependent
on an underlying false belief
109
An excluded case
mental process
content
presenting act
content of presentation
(putative) target
object of presentation
“apple”
target
present
target
absent
object
exists
object does
not exist
this combination is impossible
110
Ontological traffic rule:
to build an ontology of the types of
entities in a complex domain, focus on
the canonical instances
– in the Macbeth case we are dealing
with what happens when language goes
on holiday
111
mental act about a
real-world object
relational
(~ perception)
content
match
content
mismatch
veridical
non-relational
(~ linguistic)
content
match
content
mismatch
non-veridical
112
Veridical intentionality
mental process
content
presenting act
content of presentation
(putative) target
object of presentation
“apple”
target
present
target
absent
object
exists
object does
not exist
ordinary perception
evolutionarily most basic case
113
114
cognitive
representation
115
116
An emotion trichotomy
Occurrent emotion, e.g. when a person experiences
hate for another person on a specific occasion
Emotion disposition, e.g. when a person hates
someone for a long period of time (is predisposed
to occurrent emotions)
Personality trait = a predisposition to emotion
dispositions (e.g. sadness) of certain sorts (and thus
also to corresponding occurrent emotions)
Janna Hastings, Werner Ceusters, Barry Smith, Kevin Mulligan, “Dispositions
and Processes in the Emotion Ontology”, Proceedings of the 2nd International
Conference on Biomedical Ontology, 2011, 71-78.
117
A psychological trichotomy
Occurrent mental process, e.g. when Mary sees
that Jim has gone bald
Mental dispositions, e.g. when Mary thereafter
believes for a period of time that Jim has gone
bald
Psychological traits = predispositions to mental
cognitive dispositions (e.g. to beliefs) of certain
sorts
118
119
A psycholinguistic trichotomy
Occurrent psycholinguistic process, e.g. when
Mary reads that Jim has gone bald
Mental dispositions, e.g. when Mary thereafter
believes for a period of time what she has read
Psycholinguistic traits = predispositions to
psycholingistic dispositions of certain sorts
 including linguistic competence
120
121
122
Frishkoff: The mind as a black box
X
•
Mental processes cannot be observed.*
•
They must be inferred based on what we can observe.
What does a temperature chart
represent?
124
What does a chart representing your pulse rate represent?
85
80
75
70
65
60
Time 1
Time 2
Time 3
125
Cardiac Cycle, Left Ventricle
126
What does a chart of changes in your pulse rate
represent?
85
80
75
70
65
60
Time 1
Time 2
Time 3
127
What does a chart of changes in the Dow
Jones industrial average represent?
1399
1398
1397
1396
1395
1394
1393
1392
1391
time
128
activity during this
time interval
129
130
1398
1397
1396
1395
1394
1393
1392
1391
10.5810.59 11 11.0111.0211.0311.0411.0511.0611.0711.0811.09
11.1 11.1111.12
time
131
1398
1397
1396
1395
1394
1393
1392
1391
10.5810.59 11 11.0111.0211.0311.0411.0511.0611.0711.0811.09 11.1 11.1111.12 time
What this represents is real, and not just “real”
132
coronary heart
disease
disease during
phase of early
lesions and
small fibrous
plaques
instantiates
at t1
disease during
phase of
asymptomatic
(‘silent’)
infarction
instantiates
at t2
disease during
phase of
surface
disruption of
plaque
instantiates
at t3
unstable
angina
instantiates
at t4
stable
angina
instantiates
at t5
John’s coronary heart disease
What this represents is
real, and not just “real”
time
133
What did your temperature do over the last
month, Jim?
Jim’s temperature process profile, the
target of a certain sort of cognitive
selection, or cognitive profiling
134
The graph picks out just one dimension of
qualitative change within a much larger
conglomerate of processes within Jim
Hence ‘process profile’
135
Compare perception of polyphonic
music
• Cognitive selection of the cello part when you
listen to a string quartet
• Picking out a certain sonic partial process
within a larger body of vibrations
• Ignoring sneezes, coughs, …
• (or sometimes focusing on sneezes and
coughs for diagnostic purposes)
136
Compare perception of polyphonic
music
• Cognitive selection of the cello part when you
listen to a string quartet
• Picking out a certain sonic partial process
within a larger body of vibrations
• Ignoring sneezes, coughs, …
• (or sometimes focusing on sneezes and
coughs for diagnostic purposes)
137
time-series graph of acoustic signal,
spectrogram, formants, jaw displacement and
other speech parameters
138
adding phonetic, phonemic and syllable levels
139
g
u
t e n
140
add brain
141
speech is a process profile
the speech process is to the totality of acoustic
signal, spectrogram, formants, jaw
displacement, mental and neurological
processes
as
the pulse rate process is to the totality of aortic,
ventricular and atrial pressure, ventricular
volume, electrical activity, arterial flow, and
other processes in the heart
142
Breakthrough: First sound recordings based on reading
human auditory cortex (PLoS Biology, January 2012)
143
Top: spectrogram of words presented to subject.
Middle and bottom: reconstructions of speech based on
readings from electrodes attached to patient's brain.
144
biological pathways are process profiles
BFO: GDC
BFO:Independent
Continuant
BFO:Process
Information
Content Entity
has participant
implicitly
represents
Molecular
collective
Pathway
Pathway
diagram
BULK
has grain
MOLECULAR
explicitly
represents
Reaction
Individual
molecule
inheres in
BFO:Disposition
mental processes, too, are process profiles
146
what is a language?
something analogous to a biological
species (a population of competences)
BFO:Entity
BFO
BFO:Continuant
BFO:Independent
Continuant
BFO:Occurrent
BFO:Dependent
Continuant
MFO
BFO:Process
Bodily Process
BFO:Disposition
BFO:Quality
Linguistic
competence
Behaviour
inducing state
Linguistic competence
of a population
= a language
Linguistic competence
of an individual
Cognitive
Representation
Writing
Languagemediated
cognitive
representation
Speaking
Reading
154
154
• Examples of dispositions that are constantly
being realized:
– stock exchange
– heart beat
– brain activity
– social order
– language (social)
155