Ph.D. General Examinations
Technical Area Exam (Erik T. Mueller, Examiner)
Xinyu Hugo Liu
Submitted September, 2004
Exam Topic
Design, implement, and give analysis for an original narrative
comprehension system dealing in the realm of affect. Give a
discussion of the theory of being put forth and of the grounding for
such a theory. Also, discuss the strengths and limitations of the
system and relate this to the various theories in the “Techniques for
Narrative Comprehension with Imaginative Intelligence” literature
(cf. Ph.D. General Examinations Proposal document).
ESCADA: Experimental System for
Character Affect Dynamics Analysis
Abstract
This paper presents ESCADA, a narrative comprehension system
specialized to the analysis of how affect is interplayed between
story characters. The fundamental metaphor which underlies this
theory of affect dynamics is that of exchange. Affect-packages,
represented
in
Mehrabian’s
Pleasure-Arousal-Dominance
dimensional model, are possessed by and exchange between
characters and objects in the story world. One level of cognitive
reflexivity is addressed by modeling each character’s “mental
world.” A model of dominion and the exchange and possession of
physical objects is also implemented. Part of the theory of affect
dynamics developed in this paper is the role that objects play as
containers of affective-energy; the affective-energy of an object
affects the emotional state of the character who possesses it.
In this paper, first, we present motivation for our single-realm
approach to story understanding; second we discuss theory of
affect dynamics represented by this system with the assistance of
pertinent examples indicating the scope of the system’s capabilities;
third, we give a technical overview of the implementation; fourth,
we present the results of some indicative studies evaluating the
system’s performance; and fifth, we connect the ESCADA theory
and system to the various theories and systems in the “Techniques
for Narrative Comprehension with Imaginative Intelligence”
literature (cf. Ph.D. General Examinations Proposal document).
Introduction and Motivation
Traditionally narrative comprehension systems follow either a path
of representing the characters and the world in full detail, or of
producing very general characterizations of the document.
Following the path of full detail, all aspects of the world need to be
modeled from story text, necessitating both an enormous amount
of common sense knowledge properly formulated, and absolutely
precise deep semantic parsing capabilities; unfortunately both
prerequisite technologies are not and may not be available for quite
some time. However, rather than backing off from character-level
story understanding to document-level understanding, we take a
possible middle approach which affords more specific
understanding about characters and character interactions.
The idea is to understand in a medium amount of detail the world
under the lens of a particular realm; that entails interpreting all the
detailed events of the story by first projecting the detail into the
target realm were some interpretation can be had. Of course, the
success of a realm-specific understanding approach requires that
the realm be pertinent to a good portion of all the story details.
The realm which is the focus of our project is affect, and this seems
to be a particularly longitudinal realm, meaning that all aspects of
language and cognition have some meaningful projection in the
affect realm. David Gelernter explains the pervasiveness of affect
in human experience by suggesting that the role of affect in
cognition and memory is as an annotation mechanism which
facilitates the indexing of memories and thoughts into broader
themes, which he terms “affect linking” (Gelernter, 1994).
In order to project the consequences of story details into the
affective realm, some knowledge or linguistic resource would be
required. For the ESCADA system, we have mined and hand-
annotated three invaluable affective-lexical resources: Roget’s
Thesaurus of English Words and Phrases (Roget, 1911), the
Affective Norms for English Words (ANEW) Project at the
University of Florida (Bradley & Lang, 1999), and Beth Levin’s
English Verb Classes and Alternations (1993). Coupled with a
broad-coverage surface thematic role frame parser for English,
these affective-lexical resources enable the ESCADA system to
interpret directly from the language the affective consequences of
actions, modifiers, and (through some further reasoning), objects.
However, in light of the fact that such resources and mappings are
only approximate and are not fool proof, such resources are
interpreted by the system in a fail-soft (read: non-logical) way.
Without further ado, the following section presents a theory of
character affect dynamics which underlies ESCADA.
A Theory of Character Affect Dynamics
Let’s first consider a simple children’s story and walk through it
through the lens of the Character Affect Dynamics (CAD) Theory:
John was eating a delicious ice cream. Mary was
secretly jealous of John because she wanted his ice
cream. John didn’t know that Mary wanted his ice
cream. Suddenly, Mary snatched the ice cream from
John. John became very sad and resented Mary.
CAD advocates a squint-your-eyes approach to comprehension,
where entities are either Agents (characters) or Objects (especially
inanimate things). Agents and Objects can both be invested with,
and divested out affective energies, be they positive or negative,
and aggressive or submissive. If positive Objects are possessed,
they imbue the possessor with positive energies, but if positive
Objects are dispossessed, they bring woe to the dispossessor. If an
Agonist (acting agent) acts on an Antagonist (usually called
“patient” in the thematic role literature), in so doing he transfers an
package of affective energy to the Antagonist. Whether and how
the Antagonist reacts says something about the Antagonist’s affectrelationship with the Agonist, and about the Antagonist’s
personality archetype in general. Each Agent also possesses a
mental world, and how much an Agent utilizes his mental world
and the discrepancies between his formed mental world and the
actual world also help to betray his personality and the situation in
general. Before we discuss the theory any further, let us walk
through the above example using this theoretical framework.
Commentary appears indented
John was eating a delicious ice cream.
Delicious is positive; the IceCream Object is imbued with
positive affect; “to eat” implies possession, and so the Agent
John currently possesses the positive IceCream Object
Mary was secretly jealous of John because she wanted his ice cream.
The Agent Mary is experiencing a state of secret jealousy.
Our affective lexicon tells us that jealousy is negative,
aroused, and aggressive. The fact that the theme of jealousy
is modulated by “secretly” means that Mary’s jealousy is a
passive act against John; John does not necessarily know
about Mary’s feelings toward him. Also, Mary wants to
possess the ice cream. The causative link between the first
clause and dependent clause are not currently addressed by
our theory or implementation.
John didn’t know that Mary wanted his ice cream.
In John’s Mental World, imaginaryMary undesires the ice
cream and does not want to possess it. At this point, the
mismatch between the Real World and John’s Mental World
is a source of story conflict, and our theory can predict that
certain salient Real-Mental mismatches lead to conflict.
Also, we can make some predictions about the characters’
personalities to explain why John has false knowledge;
perhaps we can predict that John is oblivious or naïve or
insensitive; we could conversely blame Mary for being
calculating (since she has not directly communicated any
affect to John).
Suddenly, Mary snatched the ice cream from John.
The word “suddenly” informs us this is an aroused event.
Our affective lexicon says that “snatch” is a possession-verb
and is imbued with high aggression, high arousal and
slightly negative valence (thieves snatch). The act of Mary
snatching the ice cream from John means that John is
dispossessed of IceCream and Mary now possesses
IceCream. Mary benefits from the positive affect energy of
IceCream (minus whatever negative energy was imbued
upon IceCream in its participation in the negative
“snatching” event; every event affects every participating
agent or object). John is discouraged by the loss of a Positive
Object.
John became very sad and resented Mary.
From Mary’s previous act of aggression against John, John’s
reaction is telltale of the element of aggression in his
personality. However, instead of retaliating with something
aggressive, John is rather submissive; sadness is submissive
(in Mehrabian’s PAD model), and so is resentment. The fact
that John resents Mary is a purely passive and private act
which only affects John’s mental attitude toward Mary and
does not transfer any affective energies to Mary.
The type of spatial-metaphorical entity-imbuement and entityinteraction analysis employed in CAD theory has precedence in the
Artificial Intelligence literature. Jackendoff’s Trajectory Space
(1983) focuses on semantic mappings of text unto spatial
representations of objects moving along paths from sources to
destinations. Borchardt’s Transition Space (1990) represents a story
event sequence by tracking the differential change in the properties
of objects. Perhaps the closest work in the literature, however, is
Leonard Talmy’s force dynamics (1988).
The conceptual primitives in force dynamics theory are entities,
and the typical scenario calls for two entities. Each entity exerts a
force on the other and the sum of the forces yields in some
consequent. For example, if the forces are equally matched, the
result may be stalemate. CAD can be thought of in terms of
interactionary forces as well. We can think of a steadfast character
as a low-inertia entity, an aggressive character as one who exerts a
strong force, and so on and so forth. A further similarity between
Force Dynamics and CAD is that both take a cognitive linguistics
approach; both argue that inherent in basic language constructions
and in the lexicon are the needed semantics. Force dynamics uses
syntactical elements (e.g. against, despite, modals) and the
semantics of verbs such as “resisted,” “refrained” to animate an
underlying spatial representation. CAD utilizes the affective
connotations of words, and the spatial connotations of syntax (e.g.
transitivity, intransitivity, reflexivity, copula-constructions) and of
verb-semantics (e.g. passive versus active verbs) to animate an
underlying simulation of the story world.
As we have argued before, we believe there to be great benefits to
approaching story understanding in a single-realm, especially
when surface language can be used to directly map story events
into the underlying representation. Greater granularity can be
achieved than document-level characterization of text, without
requiring deep semantic parsing and not needing an enormous
corpus of common sense knowledge to aid in inferences. All the
resources needed for this approach can be found in existing
cognitive lexicon projects such as those which were mentioned in
the previous section.
Like Talmy’s force dynamics theory, the CAD framework takes a
cognitive linguistics approach to language understanding, and to
be sure, the understanding which is achieved here is only
approximate, heuristic, and connotative – concerned with the
affective gist of each linguistic transition – and the style of semantic
interpretation is also different from previous approaches. Whereas
traditional thematic role frame interpreters and frame-semantics a
la FrameNet maintains a fairly diverse and expansive case base of
types of situational semantic frames which can be parsed into
depending upon the verb-argument structure of a sentence, CAD
calls for relatively few types of cases, distinguishing only between
dominion transactions (e.g. John gave Mary the kite), affect
declarations (e.g. Mary was in a poor mood), passive affective
transactions (e.g. John resented Mary), active affective transactions
(e.g. Mary stabbed John and stole his ice cream), and contagion
transactions (e.g. The delicious ice cream; Mary’s wicked heart).
In the following subsections, we first discuss the representations of
affect and personality used in CAD theory, tying these to models of
personality from the literature; next, we discuss further grounding
for the model in psychology.
Representing Affect. In choosing a representation of affect, we
were careful to consider a model that could be generic enough to be
attributed to both Agents and Objects, and sensitive enough so that
even nuanceful affects could be captured elegantly. Immediately,
discrete ontological models such as Paul Ekman’s emotion
ontology (1993) of Happy, Sad, Angry, Fearful, Disgusted,
Surprised, derived from the study of universal facial expressions.
There is a sense that what we want to capture is not emotion, but
affect, including states which are subtle or have no linguistic label.
It was for this very reason of escaping the discourse of words
which motivated Picard to rename the discourse of emotions to that
of affect in light of computation (1997).
Our choice was the dimensional PAD model of affect proposed by
Albert Mehrabian (1995), which specifies three almost orthogonal
dimensions of Pleasure (vs. Displeasure), Arousal (vs. Nonarousal),
and Dominance (vs. Submissiveness). In this paper and in our
implemented system, the notation we will use is e.g. (P0.25 A0.5
D0.2). P0.0 is displeasure, P1.0 is pleasure; A0.0 is nonarousal, A1.0
is arousal; and D0.0 is submissiveness, D1.0 is dominance.
The PAD model is a unification model into which most of the other
models of affect can be mapped. For example, (P0.25 A0.5 D0.25)
might correspond to sadness, while (P0.25 A0.7 D0.8) would
correspond to anger. The only exception are models which include
directionality of affect. For example, “resentment” is an inwardly
kept affect whose corresponding outwardly directed affect is
“anger.” In CAD, directionality is explicitly represented, with
inwardly affect as those which reside in the Agonist’s Mental
World, and outwardly affect as those which are transferred as
affective-energy packages between Agonist and Antagonist.
Interestingly, in our review of existing emotion models, there is yet
another dimension in many accounts of human sentiment which to
our knowledge is not included in any dimensional model; this is
affective focus. Vulgarity is usually unfocused, whereas resentment
is usually focused. Incidentally, affective focus can be handled by
our CAD theory and system implementation, but its discussion is
beyond the scope of this paper.
Representing Personality. In representing personality in the CAD
theory, the primary consideration is to choose a representation
which most intuitively leverages the scaffolding established by
entity-imbuement and energy-transfer and the PAD affect model.
If personality is to arise intuitively from the CAD World model,
then there should be a complete analogical mapping between the
conceptual feature space of the World, and the space of personality.
From the personality literature, we were greatly inspired by three
sources: Jung’s theory of psychological types (trans. 1971), which
was augmented and re-presented as the familiar Myers-Briggs
Type Indicator (MBTI) personality test (Briggs-Myers & McCaulley,
1985); Carol Pearson’s six mythic archetypes (1998); and Karen
Horney’s theory of neurosis (1945), which articulates three
archetypical behaviors of Compliance, Aggression, and
Withdrawal.
The Jungian MBTI postulates four binary dimensions of
temperament:
Introversion-Extroversion,
Intuition-Sensing,
Thinking-Feeling, and Judging-Perceiving (which was added in
Myers-Briggs). We can relate MBTI to our CAD model as follows.
The first dimension describes the directionality of a person’s
psychic energies; in CAD, a character who commits passive acts
and prefers to hold inwardly directed attitudes may correlate to
Introversion, while a character who expresses his inner attitude
through extrinsic and social acts is Extroverted. The second
dimension describes the character’s chief mode of information from
the world. An Intuitive character is better able to guess (intuit) the
affective states of others, even when they are inwardly kept. For
example, if Mary is secretly sad, but John goes to console her, he is
probably Intuiting. Sensing character act primarily on what is
superliminally presented to them in explicit fashion. The third
dimension is Thinking-Feeling; a Thinker is likely to show more
cognitive planning activity and to trust his Mental World model,
while a Feeler’s decisions are dictated more by his felt affect. The
fourth dimension (not originally included in Jung’s theory of
psychological types) is Judging-Perceiving and is more tenuous to
computationalize in CAD because it requires the handling of more
advanced structures like causatives and hypotheticals, but the gist
is that a Judging character is likely to trust and follow the plan laid
out in his Mental World more than a Perceiver, who is more likely
to revise his conceptions and explore the space of alternatives.
Pearson’s six archetypes theory, a Jungian theory, is premised on
the idea that there exists a set of mythic roles which are engrained
into the unconscious of all humans past and present; these mythoi
are profound and universal. Pearson’s archetypes complement
rather than compete with psychological type theories like MBTI
because whereas MBTI describes general patterns of people’s
temperament, Pearson’s archetypes can be adopted semivolitionally by people. The six archetypes are Orphan, Wanderer,
Warrior, Altruist, Innocent, and Magician. A person may go
through all of these in various situations and at various stages in
her life. These archetypal roles are also useful ways to understand
the affect dynamics of characters in a story. For example, a
character who retaliates against an affect attack or fights hard to
improve his own mood may be described as a Warrior; a character
who gives positive affective energy to others in need may be
described as an Altruist; and finally, a character who is carefree yet
easily hurt or devastated might be an Innocent. At second glance,
Pearson’s archetypes are quite different from MBTI because
whereas the MBTI model describes an egocentric phenotype,
Pearson’s archetypes describe how an Agonist behaves in the face
of an Antagonist, and thus, it is more of an interactionist theory.
Horney’s theory of neurosis brings to light three further personality
phenotypes not addressed in MBTI. Horney postulates three broad
classes of coping strategies which people use to address their
neurotic needs. The first class is Compliance and this includes the
phlegmatic personality (sluggish), which in CAD manifests as an
unemotional character. The second class is Aggression and
describes a character who is Dominant more than Submissive. The
third class is Withdrawal and corresponds to a character who
avoids conflict rather than battling it. It is interesting the Horney
also uses a vocabulary of motion to describe these coping
strategies: Compliance, Aggression and Withdrawal are movingtoward, moving-against, and moving-away-from, respectively.
In the above presented personality theories, we’ve articulated
possible mappings into the CAD framework which we believe to be
intuitive, though admittedly these mappings are also heuristic.
Luckily, the science of Personality is also heuristic. While these
mappings illustrate that the CAD framework would be versatile
enough to accommodate all of these theories, we have selected out
aspects of each theory to construct a new model that would
resonate best and can be most readily computed from the CAD
framework. Below are the proposed dimensions of personality for
the CAD.
Inwardly-Outwardly: Similar to Jung’s Introversion-Extroversion
and Horney’s Withdrawn-NotWithdrawn. Inwardly-Outwardly is
heuristically measured by the ratio of a character’s tendencies to be
passive about affect, harboring it within, to the tendencies to be
active and extrinsic in communicating affect.
Thinking-Feeling: Straight from Jungian’s psychology type theory. A
Thinker is a character who engages in a lot of mental activity, and
involves cognitive plans in preparation for his affective acts. A
Feeler is a character who focuses primarily on feeling.
Unperceptive-Perceptive: Same as Jungian Sensing-Intuiting. A
character is perceptive if he is able to act in response to an
Antagonist’s inwardly feelings.
Uncontagiable-Contagiable:
How easily moved (in the same
direction) a character is by her contact with received affectiveenergies, and with the possession of affectively-charged objects.
Corresponds roughly to Pearson’s NonInnocent- Innocent and
MBTI’s Judging-Perceiving.
Submissive-Aggressive: Does a character usually submit to an
Antagonist, or does he aggress, initiate attack, defend, and
retaliate? Does he indulge more in submissive affects (e.g. sadness,
fear) or aggressive affects (e.g. angry)? This corresponds roughly to
Pearson’s
NonWarrior-Warrior,
Horney’s
AggressionNonAggression, and Mehrabian’s Submissiveness-Dominance.
Good-Evil: This notion may be too histrionic and draconian for real
life but it is entirely appropriate for story understanding. We can
qualify all acts and reacts in CAD as good or evil. For example,
doing something bad to someone good is evil, and taking away
something bad from someone good is good. Also deceptive
behavior, such as acting not in accordance with one’s inwardly held
feelings, is evil. Corresponds roughly to Pearson’s AltruistNonAltruist.
Unelastic-Elastic: Elasticity is proportional to the recoil time of a
character in light of distress, measured in narrative time from the
time that a character is inflicted with negative affect, to the
character’s recovery back to neutral or to positive affect. It does not
directly map into any of the three personality theories, although it
maps into archetype of obsession.
Admittedly these dimensions are not completely orthogonal, but
they are all directly available and computable from a computer
simulation of the CAD framework. Nor are these articulated
dimensions an exhaustive list of all the personality traits which can
be read off the CAD framework; however, they serve as a good
point of departure. Having described the representations of affect
and personality in CAD, the following subsection discusses the
psychological grounding for the theory.
Grounding the CAD theory. We have already discussed the
precedence in the Cognitive Linguistics literature for an agentobject-transfer model of affect dynamics; however, we have not yet
touched upon the psychological plausibility of conceiving of affect
as energy-packages which can be transferred or held, and
conceiving of objects as psychic holders of energy which can imbue
affect unto their owners; furthermore, we have not discussed the
validity of using an affective lexicon and syntactic construction to
populate events the affect realm.
In Metaphors We Live By (1980), Lakoff and Johnson put forth a
theory that all language and cognition is structured by basic
metaphors of containment, space, orientation, and movement.
They present linguistic evidence that we conceptualize of people as
CONTAINERS and interpersonal communication as flowing along
CONDUITS. In regard to emotions, they give the example, “I am in
love” to say that some emotions are containers. A person can be
inside love. We would say that the “emotions as containers”
metaphor only governs a few named and mythed states; love, long
romanticized as an institution of the human condition, is
appropriate as a public container; every person under the spell of
love is in that state. However, one would not say “I am in hate,” or
“I am in loathe.” Mostly the “emotions as states” metaphor is valid
for positively valenced emotions. There is also a cultural and social
bias at play. Love, friendship and other positive affects are public
and shared because it is socially couth to do so, but not negative
emotions. For example, you can “give her my love” and “offer her
my friendship” but not “give her my hate” or “offer her my
resentment.” However, we are quick to point out that this is most
applicable for named emotions, not necessarily for unnamed
affective states. Affective states, because they are not articulated as
emotions, remain my more personal, and we describe them as
attributes or entities contained within the self. “I have this fear in
me;” “There is a feeling in my heart;” “I am stricken by grief.”
These examples demonstrate that it is common for people to
conceive of themselves as containers, and to describe affect as that
with which they are infected or imbued.
In describing communication, Lakoff and Johnson suggest that the
CONDUIT metaphor is the most prominent. I have something I
wish to communicate and it is packaged up as an entity, and sent to
you, and you are to unpack it. If you are oblivious, you will not see
the package coming and will miss it. If you are dense, you will not
be able to unpack it properly. “He was giving her a hint.” Because
the conduit metaphor underlies communication, we would
speculate that in part, every directed communication act can be
conceptualized as a conduiting of something, regardless of whether
or not that metaphor is actively at play in structuring the linguistic
utterance; it will nevertheless structure our deep subconscious
conceptualization of each act.
The power of the CONTAINER and CONDUIT metaphors in
narratives may be even greater than in real life. Stories are a world
governed by imagination and the abstract, and thusly we speculate
that CONTAINER and CONDUIT being devices friendly to
abstraction and mental constructivism, are devices which the
reader utilizes to construct the story mentally. Each character is
constructed using our human faculties for Theory of Mind, a
concept from the Cognitive Science literature, using rational tools
such as Dan Dennett’s Intentional Stance (1987); on the flipside, the
aesthetic profile of the story also constructs in the reader’s
unconscious another dimension of understanding for the
characters. If indeed reading is an act of constructing characters
and populating their states and psyches, then perhaps it follows
that the reader will want to extract every bit of information from
the text, paying attention to the nuance of words. Verbs, it so
happens, contains a lot of nuanceful cues and implications along
different dimensions, and Cognitive Linguists know this well.
Marvin Minsky also knows this, and he proposed in Society Of
Mind (1986), using Roger Schank’s conceptual dependency
transframe construction, that a linguistic act like “John gave Mary a
kite” should be understood as a collection of transframes, one
residing in each realm: affective, physical, dominion, etc. The
implication of this is that every act can be projected into the
interpretive spaces of a variety of realms. It is then consistent with
Minsky and Lakoff and Johnson’s theories that each act is in part
interpreted by the reader as a conduiting of an affective package
from Agonist to Antagonist, consciously or unconsciously.
Ortony, Clore and Collins’s theory of emotions (1988) argues that
emotions result from cognitive appraisal, though to be sure, this
appraisal is not necessarily conscious or attentive. They postulate
that emotions only make sense when directed at an appropriate
focus, and they name three types: events, agents, and objects. This
supports CAD theory’s conceptualization of an affect as associated
with the self, with objects, or with other agents via the self’s Mental
World construction. This begs the question, “But what of those
emotions regarding events? There is no event construction in CAD
theory!”
This is a valid criticism, because the CAD is a simplification theory
meant to achieve a medium level of understanding about
characters, but not full story understanding. Certain types of events
are represented in CAD, for example, events between two
characters. “John hurt Mary” is an event, and is represented by
Mary’s receipt of negative affect-energy from John. However, in
the formulation of the CAD framework, it is not possible for John to
attribute his response to the event, unless the event is named.
Because if the event is named, the event is treated as an object, and
objects can be imbued with affect.
However, because CAD is not meant to handle the complexities of
events, it cannot relate any event to character’s goals, as more
complete story understanding demands. Schank and Abelson, for
example, interpret events based on whether or not they facilitate
goals such as Achievement, Entertainment, Instrumental, and Crisis
(1977). As a simplification framework, CAD does not keep track of
character goals, with one example: possession. CAD has a
dominion model which tracks who possesses what objects, and
how affectively valuable an object is. For example, if a story
reported that “Mary wanted a kite,” then the Kite Object is invested
with positive affect, because it is wanted. Then if the story reports,
“John bought a kite for Mary,” that event is deemed a transaction of
positive affect to Mary because the Kite Object has positive valence.
This can be viewed as the satisfaction of an Achievement goal, but
the reality of how CAD represents this is not as a goal at all.
Another way that events are handled in CAD is that if an event is
named, e.g. “the accident,” then suddenly it can be represented as
an Object and imbued with affect. With some adjustment of the
CAD framework, a character’s involvement with an event can be
represented as a character “owning” an Event Object. This may be
a natural enough way to handle events, especially traumatic ones,
where there is a sense that a character simply “can’t rid himself
from its memories;” he is unable to dispossess that event!
Another source of grounding for CAD theory is possible if we are
able to switch from a Western cultural view to an Eastern one.
Western culture does not “see” affect as anything concrete, but
Eastern culture does. In the Eastern concept of “Chi,” which means
breath and energy, objects and people are thought to possess
energy, and it is thought that psychic energies can be transferred
from agent to agent and object to agent. Such is the concept with
underlies ancient psychologies like “Feng-Shui” (wind-water) and
Jungian psychoanalysis, which views the human psyche as a
container of energy, and the environment as a field of objects
whose energies influence the psyche (cf. Symbols of Transformation,
1912) As Jung’s psychoanalysis has penetrated Western culture, we
see some examples of the belief that affective-energies are kept and
conveyed in interpersonal communication, e.g. “I’m getting some
bad vibes from her.” The belief that objects are imbued with
affective-energies and that possessing objects leads to the
inheritance of their energies has been brought closer to the
mainstream and to science by Csikszentmihalyi and RochbergHalton who developed a theory of possession around this in The
Meaning of Things (1981).
Having completed our current discussion of the theory underlying
Character Affect Dynamics, the next section discusses a system
implementation of the theory.
The ESCADA System
ESCADA is an Experimental System for Character Affect Dynamics
Analysis. It implements a good portion of the CAD theory
presented above.
The system is implemented as 1,400 lines of Python source code,
built on top of the MontyLingua NLP engine (Liu, 2004a) which is
itself 7,000 lines of Python source. A 500KB lexical knowledge base
was compiled for this project, from three sources: Roget’s
Thesaurus of English Words and Phrases (Roget, 1911), the
Affective Norms for English Words (ANEW) Project at the
University of Florida (Bradley & Lang, 1999), and Beth Levin’s
English Verb Classes and Alternations (1993). In addition, we handannotated the lexical classes present in these sources using PAD as
the annotation structure, and the annotation file is 8KB.
The system is input a narrative text; that text is segmented into
sentences, a deictic stack is implemented to track characters and
objects and resolve anaphora. Text is tokenized, part-of-speech
tagged, lemmatized, chunked, linked, and extracted into syntactic
frames of the form: Verb-Subject-Object-Object* (VSOO). The
semantic interpretation from these syntactic frames are produced
by a number of competing understanding demons, who
opportunistically recognize different types of affect-conveyance
and affect-imbuement situations and compete for their
interpretations to be accepted by a demon manager, who
alternately allows and inhibits different classes of demons, such as
cognitive-reflexive demons, conveyance demons, dominion
demons, and context demons. The architecture for this is inspired
by Oliver Selfridge’s Pandemonium feature detection system
(1958), although admittedly the demons in ESCADA do not
currently adapt themselves over experience, and coordination is
rudimentary in the demon manager; however we anticipate that as
the number of recognition demons increases, the system can scale
under this architecture.
Each demon is capable of recognizing a different syntacto-semantic
sentential case. The currently implemented demons are given
below, with names expanded for clarity, accompanied with an
explanation of their purpose and nuances, and the lambda
expression they produce which when applied to the world, updates
it.
AGONIST EXPERIENCING MOOD (e.g. John felt depressed)
Characterizes an agonist’s current affective self-concept.
(λworld: (world.find_agent(AGONIST).set_feel_about(SELF,PAD))
AGONIST PASSIVE-ACT ANTAGONIST (e.g. John sulked; Mary resented him,
John secretly cursed Mary)
Passive acts are those which form the agonist’s mental attitudes, but are not overtly
communicated to the antagonist. Iff the antagonist is Perceptive, she may intuit it.
(λworld:
(world.find_agent(AGONIST).set_feel_about(ANTAGONIST|SELF|OBJECT,PAD))
AGONIST ACTIVE-ACT ANTAGONIST (e.g. John cursed Mary; Mary shouted
obscenities at John)
Act or extrinsic acts are those which get overtly communicated as affective-energy
packages from the Agonist to Antagonist. The primary distinguishment of passive-versus
active acts are made by examining the theme (verb). Roget and Levin’s verb classes both
distinguish between mental, inwardly acts and social, extrinsic, public acts. In addition,
modifiers such as adverbials which convey stealth are used as cues.
(λworld: (world.find_agent(AGONIST).feel_at(ANTAGONIST,PAD)) ||
(λworld: (world.find_agent(READER).set_feel_about(AGONIST,PAD))
AGONIST MENTAL-ACT CLAUSE (e.g. John knew that …)
Mental acts are those which populate the Agonist’s Mental World. The setup of the
Mental World is the same as the Physical World, except that an Agonist may hold a
different version in its head. The CLAUSE can contain any utterance parseable by
another demon (e.g. John knew that Mary resented him).
(λworld: (world.find_agent(AGONIST).mental_world.(…))
AGONIST IMBUMENT (e.g. John had a horrible time sleeping; John life was one
rollercoaster ride after another)
This demon is an extremely general construction. It allows any sentence construction
because it does not consider syntax. The idea here is of affect as a contagion. Any
description whose affect can be calculated is then imbued unto the agonist. Of course, the
system is sensitive to negations, which is handled by a theme-inversion mechanism. The
demon also tries to avoid confusion by refusing to infect if more than one agent is
present.
(λworld: (world.find_agent(AGONIST).set_feel_about(SELF,PAD))
OBJECT IMBUMENT (e.g. Mary had a delicious ice cream; The accident was
particularly traumatic for John)
This demon imbues objects rather than agonists, is also general, but works with the same
confusion-avoiding caveat mechanisms. Note here that an “accident” event can be treated
as an object particularly because it is a named entity.
(λworld: (world.find_object(OBJECT).imbue_value(PAD))
OBJECT DOMINION TRANSFER (e.g. Mary had a delicious ice cream, and then
John swiped it from her.)
An object can be possessed by an Agonist, or dispossessed. Dispossession can be by an
Antagonist (e.g. John stole Mary’s heart), or can be existential (e.g. The ice cream
melted).
(λworld: (world.find_object(OBJECT).possessed|dispossessed_by(AGONIST))
STORY CONTEXT (e.g. There was a stench in the air)
This demon attends to the affect in every sentence, including those of no pertinence to
any character or object. This is used to imbue the story World itself with a sequence of
affects.
(λworld: (world.imbue_value(PAD))
The demon manager inhibits or accepts the interpretations of the
demons, and accepted interpretations are applied to the simulation
World, having the effect of continuously updating the world at
each story step. The simulation World is inhabited only by Agents
and Objects. Agents contain the following data and capabilities:
SIMULATION AGENT:
history of attitudes
current attitude
possessions
feelings about alters
mental world – same as physical world, except its agents have no mental world
agents
objects
feel-about(pad)
mental-feel-about(pad)
mental-feel-at(pad)
notified-of-incoming-affect(sender,pad)
possess(object) / dispossess(object)
pretty-print()
clone_agent() – this is used to produce an imagination scenario
Objects contain the following data and capabilities:
SIMULATION AGENT:
history of imbued values
owner
possessed-by(new-owner) / dispossessed-by(old-owner)
current-value()
To better illustrate the workings of the implementation, we walk
through a trace of a story. The following story is a paraphrase and
adaptation from the first-grade children’s story, “Up and Away,”
published by Houghton Mifflin (McKee et al., 1966). It was the
original story which Charniak used verbatim in his 1972 story
comprehension program (Charniak, 1972).
The paraphrase
eliminates character first-person utterances, which is out of the
scope of the implementation; significantly simplifies the rhetorical
structure of the text around the conventions of declarative form
(although aspectuals, moods, tenses, and dependent clauses are still
expressed); and allows the narrator to interject some subjective
commentary into the recounting of the details; the ending was also
changed.
First, here is the story in full:
Jack had some gorgeous paints given to him by his dad. By comparison,
Janet owned some lousy dirty pencils. Janet was jealous. She planned to
underhandedly swindle Jack. So she lied to him. Jack was led to believe
that the paints made his pictures look stupid. He was also made to think
that the pencils were more pleasing and could draw more splendid
pictures. Janet offered to take the paints from Jack. Jack received the
pencils from Janet. Jack was very happy. Jack thanked Janet. Janet was
regretful and ashamed.
Next, we will walk through the sentences of the story and present
the lambda interpretations derived from each (with slight float
rounding and pretty printing). We do not present a dump of the
world after each sentence (only once at the end) but we encourage
the reader to obtain the freely available source to this project and
run the given example story in the system for personal verification.
System output and commentaries are indented to here.
Recall that an affect is given in the form [p, a, d], where 0.5 is
the neutral fulcrum value.
Jack had some gorgeous paints given to him by his dad.
lambda world: world.find_object("paints")
.imbue_value([0.55, 0.53, 0.72])
lambda world: world.find_object("paints").possessed_by("Jack")
lambda world: world.imbue_mood([0.55, 0.53, 0.72])
Positive energy imbued onto paints, possessed by Jack. At
this point, if the system was asked about Jack’s feelings, the
heuristic rule of “Affective energy of possessions rub off on
possessor” could make some prediction that Jack is feeling
positive.
By comparison, Janet owned some lousy dirty pencils.
lambda world: world.find_agent("Janet")
.set_feel_about('self',[0.40, 0.48, 0.59])
lambda world: world.find_object("pencils")
.imbue_value([0.40, 0.48, 0.59])
lambda world: world.find_object("pencils").possessed_by("Janet")
lambda world: world.imbue_mood([0.40, 0.48, 0.59])
Similarly, pencils are imbued with negative energy, and the
owner is Janet. In this case, the contextual contagion demon
caused the first lambda interpretation which purports that
Janet feels negatively toward herself because she possesses
something “lousy” and “dirty.”
Janet was jealous.
lambda world: world.find_agent("Janet")
.set_feel_about('self',[0.3, 0.8, None])
lambda world: world.imbue_mood([0.3, 0.8, None])
This is a direct characterization of Janet’s affective state. The
value of the PAD’s dominance dimension is “None,”
because there is no information as to its value. When the
Janet simulation Agent code object, a history of her states is
being kept, allowing us to questions about changes or
developments in her state as the story progresses. Similarly,
the affective history of the world’s mood is being tracked, as
a record of the affective dynamics of the development of the
narrative.
She planned to underhandedly swindle Jack.
lambda world: world.find_agent("janet")
.set_feel_about('jack',[0.2, None, None])
lambda world: world.imbue_mood([0.3, None, 0.8])
The “swindle” theme has a definite affective projection: it is
the transfer of negative affective energy. However, because
theme is preceded by “plan to” and because
“underhandedly” modifies the theme, the interpretation is
that this ordinarily extrinsic act is in this case passive. Thus,
it only affects Janet’s mental affective attitude toward Jack.
Breaking at this point, some further story understanding
feats could be had. Even though ESCADA is overly
reductive in its interpretation of the situation, not knowing
about Janet’s new goal, if the system were asked to explain
why Janet was negative toward Jack, jealousy could be
reconstructed as a possible motivation, since Jack owns
positive affect and Janet owns negative affect. The system
could also predict depending on what is known currently
about Janet’s personality type, whether she is likely to
perpetrate an overt negative act against Jack. Actually,
depending on her later action, the system could learn about
Janet’s personality type since it is possible to create an
explanation case base associating various personality types
with the type of reaction they are likely to take in a given
affective situation.
For the most part, the current
implementation does not support such fine-grained
abductive explanation for character behaviors which could
betray the character personality types, but this would not be
a difficult augmentation.
So she lied to him. Jack was led to believe that the paints made his pictures
look stupid. He was also made to think that the pencils were more pleasing
and could draw more splendid pictures.
lambda world: world.imbue_mood([0.34, 0.62, 0.54])
lambda world: world.find_agent("Jack")
.set_feel_about('self',[0.47, 0.43, 0.51])
lambda world: world.find_agent("Jack")
.mental_world.find_object("paints")
.imbue_value([0.39, 0.43, 0.42])
lambda world: world.find_agent("Jack")
.mental_world.imbue_mood([0.39, 0.43, 0.42])
lambda world: world.imbue_mood([0.47, 0.43, 0.51])
lambda world: world.find_agent("Jack")
.set_feel_about('self',[0.59, 0.48, 0.69])
lambda world: world.find_agent("Jack")
.mental_world.find_object("pencils")
.imbue_value([0.65, 0.48, 0.66])
lambda world: world.find_agent("Jack")
.mental_world.imbue_mood([0.65, 0.48, 0.66])
lambda world: world.imbue_mood([0.59, 0.48, 0.69])
The theme of lying could not resolve to a substantive
consequence in the affective realm, so only global mood was
affected. In the second sentence, paints were devalued in
Jack’s mental world, but not in the real world. In the third
sentence, those shabby pencils appreciated in value in his
mental world, but not in the real one. Here, even in the
affective realm, we can see that plot tension is created. Jack’s
mental world is opposed to what exists in the real world.
Although the simplicity of the representation does not allow
us to associate Janet with causing this, it could be guessed
from Janet’s negative attitude toward Jack – that is sufficient
motive. With just a few heuristics (or that personality
archetypes explanation base), the system could also
presumably understand that Jack is gullible or naïve or
unperceptive for having such distorted perceptions. You’ll
also notice some lambda interpretation which affect Jack’s
mood as well -- this is simply the contextual contagion
keeping consistent and fluent Jack’s mental world and his
affective state.
Janet offered to take the paints from Jack. Jack received the pencils from
Janet.
lambda world: world.find_agent("Janet")
.set_feel_about('self',[0.54, 0.59, 0.67])
lambda world: world.find_object("paints")
.imbue_value([0.54, 0.59, 0.67])
lambda world: world.find_object("paints").possessed_by("Janet")
lambda world: world.imbue_mood([0.54, 0.59, 0.67])
lambda world: world.find_agent("Jack")
.set_feel_about('self',[0.59, 0.27, 0.64])
lambda world: world.find_object("pencils")
.imbue_value([0.59, 0.27, 0.64])
lambda world: world.find_object("pencils").possessed_by("Jack")
lambda world: world.imbue_mood([0.59, 0.27, 0.64])
The theme “offered to take” is given here to illustrate the
understanding limitations of the current approach. Of
course the semantics of an “offer” situation might require an
acceptance to be issued before the dominion transfer occurs,
but the current theory is perhaps overly naïve to these
details. In these two sentences, Jack and Janet swap the
paints and the pencils. There are perhaps some other
lambda interpretations which the reader disagrees with;
those are the result of contextual contagion and imbuement
demons. The fact that “offering” as a theme reflects
positively on the offerer causes Janet to be imbued with
positive mood for “offering” even though it was done under
false pretexts (she is tricking Jack). Overall though, we are
afforded some more story understanding lucidity. The fact
that Jack lost something good due to the misinformation of
his own mental world demonstrates his ignorance. Janet’s
willingness to proceed with the transaction despite her
lacking the excuse of holding false beliefs betrays her
deception.
Jack was very happy. Jack thanked Janet.
lambda world: world.find_agent("Jack")
.set_feel_about('self',[0.7, 0.69, 0.73])
lambda world: world.find_agent("jack")
.set_feel_about('self',[0.9, 0.69, 0.7])
lambda world: world.imbue_mood([0.7, 0.69, 0.73])
lambda world: world.find_agent("jack")
.feel_at('janet',[0.8, None, None])
lambda world: world.imbue_mood([0.55, None, 0.73])
Here, the “thank” theme is interpreted as an extrinsic event;
thus Janet explicitly receives an affective package from Jack.
As with the previous two sentences, Jack’s unperceptiveness
is revealed. Jack’s happiness can lead us to abductively
explain that Jack’s perceptions of the affective energies of the
paints and the pencils was based on his own mental world’s
assessment, not based on the reality the narrator has painted.
The continuing violation of the system heuristic that
“Affective energy of possessions (based on real-world
valuations) rub off on possessor” could constitute an
anomaly, as Ram used the concept in the AQUA questiondriven story understanding system to motivate further
understanding and explanation-making (Ram, 1994).
Janet was regretful and ashamed.
lambda world: world.find_agent("Janet")
.set_feel_about('self',[0.21, 0.59, 0.3])
lambda world: world.find_agent("janet")
.set_feel_about('self',[0.16, 0.59, 0.3])
lambda world: world.imbue_mood([0.21, 0.59, 0.3])
Janet’s negative self-concept at this story moment contains
so much information when correlated with the history of the
story. There are many “critical understanding agents”
which could be implemented to draw various sorts of
conclusions from the history of the character affect dynamics
of this story; for example, a critic should note that Janet went
from dominant to submissive, while Jack, despite Janet’s
negativity toward him, and despite his ignorance of her
deception, began happy and ended happy. If the critic could
look for these types of patterns, it could report its results in
the discourse of something like Lehnert’s plot units (Lehnert,
1984); this story might be something like an “Ineffective
Aggression” affective plot unit or something along those
lines. There are a great many potential simple and complex
plot units to be had, given that the CAD theory furnishes +, valences, “m” mental states, events (extrinsic and passive),
and goal achievement or failure might possibly be inferable
from the shifts in dominance posture (dominant 
submissive might signal goal failure). Other goals, such as
the attainment of positive affect, are implicit in the CAD
framework.
Finally, a pretty-print dump of the world’s state at the end of the
story is shown below:
####### THE REAL WORLD: ###############
WORLD TIME STEP: 0
OVERALL MOOD: [P0.46 A0.55 D0.63]
MOOD PROGRESSION: [P0.55 A0.53 D0.72]; [P0.4 A0.48 D0.59]; [P0.3 A0.8
D?]; [P0.3 A? D0.8]; [P0.34 A0.62 D0.54]; [P0.47 A0.43 D0.51]; [P0.59
A0.48 D0.69]; [P0.54 A0.59 D0.67]; [P0.59 A0.27 D0.64]; [P0.7 A0.69
D0.73]; [P0.55 A? D0.73]; [P0.21 A0.59 D0.3]
AGENTS:
CHARACTER: jack
FEELS: [P0.9 A0.69 D0.7]
FEELING HISTORY: [P0.47 A0.43 D0.51]; [P0.59 A0.48 D0.69]; [P0.59
A0.27 D0.64]; [P0.7 A0.69 D0.73]; [P0.9 A0.69 D0.7]
FEELS [P0.8 A? D?] ABOUT janet
HISTORY:
SENT janet [P0.8 A? D?]
OWNS:
pencils WORTH [P0.5 A0.38 D0.62]
POSSESSION HISTORY:
POSSESSED paints WORTH [P0.55 A0.53 D0.72],
DISPOSSESSED paints WORTH [P0.55 A0.56 D0.7],
POSSESSED pencils WORTH [P0.5 A0.38 D0.62]
oOoooOoo JACK's MENTAL WORLD: oOoooOoo
WORLD TIME STEP: 0
OVERALL MOOD: [P0.52 A0.45 D0.54]
MOOD PROGRESSION: [P0.39 A0.43 D0.42]; [P0.65 A0.48 D0.66]
MENTAL AGENTS:
IMAGINED OBJECTS:
IMAGINED OBJECT: paints
OWNED BY:
IMBUED VALUES: [P0.39 A0.43 D0.42]
IMAGINED OBJECT: pencils
OWNED BY:
IMBUED VALUES: [P0.65 A0.48 D0.66]
oOoooOooOOooOOoOoOOOoOOoOoOOooOoooOoo
CHARACTER: janet
FEELS: [P0.16 A0.59 D0.3]
FEELING HISTORY: [P0.4 A0.48 D0.59]; [P0.3 A0.8 D?]; [P0.54 A0.59
D0.67]; [P0.21 A0.59 D0.3]; [P0.16 A0.59 D0.3]
FEELS [P0.2 A? D?] ABOUT jack
HISTORY:
RECEIVED [P0.8 A? D?] FROM jack
OWNS:
paints WORTH [P0.55 A0.56 D0.7]
POSSESSION HISTORY:
POSSESSED pencils WORTH [P0.4 A0.48 D0.59],
POSSESSED paints WORTH [P0.55 A0.56 D0.7],
DISPOSSESSED pencils WORTH [P0.5 A0.38 D0.62]
oOoooOoo JANET's MENTAL WORLD: oOoooOoo
WORLD TIME STEP: 0
OVERALL MOOD: [P? A? D?]
MOOD PROGRESSION:
MENTAL AGENTS:
IMAGINED OBJECTS:
oOoooOooOOooOOoOoOOOoOOoOoOOooOoooOoo
OBJECTS:
OBJECT: paints
OWNED BY: janet
IMBUED VALUES: [P0.55 A0.53 D0.72]; [P0.54 A0.59 D0.67]
OBJECT: pencils
OWNED BY: jack
IMBUED VALUES: [P0.4 A0.48 D0.59]; [P0.59 A0.27 D0.64]
#######################################
Also, a most meager personality gister has thus far been
implemented, and it is extremely barebones compared to what is
possible had the personality archetypes case base (had it existed it
would be implemented like Schankian explanation patterns (1986);
patterns could be indexed by the personality archetypes which they
fit). However, for completeness, the gisted character analysis is
given below.
OVERALL STORY MOOD: [0.46166666666666667, 0.54800000000000004,
0.62909090909090915]
STAT REPORT FOR: jack
AVG EGO MOOD: [P0.65 A0.51 D0.65]
AVG ALTERS MOOD: [P0.8 A? D?]
AVG INCOMING: [P? A? D?]
AVG OUTGOING: [P0.8 A? D?]
MENTAL ACTIVITY INDEX: 2
INTRO/EXTROVERSION RATIO: 1.0
STAT REPORT FOR: janet
AVG EGO MOOD: [P0.32 A0.61 D0.47]
AVG ALTERS MOOD: [P0.2 A? D?]
AVG INCOMING: [P0.8 A? D?]
AVG OUTGOING: [P? A? D?]
MENTAL ACTIVITY INDEX: 0
INTRO/EXTROVERSION RATIO: 0
What can be understood here is that Jack is in general, a positive
person, who feels positively and acts positively toward others.
Janet in general feels negatively, and is negative toward others as
well. If deviation had been given in addition to mean, we might see
that Janet’s happiness and dominance fluctuates much more than
Jack, and is inherently the more tragic and more interesting
character (there are many directions to stretch these conclusions).
Had many of the further analyses in the aforementioned been
implemented, all of the CAD theory’s prescriptions for personality
dimensions might be analyzable; but for now, let the
aforementioned sentence-by-sentence analysis be suggestive of the
system and theory’s potentialities.
In the following section, we explore another more general mode of
operation and understanding, and discuss the results of an early
indicative evaluation of ESCADA’s performance at a real-world
understanding task.
Indicative Evaluation over Free Text
In the previous section’s example, we demonstrated that the CAD
theory and implementation is capable of a fair deal of mediumdifficulty and medium-specificity understanding tasks when the
expression of the story obeys certain pragmatic, stylistic, and
syntactic bounds. Of course, the presence of these bounds are akin
to the laws of a microworld. All deep story understanding systems
in the literature are ultimately only valid within their respective
microworld domains (folk stories are a favorite microworld).
CAD and its ESCADA system hope to distinguish itself from
previous approaches by also operating on and understanding
unrestricted natural language texts; albeit limiting understanding
to a comparatively coarse granularity and to the domain of affect
and its emergent correlate, personality. This is supported by
ESCADA’s flexible parsing and demon-recognition approach, and
by the presence of a rich and expansive affective lexicon specifically
assembled for this project and for use over general text. Of course,
this is a dual mode of operation where all the above-discussed
inferences which were dependent on the closed-world assumption
(such as action-reaction, since the reaction may not have been
detected in the free text) are unusable, but other more general
personality understanding techniques and techniques for
understanding inter-personal interactions, are still valid, especially
when the input text corpus is of a sufficient size so that the results
of this kind of opportunistic sensing of personality carries statistical
significance.
In order to gain an initial assessment for the quality and robustness
of ESCADA’s personality assessment, we performed an indicative
evaluation. The goal of this evaluation was to permit ESCADA to
analyze the personalities of some notorious characters from fiction
and real life, allowing the authors and the readers to gain a better
intuition for the true capabilities and limitations of the current
system implementation. The corpus used was summaries and
digests of great fictional works, taken from www.sparknotes.com;
this source was most appropriate because it is a non-concise thirdperson narrative paraphrase, a desirable format addressable by the
current system implementation. For each fictional work, we report
the statistics on personality means (we regret not having fully
implemented the translation of these crude statistics into the
detailed personality facets we presented in the theory section of
this paper) for each of the prominent characters, once for the corpus
of the first half of the story, and once again for the corpus of the
second half of the story. The reason for this segregation is that
either half becomes a baseline for evaluating the other; because
both samples come from the same story, corpus idiosyncrasy is
controlled for. In addition, the two-halves model allows for the
observation that characters often experience change through a text,
and calculating a personality mean over the whole of the story text
cannot possibly betray this change. Analyzing a story in halves is
also simpler than plotting the affective trajectory of each character
against an abscissa of narrative time. This alternative visualization
which demands some robustness improvements to first be
installed, will be tabled for now. For each result, we offer some
brief explanatory intuition.
For Romeo and Juliet, the whole corpus was 20,000 words, and the
average number of lambdas generated per sentence in the corpus (a
measure of recall) is 2.61. The results follow below in Table 1.
2nd Half
1st Half
Table 1. Two halves analysis of the Spark Notes chapter by chapter
recounting of Romeo and Juliet.
STAT REPORT FOR: Romeo
AVG EGO MOOD: [P0.37 A0.58 D0.42]
AVG ALTERS MOOD: [P0.2 A0.9 D0.9]
AVG INCOMING: [P0.65 A? D?]
AVG OUTGOING: [P0.24 A0.73 D0.86]
MENTAL ACTIVITY INDEX: 0
INTRO/EXTROVERSION RATIO: 0.25
STAT REPORT FOR: Romeo
AVG EGO MOOD: [P0.46 A0.63 D0.55]
AVG ALTERS MOOD: [P0.46 A? D?]
AVG INCOMING: [P0.3 A? D0.8]
AVG OUTGOING: [P0.36 A? D0.8]
MENTAL ACTIVITY INDEX: 1
INTRO/EXTROVERSION RATIO: 0.5
STAT REPORT FOR: Juliet
AVG EGO MOOD: [P0.56 A0.63 D0.56]
AVG ALTERS MOOD: [P0.5 A0.9 D0.9]
AVG INCOMING: [P0.3 A0.9 D0.65]
AVG OUTGOING: [P0.47 A0.9 D0.65]
MENTAL ACTIVITY INDEX: 4
INTRO/EXTROVERSION RATIO: 0.67
STAT REPORT FOR: Juliet
AVG EGO MOOD: [P0.59 A0.72 D0.59]
AVG ALTERS MOOD: [P0.2 A? D0.8]
AVG INCOMING: [P0.3 A? D?]
AVG OUTGOING: [P0.35 A? D0.8]
MENTAL ACTIVITY INDEX: 3
INTRO/EXTROVERSION RATIO: 0.67
A caveat on the above results: the avg ego mood is the most
statistically significant (best recall), followed by outgoing, then
alter’s mood, then incoming. The mental activity index is for now,
simply a rote count of the number of times a mental activity demon
has produced a lambda interpretation. As you can see, recall for
this is to be desired, but the results are nevertheless illustrative,
since cross-character comparison gives us good baseline.
What this analysis of Romeo and Juliet suggests is that Juliet is
consistently more positive and more affectively aroused than
Romeo, showing only slight intensification of her character at the
end. She thinks more than Romeo and so is perhaps more
introspective.
Romeo begins quite negative, and expresses
negativity toward others (recall that he was in a fight), but his
valence ameliorates in the second half. What we can read into
these character portrayals is perhaps that Juliet is a positive
steadfast character, while Romeo is troubled, but undergoes some
positive reform. The current evaluation does not reveal that the
ending is tragic, but the sequence of global mood progressions does
indeed show much affective activity and extreme emotions toward
the end of the story, and then the resolution is characterized by a
pacification of global story mood arousal.
For the next corpus, we chose The Unbearable Lightness of Being by
Milan Kundera. We used the 8,500 word SparkNotes digest of the
story. We would have preferred to use the actual next in this case
because it is a third-person narrative but an all-electronic copy
could not be obtained. Also, because the novel’s narrative is
completely out of sequence, we did not partition by halves. Table 2
holds the results of the system’s run.
Table 2. Character and pair-of-character analysis for the Spark
Notes section by section summary of The Unbearable Lightness of
Being by Milan Kundera .
STAT REPORT FOR: tomas
AVG EGO MOOD: [P0.48 A0.66 D0.52]
AVG ALTERS MOOD: [P0.67 A? D0.8]
AVG INCOMING: [P0.8 A? D?]
AVG OUTGOING: [P0.48 A? D0.8]
MENTAL ACTIVITY INDEX: 2
INTRO/EXTROVERSION RATIO: 0.6
STAT REPORT FOR: tereza
AVG EGO MOOD: [P0.56 A0.61 D0.56]
AVG ALTERS MOOD: [P0.8 A0.8 D?]
AVG INCOMING: [P0.32 A0.64 D0.78]
AVG OUTGOING: [P0.8 A0.8 D?]
MENTAL ACTIVITY INDEX: 5
INTRO/EXTROVERSION RATIO: 1.0
STAT REPORT FOR: sabina
AVG EGO MOOD: [P0.48 A0.58 D0.55]
AVG ALTERS MOOD: [P? A? D?]
AVG INCOMING: [P0.65 A0.8 D?]
AVG OUTGOING: [P? A? D?]
MENTAL ACTIVITY INDEX: 0
INTRO/EXTROVERSION RATIO: 0
STAT REPORT FOR: franz
AVG EGO MOOD: [P0.57 A0.61 D0.53]
AVG ALTERS MOOD: [P0.73 A0.7 D?]
AVG INCOMING: [P0.8 A? D?]
AVG OUTGOING: [P? A? D?]
MENTAL ACTIVITY INDEX: 0
INTRO/EXTROVERSION RATIO: 0
0
STAT REPORT FOR: tomas and tereza
AVG EGO MOOD: [P0.55 A0.74 D0.56]
AVG ALTERS MOOD: [P? A? D?]
AVG INCOMING: [P0.8 A? D?]
AVG OUTGOING: [P? A? D?]
MENTAL ACTIVITY INDEX: 0
INTRO/EXTROVERSION RATIO: 0
INTRO/EXTROVERSION RATIO:
STAT REPORT FOR: franz and sabina
AVG EGO MOOD: [P0.41 A0.62 D0.52]
AVG ALTERS MOOD: [P? A? D?]
AVG INCOMING: [P? A? D?]
AVG OUTGOING: [P? A? D?]
MENTAL ACTIVITY INDEX: 0
INTRO/EXTROVERSION RATIO: 0
STAT REPORT FOR: tomas and sabina
AVG EGO MOOD: [P0.61 A0.62 D0.64]
AVG ALTERS MOOD: [P? A? D?]
AVG INCOMING: [P? A? D?]
AVG OUTGOING: [P? A? D?]
MENTAL ACTIVITY INDEX: 0
INTRO/EXTROVERSION RATIO: 0
The above analysis of the novel’s four main characters and three
love pairings is generally correct.
The system correctly
characterized Franz as more positive than Sabina, and their love
pairing as negative; this is consistent with Kundera’s portrayal of
Franz as a optimistic romantic and one afflicted with kitsch; it is
also consistent with Sabina’s unbearable lightness, a condition of
melancholy, and the fact that Franz and Sabina’s love match was a
poor one. The system also correctly predicts that Tomas and
Sabina’s (his lover) relationship is more positive than his
relationship with his wife Tereza; and that Tereza is always
thinking (about her relationship with Tomas) and possesses inner
strength (since she is the recipient of negativity but nonetheless is
positive, acts positively, and feels positively toward others).
Of course this type of evaluation lacks rigorous control, but as an
indicative study, it was very informative and gives reason for
optimism over the ESCADA implementation and ultimately CAD
theory. There are myriad improvements to be had over the design
of an evaluation for this type of system, but these must be tabled
for future work.
Having discussed the CAD theory, ESCADA implementation,
walked through a detailed example, and presented some pilot
evaluation, the next section will conclude the paper with healthy
reflection on the potentialities of CAD theory, and the relationship
of CAD theory to the other story understanding systems presented
in the “Techniques for Narrative Comprehension with Imaginative
Intelligence” literature (cf. Ph.D. General Examinations Proposal
document).
Discussion
In this section, we stretch the story of the CAD theory in various
directions, exploring more fully its potentialities and limitations,
and making connections to other theories in the narrative
comprehension literature.
Representational issues. The representation that the ESCADA
implementation has chosen is to extract lambda interpretations out
of text, which then in turn updates a simulation world model. A
flavor of the frame problem in AI rears its ugly head here.
Although working AI systems prove that they are not incapacitated
by the frame problem, it does still introduce inconsistencies that
must usually be addressed with heuristic assumptions, and
ESCADA is no exception. The simulation world representation is
lucid (all aspects of the world are exposed and have clear
semantics), but the lambda updates to it are only fragmentary and
incomplete because the language of the narrative is incomplete; (for
example, knowledge such as common sense is suppressed). For
instance, consider that “Mary resents John.” The affective certainty
here is that Mary’s internal attitude toward John is negative;
however, does Mary’s resentment of John cause any change in the
state of John? Is John perceptive enough to have his states affected
by this act? Does Mary’s resentment of John cause a change in her
own mood and in her self-concept? It seems that so many factors
are at play that one may never be absolutely sure what is modified.
The causality of each act is tangled up in the contexts of each
character’s personality, relationship to each other, to the
environment, and so on.
The coping strategy to the frame problem employed in the STRIPS
system (Fikes and Nilsson, 1971), made the assumption that ‘all
that is not explicitly changed by an action remains unchanged.’
There is also an extended STRIPS assumption which makes a
address of consistency: ‘states not explicitly changed by an action
but which in the post-action sit in inconsistency are changed.’ This
solution, while theoretical successful within situation calculus, is
not practical to applications like ESCADA which face real-world
complexities, such as actions which are incomplete specifications,
and with numerous context-dependent entailments. ‘Assuming
that states not explicitly changed remain unchanged’ endangers the
system by making it possibly inconsistent. In this system,
inconsistency can initiate large cascades of misunderstanding. For
example, suppose that Mary resents John and then John kills Mary.
If John was perceptive, he may have known that Mary resented
him, and his killing her would be retaliation. If John did not know
about resentment toward him, then that unprovoked act could lead
one to characterize him as evil.
In ESCADA’s current implementation, the notion of contextual
contagion and imbuement is the coping strategy for the frame
problem. Because in our system, inconsistency is so sinister, we
hedge our bets. Whereas the STRIPS assumption would prescribe
that Mary’s resentment toward John only dictates her attitude
toward him and not toward herself, we assume the opposite – that
each action nudges all other plausibly related states in the same
vector direction. The implemented contextual contagion demons
cause every act to also affect the AGONIST in the act; so Mary, in
the act of resenting, is made negative herself. We should think of
contextual contagion as a form of state relaxation across the
simulation.
Not present in the current implementation, we would also propose
to implement in future work, a multiple hypothesis model of
action.
That is to say, unknown variables (e.g. character’s
personality) which may affect the entailments of an action are
instantiated as multiple hypotheses. Then, each hypothesis is
reinforced or de-reinforced depending on whether or not later
actions in the story support or contradict each hypothesis. By the
conclusion of the story, certain hypotheses will prevail over others.
In this manner, the unspecified parameters of character personality
may be learned by reinforcing various hypotheses about their
values (cf. hypothesis reinforcement in Ram’s AQUA (1994)).
Increasing understanding breadth via explanation. In the detailed
story example walkthrough discussed in the implementation
section of this paper, we suggested that the ESCADA system could
be fitted with more explanatory power, allowing it to complete a
broader range of narrative comprehension tasks. Even if not
externally prompted for explanation, the system should be able to
detect an interesting story feature or anomaly, and try to explain it.
The approach taken by Ashwin Ram’s AQUA system is most
relevant (1994) here. In AQUA, explanation is tackled with a casebased memory. The memory contains cases which Ram terms
“explanation patterns, ” which are either rote memorized
explanations (called “Explanatory Cases”); or “Abstract
Explanation Schemas,” fossilized chains of abduction from a
consequent (modeled as directed acyclic-graph i.e. decision tree).
Each case contains some prerequisites for the execution of that
case’s script, and in this sense, each case can be thought of as a
recognition demon (since a case is really just a procedure with a
condition for execution).
Here is an example of an anomaly whose explanation would bear
fruit of understanding: “John’s ice cream is delicious. John dislikes
the ice cream.” This violates the expectation established by our
assumption that the possession of positively charged objects
imbues the possessor with positive feeling. A positively charged
object should be desirable. The detection of this anomaly could be
accomplished by programming a demon to check for the truth of
axiomatic invariants over the simulation world state. In this case,
possible explanations could be that the ice cream in John’s mental
world is negative; or John is a masochist. Just as is done in AQUA,
these hypothesis could persist through the tenure of the story,
being occasionally reinforced or disavowed.
Of course, in order to address the aesthetics of narratives, especially
fictional texts, we must also accept as valid explanations some of
the following: “irony,” “tragedy,” “tension.” In and of themselves,
these are aesthetic aspects of story which are perfectly valid
explanations; some inconsistencies are actually intended!
Discovering Plot Units. Lehnert’s Plot Units (1982) sought to
define abstract prototypical forms for story structure, an ontology
of story pieces; to understand the plot unit structuring of a
narrative is an important gestalt reasoning task that can then
contextualize and inform more fine-grained understandings of
story. The elementals out of which Plot Units are composed
happen to resemble understandings mineable out of the CAD
framework. For example, Lehnert distinguishes between mental
acts, positive events, and negative events – all directly readable
from the ESCADA simulation world. The challenge in detecting
plot units is actually one of attending to certain key events, and not
attending to insignificant events. Because of the verbosity of a text,
a plot unit could unfold over a chapter or even on the scale of the
whole book; however, if the system is unable to disregard
insignificant events scattered between significant events, that is to
say, if it lacks a good heuristic of relevancy and saliency, then it
will certainly miss the important points in the story which actually
justify a plot unit. Because of the CAD theory’s cognitive linguistic
approach, a possible problem for plot unit detection is that every
transitive sentence can be viewed as an event, even though it is an
insignificant event, or simply a non-event syntactically construable
as an event; this leads to more noise which must be filtered
through. However, it may be possible to apply the chunking of
smaller events into larger events in order to detect plot units which
live at the scale of the whole story. In the Romeo and Juliet example
in the previous section, a story was segmented into two halves and
analyzed as chunks; from the trajectories of affect across halves and
between characters, we may be able to recognize some interesting
plot units (though certainly lacking the specificity of Lehnert’s
repertoire), like inspiration, or, tragedy, or betrayal.
Metaphorical representation.
The knowledge representation
employed by CAD theory is fundamentally metaphorical rather
than literal, and in that sense, CAD is a first-of-its-kind narrative
understanding theory. Not metaphorical is the literary or poetic
sense, but rather, the projection of arbitrary thematic role themes
(expressed as verbs) onto the affective plane is a metaphorical
mapping. The use of the syntacto-semantic vehicle of verb
transitivity as a CONDUIT enables the metaphorical interpretation
of affective transfer; the impact of the CONDUIT language
metaphor on thought is hypothesized in (Lakoff & Johnson, 1980).
Its metaphorical approach distinguishes CAD from story
understanding systems which deal with literal acts and engage
purely in rational reasoning.
In the discourse that Gelernter presents in The Muse in the Machine
(1994), rational reasoning and interpretation, which has been the
mainstay paradigm of AI story understanding systems, is highfocus thought, whereas metaphorical or analogical reasoning is
medium-focus thought. Medium-focus thought is an important
component of the totality of human thought, but has historically
been under-explored in AI. Medium-focus thought is quite closer
to intuitive thinking because of the relative importance of affect as a
bridge between memories and ideas; affect itself is completely
personal, living within the person-subject, and is often difficult to
articulate; in contrast, rationality and language are not inherent to
the person-subject because it is in part, defined socially and
culturally. Other than Gelernter, many in the Philosophy literature
have long held that intuition, not rationality, is the person-subject’s
chief method of real understanding (e.g., cf. An Introduction to
Metaphysics (Bergson, 1903)).
We would like to think of CAD theory’s metaphorical blur-the-eyes
approach to story understanding as at the very least, a good
complement to any rational reasoning theory. Additionally, one of
the great affordances of CAD and other metaphorical analyzers is
that their representations are simpler than with frame-semantics,
scripts, goal-schemas, et alia, and thus can use more inexact
resources like lexical libraries and connotation engines rather than
requiring the careful handcrafting of entirely detailed and precise
semantic frames.
Works Cited
Henri Bergson: 1903, “An Introduction to Metaphysics.” Kessinger
Publishing 1998.
G. C. Borchardt: 1990, “Transition space”, AI Memo 1238, Artificial
Intelligence Laboratory, Massachusetts Institute Of Technology,
Cambridge, MA
M.M. Bradley, & P.J. Lang: 1999, Affective norms for English words
(ANEW): Instruction manual and affective ratings. Technical Report C1, The Center for Research in Psychophysiology, University of
Florida.
Isabel Briggs-Myers and Mary H. McCaulley: 1985, Manual: A Guide
to the Development and Use of the Myers Briggs Type Indicator, by.
Consulting Psychologists Press.
Eugene Charniak: 1972, Toward a model of children's story
comprehension (Technical Report AITR-266). Artificial Intelligence
Laboratory, Massachusetts Institute of Technology, Cambridge,
MA.
Mihaly Csikszentmihalyi, Eugene Rochberg-Halton: 1981, The
Meaning of Things: Domestic Symbols and the Self, Cambridge
University Press, UK.
Daniel C. Dennett: 1987, The Intentional Stance Cambridge MA, MIT
Press
Paul Ekman: 1993, Facial expression of emotion. American
Psychologist, 48, 384-392.
R. E. Fikes & N.J. Nilsson: 1971, STRIPS: A new approach to the
application of theorem proving to problem solving, Artificial
Intelligence 2, 189–208.
David Gelernter: 1994, The Muse in the Machine: Computerizing the
Poetry of Human Thought. Free Press.
Karen Horney: 1945, Our Inner Conflicts: The Constructive Theory
of Neurosis. New York: WW Norton.
Ray Jackendoff: 1983, “Semantics of Spatial Expressions,” Chapter 9
in Semantics and Cognition. Cambridge, MA: MIT Press.
C.G. Jung, and H.G. Baynes (translator): 1971, Psychological Types,
C.G. Jung, H.G. Baynes (translator). Bollingen Series, Princeton U.P.
Carl Gustav Jung: 1977, Symbols of Transformation. Princeton NJ:
Princeton UP (Original work published 1912)
George Lakoff, Mark Johnson: 1980, Metaphors We Live by.
University of Chicago Press.
W.G. Lehnert: 1982, Plot units: A narrative summarization strategy.
In W. G. Lehnert & M. H. Ringle (Eds.). Strategies for natural
language processing. Hillsdale, NJ: Lawrence Erlbaum Associates.
Beth Levin: 1993, English Verb Classes And Alternations: A
Preliminary Investigation. The University of Chicago Press.
P. McKee et alii: 1966, Up and Away. Boston: Houghton Mifflin Co.
Albert Mehrabian: 1995, for a comprehensive system of measures of
emotional states: The PAD Model. Available from Albert Mehrabian,
1130 Alta Mesa Road, Monterey, CA, USA 93940.
Marvin Minsky: 1986, The Society of Mind. New York: Simon and
Schuster.
A. Ortony, G.L. Clore, A. Collins: 1988, The cognitive structure of
emotions, New York: Cambridge University Press.
Carol S. Pearson: 1998, The Hero Within: Six Archetypes We Live By,
New York, NY: HarperSanFrancisco
Rosalind W. Picard: 1997, Affective Computing, The MIT Press.
Ashwin Ram: 1994, “AQUA: Questions that drive the explanation
process.” In Roger C. Schank, Alex Kass, & Christopher K. Riesbeck
(Eds.), Inside case-based explanation (pp. 207-261). Hillsdale, NJ:
Erlbaum.
Peter Mark Roget: 1911, Roget’s Thesaurus of English Words and
Phrases. Retrieved from http://www.gutenberg.net/etext/10681.
R.C. Schank & R.P. Abelson: 1977, Scripts, plans, goals, and
understanding. Hillsdale, NJ: Lawrence Erlbaum.
R.C. Schank: 1986, Explanation Patterns: Understanding Mechanically
and Creatively. Hillsdale, NJ: Erlbaum Assoc.
O. G. Selfridge: 1958, Pandemonium: A paradigm for learning. In
Mechanisation of Thought Processes: Proceedings of a Symposium
Held at the National Physical Laboratory, London: HMSO, November.
Leonard Talmy: 1988, Force Dynamics in Language and Cognition.
Cognitive Science 12: 49-100.