Taqi Jaffri
Final Project,
CS224N - spring 2003
The Emotion Bot (Emotibot) is a conversation system that attempts to model emotions in simple situations.
There are four major aims of this system:

Chatter-Bot functionality implemented with customizable personality files, grammar, and lexicon

Information Retrieval using a Query / Response model

Enhancements using Synonym Trees

Emotions modeled with 'Emotion Matrices'
This document contains introductions to features that implement each of the four aims listed above, as well as a description of my approach to modeling emotions.
This document contains some transcripts and examples of sessions between the Bot and the User. I have found that it is not always easy to tell in these transcripts what text is input typed by the user and what text is a response by the Bot, since there are no ‘nickname’ prefixes like Instant Messaging programs. To help in this, I have adopted the following formatting standard:

All examples and sessions are in Courier Font

User input is in bold Courier Font

Comments for explanation purposes (that do not appear if you actually go through the actions depicted) are included ‘inline’ and are preceded by ##

A Makefile is included to compile the program. In order to run the Bot, you require all compiled Java classes that this Makefile creates, as well as grammar and lexicon files (same format as PP-1). I am providing one each of these that I used for testing, and which can be used for each situation/example in this write-up: mygrammar mylexicon
A set of ‘personality files’, which define the personality of the Bot, is also required to run the program correctly. These files must be organized in a ‘personality directory’, and consist of one each of the following files: ambiguousMsgSet emotions greetingSet noParseMsgSet
The *Set files include message 'seeds' for the Chatter-Bot functionality. These seeds can be 'enhanced', and
MUST be sentences that are properly parsed (not necessarily uniquely) by the provided grammar (details below). The fourth file, 'emotions', contains a 10-by-10 matrix of transition weights between emotion states
(details below).
I have provided three of these 'personality directories', and they contain all the files that are needed for testing purposes: personality_1 personality_2 personality_3
(simple 'linear' personality)
(depressed personality)
(crazy unpredictable personality)
The format to run the Bot is: java Bot <personalitydir> <grammarfile> <lexiconfile>
The command to run the Bot, using the provided grammar and lexicon, and the simple linear personality, would thus be: java Bot personality_1 mygrammar mylexicon
And similarly for the other two personalities: java Bot personality_2 mygrammar mylexicon java Bot personality_3 mygrammar mylexicon

2.
1.
The interface is pretty basic, but functional. Once the program loads up, you can type in either of two kinds of inputs:
COMMANDS
To type in a command, the first character must be '>'. To get a listing of commands at any time, you can type '>help'. The following listing comes up:
>quit Exits the program
>help Brings up this message
>trees-off Turns off tree printing
>trees-on Turns on tree printing
>en-msgs-off Turns off enhanced mesgs
>en-msgs-on <depth> Turns on Enhanced msgs, to specified search depth
They are all pretty self-explanatory, except the last two. The ‘enhanced messages’ are explained in more detail below, but it is important to note that this command only fully executes once during the lifetime of the Bot. The first time that it runs, it takes a long time to finish depending on the search depth (only search depth of 2 is recommended, otherwise it will take too long). This is because the
Bot is searching in the WordNet database to create many more messages from the few provided message ‘Seeds’ in the personality directory. To generate enhanced messages with search depth 2, type:
>en-msgs-on 2
The Bot will use create enhanced messages (please wait!), and these will be saved for the rest of the program session. You can still toggle the enhanced messages on and off, but after the first time the
Bot ignores the search depth parameter and just reinstates the old enhanced messages. This is for performance reasons since generating enhanced messages is time-consuming.
SENTENCES
All other input typed in that is not a command is an input 'sentence'. The Bot tries to parse it using the provided grammar and lexicon, and if the input is properly parsed tries to act upon the instruction or query. A tree is printed by default for each parsed sentence, but this feature can be turned off using the >trees-off command.
If a sentence cannot be parsed, then the Bot asks if you want it to 'think about it'. What this means is that the Bot has some additional features that allow it to map your unparsable input sentence to something it can understand. This is implemented using Synonym trees (details below).

A couple of different kinds of sentences are supported:
Queries and Enumerations, e.g.
How are you feeling ?
How many icecreams do you have ?
How many icecreams do i have ?
*** NOTE that the '?' needs to be separated from the last word with a space
Actions, e.g.
Give me an icecream
Take 3 icecreams from me
*** NOTE that numbers only from 1-10 are in the lexicon
The pronouns 'you', 'I', 'me', etc are supported.
Generally speaking, the provided lexicon is limited to a domain that allows the exchange of icecreams between the Bot and the user. This is adequate to experiment with emotions later on, but at first sight the lexicon might seem limited. However, the Bot has features for unparsable sentences, e.g. the Synonym trees, that somewhat alleviate this restrictiveness.
The bot loads with no icecreams of its own. The user has 10 icecreams to begin with.
The 'personality files’ allow us to provide three kinds of message sets, for different situations. These are the
'greeting messages', the 'ambiguous message error' messages, and the 'no parse error' messages in the files greetingSet, ambiguousMsgSet and noParseMsgSet respectively. You can look inside these text files for the format.
For the two Seeds:
I don't understand this sentence
I can't parse this sentence
We get (among others) the following enhanced messages at search depth 2. These have been generated by substituting synonyms for all NOUNS and VERBS in the sentence.
I don't understand this constituent
I don't dig this sentence
I can't parse this interrogative
I don't empathise this sentence
I don't comprehend this sentence
I can't parse this string
I don't fancy this sentence

As discussed above, the command to do this is:
>en-msgs-on 2
If enhanced messages are on, then an enhanced message is picked at random whenever it is needed. By default, enhanced messages are off.
An important thing to note is that any Seed must be parsable by the provided grammar for it to be successfully loaded.
A bit of the icecream game has been described in the introduction above under the SENTENCES heading.
The general idea is that you start off with 10 icecreams, and the Bot has none. Then you can exchange icecreams using different sentence structures. The program does some computational semantics by looking at the top-down parse tree generated for the sentence, and creates a representation of the sentence that you typed.
This representation is stored in an instance of the Instruction class, which is created by the static method buildInstance(Node tree) of the Instruction class. To actually perform the requested action, the program calls the .execute() method of this class.
Each Instruction also has a string representation, which is printed out each time an Instruction is built up.
For example, the following Instruction is a ‘take’ action from the subject entity ‘user’ (which contains 10 entities of type ‘icecream’, and the relevant object is ‘icecream’ with quantity 1. take an icecream from me a>take s>user: [icecream: , icecream: , icecream: , icecream: , icecream: , icecream: , icecream: , icecream: , icecream: , icecream: ] o>icecream: (1)
After an instruction is created, it is executed. In this case, that involves guessing that ‘me’ refers to the user and the ‘target’ of the action must be the Bot. The resulting states of both the user and the Bot are then printed out as follows: user: [icecream: , icecream: , icecream: , icecream: , icecream: , icecream: , icecream: , icecream: , icecream: ] emotibot: [icecream: ]
This is best illustrated by the example conversation transcript below (Tree printing is turned off after the first tree is printed, to save space).

## Sample Session
****************************************
Emotion Bot. Type >help for assistance
****************************************
Hello: my name is emotibot
>en-msgs-on 2
Please wait while enhanced messages are generated...
Enhanced personality has been created
*** Enhanced messages are now enabled how many icecreams do i have ?
[S
[Ques
[Enum
[Enum1 'how']
[Enum2 'many']]
[NP
[N 'icecreams']]
[VP
[V 'do']
[NP
[N 'i']]
[JJ 'have']]]
[Qm '?']]
a>enumerate
s>user: [icecream: , icecream: , icecream: , icecream: , icecream: , icecream: , icecream: , icecream: , icecream: , icecream: ]
o>icecream: user has 10 icecreams
## NOTE ABOVE: that the pronoun 'i' has been dereferenced successfully
>trees-off
*** Tree printing is now disabled how many icecreams do you have ?
a>enumerate
s>emotibot:
o>icecream: emotibot has 0 icecreams
## NOTE ABOVE: enumeration on the emotibot's icecreams. 'you' has been dereferenced successfully. take an icecream from me
a>take
s>user: [icecream: , icecream: , icecream: , icecream: , icecream: , icecream: , icecream: , icecream: , icecream: , icecream: ]
o>icecream: (1)

user: [icecream: , icecream: , icecream: , icecream: , icecream: , icecream: , icecream: , icecream: , icecream: ] emotibot: [icecream: ]
## NOTE ABOVE: the article 'an' is correctly interpreted as meaning 1 icecream take 4 icecreams from me
a>take
s>user: [icecream: , icecream: , icecream: , icecream: , icecream: , icecream: , icecream: , icecream: , icecream: ]
o>icecream: (4) user: [icecream: , icecream: , icecream: , icecream: , icecream: ] emotibot: [icecream: , icecream: , icecream: , icecream: , icecream: ]
## NOTE ABOVE: multiple icecreams exchanged how are you feeling ?
a>emotion-query
s>emotibot: [icecream: , icecream: , icecream: , icecream: , icecream: ] emotibot's Happy Index is at 10.0
## NOTE ABOVE: The Happy Index (0-10) is a measure of how happy the bot is.
Details below in the emotions section give me 2 icecreams
a>give
s>user: [icecream: , icecream: , icecream: , icecream: , icecream: ]
o>icecream: (2) emotibot: [icecream: , icecream: , icecream: ] user: [icecream: , icecream: , icecream: , icecream: , icecream: , icecream: , icecream: ]
## NOTE ABOVE: that you can perform the same actions in some different ways, and the system remembers the state of each user
Here you might feel restricted by the lack of other action verbs (only ones supported are give and take), as well as only 'icecreams' being exchanged. This is just to keep the concept simple, because things are made more interesting by two additional features (explained below), namely Synonym Tree guesses and Emotion
Matrices

The idea here is to expand the limited functionality provided by the tiny lexicon, and allow the user to be freer with input. When the program sees a sentence that it cannot parse, the idea here is to see whether you can replace words in the sentence with their synonyms, and maybe get something you can parse that way.
The WordNet database that I have used for getting Synonyms is huge, and therefore performance is pretty bad when extracting synonyms. The Synonyms.java
class is actually pretty self contained, and has been modified from what I submitted with PP2 to support 'deep searches'. This means that you can find 'the synonyms of the synonyms of the synonyms... ' of a word, up to a specified depth, in the form of a Synonym
Tree.
This search is obviously slow, and you have to specify what kind of word you are searching for, to ensure that 'noun' and 'verb' etc versions of the word do not get mixed up. An example use of the Synonyms class directly is provided below: elaine24:~/spring03/cs224n/final> java Synonyms give verb 1
[pass, generate, pay, leave, establish, feed, grant, throw, sacrifice, commit, dedicate, collapse, open,break, hand, make, hold, render, contribute, gift, reach, devote, impart, present, grub, have, return, afford, founder, apply, yield]
Ofcourse, this is for illustration only and you don't need to manually create Synonyms to operate the Bot.
So now we can say grandiose and fun things like... "gift me an icecream". The program will obviously not be able to parse this off the bat, since 'gift' is not a part of its limited lexicon. But it will prompt you to allow it to search with synonyms, and if you agree then it will try to match 'gift' with a verb in its lexicon. In this case, it is successful, since the depth-1 synonyms that the Synonyms class finds for the verb gift include
‘give’, which is in the lexicon.
## INITIAL STATE user: [icecream: , icecream: , icecream: , icecream: , icecream: , icecream: , icecream: , icecream: , icecream: ] emotibot: [icecream: ]
## gift me an icecream i can't parse this sentence
Do you want me to think about this ? (yes/no) yes
Please wait while I try to guess what you meant...
Is this what you meant ? (yes/no)
[S
[Action
[VAC 'give']
[NP
[N 'me']]
[PP

[DT 'an']
[NP
[N 'icecream']]]]] yes
a>give
s>user: [icecream: , icecream: , icecream: , icecream: , icecream: , icecream: , icecream: , icecream: , icecream: ]
o>icecream: (1)
## FINAL STATE emotibot: user: [icecream: , icecream: , icecream: , icecream: , icecream: , icecream: , icecream: , icecream: , icecream: , icecream: ]
Other fun possibilities that also work are: hand me an icecream feed me an icecream devote me an icecream acquire an icecream from me accept an icecream from me consume an icecream from me
None of these verbs are in the lexicon! For performance reasons, though, the search is limited to verbs. The adjectives, etc, might also be interesting candidates for this kind of guessing.
As you might have noticed above, the Bot maintains a state of emotion, and you can always ask it ‘how are you feeling ?’
Emotions are measured with a 'happy index' which is a range from 0 to 10, with larger numbers meaning that the Bot is happier. Thus we are only looking at a happy-sad emotion, but the possibilities are continuous, i.e. any real value between 0 and 10. This allows for some interesting behavior.
For the Bot, reaction to loss or gain of icecreams (with more or less happiness) is a personality trait, and is defined in the ‘emotions’ file required in the 'personality directory'. All that this file contains is a 10 by 10 matrix of numbers, each of which represents how many 'points' the index moves when the Bot moves from one state to the other. This is a bit like the ‘crazy softdrink machine’ example in the textbook, and by playing around with the matrix we can model different kinds of ‘personalities’.
A 'linear' personality is the simplest kind, one that reacts with 1.0 happy-points gained or dropped if you give or take an icecream respectively. Multiple icecreams result in multiple happy points dropped or gained, but the changes are always linear and predictable. The 'emotions' file in the 'personality directory' contains an Emotions Matrix with a ‘flat metric’, i.e. all transitions have 1.0 points. Such a personality is provided in the directory personality_1.

## -----------------------------------------------
## Emotions transition matrix for a linear personality
## -----------------------------------------------
1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
Other personalities can be modeled by changing the values of these numbers, with the only restriction that the numbers must be positive. If a state transition not possible, then its value is 0, and this serves as an effective limit on the happy-index. For example, if we set the entire column 7 to 0, this means that each time the Bot has a happy-index in the range 6-7, it is not possible for it to go above that. This is because the next transition has zero weight. This is exactly how the ‘depressed personality’ (in directory personality_2) is modeled:
## -----------------------------------------------
## Emotions transition matrix for a depressed personality
##
## Happy index cannot go above 6
## and below 3, emotions are 'volatile', i.e.
## each transition carries more weight
## -----------------------------------------------
2.00 2.00 2.00 2.00 1.00 1.00 1.00 0.00 0.00 0.00
2.00 2.00 2.00 2.00 1.00 1.00 1.00 0.00 0.00 0.00
2.00 2.00 2.00 2.00 1.00 1.00 1.00 0.00 0.00 0.00
2.00 2.00 2.00 2.00 1.00 1.00 1.00 0.00 0.00 0.00
2.00 2.00 2.00 2.00 1.00 1.00 1.00 0.00 0.00 0.00
2.00 2.00 2.00 2.00 1.00 1.00 1.00 0.00 0.00 0.00
2.00 2.00 2.00 2.00 1.00 1.00 1.00 0.00 0.00 0.00
2.00 2.00 2.00 2.00 1.00 1.00 1.00 0.00 0.00 0.00
2.00 2.00 2.00 2.00 1.00 1.00 1.00 0.00 0.00 0.00
2.00 2.00 2.00 2.00 1.00 1.00 1.00 0.00 0.00 0.00
Finally, an unpredictable ‘random’ personality is provided in directory personality_3:
## -----------------------------------------------
## Emotions transition matrix for an unpredictable personality
##
## state transitions were generated with the Excel function rand()*3
## -----------------------------------------------
2.60 1.46 0.98 1.36 1.69 0.72 1.65 1.53 0.24 0.79
1.47 0.32 1.03 0.53 0.85 1.14 0.92 0.14 0.21 0.04
0.79 0.50 0.89 0.42 0.88 1.90 1.25 0.06 0.76 0.56
1.46 1.81 0.43 0.47 0.86 1.89 0.05 0.85 0.27 0.41
0.27 0.11 1.40 1.83 0.74 0.75 0.17 1.03 0.70 1.21
1.00 0.78 0.99 1.45 0.62 1.27 0.98 1.75 1.37 1.88
0.69 1.19 1.08 0.92 1.10 0.42 0.20 0.30 0.13 0.11
1.45 0.72 0.53 1.52 1.21 1.33 0.56 1.70 0.05 1.47
1.08 1.08 1.62 0.01 0.89 1.37 1.97 1.39 0.24 1.52
0.97 1.28 0.12 1.02 0.04 1.38 1.90 0.53 1.49 0.17

The first three aims listed on the first page of this write-up work well, but there was no way to quantify the performance of the model that I used to model emotions. Mainly, this was because I had no datasets available, other than the few times that I asked friends to ‘simulate’ the emotions Bot in a sequence of transactions of icecreams. They were then asked to give a ‘happy index’ value based on whatever emotion they might think a Bot with their personality would feel:
#User #Bot Sim. Index Linear Index %diff over range
10 0 5 5 0%
9
5
9
8
3
10
7
6
1
3
9
4
5
2
1
6
5
8
3
4
9
7
1
5
1
2
7
0
1
6.5
8
7
5
6
6.5
7
3
9
1
2
7
0
6
10
9
10
5
6
10
8
2
6
2
3
8
1
-50%
-35%
-10%
-30%
0%
0%
-35%
-10%
10%
30%
-10%
-10%
-10%
-10%
Average % difference -11%
The average percentage difference above can be used as a measure of how successfully the personality models that of the human simulating the Bot. In the short sample above, we can see that the ‘linear’ personality is actually quite successful at modeling the personality of the human I asked to participate in the test simulation. The results are not statistically significant, though, since there is not enough data to evaluate the user, but such methods might be successful in both training and evaluation. An interesting application would be to ‘train’ the emotions matrix if a large sample of human simulation data was available.
http://www.ellaz.com/WordNet.aspx http://www.botspot.com/search/s-chat.htm http://www.simonlaven.com