Natural Language Processing
Speech acts
- definition, what is implied
- Conducting a conversation – conditional planning
- Understanding an utterance
Natural and formal languages
- Phrase structure
7 component steps of communication
Intention
Perception
Generation
Analysis
Synthesis
Disambiguation
Incorporation
- Discussion on how analysis may influence perception
Syntax, Semantics, Pragmatics
Situated language – the meaning of a message depends on both the words
and the situation in which the words are uttered.
Types of communicating agents
1 Agents that share a common internal communication language
– they can communicate without any external language at all
Problems:
a. easy to agree on static symbols but difficult to agree on dynamic
symbols
b. need a renaming policy
c. need a way to relate symbols introduced by different agents
d. what to communicate and what new things the other agents have
found out
2 Agents that make no assumption about each other’s internal language
– they share a common communication language
Communication language (external)
Internal representation
language
External communication language: generation, analysis
1
We still have problem (d) of the first type of agents
2.1 Defining the language (subset of English)
Lexicon – list of allowable vocabulary words
- grouped in categories (parts of speech)
- categories: open classes, closed classes
Lexicon
Noun  stench | breeze | wumpus ..
Verb  is | see | smell ..
Adjective  right | left | smelly …
Adverb  here | there | ahead …
Pronoun  me | you | I | it
RelPronoun  that | who
Name  John | Mary
Article  the | a | an
Preposition  to | in | on
Conjunction  and | or | but
Grammar
S  NP VP | S Conjunction S
NP  Pronoun | Noun | Article Noun | NP PP | NP RelClause
VP  Verb | VP NP | VP Adjective | VP PP | VP Adverb
PP  Preposition NP
RelClause  RelPronoun VP
2.2 Syntactic analysis
Top down parsing
Bottom-up parsing
 Definite Clause Grammars
A grammar written with logical sentences is called a logical grammar.
Definite Clause Grammars
Definite Horn Clauses (remember def from FOPL)
2
BNF
S  NP VP
NP  stench
NP  wumpus
VP  smells
VP  kills
FOPL
NP(s1)  VP(s2)  S(append(s1,s2))
(s = “stench”  s = “wumpus”) 
Noun(s)
(v = “smells”  v = “kills”) 
Verb(v)
PROLOG
sentence([S1, S2]) :np(S1), vp(S2).
np(stench).
np(wumpus).
vp(smells).
vp(kills).
?- sentence([wumpus,
smells]).
?-sentence([S1, S2]).
 Augmenting the grammar
 Add one extra argument for the semantics – see also semantic analysis
further on
DCG
S(sem)  NP(sem1) VP(sem2)
{compose(sem1, sem2, sem)}
FOPL
NP(s1, sem1)  VP(s2, sem2) 
S(append(s1, s2)),
compose(sem1, sem2m sem)
PROLOG
See later on
Compositional semantics
 Verify grammatical correct sentences
Problem: the previous grammar will generate sentences that are not
grammatically correct.
NL is not a context free language
 We must deal with
o cases
o agreement between subject and main verb in the sentence
(predicate)
o verb subcategorization: the complements that a verb can accept
3
CASES
Nominative case (subjective case) + agreement
I take the bus
Je prends l’autobus
You take the bus
Tu prends l’autobus
He takes the bus
Il prend l’autobus
Eu iau autobuzul
Tu iei autobuzul
El ia autobuzul
Accusative case (objective case)
He gives me the book
Il me donne le livre
Dative case
You are talking to me
El imi da cartea
Il parle avec moi El vorbeste cu mine
Solution to cases: new categories, e.g. NPS, NPO - not very efficient, too
many rules
 Augment the grammar with a new parameter to describe the case
S  NP(Subjective) VP
NP(case)  Pronoun (case) | Noun | Article Noun
VP  VP NP(Objective)
VP  VP PP
PP  Preposition NP(Objective)
I
give him the book
turn to the right
Pronoun(Subjective)  I | you | he | she
Pronoun(Objective)  me | you | him | her
 Augment the grammar with a new parameter to describe the verb
subcategorization
Verb subcategories – specify which verb can be followed by which other
categories
- each verb has a list of complements
4
1. augment VP to take a subcategorization argument
VP(subcat)  VP([NP | subcat]) NP(Objective)
| VP([Adjective | subcat]) Adjective
| VP ([PP | subcat]) PP
| Verb(subcat)
2. change S so that it has a VP with subcategories
S  NP(Subjective) VP( [])
3. add adjuncts to VP – verb phrases that may follow any verb, regardless
of the subcategory
VP(subcat)  VP(subcat) PP
| VP(subcat) Adverb
I smell the wumpus now
Resulting augmented DCG
S  NP(Subjective) VP( [])
NP(case)  Pronoun (case) | Noun | Article Noun
Pronoun(Subjective)  I | you | he | she
Pronoun(Objective)  me | you | him | her
VP(subcat)  VP([NP | subcat]) NP(Objective)
| VP([Adjective | subcat]) Adjective
| VP ([PP | subcat]) PP
| Verb(subcat)
| VP(subcat) PP
| VP(subcat) Adverb
Note: top down parsing may be problematic with this DCG as it is left
recursive.
5
We can try to implement it in Prolog but we must take care at left recursive
clauses
We can use bottom-up parsing
PROLOG
sentence(S) :- np(S1,subjective), vp(S2, subcat), append(S1, S2, S).
np([S], Case) :- pronoun(S, Case).
np([S], _ ) :- noun(S).
np([S1, S2], _ ) :- article(S1), noun(S2).
pronoun(i, subjective).
pronoun(me, objective).
pronoun(you, _ ).
pronoun(him, objective).
pronoun(he, subjective).
pronoun(her, objective).
pronoun(she, subjective).
vp(S., [np| Subcat]) :- vp(S1, Subcat), np(S2, objective), append(S1, S2, S).
OK up to now
But dangerous to translate
VP(subcat)  VP(subcat) PP
vp(S, Subcat) :- vp1(s1, Subcat), pp(S2), append(S1, S2, S).
2.3 Semantic analysis
Compositional semantics
The dog has legs.
The ball is yellow.
The ball is red.
The dog bites.
(dog parts legs)
(ball color yellow)
(ball color red)
(dog property bytes)
6
PROLOG (without case and subcategories)
sentence(S, Sem) :- np(S1, Sem1), vp(S2, Sem2), append(S1, S2, S),
Sem = [Sem1 | Sem2].
np([S1, S2], Sem) :- article(S1), noun(S2, Sem).
vp([S], Sem) :- verb(S, Sem1), Sem = [property, Sem1].
vp([S1, S2], Sem) :- verb(S1), adjective(S2, color, Sem1),
Sem = [color, Sem1].
vp([S1, S2], Sem) :- verb(S1), noun(S2, Sem1), Sem = [parts, Sem1].
NL does not have a compositional semantics for the general case
Ambiguity
A clear sky
The way is clear
A clear profit
John is clear
It is clear that …
I saw John in a restaurant with a telescope
John loves Mary
S(rel(obj))  NP(obj) VP(rel)
VP(rel(obj))  Verb(rel) NP(obj)
NP(obj)  name(obj)
name(John)  John
name(Mary)  Mary
Verb(x y Loves(x, y))  Loves
John loves Mary
(x (y Loves(x, y) Mary) John)
7
Intermediate form – quasi-logical form
NP(sem)  pronoun(sem)
S(rel(obj))  NP(obj) VP(rel)
VP(x rel1(x)  rel2(Event(rel1)))  VP(rel1) Adverb(rel2)
x e e Sees(x, Speaker)  After(Now, e)  During(e, Yesterday)
“ .. saw me yesterday”
e – event variable
2.4 Pragmatic interpretation
Complete the semantic interpretation by adding information about the
current situation
Indexical = phrases that refer directly to the current situation
 I am in Paris today
 Block A is clear and it is on block B
Anaphora = the occurrence of phrases referring to objects that have been
mentioned previously
 Mary was sleepy. She went to bed.
 The ball hit the house. It broke a window.
 John meet Mary at the bus station. They went to a restaurant.
AMBIGUITY
Lexical ambiguity
8
Syntactic ambiguity
 I saw the Statue of liberty flying over New York
 Salespeople sold the dog biscuits
 I saw John in a restaurant with a telescope
 Time flies like an arrow
Referential ambiguity
 Block A is on block B and it is not clear
 John met Mary and Tom. They went to a restaurant.
Pragmatic ambiguity
 I’ll meet you next Friday
 I am in the room with John
Disambiguation
- the world model
- the mental model
- the language model
- the acoustic model
9