CSE 576
Natural Language Processing
Final Report
Ravi Palla
Tae-Won Kim
Yunsong Meng
Introduction
It is known that monotonic semantics are not appropriate for knowledge representation and reasoning.
This is due to the fact that generally reasoning is done based on the knowledge available regarding the
domain which in most cases is incomplete. This implies that inferences might have to change if the
knowledge available changes. As an example, consider the following scenario where in the initial
knowledge available is

Normally, every car has a tank.

Reva is a car.
Given this information, we should be able to conclude that Reva has a tank.
Now, if we add

Reva is an electric car

Every electric has no tank
to the knowledge base, we should no longer be able to conclude that Reva has a tank. Under monotonic
semantics such as the semantics of first order logic, this is not possible. In order to derive the correct
conclusions, we need to reason based on non-monotonic semantics.
[Baral, Dzifcak, Son; AAAI 2008] proposed Answer set programming (ASP) as the target representation
for english sentences and showed how certain sentences can be converted to the syntax of ASP. Since
the underlying semantics of ASP (answer set semantics) is non-monotonic, this approach allows us to
make inferences based on non-monotonic reasoning. Though, this approach makes non-monotonic
reasoning possible, there are certain limitations to this approach. Below, we discuss 3 limitations of this
approach.
1. Syntax of ASP is restricted. Every answer set program is a set of rules of the form
a. A_1 v A_2 v … v A_k :- B_1, B_2, …, B_n
where A_i and B_j are literals. This implies that explicit quantifiers are not allowed in the syntax
of ASP.
Quantifiers play a major role in first order representations of knowledge and it is well
acknowledged that first order representation is natural and simple. As an example, consider the
sentence “Every car has a tank”. A first order representation of the sentence would be
x (car(x) => y( tank(y) & has(x, y) ) ). However, this representation cannot be used if we use
the ASP approach.
2. Since the syntax of ASP does not allow explicit quantifiers, any representation in the syntax of
ASP should be such that reasoning about unnamed objects in the domain is possible. Consider
the above example. An appropriate representation in ASP would be “hasTank(x) :- car(x)”. For
certain examples, it is hard to find such representations.
3. Answer set semantics consider only the Herbrand universe which implies that every object
constant represents a unique object. This implies that sentences like “Robert is also Bob” are
hard to represent (in first order representation, this can be represented as “robert = bob”.).
[Ferraris, et all; IJCAI 2007] generalized the answer set semantics (also called as stable model
semantics) to the syntax of first order sentences under which the above limitations are no longer valid. In
this project, we show how to obtain first order representations of certain sentences that enable us to
deduce correct inferences using the new language of stable models.
Aim of the project
Given any <T,H> where T and H are as follows:
T: A set of English sentences.
H: Hypothesis,
our aim is to develop a system that generates first order representations of T and H such that
SM[FOL(T)] entails FOL(H) iff H follows from T
where FOL(T) and FOL(H) are the first order representations of T and H respectively and SM[F] where F
is a first order sentence stands for a second order formula, the models of which are the stable models of
F.
For this project, we consider 20 sets shown below along with the expected target representation.
Abbreviations and Notations:

_n y F(y) stands for  y_1 …  y_n

t_cur is a constant indicating current time.

In some of the first order representations, x (F(x,t) & x = a) has been simplified as
( F(y_1) & … & F(y_n) &_{i != j} y_i != y_j ).
F(a,t).
Sentences with every that need explicit quantifiers.
1.
T:
Normally, every married man has a wife.
xt (married(x,t) & man(x,t) & -Ab(x,t) => w( wife(x,w,t) & has(x,w,t) ) )
Kofi is a married man.
t (man(Kofi,t) & married(Kofi,t) & t=t_cur)
H:
Kofi has a wife.
t w(has(Kofi,w,t) & wife(Kofi,w,t) & t=t_cur)
2.
T:
%current time
%current time
Every normal student has a favorite book.
xt (-Ab(x,t) & student(x,t) => b ( has(x,b,t) & favorite(x,b,t) & book(b,t) ) )
John is an abnormal student.
t (student(John,t) & Ab(John,t) & t=t_cur)
H:
%current time
John has a favorite book.
t b (has(John,b,t) & book(b,t) & favorite(John,b,t) & t=t_cur)
%current time
3.
T:
Every normal student is not married.
xt (-Ab(x,t) & student(x,t) => -married(x,t) )
Michael is an abnormal student.
t (student(Michael,t) & Ab(Michael,t) & t=t_cur)
H:
Michael is married.
t (married(Michael,t) & t=t_cur)
4.
T:
%current time
%current time
Normally, every professor teaches a class.
pt (professor(p,t) & -Ab(p,t) => c( class(c,t) & teach(p,c,t) ) )
James is a new professor.
t (professor(James,t) & new(James,t) & t=t_cur)
H:
%current time
James teaches a class.
t c (teach(James,c,t) & class(c,t) & t=t_cur)
%current time
Sentences dealing with properties that change with time.
5.
T:
Kofi was a married man.
t (married(Kofi, t) & t<t_cur & man(Kofi, t))
Normally, every married man has a wife.
 xt (man(x, t) & married(x, t) & -Ab(x, t) =>  y (has(x, y, t) & wife(x, y, t)))
Kofi is divorced.
t (divorced(Kofi, t) & t=t_cur)
Every divorced man is not married.
 xt (man(x, t) & divorced(x, t) => -married(x, t))
H:
Kofi has a wife.
 x t (has(Kofi, x, t) & wife(Kofi, x, t) & t=t_cur)
6.
T:
John was rude.
t (rude(John, t) & t<t_cur)
H:
John is rude.
t (rude(John, t) & t=t_cur)
7.
T:
John was a rude person.
t(rude(John, t) & person(John, t) & t<t_cur )
John is polite now.
t (polite(John, t) & t=t_cur)
Every polite person is not rude.
xt (polite(x, t) & person(x, t) => -rude(x, t))
H:
John is rude.
t (rude(John, t) & t=t_cur).
8.
T:
James was a basketball player.
t (player(James, basketball,t) & t<t_cur)
Every basketball player has a team.
xt (player(x, basketball,t) => y (has(x, y, t) & team(y, t)))
James is retired.
t (retired(James, t) & t=t_cur)
Every retired player has no team.
xzt (retired(x, t) & player(x,z,t)=> -y (has(x, y, t) & team(y, t)))
H:
James has a team.
xt (has(James, x, t) & team(x, t) & t=t_cur)
Sentences with numbers.
9.
T:
Every student with 3 noisy roommates is unhappy.
xt (student(x,t) & _3 y (noisy(y,t) & has(x,y,t) & roommate(x,y,t)) => unhappy(x,t))
John has 2 noisy roommates.
 t _2 y (noisy(y,t) & has(John,y,t) & roommate(John,y,t) & t=t_cur)
H:
John is unhappy.
 t (unhappy(John,t) & t=t_cur).
10.
T:
Every car that has 4 airbags is safe.
xt (car(x,t) & _4 y (airbag(y,t) & has(x,y,t)) => safe(x,t))
Civic is a car that has 2 airbags.
 t (car(civic,t) & _2 y (airbag(y,t) & has(civic,y,t)) & t=t_cur).
H:
Civic is a safe car.
 t (car(civic,t) & safe(civic,t) & t=t_cur).
11.
T:
Every professor has 20 publications.
xt (professor(x,t) => _20 y (publication(y,t) & has(x,y,t)))
John is a professor.
 t (professor(John,t) & t=t_cur).
H:
John has 20 publications.
 t _20 y (publication(y,t) & has(John,y,t) & t=t_cur).
12.
T:
There are 2 apples on the table.
 t _2 y (apple(y,t) & on(y,table,t) & t=t_cur).
H:
There are 3 apples on the table.
 t _3 y (apple(y,t) & on(y,table,t) & t=t_cur).
Sentences with numbers and "normally".
13.
T:
Normally, every car that has 4 airbags is safe.
xt (car(x,t) & -Ab(x,t) & _4 y (airbag(y,t) & has(x,y,t)) => safe(x,t))
Civic is a car that has 2 airbags.
 t (car(civic,t) & _2 y (airbag(y,t) & has(civic,y,t)) & t=t_cur).
Civic is safe.
 t (safe(civic,t) & t=t_cur).
John drives a Civic.
 t (drives(John,civic,t) & t=t_cur).
H:
John drives a safe car.
 t  x (drives(John,x,t) & safe(x,t) & car(x,t) & t=t_cur ).
14.
T:
Normally, every student with 3 noisy roommates is unhappy.
xt (student(x,t) & -Ab(x,t) & _3 y (noisy(y,t) & has(x,y,t) & roommate(x,y,t)) =>
unhappy(x,t))
John has 3 noisy roommates.
 t _3 y (noisy(y,t) & has(John,y,t) & roommate(John,y,t) & t=t_cur)
H:
John is unhappy.
 t (unhappy(John,t) & t=t_cur).
15.
T:
Every normal plane carries 200 passengers.
xt (plane(x,t) & -Ab(x,t) => _200 y (passenger(y,t) & carries(x,y,t)))
Every damaged plane is abnormal.
xt (plane(x,t) & damaged(x,t) => Ab(x,t))
DA is damaged.
 t (damaged(DA,t) & t=t_cur).
H:
DA carries 200 passengers.
t
16.
T:
_200 y (passenger(y,t) & carries(DA,y,t) & t=t_cur).
Every normal professor takes 2 classes.
xt (professor(x,t) & -Ab(x,t) => _2 y (class(y,t) & takes(x,y,t)))
John is not a normal professor.
 t ((-professor(John,t) | Ab(John,t)) & t = t_cur).
H:
John takes 2 classes.
 t _2 y (class(y,t) & takes(x,y,t) & t=t_cur).
Sentences giving different names to the same object (no UNA).
17.
T:
Mr.Obama is a president.
t (president(Mr.Obama, t) & t=t_cur)
Barak is a man.
t (man(Barak, t) & t=t_cur)
Barak is Mr.Obama.
Barak=Mr.Obama
H:
Barak is a male president.
t (man(Barak, t) & president(Barak,t) & t=t_cur)
18.
T:
Robert is a student.
t (student(Robert, t) & t=t_cur)
Bob is a mechanic.
t (mechanic(Bob, t) & t=t_cur)
Robert is Bob.
Robert=Bob
H:
Bob is a student and a mechanic.
t (student(Bob, t) & mechanic(Bob, t) & t=t_cur)
19.
T:
Normally, every person named John is not from Korea.
 xt (person(x, t) & named(x, John, t) & -Ab(x, t) => -from(x, Korea, t))
Normally, every person named Mr.Kim is from Korea.
 xt (person(x, t) & named(x, Mr.Kim, t) & -Ab(x, t) => from(x, Korea, t))
John is Mr.Kim.
John=Mr.Kim
 xt (named(x, x, t)). % background knowledge
H:
John is from Korea.
t (from(John, Korea, t) & t=t_cur)
Sentences with numbers and “not”.
20.
T:
John has a red ball.
t b (has(John,b,t) & ball(b,t) & red(b,t) & t=t_cur)
%current time
Every ball John has is red.
bt (has(John,b,t) & ball(b,t) & t=t_cur => red(b,t) )
H:
%current time
John does not have 2 balls.
t -_2b(has(John,b,t) & ball(b,t) & t=t_cur)
% current time
CCG and 
1,2,3, etc
NP/NP
With has - p t y _n w ( ((p@t)@y)@w)
With has and not - p t y -_n w ( ((p@t)@y)@w)
Without has - p q t x _n w ( (p@t)@w & ((q@t)@x)@w)
Without has and subject - p q t _n w ( (p@t)@w & (q@t)@w)
A
NP/NP
With that - p q t y x ( (p@t)@x & q@x & x=y )
Without that and every/direct ref - p t y x ( (p@t)@x & x=y )
(kofi is a married man).
Referring to object directly - p p
Without has/with verb before it –
p q t x y ( (p@t)@y & ((q@t)@x)@y )
With not - p t y - w ( (p@t)@w & w=y)
a_have: u t z xy ( ((u@t)@x)@y & y=z)
a_be: u t z x ((u@t)@x & x=z)
With every and without that/ own - p t y x ( ((p@t)@y)@x) –
(own : John has a ball)
Abnormal
NP, NP/NP
t x (Ab(x,t)),
p t x (Ab(x,t) & (p@t)@x )
and
(NP\NP)/NP
u v t z (u @t@z & v @t@z)
An
NP/NP
p t y x ( (p@t)@x & x=y )
Airbag
NP
t y x (airbag(x,y,t))
Apple
NP
t x (apple(x,t))
Are
(S\NP)/NP
p t ( p@t & t=t_cur)
Ball
NP
t x y ball(x, y, t)
Barak
NP
Barak
Basketball
NP/NP
u u @ basketball
Book
NP
t y (book(y,t))
Bob
NP
Bob
can
(S\NP)/(S\NP)
p x t ( (p@t)@x & t=t_cur)
Car, Civic
NP
t y (car(y,t)), civic
Carries
(S\NP)/NP
p q q@(p@( t x y carries(x,y,t) ) )
Class
NP
t x (class(x,t))
currently
(S\NP)/(S\NP)
p x t ( (p@t)@x & t=t_cur)
DA
NP
DA
Damaged
NP/NP,NP
p t x (damaged(x,t) & (p@t)@x ), t x (damaged(x,t))
Divorced
NP/NP
t  x (divorced(x, t))
Does
(S\NP) / (S\NP)
p p
Drives
(S\NP)/NP
p p@(t x y (drives(x,y,t) )
Every
NP/NP
With with/that –
-
p r q x t ( (p @ t)@x
& (r @ t)@x
-> (q @ t)@x)
Without with/that - p q x t ( (p @ t)@x
-> (q @ t)@x)
With with/that and normally –
p r q s x t ( (p @ t)@x
& (r @ t)@x
& (s @ t)@x
->
(q @ t)@x)
Without with/that and with normally p r q x t ( (p @ t)@x
& (q @ t)@x
-> (r @ t)@x)
With that and ownership –
uvw xt (u@t@x@v=> w@t@x)
Favorite
NP/NP
p t z x (favorite(z,x,t) & (p@t)@x )
From
NP/NP
y t x from(x, y, t)
Has
(S\NP)/NP
Without that and every / own - p y t ( (p@t)@y & t=t_cur) ( John
has a ball)
With that and every
- p v v@( t y (p@t)@y)
With that and without every - p y t ( (p@t)@y & t=t_cur)
Without that and with every - p q q@p
Has no_every: u s. s @ -y u@y
Have
(S\NP)/NP
p p
Is
(S\NP)/NP
With every and that/with and dirref - x y y@x (every student is not
married)
Without every / direct ref - p x t ( (p@t)@x & t=t_cur) (Kofi is a
married man)
With every and without that/with - p v v@( t x (p@t)@x)
Is_equal -  u  w u=w
With every and ownership - u t w ((u @t) @w)
Was
(S\NP)/NP
Was: u w t (u @t @w & t < t_cur)
James, John
NP
James, John
Kofi, Korea
NP
Kofi, Korea
Male
NP/NP
u t x (male(x, t) & (u @t) @x)
Man
NP
t x man(x, t)
Married
NP/NP, NP
u t x (married(x, t) & (u @t) @x)
With not: t y -(married(y,t)), t x (married(x, t))
Michael, Mr.
NP
Michael, Mr.Obama, Mr. Kim
Mechanic
NP
t x mechanic(x, t)
Named
(NP\NP)/NP
y u t x (u@t@x & named(x, y, t))
New
NP/NP
u t x (new(x, t) & (u @t) @x)
Now
S\S
s. s @s
Noisy
NP/N
p t y x (noisy(x,t) & ((p@t)@y)@x)
No
NP/NP
Coupled with has
Normal
NP/NP
p t x (-Ab(x,t) & (p@t)@x )
Normally
S/S
p p@(t x -Ab(x,t))
Not
(S\NP)\(S\NP)
p p
Obama, Mr.
Kim
Is_every not: u t w (-((u @t) @w))
On
(NP\NP)/NP
y t x (on(x,y,t))
Passenger
NP
t x (passenger(x,t))
Person
NP
t x person(x, t)
Professor
NP
t x (professor(x,t))
president
NP
t x president(x, t)
Publication
NP
t y x (publication(x,y,t))
player
NP
y t x (player(x, y, t))
Plane
NP
t x (plane(x,t))
Polite
NP, NP/NP
t x (polite(x, t)), u t x (polite(x, t) & (u @t) @x)
Retired
NP, NP/NP
t x (retired(x, t)), u t x (retired(x, t) & (u @t) @x)
Red
NP/NP, NP
u t (x y (red(x, t) & (u @t) @x) @y)
Rude
NP, NP/NP
t x (rude(x, t)), u t x (rude(x, t) & (u @t) @x)
Robert
NP
Robert
Roommate
N
t y x (roommate(x,y,t))
Safe
NP, NP/NP
Dirref - t
z (safe(z,t)) (Civic is safe),
p t x (safe(x,t) & (p@t)@x)
Student
NP
t y (student(y,t))
Table
NP
table
Takes
(S\NP)/NP
p q q@(p@( t x y takes(x,y,t) ) )
Team
NP
t x team(x, t)
Teaches
(S\NP)/NP
p p@ (t x y (teaches(x,y,t)))
That
(NP\NP)/(S\NP)
With every - p p
Without every - q p p@q
The
NP/NP
y y
There
NP
y y
Unhappy
NP
t z (unhappy(z,t))
Wife
NP
t xy wife(x, y, t)
With
(NP\NP)/NP
p v v@( t y (p@t)@y)
Example:
Every professor
has
20
NP/NP
(S\NP)/NP
NP/NP
NP
---------------------
publications.
NP
--------------------------
NP
NP
-------------------------------------------S\NP
------------------------------------------------------------------------S
Every :
p. q. x. t. ( (p @ t)@x
professor:
t. y. (professor(y,t))
has:
p. q. q@p
20:
p. t. y. _20 w. ( ((p@t)@y)@w)
publications:
t. y. x. (publication(x,y,t))
-> (q @ t)@x)
This gives the representation x t ( professor(x,t) => _20 w publication(w,x,t) )
The System
The system is a modification of the system by Juraj Dzifcak. The parser does forward and backward
composition. The system takes two files, one containing the English text and the other containing the
categories and lambda expressions (dictionary), as input and outputs a first order sentence
corresponding to each English sentence. Each sentence of the English text should be in a separate line.
Similarly, the categories and lambda expressions for each word should be in a separate line. The word,
categories and the lambda expressions should be separated by “:” and different categories and lambda
expressions for the same word should be separated by “,”. This implies “,” and “:” cannot be part of any
lambda expression. Also “” is represented using “#” and “\” is represented by “|”. To avoid conflict with
variables, all predicate names, EXISTS and FORALL are in uppercase and every variable is followed by
a dot (.).
The dictionary for the sentence “Every professor has 20 publications” is shown below:
Every: NP[every]/NP: #p.#q.FORALL x. FORALL t. ((p @ t)@x)
-> ((q @ t)@x)
professor: NP: #t.#y. PROFESSOR(y_t)
has: (S|NP[every])/NP, (S|NP)/NP: #p.#q.(q@p), #p.#y.EXISTS t.((p@t)@y) & t=a
20: NP/N: #p.#t.#y. EXISTS20 w. ( ((p@t)@y)@w)
publications: N: #t.#y.#x. PUBLICATION(x_y_t)
When the sentence and the dictionary is provided as input to the system, it outputs
FORALLv24.FORALLv25.PROFESSOR(v24,v25)->EXISTS20v22.PUBLICATION(v22,v24,v25).
Every T, H set has an English text file and corresponding dictionary. However, it is also possible to have
a common dictionary for all the 20 sets and the procedure used to merge dictionaries is explained in the
following section.
Merging dictionaries
It is possible that the same word with the same category across different sentences can have different
lambda expressions. For example, the word “every” in the following sentences has the same category
but different lambda expressions:

Every professor has 20 publications. (Every: NP/NP: #p.#q.FORALL x. FORALL t. ((p @ t)@x)
-> ((q @ t)@x) ).

Every student with 3 noisy roommates is unhappy. ( Every: NP/NP: #p.#r.#q.FORALL x.
FORALL t. ((p @ t)@x)
& ((r @ t)@x)
-> ((q @ t)@x) ).
In order to obtain the correct representation for the sentence we need to make sure that the parser picks
up the correct lambda expression for the word. This is done by adding conditions to the categories. The
conditions basically represent the properties of sentences in which the words occur. For example, in the
second sentence above, “every” occurs in a sentence that contains “with”. So, changing the category of
“every” in the sentence to NP[with]/NP and writing the categories of “with”
and “is” as
(NP[with] | NP[with])/NP and (S | NP[with])/NP will force the parser to pick the correct lambda expression
for “every”. The following shows the combined dictionary for the two sentences above:
Every: NP/NP, NP[with]/NP: #p.#q.FORALL x. FORALL t. ((p @ t)@x)
#p.#r.#q.FORALL x. FORALL t. ((p @ t)@x)
& ((r @ t)@x)
-> ((q @ t)@x),
-> ((q @ t)@x)
professor: NP: #t.#y. PROFESSOR(y_t)
has: (S | NP)/NP: #p.#q.(q@p)
20: NP/N: #p.#t.#y. EXISTS20 w. ( ((p@t)@y)@w)
publications: N: #t.#y.#x. PUBLICATION(x_y_t)
student: NP: #t.#y. STUDENT(y_t)
with: (NP[with] | NP[with])/NP: #u.#v.(v@ #t.#z.((u@t)@z))
3: NP/NP: #p.#t.#y. EXISTS3 w. ( ((p@t)@y)@w)
noisy: NP/NP: #p.#t.#y.#x. NOISY(x_t) & (((p@t)@y)@x)
roommates: NP: #t.#y.#x. ROOMMATE(x_y_t)
is: (S |NP[with])/NP: #x.#y. (y@x)
unhappy: NP: #t.#z. UNHAPPY(z_t)
Now in order to for the parser to obtain “S” after parsing the second sentence, the parser has to pick
NP[with]/NP and #p.#r.#q.FORALL x. FORALL t. ((p @ t)@x)
& ((r @ t)@x)
-> ((q @ t)@x) for the
word “every”.
As part of the project, we have merged the dictionaries of the <T, H> sets 1 – 4 and 9-10. So, all the
sentences corresponding to these sets can be run using a common dictionary (dic_common.txt).
Testing new sentences
The common dictionary generated is not really useful unless it can be used to obtain the correct
representation for any sentence such that

every word with category X in the sentence is present in the dictionary with category X.
For example, dic_common.txt contains all the categories and lambdas for

John has a car

Mary has a favorite book.
However, it also works for
“Kofi has a favorite book”.
In addition to the above, if with respect to a sentence, some word with category X in the sentence is
missing in the dictionary, minimal updation of the dictionary should be sufficient to obtain the correct
representation of the sentence.
For example, consider the “electric car” example modified as follows:
T:
Normally every car has a tank.
Reva is a car.
Reva is an electric car.
Every electric car is an abnormal car.
Every electric car has no tank.
H:
Reva has a tank.
Consider dic_common.txt. The dictionary already contains all words with corresponding categories in the
above sentences except:
tank: NP: #t.#x.#y. TANK(y_x_t)
Reva: NP: reva
electric: NP/NP: #p.#t.#x. ELECTRIC(x_t) & ((p@t)@x)
no: NP[own]/NP: #p.#t.#x.-EXISTS y. (((p@t)@x)@y).
Adding the above to dic_common.txt produces correct representations for all the above sentences.
Conclusions and future work
In this project, we have shown how to obtain first order representations of English sentences for nonmonotonic reasoning under the stable model semantics. We also presented a system that generates the
first order representations for certain kinds of sentences. However, there is no system that can check if
FOL(H) follows from SM[FOL(T)] since the available solvers that compute stable models only work for
the syntax of ASP and not for the general syntax of first order logic. Since the new stable model
semantics is based on second order logic and computing models of second order logic is undecidable,
we may not be able to perform entailment checking as required for the project. However, under certain
conditions (such as safety and finite complete set of loops), SM[F], where F is a first order sentence, is
equivalent to a first order sentence which enables us to use first order theorem provers to check
entailment. It is interesting to have such a system implemented and see how it can actually be used to
solve various problems in Natural Language Processing and Knowledge representation and reasoning.
Appendix
In the submitted system,

dicN_1 and senN_1 are the dictionary and sentences for the <T,H> set N.

dic_common.txt is the common dictionary for <T,H> sets 1-4, 9-10 and the “electric car”
example.

sentences_covered.txt contains all the sentences for <T,H> sets 1-4, 9-10 and the “electric car”
example.

sen_electriccar.txt contain the sentences for the electric car example.

test.txt contains a few new sentences that have been tested using dic_common.txt.