Logic Programming for Linguist(ic)s
+
Logic Grammars with Constraints
≈ an easy way to Semantics
Tutorial at GRLMC, Tarragona, 20 November 2009
Henning Christiansen
professor of Computer Science
Research group PLIS: Programming, Logic and Intelligent Systems
Department of Communication, Business and Information Technologies
Roskilde University
P.O.Box 260, DK-4000 Roskilde, DENMARK
http://www.ruc.dk/~henning
Speaker’s background
• Computer Science; 1980ies and onwards programming
languages, compilers and syntax (“extensible syntax”)
• Current interests: Logic Programming, Databases, AI,
Implementation of advanced reasoning, applications for
NLP, Bioinformatics, Probabilistic Models, etc.
• Background in linguistics = Ø, but inspiration from
– working with people who know more about language than I do
– the reception of my LP work by such people
• Projects
– The CONTROL research project (2004–8)
• “CONstrainTs for RObust Language processing”
– The LoSt research project (2007-2012)
• "Logic-Statistical analysis of biological sequence data"
• 3 PhD students, 2 postdocs, 2 companies, partners in Denmark,
Tokyo, ... contacts to GB, Norway...
1
Motivation & intuition
Logic programming in linguistics / for linguistic students:
• You can program without being a programmer 
• Logic programming in Prolog is simple, intuitive and wellsuited for phrase-structure grammars
Involving constraints in logic grammars:
• Provides an easy way to semantics/pragmatics
• while keeping the qualities of Prolog programming
2
Outline of this tutorial
1. Crash course on Prolog programming
–
with references if you want to continue yourselves
2. Definite Clause Grammars, DCGs (simple add-on to
Prolog)
3. Introducing Constraint Handling Rules, CHR (another
add-on to Prolog)
4.
5.
6.
7.
Relation to Abductive Logic Programming
Using CHR for to add meanings to DCGs
A practical application
Conclusion
• Approach won’t solve all problems, but good for illustrative purposes
• and in teaching
• (prototype) applications
• and (not least) fun to play with!
3
1. A crash course on Prolog
Prolog differs from Java, C, etc.:
• based on logic
–
–
–
–
focuses on "what" rather that "how"
programs are concise and compact
programs can be tested and built incrementally
rule-based
• symbols and structure (rather that numbers)
• and very easy to learn!
History
• A.Colmerauer & co. (Marseille), ca. 1970: "Prolog"
• D.H.D. Warren: Efficient compiler, 1975,
• Language made known by R.Kowalski "Logic for Problem
solving", 1979, ....
4
• Since then: several available systems...
Program is a description of data
parent(
parent(
parent(
parent(
parent(
parent(
pam,
tom,
tom,
bob,
bob,
pat,
bob). % Pam is a parent of Bob
bob).
liz).
pam
tom
ann).
pat).
jim).
bob
liz
ann
pat
jim
Basic notions:
• predicates: parent
– describes a relation
– defined by facts, rules, collectively called clauses
• constants: tom, bob, x, y
• variables: X, Y, Tom
• simple goals: parent(A,a)
• Queries....
Queries
Atomic queries
?- parent(X,Y).
... give me values of X and Y so parent(X,Y) logically
follows from program
Compound query
?- parent(pam, X), parent(X, Y).
... give me X and Y, so that...
Procedural semantics
parent( pam, bob).
X=bob
•Unification term=term?
parent( tom, bob). ?- parent(pam,
•from left toX),
rightparent(X, Y).
Y).
•from start to
end
parent( tom, liz). ?- parent(bob,
parent( bob, ann).
parent( bob, pat).
Y=ann
•backtracking
Y=pat
≈ undo and trySuccess!
new
solutions?
choices Other Solutions?
parent( pat, jim).
No
Nomore
morepossible
possible
solutions
solutionsathere
all :(:(
Rules
Procedural semantics
as before + rewrite subgoal using
rules
grandparent(X,Z):parent(X, Y),
parent(Y, Z).
Declarative semantics ≈ logical
consequence in 1st order logic
The nice property:
procedural ≈ declarative
Other features
• structures and lists (will see later)
• control mechanisms (advanced; not needed for today)
2. Definite Clause Grammars
Syntactic sugar for Prolog programs that perform topdown, backtracking parsing.
In Prolog since 1970ies [Colmerauer; Pereira, Warren; ...]
Features ≈ arguments and terms as in Prolog predicates
Example:
?- [simpleDCG].
Popular, easy to work with, fun to play with, etc.
10
Adding contexts ≈ knowledge bases
Usual approach: more and complex features.
Discourse analysis with semantics
story(C0,M1+M2) --> [] ; s(C0,C1,M1), ['.'], story(C1,M2).
We try another something else: use a global resource to
store context and meanings.
Thus stay with the rule
story --> [] ; s, ['.'], story.
But “global resource” in Prolog??
= constraint store and meanings etc. as constraints...
11
3. Constraint Handling Rules, CHR
A recent addition to Prolog; available in several major
Prologs [Frühwirth, 1992, 1998]
A “glass box” approach to constraint solving:
• defining constraints and their behaviour in a
declarative language
Prolog: Backward chaining
CHR: Forward chaining
Prolog+CHR by an example that anticipates
applications for language analysis...
?- [happy1].
?- [happy2].
?- [happy3].
12
3. Theoretical interlude:
You have just seen an implementation of Abductive Logic
Programming using Prolog+CHR (Peirce, 18-something);
(Conan Doyle, 19-something - Sherlock Holmes); Kakas,
Kowalski, etc. 1980ies, ......)
Def: Given Rules and Observations, abducible predicates and
Integrity Constraints, a set of abducible facts A is an abductive
?- [RulesAbdPredsICs].
answer whenever
?- Observations.
Rules U A |= Observations
abductive-answer ? ;
Rules U A |= Integrity Constraints
Relationship ALP <-> CHR:
[S.Abdennadher, HC, 2000; HC, V.Dahl, 2004, etc.]
H. Christiansen. Executable specifications for hypothesis-based
13
3. Back to CHR
Three kinds of rules:
Propagate
c, c, ..., c ==> Guard | ... c ...
adds constraints
Simplify
c, c, ..., c <=> Guard | ... c ...
replaces constraints
Simpagate
c, ... \ c, ... <=> Guard | ... c ...
remove some, keep some and add others
———————————————————————
Declarative semantics: as indicated by the arrow shapes
Obs: Prolog+CHR sum up to a language with declarative semantics
Theorem: Implementation of abduction is correct :)
14
5. Back to discourse analysis
First version, just collecting facts
?- [discourse1].
Second version, adding rules about semantic/pragmatic universe
?- [discourse2].
—————————————————————————————
Reflections:
• An instance of “Interpretation of abduction”, [Hobbs & al, 1993]
• Notice that distinction semantics/pragmatics disappears, i.e.,
– i.e., no “objective and context independent” meaning before mapping
to “real world objects”,
• [Christiansen, Dahl, Context05] formalize “Meaning in Context”
with possible-worlds semantics and relates to abduction and CHR
15
A taste of “Meaning in Context”
[Christiansen, Dahl, Context05]
Meaning of
The tall, red-haired man carrying a laptop won a brand new Ferrari.
is
won(X,Z)
in the context
tall(X), red_haired(X), carries(X,Y), laptop(Y),
won(X,Z), ferrari(Z), brand_new(Z)
Might be the same as the meaning of
He won it
16
Context in disambiguation
• version 1, static context
?- [contextLex1].
• version 2, non-monotonic change of context ?- [contextLex2].
Theoretical interlude
• Version 2 goes beyond the indicated declarative
semantics
• A programmer’s mind uses procedural knowledge
to implement a sort of non-monotonic logic.
• [Betz, Frühwirth, CP05] has defined a linear-logic
semantics for CHR which fits such usages
17
Assumptions in logic grammars
• [Dahl & al., 1997; Christiansen, Dahl, 2004, ...]
• Similar to abduction but with explicit creation and
application + simple scoping
• Can be implemented in CHR more or less the same way
• Included in the Hyprolog system
+A
Assert linear assumption A for subsequent proof steps.
Linear means “can be used once”.
*A
Assert intuitionistic assumption A for subsequent proof steps.
Intuitionistic means “can be used any number of times”.
-A
Expectation: consume/apply existing intuitionistic assumption in the state which
unifies with A.
=+A, =*A, =-A
Timeless versions of the above, meaning that order of assertion of assumptions
and their application or consumption can be arbitrary.
Examples...
18
Assumptions in Hyprolog
An example: Pronoun resolution
assumptions acting/1.
abducibles fact/1.
sentence --> np(A,_), verb(V), np(B,_), {fact(A,V,B)}.
sentences --> [] ; sentence(S1),sentences(S2).
np(X,Gender) --> name(X,Gender), {*acting(X,Gender)}.
name(peter,masc) --> [peter].
...
np(X,Gender) --> {-acting(X,Gender)}, pronoun(Gender).
pronoun(fem) --> [her].
“Peter likes Mary. She likes him”
...
verb(like) --> [likes]. *acting(peter,masc) *acting(mary,fem)
-acting(X,fem)  X=mary
-acting(X, masc)  X=peter
fact(peter,like,mary)
fact(mary, like,peter)
19
Assumptions in Hyprolog
Another example: Coordination
assumptions ..., verb_used/1.
sentence --> np(A,_), verb(V), np(B,_),
{*verb_used(V), fact(A,V,B)}.
...
verb(like) --> [likes].
verb(V) --> [],{-verb_used(V)}.
“Peter likes Mary, and she him”
...
*verb_used(like)
-verb_used(X)  X=like
fact(peter,like,mary)
fact(mary,like,peter)
20
Side remark
What we have shown until now, may be seen as logic
programming dressed up by a nice syntax.
It has been described in a formal way:
Henning Christiansen, Verónica Dahl
Abductive Logic Grammars.
Logic, Language, Information and Computation. 16th International
21
6. An practical application in more detail
Mapping use case text (a la OOP/OOD) into UML
diagrams
Example of discourse analysis, building a knowledge base
= context
Features:
• non-trivial properties of semantic/pragmatic universe
• resolution of pronouns based on well-defined heuristics
[Christiansen, Have, Tveitane, RANLP2007, CSLP2007]
22
The setting...
• Use Cases
– Textual description of what a “system” does without
specifying how.
– Used in the inception phase of the object oriented software
development process.
• Reasoning about Use Cases
– Parsing use cases to uncover programmatic semantics
– Result: Diagrammatic representation (UML)
Language for Use Cases
•
•
•
•
Restricted natural language (English)
Natural flow, style and expressiveness
Unambiguous parsing
Why avoid ambiguity
Programmatic Semantics
•
•
•
•
•
•
Classes and objects
Methods
Properties
Inheritance
Instantiation
Temporality (not shown here)
Programmatic semantics, methods
• The professor teaches.
Generates one class:
With one method
Programmatic semantics, methods
• The professor teaches.
• A student reads, writes
projects and takes
exams.
Programmatic Semantics, properties
• A professor has an
office.
Programmatic Semantics, properties
• A professor has an
office.
• The university has five
study lines.
Programmatic Semantics, inheritance
• Students and professors
are persons.
System Architecture
Architecture = Prolog + DCG + CHR
 Diagrams are generated by GraphViz

fc_
“cats and dogs”
Input
grammar
indiv_ “her, Peter and Paul”
cat+dog
mary+peter+paul
The use case text is parsed using
Grammar
rc_ a Definite
“MaryClause
and the
boys” (DCG) woman+boy
q_
“a tail and some legs”
tail:1+legs:n
sentence -->
fc_noun_phrase(Number,_ , subj, IdSub),
subord_verb(Number,_ ),
fc_noun_phrase(_ , Number, obj, IdObj),
{extends(IdSub,IdObj)}.
'extends' is not a
usual Prolog predicate,
but declared as
a constraint
There are sentence grammar rules like this for different sentence types
 Each rule captures the semantic features of a particular type of sentence

Reasoning with Constraints, I
Example:
Professors and students are persons.
Example:
A professor is a teacher and a researcher.
Constraint Handling rule:
Constraint Handling rule:
extends(A+B, C) <=>
extends(A,C),
extends(B,C).
extends(A, B+C) <=>
extends(A, B),
extends(A, C).
Constraint store:
Constraint store:
extends(professor+student, person)
extends(professor, teacher+researcher)
extends(professor, person)
extends(student, person)
extends(professor, teacher)
extends(professor, researcher)
Reasoning with Constraints, II
Example: Paul has a dog and Peter has five dogs. (Peter and Paul are men).
Constraint Handling rule:
property(C,P:N), property(C,P:M) <=>
q_count(N), q_count(M), q_less_eq(N,M)
| property(C,P:(N..M)).
...
property(man, dog:1)
property(man, dog:5)
property(man, dog:(1..5))
Reasoning with Constraints, III
Ranges can be merged:
property(man, dog:(0..2))
property(man, dog:(1..n))
property(man, dog:(0..n))
Constraint Handling rule:
property(C,P:(N1..M1)),property(C,P:(N2..M2)) <=>
q_min(N1,N2,N), q_max(M1,M2,M),
property(C,P:(N..M)).
Reasoning with Constraints, IV
Anaphora resolution - Individuals as prototypes.
Example: John is a student. John studies.
class(student)
object(john, student)
method(john, study)
class(student)
object(john, student)
method(student, study)
Constraint Handling Rule:
object(Id, Cl) \ method(Id, M) <=> method(Cl, M)
Reasoning with Constraints, V
Anaphora resolution - Pronouns.
Example: Jack and John are teachers. Jack teaches music. John teaches
computer science. Mary is a student. He has many students.
Constraint store:
...
referent(sing,masc,1,jack)
referent(sing,masc,1,john)
referent(sing,masc,2,jack)
referent(sing,masc,3,john)
referent(sing,fem,4,mary)
expect_referent(sing,masc,X)
He refers to John.
Referents are labelled with the
sentence number in which they occur.
Heuristic method resolves the entity
matching number and gender in
the most recent sentence.
Ambiguous pronouns are rejected.
Constraint Handling Rule:
sentence_no(Now), referent(No,G,Id,T) \ expect_referent(No,G,X) <=>
T < Now, there is no other relevant referent with Timestamp > T
|
if there is another relevant referent with Timestamp = T then
X = errorcode(ambiguous)
else
X = Id.
Conclusion of the example
Shows an application of discourse analysis with
• restricted language
• “concise” semantics
• a desire of an unambiguous interpretation
Demonstrates that Prolog+DCG+CHR applied for
“interpretation as abduction” in this case provides
• flexibility and ease of modeling
• concise, executable specifications
– logic embedded in a full programming language, i.e., you can hack
the logic if you feel like it
38
Other approaches to languages
analysis using CHR
CHR grammars [Christiansen, 2002-3, TPLP2005]
Grammar notation for bottom-up parsers entirely with CHR; features
• Context-sensitive rules
a -\ b,c /- ..., d ::> e
compiled into
a(_,N1), b(N1,N2), c(N2,N3), d(N4,_) ==> N3=<N4 | e(N1,N3)
•
•
•
•
Ambiguity and partial parsing handled for free
Includes abduction and assumption
Disadvantage: Alternative interpretations all mixed up
a version exists, which handles this correctly; never documented or
released (awkward implementation in CHR; low-level impl. considered)
Applications
• Blache & Dahl: Property grammars (repr. phrase as set of sat/unsat constraints)
• Dahl and her students: Biomedical text; sequence analysis; etc.
• My students: Hieroglyph inscriptions; use cases (a la OOP);
39
sign language (strong potential; but, sic!, we did not get the grant)
Conclusions
• Constraints adds a state component to logic grammars
• The state can be first-order formulas or other other
kinds of knowledge representation
• Interpretation-as-abduction for discourse analysis fits in
naturally
• Nonmonotonic updates can be modelled, e.g., reflecting
a time dimension over a changing world
• Prolog + CHR provides
– a flexibility for a programmer’s mind to hack the logic (and
perhaps formalize afterwards;-)
– an effective environment for prototyping / executable
specifications / implementation
40
Recent results
Probabilistic & prioritized abduction in CHR, best first search
•
H.Christiansen. Implementing Probabilistic Abductive Logic Programming with
Constraint Handling Rules, 35 pp., Constraint Handling Rules, Current Research
Topics. T. Schrijvers, T. Frühwirth (eds.). Lecture Notes in Artificial Intelligence 5388,
Springer 2008. pp. 85-118
H. Christiansen. Prioritized Abduction with CHR, Proc. Fifth Workshop on Constraint
Handling Rules, Hagenberg, Austria, July 14, 2008.
•
Probabilistic DCGs
•
C.T.Have. Stochastic Definite Clause Grammars. Proc. RANLP 2009, Recent
Advances in Natural Language Processing, Bulgaria, September 14-17, 2009.
C.T.Have., Stochastic Definite Clause Grammars; Master thesis, IT-University of
Copenhagen, 2008
•
Work in progress
General probabilistic-logic models for bioinformatics, e.g.
•
H. Christiansen. C.T. Have, O.T. Lassen, M. Petit. A Constraint Model for
Constrained Hidden Markov Models: a first Biological Application WCB09,
Workshop on Constraint Based Methods for Bioinformatics, Associated to CP 2009.
Lisbon, Portugal, September 20, 2009.
41
References
Texts
• Intro to Prolog with CHR and AI applications
H.Christiansen: Introduction to Prolog as a framework for Knowledge
Representation and Artificial Intelligence, (2006)
http://www.ruc.dk/~henning/KIIS08/courseMaterial/CourseNote.pdf
• Intro to Prolog, focus on DCG
P.Blackburn, J.Bos, and K.Striegnitz: Learn Prolog Now! (2001)
http://www.learnprolognow.org
• Overview of Prolog+CHR for logical reasoning
Henning Christiansen. Executable specifications for hypothesis-based reasoning with Prolog and
Constraint Handling Rules, Journal of Applied Logic, vol 7, 2009. pp. 341-362
Preliminary version: http://www.ruc.dk/~henning/publications/HypoReason2008_HC.pdf
Good Prolog systems with CHR
• SICStus Prolog (costs money, institution license?, 30 days test version)
http://www.sics.se/isl/sicstuswww/site/index.html
• SWI Prolog, free
http://www.swi-prolog.org/
Software by the speaker:
• HYPROLOG: http://www.ruc.dk/~henning/hyprolog/
• CHR Grammars: http://www.ruc.dk/~henning/chrg/
42
The end
43