Semantics and Semantic
Analysis
CSC 9010: Special Topics. Natural Language Processing.
Paula Matuszek, Mary-Angela Papalaskari
Spring, 2005
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
1
Meaning
• So far, we have focused on the structure of
language, not on what things mean
• We have been doing natural language
processing, but not natural language
understanding.
• So what is natural language understanding?
–
–
–
–
Answering an essay question on an exam?
Deciding what to order at a restaurant by reading a menu?
Realizing you’ve been insulted?
Appreciating a sonnet?
• As hard as answering "What is artificial
intelligence?"
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
2
Meaning, cont
• On the practical side, we want to “understand”
natural language because morphology- and
syntax-based methods will only take us so far in
some things:
– Machine translation
– Generation
– Question answering
• So we saw how we could use n-grams to
choose the more likely translation for a word
given other words. But n-grams can’t give us
the potential translations in the first place.
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
3
Semantics
• What kinds of things can we not do well with
the tools we have already looked at?
– Retrieve information in response to unconstrained
questions: e.g., travel planning
– Accurate translations
– Play the "chooser" side of 20 Questions
– Read a newspaper article and answer questions
about it
• These tasks require that we also consider
semantics: the meaning of our tokens and
their sequences
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
4
So What IS Meaning?
• From NLP viewpoint, meaning is a mapping
from linguistic forms to some kind of
representation of knowledge of the world
• It is interpreted within the framework of some
sort of action to be taken.
• Often we manipulate symbols all the way
through; the “meaning” is put in by the human
user. Translations, for instance.
• But not always – voice command systems, for
instance, may map from the representation into
actions.
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
5
Example
• Question: Is there a restaurant in King of
Prussia serving vegetarian dinners?
• From a restaurant database
– Lemon Grass is a sister restaurant to the one in West
Philadelphia serving traditional Thai dishes in addition to a
complete vegetarian Thai menu. The service and
atmosphere are quite pleasant. A welcome addition to the
King of Prussia area.
• What do we need to know to answer this
question from this text?
• Can we unambiguously answer it?
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
6
Example, continued
• Is there a restaurant in King of Prussia serving
vegetarian dinners? Yes.
• Lemon Grass is a sister restaurant to the one in West Philadelphia
serving traditional Thai dishes in addition to a complete
vegetarian Thai menu. The service and atmosphere are quite
pleasant. A welcome addition to the King of Prussia area.
• What do we need to know?
– Vegetarian Thai menu = vegetarian dinners
– Welcome addition to the KoP area = in KoP
• Can we unambiguously answer it?
– No. But we can be fairly certain.
• The only parse that makes sense
• Restaurants normally do serve the dinner meal.
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
7
Knowledge Representation for NLP
• So what representation?
• A useful KR needs to give us a way to encode the
knowledge relevant to our problem in a way which
allows us to use it. Any representation which does
this will work.
• How do we decide what we want to
represent?
– Entities, categories, events, time, aspect
– Predicates, relationships among entities,
arguments (constants, variables)
– And…quantifiers, operators (e.g. temporal)
• How much structure do we capture?
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
8
Structured Knowledge Representations
• Model-based representations reflect the
structure of the domain, and then
reason based on the model.
– Semantic Nets
– Frames
– Scripts
• Sometimes called associative networks
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
9
Basics of Associative Networks
• All include
– Concepts
– Various kinds of links between concepts
• “has-part” or aggregation
• “is-a” or specialization
• More specialized depending on domain
• Typically also include
– Inheritance
– Some kind of procedural attachment
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
10
Semantic Nets
labeled, directed graph
nodes represent objects, concepts, or situations
labels indicate the name
nodes can be instances (individual objects) or
classes (generic nodes)
links represent relationships
the relationships contain the structural information of
the knowledge to be represented
the label indicates the type of the relationship
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
11
Semantic Net Examples
HasA
Ship
HasA
Propulsion
Hull
Person
InstanceOf
Instance
Instance
Steamboat
Bob
give
recipient
agent
Mary
object
Candy
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
12
Generic/Individual
• Generic describes the idea--the “notion”
– static
• Individual or instance describes a real entity
– must conform to notion of generic
– dynamic
– “individuate” or “instantiate”
• A lot of NLP using semantic nets involves
instantiating generic nets based on a given piece of
text.
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
13
Individuation example
Person
give
recipient
agent
Person
object
Thing
Generic Representation
Process the sentence
“Bob gave Mary some candy”.
Bob
give
recipient
agent
Mary
object
Candy
Instantiation
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
14
Frames
Represent related knowledge about a subject
Frame has a title and a set of slots
 Title is what the frame “is” – the concept
 Slots capture relationships of the concept to other things
Typically can be organized hierarchically
 Most frame systems have an is-a slot
 allows the use of inheritance
Slots can contain all kinds of items
 Rules, facts, images, video, questions, hypotheses, other frames
 In NLP, typically capture relationships to other frames or entities
Slots can also have procedural attachments
 on creation, modification, removal of the slot value
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
15
Simple Frame Example
Slot Name
Restaurant
Cuisine
Price
Service
Atmosphere
Location
Web page
Filler
Lemon Grass
Thai, Vegetarian
Expensive
Excellent
Good
KoP
Ask Google.
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
16
Usage of Frames
• Most operations with frames do one of
two things:
• Fill slots
– Process a piece of text to identify an entity
for which we have a frame
– Fill as many slots as possible
• Use contents of slots
– Look up answers to questions
– Generate new text
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
[Rogers 1999]
17
Scripts
• Describe typical events or sequences
• Components are
– script variables (players, props)
– entry conditions
– transactions
– exit conditions
• Create instance by filling in variables
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
18
Restaurant Script Example
• generic template for restaurants
– different types
– default values
• script for a typical sequence of activities
at a restaurant
• Often has a frame behind it; script is
essentially instantiating the frame
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
[Rogers 1999]
19
EAT-AT-RESTAURANT Script
Restaurant Script
Props:
(Restaurant, Money, Food, Menu, Tables, Chairs)
Roles:
(Hungry-Persons, Wait-Persons, Chef-Persons)
Point-of-View:
Hungry-Persons
Time-of-Occurrence: (Times-of-Operation of Restaurant)
Place-of-Occurrence: (Location of Restaurant)
Event-Sequence:
first:
Enter-Restaurant Script
then:
if (Wait-To-Be-Seated-Sign or Reservations)
then Get-Maitre-d's-Attention Script
then:
Please-Be-Seated Script
then:
Order-Food-Script
then:
Eat-Food-Script unless (Long-Wait) when Exit-Restaurant-Angry
Script
then:
if (Food-Quality was better than Palatable)
then Compliments-To-The-Chef Script
then:
Pay-For-It-Script
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSCfinally:
Leave-Restaurant
Script
481/W03/Slides/3-Knowledge-Representation.ppt
and Hirschberg:
www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
[Rogers 1999]
20
Comments on Scripts
• Obviously takes a lot of time to develop
them initially.
– The script itself has much of the knowledge
– May be serious overkill for most NLP tasks
• We need this level of detail if we want to
include answers based on reasoning
like “Most restaurants do serve dinner.”
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
21
First Order Predicate Calculus (FOPC)
• Terms
– Constants: Lemon Grass
– Functions: LocationOf(Lemon Grass)
– Variables: x in LocationOf(x)
• Predicates: Relations that hold among
objects
– Serves(Lemon Grass,VegetarianFood)
• Logical Connectives: Permit compositionality
of meaning
– I only have $5 and I don’t have a lot of time
– Have(I,$5)
Have(I,LotofTime)

Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
22
FOPC Semantics
• Sentences in FOPC can be assigned
truth values True or False
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
23
Variables and Quantifiers
• Existential (): There exists
– A restaurant that serves Vegetarian food in KoP
– (x) Restaurant(x) Serves(x,VegetarianFoor)
Near(LocationOf(x), KoP)
• Universal (): For all


– All vegetarian restaurants serve vegetarian food
– (x) VegetarianRestaurant(x) ->


Serves(x,VegetarianFood)
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
24
FOPC Examples
• John gave Mary a book
– Previously: Give(John,Mary,book)
• Better:
– (x) Giving(x) Giver(John,x)
Givee(Mary,x) Given(book,x) 

• Full Definition of Give:
– (w,x,y,z) Giving(x) Giver(w,x)

Givee(z,x)
Given(y,x)



Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
25
Choosing a Representation
• We would like our representation to
support:
– Verifiability
– Unambiguous Representation
– Canonical Form
– Inference
– Expressiveness
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
26
Verifiability
• System can match input representation
against representations in knowledge
base. If it finds a match, it can return
Yes; Otherwise No.
• Does Lemon Grass serve vegetarian
food?
Serves(Lemon Grass,vegetarian food)
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
27
Unambiguous Representation
• Single linguistic input can have different
meaning representations
• Each representation unambiguously
characterizes one meaning.
• Example: small cars and motorcycles are
allowed
– car(x) & small(x) & motorcycle(y) & small(y) &
allowed(x) & allowed(y)
– car(x) & small(x) & motorcycle(y) & allowed(x) &
allowed(y)
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
28
Ambiguity and Vagueness
• An expression is ambiguous if, in a given
context, it can be disambiguated to have a
specific meaning, from a number of discrete,
possible meanings. E.g., bank (financial
institution) vs bank (river bank)
• An expression is vague, if it refers to a range
of a scalar variable, such that, even in a
specific context, it’s hard to specify the range
entirely. E.g., he’s tall, it’s warm, etc.
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
29
Representing Similar Concepts
• Distinct inputs could have the same meaning
–
–
–
–
Does Lemon Grass have vegetarian dishes?
Do they have vegetarian food at Lemon Grass?
Are vegetarian dishes served at Lemon Grass?
Does Lemon Grass serve vegetarian fare?
• Alternatives
– Four different semantic representations
– Store all possible meaning representations in KB
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
30
Canonical Form
• Solution: Inputs that mean same thing
have same meaning representation
• Is this easy? No!
– Vegetarian dishes, vegetarian food,
vegetarian fare
– Have, serve
• What to do?
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
31
How to Produce a Canonical Form
• Systematic Meaning Representations can be
derived from thesaurus
– food ___
– dish ___|____one overlapping meaning sense
– fare ___|
• We can systematically relate syntactic
constructions
– [S [NP Lemon Grass] serves
[NP
vegetarian dishes]]
– [S [NP vegetarian dishes] are served at [NP
Lemon Grass]]
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
32
Inference
• Consider a more complex request
– Can vegetarians eat at Lemon Grass?
– Vs: Does Lemon Grass serve vegetarian food?
• Why do these result in the same answer?
• Inference: Draw conclusions about truth of
propositions not explicitly stored in KB
• serve(Lemon Grass,VegetarianFood) =>
CanEat(Vegetarians,At Lemon Grass)
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
33
Non-Yes/No Questions
• Example: I'd like to find a
restaurant where I can get
vegetarian food.
• serve(x,VegetarianFood)
• Matching succeeds only if variable
x can be replaced by known object
in KB.
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
34
Meaning Structure of Language
• Human Languages
– Display a basic predicate-argument
structure
– Make use of variables
– Make use of quantifiers
– Display a partially compositional semantics
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
35
Compositional Semantics
• Assumption: The meaning of the whole is
comprised of the meaning of its parts
– George cooks. Dan eats. Dan is sick.
– Cook(George) Eat(Dan) Sick(Dan)
– If George cooks and Dan eats, Dan will get sick.
(Cook(George) ^ eat(Dan))  Sick(Dan)
• Meaning derives from
– The people and activities represented (predicates
and arguments, or, nouns and verbs)
– The way they are ordered and related: syntax of
the representation, which may also reflect the
syntax of the sentence
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
36
Syntax-Driven Semantics
S
NP
VP eat(Dan)
Nom V
N
Dan eats
• So….can we link syntactic structures to a
corresponding semantic representation to
produce the ‘meaning’ of a sentence in the
course of parsing it?
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
37
Specific vs. General-Purpose Rules
• We don’t want to have to specify for every
possible parse tree what semantic
representation it maps to
• We want to identify general mappings from
parse trees to semantic representations
• One approach:
– Augment the lexicon and the grammar
– Devise a mapping between rules of the grammar
and rules of semantic representation
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
38
Semantic Attachments
Extend each grammar rule with instructions on
how to map the components of the rule to a
semantic representation
S  NP VP {VP.sem(NP.sem)}
• Each semantic function defined in terms of
the semantic representation of choice
• Problem: how to define these functions and
how to specify their composition so we
always get the meaning representation we
want from our grammar?
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
39
A Simple Example
McDonald’s serves burgers.
• Associating constants with constituents
– ProperNoun  McDonald’s {McDonald’s}
– PlNoun  burgers {burgers}
• Defining functions to produce these from
input
– NP  ProperNoun {ProperNoun.sem}
– NP  PlNoun {PlNoun.sem}
– Assumption: meaning representations of children
are passed up to parents for non-branching
constuents
• Verbs are where the action is
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
40
• V  serves {E(e,x,y) Isa(e,Serving) ^
Server(e,x) ^ Served(e,y)} where e = event, x
= agent, y = patient
• Will every verb have its own distinct
representation?
– McDonald’s hires students.
– McDonald’s gave customers a bonus.
– Predicate(Agent, Patient, Beneficiary)
• Typically, divide verbs into classes based on
what roles participate in the verb.
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
41
Doing Compositional Semantics
• To incorporate semantics into grammar we
must
– Figure out right representation for each constituent
based on the parts of that constituent (e.g. Adj)
– Figure out the right representation for a category
of constituents based on other grammar rules,
making use of that constituent (e.g. V.sem)
• This gives us a set of function-like semantic
attachments incorporated into our CFG
– E.g. Nom  Adj Nom {x Nom.sem(x) ^
Isa(x,Adj.sem)}
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
42
What do we do with them?
• Alter a parser such as an Early-style parser
so when constituents (dot at the end of the
rule) are completed, the attached semantic is
function applied and meaning representation
is created and stored with state
• Or, let parser run to completion and then walk
through resulting tree running semantic
attachments from bottom-up
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
43
Option 1 (Integrated Semantic Analysis)
S  NP VP {VP.sem(NP.sem)}
– VP.sem has been stored in state representing VP
– NP.sem stored with the state for NP
– When rule completed, retrieve value of VP.sem and of
NP.sem, and apply VP.sem to NP.sem
– Store result in S.sem.
• As fragments of input parsed, semantic fragments
created
• Can be used to block ambiguous representations
• But: you also perform semantic analysis on
orphaned constituents that play no role in final parse
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
44
Option 2: Post-hoc Semantic Analysis
• Do semantics after syntactic parse
– Don’t do any semantic work on
constituents we will back out of – can be a
significant efficiency gain
– But can’t use semantics to aid the parse.
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
45
Compositional Semantics Summary
• Hypothesis: Principle of Compositionality
– Semantics of NL sentences and phrases can be
composed from the semantics of their subparts
• Rules can be derived which map syntactic
analysis to semantic representation (Rule-toRule Hypothesis)
• Parsing and mapping can be done during
syntactic analysis or as a separate phase.
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
46
Non-Compositional Language
• What do we do with language whose meaning isn’t
derived from the meanings of its parts
– Non-compositional modifiers: fake, former, local
– Metaphor:
• You’re the cream in my coffee. She’s the cream in George’s
coffee.
• The break-in was just the tip of the iceberg. This was only the
tip of Shirley’s iceberg.
– Idioms:
• The old man finally kicked the bucket. The old man finally
kicked the proverbial bucket.
– Deferred reference: The ham sandwich wants his check.
• Mix lexical items with special grammar rules? Take a
totally different approach?
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
47
Feature and Information Extraction
• We don’t always need a complete parse.
– We may be after only some specific information
– And we know what that information is
• Lemon Grass is a sister restaurant to the one in
West Philadelphia serving traditional Thai
dishes in addition to a complete vegetarian
Thai menu. The service and atmosphere are
quite pleasant. A welcome addition to the King
of Prussia area.
– We don’t care at all about the information in red
• Other semantic methods are more useful here.
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
48
Feature Extraction
• Group individual terms into more complex entities (which
in turn become tokens)
• Examples
– Dates, times, names, places
– URLs, HREFs and IMG tags
– Relationships like “X is president of Y”
• Can involve quite high-level features: language of a
sample of text
• Not usually a goal in itself
• In this case we are generating semantics for further use
by an application
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
49
Feature Extraction
• Human-meaningful features: Parse token
stream, applying pattern-matching rules
– general, broadly applicable features (dates)
– domain-specific features (chemical names)
– Can involve very sophisticated domain knowledge.
• Statistical features:
– document length, vocabulary used, sentence
length, document complexity, etc, etc
• Often first step in document-based analysis
such as classification
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
50
Information Extraction
• Retrieve some specific information which is
located somewhere in this set of documents.
• Don’t want all the information, just what we
have specified.
• Information may occur multiple times in the
text but we just need to find it once
• Often what is really wanted from a web
search, for instance.
• Often involves long sentences, complex
syntax, extended discourse, multiple writers
• May involve sentence fragments and other
non-grammatical text
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
51
Some Examples of Information Extraction
• Financial Information
– Who is the CEO/CTO of a company?
– What were the dividend payments for stocks I’m interested in for
the last five years?
• Biological Information
– Are there known inhibitors of enzymes in a pathway?
– Are there chromosomally located point mutations that result in a
described phenotype?
• Other typical questions
– Who is expert in some area?
– Is there a vegetarian restaurant in KoP?
– Can I get a flight to Chicago tomorrow?
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
52
Information Extraction -- How
• Create a model of information to be extracted
• Create knowledge base of rules for extraction
– concepts
– relations among concepts
• Process information applying a series of tools
• Always involves some level of domain modeling
and considerable natural language processing
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
53
Staged Approach
• Apply a series of tools to an input text
• At each stage an element of is identified
for possible use in the next level
• The end result is a set of relations
suitable for entry into a database
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
54
Cascaded Process
• Tokenize
• POS Tag
• Tag named entities: Names, dates,
locations
• Tag complex entities and relationships
– Grammatical, such as noun phrases
– Semantic, such as CEO
• Bind values to tags throughout.
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
55
Named Entities
• Named entity recognition…
– Labeling all the occurrences of named entities in a
text…
• People, organizations, lakes, bridges, hospitals,
mountains, etc…
• This is well-understood technology, readily
available and works well.
• Typically uses a combination of enumerated
lists (often called gazetteers) and regular
expressions.
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
56
Complex Entities and Relationships
• Uses Named Entities as components
• Pattern-matching rules specific to a given
domain
• May be multi-pass: One pass creates entities
which are used as part of later passes.
– EG: First pass locates CEOs, second pass
locates dates for CEOs being replaced.
• May involve syntactic analysis as well,
especially for things like negatives and
reference resolution
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
57
Why Does This Work?
• The problem is very constrained.
– Only looking for a small set of items that
can appear in a small set of roles
• We can ignore stuff we don’t care
about.
• BUT: creating the knowledge bases can
be very complex and time-consuming.
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
58
Summary
• Some NLP tasks require more than morphology and
syntax; they require some model of meaning: semantics
• Semantics can be modeled in a variety of different
formalisms. The goal of all of them is to map from the
sample of text being processed into some representation
relevant to a real-world task
• Complete generic semantic parses gives us a very rich
representation of text, but typically depend on the
assumption that language is compositional.
• Other tasks such as Information Extraction may not
require a detailed parse, but may depend on extensive
domain knowledge
• How you represent and work with semantics will always
depend on the NLP task you are trying to accomplish.
Some slides adapted from Dorr, www.umiacs.umd.edu/~christof/courses/cmsc723-fall04 , Kurfess: www.csc.calpoly.edu/~fkurfess/Courses/CSC481/W03/Slides/3-Knowledge-Representation.ppt and Hirschberg: www1.cs.columbia.edu/~julia/cs4705/syllabus.html
CSC 9010: Special Topics, Natural Language Processing. Spring, 2005. Matuszek & Papalaskari
59