Machine Learning Group
Learning for Semantic Parsing of
Natural Language
Raymond J. Mooney
Ruifang Ge, Rohit Kate, Yuk Wah Wong
John Zelle, Cynthia Thompson
Machine Learning Group
Department of Computer Sciences
University of Texas at Austin
July 31, 2005
University of Texas at Austin
Syntactic Natural Language Learning
• Most computational research in natural-language
learning has addressed “low-level” syntactic
processing.
–
–
–
–
Morphology (e.g. past-tense generation)
Part-of-speech tagging
Shallow syntactic parsing
Syntactic parsing
2
Semantic Natural Language Learning
• Learning for semantic analysis has been restricted
to relatively small, isolated tasks.
– Word sense disambiguation (e.g. SENSEVAL)
– Semantic role assignment (determining agent, patient,
instrument, etc., e.g. FrameNet, PropBank)
– Information extraction
3
Semantic Parsing
• A semantic parser maps a natural-language
sentence to a complete, detailed semantic
representation (logical form).
• For many applications, the desired output is
immediately executable by another program.
• Two application domains:
– CLang: RoboCup Coach Language
– GeoQuery: A Database Query Application
4
CLang: RoboCup Coach Language
• In RoboCup Coach competition teams compete to
coach simulated players
• The coaching instructions are given in a formal
language called CLang
Coach
If the ball is in our
penalty area, then all our
players except player 4
should stay in our half.
Simulated soccer field
Semantic Parsing
CLang
((bpos (penalty-area our))
(do (player-except our{4}) (pos (half our)))
5
GeoQuery: A Database Query Application
• Query application for U.S. geography database
containing about 800 facts [Zelle & Mooney, 1996]
How many
cities are
there in the
US?
User
Semantic Parsing
Query answer(A,
count(B, (city(B), loc(B, C),
const(C, countryid(USA))),A))
6
Learning Semantic Parsers
• Manually programming robust semantic parsers is
difficult due to the complexity of the task.
• Semantic parsers can be learned automatically from
sentences paired with their logical form.
NLLF
Training Exs
Natural
Language
Semantic-Parser
Learner
Semantic
Parser
Logical
Form
7
Engineering Motivation
• Most computational language-learning research
strives for broad coverage while sacrificing depth.
– “Scaling up by dumbing down”
• Realistic semantic parsing currently entails
domain dependence.
• Domain-dependent natural-language interfaces
have a large potential market.
• Learning makes developing specific applications
more tractable.
• Training corpora can be easily developed by
tagging existing corpora of formal statements
with natural-language glosses.
8
Cognitive Science Motivation
• Most natural-language learning methods require
supervised training data that is not available to a
child.
– General lack of negative feedback on grammar.
– No POS-tagged or treebank data.
• Assuming a child can infer the likely meaning of
an utterance from context, NLLF pairs are more
cognitively plausible training data.
9
Our Semantic-Parser Learners
• CHILL+WOLFIE (Zelle & Mooney, 1996; Thompson &
Mooney, 1999, 2003)
– Separates parser-learning and semantic-lexicon learning.
– Learns a deterministic parser using ILP techniques.
• SILT (Kate, Wong & Mooney, 2005)
– Learns symbolic transformation rules for mapping directly from NL to
LF.
• SCISSOR (Ge & Mooney, 2005)
– Integrates semantic interpretation into a statistical syntactic parser
(Collins model-3).
10
CHILL
(Zelle & Mooney, 1992-96)
• Semantic parser acquisition system using Inductive
Logic Programming (ILP) to induce a parser
written in Prolog.
• Starts with a parsing “shell” written in Prolog and
learns to control the operators of this parser to
produce the given I/O pairs.
• Requires a semantic lexicon, which for each word
gives one or more possible semantic
representations.
• Parser must disambiguate words, introduce proper
semantic representations for each, and then put
them together in the right way to produce a proper
representation of the sentence.
11
CHILL Example
• U.S. Geographical database
– Sample training pair
• Cuál es el capital del estado con la población más grande?
• answer(C, (capital(S,C), largest(P, (state(S), population(S,P)))))
– Sample semantic lexicon
•
•
•
•
•
cuál :
capital:
estado:
más grande:
población:
answer(_,_)
capital(_,_)
state(_)
largest(_,_)
population(_,_)
12
WOLFIE
(Thompson & Mooney, 1995-1999)
• Learns a semantic lexicon for CHILL from the
same corpus of semantically annotated sentences.
• Determines hypotheses for word meanings by
finding largest isomorphic common subgraphs
shared by meanings of sentences in which the
word appears.
• Uses a greedy-covering style algorithm to learn a
small lexicon sufficient to allow compositional
construction of the correct representation from the
words in a sentence.
13
WOLFIE + CHILL
Semantic Parser Acquisition
NLLF
Training Exs
WOLFIE
Lexicon Learner
Semantic
Lexicon
CHILL
Parser Learner
Natural
Language
Semantic
Parser
Logical
Form
14
Semantic Parsing using Transformation Rules
• SILT (Semantic Interpretation by Learning
Transformations)
• Uses pattern-based transformation rules which
map natural language phrases to formal language
constructs
• Transformation rules are repeatedly applied to the
sentence to construct its formal language
expression
15
Formal Language Grammar
NL:
If our player 4 has the ball, our player 4 should shoot.
CLang: ((bowner our {4}) (do our {4} shoot))
CLang Parse:
RULE
CONDITION
bowner
DIRECTIVE
TEAM
UNUM
our
4
do
TEAM
UNUM
ACTION
our
4
shoot
• Non-terminals: RULE, CONDITION, ACTION…
• Terminals: bowner, our, 4…
• Productions: RULE  CONDITION DIRECTIVE
DIRECTIVE  do TEAM UNUM ACTION
ACTION  shoot
16
Transformation Rule Representation
• Rule has two components: a natural language pattern and an
associated formal language template
• Two versions of SILT:
– String-based rules: used to convert natural language
word gap
sentence directly to formal language
String-pattern
TEAM
[1] ball tree to formal
– Tree-based
rules: used
to UNUM
converthas
syntactic
language
Template
CONDITION  (bowner TEAM {UNUM})
S
Treepattern
VP
NP
TEAM
Template
UNUM
NP
VBZ
has
DT
NN
the
ball
CONDITION  (bowner TEAM {UNUM})
17
Example of Semantic Parsing
If
our player 4 has the ball,
our player 4 should shoot.
our
player 4
shoot
TEAM  our
UNUM  4
ACTIONshoot
TEAM UNUM has [1] ball
If CONDITION, DIRECTIVE.
CONDITION  (bowner TEAM {UNUM})
RULE  (CONDITION DIRECTIVE)
TEAM UNUM should ACTION
DIRECTIVE  (do TEAM {UNUM} ACTION)
18
Example of Semantic Parsing
If TEAM
our player 4 has the ball, TEAM
our player 4 should shoot .
our
our
our
player 4
shoot
TEAM  our
UNUM  4
ACTIONshoot
TEAM UNUM has [1] ball
If CONDITION, DIRECTIVE.
CONDITION  (bowner TEAM {UNUM})
RULE  (CONDITION DIRECTIVE)
TEAM UNUM should ACTION
DIRECTIVE  (do TEAM {UNUM} ACTION)
19
Example of Semantic Parsing
If TEAM player 4 has the ball, TEAM player 4 should shoot .
our
our
our
player 4
shoot
TEAM  our
UNUM  4
ACTIONshoot
TEAM UNUM has [1] ball
If CONDITION, DIRECTIVE.
CONDITION  (bowner TEAM {UNUM})
RULE  (CONDITION DIRECTIVE)
TEAM UNUM should ACTION
DIRECTIVE  (do TEAM {UNUM} ACTION)
20
Example of Semantic Parsing
UNUM4 has the ball, TEAM player
UNUM4 should shoot .
If TEAM player
our
4
our
4
our
player 4
shoot
TEAM  our
UNUM  4
ACTIONshoot
TEAM UNUM has [1] ball
If CONDITION, DIRECTIVE.
CONDITION  (bowner TEAM {UNUM})
RULE  (CONDITION DIRECTIVE)
TEAM UNUM should ACTION
DIRECTIVE  (do TEAM {UNUM} ACTION)
21
Example of Semantic Parsing
If TEAM UNUM has the ball, TEAM
our
4
our
UNUM should shoot .
4
our
player 4
shoot
TEAM  our
UNUM  4
ACTIONshoot
TEAM UNUM has [1] ball
If CONDITION, DIRECTIVE.
CONDITION  (bowner TEAM {UNUM})
RULE  (CONDITION DIRECTIVE)
TEAM UNUM should ACTION
DIRECTIVE  (do TEAM {UNUM} ACTION)
22
Example of Semantic Parsing
If TEAM UNUM has the ball, TEAM
our
4
our
UNUM should ACTION
shoot .
4
our
player 4
shoot
TEAM  our
UNUM  4
ACTIONshoot
shoot
TEAM UNUM has [1] ball
If CONDITION, DIRECTIVE.
CONDITION  (bowner TEAM {UNUM})
RULE  (CONDITION DIRECTIVE)
TEAM UNUM should ACTION
DIRECTIVE  (do TEAM {UNUM} ACTION)
23
Example of Semantic Parsing
If TEAM UNUM has the ball, TEAM
our
4
our
UNUM should ACTION.
4
our
player 4
shoot
TEAM  our
UNUM  4
ACTIONshoot
shoot
TEAM UNUM has [1] ball
If CONDITION, DIRECTIVE.
CONDITION  (bowner TEAM {UNUM})
RULE  (CONDITION DIRECTIVE)
TEAM UNUM should ACTION
DIRECTIVE  (do TEAM {UNUM} ACTION)
24
Example of Semantic Parsing
CONDITION
If TEAM UNUM
has the ball , TEAM
our
(bowner
4 our {4})
our
UNUM should ACTION.
4
our
player 4
shoot
TEAM  our
UNUM  4
ACTIONshoot
shoot
TEAM UNUM has [1] ball
If CONDITION, DIRECTIVE.
CONDITION  (bowner TEAM {UNUM})
RULE  (CONDITION DIRECTIVE)
TEAM UNUM should ACTION
DIRECTIVE  (do TEAM {UNUM} ACTION)
25
Example of Semantic Parsing
If
, TEAM
CONDITION
(bowner our {4})
our
UNUM should ACTION.
4
our
player 4
shoot
TEAM  our
UNUM  4
ACTIONshoot
shoot
TEAM UNUM has [1] ball
If CONDITION, DIRECTIVE.
CONDITION  (bowner TEAM {UNUM})
RULE  (CONDITION DIRECTIVE)
TEAM UNUM should ACTION
DIRECTIVE  (do TEAM {UNUM} ACTION)
26
Example of Semantic Parsing
If
, TEAM
CONDITION
(bowner our {4})
our
UNUM
DIRECTIVE
should ACTION .
(do our
4 {4} shoot)
our
player 4
shoot
TEAM  our
UNUM  4
ACTIONshoot
shoot
TEAM UNUM has [1] ball
If CONDITION, DIRECTIVE.
CONDITION  (bowner TEAM {UNUM})
RULE  (CONDITION DIRECTIVE)
TEAM UNUM should ACTION
DIRECTIVE  (do TEAM {UNUM} ACTION)
27
Example of Semantic Parsing
If
,
CONDITION
.
DIRECTIVE
(bowner our {4})
(do our {4} shoot)
our
player 4
shoot
TEAM  our
UNUM  4
ACTIONshoot
TEAM UNUM has [1] ball
If CONDITION, DIRECTIVE.
CONDITION  (bowner TEAM {UNUM})
RULE  (CONDITION DIRECTIVE)
TEAM UNUM should ACTION
DIRECTIVE  (do TEAM {UNUM} ACTION)
28
Example of Semantic Parsing
If
CONDITION RULE ,
.
DIRECTIVE
(bowner
((bowner
ourour
{4})
{4}) (do our {4} shoot))
(do our {4} shoot)
our
player 4
shoot
TEAM  our
UNUM  4
ACTIONshoot
TEAM UNUM has [1] ball
If CONDITION, DIRECTIVE.
CONDITION  (bowner TEAM {UNUM})
RULE  (CONDITION DIRECTIVE)
TEAM UNUM should ACTION
DIRECTIVE  (do TEAM {UNUM} ACTION)
29
Learning Transformation Rules
• SILT induces rules from a corpora of NL sentences
paired with their formal representations
• Patterns are learned for each production by
bottom-up rule learning
• For every production:
– Call those sentences positives whose formal
representations’ parses use that production
– Call the remaining sentences negatives
30
Rule Learning for a Production
CONDITION
(bpos REGION)
positives
•
•
•
•
•
•
•
The ball is in REGION , our player 7 is in REGION and no
opponent is around our player 7 within 1.5 distance.
If the ball is in REGION and not in REGION then player 3
should intercept the ball.
During normal play if the ball is in the REGION then player 7 ,
9 and 11 should dribble the ball to the REGION .
When the play mode is normal and the ball is in the REGION
then our player 2 should pass the ball to the REGION .
All players except the goalie should pass the ball to REGION if
it is in RP18.
If the ball is inside rectangle ( -54 , -36 , 0 , 36 ) then player 10
should position itself at REGION with a ball attraction of
REGION .
Player 2 should pass the ball to REGION if it is in REGION .
negatives
•
•
•
•
•
•
•
•
If our player 6 has the ball then he should take a shot on goal.
If player 4 has the ball , it should pass the ball to player 2 or 10.
If the condition DR5C3 is true , then player 2 , 3 , 7 and 8
should pass the ball to player 3.
During play on , if players 6 , 7 or 8 is in REGION , they should
pass the ball to players 9 , 10 or 11.
If "Clear_Condition" , players 2 , 3 , 7 or 5 should clear the ball
REGION .
If it is before the kick off , after our goal or after the opponent's
goal , position player 3 at REGION .
If the condition MDR4C9 is met , then players 4-6 should pass
the ball to player 9.
If Pass_11 then player 11 should pass to player 9 and no one
else.
• SILT applies greedy-covering, bottom-up rule induction
method that repeatedly generalizes positives until they start
covering negatives
31
Generalization of String Patterns
ACTION  (pos REGION)
Pattern 1: Always position player UNUM at REGION .
Pattern 2: Whenever the ball is in REGION, position player
UNUM near the REGION .
Find the highest scoring common subsequence:
score (c)  length(c)   * ( sum of word gaps )
32
Generalization of String Patterns
ACTION  (pos REGION)
Pattern 1: Always position player UNUM at REGION .
Pattern 2: Whenever the ball is in REGION, position player
UNUM near the REGION .
Find the highest scoring common subsequence:
score (c)  length(c)   * ( sum of word gaps )
Generalization: position player UNUM [2] REGION .
33
Generalization of Tree Patterns
REGION  (penalty-area TEAM)
Pattern 1:
Pattern 2
NP
NP
NN NN
TEAM POS penalty box
NP
PRP$
NN
NN
TEAM penalty area
’s
Find common subgraphs.
34
Generalization of Tree Patterns
REGION  (penalty-area TEAM)
Pattern 1:
Pattern 2
NP
NP
NP
PRP$
NN NN
NN
NN
TEAM penalty area
TEAM POS penalty box
’s
NP
Find common subgraphs.
*
Generalization:
NN
NN
TEAM penalty
35
Rule Learning for a Production
CONDITION  (bpos REGION)
positives
negatives
•
•
•
•
•
•
•
The ball is in REGION , our player 7 is in REGION and no
opponent is around our player 7 within 1.5 distance.
If the ball is in REGION and not in REGION then player 3
should intercept the ball.
During normal play if the ball is in the REGION then player 7 ,
9 and 11 should dribble the ball to the REGION .
When the play mode is normal and the ball is in the REGION
then our player 2 should pass the ball to the REGION .
All players except the goalie should pass the ball to REGION if it
is in REGION.
If the ball is inside REGION then player 10 should position itself
at REGION with a ball attraction of REGION .
Player 2 should pass the ball to REGION if it is in REGION .
•
•
•
•
•
•
•
•
If our player 6 has the ball then he should take a shot on goal.
If player 4 has the ball , it should pass the ball to player 2 or 10.
If the condition DR5C3 is true , then player 2 , 3 , 7 and 8
should pass the ball to player 3.
During play on , if players 6 , 7 or 8 is in REGION , they should
pass the ball to players 9 , 10 or 11.
If "Clear_Condition" , players 2 , 3 , 7 or 5 should clear the ball
REGION .
If it is before the kick off , after our goal or after the opponent's
goal , position player 3 at REGION .
If the condition MDR4C9 is met , then players 4-6 should pass
the ball to player 9.
If Pass_11 then player 11 should pass to player 9 and no one
else.
Bottom-up Rule Learner
ball is [2] REGION
it is in REGION
CONDITION  (bpos REGION)
CONDITION  (bpos REGION)
36
Rule Learning for a Production
CONDITION  (bpos REGION)
positives
negatives
•
•
•
•
•
•
•
The
CONDITION , our player 7 is in REGION and no
opponent is around our player 7 within 1.5 distance.
If the CONDITION and not in REGION then player 3 should
intercept the ball.
During normal play if the CONDITION
then player 7 , 9
and 11 should dribble the ball to the REGION .
When the play mode is normal and the
CONDITION then
our player 2 should pass the ball to the REGION .
All players except the goalie should pass the ball to REGION if
CONDITION.
If the
CONDITION then player 10 should position itself at
REGION with a ball attraction of REGION .
Player 2 should pass the ball to REGION if CONDITION .
•
•
•
•
•
•
•
•
If our player 6 has the ball then he should take a shot on goal.
If player 4 has the ball , it should pass the ball to player 2 or 10.
If the condition DR5C3 is true , then player 2 , 3 , 7 and 8
should pass the ball to player 3.
During play on , if players 6 , 7 or 8 is in REGION , they should
pass the ball to players 9 , 10 or 11.
If "Clear_Condition" , players 2 , 3 , 7 or 5 should clear the ball
REGION .
If it is before the kick off , after our goal or after the opponent's
goal , position player 3 at REGION .
If the condition MDR4C9 is met , then players 4-6 should pass
the ball to player 9.
If Pass_11 then player 11 should pass to player 9 and no one
else.
Bottom-up Rule Learner
ball is [2] REGION
it is in REGION
CONDITION  (bpos REGION)
CONDITION  (bpos REGION)
37
Rule Learning for All Productions
• Transformation rules for productions should
cooperate globally to generate complete semantic
parses
• Redundantly cover every positive example by
β = 5 best rules
coverage
accuracy
pos(r )
• Find goodness
the subset
these
(r ) ofpos
(r ) *rules which best cooperate
pos(r )  neg (r )
to generate complete semantic
parses on the
training data
38
SCISSOR:
Semantic Composition that Integrates Syntax
and Semantics to get Optimal Representations
• Based on a fairly standard approach to compositional
semantics [Jurafsky and Martin, 2000]
• A statistical parser is used to generate a semantically
augmented parse tree (SAPT)
– Augment Collins’ head-driven model 2 (Bikel’s
S-bowner
implementation, 2004)
to incorporate semantic labels
NP-player
VP-bowner
• Translate
a complete
formal meaning
PRP$-team SAPT
NN-playerinto
CD-unum
VB-bowner
NP-null
representation
(MR) 2
our
player
has
DT-null
NN-null
MR: bowner(player(our,2))
the
ball
39
Overview of SCISSOR
NL Sentence
SAPT Training Examples
learner
Integrated Semantic Parser
SAPT
TRAINING
TESTING
ComposeMR
MR
40
ComposeMR
bowner
player
team
player
our
player
bowner
unum
2
null
bowner
has
null
null
the
ball
41
ComposeMR
bowner(_)
player(_,_)
team
player(_,_)
our
player
bowner(_)
unum
2
null
bowner(_)
has
null
null
the
ball
42
ComposeMR
bowner(player(our,2))
bowner(_)
bowner(_)
player(our,2)
player(_,_)
player(_,_)
team
player(_,_)
our
player
unum
2
null
bowner(_)
has
null
null
the
ball
player(team,unum)
bowner(player)
43
Collins’ Head-Driven Model 2
• A generative, lexicalized model
• Each node on the tree has a syntactic label, it is
also lexicalized with its head word
S(has)
NP(player)
VP(has)
NP(ball)
PRP$
NN
CD
VB
DT
NN
our
player
2
has
the
ball
44
Modeling Rule Productions as Markov Processes
S(has)
VP(has)
Ph(VP | S, has)
45
Modeling Rule Productions as Markov Processes
S(has)
VP(has)
{NP } { }
Ph(VP | S, has) ×
Plc({NP} | S, VP, has) × Prc({} | S, VP, has)
46
Modeling Rule Productions as Markov Processes
S(has)
NP(player)
VP(has)
{NP } { }
Ph(VP | S, has) ×
Plc({NP} | S, VP, has) × Prc({} | S, VP, has) ×
Pd(NP(player) | S, VP, has, LEFT, {NP})
47
Modeling Rule Productions as Markov Processes
S(has)
NP(player)
VP(has)
{} {}
Ph(VP | S, has) ×
Plc({NP} | S, VP, has) × Prc({} | S, VP, has) ×
Pd(NP(player) | S, VP, has, LEFT, {NP})
48
Modeling Rule Productions as Markov Processes
S(has)
STOP
NP(player)
VP(has)
{} {}
Ph(VP | S, has) ×
Plc({NP} | S, VP, has) × Prc({} | S, VP, has) ×
Pd(NP(player) | S, VP, has, LEFT, {NP}) × Pd(STOP | S, VP, has, LEFT, {})
49
Modeling Rule Productions as Markov Processes
S(has)
STOP
NP(player)
VP(has)
{} {}
STOP
Ph(VP | S, has) ×
Plc({NP} | S, VP, has) × Prc({} | S, VP, has) ×
Pd(NP(player) | S, VP, has, LEFT, {NP}) × Pd(STOP | S, VP, has, LEFT, {}) ×
Pd(STOP | S, VP, has, RIGHT, {})
50
Integrating Semantics into the Model
• Use the same Markov processes
• Add a semantic label to each node
• Add semantic subcat frames
– Give semantic subcategorization preferences
– bowner takes a player as its argument
S(has)
S-bowner(has
)
NP-player(player)
NP(player)
VP(has)
VP-bowner(has)
NP(ball)
NP-null(ball)
PRP$-team
PRP$ NN-player
NN
CD-unum
CD
VB
VB-bowner
DTDT-nullNN
NN-null
our
our
player
player
2 2
has has
the the ball ball
51
Adding Semantic Labels into the Model
S-bowner(has)
VP-bowner(has)
Ph(VP-bowner | S-bowner, has)
52
Adding Semantic Labels into the Model
S-bowner(has)
VP-bowner(has)
{NP}-{player} { }-{ }
Ph(VP-bowner | S-bowner, has) ×
Plc({NP}-{player} | S-bowner, VP-bowner, has) × Prc({}-{}| S-bowner, VP-bowner, has)
53
Adding Semantic Labels into the Model
S-bowner(has)
NP-player(player) VP-bowner(has)
{NP}-{player} { }-{ }
Ph(VP-bowner | S-bowner, has) ×
Plc({NP}-{player} | S-bowner, VP-bowner, has) × Prc({}-{}| S-bowner, VP-bowner, has) ×
Pd(NP-player(player) | S-bowner, VP-bowner, has, LEFT, {NP}-{player})
54
Adding Semantic Labels into the Model
S-bowner(has)
NP-player(player) VP-bowner(has)
{ }-{ } { }-{ }
Ph(VP-bowner | S-bowner, has) ×
Plc({NP}-{player} | S-bowner, VP-bowner, has) × Prc({}-{}| S-bowner, VP-bowner, has) ×
Pd(NP-player(player) | S-bowner, VP-bowner, has, LEFT, {NP}-{player})
55
Adding Semantic Labels into the Model
S-bowner(has)
STOP NP-player(player) VP-bowner(has)
{ }-{ } { }-{ }
Ph(VP-bowner | S-bowner, has) ×
Plc({NP}-{player} | S-bowner, VP-bowner, has) × Prc({}-{}| S-bowner, VP-bowner, has) ×
Pd(NP-player(player) | S-bowner, VP-bowner, has, LEFT, {NP}-{player}) ×
Pd(STOP | S-bowner, VP-bowner, has, LEFT, {}-{})
56
Adding Semantic Labels into the Model
S-bowner(has)
STOP NP-player(player) VP-bowner(has)
STOP
{ }-{ } { }-{ }
Ph(VP-bowner | S-bowner, has) ×
Plc({NP}-{player} | S-bowner, VP-bowner, has) × Prc({}-{}| S-bowner, VP-bowner, has) ×
Pd(NP-player(player) | S-bowner, VP-bowner, has, LEFT, {NP}-{player}) ×
Pd(STOP | S-bowner, VP-bowner, has, LEFT, {}-{}) ×
Pd(STOP | S-bowner, VP-bowner, has, RIGHT, {}-{})
57
Smoothing
• Each label in SAPT is the combination of a
syntactic label and a semantic label
• Increases data sparsity
• Use Bayes rule to break the parameters down
Ph(H | P, w)
= Ph(Hsyn, Hsem | P, w)
= Ph(Hsyn | P, w) × Ph(Hsem | P, w, Hsyn)
• Details in the paper
58
Experimental Corpora
• CLang
– 300 randomly selected pieces of coaching advice from
the log files of the 2003 RoboCup Coach Competition
– 22.52 words on average in NL sentences
– 14.24 tokens on average in formal expressions
• GeoQuery [Zelle & Mooney, 1996]
– 250 queries for the given U.S. geography database
– 6.87 words on average in NL sentences
– 5.32 tokens on average in formal expressions
59
Experimental Methodology
• Evaluated using standard 10-fold cross validation
• Syntactic parses for tree-based version of SILT were
obtained by training Collins’ parser [Bikel, 2004] on WSJ
treebank and gold-standard parses of training sentences
• Correctness
– CLang: output exactly matches the correct representation
– Geoquery: the resulting query retrieves the same answer as the
correct representation
• Metrics
Precision 
| Correct Completed Parses |
| Completed Parses |
Recall 
|Correct Completed Parses|
|Sentences|
60
Compared Systems
• CHILL [Tang & Mooney, 2001]
– Learn control rules for parsing based on Inductive Logic
Programming (ILP)
• SILT [Kate, Wong, and Mooney, 2005]
– Learns pattern-based transformation rules
– SILT-string
– SILT-tree
• SCISSOR [Ge and Mooney, 2005]
– Integrated syntactic/semantic statistical parser
• GEOBASE
– The original hand-built parser on Geoquery
61
Precision Learning Curve for CLang
62
Recall Learning Curve for CLang
63
F1 Measure Learning Curve for CLang
64
Precision Learning Curve for Geoquery
65
Recall Learning Curve for Geoquery
66
F1 Measure Learning Curve for Geoquery
67
Related Work
• [Zettlemoyer and Collins, 2005] use Combinatorial
Categorial Grammar (CCG) formalism to learn a statistical
semantic parser.
– Requires an initial CCG syntactic grammar of English.
• PRECISE [Popescu, 2003]
– Designed to work specially on NL database interfaces
– Not a learning system
• [Miller et al., 1996; Miller et al., 2000] use an approach
similar to SCISSOR to train a statistical parser integrating
syntax and semantics.
– Does not utilize subcat information
– Task: Information Extraction
68
Future Work
• Explore methods that can automatically generate
SAPTs to minimize the annotation effort for
SCISSOR.
• Use of methods from statistical MT (e.g. word
alignment) for semantic parsing.
• Learning semantic parsers just from sentences
paired with “perceptual context.”
69
Contextually Ambiguous
Sentence Meaning
• Sentences are uttered in complex situations
composed of numerous potential meanings.
• Could assume each sentence is annotated with
multiple possible meanings inferred from context
(Siskind, 1996).
– Multiple instance learning (Dietterich et al., 1997)
• Assuming context meaning is represented as a
semantic network, sentence meaning could be
assumed to be any connected subgraph of the
context.
70
Sample Ambiguous Context
“Juvenile caresses canine.”
Dog
isa
Chewing
HasColor
attr
obj
Black
Spot patient Petting
agent
agent
patient
Thing1
isa
Mary
isa
Child
HasColor
attr
obj
Blonde
Possess
patient
agent
isa
Thing2
Barbie
obj
part
isa
HasPart
Doll
Hair
Bone
71
Sample Ambiguous Context
“Juvenile caresses canine.”
Dog
isa
Chewing
HasColor
attr
obj
Black
Spot patient Petting
agent
agent
patient
Thing1
isa
Mary
isa
Child
HasColor
attr
obj
Blonde
Possess
patient
agent
isa
Thing2
Barbie
obj
part
isa
HasPart
Doll
Hair
Bone
72
Sample Ambiguous Context
“Juvenile caresses canine.”
Dog
isa
Chewing
HasColor
attr
obj
Black
Spot patient Petting
agent
agent
patient
Thing1
isa
Mary
isa
Child
HasColor
attr
obj
Blonde
Possess
patient
agent
isa
Thing2
Barbie
obj
part
isa
HasPart
Doll
Hair
Bone
73
Sample Ambiguous Context
“Juvenile caresses canine.”
Dog
isa
Chewing
HasColor
attr
obj
Black
Spot patient Petting
agent
agent
patient
Thing1
isa
Mary
isa
Child
HasColor
attr
obj
Blonde
Possess
patient
agent
isa
Thing2
Barbie
obj
part
isa
HasPart
Doll
Hair
Bone
74
Sample Ambiguous Context
“Juvenile caresses canine.”
Dog
isa
Chewing
HasColor
attr
obj
Black
Spot patient Petting
agent
agent
patient
Thing1
isa
Mary
isa
Child
HasColor
attr
obj
Blonde
Possess
patient
agent
isa
Thing2
Barbie
obj
part
isa
HasPart
Doll
Hair
Bone
75
Sample Ambiguous Context
“Juvenile caresses canine.”
Dog
isa
Chewing
HasColor
attr
obj
Black
Spot patient Petting
agent
agent
patient
Thing1
isa
Mary
isa
Child
HasColor
attr
obj
Blonde
Possess
patient
agent
isa
Thing2
Barbie
obj
part
isa
HasPart
Doll
Hair
Bone
76
Sample Ambiguous Context
“Juvenile caresses canine.”
Dog
isa
Chewing
HasColor
attr
obj
Black
Spot patient Petting
agent
agent
patient
Thing1
isa
Mary
isa
Child
HasColor
attr
obj
Blonde
Possess
patient
agent
isa
Thing2
Barbie
obj
part
isa
HasPart
Doll
Hair
Bone
77
Sample Ambiguous Context
“Juvenile caresses canine.”
Dog
isa
Chewing
HasColor
attr
obj
Black
Spot patient Petting
agent
agent
patient
Thing1
isa
Mary
isa
Child
HasColor
attr
obj
Blonde
Possess
patient
agent
isa
Thing2
Barbie
obj
part
isa
HasPart
Doll
Hair
Bone
78
Sample Ambiguous Context
“Juvenile caresses canine.”
Dog
isa
Chewing
HasColor
attr
obj
Black
Spot patient Petting
agent
agent
patient
Thing1
isa
Mary
isa
Child
HasColor
attr
obj
Blonde
Possess
patient
agent
isa
Thing2
Barbie
obj
part
isa
HasPart
Doll
Hair
Bone
79
Conclusions
• Learning semantic parsers is an important and
challenging problem in natural-language learning.
• We have obtained promising results on several
applications using a variety of approaches with
different strengths and weaknesses.
• Not many others have explored this problem, I
would encourage others to consider it.
• More and larger corpora are needed for training
and testing semantic parser induction.
80
Thank You!
Our papers on learning semantic parsers are on-line at:
http://www.cs.utexas.edu/~ml/publication/lsp.html
Our corpora can be downloaded from:
http://www.cs.utexas.edu/~ml/nldata.html
Questions??
81