Parsing of Context-Free Grammars
Bernd Kiefer
Bernd.Kiefer@dfki.de
Deutsches Forschungszentrum für künstliche Intelligenz
Introduction to CL - Context-Free Grammars – p.1/32
Limits of Regular Expressions
Assignment: Write a regular expression for fully bracketed
arithmetic expressions!
Answer: ?
Introduction to CL - Context-Free Grammars – p.2/32
Limits of Regular Expressions
Assignment: Write a regular expression for fully bracketed
arithmetic expressions!
Answer:
This is not possible!
Regular expressions can only count finite amounts of
brackets
Introduction to CL - Context-Free Grammars – p.2/32
Limits of Regular Expressions
Assignment: Write a regular expression for fully bracketed
arithmetic expressions!
Answer:
This is not possible!
Regular expressions can only count finite amounts of
brackets
We need a more powerful formal device: context-free
grammars
Introduction to CL - Context-Free Grammars – p.2/32
Limits of Regular Expressions
Assignment: Write a regular expression for fully bracketed
arithmetic expressions!
Answer:
This is not possible!
Regular expressions can only count finite amounts of
brackets
We need a more powerful formal device: context-free
grammars
Context-free grammars provide a (finite) inventory of
named brackets
All regular languages are also context-free, i.e.:
for every regular expression, there is a context-free
grammar that accepts / derives the same language
Introduction to CL - Context-Free Grammars – p.2/32
Context-Free Grammars
A context-free grammar (CFG) consists of:
The set of terminal symbols Σ = a, b, c, . . . (the words or
letters of the language)
The set of non-terminal symbols N = A, B, C, . . .
The startsymbol S ∈ N
The set of productions (rules) P , where
P ∋ r = A → α with α ∈ (Σ ∪ N )∗
(we use greek letters for strings of Σ ∪ N )
Example: A grammar for arithmetic expressions:
Σ = { int, +, *, (, ) } , N = {E} , S = E
P = { E → E +E, E → E *E, E → (E ), E → int }
Introduction to CL - Context-Free Grammars – p.3/32
The Language of a CFG
Given a CFG G, the language L(G) is defined as the set of
all strings that can be derived from S
Given a string α from (Σ ∪ N )∗ , derive a new string β :
Choose one of the nonterminals in α, say, A
Choose one of the productions with A on the left hand
side
Replace A in α with the right hand side (rhs) of the
production to get the derived string β
If α contains only symbols in Σ, then α ∈ L(G)
Example:
α = int*(E); choose E → E +E ; β = int*(E +E )
Introduction to CL - Context-Free Grammars – p.4/32
Derivations: Formally
A string α derives a string β , (α ⇒ β ) α, β ∈ (Σ ∪ N )∗ , if:
G
there are γ, δ, η ∈ (Σ ∪ N )∗ , A ∈ N such that
α = γ A δ , β = γ η δ and A −→ η ∈ P
We write α ⇒ β for a one-step derivation
G
∗
α ⇒ β is a many-step derivation: α ⇒ α0 ⇒ α1 . . . ⇒ β
G
G
G
G
∗
Language L(G) generated by G: L(G) = {s ∈ Σ∗ |S ⇒ s}
G
The task of a parser: find one (or all) derivation(s) of a
string in Σ∗ , given a CFG G
Introduction to CL - Context-Free Grammars – p.5/32
Derivations: Example
Σ = {john, girl, car, saw, walks, in, the, a}
N = {S, NP, VP, PP, D, N, V, P}


S →NP VP|N VP|N V|NP V




 VP→V NP|V N|VP PP
P =

NP→D N|NP PP|N PP




 PP→P NP|P N

N→john, girl, car 




V→saw, walks 
P→in
D→the, a






S⇒
G
john saw the girl in a car
Introduction to CL - Context-Free Grammars – p.6/32
Derivations: Example
Σ = {john, girl, car, saw, walks, in, the, a}
N = {S, NP, VP, PP, D, N, V, P}


S →NP VP|N VP|N V|NP V




 VP→V NP|V N|VP PP
P =

NP→D N|NP PP|N PP




 PP→P NP|P N

N→john, girl, car 




V→saw, walks 
P→in
D→the, a






S ⇒ N VP ⇒
G
G
john saw the girl in a car
Introduction to CL - Context-Free Grammars – p.6/32
Derivations: Example
Σ = {john, girl, car, saw, walks, in, the, a}
N = {S, NP, VP, PP, D, N, V, P}


S →NP VP|N VP|N V|NP V




 VP→V NP|V N|VP PP
P =

NP→D N|NP PP|N PP




 PP→P NP|P N

N→john, girl, car 




V→saw, walks 
P→in
D→the, a






S ⇒ N VP ⇒ john VP ⇒
G
G
G
john saw the girl in a car
Introduction to CL - Context-Free Grammars – p.6/32
Derivations: Example
Σ = {john, girl, car, saw, walks, in, the, a}
N = {S, NP, VP, PP, D, N, V, P}


S →NP VP|N VP|N V|NP V




 VP→V NP|V N|VP PP
P =

NP→D N|NP PP|N PP




 PP→P NP|P N

N→john, girl, car 




V→saw, walks 
P→in
D→the, a






S ⇒ N VP ⇒ john VP ⇒ john V NP ⇒
G
G
G
G
john saw the girl in a car
Introduction to CL - Context-Free Grammars – p.6/32
Derivations: Example
Σ = {john, girl, car, saw, walks, in, the, a}
N = {S, NP, VP, PP, D, N, V, P}


S →NP VP|N VP|N V|NP V




 VP→V NP|V N|VP PP
P =

NP→D N|NP PP|N PP




 PP→P NP|P N

N→john, girl, car 




V→saw, walks 
P→in
D→the, a






S ⇒ N VP ⇒ john VP ⇒ john V NP ⇒ john saw NP ⇒
G
G
G
G
G
john saw the girl in a car
Introduction to CL - Context-Free Grammars – p.6/32
Derivations: Example
Σ = {john, girl, car, saw, walks, in, the, a}
N = {S, NP, VP, PP, D, N, V, P}


S →NP VP|N VP|N V|NP V




 VP→V NP|V N|VP PP
P =

NP→D N|NP PP|N PP




 PP→P NP|P N

N→john, girl, car 




V→saw, walks 
P→in
D→the, a






S ⇒ N VP ⇒ john VP ⇒ john V NP ⇒ john saw NP ⇒
G
G
G
G
G
john saw NP PP ⇒
G
john saw the girl in a car
Introduction to CL - Context-Free Grammars – p.6/32
Derivations: Example
Σ = {john, girl, car, saw, walks, in, the, a}
N = {S, NP, VP, PP, D, N, V, P}


S →NP VP|N VP|N V|NP V




 VP→V NP|V N|VP PP
P =

NP→D N|NP PP|N PP




 PP→P NP|P N

N→john, girl, car 




V→saw, walks 
P→in






D→the, a
S ⇒ N VP ⇒ john VP ⇒ john V NP ⇒ john saw NP ⇒
G
G
G
G
G
john saw NP PP ⇒ john saw D N PP ⇒ john saw the N PP ⇒ john saw the girl
G
G
G
PP ⇒ john saw the girl P NP ⇒ john saw the girl in NP ⇒ john saw the girl in D
G
G
G
N ⇒ john saw the girl in a N ⇒
G
G
john saw the girl in a car
Introduction to CL - Context-Free Grammars – p.6/32
Derivation (Parse) Trees
S
S
NP
N
V
john
saw
NP
N
VP
NP
NP
PP
D
N
P
the
girl
in
john
NP
VP
VP
V
saw
D
N
a
car
PP
NP
P
D
N
the
girl
in
NP
D N
a
car
Encodes many possible derivations
PP node in the example can be attached to two nodes: the
grammar is ambigous
CF Parsers/Recognizers differ in the way the derivation
trees are build
Introduction to CL - Context-Free Grammars – p.7/32
Context-free Recognition
Task: given s ∈ Σ∗ and G, is s ∈ L(G) ?
Two ways to go:
start with the start symbol S and try to derive s by
systematic application of the productions:
top down recognition (goal driven)
start with the string s and try to reduce it to the start
symbol:
bottom up recognition (data driven)
Introduction to CL - Context-Free Grammars – p.8/32
Recursive Descent Parsing
Idea: Recursively compute all expansions of a nonterminal at
some input position
E → S +S, E → S *S,
expand(S , 0)
S → (E ), S → int
S
(
int
+
(
int
*
int
)
)
Introduction to CL - Context-Free Grammars – p.9/32
Recursive Descent Parsing
Idea: Recursively compute all expansions of a nonterminal at
some input position
E → S +S, E → S *S,
expand(S , 0)
S → (E ), S → int
S
(
E
(
int
)
+
(
int
*
int
)
)
Introduction to CL - Context-Free Grammars – p.9/32
Recursive Descent Parsing
Idea: Recursively compute all expansions of a nonterminal at
some input position
E → S +S, E → S *S,
expand(E , 1)
S → (E ), S → int
S
(
(
E
S
int
)
*
+
S
(
int
*
int
)
)
Introduction to CL - Context-Free Grammars – p.9/32
Recursive Descent Parsing
Idea: Recursively compute all expansions of a nonterminal at
some input position
E → S +S, E → S *S,
expand(S , 1)
S → (E ), S → int
S
(
E
S
int
(
int
)
*
+
S
(
int
*
int
)
)
Introduction to CL - Context-Free Grammars – p.9/32
Recursive Descent Parsing
Idea: Recursively compute all expansions of a nonterminal at
some input position
E → S +S, E → S *S,
expand(E , 1)
S → (E ), S → int
S
(
E
S
int
S
(
int
)
*
S
+
S
+
(
int
*
int
)
)
Introduction to CL - Context-Free Grammars – p.9/32
Recursive Descent Parsing
Idea: Recursively compute all expansions of a nonterminal at
some input position
E → S +S, E → S *S,
expand(S , 1)
S → (E ), S → int
S
(
E
S
int
S
)
*
S
+
S
int
(
int
+
(
int
*
int
)
)
Introduction to CL - Context-Free Grammars – p.9/32
Recursive Descent Parsing
Idea: Recursively compute all expansions of a nonterminal at
some input position
E → S +S, E → S *S,
expand(S , 1)
S → (E ), S → int
S
(
E
S
int
S
)
*
S
+
S
int
(
int
+
(
int
*
int
)
)
Introduction to CL - Context-Free Grammars – p.9/32
Recursive Descent Parsing
Idea: Recursively compute all expansions of a nonterminal at
some input position
E → S +S, E → S *S,
expand(S , 3)
S → (E ), S → int
S
(
E
S
int
S
)
*
S
+
S
int
(
int
+
(
E
(
int
)
*
int
)
)
Introduction to CL - Context-Free Grammars – p.9/32
Recursive Descent Parsing
Idea: Recursively compute all expansions of a nonterminal at
some input position
E → S +S, E → S *S,
expand(E , 4)
S → (E ), S → int
S
(
E
S
int
S
)
*
S
+
S
(
int
(
int
+
(
E
S
)
*
int
*
S
int
)
)
Introduction to CL - Context-Free Grammars – p.9/32
Recursive Descent Parsing
Idea: Recursively compute all expansions of a nonterminal at
some input position
E → S +S, E → S *S,
expand(S , 4)
S → (E ), S → int
S
(
E
S
int
S
)
*
S
+
S
(
int
E
S
int
(
int
+
(
int
)
*
*
S
int
)
)
Introduction to CL - Context-Free Grammars – p.9/32
Recursive Descent Parsing
Idea: Recursively compute all expansions of a nonterminal at
some input position
E → S +S, E → S *S,
expand(S , 6)
S → (E ), S → int
S
(
E
S
int
S
)
*
S
+
S
(
int
E
S
int
(
int
+
(
int
)
*
*
S
( E )
int
)
)
Introduction to CL - Context-Free Grammars – p.9/32
Recursive Descent Parsing
Idea: Recursively compute all expansions of a nonterminal at
some input position
E → S +S, E → S *S,
expand(S , 6)
S → (E ), S → int
S
(
E
S
int
S
)
*
S
+
S
(
int
E
S
int
(
int
+
(
int
)
*
*
S
( E )
int
int
)
)
Introduction to CL - Context-Free Grammars – p.9/32
Recursive Descent Parsing
Idea: Recursively compute all expansions of a nonterminal at
some input position
E → S +S, E → S *S,
expand(S , 3)
S → (E ), S → int
S
(
E
S
int
S
)
*
S
+
S
(
int
E
S
int
(
int
+
(
int
)
*
*
S
( E )
int
int
)
)
Introduction to CL - Context-Free Grammars – p.9/32
Recursive Descent Parsing
Idea: Recursively compute all expansions of a nonterminal at
some input position
E → S +S, E → S *S,
expand(S , 0)
S → (E ), S → int
S
(
E
S
int
S
)
*
S
+
S
(
int
E
S
int
(
int
+
(
int
)
*
*
S
( E )
int
int
)
)
Introduction to CL - Context-Free Grammars – p.9/32
Recursive Descent Parsing II
expand returns a set of possible end positions
During expansion of one production
Keep track of a set of intermediate positions for the
already expanded part
Expand the next terminal or nonterminal from every
intermediate position
If the set of possible intermediate position gets empty,
the production fails
When a production was successfully completed, the
intermediate set is added to the set of end positions
May run into an infinite loop with left-recursive grammars,
∗
i.e. grammars where A ⇒ Aβ for some A
G
Introduction to CL - Context-Free Grammars – p.10/32
Top Down Parsing: Optimizations
For every nonterminal, compute the set of terminals that
can occur in the first position
Expand only those nonterminals / productions that are
compatible with the current input symbol
Avoid performing duplicate calls with identical
nonterminal/position pair: memoize previous calls
use a data structure whose index are tuples consisting
of the function arguments
the result of the lookup is the result of a previous call
with the same arguments (if available)
The memoized method has to be strictly functional for
this to work
Introduction to CL - Context-Free Grammars – p.11/32
Epsilon Reduction
A CFG may contain productions of the form A → ǫ
Construct a CFG G′ with the same language as G and at
most one epsilon production: S → ǫ
for all nonterminals A with A → ǫ ∈ P :
mark A as ǫ-deriving and add it to the set Q
while Q is not empty, remove a nonterminal X from Q:
for all Y → α X β ∈ P , with α or β not empty, add Y → α β to P ′
for all Y → X, if Y is not marked as ǫ-deriving:
mark Y as ǫ-deriving and add it to Q
if S is ǫ-deriving, add S → ǫ to P ′
add all non-ǫ productions of P to P ′
Introduction to CL - Context-Free Grammars – p.12/32
Chomsky Normal Form
A context-free grammar is in Chomsky Normal Form if:
(i) it is ǫ-free,
(ii) all productions have one of two forms:
• A → a with a ∈ Σ
• A → B C with B, C ∈ N
Every CFG can be transformed into a CNF grammar with
the same language
Drawback: The original structure of the parse trees must
be reconstructed, if necessary
The original and transformed grammar are said to be
weakly equivalent
Introduction to CL - Context-Free Grammars – p.13/32
Chomsky Normal Form II
Convert an arbitrary CFG into CNF:
Introduce new nonterminals and productions Aa → a
Replace all occurences of a by Aa
Eliminate unary productions A → B :
add productions where A is replaced by B in the right
hand sides
Replace productions with more than two symbols on
the right hand side by a sequence of productions:
A → R1 R2 . . . Rn ⇒
A → R1 A(1) , A(1) → R2 A(2) , · · ·
A(n−2) → Rn−1 Rn
Introduction to CL - Context-Free Grammars – p.14/32
Chart Parsing
First algorithm independently developed by Cocke,
Younger and Kasami (late 60s)
Given a string w of length n, use an n × n table to store
subderivations (hence chart or tabular parsing)
Works for all kinds of grammars: left/right recursive,
ambiguous
Storing subderivations avoids duplicate computation: an
instance of dynamic programming
polynomial space and time bounds, although an
exponential number of parse trees may be encoded!
Introduction to CL - Context-Free Grammars – p.15/32
CYK Parsing
Input: G in Chomsky normal form, input string w1 , . . . , wn
Systematically explore all possible sub-derivations
bottom-up
Use an n × n array C such that
∗
If nonterminal A is stored in C(i, k): A ⇒ wi+1 , . . . , wk
G
Maintain a second table B, such that if j ∈ B(i, k): ex.
A → B C ∈ P, A ∈ C(i, k), B ∈ C(i, j) and C ∈ C(j, k)
B enables us to extract the parse trees
Implement C and B as three-dimensional boolean arrays of
size n × n × |N | and n × n × n, respectively
Introduction to CL - Context-Free Grammars – p.16/32
CYK – The Original
For i = 1 to n
For each Rj −→ ai , set C(i − 1, i, j) = true
For l = 2 to n
– Length of new constituent
For i = 0 to n − l
– Start of new constituent
For m = 1 to l − 1
– Length of first subconstituent
For each production Ra −→ Rb Rc
If C(i, i + m, b) and C(i + m, i + l, c) then
set C(i, i + l, a) = true
set B(i, i + l, i + m) = true
If C(1, n, S) is true, w ∈ L(G)
Introduction to CL - Context-Free Grammars – p.17/32
CYK Chart Example
C
0
john
1
saw
2
the
B
3
girl
4
in
5
a
6
car
7
Introduction to CL - Context-Free Grammars – p.18/32
CYK Chart Example
N
C
V
B
D
N
P
D
N
N
0
john
V
1
saw
D
2
the
N
3
girl
P
4
in
N
D
5
a
6
car
7
Introduction to CL - Context-Free Grammars – p.18/32
CYK Chart Example
N S
V
C
1
B
3
D NP
N
P
6
D NP
N
S
N
0
john
NP
V
1
saw
D
2
the
N
3
girl
NP
P
4
in
D
5
a
6
N
car
7
Introduction to CL - Context-Free Grammars – p.18/32
CYK Chart Example
N S
V
C
VP
D NP
N
P
1
B
2
3
5
6
PP
D NP
N
VP
S
N
0
john
NP
V
1
PP
saw
D
2
the
N
3
girl
NP
P
4
in
D
5
a
6
N
car
7
Introduction to CL - Context-Free Grammars – p.18/32
CYK Chart Example
N S
V
S
VP
D NP
N
C
1
B
NP
PP
P
D NP
N
4
5
6
S
NP
VP
S
N
0
john
PP
NP
V
1
1
2
3
saw
D
2
the
N
3
girl
NP
P
4
in
D
5
a
6
N
car
7
Introduction to CL - Context-Free Grammars – p.18/32
CYK Chart Example
N S
V
S
VP
D NP
N
C
1
B
NP
NP
PP
P
D NP
N
NP
VP
S
0
john
1
PP
NP
V
saw
D
2
4
4
5
6
NP
S
N
1
2
3
the
N
3
girl
NP
P
4
in
D
5
a
6
N
car
7
Introduction to CL - Context-Free Grammars – p.18/32
CYK Chart Example
N S
V
S
VP
D NP
N
VP
NP
NP
PP
P
D NP
N
C
1
B
1
2
3
2,4
4
4
5
6
VP
NP
S
NP
VP
S
N
0
john
NP
V
1
PP
saw
D
2
the
N
3
girl
NP
P
4
in
D
5
a
6
N
car
7
Introduction to CL - Context-Free Grammars – p.18/32
CYK Chart Example
N S
V
S
VP
D NP
N
S
VP
NP
NP
PP
P
D NP
N
C
1
B
1
2,4
4
4
5
6
1
2
3
S
VP
NP
S
NP
VP
S
N
0
john
NP
V
1
PP
saw
D
2
the
N
3
girl
NP
P
4
in
D
5
a
6
N
car
7
Introduction to CL - Context-Free Grammars – p.18/32
Example Chart
S →NP VP|N VP|N V|NP V
VP→V NP|V N|VP PP
NP→D N|NP PP|N PP
PP→P NP|P N
N →john, girl, car
V →saw, walks
P →in
D →the, a
1 2 3 4 5 6
7
0 N S
S
S
1
V
VP
VP(2)
2
D NP
NP
3
N
NP
4
P
PP
5
D NP
6
N
0 john 1 saw 2 the 3 girl 4 in 5 a 6 car 7
Introduction to CL - Context-Free Grammars – p.19/32
Example Chart
S →NP VP|N VP|N V|NP V
VP→V NP|V N|VP PP
NP→D N|NP PP|N PP
PP→P NP|P N
N →john, girl, car
V →saw, walks
P →in
D →the, a
1 2 3 4 5 6
7
0 N S
S
S
1
V
VP
VP(2)
2
D NP
NP
3
N
NP
4
P
PP
5
D NP
6
N
0 john 1 saw 2 the 3 girl 4 in 5 a 6 car 7
Introduction to CL - Context-Free Grammars – p.19/32
Example Chart
S →NP VP|N VP|N V|NP V
VP→V NP|V N|VP PP
NP→D N|NP PP|N PP
PP→P NP|P N
N →john, girl, car
V →saw, walks
P →in
D →the, a
1 2 3 4 5 6
7
0 N S
S
S
1
V
VP
VP(2)
2
D NP
NP
3
N
NP
4
P
PP
5
D NP
6
N
0 john 1 saw 2 the 3 girl 4 in 5 a 6 car 7
Introduction to CL - Context-Free Grammars – p.19/32
Example Chart
S →NP VP|N VP|N V|NP V
VP→V NP|V N|VP PP
NP→D N|NP PP|N PP
PP→P NP|P N
N →john, girl, car
V →saw, walks
P →in
D →the, a
1 2 3 4 5 6
7
0 N S
S
S
1
V
VP
VP(2)
2
D NP
NP
3
N
NP
4
P
PP
5
D NP
6
N
0 john 1 saw 2 the 3 girl 4 in 5 a 6 car 7
Introduction to CL - Context-Free Grammars – p.19/32
Encoding Ambiguities
Σ = {john, girl, car, sees, in, the, a}


S →NP VP,




 VP→V|V NP|V NP PP
P =

NP→N|D N|N PP|D N PP




 PP→P NP
S
N = {S, NP, VP, PP, D, N, V, P}

N→john, girl, car 





V→sees

P→in





D→the, a
VP
NP
N
V
john
sees
NP
D
N
the
girl
PP
P
in
NP
D
N
a
car
Introduction to CL - Context-Free Grammars – p.20/32
Describing Chart Parsing Algorithms
Parsing algorithm should use original grammars for comparability
Introduction to CL - Context-Free Grammars – p.21/32
Describing Chart Parsing Algorithms
Parsing algorithm should use original grammars for
comparability
They are described using chart items, which consist of
a symbol, derived from the grammar
a start and end position from 0, . . . , n
Introduction to CL - Context-Free Grammars – p.21/32
Describing Chart Parsing Algorithms
Parsing algorithm should use original grammars for
comparability
They are described using chart items, which consist of
a symbol, derived from the grammar
a start and end position from 0, . . . , n
The symbol of complete items is one of Σ ∪ N
Introduction to CL - Context-Free Grammars – p.21/32
Describing Chart Parsing Algorithms
Parsing algorithm should use original grammars for
comparability
They are described using chart items, which consist of
a symbol, derived from the grammar
a start and end position from 0, . . . , n
The symbol of complete items is one of Σ ∪ N
Incomplete chart items encode partially filled rules
the symbol is a pair (r, i) of rule and dot position if
P ∋ r : A → αβ with |α| = i
write alternatively: A → α rβ
Introduction to CL - Context-Free Grammars – p.21/32
Bottom-Up Chart Parsing
How, when and which chart items are created or combined
characterizes a parsing algorithm or parsing strategy
First: A modified variant of Cocke-Younger-Kasami (CYK)
algorithm
Prerequisites: CFG G, input string w = a1 , . . . , an
Data Structures:
A n + 1 × n + 1 chart C , where each cell contains a set of
(complete or incomplete) chart items
A set of chart items A (those must still be treated in
some way)
Initialization: add all (ai , i − 1, i) to A and Ci−1,i
Introduction to CL - Context-Free Grammars – p.22/32
Bottom-Up Parsing Algorithm
while A not empty
take an (X, i, j) from A and remove it
if X ∈ Σ ∪ N
for P ∋ r ≡ A → Xα do
check and add(A → X rα, i, j)
for k ∈ 0, . . . , i − 1 do
for all (A → β rXα, k, i) ∈ C do
check and add(A → βX rα, k, j)
else
– incomplete item: X ≡ A → β rY α
for k ∈ j + 1, . . . , n do
if (Y, j, k) ∈ C check and add(A → βY rα, i, k)
check and add(X
≡ A → α rβ, i, j) ≡
if β = ǫ then if (A, i, j) 6∈ C add (A, i, j) to A and C endif
else if (A → α rβ, i, j) 6∈ C add (A → α rβ, i, j) to A and C endif
Introduction to CL - Context-Free Grammars – p.23/32
Bottom-Up: The Details
How to implement A and C efficiently?
Implementation of the (n + 1)2 sets in C :
Introduction to CL - Context-Free Grammars – p.24/32
Bottom-Up: The Details
How to implement A and C efficiently?
Implementation of the (n + 1)2 sets in C :
Operations: add single element, contains element
Introduction to CL - Context-Free Grammars – p.24/32
Bottom-Up: The Details
How to implement A and C efficiently?
Implementation of the (n + 1)2 sets in C :
Operations: add single element, contains element
P
bit vector of size |G| := |Σ| + |N | + P ∋r:A→α |Aα|
Introduction to CL - Context-Free Grammars – p.24/32
Bottom-Up: The Details
How to implement A and C efficiently?
Implementation of the (n + 1)2 sets in C :
Operations: add single element, contains element
P
bit vector of size |G| := |Σ| + |N | + P ∋r:A→α |Aα|
Implementation of the set A:
Introduction to CL - Context-Free Grammars – p.24/32
Bottom-Up: The Details
How to implement A and C efficiently?
Implementation of the (n + 1)2 sets in C :
Operations: add single element, contains element
P
bit vector of size |G| := |Σ| + |N | + P ∋r:A→α |Aα|
Implementation of the set A:
Operations: add , get and remove some element
Introduction to CL - Context-Free Grammars – p.24/32
Bottom-Up: The Details
How to implement A and C efficiently?
Implementation of the (n + 1)2 sets in C :
Operations: add single element, contains element
P
bit vector of size |G| := |Σ| + |N | + P ∋r:A→α |Aα|
Implementation of the set A:
Operations: add , get and remove some element
(priority) queue, stack
A is called agenda and can be used to implement
search strategies
Keep terminal items separate from the chart for space and
time efficiency
Introduction to CL - Context-Free Grammars – p.24/32
Runtime of CYK Algorithm
check and add(X
≡ A → α rβ, i, j) ≡
if β = ǫ then if (A, i, j) 6∈ C add (A, i, j) to A and C endif
else if (A → α rβ, i, j) 6∈ C add (A → α rβ, i, j) to A and C endif
Introduction to CL - Context-Free Grammars – p.25/32
Runtime of CYK Algorithm
check and add(X
≡ A → α rβ, i, j) ≡
all operations O(1)
if β = ǫ then if (A, i, j) 6∈ C add (A, i, j) to A and C endif
else if (A → α rβ, i, j) 6∈ C add (A → α rβ, i, j) to A and C endif
Introduction to CL - Context-Free Grammars – p.25/32
Runtime of CYK Algorithm
check and add(X
≡ A → α rβ, i, j) ≡
all operations O(1)
if β = ǫ then if (A, i, j) 6∈ C add (A, i, j) to A and C endif
else if (A → α rβ, i, j) 6∈ C add (A → α rβ, i, j) to A and C endif
while A not empty
take an (X, i, j) from A and remove it
if X ∈ Σ ∪ N
for P ∋ r ≡ A → Xα do
check and add(A → X rα, i, j)
for k ∈ 0, . . . , i − 1 do
for all (A → β rXα, k, i) ∈ C do
check and add(A → βX rα, k, j)
else
– incomplete item: X ≡ A → β rY α
for k ∈ j + 1, . . . , n do
if (Y, j, k) ∈ C check and add(A → βY rα, i, k)
Introduction to CL - Context-Free Grammars – p.25/32
Runtime of CYK Algorithm
check and add(X
≡ A → α rβ, i, j) ≡
all operations O(1)
if β = ǫ then if (A, i, j) 6∈ C add (A, i, j) to A and C endif
else if (A → α rβ, i, j) 6∈ C add (A → α rβ, i, j) to A and C endif
while A not empty
take an (X, i, j) from A and remove it
max. |G| × (n + 1)2
if X ∈ Σ ∪ N
for P ∋ r ≡ A → Xα do
check and add(A → X rα, i, j)
for k ∈ 0, . . . , i − 1 do
for all (A → β rXα, k, i) ∈ C do
check and add(A → βX rα, k, j)
else
– incomplete item: X ≡ A → β rY α
for k ∈ j + 1, . . . , n do
if (Y, j, k) ∈ C check and add(A → βY rα, i, k)
Introduction to CL - Context-Free Grammars – p.25/32
Runtime of CYK Algorithm
check and add(X
≡ A → α rβ, i, j) ≡
all operations O(1)
if β = ǫ then if (A, i, j) 6∈ C add (A, i, j) to A and C endif
else if (A → α rβ, i, j) 6∈ C add (A → α rβ, i, j) to A and C endif
while A not empty
take an (X, i, j) from A and remove it
max. |G| × (n + 1)2
if X ∈ Σ ∪ N
for P ∋ r ≡ A → Xα do
max. |G|
check and add(A → X rα, i, j)
for k ∈ 0, . . . , i − 1 do
for all (A → β rXα, k, i) ∈ C do
check and add(A → βX rα, k, j)
else
– incomplete item: X ≡ A → β rY α
for k ∈ j + 1, . . . , n do
if (Y, j, k) ∈ C check and add(A → βY rα, i, k)
Introduction to CL - Context-Free Grammars – p.25/32
Runtime of CYK Algorithm
check and add(X
≡ A → α rβ, i, j) ≡
all operations O(1)
if β = ǫ then if (A, i, j) 6∈ C add (A, i, j) to A and C endif
else if (A → α rβ, i, j) 6∈ C add (A → α rβ, i, j) to A and C endif
while A not empty
take an (X, i, j) from A and remove it
max. |G| × (n + 1)2
if X ∈ Σ ∪ N
for P ∋ r ≡ A → Xα do
max. |G|
check and add(A → X rα, i, j)
max. n times
for k ∈ 0, . . . , i − 1 do
for all (A → β rXα, k, i) ∈ C do
check and add(A → βX rα, k, j)
else
– incomplete item: X ≡ A → β rY α
for k ∈ j + 1, . . . , n do
if (Y, j, k) ∈ C check and add(A → βY rα, i, k)
Introduction to CL - Context-Free Grammars – p.25/32
Runtime of CYK Algorithm
check and add(X
≡ A → α rβ, i, j) ≡
all operations O(1)
if β = ǫ then if (A, i, j) 6∈ C add (A, i, j) to A and C endif
else if (A → α rβ, i, j) 6∈ C add (A → α rβ, i, j) to A and C endif
while A not empty
take an (X, i, j) from A and remove it
max. |G| × (n + 1)2
if X ∈ Σ ∪ N
for P ∋ r ≡ A → Xα do
max. |G|
check and add(A → X rα, i, j)
max. n times
for k ∈ 0, . . . , i − 1 do
for all (A → β rXα, k, i) ∈ C do
max. |G| times
check and add(A → βX rα, k, j)
else
– incomplete item: X ≡ A → β rY α
for k ∈ j + 1, . . . , n do
if (Y, j, k) ∈ C check and add(A → βY rα, i, k)
Introduction to CL - Context-Free Grammars – p.25/32
Runtime of CYK Algorithm
check and add(X
≡ A → α rβ, i, j) ≡
all operations O(1)
if β = ǫ then if (A, i, j) 6∈ C add (A, i, j) to A and C endif
else if (A → α rβ, i, j) 6∈ C add (A → α rβ, i, j) to A and C endif
while A not empty
take an (X, i, j) from A and remove it
max. |G| × (n + 1)2
if X ∈ Σ ∪ N
for P ∋ r ≡ A → Xα do
max. |G|
check and add(A → X rα, i, j)
max. n times
for k ∈ 0, . . . , i − 1 do
for all (A → β rXα, k, i) ∈ C do
max. |G| times
check and add(A → βX rα, k, j)
else
– incomplete item: X ≡ A → β rY α
max. n times
for k ∈ j + 1, . . . , n do
if (Y, j, k) ∈ C check and add(A → βY rα, i, k)
Introduction to CL - Context-Free Grammars – p.25/32
CYK – Summary
Polynomial complexity: O(|G|2 n3 )
Explores all possible sub-derivations
Advantageous for robust parsing:
Extract the biggest/best chunks for ungrammatical input
That a derivation must start at S is not used at all
Average time is near or equal to the worst case
May lead to poor performance in practice
Two main steps:
if (X, i, j) ∈ C, X ∈ Σ ∪ N and A → Xα ∈ P :
add (A → X rα, i, j) to C
if (A → β rY α, i, j) ∈ C and (Y, j, k) ∈ C :
add (A → βY rα, i, k) to C
Introduction to CL - Context-Free Grammars – p.26/32
Predictive Bottom-Up: Earley Parsing
Described by J. Earley (1970): Predict Top-Down and
Complete Bottom-Up
Initialize by adding the terminal items and (S → α, 0, 0) for
all S → rα ∈ P
Three main operations:
Prediction if (A → β rY α, i, j) ∈ C , for every Y → γ ∈ P
add (Y → rγ, j, j) to C
Scanning if (A → β raj+1 α, i, j) ∈ C ,
add (A → βaj+1 rα, i, j + 1) to C
Completion if (Y, i, j), Y ∈ N and (A → β rY α, j, k) ∈ C ,
add (A → βY rα, i, k)
Introduction to CL - Context-Free Grammars – p.27/32
Earley Parsing Example
john
saw
the
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.28/32
Earley Parsing Example
john
S→ rNP VP
saw
the
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.28/32
Earley Parsing Example
john
S→ rNP VP
NP→ rD N PP
NP→ rN
NP→ rD N
NP→ rN PP
saw
the
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.28/32
Earley Parsing Example
john
S→ rNP VP
NP→ rD N PP
NP→ rN
NP→ rD N
NP→ rN PP
N→ rjohn
N→ rgirl
N→ rcar
D→ rthe
D→ ra
saw
the
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.28/32
Earley Parsing Example
N
john
S→ rNP VP
NP→ rD N PP
NP→ rN
NP→ rD N
NP→ rN PP
N→ rjohn
N→ rgirl
N→ rcar
D→ rthe
D→ ra
saw
the
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.28/32
Earley Parsing Example
NP
N
john
S→ rNP VP
NP→ rD N PP
NP→ rN
NP→ rD N
NP→ rN PP
N→ rjohn
N→ rgirl
N→ rcar
D→ rthe
D→ ra
saw
the
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.28/32
Earley Parsing Example
NP→N rPP
NP
N
saw
john
S→ rNP VP PP→ rP NP
NP→ rD N PP
NP→ rN
NP→ rD N
NP→ rN PP
N→ rjohn
N→ rgirl
N→ rcar
D→ rthe
D→ ra
the
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.28/32
Earley Parsing Example
S→NP rVP
NP→N rPP
NP
N
saw
john
S→ rNP VP PP→ rP NP
NP→ rD N PP VP→ rV
NP→ rN VP→ rV NP PP
NP→ rD N VP→ rV NP
NP→ rN PP V→ rsaw
N→ rjohn
N→ rgirl
N→ rcar
D→ rthe
D→ ra
the
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.28/32
Earley Parsing Example
S→NP rVP
NP→N rPP
NP
VP
N
V
saw
john
S→ rNP VP PP→ rP NP
NP→ rD N PP VP→ rV
NP→ rN VP→ rV NP PP
NP→ rD N VP→ rV NP
NP→ rN PP V→ rsaw
N→ rjohn
N→ rgirl
N→ rcar
D→ rthe
D→ ra
the
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.28/32
Earley Parsing Example
S
S→NP rVP VP→V rNP PP
NP→N rPP VP→V rNP
NP
VP
N
V
saw
john
S→ rNP VP PP→ rP NP
NP→ rD N PP VP→ rV
NP→ rN VP→ rV NP PP
NP→ rD N VP→ rV NP
NP→ rN PP V→ rsaw
N→ rjohn
N→ rgirl
N→ rcar
D→ rthe
D→ ra
the
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.28/32
Earley Parsing Example
S
S→NP rVP VP→V rNP PP
NP→N rPP VP→V rNP
NP
VP
N
V
saw
john
the
S→ rNP VP PP→ rP NP NP→ rD N PP
NP→ rD N
NP→ rD N PP VP→ rV
NP→ rN VP→ rV NP PP NP→ rN
NP→ rD N VP→ rV NP NP→ rN PP
N→ rjohn
NP→ rN PP V→ rsaw
N→ rgirl
N→ rjohn
N→ rgirl
N→ rcar
D→ rthe
N→ rcar
D→ ra
D→ rthe
D→ ra
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.28/32
Earley Parsing Example
S
S→NP rVP VP→V rNP PP
NP→N rPP VP→V rNP NP→D rN PP
NP
VP
NP→D rN
N
V
D
saw
john
the
S→ rNP VP PP→ rP NP NP→ rD N PP
NP→ rD N
NP→ rD N PP VP→ rV
NP→ rN VP→ rV NP PP NP→ rN
NP→ rD N VP→ rV NP NP→ rN PP
N→ rjohn
NP→ rN PP V→ rsaw
N→ rgirl
N→ rjohn
N→ rgirl
N→ rcar
D→ rthe
N→ rcar
D→ ra
D→ rthe
D→ ra
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.28/32
Earley Parsing Example
S
S→NP rVP VP→V rNP PP
NP→N rPP VP→V rNP NP→D rN PP
NP
VP
NP→D rN
N
V
D
saw
john
girl
the
r
S→ rNP VP PP→ rP NP NP→ rD N PP N→ john
N→ rgirl
NP→ rD N
NP→ rD N PP VP→ rV
N→ rcar
NP→ rN VP→ rV NP PP NP→ rN
NP→ rD N VP→ rV NP NP→ rN PP
N→ rjohn
NP→ rN PP V→ rsaw
N→ rgirl
N→ rjohn
N→ rgirl
N→ rcar
D→ rthe
N→ rcar
D→ ra
D→ rthe
D→ ra
in
a
car
Introduction to CL - Context-Free Grammars – p.28/32
Earley Parsing Example
S
S→NP rVP VP→V rNP PP
NP
NP→N rPP VP→V rNP NP→D rN PP
NP
VP
NP→D rN
N
V
D
N
saw
john
girl
the
r
S→ rNP VP PP→ rP NP NP→ rD N PP N→ john
N→ rgirl
NP→ rD N
NP→ rD N PP VP→ rV
N→ rcar
NP→ rN VP→ rV NP PP NP→ rN
NP→ rD N VP→ rV NP NP→ rN PP
N→ rjohn
NP→ rN PP V→ rsaw
N→ rgirl
N→ rjohn
N→ rgirl
N→ rcar
D→ rthe
N→ rcar
D→ ra
D→ rthe
D→ ra
in
a
car
Introduction to CL - Context-Free Grammars – p.28/32
Earley Parsing Example
S
VP→V NP rPP
VP
S
NP→D N rPP
S→NP rVP VP→V rNP PP
NP
NP→N rPP VP→V rNP NP→D rN PP
NP
VP
NP→D rN
N
V
D
N
saw
john
girl
the
r
S→ rNP VP PP→ rP NP NP→ rD N PP N→ john
N→ rgirl
NP→ rD N
NP→ rD N PP VP→ rV
N→ rcar
NP→ rN VP→ rV NP PP NP→ rN
NP→ rD N VP→ rV NP NP→ rN PP
N→ rjohn
NP→ rN PP V→ rsaw
N→ rgirl
N→ rjohn
N→ rgirl
N→ rcar
D→ rthe
N→ rcar
D→ ra
D→ rthe
D→ ra
in
a
car
Introduction to CL - Context-Free Grammars – p.28/32
Earley Parsing Example
S
VP→V NP rPP
VP
S
NP→D N rPP
S→NP rVP VP→V rNP PP
NP
NP→N rPP VP→V rNP NP→D rN PP
NP
VP
NP→D rN
PP→P rNP
N
V
D
N
P
saw
john
girl
in
the
r
P→ rin
S→ rNP VP PP→ rP NP NP→ rD N PP N→ john
N→ rgirl PP→ rP NP
NP→ rD N
NP→ rD N PP VP→ rV
N→ rcar
NP→ rN VP→ rV NP PP NP→ rN
NP→ rD N VP→ rV NP NP→ rN PP
N→ rjohn
NP→ rN PP V→ rsaw
N→ rgirl
N→ rjohn
N→ rgirl
N→ rcar
D→ rthe
N→ rcar
D→ ra
D→ rthe
D→ ra
a
car
Introduction to CL - Context-Free Grammars – p.28/32
Earley Parsing Example
S
VP→V NP rPP
VP
S
NP→D N rPP
S→NP rVP VP→V rNP PP
NP
NP→N rPP VP→V rNP NP→D rN PP
NP
VP
NP→D rN
PP→P rNP
N
V
D
N
P
saw
a
john
girl
in
the
r
P→ rin NP→ rD N PP
S→ rNP VP PP→ rP NP NP→ rD N PP N→ john
N→ rgirl PP→ rP NP NP→ rD N
NP→ rD N
NP→ rD N PP VP→ rV
N→ rcar
NP→ rN VP→ rV NP PP NP→ rN
NP→ rN PP
NP→ rN
NP→ rD N VP→ rV NP NP→ rN PP
N→ rjohn
N→ rjohn
NP→ rN PP V→ rsaw
N→ rgirl
N→ rjohn
N→ rgirl
N→ rgirl
N→ rcar
N→ rcar
D→ rthe
N→ rcar
D→ rthe
D→ ra
D→ rthe
D→ ra
D→ ra
car
Introduction to CL - Context-Free Grammars – p.28/32
Earley Parsing Example
S
VP→V NP rPP
VP
S
NP→D N rPP
S→NP rVP VP→V rNP PP
NP
NP
NP→N rPP VP→V rNP NP→D rN PP
NP→D rN PP
NP
VP
NP→D rN
PP→P rNP NP→D rN
N
V
D
N
P
D
N
saw
a
car
john
girl
in
the
r
r
P→ rin NP→ rD N PP N→ john
S→ rNP VP PP→ rP NP NP→ rD N PP N→ john
N→ rgirl PP→ rP NP NP→ rD N
N→ rgirl
NP→ rD N
NP→ rD N PP VP→ rV
N→ rcar
NP→ rN VP→ rV NP PP NP→ rN
NP→ rN PP N→ rcar
NP→ rN
NP→ rD N VP→ rV NP NP→ rN PP
N→ rjohn
N→ rjohn
NP→ rN PP V→ rsaw
N→ rgirl
N→ rjohn
N→ rgirl
N→ rgirl
N→ rcar
N→ rcar
D→ rthe
N→ rcar
D→ rthe
D→ ra
D→ rthe
D→ ra
D→ ra
Introduction to CL - Context-Free Grammars – p.28/32
Earley Parsing Example
S
VP→V NP rPP
VP
S
NP→D N rPP
NP→D N rPP
S→NP rVP VP→V rNP PP
NP
NP
NP→N rPP VP→V rNP NP→D rN PP
NP→D rN PP
NP
VP
NP→D rN
PP→P rNP NP→D rN
N
V
D
N
P
D
N
saw
a
car
john
girl
in
the
r
r
P→ rin NP→ rD N PP N→ john P→ rin
S→ rNP VP PP→ rP NP NP→ rD N PP N→ john
N→ rgirl PP→ rP NP NP→ rD N
N→ rgirl PP→ rP NP
NP→ rD N
NP→ rD N PP VP→ rV
N→ rcar
NP→ rN VP→ rV NP PP NP→ rN
NP→ rN PP N→ rcar
NP→ rN
NP→ rD N VP→ rV NP NP→ rN PP
N→ rjohn
N→ rjohn
NP→ rN PP V→ rsaw
N→ rgirl
N→ rjohn
N→ rgirl
N→ rgirl
N→ rcar
N→ rcar
D→ rthe
N→ rcar
D→ rthe
D→ ra
D→ rthe
D→ ra
D→ ra
Introduction to CL - Context-Free Grammars – p.28/32
Earley Parsing Example
S
VP→V NP rPP
VP
PP
r
S
NP→D N PP
NP→D N rPP
S→NP rVP VP→V rNP PP
NP
NP
NP→N rPP VP→V rNP NP→D rN PP
NP→D rN PP
NP
VP
NP→D rN
PP→P rNP NP→D rN
N
V
D
N
P
D
N
saw
a
car
john
girl
in
the
r
r
P→ rin NP→ rD N PP N→ john P→ rin
S→ rNP VP PP→ rP NP NP→ rD N PP N→ john
N→ rgirl PP→ rP NP NP→ rD N
N→ rgirl PP→ rP NP
NP→ rD N
NP→ rD N PP VP→ rV
N→ rcar
NP→ rN VP→ rV NP PP NP→ rN
NP→ rN PP N→ rcar
NP→ rN
NP→ rD N VP→ rV NP NP→ rN PP
N→ rjohn
N→ rjohn
NP→ rN PP V→ rsaw
N→ rgirl
N→ rjohn
N→ rgirl
N→ rgirl
N→ rcar
N→ rcar
D→ rthe
N→ rcar
D→ rthe
D→ ra
D→ rthe
D→ ra
D→ ra
Introduction to CL - Context-Free Grammars – p.28/32
Earley Parsing Example
NP
S
VP→V NP rPP
VP
PP
r
S
NP→D N PP
NP→D N rPP
S→NP rVP VP→V rNP PP
NP
NP
NP→N rPP VP→V rNP NP→D rN PP
NP→D rN PP
NP
VP
NP→D rN
PP→P rNP NP→D rN
N
V
D
N
P
D
N
saw
a
car
john
girl
in
the
r
r
P→ rin NP→ rD N PP N→ john P→ rin
S→ rNP VP PP→ rP NP NP→ rD N PP N→ john
N→ rgirl PP→ rP NP NP→ rD N
N→ rgirl PP→ rP NP
NP→ rD N
NP→ rD N PP VP→ rV
N→ rcar
NP→ rN VP→ rV NP PP NP→ rN
NP→ rN PP N→ rcar
NP→ rN
NP→ rD N VP→ rV NP NP→ rN PP
N→ rjohn
N→ rjohn
NP→ rN PP V→ rsaw
N→ rgirl
N→ rjohn
N→ rgirl
N→ rgirl
N→ rcar
N→ rcar
D→ rthe
N→ rcar
D→ rthe
D→ ra
D→ rthe
D→ ra
D→ ra
Introduction to CL - Context-Free Grammars – p.28/32
Earley Parsing Example
VP
NP
S
VP→V NP rPP
VP
PP
r
S
NP→D N PP
NP→D N rPP
S→NP rVP VP→V rNP PP
NP
NP
NP→N rPP VP→V rNP NP→D rN PP
NP→D rN PP
NP
VP
NP→D rN
PP→P rNP NP→D rN
N
V
D
N
P
D
N
saw
a
car
john
girl
in
the
r
r
P→ rin NP→ rD N PP N→ john P→ rin
S→ rNP VP PP→ rP NP NP→ rD N PP N→ john
N→ rgirl PP→ rP NP NP→ rD N
N→ rgirl PP→ rP NP
NP→ rD N
NP→ rD N PP VP→ rV
N→ rcar
NP→ rN VP→ rV NP PP NP→ rN
NP→ rN PP N→ rcar
NP→ rN
NP→ rD N VP→ rV NP NP→ rN PP
N→ rjohn
N→ rjohn
NP→ rN PP V→ rsaw
N→ rgirl
N→ rjohn
N→ rgirl
N→ rgirl
N→ rcar
N→ rcar
D→ rthe
N→ rcar
D→ rthe
D→ ra
D→ rthe
D→ ra
D→ ra
Introduction to CL - Context-Free Grammars – p.28/32
Earley Parsing Example
S
VP
NP
S
VP→V NP rPP
VP
PP
r
S
NP→D N PP
NP→D N rPP
S→NP rVP VP→V rNP PP
NP
NP
NP→N rPP VP→V rNP NP→D rN PP
NP→D rN PP
NP
VP
NP→D rN
PP→P rNP NP→D rN
N
V
D
N
P
D
N
saw
a
car
john
girl
in
the
r
r
P→ rin NP→ rD N PP N→ john P→ rin
S→ rNP VP PP→ rP NP NP→ rD N PP N→ john
N→ rgirl PP→ rP NP NP→ rD N
N→ rgirl PP→ rP NP
NP→ rD N
NP→ rD N PP VP→ rV
N→ rcar
NP→ rN VP→ rV NP PP NP→ rN
NP→ rN PP N→ rcar
NP→ rN
NP→ rD N VP→ rV NP NP→ rN PP
N→ rjohn
N→ rjohn
NP→ rN PP V→ rsaw
N→ rgirl
N→ rjohn
N→ rgirl
N→ rgirl
N→ rcar
N→ rcar
D→ rthe
N→ rcar
D→ rthe
D→ ra
D→ rthe
D→ ra
D→ ra
Introduction to CL - Context-Free Grammars – p.28/32
Earley Parsing – Summary
The number of useless items is reduced
Superior runtime for unambiguous grammars: O(n2 )
Valid prefix property
Not all sub-derivations are computed
Introduction to CL - Context-Free Grammars – p.29/32
Left Corner Parsing
Observation: Earley parsing predicts items that can not
succeed
Idea: predict only items that can also be derived from the
leftmost terminal item
Formalization: left-corner relation
A >l B ⇐⇒ ∃β : A → Bβ ∈ P, B ∈ Σ ∪ N
A >∗l is the transitive closure of >l
Reformulation of the prediction step:
If (A → β rY α, i, j) and (B, j, k) ∈ C , with B ∈ Σ ∪ N add
(C → B rγ, j, k) if C → Bγ ∈ P and Y >∗l C
This formulation also avoids the zero-length predictions
with the dot in initial position
Introduction to CL - Context-Free Grammars – p.30/32
Left-Corner Example
john
saw
the
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.31/32
Left-Corner Example
N
john
saw
the
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.31/32
Left-Corner Example
NP→N rPP
N
john
saw
the
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.31/32
Left-Corner Example
NP
NP→N rPP
N
john
saw
the
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.31/32
Left-Corner Example
S→NP rVP
NP
NP→N rPP
N
john
saw
the
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.31/32
Left-Corner Example
S→NP rVP
NP
NP→N rPP
N
V
john
saw
the
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.31/32
Left-Corner Example
S→NP rVP
NP
NP→N rPP
VP
N
V
john
saw
the
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.31/32
Left-Corner Example
S→NP rVP
NP
VP→V rNP
NP→N rPP
VP
N
V
john
saw
the
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.31/32
Left-Corner Example
S→NP rVP VP→V rNP PP
NP
VP→V rNP
NP→N rPP
VP
N
V
john
saw
the
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.31/32
Left-Corner Example
S
S→NP rVP VP→V rNP PP
NP
VP→V rNP
NP→N rPP
VP
N
V
john
saw
the
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.31/32
Left-Corner Example
S
S→NP rVP VP→V rNP PP
NP
VP→V rNP
NP→N rPP
VP
N
V
D
john
saw
the
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.31/32
Left-Corner Example
S
S→NP rVP VP→V rNP PP
NP
VP→V rNP
NP→N rPP
VP
NP→D rN
N
V
D
john
saw
the
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.31/32
Left-Corner Example
S
S→NP rVP VP→V rNP PP
NP
VP→V rNP NP→D rN PP
NP→N rPP
VP
NP→D rN
N
V
D
john
saw
the
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.31/32
Left-Corner Example
S
S→NP rVP VP→V rNP PP
NP
VP→V rNP NP→D rN PP
NP→N rPP
VP
NP→D rN
N
V
D
N
john
saw
the
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.31/32
Left-Corner Example
S
S→NP rVP VP→V rNP PP
NP
NP
VP→V rNP NP→D rN PP
NP→N rPP
VP
NP→D rN
N
V
D
N
john
saw
the
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.31/32
Left-Corner Example
S
NP→D N rPP
S→NP rVP VP→V rNP PP
NP
NP
VP→V rNP NP→D rN PP
NP→N rPP
VP
NP→D rN
N
V
D
N
john
saw
the
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.31/32
Left-Corner Example
VP
S
NP→D N rPP
S→NP rVP VP→V rNP PP
NP
NP
VP→V rNP NP→D rN PP
NP→N rPP
VP
NP→D rN
N
V
D
N
john
saw
the
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.31/32
Left-Corner Example
VP→V NP rPP
VP
S
NP→D N rPP
S→NP rVP VP→V rNP PP
NP
NP
VP→V rNP NP→D rN PP
NP→N rPP
VP
NP→D rN
N
V
D
N
john
saw
the
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.31/32
Left-Corner Example
S
VP→V NP rPP
VP
S
NP→D N rPP
S→NP rVP VP→V rNP PP
NP
NP
VP→V rNP NP→D rN PP
NP→N rPP
VP
NP→D rN
N
V
D
N
john
saw
the
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.31/32
Left-Corner Example
S
VP→V NP rPP
VP
S
NP→D N rPP
S→NP rVP VP→V rNP PP
NP
NP
VP→V rNP NP→D rN PP
NP→N rPP
VP
NP→D rN
N
V
D
N
P
john
saw
the
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.31/32
Left-Corner Example
S
VP→V NP rPP
VP
S
NP→D N rPP
S→NP rVP VP→V rNP PP
NP
NP
VP→V rNP NP→D rN PP
NP→N rPP
VP
NP→D rN
PP→P rNP
N
V
D
N
P
john
saw
the
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.31/32
Left-Corner Example
S
VP→V NP rPP
VP
S
NP→D N rPP
S→NP rVP VP→V rNP PP
NP
NP
VP→V rNP NP→D rN PP
NP→N rPP
VP
NP→D rN
PP→P rNP
N
V
D
N
P
D
john
saw
the
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.31/32
Left-Corner Example
S
VP→V NP rPP
VP
S
NP→D N rPP
S→NP rVP VP→V rNP PP
NP
NP
VP→V rNP NP→D rN PP
NP→N rPP
VP
NP→D rN
N
V
D
john
saw
the
PP→P rNP
NP→D rN
N
P
D
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.31/32
Left-Corner Example
S
VP→V NP rPP
VP
S
NP→D N rPP
S→NP rVP VP→V rNP PP
NP
NP
VP→V rNP NP→D rN PP
NP→N rPP
VP
NP→D rN
N
V
D
john
saw
the
NP→D rN PP
PP→P rNP
NP→D rN
N
P
D
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.31/32
Left-Corner Example
S
VP→V NP rPP
VP
S
NP→D N rPP
S→NP rVP VP→V rNP PP
NP
NP
VP→V rNP NP→D rN PP
NP→N rPP
VP
NP→D rN
N
V
D
john
saw
the
NP→D rN PP
PP→P rNP
NP→D rN
N
P
D
N
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.31/32
Left-Corner Example
S
VP→V NP rPP
VP
S
NP→D N rPP
S→NP rVP VP→V rNP PP
NP
NP
NP
VP→V rNP NP→D rN PP
NP→N rPP
VP
NP→D rN
N
V
D
john
saw
the
NP→D rN PP
PP→P rNP
NP→D rN
N
P
D
N
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.31/32
Left-Corner Example
S
VP→V NP rPP
VP
S
NP→D N rPP
NP→D N rPP
NP
NP
S→NP rVP VP→V rNP PP
NP
VP→V rNP NP→D rN PP
NP→N rPP
VP
NP→D rN
N
V
D
john
saw
the
NP→D rN PP
PP→P rNP
NP→D rN
N
P
D
N
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.31/32
Left-Corner Example
S
VP→V NP rPP
VP
S
NP→D N rPP
NP→D N rPP
NP
NP
S→NP rVP VP→V rNP PP
NP
PP
VP→V rNP NP→D rN PP
NP→N rPP
VP
NP→D rN
N
V
D
john
saw
the
NP→D rN PP
PP→P rNP
NP→D rN
N
P
D
N
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.31/32
Left-Corner Example
NP
S
VP→V NP rPP
VP
S
NP→D N rPP
NP→D N rPP
NP
NP
S→NP rVP VP→V rNP PP
NP
PP
VP→V rNP NP→D rN PP
NP→N rPP
VP
NP→D rN
N
V
D
john
saw
the
NP→D rN PP
PP→P rNP
NP→D rN
N
P
D
N
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.31/32
Left-Corner Example
S
VP
NP
S
VP→V NP rPP
VP
S
NP→D N rPP
NP→D N rPP
NP
NP
S→NP rVP VP→V rNP PP
NP
PP
VP→V rNP NP→D rN PP
NP→N rPP
VP
NP→D rN
N
V
D
john
saw
the
NP→D rN PP
PP→P rNP
NP→D rN
N
P
D
N
girl
in
a
car
Introduction to CL - Context-Free Grammars – p.31/32
Context-Free Parsing: Summary
Context-free grammars provide you with a finite set of
infinitely embeddable brackets
Two main approaches to CF recognition: top down
(goal-driven) and bottom-up (data driven)
Storing sub-derivations for re-use (dynamic programming)
in a chart lead to a polynomial algorithm with worst case n3
The chart offers a compact (polynomial size) storage for a
possibly exponential number of results
Earley and Left Corner Parsing improve the average
runtime over the naïve CYK algorithm, and have a better
worst case complexity for some classes of context-free
grammars
Introduction to CL - Context-Free Grammars – p.32/32