Module 28
• Context Free Grammars
– Definition of a grammar G
– Deriving strings and defining L(G)
• Context-Free Language definition
1
Context-Free Grammars
Definition
2
Definition
• A context-free grammar G = (V, S, S, P)
– V: finite set of variables (nonterminals)
– S: finite set of characters (terminals)
– S: start variable
• element of V
• role is similar to that of q0 for an FSA or NFA
– P: finite set of grammar rules or production rules
• Syntax of a production
• variable → string of variables and terminals
3
English Context-Free Grammar
• ECFG = (V, S, S, P)
– V = {<sentence>, <noun phrase>, <verb phrase>, ... }
• people sometimes use < > to delimit variables
• In this course, we generally will use capital letters to denote
variables
– S = {a, b, c, ..., z, ;, ,, ., ...}
– S = <sentence>
– P = { <sentence> → <noun phrase> <verb phrase>
<pct>, <noun phrase> → <article> <adj> <noun>, ...}
4
i
i
{a b
| i>0} CFG
• ABG = (V, S, S, P)
– V = {S}
– S = {a, b}
– S=S
– P = {S → aSb, S → ab} or S → aSb | ab
• second format saves some space
5
Context-Free Grammars
Deriving strings, defining L(G), and
defining context-free languages
6
Defining →, ==> notation
• First: → notation
– This is used to define the productions of a grammar
• S → aSb | ab
• Second: ==>G notation
– This is used to denote the application of a production
rule from a grammar G
• S ==>ABG aSb ==>ABG aaSbb ==>ABG aaabbb
– We say that string S derives string aSb (in one step)
– We say that string aSb derives string aaSbb (in one step)
– We say that string aaSbb derives string aaabbb (in one step)
• We often omit the grammar subscript when the intended
grammar is unambiguous
7
Defining ==> continued
• Third: ==>kG notation
– This is used to denote k applications of production rules
from a grammar G
• S ==>2ABG aaSbb
– We say that string S derives string aaSbb in two steps
• aSb ==>2ABG aaabbb
– We say that string aSb derives string aaabbb in two steps
• We often omit the grammar subscript when the intended
grammar is unambiguous
8
Defining ==> continued
• Fourth: ==>*G notation
– This is used to denote 0 or more applications of
production rules from a grammar G
• S ==>*ABG S
– We say that string S derives string S in 0 or more steps
• S ==>*ABG aaSbb
– We say that string S derives string aaSbb in 0 or more steps
• aSb ==>*ABG aaSbb
– We say that string aSb derives string aaSbb in 0 or more steps
• aSb ==>*ABG aaabbb
– We say that string aSb derives string aaabbb in 0 or more steps
• We often omit the grammar subscript when the intended
grammar is unambiguous
9
Defining derivations *
• Derivation of a string x
– The complete step by step derivation of a string x from
the start variable S
– Key fact: each step in a derivation makes only one
application of a production rule from G
– Example: Derivation of string aaabbb using ABG
• S ==>ABG aSb ==>ABG aaSbb ==>ABG aaabbb
– Example 2: AG= (V, S, S, P) where P = S →SS | a
• Deriving string aaa
• S ==> SS ==> Sa ==> SSa ==> aSa ==> aaa
10
Defining L(G) *
• Generating strings
– If S ==>G* x, then grammar G generates string x
• Note G generates strings which contain terminals and
nonterminals
– aSb contains nonterminals and terminals
– S contains only nonterminals
– aaabbb contains only terminals
• L(G)
– The set of strings over S generated by grammar G
• Note we only consider terminal strings generated by G
– {aibi | i > 0} = L(ABG)
– {ai | i > 0} = L(AG)
11
Context-Free Languages *
• Context-Free Languages
– A language L is a context-free language (CFL) iff
• Results so far
– {ai | i > 0} is a CFL
• One CFG G such that L(G) = this language is AG
• Note this language is also regular
– {aibi | i > 0} is a CFL
• One CFG G such that L(G) = this language is ABG
• Note this language is NOT regular
12
Example *
• Let BAL = the set of strings over {(,)} in which
the parentheses are balanced
• Prove that BAL is a CFL
– To prove this, you need to come up with a CFG BALG
such that L(BALG) = BAL
• BALG = (V, S, S, P)
–
–
–
–
V = {S}
S = {(, )}
S=S
P=?
• Give derivations of ((( ))) and ( )(( )) with your grammar
13
Module 29
• Parse/Derivation Trees
– Leftmost derivations, rightmost derivations
• Ambiguous Grammars
– Examples
• Arithmetic expressions
• If-then-else Statements
– Inherently ambiguous CFL’s
14
Context-Free Grammars
Parse Trees
Leftmost/rightmost derivations
Ambiguous grammars
15
Parse Tree
• Parse/derivation trees are structured
derivations
– The structure graphically illustrates semantic
information about the string
• Formalization of concept we encountered in
regular languages unit
– Note, what we saw before were not exactly
parse trees as we define them now, but they
were close
16
Parse Tree Example
• Parse tree for string ( )(( )) and grammar BALG
– BALG = (V, S, S, P)
• V = {S}, S = {(, )}, S = S
• P = S → SS | (S) | l
– One derivation of ( )(( ))
• S ==> SS ==> (S)S ==> ( )S ==> ( )(S) ==> ( )((S)) ==> ( )(( ))
S
– Parse tree
S
(
S
l
S
)
( S )
( S )
17
l
Comments about Example *
• Syntax:
S
S
(
S
l
S
)
( S )
( S )
l
– draw a unique arrow from each
variable to each character that is a
direct child of that variable
• A line instead of an arrow is ok
– The derived string can be read in a left
to right traversal of the leaves
• Semantics
– The tree graphically illustrates the
nesting structure of the string of
parentheses
18
Leftmost/Rightmost Derivations
• There is more than one derivation of the
string ( )(( )).
S
S
(
S
l
S
)
( S )
( S )
– S ==> SS ==> (S)S ==>( )S ==> ( )(S)
==> ( )((S)) ==> ( )(( ))
– S ==> SS ==> (S)S ==> (S)(S) ==> ( )(S)
==> ( )((S)) ==> ( )(( ))
– S ==> SS ==> S(S) ==> S((S)) ==> S(( ))
==> (S)(( )) ==>( )(( ))
• Leftmost derivation
l
– Leftmost variable is always expanded
– Which one of the above is leftmost?
• Rightmost derivation
– Rightmost variable is always expanded
– Which one of the above is rightmost?
19
Comments
S
S
(
S
l
S
)
( S )
( S )
l
– S ==> SS ==> (S)S ==>( )S ==> ( )(S)
==> ( )((S)) ==> ( )(( ))
– S ==> SS ==> (S)S ==> (S)(S) ==> ( )(S)
==> ( )((S)) ==> ( )(( ))
– S ==> SS ==> S(S) ==> S((S)) ==> S(( ))
==> (S)(( )) ==>( )(( ))
• Fix a string and a grammar
– Any derivation corresponds to a
unique parse tree
– Any parse tree can correspond to
many different derivations
• Example
– The one parse tree corresponds to all
three derivations
• Unique mappings
– For any parse tree, there is a unique
leftmost/rightmost derivation that it
corresponds to
20
Example *
• S ==> SS ==> SSS ==> (S)SS ==> ( )SS ==> ( )S ==> ( )
– The above is a leftmost derivation of the string ( ) from the
grammar BALG
– Draw the corresponding parse tree
– Draw the corresponding rightmost derivation
• S ==> (S) ==> (SS) ==> (S(S)) ==> (S( )) ==> (( ))
– The above is a rightmost derivation of the string (( )) from the
grammar BALG
– Draw the corresponding parse tree
– Draw the corresponding leftmost derivation
21
Ambiguous Grammars
Examples:
Arithmetic Expressions
If-then-else statements
Inherently ambiguous grammars
22
Ambiguous Grammars
• A grammar G is ambiguous if there exists a string
x in L(G) with two or more distinct parse trees
– (2 or more distinct leftmost/rightmost derivations)
• Example
– Grammar AG is ambiguous
• String aaa in L(AG) has 2 rightmost derivations
– S ==> SS ==> SSS ==> SSa ==> Saa ==> aaa
– S ==> SS ==> Sa ==> SSa ==> Saa ==> aaa
23
2 Simple Examples
• Grammar BALG is ambiguous
– String ( ) in L(BALG) has >1 leftmost
derivation
• S ==> (S) ==> ( )
• S ==> (S) ==> (SS) ==>(S) ==>( )
• Give another leftmost derivation of ( ) from BALG
• Grammar ABG is NOT ambiguous
– Consider any string x in {aibi | i > 0}
• There is a unique parse tree for x
24
Legal Arithmetic Expressions
• Develop a grammar MATHG = (V, S, S, P) for the
language of legal arithmetic expressions
– S = {0, 1, +, *, -, /, (, )}
– Strings in the language include
•
•
•
•
0
10
10*11111+100
10*(11111+100)
– Strings not in the language include
• 10+
• 11++101
• )(
25
Grammar MATHG1
•
•
•
•
V = {E, N}
S = {0, 1, +, *, -, /, (, )}
S=E
P:
– E → N | E+E | E*E | E/E | E-E | (E)
– N → N0 | N1 | 0 | 1
26
MATHG1 is
ambiguous
E → N | E+E | E*E | E/E | E-E | (E)
N → N0 | N1 | 0 | 1
• Come up with two distinct leftmost derivations of
the string 11+0*11
– E ==> E+E ==> N+E ==> N1+E ==> 11+E ==>
11+E*E ==> 11+N*E ==> 11+0*E ==> 11+0*N ==>
11+0*N1 ==> 11+0*11
– E ==> E*E ==> E+E*E ==> N+E*E ==> N1+E*E ==>
11+E*E ==> 11+N*E ==> 11+0*E ==> 11+0*N ==>
11+0*N1 ==>11+0*11
• Draw the corresponding parse trees
27
Corresponding Parse Trees
•
E ==> E+E ==> N+E ==> N1+E ==>
11+E ==> 11+E*E ==> 11+N*E ==>
11+0*E ==> 11+0*N ==> 11+0*N1
==> 11+0*11
•
E ==> E*E ==> E+E*E ==> N+E*E
==> N1+E*E ==> 11+E*E ==>
11+N*E ==> 11+0*E ==> 11+0*N ==>
11+0*N1 ==>11+0*11
E
E
E
+
N
N
1
1
E
E
E
*
E
* E
E
+ E
N
N
N
N
N
0
N
0
1
1
1
N
1
1
N
1
28
Parse Tree Meanings
E
E
E
+
N
N
1
1
E
E
E
*
E
* E
E
+ E
N
N
N
N
N
0
N
0
1
1
1
N
1
N
1
1
Note how the parse trees captures the semantic meaning of string 11+0*11.
More specifically, what number does the first parse tree represent?
What number does the second parse tree represent?
29
Implications
• Two interpretations of string 11+0*11
– 11+(0*11) = 11
– (11+0)*11 = 1001
• What if a line in a program is
– MSU_Tuition = 11+0*11;
– What is MSU_Tuition?
• Depends on how the expression 11+0*11 is parsed.
• This is not good.
• Ambiguity in grammars is undesirable, particularly if the grammar is
used to develop a compiler for a programming language like C++.
• In this case, there is an unambiguous grammar for the
language of arithmetic expressions
30
If-Then-Else Statements
• A grammar ITEG = (V, S, S, P) for the language
of legal If-Then-Else statements
–
–
–
–
V = (S, BOOL)
S = {D<85, D>50, grade=3.5, grade=3.0, if, then, else}
S=S
P:
• S → if BOOL then S else S | if BOOL then S |grade=3.5 |
grade=3.0
• BOOL → D<85 | D>50
31
ITEG is
ambiguous
S → if BOOL then S |grade=3.5 |
grade=3.0 | if BOOL then S else S
BOOL → D<85 | D>50
• Come up with two distinct leftmost derivations of
the string
– if D<85 then if D>50 then grade=3.5 else grade=3.0
– S ==>if BOOL then S else S ==> if D<85 then S else S ==> if D<85 then
if BOOL then S else S ==> if D<85 then if D>50 then S else S ==> if
D<85 then if D>50 then grade=3.5 else S ==> if D<85 then if D>50 then
grade=3.5 else grade=3.0
– S ==>if BOOL then S ==> if D<85 then S ==> if D<85 then if BOOL
then S else S ==> if D<85 then if D>50 then S else S ==> if D<85 then if
D>50 then grade=3.5 else S ==> if D<85 then if D>50 then grade=3.5 else
grade=3.0
• Draw the corresponding parse trees
32
Corresponding Parse Trees
•
S ==>if BOOL then S else S ==> if D<85
then S else S ==> if D<85 then if BOOL then
S else S ==> if D<85 then if D>50 then S else
S ==> if D<85 then if D>50 then grade=3.5
else S ==> if D<85 then if D>50 then
grade=3.5 else grade=3.0
•
S ==>if BOOL then S ==> if D<85 then S
==> if D<85 then if BOOL then S else S ==>
if D<85 then if D>50 then S else S ==> if
D<85 then if D>50 then grade=3.5 else S ==>
if D<85 then if D>50 then grade=3.5 else
grade=3.0
S
if
D<85
B
then
S
S
else
if B then S
D>50
S
grade=3.0
grade=3.5
if
D<85
D>50
B
then
S
S
if B then S else
grade=3.5
grade=3.0
33
Parse Tree Meanings
S
S
if
D<85
B
then
S
else
if B then S
D>50
if
S
grade=3.0
grade=3.5
D<85
D>50
B
then
S
S
if B then S else
grade=3.5
grade=3.0
If you receive a 90 on type D points, what is your grade?
By parse tree 1
By parse tree 2
34
Implications
• Two interpretations of string
–
–
–
–
if D<85 then if D>50 then grade=3.5 else grade=3.0
Issue is which if-then does the last ELSE attach to?
This phenomenon is known as the “dangling else”
Answer: Typically, else binds to NEAREST if-then
• In this case, there is an unambiguous grammar for handling
if-then’s as well as if-then-else’s
35
Inherently ambiguous CFL’s
• A CFL L is inherently ambiguous iff for all CFG’s G such
that L(G) = L, G is ambiguous
• Examples so far
– None of the CFL’s we’ve seen so far are inherently ambiguous
– While the CFG’s we’ve seen ambiguous, there do exist
unambiguous CFG’s for those CFL’s.
• Later result
– There exist inherently ambiguous CFL’s
– Example: {aibjck | i=j or j=k or i=j=k}
• Note i=j=k is unnecessary, but I added it here for clarity
36
Summary
• Parse trees illustrate “semantic” information about strings
• Ambiguous grammars are undesirable
– This means there are multiple parse trees for some string
– These strings can be interpreted in multiple ways
• There are some heuristics people use for taking an
ambiguous grammar and making it unambiguous, but this
is not the focus of this course
• There are some inherently ambiguous CFL’s
– Thus, the above heuristics do not always work
37
Module 30
• EQUAL language
– Designing a CFG
– Proving the CFG is correct
38
EQUAL language
Designing a CFG
39
EQUAL
• EQUAL is the set of strings over {a,b} with an equal
number of a’s and b’s
• Strings in EQUAL include
– aabbab
– bbbaaa
– abba
• Strings in {a,b}* not in EQUAL include
–
–
–
–
aaa
bbb
aab
ababa
40
Designing a CFG for EQUAL
• Think recursively
• Base Case
– What is the shortest possible string in EQUAL?
– Production Rule:
41
Recursive Case
• Recursive Case
– Now consider a longer string x in EQUAL
– Since x has length > 0, x must have a first character
• This must be a or b
– Two possibilities for what x looks like
• x = ay
– What must be true about relative number of a’s and b’s in y?
• x = bz
– What must be true about relative number of a’s and b’s in z?
42
Case 1: x=ay
• x = ay where y has one extra b
– What must y look like?
• Some examples
–
–
–
–
b
babba
aabbbab
aaabbbb
• Is there a general pattern that applies to all of the above
examples?
• More specifically, show how we can decompose all of the
above strings y into 3 pieces, two of which belong to EQUAL.
– Some of these pieces might be the empty string l
43
Decomposing y
• y has one extra b
– Possible examples
• b, babba, aabbbab, aaabbbb
– Decomposition
• y = ubv where
– u and v both have an equal number of a’s and b’s
• Decompose the 4 strings above into u, b, v
– lbl, aabbbab, lbabba, aaabbbbl
44
Implication
• Case 1: x=ay
– y has one extra b
• Case 1 refined: x=aubv
– u, v belong to EQUAL
• Production rule for this case?
45
Case 2: x=bz
• Case 2: x=bz
– z has one extra a
• Case 2 refined: x=buav
– u, v belong to EQUAL
• Production rule for this case?
46
Final Grammar
• EG = (V, S, S, P)
–
–
–
–
V = {S}
S = {a,b}
S=S
P:
47
EQUAL language
Proving CFG is correct
48
Is our grammar correct?
• How do we prove our grammar is correct?
– Informal
• Test some strings
• Review logic behind program (CFG) design
– Formal
• First, show every string derived by EG belongs to EQUAL
– That is, show L(EG) is a subset of EQUAL
• Second, show every string in EQUAL can be derived by EG
– That is, show EQUAL is a subset of L(EG)
• Both proofs will be inductive proofs
– Inductive proofs and recursive algorithms go well together
49
L(EG) subset of EQUAL
• Let x be an arbitrary string in L(EG)
• What does this mean?
– S ==>*EG x
• Follows from definition of x in L(EG)
– We will prove the following
•
•
•
•
•
If S ==>1EG x, then x is in EQUAL
If S ==>2EG x, then x is in EQUAL
If S ==>3EG x, then x is in EQUAL
If S ==>4EG x, then x is in EQUAL
...
50
Base Case
• Statement to be proven:
– For all n ≥ 1, if S ==>nEG x, then x is in EQUAL
– Prove this by induction on n
• Base Case:
– n=1
– What is the set of strings {x | S ==>1EG x}?
– What do we need to prove about this set of strings?
51
Inductive Case
• Inductive Hypothesis:
– For 1 ≤ j ≤ n, if S ==>jEG x, then x is in EQUAL
• Note, this is a “strong” induction hypothesis
• Traditional inductive hypothesis would take form:
– For some n ≥ 1, if S ==>nEG x, then x is in EQUAL
• The difference is we assume the basic hypothesis for all
integers between 1 and n, not just n
• Statement to be Proven in Inductive Case:
– If S ==>n+1EG x, then x is in EQUAL
52
“Regular” induction vs Strong
induction
• Infinite Set of Facts • Base Case
–
–
–
–
–
–
–
Fact 1
Fact 2
Fact 3
Fact 4
Fact 5
Fact 6
…
– Prove fact 1
• Regular inductive case
– For n 1,
• Fact n --> Fact n+1
• Strong inductive case
– For n ≥ 1,
• Fact 1 to Fact n --> Fact n+1
53
Visualization of Induction
Regular Induction
Strong Induction
Fact 1
Fact 2
Fact 3
Fact 4
Fact 5
Fact 6
Fact 7
Fact 8
Fact 9
…
Fact 1
Fact 2
Fact 3
Fact 4
Fact 5
Fact 6
Fact 7
Fact 8
Fact 9
…
54
Proving Inductive Case
• If S ==>n+1EG x, then x is in EQUAL
– Let x be an arbitrary string such that S ==>n+1EG x
– Examining EG, what are the three possible first
derivation steps
• Case 1: S ==>
• Case 2: S ==>
• Case 3: S ==>
==>nEG x
==>nEG x
==>nEG x
– One of the cases is impossible. Which one and why?
55
Case 2: S ==>
==>nEG x
• This means x has the form aubv where
– What can we conclude about u (don’t apply IH)?
– What can we conclude about v (don’t apply IH)?
• Apply the inductive hypothesis
– u and v belong to EQUAL
– Why do we need the strong inductive hypothesis?
• Conclude x belongs to EQUAL
– x = aubv where u and v belong to EQUAL
• Clearly the number of a’s in x equals the number of b’s in x
56
Case 3: S ==>
==>nEG x
• This means x has the form buav where
– What can we conclude about u (no IH)?
– What can we conclude about v (no IH)
• Apply the inductive hypothesis
– u and v belong to EQUAL
– Why do we need the strong inductive hypothesis?
• Conclude x belongs to EQUAL
– x = buav where u and v belong to EQUAL
• Clearly the number of a’s in x equals the number of b’s in x
57
L(EG) subset of EQUAL
• Wrapping up inductive case
– In all possible derivations of x, we have shown that x
belongs to EQUAL
– Thus, we have proven the inductive case
• Conclusion
– By the principle of mathematical induction, we have
shown that L(EG) is a subset of EQUAL
58
EQUAL subset of L(EG)
• Let x be an arbitrary string in EQUAL
• What does this mean?
• We will prove the following
•
•
•
•
•
If |x| = 0 and x is in EQUAL, then x is in L(G)
If |x| = 1 and x is in EQUAL, then x is in L(G)
If |x| = 2 and x is in EQUAL, then x is in L(G)
If |x| = 3 and x is in EQUAL, then x is in L(G)
...
59
EQUAL subset of L(EG)
• Statement to be proven:
– For all n ≥ 0, if |x| = n and x is in EQUAL, then x is in
L(EG)
– Prove this by induction on n
• Base Case:
– n=0
– What is the only string x such that |x|=0 and x is in
EQUAL?
– Prove this string belongs to L(EG)
60
Inductive Case
• Inductive Hypothesis:
– For 0 ≤ j ≤ n, if |x| =j and x is in EQUAL, then x is in
L(EG)
• Again, this is a “strong” induction hypothesis
• Statement to be Proven in Inductive Case:
– For n ≥ 0,
– if |x| = n+1 and x is in EQUAL, then x is in L(EG)
61
Proving Inductive Case
• If |x|=n+1 and x is in EQUAL, then x is in L(EG)
– Let x be an arbitrary string such that |x|=n+1 and x is in
L(EG)
– Examining S, what are the two possibilities for the first
character in x?
• Case 1: first character in x is
• Case 2: first character in x is
– In each case, what can we say about the remainder of
x?
• Case 1: the remainder of x
• Case 2: the remainder of x
62
Case 1: x = ay
• What can we say about y in this case?
• This means x has the form aubv where
– u is in EQUAL and has length ≤ n
– v is in EQUAL and has length ≤ n
– Proving this statement true
• Consider all the prefixes of string y
–
–
–
–
–
length 0: l
length 1: y1
length 2: y1y2
…
length n: y1y2 … yn = y
63
Case 1: x = ay
• Consider all the prefixes of string y
–
–
–
–
–
length 0: l
length 1: y1
length 2: y1y2
…
length n: y1y2 … yn = y
• The first prefix l has the same number of a’s as b’s
• The last prefix y has one extra b
• The relative number of a’s and b’s changes in the length i
prefix differs by only one from the length i-1 prefix
• Thus, there must be a first prefix t of y where t has one extra b
• Furthermore, the last character of t must be b
– Otherwise, t would not be the FIRST prefix of y with one extra b
• Break t into u and b and let the remainder of y be v
• The statement follows
64
Case 1: x = aubv *
• x = aubv
– u is in EQUAL and has length ≤ n
– v is in EQUAL and has length ≤ n
• Apply the induction hypothesis
– What can we conclude from applying the IH?
– Why did we need a strong inductive hypothesis?
• Conclude x is in L(EG) by constructing a derivation
– S ==> aSbS ==>*EG aubS ==>*EG aubv
65
Case 2: x = buav
• x = buav
– u is in EQUAL and has length ≤ n
– v is in EQUAL and has length ≤ n
• Apply the induction hypothesis
– What can we conclude about u and v?
• Conclude x is in L(EG) by constructing a derivation
– S ==> bSaS ==>*EG buaS ==>*EG buav
• Justify each of the steps in this derivation
66
EQUAL subset of L(EG)
• Wrapping up inductive case
– For all possible first characters of x, we have shown
that x belongs to L(EG)
– Thus, we have proven the inductive case
• Conclusion
– By the principle of mathematical induction, we have
shown that EQUAL is a subset of L(EG)
67
Module 31
• Closure Properties for CFL’s
– Kleene Closure
• construction
• examples
• proof of correctness
– Others covered less thoroughly in lecture
• union, concatenation
• CFL’s versus regular languages
– regular languages subset of CFL
68
Closure Properties for CFL’s
Kleene Closure
69
CFL closed under Kleene Closure
• Let L be an arbitrary CFL
• Let G1 be a CFG s.t. L(G1) = L
– G1 exists by definition of L1 in CFL
•
•
•
•
Construct CFG G2 from CFG G1
Argue L(G2) = L*
There exists CFG G2 s.t. L(G2) = L*
L* is a CFL
70
Visualization
• Let L be an arbitrary CFL
• Let G1 be a CFG s.t. L(G1) = L
L
L*
– G1 exists by definition of L1 in CFL
•
•
•
•
Construct CFG G2 from CFG G1
Argue L(G2) = L*
There exists CFG G2 s.t. L(G2) = L*
L* is a CFL
CFL
G1
G2
CFG’s
71
Algorithm Specification
• Input
– CFG G1
• Output
– CFG G2 such that L(G2) =
CFG G1
A
CFG G2
72
Construction
• Input
– CFG G1 = (V1, S, S1, P1)
• Output
– CFG G2 = (V2, S, S2, P2)
• V2 = V1 union {T}
– T is a new symbol not in V1 or S
• S2 = T
• P2 = P1 union ??
73
Closure Properties for CFL’s
Kleene Closure Examples
74
Example 1
• Input grammar:
–
–
–
–
V = {S}
S = {a,b}
S=S
P:
V2 = V1 union {T}
T is a new symbol not in V1 or S
S2 = T
P2 = P1 union {T → ST | l}
• Output grammar
–
–
–
–
V=
S = {a,b}
Start symbol is
P:
S → aa | ab | ba | bb
75
Example 2
• Input grammar:
–
–
–
–
V = {S, T}
S = {a,b}
Start symbol is T
P:
V2 = V1 union {T}
T is a new symbol not in V1 or S
S2 = T
P2 = P1 union {T → ST | l}
• Output grammar
–
–
–
–
V=
S = {a,b}
Start symbol is
P:
T → ST | l
S → aa | ab | ba | bb
76
Closure Properties for CFL’s
Kleene Closure Proof of Correctness
77
Is our construction correct?
• How do we prove our construction is correct?
– Informal
• Test some strings
• Review logic behind construction
– Formal
• First, show every string derived by G2 belongs to (L(G1))*
– That is, show L(G2) is a subset of (L(G1))*
• Second, show every string in (L(G1))* can be derived by G2
– That is, show (L(G1))* is a subset of L(G2)
• Both proofs will be inductive proofs
– Inductive proofs and recursive algorithms go well together
78
L(G2) is a subset of (L(G1))*
• We want to prove the following
– If x in L(G2), then x is in (L(G1))*
• This is equivalent to the following
– If T ==>*G2 x, then x is in (L(G1))*
– The two statements are equivalent because
• x in L(G2) means that T ==>*G2 x
• We break the second statement down as follows:
– If T ==>1G2 x, then x is in (L(G1))*
– If T ==>2G2 x, then x is in (L(G1))*
– If T ==>3G2 x, then x is in (L(G1))*
– ...
79
L(G2) is a subset of (L(G1))*
• Statement to be proven:
– For all n ≥ 1, if T ==>nG2 x, then x is in (L(G1))*
– Prove this by induction on n
• Base Case:
– n=1
– Examining grammer G2, what is the only string x such
that T ==>1G2 x ?
– Prove this string is in (L(G1))*
80
Inductive Case
• Inductive Hypothesis:
– For 1 ≤ j ≤ n, if T ==>jG2 x, then x is in (L(G1))*
• Note, this is a “strong” induction hypothesis
• Statement to be Proven in Inductive Case:
– For n above, if T ==>n+1G2 x, then x is in (L(G1))*
• Proving this statement
– Let x be an arbitrary string such that T ==>n+1G2 x
– Examining G2, what are the two possible first
derivation steps?
• Case 1: T ==>G2
• Case 2: T ==>G2
==>nG2 x
==>nG2 x
81
Case Analysis
• Case 1: T ==>G2 ==>n x is not possible
– Why not?
• Case 2: T ==>G2 ==>nG2 x
– This means x has the form uv where
• What can we say about u (no IH)?
• What can we say about v (no IH)?
– Applying the inductive hypothesis, what can we conclude?
82
Concluding Case 2:
T ==>G2 ==>nG2 x
– Concluding string u belongs to L(G1)
• Follows from S ==>* G2 u and
• Our construction insures that all strings derived from S in L(G2) are
also in L(G1)
– How do we conclude that x belongs to (L(G1))*
• Wrapping up inductive case
– In all possible derivations of x, we have shown that x belongs to
(L(G1))*
– Thus, we have proven the inductive case
• Conclusion
– By the principle of mathematical induction, we have shown that
L(G2) is a subset of (L(G1))*
83
(L(G1))* is a subset of L(G2)
• We want to prove the following
– If x is in (L(G1))*, then x is in L(G2)
• This is equivalent to the following
– If x is in (L(G1))*, then T ==>*G2 x
– The two statements are equivalent because
• x in L(G2) means that T ==>*G2 x
• We break the second statement down as follows:
– If x is in (L(G1))0, then T ==>*G2 x
– If x is in (L(G1))1, then T ==>*G2 x
– If x is in (L(G1))2, then T ==>*G2 x
– ...
84
(L(G1))* is a subset of L(G2)
• Statement to be proven:
– For all n ≥ 0, if x is in (L(G1))n, then x is in L(G2)
– Prove this by induction on n
• Base Case:
– n=0
– What is the only string x in (L(G1))0?
– Show this string belongs to L(G2)
85
Inductive Case
• Inductive Hypothesis:
– For n ≥ 0, if x is in (L(G1))j, then T ==>*G2 x
• Note, this is a “normal” induction hypothesis
• Statement to be Proven in Inductive Case:
– For n ≥ 0, if x is in (L(G1))n+1, then T ==>*G2 x
• Proving this statement
– Let x be an arbitrary string in (L(G1))n+1
– This means x = uv where
• u in L(G1)
• What can we say about v?
86
Deriving x
– x = uv where
• u is a string in L(G1)
• v is a string in
– Justify all the steps in the following derivation
– T ==> G2 ST ==>* G2 Sv ==>* G2 uv = x
• First step:
• Second step:
• Third step:
– Thus T ==>* G2 x
• The inductive case follows
• The result is proven by the principle of mathematical
induction
87
Construction for Set Union
• Input
– CFG G1 = (V1, S, S1, P1)
– CFG G2 = (V2, S, S2, P2)
• Output
– CFG G3 = (V3, S, S3, P3)
• V3 = V1 union V2 union {T}
– Variable renaming to insure no names shared between V1 and V2
– T is a new symbol not in V1 or V2 or S
• S3 = T
• P3 =
88
Construction for Set
Concatenation
• Input
– CFG G1 = (V1, S, S1, P1)
– CFG G2 = (V2, S, S2, P2)
• Output
– CFG G3 = (V3, S, S3, P3)
• V3 = V1 union V2 union {T}
– Variable renaming to insure no names shared between V1 and V2
– T is a new symbol not in V1 or V2 or S
• S3 = T
• P3 =
89
CFL’s and regular languages
90
CFL Closure Properties
• What have we just proven
– CFL’s are closed under Kleene closure
– CFL’s are closed under set union
– CFL’s are closed under set concatenation
• What can we conclude from these 3 results?
– It follows that regular languages are a subset of
CFL’s
91
Regular languages subset of CFL
• Recursive definition of regular languages
– Base Case:
• {}, {l}, {a}, {b} are regular languages over {a,b}
• P={}, P={S → l}, P={S → a}, P={S → b}
– Inductive Case:
• If L1 and L2 are are regular languages, then L1*,
L1L2, L1 union L2 are regular languages
• Use previous constructions to see that these
resulting languages are also context-free
92
Other CFL Closure Properties
• We will show that CFL’s are NOT closed under
many other set operations
• Examples include
– set complement
– set intersection
– set difference
93
Language class hierarchy
?
H
H
Equal
REG
CFL
REC
RE
All languages over alphabet S
94
Module 32
• Pushdown Automata (PDA’s)
– definition
– Example
• We define configurations and computations
of PDA’s
• We define L(M) for PDA’s
95
Pushdown Automata
Definition and Motivating Example
96
Pushdown Automata (PDA)
• In this presentation we introduce the PDA
model of computation (programming
language).
– The key addition to a PDA (from an NFA-/\) is
the addition of external memory in the form of
an infinite capacity stack
• The word “pushdown” comes from the stacks of
trays in cafeterias where you have to pushdown on
the stack to add a tray to it.
97
NFA for
a
b
/\
I
m
n
{a b
/\
B
C
• What strings end up in
each state of the above
NFA?
– I:
– B:
– C:
| m,n ≥ 0}
• Consider the language
{anbn | n ≥ 0}.
• This NFA can recognize
strings which have the
correct form,
– a’s followed by b’s.
• However, the NFA cannot
remember the relative
number of a’s and b’s seen
at any point in time.
98
PDA for
a
b
/\
I
/\
B
C
NFA
a;push a
I
Initialize stack to empty
|n≥0} *
Imagine we now have memory in the form of a stack which
we can use to help remember how many a’s we have seen by
pushing onto and popping from the stack
When we see an a in state I, we do the following two actions:
1) We push an a on the stack.
2) We stay in state I.
b;pop
/\
n
n
{a b
When we see a b in state B, we do the following two actions:
1) We pop an a from the stack.
/\;only if stack is empty
2) We stay in state B.
PDA
B
C
From state B, we allow a /\-transition to state C only if
1) The stack is empty.
Finally, when we begin, the stack should be empty.
99
Formal PDA definition
• PDA M = (Q, S, G, q0, Z, A, d)
• Modified elements
– G is the stack alphabet
• Z is a special character that is initially on the stack
• Often used to represent an empty stack
– d is modified as follows
• Pop to read the top character on the stack
• Stack update action
– What to push back on the stack
– If we push /\, then the net result of the action is a pop
100
Example PDA
a;a; aa
a;Z; aZ
I
Initialize stack to
only contain Z
/\;Z;Z
/\;a;a
b;a;/\
B
/\;Z;Z
C
Q = {I, B, C}
S = {a,b}
G = {Z, a}
q0 = I
Z is the initial stack character
A = {C}
d:
Example PDA
S
I
I
I
I
B
B
a
a
a
/\
/\
b
/\
TopSt
a
Z
a
Z
a
Z
NS
I
I
B
B
B
C
stack update
push aa
push aZ
push a
push Z
push /\
push Z
101
Computing with PDA’s *
• Configurations change compared with NFA-/\’s
– Configuration components:
• current state
• remaining input to be processed
• stack contents
• Computations are essentially the same as with
NFA-/\’s given the modified configurations
– Determining which transitions of a PDA can be
applied to a given configuration is more
complicated though
102
Computation Graph of PDA
Q = {I, B, C}
S = {a,b}
G = {Z, a}
q0 = I
Z is the initial stack character
A = {C}
d:
S
I
I
I
I
B
B
a
a
a
/\
/\
b
/\
TopSt
a
Z
a
Z
a
Z
NS
I
I
B
B
B
C
Computation graph for this PDA on the input string aabb
(I,aabb,Z)
(B,aabb,Z)
(I,abb,aZ)
(I,bb,aaZ)
stack update
push aa
push aZ
push a
push Z
push /\
push Z
(B,abb,aZ)
(C,aabb,Z)
(B,bb,aaZ)
(B,b,aZ)
(B,/\,Z)
(C,/\,Z)
103
a;a; aa
a;Z; aZ
Definition of ├
I
/\;Z;Z
/\;a;a
b;a;/\
B
/\;Z;Z
C
Input string aabb
(I, aabb, Z) ├ (I,abb,aZ)
(I,aabb,Z)
(I, aabb, Z) ├ (B, aabb, Z)
(B,aabb,Z)
(I,abb,aZ)
(I,bb,aaZ)
(B,bb,aaZ)
(B,b,aZ)
(B,/\,Z)
(B,abb,aZ)
(C,aabb,Z)
(I, aabb, Z) ├ 2 (C, aabb, Z)
(I, aabb, Z) ├ 3 (B, bb, aaZ)
(I, aabb, Z) ├ * (B, abb, aZ)
(I, aabb, Z) ├ * (B, /\, Z)
(I, aabb, Z) ├ * (C, /\, Z)
(C,/\,Z)
104
a;a; aa
a;Z; aZ
Acceptance and
Rejection
(I,aabb,Z)
(B,aabb,Z)
(I,abb,aZ)
(I,bb,aaZ)
(B,abb,aZ)
(C,aabb,Z)
I
/\;Z;Z
/\;a;a
b;a;/\
B
/\;Z;Z
C
Input string aabb
M accepts string x if one of the
configurations reached is an accepting
configuration
(q0, x, Z) ├* (f, /\, a),f in A, a in G*
Stack contents can be anything
(B,bb,aaZ)
(B,b,aZ)
Not an accepting
configuration since input
not completely processed
(B,/\,Z)
(C,/\,Z)
An accepting
configuration
Not an accepting
configuration since state
is not accepting
M rejects string x if all configurations
reached are either not halting
configurations or are rejecting
configurations
105
Defining L(M) and LPDA
M accepts string x if one of the
configurations reached is an accepting
configuration
(q0, x, Z) ├* (f, /\, a),f in A, a in G*
Stack contents can be anything
M rejects string x if all configurations
reached are either not halting
configurations or are rejecting
configurations
• L(M) (or Y(M))
– The set of strings ?
• N(M)
– The set of strings ?
• LPDA
– Language L is in language
class LPDA iff ?
106
Deterministic PDA’s
• A PDA is deterministic if its transition function
satisfies both of the following properties
– For all q in Q, a in S union {/\}, and X in G,
• the set d(q,a,X) has at most one element
– For all q in Q and X in G,
• if d(q, /\, X) ≠ { }, then d(q,a,X) = { } for all a in S
• A computation graph is now just a path again
• Our default assumption is that PDA’s are
nondeterministic
107
Two forms of nondeterminism
Trans Current Input Top of Next Stack
#
State Char. Stack State Update
------------------------------------------------------1
q0
a
Z
q0
aZ
2
q0
a
Z
q0
aa
3
4
q0
q0
/\
a
Z
Z
q0
q0
aZ
aa
108
LPDA and DCFL
• A language L is in language class LPDA if and
only if there exists a PDA M such that L(M) = L
• A language L is in language class DCFL
(Deterministic Context-Free Languages) if and
only if there exists a deterministic PDA M such
that L(M) = L
• To be proven
– LPDA = CFL
– CFL is a proper superset of DCFL
109
PDA Comments
• Note, we can use the stack for much more
than just a counter
• See examples in chapter 7 for some details
110
Module 33
• Pushdown Automata (PDA’s)
– Another example
111
Palindromes
• Let PAL be the set of palindromes over {a,b}
– Let PAL1 be the following related language:
• {wcwr | w consists only of a’s and b’s}
– we add c to the input alphabet as a special “marker” character
– Strings in PAL1
» aca, bcb, abcba, aabcbaa, c
– strings not in PAL1
» aaca, aaccaa, abccba, abcb, abba
– Let PAL2 be the set of even length palindromes
• {wwr | w consists only of a’s and b’s}
112
PAL1
• Lets first construct a PDA for PAL1
• Basic ideas
– Have one state remember first “half” of string
– Have one state “match” second half of string to
first half
– Transition between these two states when the
first c is encountered
113
PDA for PAL1
• M = (Q, S, G, q0, Z, A, d)
–
–
–
–
–
–
Q = {q0, qm, qf}
S = {a, b, c}
G = {Z, a, b}
q0 = q 0
Z=Z
A = {qf}
114
Transition Function
Trans Current Input Top of Next Stack
#
State Char. Stack State Update
------------------------------------------------------1
q0
a
Z
q0
aZ
2
q0
a
a
q0
aa
3
q0
a
b
q0
ab
4
5
6
q0
q0
q0
b
b
b
Z
a
b
q0
q0
q0
bZ
ba
bb
7
8
9
q0
q0
q0
c
c
c
Z
a
b
qm
qm
qm
Z
a
b
10
11
12
qm
qm
qm
a
b
l
a
b
Z
qm
qm
qf
l
l
Z
First three transitions push a on
top of the stack
Second three transitions push b
on the stack
Third three transitions switch
state q0 to qm
No change to stack
Transitions 10 and 11 “match”
characters from first and last
half of input string
115
Notation comment
Trans Current Input Top of Next Stack
#
State Char. Stack State Update
------------------------------------------------------1
q0
a
Z
q0
aZ
2
q0
a
a
q0
aa
3
q0
a
b
q0
ab
4
5
6
q0
q0
q0
b
b
b
Z
a
b
q0
q0
q0
bZ
ba
bb
7
8
9
q0
q0
q0
c
c
c
Z
a
b
qm
qm
qm
Z
a
b
10
11
12
qm
qm
qm
a
b
l
a
b
Z
qm
qm
qf
l
l
Z
We might represent transition 1
in two other ways
d(q0,a,Z) = (q0, aZ)
(q0, a, Z, q0, aZ)
•Question
•Is this PDA deterministic?
116
Computation Graph 1
Trans Current Input Top of Next Stack
#
State Char. Stack State Update
------------------------------------------------------1
q0
a
Z
q0
aZ
2
q0
a
a
q0
aa
3
q0
a
b
q0
ab
4
5
6
q0
q0
q0
b
b
b
Z
a
b
q0
q0
q0
bZ
ba
bb
7
8
9
q0
q0
q0
c
c
c
Z
a
b
qm
qm
qm
Z
a
b
10
11
12
qm
qm
qm
a
b
l
a
b
Z
qm
qm
qf
l
l
Z
(q0, abcba, Z)
(q0, bcba, aZ)
(q0, cba, baZ)
(qm, ba, baZ)
(qm, a, aZ)
(qm, l, Z)
(qf, l, Z)
117
Computation Graph 2
Trans Current Input Top of Next Stack
#
State Char. Stack State Update
------------------------------------------------------1
q0
a
Z
q0
aZ
2
q0
a
a
q0
aa
3
q0
a
b
q0
ab
4
5
6
q0
q0
q0
b
b
b
Z
a
b
q0
q0
q0
bZ
ba
bb
7
8
9
q0
q0
q0
c
c
c
Z
a
b
qm
qm
qm
Z
a
b
10
11
12
qm
qm
qm
a
b
l
a
b
Z
qm
qm
qf
l
l
Z
(q0, abcab, Z)
(q0, bcab, aZ)
(q0, cab, baZ)
(qm, ab, baZ)
118
Computation Graph 3
Trans Current Input Top of Next Stack
#
State Char. Stack State Update
------------------------------------------------------1
q0
a
Z
q0
aZ
2
q0
a
a
q0
aa
3
q0
a
b
q0
ab
4
5
6
q0
q0
q0
b
b
b
Z
a
b
q0
q0
q0
bZ
ba
bb
7
8
9
q0
q0
q0
c
c
c
Z
a
b
qm
qm
qm
Z
a
b
10
11
12
qm
qm
qm
a
b
l
a
b
Z
qm
qm
qf
l
l
Z
(q0, acab, Z)
(q0, cab, aZ)
(qm, ab, aZ)
(qm, b, Z)
(qf, b, Z)
119
PAL2
• Lets now construct a PDA for PAL
• What is harder this time?
– When do we switch from putting strings on the
stack to matching?
– Example
• After seeing aab, should we switch to match mode
or stay in stack mode?
– Solution
• Do both using nondeterminism
120
PDA for PAL2
• M = (Q, S, G, q0, Z, A, d)
–
–
–
–
–
–
Q = {q0, qm, qf}
S = {a, b}
G = {Z, a, b}
q0 = q 0
Z=Z
A = {qf}
121
Transition Relation
Trans Current Input Top of Next Stack
#
State Char. Stack State Update
------------------------------------------------------1
q0
a
Z
q0
aZ
2
q0
a
a
q0
aa
3
q0
a
b
q0
ab
4
5
6
q0
q0
q0
b
b
b
Z
a
b
q0
q0
q0
bZ
ba
bb
7
8
9
q0
q0
q0
l
l
l
Z
a
b
qm
qm
qm
Z
a
b
10
11
12
qm
qm
qm
a
b
l
a
b
Z
qm
qm
qf
l
l
Z
First three transitions push a on
top of the stack
Second three transitions push b
on the stack
Third three transitions switch
state q0 to qm
Is the PDA deterministic or
nondeterministic?
122
Computation Graph 1
Trans Current Input Top of Next Stack
#
State Char. Stack State Update (q0, abba, Z)
------------------------------------------------------1
q0
a
Z
q0
aZ
(q0, bba, aZ)
2
q0
a
a
q0
aa
3
q0
a
b
q0
ab
(q0, ba, baZ)
4
5
6
q0
q0
q0
b
b
b
Z
a
b
q0
q0
q0
bZ
ba
bb
7
8
9
q0
q0
q0
l
l
l
Z
a
b
qm
qm
qm
Z
a
b
10
11
12
qm
qm
qm
a
b
l
a
b
Z
qm
qm
qf
l
l
Z
(qm, abba, Z)
(qm, bba, aZ)
(q0, a, bbaZ)
(q0, l, abbaZ)
(qf, abba, Z)
(qm, ba, baZ)
(qm, a, bbaZ)
(qm, l, abbaZ)
(qm, a, aZ)
(qm, l, Z)
(qf, l, Z)
123
Computation Graph 2
Trans Current Input Top of Next Stack
#
State Char. Stack State Update
------------------------------------------------------1
q0
a
Z
q0
aZ
2
q0
a
a
q0
aa
3
q0
a
b
q0
ab
4
5
6
q0
q0
q0
b
b
b
Z
a
b
q0
q0
q0
bZ
ba
bb
7
8
9
q0
q0
q0
l
l
l
Z
a
b
qm
qm
qm
Z
a
b
10
11
12
qm
qm
qm
a
b
l
a
b
Z
qm
qm
qf
l
l
Z
(q0, aba, Z)
(q0, ba, aZ)
(qm, aba, Z)
(q0, a, baZ)
(qm, ba, aZ)
(q0, l, abaZ)
(qf, aba, Z)
(qm, a, baZ)
(qm, l, abaZ)
124
PAL
• Challenge
– Construct a PDA for PAL
– First step
• Construct a PDA for odd length palindromes
– Then
• Combine PDA’s for odd length and even length
palindromes
125
Module 34
• CFG → PDA construction
– Shows that for any CFL L, there exists a PDA
M such that L(M) = L
– The reverse is true as well, but we do not prove
that here
126
CFL subset LPDA
• Let L be an arbitrary CFL
• Let G be the CFG such that L(G) = L
– G exists by definition of L is CF
•
•
•
•
Construct a PDA M such that L(M) = L(G)
Argue L(M) = L
There exists a PDA M such that L(M) = L
L is in LPDA
– By definition of L in LPDA
127
Visualization
•Let L be an arbitrary CFL
•Let G be the CFG such that L(G) = L
•G exists by definition of L is CF
•Construct a PDA M such that L(M) = L
•M is constructed from CFG G
•Argue L(M) = L
•There exists a PDA M such that L(M) = L
•L is in LPDA
•By definition of L in LPDA
L
L
CFL
G
CFG’s
LPDA
M
PDA’s
128
Algorithm Specification
• Input
– CFG G
• Output
– PDA M such that L(M) =
CFG G
A
PDA M
129
Construction Idea
• The basic idea is to have a 2-phase PDA
– Phase 1:
• Derive all strings in L(G) on the stack nondeterministically
• Do not process any input while we are deriving the string on
the stack
– Phase 2:
• Match the input string against the derived string on the stack
– This is a deterministic process
• Move to an accepting state only when the stack is empty
130
Illustration
• Input Grammar G
–
–
–
–
V = {S}
S = {a,b}
S=S
P:
S → aSb | l
• What is L(G)?
1. Derive all strings in L(G) on the stack
2. Match the derived string against input
Illustration of how the PDA might work,
though not completely accurate.
(q0, aabb, Z)
/* put S on stack */
(q1, aabb, SZ)
/* derive aabb on stack */
(q1, aabb, aSbZ)
(q1, aabb,
aaSbbZ)
(q1, aabb, aabbZ)
/* match stack vs input */
(q2, aabb, aabbZ)
(q2, abb, abbZ)
(q2, bb, bbZ)
(q2, b, bZ)
(q2,l, Z)
131
(q3, l, Z)
Difficulty
(q0, aabb, Z)
/* put S on stack */
(q1, aabb, SZ)
/* derive aabb on stack */
(q1, aabb, aSbZ)
(q1, aabb,
aaSbbZ)
(q1, aabb, aabbZ)
/* match stack vs input */
(q2, aabb, aabbZ)
(q2, abb, abbZ)
(q2, bb, bbZ)
(q2, b, bZ)
(q2,l, Z)
(q3, l, Z)
1. Derive all strings in L(G) on the stack
2. Match the derived string against input
What is illegal with the
computation graph on the
left?
132
Construction
• Input Grammar
– G=(V,S, S, P)
• Output PDA
–
–
–
–
–
–
–
M=(Q, S, G, q0, Z, F, d)
Q = {q0, q1, q2}
S=S
G = V union S union {Z}
Z=Z
q0 = q0
F = {q2}
• d:
– d(q0, l, Z) = (q1, SZ)
– d(q1, l, Z) = (q2, Z)
– For all productions A → a
• d(q1, l, A) = (q1, a)
– For all a in S
• d(q1, a, a) = (q1, l)
133
Examples
134
Palindromes
• PALG:
–
–
–
–
V = {S}
S = {a,b}
S=S
P:
• S → aSa | bSb | a | b | l
• Output PDA
M=(Q,S,G,q0,Z,F,d)
–
–
–
–
–
•
d:
Q = {q0, q1, q2}
G = {a,b,S,Z}
q0 = q0
Z=Z
F = {q2}
– d(q0, l, Z) = (q1, SZ)
– d(q1, l, Z) = (q2, Z)
– Production Transitions
•
•
•
•
•
d(q1, l, S) = (q1, aSa)
d(q1, l, S) = (q1, bSb)
d(q1, l, S) = (q1, a)
d(q1, l, S) = (q1, b)
d(q1, l, S) = (q1, l)
– Matching transitions
• d(q1, a, a) = (q1, l)
• d(q1, b, b) = (q1, l)
135
Palindrome Transition Table
Transition
Current Input
Top of Next Stack
Number
State
Symbol Stack
State Update
--------------------------------------------------------------------------------1
q0
l
Z
q1
SZ
2
q1
l
Z
q2
Z
3
q1
l
S
q1
aSa
4
q1
l
S
q1
bSb
5
q1
l
S
q1
a
6
q1
l
S
q1
b
7
q1
l
S
q1
l
8
q1
a
a
q1
l
9
q1
b
b
q1
l
136
Partial Computation Graph
(q0, aba, Z)
(q1, aba, SZ)
(q1, aba, aSaZ) (other branches not shown)
(q1, ba, SaZ)
(q1, ba, baZ) (other branches not shown)
(q1, a, aZ)
(q1, l, Z)
(q2, l, Z)
On your own, draw computation trees for other strings
not in the language and see that they are not accepted.
137
n
n
{a b
• Grammar G:
–
–
–
–
V = {S}
S = {a,b}
S=S
P:
| n ≥ 0}
• δ:
– δ(q0, l, Z) = (q1, SZ)
– δ(q1, l, Z) = (q2, Z)
– Production Transitions
•
• S → aSb | l
• Output PDA
M=(Q,S,G,q0,Z,F,d)
–
–
–
–
–
– Matching transitions
Q = {q0, q1, q2}
G = {a,b,S,Z}
q0 = q0
Z=Z
F = {q2}
138
n
n
{a b
| n ≥ 0} Transition Table
Transition
Current Input
Top of Next Stack
Number
State
Symbol Stack
State Update
--------------------------------------------------------------------------------1
q0
l
Z
2
q1
l
Z
3
q1
l
S
4
q1
l
S
5
q1
a
a
6
q1
b
b
139
Partial Computation Graph
(q0, aabb, Z)
(q1, aabb, SZ)
(q1, aabb, aSbZ) (other branch not shown)
(q1, abb, SbZ)
(q1, abb, aSbbZ) (other branch not shown)
(q1, bb, SbbZ)
(q1, bb, bbZ) (other branch not shown)
(q1, b, bZ)
(q1, l, Z)
(q2, l, Z)
140
i
j
{a b
• Grammar G:
–
–
–
–
V = {S,T,U}
S = {a,b}
S=S
P:
• S→T|U
• T → aTb | l
• U → aaUb | l
• Output PDA
M=(Q,S,G,q0,Z,F,d)
–
–
–
–
–
Q = {q0, q1, q2}
G = {a,b,S,T,U,Z}
q0 = q0
Z=Z
F = {q2}
| i = j or i = 2j}
•
d
– d(q0, l, Z) = (q1, SZ)
– d(q1, l, Z) = (q2, Z)
– Production Transitions
•
– Matching transitions
•
141
{aibj | i = j or i = 2j} Transition Table
Transition
Current Input
Top of Next Stack
Number
State
Symbol Stack
State Update
--------------------------------------------------------------------------------1
q0
l
Z
q1
SZ
2
q1
l
Z
q2
Z
3
q1
l
S
q1
T
4
q1
l
S
q1
U
5
q1
l
T
q1
aTb
6
q1
l
T
q1
l
7
q1
l
U
q1
aaUb
8
q1
l
U
q1
l
9
q1
a
a
q1
l
10
q1
b
b
q1
l
142
Partial Computation Graph
(q0, aab, Z)
(q1, aab, SZ)
(q1, aab, UZ) (other branch not shown)
(q1, aab, aaUbZ) (other branch not shown)
(q1, ab, aUbZ)
(q1, b, UbZ)
(q1, b, bZ) (other branch not shown)
(q1, l, Z)
(q2, l, Z)
143
Things you should be able to do
• You should be able to execute this algorithm
– Given any CFG, construct an equivalent PDA
• You should understand the idea behind this
algorithm
– Derive string on stack and then match it against input
• You should understand how this construction can
help you design PDA’s
• You should understand that it can be used in
answer-preserving input transformations between
decision problems about CFL’s.
144
Module 35
• Attempt to prove that CFL’s are closed
under intersection
– Review previous constructions
– Translate previous constructions to current
setting
– Prove modified result
145
High Level Overview
146
CFL closed under set intersection
• Let L1 and L2 be arbitrary CFL’s
• Let M1 and M2 be PDA’s s.t. L(M1) = L1, L(M2) =
L2
– M1 and M2 exist by definition of L1 and L2 are CFL’s and
the fact that for every CFG, there is an equivalent PDA
•
•
•
•
Construct PDA M3 from PDA’s M1 and M2
Argue L(M3) = L1 intersect L2
There exists a PDA M3 s.t. L(M3) = L1 intersect L2
L1 intersect L2 is a CFL
147
Visualization
•Let L1 and L2 be arbitrary CFL’s
•Let M1 and M2 be PDA’s s.t. L(M1) = L1,
L(M2) = L2
•M1 and M2 exist by definition of L1
and L2 are CFL’s and the fact that for
every CFG, there is an equivalent PDA
•Construct PDA M3 from PDA’s M1 and M2
•Argue L(M3) = L1 intersect L2
•There exists a PDA M3 s.t. L(M3) = L1
intersect L2
•L1 intersect L2 is a CFL
L1
L1 intersect L2
L2
CFL
M1
M3
M2
PDA’s
148
Algorithm Specification
• Input
– Two PDA’s M1 and M2
• Output
– PDA M3 such that L(M3) =
PDA M1
PDA M2
A
PDA M3
149
Review Previous Results
150
Underlying Idea
• Previous Results
– recursive languages are closed under set intersection
– r.e. languages are closed under set intersection
– regular languages are closed under set intersection
• What is the idea underlying the constructions used
to prove these previous results?
151
Implementation with FSA’s *
• Given the basic idea underlying these
constructions, how was this idea implemented in
when dealing with FSA’s?
• That is, restate the construction used to prove that
the regular languages are closed under set
intersection.
– Specify the output FSA in terms of the input FSA’s
152
Applying previous approach to
PDA’s
153
Applying approach to PDA’s *
• Given the basic idea underlying these
constructions, try and implement this idea in a
construction working with PDA’s rather than
FSA’s.
• That is, give a construction specifying how the
output PDA is built out of the input PDA’s
154
Problem
• Describe what goes wrong when applying this
idea to PDA’s instead of FSA’s.
• Does this prove that CFL’s are NOT closed under
set intersection?
155
Modified Result *
• What happens if the inputs are 1 FSA and 1 PDA?
• What modified result does the resulting
construction prove?
156
Module 36
• Non context-free languages
– Examples and Intuition
• Pumping lemma for CFL’s
– Pumping condition
– No proof of pumping lemma
– Applying pumping lemma to prove that some
languages are not CFL’s
157
Examples and Intuition
158
Examples
• What are some examples of nonregular languages?
• Can we build on any of these languages to create a
non context-free language?
159
Intuition
• Try and prove that these languages are CFL’s and
identify the stumbling blocks
– Why can’t we construct a CFG to generate this language?
– Why can’t we construct a PDA to accept this language?
– Compare to similar CFL languages to try and identify
differences.
160
Pumping Lemma for CFL’s
161
Comparison to regular language
pumping lemma/condition
162
What’s different about CFL’s than
regular languages? *
• In regular languages, a single substring “pumps”
– Consider the language of even length strings over {a,b}
– We can identify a single substring which can be pumped
• In CFL’s, multiple substrings can “pump”
– Consider the language {anbn | n > 0}
– No single substring can be pumped and allow us to stay
in the language
– However, there do exist pairs of substrings which can be
pumped resulting in strings which stay in the language
• This results in a modified pumping condition
163
Modified Pumping Condition
• A language L satisfies the
regular language pumping
condition if:
– there exists an integer n > 0
such that
– for all strings x in L of length
at least n
– there exist strings u, v, w such
that
•
•
•
•
x = uvw and
|uv| ≤ n and
|v| ≥ 1 and
For all k ≥ 0, uvkw is in L
• A language L satisfies the CFL
pumping condition if:
– there exists an integer n > 0
such that
– for all strings x in L of length
at least n
– there exist strings u, v, w, y, z
such that
• x = uvwyz and
• |vwy| ≤ n and
• |vy| ≥ 1 and
• For all k ≥ 0, uvkwykz is in L
164
Pumping Lemma
• All CFL’s satisfy the CFL pumping
condition
CFL’s
“Pumping Languages”
All languages over {a,b}
165
Pumping
Implications
CFL
• We can use the pumping lemma to prove a
language L is not a CFL
– Show L does not satisfy the CFL pumping
condition
• We cannot use the pumping lemma to prove
a language is context-free
– Showing L satisfies the pumping condition does
not guarantee that L is context-free
166
Pumping Lemma
What does it mean?
167
Pumping Condition
• A language L satisfies the CFL pumping condition if:
– there exists an integer n > 0 such that
– for all strings x in L of length at least n
– there exist strings u, v, w, y, z such that
• x = uvwyz and
• |vwy| ≤ n and
• |vy| ≥ 1 and
• For all k ≥ 0, uvkwykz is in L
168
v and y can
be pumped
1) x in L
2) x = uvwyz
3) For all k ≥ 0, uvkwykz is in L
• Let x = abcdefg be in L
• Then there exist 2 substrings v and y in x such that v and y
can be repeated (pumped) in place any number of times
and the resulting string is still in L
– uvkwykz is in L for all k ≥ 0
• For example
– v = cd and y = f
• uv0wy0z = uwz = abeg is in L
• uv1wy1z = uvwyz = abcdefg is in L
• uv2wy2z = uvvwyyz = abcdcdeffg is in L
• uv3wy3z = uvvvwyyyz = abcdcdcdefffg is in L
• …
169
What the other parts mean
• A language L satisfies the CFL pumping condition if:
– there exists an integer n > 0 such that
• Since we skip this proof, we will not see what n really means
– for all strings x in L of length at least n
• x must be in L and have sufficient length
– there exist strings u, v, w, y, z such that
• x = uvwyz and
• |vwy| ≤ n and
– v and y are contained within n characters of x
– Note: these are NOT necessarily the first n characters of x
• |vy| ≥ 1 and
– v and y cannot both be l,
– One of them might be l, but not both
• For all k ≥ 0, uvkwykz is in L
170
Example
• Let L be the set of palindromes over {a,b}
– Let x = aabaa
– Let n = 3
– What are the possibilities for v and y ignoring the
pumping constraint?
– Which ones satisfy the pumping lemma?
171
Pumping Lemma
Applying it to prove a specific
language L is not context-free
172
How we use the Pumping Lemma
• We choose a specific language L
– For example, {ajbjcj | j > 0}
• We show that L does not satisfy the
pumping condition
• We conclude that L is not context-free
173
Showing L “does not pump”
• A language L satisfies the
CFL pumping condition if:
– there exists an integer n > 0
such that
– for all strings x in L of
length at least n
– there exist strings u, v, w, y,
z such that
• x = uvwyz and
• |vwy| ≤ n and
• |vy| ≥ 1 and
• For all k ≥ 0, uvkwykz is
in L
• A language L does not
satisfy the CFL pumping
condition if:
– for all integers n of
sufficient size
– there exists a string x in L
of length at least n such that
– for all strings u, v, w, y, z
such that
• x = uvwyz and
• |vwy| ≤ n and
• |vy| ≥ 1
– There exists a k ≥ 0 such
that uvkwykz is not in L
174
Example Proof
•
A language L does not satisfy the
CFL pumping condition if:
– for all integers n of sufficient size
– there exists a string x in L of
length at least n such that
– for all strings u, v, w, y, z such that
•
•
•
•
• x = uvwyz and
• |vwy| ≤ n and
• |vy| ≥ 1
– There exists a k ≥ 0 such that
uvkwykz is not in L
Proof that L = {aibici | i>0} does not
satisfy the CFL pumping condition
Let n be the integer from the pumping
lemma
Choose x = anbncn
Consider all strings u, v, w, y, z s.t.
• x = uvwyz and
• |vwy| ≤ n and
• |vy| ≥ 1
•
Argue that uvkwykz is not in L for
some k ≥ 0
– Argument must apply to all possible
u,v,w,y,z
– Continued on next slide
175
Example Proof Continued
•
•
•
•
Proof that L = {aibici | i>0} does not
satisfy the CFL pumping condition
Let n be the integer from the pumping
lemma
Choose x = anbncn
Consider all strings u, v, w, y, z s.t.
• x = uvwyz and
• |vwy| ≤ n and
• |vy| ≥ 1
•
Argue that uvkwykz is not in L for
some k ≥ 0
– Argument must apply to all possible
u,v,w,y,z
– Continued next column
•
•
Identify possible cases for vwy
What is impossible for vwy?
•
Case 1
– vwy contains no a’s
•
Case 2
– vwy contains no c’s
•
Must argue uvkwykz is not in L for
both cases described above
– Can use different values of k
– Continued on next slide
176
Example Proof Continued
• Identify possible cases for vwy
• What is impossible for vwy?
• Case 1: vwy contains no a’s
– vy contains at least 1 b or c
• follows from
• Case 1
– vwy contains no a’s
• Case 2
– vwy contains no c’s
• Must argue uvkwykz is not in L
for both cases described above
– Can use different values of k
– Continued next column
– vwy contains no a’s and
– |vy| ≥ 1
– uwz is not in L
• uwz has n a’s
– follows from fact vwy contains
no a’s and x originally had n
a’s
• uwz has fewer than n b’s or
fewer than n c’s
– follows from vy contains at
least 1 b or c and x originally
only had n b’s and n c’s
• Continued next slide
177
Example Proof Continued
• Case 1: vwy contains no a’s
– vy contains at least 1 b or c
• Case 2: vwy contains no c’s
– vy contains at least
• follows from
– vwy contains no a’s and
– |vy| ≥ 1
– uwz is not in L
• uwz has n a’s
– follows from fact vwy contains
no a’s and x originally had n
a’s
• uwz has fewer than n b’s or
fewer than n c’s
– follows from vy contains at
least 1 b or c and x originally
only had n b’s and n c’s
• Continued next column
– uv2wy2z is not in L
• uv2wy2z has n c’s
– follows from fact vwy contains
no c’s and x originally had n
c’s
• uv2wy2z has more than n a’s or
more than n b’s
– follows from vy contains at
least 1 a or b and x originally
has n a’s and n b’s
• Continued next slide
178
Example Proof Completed
• Case 2: vwy contains no c’s
– vy contains at least
– uv2wy2z is not in L
•
uv2wy2z
has n c’s
– follows from fact vwy
contains no c’s and x
originally had n c’s
• uv2wy2z has more than n a’s
or more than n b’s
– follows from vy contains at
least 1 a or b and x
originally has n a’s and n
b’s
• For all possible u, v, w, y, z, we
have shown there exists a k ≥ 0
such that
– uvkwykz is not in L
• Note, we used a different value
of k for each case (though we
didn’t have to)
• Therefore L does not satisfy the
CFL pumping condition
• There L is not a CFL
• Continued next column
179
Other example languages
• TWOCOPIES = {ww | w is in {a,b}* }
– abbabb is in TWOCOPIES but abaabb is not
• EQUAL3 = the set of strings over {a, b, c}
such that the number of a’s equals the
number of b’s equals the number of c’s
• {aibjck | i < j < k}
180
Pumping Lemma
Two rules of thumb
181
Two Rules of Thumb
• Try to use blocks of at least n characters in x
– For TWOCOPIES, choose x = anbnanbn rather than
anbanb
• Guarantees v and y cannot be in more than 2 blocks of x
• Try k=0 or k=2
– k=0
• This reduces number of occurrences of v and y
– k=2
• This increases number of occurrences of v and y
182
Summary
• We use the Pumping Lemma to prove a
language is not a CFL
– Note, does not work for all non CFL languages
– Can be strengthened to Ogden’s Lemma
• In book
• Choosing a good string x is first key step
• Choosing a good k is second key step
• Typically have several cases for v, w, y
183
Module 37
• Showing CFL’s not closed under set
intersection and set complement
184
Nonclosure Properties for CFL’s
185
CFL’s not closed under set
intersection
• How can we prove that CFL’s are not closed
under set intersection?
186
Counterexample
• What is a possible L1 intersect L2?
– What non-CFL languages do we know?
• What could L1 and L2 be?
– L1 =
– L2 =
– How can we prove that L1 and L2 are contextfree?
187
CFL’s not closed under
complement
• How can we prove that CFL’s are not closed
under complement?
– Another way
• Use fact that any language class which is closed
under union and complement must also be closed
under intersection
188
Language class hierarchy
H
H
Equal
Equal-3
REG
CFL
REC
RE
All languages over alphabet S
189