Basic Parsing with Context-Free Grammars CS 4705 1 Analyzing Linguistic Units • Morphological parsing: – analyze words into morphemes and affixes – rule-based, FSAs, FSTs • Ngrams for Language Modeling • POS Tagging • Syntactic parsing: – identify constituents and their relationships – to see if a sentence is grammatical – to assign an abstract representation of meaning 2 Syntactic Parsing • Declarative formalisms like CFGs, FSAs define the legal strings of a language -- but only tell you ‘this is a legal string of the language X’ • Parsing algorithms specify how to recognize the strings of a language and assign each string one (or more) syntactic analyses • Parsing useful for grammar checking, semantic analysis, MT, QA, information extraction, speech recognition…almost every task in NLP…but… 3 Parsing as a Form of Search • Searching FSAs – Finding the right path through the automaton – Search space defined by structure of FSA • Searching CFGs – Finding the right parse tree among all possible parse trees – Search space defined by the grammar • Constraints provided by the input sentence and the automaton or grammar 4 CFG for Fragment of English S NP VP S Aux NP VP S VP NP Det Nom NP PropN Nom N Nom Nom N Nom Nom PP VP V NP TopD BotUp VP V PP -> Prep NP N book | flight | meal | money V book | include | prefer Aux does Prep from | to | on PropN Houston | TWA Det that | this | a E.g. LC’s 5 Parse Tree for ‘Book that flight’ for Prior CFG S VP NP Nom V Det N Book that flight 6 Rule Expansion S NP VP S Aux NP VP S VP (1) NP Det Nom (3) NP PropN Nom N Nom Nom N (4) Nom Nom PP VP V NP (2) TopD BotUp VP V PP -> Prep NP N book | flight | meal | money V book | include | prefer Aux does Prep from | to | on PropN Houston | TWA Det that | this | a E.g. LC’s 7 Top-Down Parser • Builds from the root S node to the leaves • Assuming we build all trees in parallel: – – – – Find all trees with root S (or all rules w/lhs S) Next expand all constituents in these trees/rules Continue until leaves are pos Candidate trees failing to match pos of input string are rejected (e.g. Book that flight matches only one subtree) 8 Top-Down Search Space for CFG (expanding only leftmost leaves) S NP S NP S VP S Aux NP S VP S VP S S S VP NP VP Aux NP VP Aux NP VP VP VP Det Nom PropN Det Nom PropN V NP V Det Nom N 9 Bottom-Up Parsing • Parser begins with words of input and builds up trees, applying grammar rules whose rhs match – Book that flight N Det N V Det N Book that flight Book that flight – ‘Book’ ambiguous (2 pos appear in grammar) – Parse continues until an S root node reached or no further node expansion possible 10 Two Candidates: One Successful Parse S VP VP V Book NP Det that Nom N flight NP Nom V Det N Book that flight S ~ VP NP 11 What’s right/wrong with…. • Top-Down parsers – they never explore illegal parses (e.g. which can’t form an S) -- but waste time on trees that can never match the input • Bottom-Up parsers – they never explore trees inconsistent with input -- but waste time exploring illegal parses (with no S root) • For both: find a control strategy -- how explore search space efficiently? – Pursuing all parses in parallel or backtrack or …? – Which rule to apply next? – Which node to expand next? 12 A Possible Top-Down Parsing Strategy • Depth-first search: – Agenda of search states: expand search space incrementally, exploring most recently generated state (tree) each time – When you reach a state (tree) inconsistent with input, backtrack to most recent unexplored state (tree) • Which node to expand? – Leftmost or rightmost • Which grammar rule to use? – Order in the grammar? How? 13 Top-Down, Depth-First, Left-Right Strategy • Initialize agenda with ‘S’ tree and ptr to first word (cur) • Loop: Until successful parse or empty agenda – Apply next applicable grammar rule to leftmost unexpanded node (n) of current tree (t) on agenda and push resulting tree (t’) onto agenda • If n is a POS category and matches the POS of cur, push new tree (t’’) onto agenda • Else pop t’ from agenda – Final agenda contains history of successful parse • Does this flight include a meal? 14 Fig 10.7 CFG 15 Left Corners: Top-Down Parsing with Bottom-Up Filtering • We saw: Top-Down, depth-first, L2R parsing – Expands non-terminals along the tree’s left edge down to leftmost leaf of tree – Moves on to expand down to next leftmost leaf… – Note: In successful parse, current input word will be first word in derivation of node the parser currently processing – So….look ahead to left-corner of the tree • B is a left-corner of A if A =*=> Bα • Build table with left-corners of all non-terminals in grammar and consult before applying rule 16 Left Corners 17 Left-Corner Table for CFG Category Left Corners S Det, PropN, Aux, V NP Det, PropN Nom N VP V 18 Left Recursion vs. Right Recursion • Depth-first search will never terminate if grammar is left recursive (e.g. NP --> NP PP) * * ( , ) 19 • Solutions: – Rewrite the grammar (automatically?) to a weakly equivalent one which is not left-recursive e.g. The man {on the hill with the telescope…} NP NP PP (wanted: Nom plus a sequence of PPs) NP Nom PP NP Nom Nom Det N …becomes… NP Nom NP’ Nom Det N NP’ PP NP’ (wanted: a sequence of PPs) NP’ e • Not so obvious what these rules mean… 20 – Harder to detect and eliminate non-immediate left recursion – NP --> Nom PP – Nom --> NP – Fix depth of search explicitly – Rule ordering: non-recursive rules first • NP --> Det Nom • NP --> NP PP 21 An Exercise: The city hall parking lot in town • • • • • • • • • • NP NP NP PP NP Det Nom NP Adj Nom NP Nom Nom Nom NP Nom Nom N PP Prep NP N city | hall | lot | town Adj parking Prep to | for | in 22 Another Problem: Structural ambiguity • Multiple legal structures – Attachment (e.g. I saw a man on a hill with a telescope) – Coordination (e.g. younger cats and dogs) – NP bracketing (e.g. Spanish language teachers) 23 NP vs. VP Attachment 24 • Solution? – Return all possible parses and disambiguate using “other methods” 25 Summing Up • Parsing is a search problem which may be implemented with many control strategies – Top-Down or Bottom-Up approaches each have problems • Combining the two solves some but not all issues – Left recursion – Syntactic ambiguity • Next time: Making use of statistical information about syntactic constituents – Read Ch 12 26