Recursive descent parsing
26-Jul-16
Some notes on recursive descent

The starter code that I gave you did not exactly fit the
grammar that I gave you


Both work, both are correct
Many things can be coded either recursively or
iteratively

I gave you an iterative grammar and recursive code
Recursive rule for <term>


<term> ::= <factor> <multiply_operator> <term>
public boolean term() {
if (!factor()) return false;
if (!multiplyOperator()) return true;
if (!term()) error("No term after '*' or '/' ");
return true;
}
Iterative rule for <term>


<term> ::= <factor> { <multiply_operator> <factor> }
public boolean term() {
if (!factor()) return false;
while (multiplyOperator()) {
if (!factor()) error("No factor after '*' or '/' ");
}
return true;
}
Parse trees

Every construct (statement, expression, etc.) in a programming
language can be represented as a parse tree
Recognizers and parsers



A recognizer tells whether a given string “belongs to” (is
correctly described by) a grammar
A parser uses a grammar to construct a parse tree from a given
string
One kind of parser is a recursive descent parser

Recursive descent parsers have some disadvantages:




And some advantages:



They are not as fast as some other methods
It is difficult to provide really good error messages
They cannot do parses that require arbitrarily long lookaheads
They are exceptionally simple
They can be constructed from recognizers simply by doing some extra
work—specifically, building a parse tree
Recursive descent parsers are great for “quick and dirty” parsing jobs
The Stack

One easy way to do recursive descent parsing is to
have each parse method take the tokens it needs, build
a parse tree, and put the parse tree on a global stack

Write a parse method for each nonterminal in the grammar



Each parse method should get the tokens it needs, and only those
tokens
 Those tokens (usually) go on the stack
Each parse method may call other parse methods, and expect those
methods to leave their results on the stack
Each (successful) parse method should leave one result on the stack
Example: while statement


<while statement> ::= “while” <condition> <block>
The parse method for a while statement:



Calls the Tokenizer, which returns a “while” token
Makes the “while” into a tree node, which it puts on the stack
Calls the parser for <condition>, which parses a condition and puts it on
the stack


“while” <condition> (“top” is on right)
Calls the parser for <block>, which parses a block and puts it on the stack


Stack now contains:
Stack now contains:
“while” <condition> <block>
Pops the top three things from the stack, assembles them into a tree, and
pushes this tree onto the stack
Recognizing a while statement

// <while statement> ::= “while” <condition> <block>
private boolean whileStatement() {
if (keyword("while")) {
if (condition()) {
if (block()) {
return true;
} else error("Missing '{' ");
} else error("Missing <condition>");
}
return false;
}
Why do you suppose I named this method whileStatement() instead of while() ?
Parsing a while statement



// <while statement> ::= “while” <condition> <block>
private boolean whileStatement() {
if (keyword("while")) {
if (condition()) {
if (block()) {
makeTree(3, 2, 1);
return true;
} else error("Missing '{' ");
} else error("Missing <condition>");
}
return false;
}
This code assumes that keyword(String), condition(), and
block() all leave their results on a stack
On the stack, while = 3, <condition> = 2, <block> = 1
Alternative code


public boolean whileStatement() {
if (keyword("while") && condition() && block()) {
makeTree(3, 2, 1);
return true;
}
return false;
}
No room for an error condition in this code
Alternative code with one message

public boolean whileStatement() {
if (keyword("while")) {
if (condition()) && (block()) {
makeTree(3, 2, 1);
return true;
}
error("Error in \"while\" statement");
}
return false;
}
Simple makeTree() method



After recognizing a while loop, the stack looks
like this:
And I could have written code like this:
private void makeTree() {
Tree right = stack.pop();
Tree left = stack.pop();
Tree root = stack.pop();
root.addChild(left);
root.addChild(right);
stack.push(root);
}
<block>
<condition>
while
while
<condition>
<block>

This code assumes that the root is the third item down, etc., and
that isn’t always the case

I found it more convenient to write more flexible methods
More general makeTree method

private void makeTree(int keyword, int left, int right) {
Tree root = getStackItem(keyword);
Tree leftChild = getStackItem(left);
Tree rightChild = getStackItem(right);
stack.pop();
stack.pop();
stack.pop();
}
root.addChild(leftChild);
root.addChild(rightChild);
stack.push(root);
Parser methods

In the BNF, I have one long definition for <command>


<command> ::=
<move> <expression> <eol>
| "turn" <direction> <eol>
| "take" <object> <eol>
| "drop" <object> <eol>
...
In my code, I broke that into multiple methods

<command> ::= <move command>
| <turn command>
| <take command>
| <drop command>
...
My command() method

public boolean command() {
if (move()) return true;
if (turn()) return true;
if (take()) return true;
if (drop()) return true;
...
return false;
}
Helper methods


I wrote a number of helper methods for the Parser and for the
ParserTest classes
It’s helpful to have methods to build trees quickly and easily




Another useful method is assertStackTop, which is just


private makeTree(String op)
private Tree makeTree(String op, Tree left, Tree right)
Java 5 allows a variable number of arguments, so you could write
private Tree makeTree(String op, Tree... children)
private void assertStackTop(Tree expected) {
assertEquals(expected, parser.stack.peek());
}
Example:

Tree condition = makeTree("=", "2", "2");
assertStackTop(makeTree("if", condition, "list"));
Final comments

You are welcome to use any of this code, but...



...my code is never the last word on anything!
Code can always be improved
While I think my code is generally useful, you always
have to understand it well enough to adapt it to your
particular needs
The End