CS 163
Data Structures
Chapter 10
Symbolic Differentiation
Herbert G. Mayer, PSU
Status 6/11/2015
1
Syllabus
 Problem Assessment
 Rules of First Derivative
 Data Structures and Types
 Make Node
 Copy Tree
 Print Tree
 Build Tree
 Simplify
 Main
 References
2
Problem Assessment
Goal of this assignment is to design, code, and execute symbolic
differentiation of mathematical formulae or equations
These formulae are simple, meaning:





No partial differentiation
Differentiation only w.r.t a single variable
And that variable is pre-defined to be ‘x’
All numeric constants are integer, and of small value, i.e. single
decimal digits; but leave “room” for adding up larger integer values
Goal was not to focus on scanning, i.e. not on lexical analysis
All variable names different from ‘x’ are constant w.r.t.
differentiation toward ‘x’
If two derivatives are needed, then the output of the first can serve
as input to the second differentiation
So all formulae are simple, yet the mathematical problem remains
general and highly interesting
3
Problem Assessment
When applying the rules of differentiation, the result can be
a formula with redundancies, e.g. +0, *1, /1 etc.
In such cases: advisable to simplify the result
Each input formula is terminated by a special symbol, the
‘$’ character
For example:




Input f(x) = 2*x+x*3+5+6*x$
Normalized input f(x) = ((((2*x)+(x*3))+5)+(6*x)) $
First derivative output f’(x) =
(((((0*x)+(2*1))+((1*3)+(x*0)))+0)+((0*x)+(6*1)))
Simplified output: 11
4
Rules of First Derivative Toward x
Function references u and v below are functions of x, i.e. u(x) and v(x)
the ‘ operator symbolizes the first derivative
f(x)
f’(x)
f(x) = some variable != x
f’(x) = 0
f(x) = integer constant
f’(x) = 0
f(x) = x
f’(x) = 1
f(x) = u + v
f’(x) = u’ + v’
f(x) = u - v
f’(x) = u’ - v’
f(x) = u * v
f’(x) = u’*v + u*v’
f(x) = u / v
f’(x)= (u’*v -u*v’) / v2
f(x) = ln(u) = & u
f’(x) = u’ / u
f(x) = u ^ v
f’(x) = u' * v * u ^ ( v - 1 ) + & u * v' * u ^ v
f’(x) = 1 * x * x^(x–1) + & x * 1 * x^x
Example: f(x) = x ^ x
= x^x + & x * x^x
= (& x + 1) * x^x
5
Data Structures and Types
Element of symbolic-differentiation is a node
Each mathematical function f(x) is represented internally as a
binary tree, pointed to by “root”
Root’s type is pointer to structure of node type
Any node is either:



A literal with a stored integer value - single decimal digit for now!
A variable, which could either be the select variable ‘x’ or some
other
An operator, stored as a single character, like ‘+’ ‘*’ ‘/’ ...
Each node has all the following fields, sometimes NOT needed:




Class, specifying enumeration type { Literal, Identifier, Operator }
The single character Symbol, e.g. ‘x’ for the special variable
The integer literal value LitVal to remember the integer value, e.g. 7
And node pointers Left and Right to the respective subtrees
6
Data Structures and Types
// each node has 1 of these class states:
// a Literal, an Identifier (for variable), or an Operator.
// Parenthesized expressions have been reduced
typedef enum { Literal, Identifier, Operator } NodeClass;
typedef struct NodeType * NodePtr;
// forward announcement
// now comes the actual node-type structure,
// using the forward declared pointer type: NodePtr
typedef struct NodeType
{
NodeClass Class;
// 1 of the 3 classes.
char
Symbol;
// store: Identifier, Operator
int
LitVal;
// if Literal, this is its value
NodePtr
Left;
// subtree
NodePtr
Right;
// subtree
} s_node_tp;
7
Make Node
// malloc() new node from heap. All fields are passed in;
// return the pointer to the new node to caller
NodePtr Make( NodeClass Class, char Symbol, int value,
NodePtr Left, NodePtr Right )
{ // Make
NodePtr Node = (NodePtr)malloc( sizeof( struct NodeType ) );
ASSERT( ... node’s space is really there ... );
Node->Class = Class;
Node->Symbol = Symbol;
Node->LitVal = value;
Node->Left
= Left;
Node->Right = Right;
return Node;
} //end Make
8
Copy Tree
// recursively copy tree pointed to by Root.
// return a pointer to the copy to caller
NodePtr Copy( NodePtr Root ) // clever code!!!
{ // Copy
if ( NULL == Root ) {
return NULL;
}else{
return Make( Root->Class, Root->Symbol,
Root->LitVal,
Copy( Root->Left ),
Copy( Root->Right )
);
} //end if
} //end Copy
9
Print Tree
void PrintTree( NodePtr Root )
{ // PrintTree
if ( Root != NULL ) {
if ( Root->Class == Operator ) {
printf( "(" );
} //end if
PrintTree( Root->Left );
if ( Root->Class == Literal ) {
printf( "%d", Root->LitVal ); // prints ints > 9
}else{
printf( "%c", Root->Symbol );
} //end if
PrintTree( Root->Right );
if ( Root->Class == Operator ) {
printf( ")" );
} //end if
} //end if
} //end PrintTree
10
Build Tree for Expression()
Quick BNF Intro (Backus Naur Form)
1. Grammar is a set of rules, defining a language
2. Rules define nonterminals via terminals, nonterminals,
or special symbols (like ‘+’ or ‘;’ or ‘/’)
3. Left side nonterminal : is defined by right side
4. One nonterminal is the start symbol, here Expression
5. : operator separates left from right side
6. | introduces another alternative on r.h.s.
7. { and } group 0 or more phrases on r.h.s.
8. Special symbols are numeric literals (e.g. 6), identifiers
(e.g. y), and char and string literals (e.g. ‘#’)
11
Build Tree for Expression()
Make C program from BNF Grammar
1. For each left nonterminal define a suitable C
function by that nonterminal name
2. For each nonterminal used, call that C function
3. For each terminal used in a phrase in a non-first
position, require that symbol, i.e. error if not found
4. For each terminal at the start of an alternative, see
if this terminal is in the source, then enter that
alternative, else find another alternative
5. Insert needed semantic actions
12
Build Tree for Expression()
Expression
:
Term { plus_op Term }
plus_op
:
‘+’ | ‘-’
Term
:
Factor { mult_op Factor }
mult_op
:
‘*’ | ‘/’
Factor
:
Primary { ‘^’ Primary }
Primary
:
|
|
|
IDENT
LITERAL
‘(‘ Expression ‘)’
‘&’ Primary
13
// start symbol
// for ln()
Build Tree: Expression()
// parse expression and build tree
// using Term() and higher priority functions/ops
// all returning pointers to nodes
// in Expression() handle ‘+’ and ‘-’ operators
NodePtr Expression()
{ // Expression
char Op;
// remember ‘+’ or ‘-’
NodePtr Left = Term(); // handle all higher prior.
while ( NextChar == ‘+’ || NextChar == ‘-’ ) {
Op = NextChar;
// remember ‘+’ or ‘-’
GetNextChar();
// skip Op ‘+’ or ‘-’
// note 0 below for LitVal is just a dummy
Left = Make( Operator, Op, 0, Left, Term() );
} //end while
return Left;
} //end Expression
14
Build Tree: Term()
// multiply operators ‘*’ and ‘/’, later add ‘%’
NodePtr Term( )
{ // Term
char Op;
// remember ‘*’ or ‘/’
NodePtr Left = Factor();
while ( NextChar == ‘*' || NextChar == ‘/' ) {
Op = NextChar;
// remember ‘*’ or ‘/’
GetNextChar();
// skip over Op
// note 0 below for LitVal is just a dummy
Left = Make( Operator, Op, 0, Left, Factor() );
} //end while
return Left;
} //end Term
15
Build Tree: Factor() Left-Assoc.
// exponentiation operator ‘^’ left-associatively
NodePtr Factor()
{ // Factor
NodePtr Left = Primary();
while ( NextChar == ‘^’ ) {
GetNextChar();
// skip over ‘^’
Left = Make( Operator, ‘^’, 0, Left, Primary() );
} //end while
return Left;
} //end Factor
// Think about left- versus right-associativity!!!
// How would you change the code –-and grammar—
// if indeed you make ‘^’ right associative?
16
Build Tree: Factor () Right-Assoc.
// exponentiation operator ‘^’ right-associative
NodePtr Factor()
{ // Factor
NodePtr Left = Primary();
if ( NextChar == ‘^’ ) {
GetNextChar();
// skip over ‘^’
Left = Make( Operator, ‘^’, 0, Left, Factor() );
} //end if
return Left;
} //end Factor
// now multiple ^ operators are handled right-to-left
// in line with common precedence of exponentiation
17
Build Tree: Primary()
NodePtr Primary( )
{ // Primary
char Symbol = NextChar;
NodePtr Temp;
// first_set = { ‘(‘, ‘&’, IDENT, LIT }
GetNextChar();
// skip over current Symbol
if ( IsDigit( Symbol ) ) {
// end node: don’t recurse
return Make( Literal, Symbol, (int)(Symbol-'0’), NULL, NULL );
}else if ( IsLetter( Symbol ) ) {
// also end node: don’t recurse
return Make( Identifier, tolower( Symbol ), 0, NULL, NULL );
}else if ( ‘(‘ == Symbol ) {
Temp = Expression();
Must_Be( ‘)’ );
return Temp;
}else if ( Symbol == '&' ) {
return Make( Operator, '&', 0, NULL, primary() );
}else{
printf( "Illegal character '%c'.\n", Symbol );
return NULL;
} //end if
// impossible to reach! No need to check Herb!!
} //end Primary
18
Derive, 1
// Real action: Derive(Root) derives tree pointed to by Root
// First left, then right subtree
// When done, focus on the Root node
// Both u and v are f(x)
// derive( x )
= 1
-- derive x -> 1
// derive( a )
x -> 0
= 0
-- any variable derived except
// derive( # )
= 0
-- any number derived -> 0
// derive( u + v ) = u' + v'
-- where u = f(x) and v = g(x)
// derive( u - v ) = u' - v'
// derive( u * v ) = u' * v + u * v'
// derive( u / v ) = u' * v - u * v' / ( v * v)
// derive( u ^ v ) = u' * v * u ^ ( v - 1 ) + ln u * v' * u ^ v
// derive( ln u )
= derive( & u )
19
=
u' / u
Derive, 2
NodePtr Derive( NodePtr Root )
{ // Derive
if ( Root == NULL ) {
return NULL;
}else{
switch ( Root->Class ) {
case Literal:
return Make( Literal, '0', 0, NULL, NULL );
case Identifier:
if ( ( Root->Symbol == 'x' ) || ( Root->Symbol == 'X' ) ) {
return Make( Literal, '1', 1, NULL, NULL );
}else{
return Make( Literal, '0', 0, NULL, NULL );
} //end if
case Operator:
switch ( Root->Symbol ) {
case '+': case '-':
return Make( Operator, Root->Symbol, 0,
Derive( Root->Left ), Derive( Root->Right ) );
case '*':
return Make( Operator, '+', 0, . . Next page
20
Derive, 3
case '*':
return Make( Operator, '+', 0,
Make( Operator, '*', 0,
Derive( Root->Left ), Copy( Root->Right ) ),
Make( Operator, '*', 0,
Copy( Root->Left ), Derive( Root->Right ) ) );
case '/':
return Make( Operator, '/', 0, Make( Operator, '-', 0,
Make( Operator, '*', 0,
Derive( Root->Left ), Copy( Root->Right ) ),
Make( Operator, '*', 0,
Copy( Root->Left ), Derive( Root->Right ) ) ),
Make( Operator, '*', 0,
Copy( Root->Right ), Copy( Root->Right ) ) );
case '^':
. . . Next page
21
Derive, 4
case '^':
return Make( Operator, '+', 0,
Make( Operator, '*', 0, Derive( Root->Left ),
Make( Operator, '*', 0, Copy( Root->Right ),
Make( Operator, '^', 0, Copy( Root->Left ),
Make( Operator, '-', 0, Copy( Root->Right ),
Copy( & OneNode ) )
)
)
),
Make( Operator, '*', 0,
Make( Operator, '*', 0,
Make( Operator, '&', 0, NULL, Copy( Root->Left ) ),
Derive( Root->Right ) ),
Make( Operator, '^', 0,
Copy( Root->Left ), Copy( Root->Right ) )
)
);
case '&': . . . Next page
22
Derive, 5
case '&':
if ( Root->Left != NULL ) {
printf( "ln has only one operand.\n" );
} //end if
return Make( Operator, '/', 0,
Derive( Root->Right ), Copy( Root->Right ) );
default:
printf( "Impossible operator.\n" );
return NULL;
} //end switch Root->Symbol
default:
printf( "Unknown Root->Class\n" );
return NULL;
} //end switch Root->Class
23
Opportunities for Simplification
#
1
2
3
4
5
6
7
8
9
10
11
12
13
14
Original expression
x+0
0+x
x-0
x-x
x*0
0*x
x*1
1*x
x/x
x/1
x^1
x^0
1^x
&1
Simplified expression
x
x
x
0
0
0
x
x
1
x
x
1
1
0
24
Simplify, 1
NodePtr Simplify( NodePtr Root )
{ // Simplify
int val = 0;
// accumulate integer values from + - * etc.
if ( !Root ) {
return Root;
}else{
switch ( Root->Class ) {
case Literal:
case Identifier:
return Root;
case Operator:
Root->Left = Simplify( Root->Left );
Root->Right = Simplify( Root->Right );
switch ( Root->Symbol ) {
case '+':
if ( IsLit( '0', Root->Left ) ) {
return Root->Right;
}else if ( IsLit( '0', Root->Right ) ) {
return Root->Left;
}else if ( BothLit( Root->Left, Root->Right ) ) {
val = Root->Left->LitVal + Root->Right->LitVal;
return Make( Literal, (char)( val + '0' ), val,
NULL, NULL );
}else{
return Root; // no other simplifiction for ‘+’
} //end if
. . .
25
Simplify, 2
case '-':
if ( IsLit( '0', Root->Right ) ) {
return Root->Left;
}else if ( BothLit( Root->Left, Root->Right ) ) {
val = Root->Left->LitVal - Root->Right->LitVal;
return Make( Literal, (char)( val + '0' ), val, NULL, NULL );
}else if ( IsEqual( Root->Left, Root->Right ) ) {
return & NullNode;
}else{
return Root;
} //end if
case '*':
if ( IsLit( '1', Root->Left ) ) {
return Root->Right;
}else if ( IsLit( '1', Root->Right ) ) {
return Root->Left;
}else if ( IsLit( '0', Root->Left ) || IsLit( '0', Root->Right ) ) {
return & NullNode;
}else{
return Root;
}//end if
case '/':
if ( IsLit( '1', Root->Right ) ) {
return Root->Left;
}else if ( IsLit( '0', Root->Left ) ) {
return & NullNode;
}else if ( IsEqual( Root->Left, Root->Right ) ) {
return & OneNode;
}else{
return Root;
} //end if
case '^':
if ( IsLit( '0', Root->Right ) ) {
// x^0 = 1
return & OneNode;
}else if ( IsLit( '1', Root->Right ) ) {
// x^1 = x
return Root->Left;
}else if ( IsLit( '1', Root->Left ) ) {
// 1^x = 1
return & OneNode;
}else{
return Root;
} //end if
. . .
26
Two Equal Trees
Students write code for:
bool IsEqual( NodePtr Left, NodePtr Right )
27
Two Equal Trees
// return true only if both subtrees left and right are equal
bool IsEqual( NodePtr Left, NodePtr Right )
{ // IsEqual
if ( ( !Left ) && ( !Right ) ) {
return TRUE;
}else if ( NULL == Left ) {
// Right is known to be not NULL
return FALSE;
}else if ( NULL == Right ) {
// Left is known to be NOT NULL
return FALSE;
}else if ( ( Left->Class == Literal ) && ( Right->Class == Literal ) ) {
return ( Left->LitVal ) == ( Right->LitVal );
}else if ( ( Left->Class == Identifier ) && ( Right->Class == Identifier )){
return ( Left->Symbol ) == ( Right->Symbol );
}else{
// must be Operator; same?
if ( ( Left->Symbol ) == ( Right->Symbol ) ) {
// IsEqual yields true, only if both subtrees are equal
return ( IsEqual( Left->Left, Right->Left ) &&
IsEqual( Left->Right, Right->Right ) ) ||
( is_associative( Left->Symbol ) &&
IsEqual( Left->Left, Right->Right ) &&
IsEqual( Left->Right, Right->Left ) );
}else{
return FALSE;
} //end if
} //end if
printf( "Impossible to reach in IsEqual.\n" );
} //end IsEqual
28
main()
int main ()
{ // main: Differentiation
NodePtr root = NULL;
Initialize();
root = Expression();
VERIFY( ( NextChar == '$'
SHOW( " original
f(x) =
root = Simplify( root );
SHOW( " Simplified f(x) =
root = Derive( root );
SHOW( " derived
f'(x) =
root = Simplify( root );
SHOW( " reduced
f'(x) =
), "$ expected, not found\n" );
", root );
", root );
", root );
", root );
Or else:
print_tree( simplify( derive( simplify( expression( root )))));
return 0;
} //end main: Differentiation
29
References
1. Differentiation rules, implementation code samples:
http://www.codeproject.com/KB/recipes/Differentiation.aspx
2. More code samples in Lisp: http://mitpress.mit.edu/sicp/fulltext/sicp/book/node39.html
30