First-class Functions in
Scientific Programming
Steve Johnson
Languages for Science and
Engineering
• Support floating-point operations
• Support vectors and matrices
• Support for big (data) long (time) problems
– Parallel and Vector Dialects
• Influenced by mathematical notation
• Examples
– FORTRAN and its descendents
– ALGOL
– MATLAB, Mathematica, Octave, ...
May 23, 2005
Copyright (c) 2005 The
MathWorks
2
Rich Data Descriptions
•
•
•
•
•
Started with vectors, matrices
Added structures, objects
Operator overloading
Dynamic evaluation and binding
Express a lot of mathematics quite
naturally
May 23, 2005
Copyright (c) 2005 The
MathWorks
3
Functions, but not Notation
• In mathematics, if f and g are functions,
we have a simple notation for the
functions:
f + g (sum)
f * g (product)
f ○ g (composition)
• But in most of these languages, functions
are described imperatively (by giving a list
of instructions to be done)
May 23, 2005
Copyright (c) 2005 The
MathWorks
4
I Will Be Discussing MATLAB
• 20-year old language
– Initially, a front end to Linpack
– Now, a platform for many diverse engineering
applications
• Original language was interpreted, very
dynamic
– No declarations
– Highly Interactive
– Focus on data analysis and visualization
May 23, 2005
Copyright (c) 2005 The
MathWorks
5
MATLAB (continued)
• MATLAB today...
– uses JIT technology for performance
– increasingly used to target embedded and
deployed applications
– Applications go far beyond the numerical
•
•
•
•
May 23, 2005
Biological and Financial modeling
Control theory
Digital Filters
Simulation
Copyright (c) 2005 The
MathWorks
6
Summing Two Functions
(MATLAB 6)
% definition
function y = fsum( f, g, x )
y = feval(f,x) + feval(g,x);
...
% use
v = fsum( @sin, @cos, theta);
May 23, 2005
Copyright (c) 2005 The
MathWorks
7
This isn’t very Compelling
• The functions to be added have to be
towed around and passed to fsum for each
value of x.
• There is no real advantage over writing
v = sin(theta) + cos(theta);
May 23, 2005
Copyright (c) 2005 The
MathWorks
8
Summing Two Functions
(MATLAB 7)
% definition
h = fsum( @sin, @cos );
...
% use
v = h( theta );
May 23, 2005
Copyright (c) 2005 The
MathWorks
9
With Some Work,
You Can Say...
% definition
h = @sin + @cos;
. . .
% use
v = h( theta )
May 23, 2005
Copyright (c) 2005 The
MathWorks
10
The Remainder of Talk
• Discuss “function factories” like fsum and
give some examples of useful ones
• Talk about the language features needed
to make function factories, and show how
to write fsum in MATLAB 7
• Say a bit about implementation issues
May 23, 2005
Copyright (c) 2005 The
MathWorks
11
Often, a Function is Desired
• Interpolation
– Have Data: X, Y vectors
– Want to do an nth degree polynomial
interpolation
• MATLAB 6:
– Call one function to get the coefficients
– Then, to evaluate the function, call another
function with the coefficients and the desired
value.
May 23, 2005
Copyright (c) 2005 The
MathWorks
12
Interpolation in MATLAB 6...
% get coefficients
p = polyfit( X, Y, n );
% p a vector of coeffs.
% evaluate polynomial at x
y = polyval( p, x )
May 23, 2005
Copyright (c) 2005 The
MathWorks
13
Same Problems as fsum
• What you really want from a polynomial
fitting routine is a function:
f = polyfitfun( X, Y, n );
% f a function handle
%
evaluate f at x
y = f(x);
May 23, 2005
Copyright (c) 2005 The
MathWorks
14
Now Consider Spline Interpolation
• Similar to polynomial interpolation
• Returns a structure array instead of coefficients
pp = spline( X, Y );
% pp a structure array
• Evaluation uses another function
y = ppval( pp, x );
• Once again, you would rather have
f = splinefun( X, Y )
May 23, 2005
Copyright (c) 2005 The
MathWorks
15
Unify Interpolations
• Not only are polynomial and spline
interpolation functions more natural, they
have the same shape
– Could generate both, choose between them
– Could write a “meta function” that would
decide which one to generate
– Easy to extend to other fitting methods
• The right notation suggests better ways to
organize the problem
May 23, 2005
Copyright (c) 2005 The
MathWorks
16
Other Generated Functions
• Integral of a function over a range
(numerical quadrature)
– As a function of the range
– As a function of some parameters of the
function
• Derivative / gradient functions
• Functions resulting from model fitting
• Error estimate functions
May 23, 2005
Copyright (c) 2005 The
MathWorks
17
Non-Numeric Examples:
String Matching
%
%
%
%
f = strmatchfun( SS );
SS is an array of strings
use:
ix = f( string );
ix = index of string in SS
... or 0 if not in SS
May 23, 2005
Copyright (c) 2005 The
MathWorks
18
Parsing
• A parser is a function that takes an input
string and produces a parse tree
• A parser-generator such as Yacc takes a
grammar and returns a parser
parser = yacc( grammar );
. . .
tree = parser( input_string );
May 23, 2005
Copyright (c) 2005 The
MathWorks
19
Other Non-numeric Examples
• GUI-building
– An application would often like to call a GUI,
e.g. to find an input file name to open
– GUI-builder could return such a function
• Control and Optimization functions
• Adaptive Simulation
• I/O and data access (see below)
May 23, 2005
Copyright (c) 2005 The
MathWorks
20
Essential Ingredients
• Make a new function, dynamically, with a
handle that can be used to call it
• Provide a place where the new function
stores information supplied when it is
created, and accessed when it is called
• Solution: nested functions
– A function defined inside of another function
– “function factories”
May 23, 2005
Copyright (c) 2005 The
MathWorks
21
fsum Definition
function h = fsum( f, g )
h = @sss;
function y = sss( x )
y = f(x) + g(x);
end
end
May 23, 2005
Copyright (c) 2005 The
MathWorks
22
Nested Semantics
• Values in the parent’s workspace remain
accessible to the child function, even after
the parent returns.
• Each function handle to a child function
invokes a unique instance of the child
function, “customized” by the variables of
the caller instance that created it
h = fsum( @sin, @cos );
hh = fsum( @sinh, @cosh );
May 23, 2005
Copyright (c) 2005 The
MathWorks
23
Shortcut – Anonymous Functions
• Generate simple functions without names, when
all you need is a handle
function h = fsum( f, g )
h = @(x) f(x)+g(x);
end
• Non-parameter variables are evaluated at the
time the anonymous function is created
• Parameters and functions are evaluated at the
time of the call
May 23, 2005
Copyright (c) 2005 The
MathWorks
24
polyfitfun
function f = polyfitfun( X, Y, n )
p = polyfit( X, Y, n );
f = @(x) polyval( p, x );
end
May 23, 2005
Copyright (c) 2005 The
MathWorks
25
More Ornate Example
• Function fcw is called with no arguments
• It returns two functions—one is for
collecting strings, the second for writing
the string out to a file
• The file name need only be supplied at the
time the file is written
May 23, 2005
Copyright (c) 2005 The
MathWorks
26
fcw
function [w,c] = fcw
w = @ws; % write string
c = @wf; % write file
X = { }; % file contents
function ws( str )
X{end+1} = str;
end
function wf( fname )
f = fopen( fname, ‘w’ );
fprintf( f, ‘%s\n’, C{:} );
end
end
May 23, 2005
Copyright (c) 2005 The
MathWorks
27
fcw use
[ ws, wf ] = fcw();
. . .
ws( ‘hello’ );
ws( ‘world’ );
...
wf( ‘greeting’ );
May 23, 2005
Copyright (c) 2005 The
MathWorks
28
fcw Notes
• A practical version of fcw would need error
handling and more attention to efficiency
(preallocate the string array)
• This function is nicely encapsulated
– Could change algorithm in various ways
• Always write to a temp file
• Write files > 1000 lines to a temp file
• Do compression and/or encryption before writing
– Can write multiple files simultaneously
May 23, 2005
Copyright (c) 2005 The
MathWorks
29
Another Application: Callbacks
• Callbacks used in GUIs, event handlers,
watchers, etc.
• GUI callbacks need to understand both the
GUI and the application
– For example, clicking a button may highlight
something on the GUI and set a variable in
the application
– Often leads to two copies of various data
– Nesting callbacks in the application makes
this easy
May 23, 2005
Copyright (c) 2005 The
MathWorks
30
Nested Callbacks
... % in the application
G = make_gui(..., @callbk, )
% G has GUI information
...
function callbk( ... )
% can reference G,
% also application data
...
end
May 23, 2005
Copyright (c) 2005 The
MathWorks
31
First-class Function Notation
• Make a directory @function_handle
• Put in it a file, plus.m, with
function h=plus(f,g)
if isnumeric(f)
h = @(x) f+g(x);
elseif isnumeric(g)
h = @(x) f(x)+g;
else
h = @(x) f(x)+g(x);
end
end
May 23, 2005
Copyright (c) 2005 The
MathWorks
32
Then you can Write
f = @sin + @cos;
g = 1 + @atan;
and so on...
• Similarly, can overload -, *, / and define
function composition
May 23, 2005
Copyright (c) 2005 The
MathWorks
33
Implementation Details
• Nested functions: stack frames
(workspaces) can be referenced by child
functions
• MATLAB 6 environment
– Reference counts used to implement data
sharing, copy-on-write, etc.
– No circular data structures
May 23, 2005
Copyright (c) 2005 The
MathWorks
34
To Support Nesting...
• Need to allow frames (workspaces) to
persist after call returns
• Need to clean up workspaces when they
can no longer be reached
• Need to detect reference circularities to
prevent memory leaks
May 23, 2005
Copyright (c) 2005 The
MathWorks
35
Other Challenging Issues
• Serialization of function handles
– Can store workspaces in .mat files
– When reloaded, what if the function definitions
have changed?
• Debugging, single stepping, etc.
May 23, 2005
Copyright (c) 2005 The
MathWorks
36
Anonymous Implementation
• Uses the same machinery as nested
functions
• The dynamic nature of MATLAB makes it
difficult, at times, to distinguish functions
and variables
May 23, 2005
Copyright (c) 2005 The
MathWorks
37
Future Work
• Improve the efficiency of the
implementation
– (and fix a few corner-case bugs)
• Use nested functions more in our legacy
code base
– But keep customer code working, too!
• Possible scalability features
– “uplevel” keyword to control sharing?
May 23, 2005
Copyright (c) 2005 The
MathWorks
38
Summary
• Functions can now be dynamically
constructed in MATLAB 7
– This changes the way we think about lots of
traditional activities
– Function factories are popular and getting
more so
• It’s a return to our “mathematical” roots
May 23, 2005
Copyright (c) 2005 The
MathWorks
39
Acknowledgements
Thanks to Mike Karr, Steve Eddins, Loren
Shure, and my other MathWorks
colleagues.
May 23, 2005
Copyright (c) 2005 The
MathWorks
40