Numerical Algorithms Group
Mathematics and technology for optimized performance
Optimisation and the
NAG Toolbox for MATLAB
University of Warwick
Mick Pont, John Holden - NAG
Results Matter. Trust NAG.
Agenda…
 NAG licences at Warwick
 NAG Toolbox for MATLAB
 Optimization and other topics
University of Warwick - January 31st 2008
The NAG Toolbox for MATLAB
2
Licences at the University of Warwick
Site Licence for eight implementations:
Site contact: patrick.ryan@warwick.ac.uk
 NAG’s Serial Libraries (Fortran 77, 90, C & DMC)
 REVIEW OF LICENCES UNDERWAY – PROBABLE CHANGES:
 Windows (Fortran and C DLLs)
 32-bit Linux (Fortran-g77 and C-gcc)
 64-bit Linux (Fortran and C)
NAG Toolbox for MATLAB
 Licence upgrade “in progress”
 Maple-NAG Connector
 collaborative licence opportunities – contact NAG direct
University of Warwick - January 31st 2008
The NAG Toolbox for MATLAB
3
What does the University licence cover?
 Unlimited use for the licensed implementations
 As long as for academic or research purposes
 Installation may be on any university, staff or student
machine
 Full access to NAG Support support@nag.co.uk
 Need to use an @xxxx.ac.uk e-mail address
 Our software:
 Includes online documentation - also www.nag.co.uk
 Supplied with extensive example programs
 data and results
University of Warwick - January 31st 2008
The NAG Toolbox for MATLAB
4
Technical Agenda
 The NAG Engine
 Wrapper functionality
 Algorithmic contents
 Navigating around the NAG toolbox in MATLAB
 NAG Optimization Chapters
 Random numbers
University of Warwick - January 31st 2008
The NAG Toolbox for MATLAB
5
The NAG Engine
NAG software is based on NAG Engine technology
NAG C
Library
User-callable
library routines are
thin wrappers
NAG
Toolbox for
MATLAB
University of Warwick - January 31st 2008
NAG Fortran
Library
NAG
Engine
(algorithmic
repository)
NAG SMP
Library
Other NAG
Software
The NAG Toolbox for MATLAB
6
NAG Libraries Ease of Integration
 C++ (various)
 Excel
 C# / .NET
 MATLAB
 Visual Basic
 Maple
 Java
 LabVIEW
 Borland Delphi
 R and S-Plus
 Python
 SAS
…
 Simfit
…
…
 and more
 and more
University of Warwick - January 31st 2008
The NAG Toolbox for MATLAB
7
NAG Toolbox for MATLAB
 Built as MATLAB mex files
 Auto-generated from XML documentation
 Contains essentially all NAG functionality
 not a subset
 Currently runs under Windows (32-bit) or Linux
(32/64-bit)
 Installed under the usual MATLAB toolbox
directory
 Makes use of a DLL or shared version of the NAG
Library
<start MATLAB here>
University of Warwick - January 31st 2008
The NAG Toolbox for MATLAB
8
Chapter e04 – Minimization / Maximization
Problem: minimize F(x1, x2, …, xn)
possibly subject to constraints
The function F(x) is called the objective function.
We wish to determine x, the n-vector of variables.
May have:
 No constraints
 Bound constraints: li <= xi <= ui
 Linear or nonlinear constraints: l <= G(x) <= u
University of Warwick - January 31st 2008
The NAG Toolbox for MATLAB
11
Unconstrained optimization
University of Warwick - January 31st 2008
The NAG Toolbox for MATLAB
12
Linearly constrained optimization
University of Warwick - January 31st 2008
The NAG Toolbox for MATLAB
13
Nonlinear constraints
University of Warwick - January 31st 2008
The NAG Toolbox for MATLAB
14
Chapter e04
Problems categorized according to properties of
objective function:
 nonlinear
 sum of squares of nonlinear functions
 quadratic
 sum of squares of nonlinear functions
 linear
Example – nonlinear objective and constraints:
Minimize f(x,y) = (1 − x)2 + 100(y − x2)2
subject to x2 + y2 <= 2
-2 <= x <= 2
University of Warwick - January 31st 2008
The NAG Toolbox for MATLAB
15
E04WD
 Sequential quadratic programming (SQP)
algorithm
 obtains search directions from a sequence of QP
subproblems.
 designed for problems with many variables and
constraints
 P. Gill (San Diego), W. Murray (Stanford) and M.
Saunders (Stanford)
University of Warwick - January 31st 2008
The NAG Toolbox for MATLAB
16
Chapter e04
It is important to choose a method appropriate to
your problem type, for efficiency and the best
chance of success.
NAG documentation is comprehensive – for advice
see the Chapter Introduction for e04:
www.nag.co.uk/numeric/FL/manual/pdf/E04/e04_intro.pdf
www.nag.co.uk/numeric/CL/nagdoc_cl08/pdf/E04/e04_intro.pdf
<run rosenbrock_sd_demo, rosenbrock_sqp _demo, rosenbrock_lsq
_demo here>
University of Warwick - January 31st 2008
The NAG Toolbox for MATLAB
17
Some routines available in Chapter e04
 e04ab: minimize a function of one variable
 e04dg: minimization using conjugate gradients
 e04mf: linear programming
 e04nc: linear least-squares
 e04nf: quadratic programming
 e04nq: LP or QP (for sparse problems)
 e04un: nonlinear least-squares
 e04vh: general sparse constrained nonlinear
 e04wd: general nonlinear all-purpose
 etc.
University of Warwick - January 31st 2008
The NAG Toolbox for MATLAB
18
New optimization coming at next Mark
Currently many optimization routines in NAG, but
these have all been for local optimization. No
guarantee about which minimum (or maximum) is
returned.
University of Warwick - January 31st 2008
The NAG Toolbox for MATLAB
19
Local optimization
University of Warwick - January 31st 2008
The NAG Toolbox for MATLAB
20
Global requirements
Users often ask for global optimization methods.
In next releases of NAG Libraries we will have
software based on 'multilevel coordinate search'
(MCS) method - Huyer and Neumaier:
http://www.mat.univie.ac.at/~neum/ms/mcs.pdf
Search space is recursively split into sub-boxes, looking for child
boxes where gain in objective is expected. Boxes swept through in
turn, perhaps being split, until a box with maximum level exists. Then
a local search is performed.
Already in NAG Engine - new Chapter e05
Beta available now on request
University of Warwick - January 31st 2008
The NAG Toolbox for MATLAB
21
New NAG Chapter – E05
 Main routine named E05JB
 Plus initialization and option setting routines
 Currently handles only bound constraints:
Minimize f(x1, x2 … xn)
Subject to li <= xi <= ui
<run e05jb_demo here>
University of Warwick - January 31st 2008
The NAG Toolbox for MATLAB
22
Chapter g05 – Random Numbers
Random numbers – used to model real-life
processes.
Humans are bad at choosing them.
 e.g. faking a random sequence of coin tosses is very
difficult: HTHHTHTTHTHHTTTHTTHHTTHTHHTHTH …
 Most people would choose a sequence easily proved
not to be random
 (Do you play the lottery?)
Therefore good algorithms are required.
University of Warwick - January 31st 2008
The NAG Toolbox for MATLAB
23
Chapter g05 – Random Numbers
Random numbers from non-uniform distributions
 Generated from uniform numbers
 Transformation methods
 Rejection methods
 Table search methods
All the likely suspects are there:
 Normal (Gaussian) distribution
 Student’s t distribution
 Beta and exponential
 Chi-squared and binomial
 Geometric, poisson, F, Gamma, …
University of Warwick - January 31st 2008
The NAG Toolbox for MATLAB
24
Some routines available in Chapter g05
 g05ca: pseudo-random from uniform distribution
 g05cb / g05cc: initialize generator to repeatable or
non-repeatable sequence respectively
 g05db: exponential distribution
 g05dd: normal distribution
 g05fe: beta distribution
 g05ff: gamma distribution
 g05ya: quasi-random numbers
 Other distributions, plus GARCH, copula, etc.
University of Warwick - January 31st 2008
The NAG Toolbox for MATLAB
25
Pseudo- versus Quasi-Random Numbers
Pseudo-random numbers:
 generated systematically
 e.g. multiplicative congruential ni = a ni-1 mod m
 properties close to true random numbers
 (assuming that a and m are chosen wisely)
 negligible correlation between consecutive numbers
Quasi-random numbers:
 not statistically independent
 give more even distribution in space (“looks more random”)
 useful for Monte Carlo integration
<run random_demo here>
University of Warwick - January 31st 2008
The NAG Toolbox for MATLAB
26
Other NAG software
We’d welcome contact on our other software:
 Maple-NAG Connector
 NAG's Excel related products
 Fortran Builder (NAG’s Windows Fortran compiler)
 New library functionality
 NAG’s High Performance libraries
 ..
University of Warwick - January 31st 2008
The NAG Toolbox for MATLAB
27
NAG Toolbox for MATLAB - summary
 Runs under Windows and Linux
 Currently on beta test
 Downloadable from NAG web site
 Can be used with MATLAB compiler
 Release version coming soon
Questions?
sales@nag.co.uk or support@nag.co.uk
www.nag.co.uk/about/careers.asp
University of Warwick - January 31st 2008
The NAG Toolbox for MATLAB
28