Pre-Engineering 220
Introduction to MatLab® &
Scientific Programming
J Kiefer
Gottfried W. Leibnitz:
It is unworthy for excellent men to lose hours like slaves in the labour of calculation
which could be safely relegated to anyone else if machines were used.
© 2013
Table of Contents
Table of Contents ............................................................................................................................ 1
I.
Introduction ........................................................................................................................... 3
A.
Numerical Methods or Numerical Analysis ................................................................... 3
1.
Numerical Analysis ..................................................................................................... 3
2.
Newton’s Method for Solving a Nonlinear Equation—an example ........................... 3
3.
Series ........................................................................................................................... 5
4.
Error ............................................................................................................................ 5
B.
Programming................................................................................................................... 6
1.
Program Design .......................................................................................................... 6
2.
Branching .................................................................................................................... 6
3.
Loops........................................................................................................................... 6
4.
I/O ............................................................................................................................... 6
5.
Precision Issues ........................................................................................................... 7
6.
Debugging ................................................................................................................... 7
II. MatLab® ................................................................................................................................ 8
A.
Program Features ............................................................................................................ 8
1.
Commands .................................................................................................................. 8
2.
Arrays ........................................................................................................................ 10
3.
Array Operations ....................................................................................................... 11
B.
Files ............................................................................................................................... 11
1.
m-files ....................................................................................................................... 11
2.
Script files ................................................................................................................. 12
3.
Function files ............................................................................................................ 12
C.
Plots............................................................................................................................... 13
1.
Two Dimensional Graphs (pp. 133-158.................................................................... 13
2.
Three Dimensional Graphs ....................................................................................... 13
D.
Programs ....................................................................................................................... 14
1.
Branches .................................................................................................................... 14
2.
Loops (pp. 190-200).................................................................................................. 16
3.
Input/output (pp 114-118) ......................................................................................... 17
III. Numerical Solution of Nonlinear Equations ....................................................................... 18
A.
Non-Linear Equations—one at a time .......................................................................... 18
1.
The Problem .............................................................................................................. 18
2.
Bisection ................................................................................................................... 18
3.
Newton’s Method or the Newton-Raphson Method ................................................. 19
4.
Secant Method .......................................................................................................... 20
5.
Hybrid Methods ........................................................................................................ 20
B.
Systems of Nonlinear Equations ................................................................................... 21
1.
Newton-Raphson....................................................................................................... 21
2.
Implicit Iterative Methods......................................................................................... 21
IV. Linear Algebra..................................................................................................................... 23
A.
Matrix Arithmetic ......................................................................................................... 23
1.
Matrices..................................................................................................................... 23
2.
Addition & Subtraction ............................................................................................. 23
3.
Multiplication............................................................................................................ 23
1
4.
Inverse Matrix ........................................................................................................... 24
B.
Simultaneous Linear Equations .................................................................................... 25
1.
The Problem .............................................................................................................. 25
2.
Gaussian Elimination ................................................................................................ 25
3.
Matrix Operations ..................................................................................................... 26
4.
Gauss-Jordan Elimination ......................................................................................... 28
C.
Iterative Methods .......................................................................................................... 30
1.
Jacobi Method ........................................................................................................... 30
2.
Gauss-Seidel Method ................................................................................................ 31
D.
Applications .................................................................................................................. 32
1.
Electrical Circuit ....................................................................................................... 32
2.
Truss System ............................................................................................................. 33
V. Interpolation and Curve Fitting ........................................................................................... 34
A.
Polynomial Interpolation .............................................................................................. 34
1.
Uniqueness ................................................................................................................ 34
2.
Newton’s Divided Difference Interpolating Polynomial .......................................... 35
B.
Least Squares Fitting..................................................................................................... 38
1.
Goodness of Fit ......................................................................................................... 38
2.
Least Squares Fit to a Polynomial ............................................................................ 38
3.
Least Squares Fit to Non-polynomial Function ........................................................ 40
MatLab® Sidelight Number One ............................................................................................. 41
1.
Polynomials............................................................................................................... 41
2.
Curve Fitting & Interpolation ................................................................................... 42
VI. Integration ........................................................................................................................... 43
A.
Newton-Cotes Formulæ ................................................................................................ 43
1.
Trapezoid Rule .......................................................................................................... 43
2.
Extension to Higher Order Formulæ......................................................................... 44
B.
Numerical Integration by Random Sampling ............................................................... 47
1.
Random Sampling..................................................................................................... 47
2.
Samples of Random Sampling .................................................................................. 48
3.
Integration ................................................................................................................. 48
MatLab® Sidelight Number Two ............................................................................................ 53
1.
Nonlinear Equations.................................................................................................. 53
2.
Integration ................................................................................................................. 53
VII.
Ordinary Differential Equations ................................................................................... 55
A.
Linear First Order Equations ........................................................................................ 55
1.
One Step Methods ..................................................................................................... 55
2.
Error .......................................................................................................................... 56
MatLab® Sidelight Number Three .......................................................................................... 58
1.
First Order Ordinary Differential Equations (ODE) ................................................. 58
B.
Second Order Ordinary Differential Equations ............................................................ 59
1.
Reduction to a System of First Order Equations ...................................................... 59
2.
Difference Equations ................................................................................................ 60
2
I.
Introduction
A.
Numerical Methods or Numerical Analysis
1.
Numerical Analysis
a.
Definition
“Concerned with solving mathematical problems by the operations of arithmetic.” That is, we
manipulate (  / ,, , etc.) numerical values rather than derive or manipulate analytical
d
mathematic expressions ( ,  dx, e x , x b , ln x , etc.).
dx
We will be dealing always with approximate values rather than exact formulæ.
b.
History
Recall the definition of a derivative in Calculus:
df
f
 lim
 g ( x) ,
dx x0 x
where f  f ( x2 )  f ( x1 ) and x  x2  x1 . We will work it backwards, using
df f

.
dx x
In fact, before Newton and Leibnitz invented Calculus, the numerical methods were the methods.
Mathematical problems were solved numerically or geometrically, e.g., Kepler and Newton with
their orbits and gravity. Many of the numerical methods still used today were developed by
Newton and his predecessors and contemporaries.
They, or their “computers,” performed numerical calculations by hand. That’s one reason it
could take Kepler so many years to formulate his “Laws” of planetary orbits. In the 19th and
early 20th centuries adding machines were used, mechanical and electric. In business, also,
payroll and accounts were done by “hand.”
Today, we use automatic machines to do the arithmetic, and the word computer no longer refers
to a person, but to the machine. The machines are cheaper and faster than people; however, they
still have to be told what to do, and when to do it—computer programming.
2.
Newton’s Method for Solving a Nonlinear Equation—an example
a.
Numerical solution
Let’s say we want to evaluate the cube root of 467. That is, we want to find a value of x such
that x 3  467 . Put another way, we want to find a root of the following equation:
f ( x)  x 3  467  0 .
3
If f(x) were a straight line, then
df x  xo 
x1  xo   0 .
f ( x1 )  f ( xo ) 
dx
In fact, f ( x1 )  0 , but let’s say that f ( x1 )  0
and solve for x1.
f x1   f xo 
f  xo 
.
x1  xo 
 xo 
df ( xo )
f xo 
dx
df ( x  xo )
Note that we are using f ( xo ) 
.
dx
Having now obtained a new estimate for the
root, we repeat the process to obtain a sequence
of estimated roots which we hope converges on
the exact or correct root.
f x1 
x2  x1 
f x1 
f x2 
x3  x 2 
f x2 
etc.
3
In our example, f ( x)  x  467 and f ( x)  3 x 2 . If we take our initial guess to be xo  6 ,
then by iterating the formula above, we generate the following table:
i
0
1
2
3
x1  xo 
xi
6
8.324
7.796
7.759
f ( xi )
-251
109.7718
6.8172
0.108
f  xo 
 251
 6
 8.32407
f x1o 
108
f ( xi )
108
207.8706
182.3316
0.0350
f x1 
109.7768
 8.32407 
 7.79597
f x1 
207.8706
f x2 
6.817273
x3  x2 
 7.79597 
 7.75858
f x2 
182.33156
[Note: The pocket calculator has a (yx) button, but a computer may do x  x  x to get x3.]
x2  x1 
b.
Analytical solution
How might we solve for the cube root of 467 analytically or symbolically? Take logarithms.
x 3  467
3 ln x  ln 467
1
ln x  ln 467
3
4
x
ln 467
e 3 =
7.758402264. . .
We used the (ln) button on our pocket calculator, followed by the (ex) button. In earlier times,
we’d have used log tables. But, whence cometh those tables and how does the calculator
evaluate ln 467 or e2.0488?
3.
Series
x 1 1  x 1
1  x 1
ln x 
 
  
 
x
2 x 
3 x 
2
3
x3 x5 x7



3! 5! 7!
x2 x3 x4
ex  1 x 



2! 3! 4!
The infinite series are exact. However, in practice we always keep a finite number of terms. In
principle, we can achieve arbitrary precision, if we have the necessary patience. Pocket
calculators and computer programs add up enough terms in a series to achieve a specified
precision, say 8 or 16 significant digits.
sin x  x 
4.
Error
In this context, the term error does not refer to a mistake. Rather, it refers to the ideas of
deviation or of uncertainty. Every measured value is uncertain, according to the precision of the
measuring instrument. Every computed value is uncertain, according to the number of
significant digits carried along or according to the number of terms retained in the summation of
a series. Consequently, all numerical solutions are approximate.
Oftentimes, in discussing an example problem, the correct exact solution is known, so it is
possible to determine how an approximate numerical solution deviates from that exact solution.
Indeed, algorithms are often tested by applying them to problems having known exact solutions.
However, in real life, we don’t know the correct exact solution. We can’t know how far our
approximate solutions deviate from the correct, exact, but unknown solution. In other words, we
have to approximate the solution to a problem, but also we can only estimate the error.
Fortunately, we have means of estimating error. A goodly portion of the discussion in a
Numerical Methods textbook is devoted to rigorous estimation of error. In this course, we won’t
concern ourselves with a detailed discussion of error analysis. Nonetheless, we want to be
always aware of the error issue, keeping in mind at least qualitatively the limitations of a
numerical solution. From time to time in the paragraphs that follow some aspects of the error
involved with a particular algorithm will be briefly discussed.
5
B.
Programming
The computer carries out the tedious arithmetic, but it must be told what to do. That is the
function of a computer program. A program may be written in one of any number of
programming languages, however there are certain features or issues that all languages have in
common.
1.
Program Design
a.
Stages
Conception—define the problem
Develop the algorithm—map out or outline the solution
Code—write the program
Debug & verify—trace the program; perform trial runs with known results; correct logical
& syntax errors
b.
Building blocks
Sequential operations—instructions done one after the other in a specified order
Branching operations—selecting alternative sequences of operations
Looping operations—repeating subsets of operations
I/O operations—reading and writing data
2.
Branching
a.
Simple yes or no—select between just 2 alternative actions
b.
Nested branches—a sequence of decisions or branches; decision tree
c.
Select case—more than two alternative actions
3.
Loops
a.
Counted loop—a section of code is executed a specified number of times
b.
Conditional loop—a section of code is iterated until a specified condition is met
c.
“Infinite’ loop—the condition for ending the loop never is encountered, so the program
never ends
4.
I/O
a.
Input—keyboard or data file
b.
Output—monitor, output file, printer; numbers, text, graphics
6
5.
Precision Issues
a.
Binary
The computer does its arithmetic with binary numbers, that is, base-2. E.g., 0, 1, 10, 11, 100,
101, 110, 111, etc. We are accustomed to working and thinking with base-10 numbers. In
producing the machine language code (the “executable”) and carrying out calculations, all
numerical values are translated from base-10 to base-2 then back again for output. Usually, we
don’t need to care about this. However, it can be a source of loss of precision in our numerical
values because the machine stores values with only finite precision.
b.
Precision
A single binary digit (0 or 1) is called a bit. Eight bits make up a byte. Within the machine, the
unit of information that is transferred at one time to/from the CPU and main memory is called a
word. The size of a word, or the word length, varies from one machine to another. Typically,
it’ll be from 4 to 64 bits. A 4-byte word contains 32 bits, etc.
One memory cell or memory location holds one or more words. Let’s say it’s one word, or 4
bytes. Whatever information (number) is stored in one such memory cell must be expressible as
a string of 32 bits and no more. For instance, a non-terminating binary fraction will be
truncated, e.g., (0.1)10 = (0.00011001100110011. . .)2. Only 32 digits will be stored in memory.
When translated back into decimal, the number will be (0.09999997)10, not (0.1)10. Similarly,
the finite precision places a limit on the largest and the smallest numerical value that can be
stored in a memory cell.
In the back of our minds, we always remain aware of the physical limitations of the machine.
6.
Debugging
When syntax errors are all eliminated, the program may very well run smoothly to completion.
Perhaps it produces results which are clearly absurd; perhaps the results appear quite plausible.
A programmer must always take steps to convince itself that the program is working correctly;
the temptation to assume must be resisted.
One of the most insidious assumptions is that the program is doing what the programmer
intended it to do. Perhaps, a typing error has produced a statement that has no syntax error, but
does a different operation from that intended. Perhaps the logical sequence of steps written by
the programmer doesn’t accomplish the task intended by the programmer. This why program
tracing is so important, why it is essential to insert print statements all through the program to
display the intermediate values of variables, why it is essential to check and double check such
things as argument lists and dimensions and the values of indices—checking not what the
programmer intended, but what the program actually does.
The other, almost easier, aspect of debugging involves applying the program to a problem whose
solution is already known. It also involves repeating a numerical solution with different values
of various parameters such as step size and convergence tolerance. It involves comparing a
numerical solution for consistency with previous experience.
7
II.
MatLab®
A. Program Features
Work in MatLab® is done in a variety of windows. The windows used most often are the
Command, Figure, Editor, and Help windows. When the program is started, three windows are
displayed—Command, Current Directory, and Command History windows. The first thing to do
upon starting the program is to select the Desktop Menu, select Desktop Layout, select
Command Window Only.
1.
Commands
a.
Command lines (p. 9)
Commands are entered at the command prompt (>>). When the enter key is pressed, the
command is executed and the output (if any) is displayed at once. All commands are recorded in
the Command History. Results from those previous commands are remembered.
More than one command may be entered on one line, separated by commas. The commands are
executed in order when enter is pressed.
A command can be continued to the next line with an ellipsis followed by enter.
The command history can be accessed with the up and down arrow keys.
Suppress command output--If a command is ended with a semicolon, display of its output (if
any) is suppressed. The product of the command is still available, just not displayed in the
command window.
Comments
Comment lines are started with the % symbol. They are not executed when the enter key is
pressed. A comment may also be attached to the end of a command, before pressing the enter
key.
Clearing the Command Window
The clc command clears the Command Window, but does not erase the command history.
b.
Arithmetic operators (p. 10)
Operation
Addition
Subtraction
Multiplication
Right division
Left division
Exponentiation
Symbol
+
*
/
\
^
precedence
4
4
3
3
3
2
Notice the distinction between right & left division. Left division is right division raised to the
–1 power: 3\5 = 5/3.
8
Expressions enclosed in parentheses are evaluated first. Nested parentheses are executed from
innermost outward.
c.
Built-in functions (pp. 13-16)
Commonly used math functions are built-in. There are the usual sqrt, exp, sin, cos, etc., as
shown in the tables in the text. In addition, there are so-called rounding functions. The
argument, x, may be an expression.
Function
Description
Round(x)
Round to nearest integer
Fix(x)
Round toward zero
Ceil(x)
Round toward infinity
Floor(x)
Round toward –infinity
Rem(x,y)
Remainder of x/y
Sign(x)
Returns the algebraic sign of x: 1, -1, or 0
d.
Scalars
A scalar is a numerical constant, like 5 or –8746 or 45.998, etc. A scalar variable is a name,
really the label of a memory location. A numerical value is stored in a variable. That numerical
value may be changed at any time. A variable name must begin with a letter, but may otherwise
contain letters, digits and the underscore character. There is a limit to how many characters the
name may be, but that varies with the MatLab® version.
Built-in scalar variables: ans, pi, eps = 2^(-52), inf (infinity), and my favorite, NaN (not a
number). The variable ans is used to store the value of an expression or command that has not
been assigned a variable name. Caution! The built-in scalar variables may be reassigned,
whether inadvertently, or advertently.
The values stored in variables are all retained until or unless they are removed from memory
with the clear command. A list of variables presently in memory is obtained with the who or
whos commands.
e.
Assignment operator
Numerical values are “assigned” to a variable name with the assignment operator. The
assignment operator is the = sign, but it does not mean equal to. It means store this value in the
memory location labeled by the specified variable name. Only a single variable name can be on
the left-hand side, while the right-hand side may be a single number or a computable expression
including other, previously defined, variables.
The initial assignment of a value to a variable serves to define that variable. There is no special
declaration of variable types as is seen in some programming languages.
f.
Numerical display formats (p. 12-13)
The format command sets the display format of numerical values. See Table 1-2 in the text.
Basically, the number of digits displayed can be either 4 or 14(15) in either fixed point or
exponential notation.
9
2.
Arrays
An array is a matrix, or rather a matrix is an array of numbers. An n by m matrix has n rows and
m columns. All variables in Matlab® are arrays, even scalars, which are 1x1 arrays.
a.
Vectors
A vector is a one-dimensional array. A row vector has one row and n columns. A row vector is
defined by listing its elements enclosed by square brackets and separated by commas or spaces.
E.g., a three element row vector is defined by A = [a1 , a2 , a3]. Similarly, a column vector is
defined by listing its elements enclosed by square brackets and separated by semicolons. B = [b1
; b2 ; b3] The column vector has one column and m rows.
Alternatively, row vectors may be defined by first element (zi), last element (zf) and the spacing
between the elements (q). Z = [ zi : q : zf ]
The linspace command creates a row vector by specifying the first and last element and the
number of elements. Z = linspace(z1,zf,n)
A character string is stored in MatLab® as a vector, one character to one element. For instance,
B = ‘Now is the time for all’ creates a 23-element vector, as there are 23 characters (including
spaces) in the phrase enclosed in the single quote marks. Each element may be addressed and
altered/replaced/deleted individually.
b.
Two-dimensional arrays
A = [first row ; second row ; third row ; . . .]
The rows can be specified as individual row vectors. The elements can be expressions.
Special arrays are zeros (elements all zero), ones (elements all ones), and eye (the identity
matrix).
The matrix transpose operator is the single quote mark. B = A’  B is the transpose of A.
c.
Addressing matrix elements
Individual elements of an array are referred to by their indices.
A(k) is the kth element of the vector A. B(m,n) is the element in the mth row & nth column.
It may be desirable to address an entire row or column of a matrix, perhaps a subset of a row or
column. In that case a colon (:) is used to indicate a range.
The 3rd through 6th elements of a vector are addressed by A(3:6), etc. Likewise, all the elements
of the mth row of a matrix are addressed by B(m,:). The m through n columns of all the rows of
a matrix B are designated by B(:,m:n).
The most general case would be a block of elements within the matrix—B(m:n,p:q).
10
d.
Adding or deleting matrix elements
It is possible alter the sizes of a previously defined array variables. This done simply by
addressing additional vector(matrix) elements and assigning them values.
Say that A is a 4-element vector. We add elements to the vector by assigning values to the extra
elements. A(5)=5 , A(6)=7 , A97)=-98 , etc. Alternatively, a preexisting vector may be
appended to another. C=[ A B] or C=[G ; H]. Likewise, rows, columns, or entire matrices may
be appended to a matrix. Of course, the dimensions of the added rows, columns, & matrices
must match the matrix being enlarged.
A vector or matrix can be reduced in size, as well, by assigning “nothing” to some the elements,
thusly: B(:,4,9)=[]. This particular example will eliminate all rows from columns 4 – 9.
e.
Built in array manipulations
Some common array handling functions are built-in. These are listed on pages 41 – 43 of the
text.
3.
Array Operations
a.
Matrix operations
Arrays are multiplied, divided, added, subtracted, etc. according to the usual rules of matrix
arithmetic.
Inverse A-1 = A^-1
Left & right division X = A-1B = A\B
X = DC-1 = D/C
Left and right division arise because matrix multiplication is not commutative.
b.
Element by element operations
There exist also what are called element-by-element operations. In that case, an operation is
carried out on every element of an array. A period is added in front of the math operator to
indicate element-by-element operation. E.g., .* or .^
Notice that a dot product between two vectors can be carried out by an element-by-element
multiplication: sum(A.*B) = a1b1 + a2b2 + a3b3 + . . .
c.
Analyzing arrays
The built-in array functions are listed in Table 3-1, pages 64-65. These include Inv and Det.
B.
Files
1.
m-files
MatLab® commands can be stored in a plain text file, and then “run” in the Command window.
The general term for a series of commands is a script. Writing such a series of commands is
called scripting. In MatLab®, script files are saved with the extension .m, hence the term m-files.
11
The m-file may be created & edited in any plain text editor, such as Notepad, or by any word
processing program that is capable of storing plain text. There is also an Edit Window in
MatLab® itself.
a.
Editor
Script files are created, edited, saved, and run in the Edit window.
b.
I/O (pp 95-117)
Input
Assign variables in the Command window before running the script.
Use the Input command or function within the script to interactively enter data.
Variable = input(‘message string’)
Output
Disp—writes to the workspace
Fprintf—allows formatting of the printed line(s).
2.
Script files
a.
Running
Run by entering the file name at the prompt in the Command window.
Run by pressing the run button in the Editor window
In either case, commands previously issued and variables previously defined in the Command
window are known to the script file.
b.
Comments & documentation
There must be comments throughout a script file describing the purpose of the script, defining
the variables used, describing the required input, etc. The purpose of the documentation is to
make plain what is happening in the script to yourself or another programmer at some later date,
not to mention to the instructor. Get in the habit early of over-commenting your scripts.
c.
Inline & feval
These are commands to create one-liners.
Functioname = inline(‘math expression as character string’)
x = functioname(arguments)
variable = feval(‘function name’,argument value)
3.
Function files
function command
A function file differs from the general script file in that it is self-contained. Variables assigned
in the work space (Command window) are not available inside the function file in general.
Likewise, variables assigned within the function file are not available outside the function file.
Variables have to be assigned inside the function file, or passed via the argument list in the
function statement, or of course by input commands. The first line of a function file is
function[arguments-out] = functioname(arguments-in)
12
Typically, the function is saved in the file functioname.m; that is, the file name is the same as the
function. The function is invoked by entering the functioname(arguments-in)
Data can be passed to the function through global variables, the argument-in list, and through
input commands within the function, as well as xlsread commands.
The function produces output through disp, fprintf, and plot commands within the script, or
through the arguments-out list.
It is possible to define variables to be global variables by including the Global command in all
script files, and the Command window as well.
Global variable list
C.
Plots
1.
Two Dimensional Graphs (pp. 133-158
a.
Line plots
Executing the plot or the fplot command automatically opens a Figure Window.
Plot(X,Y) – plots Y vs X, where X & Y are vectors of the same length. If no other parameters
are specified, the graph is plotted in a bare-bones fashion, with a line connecting the data points,
but no axis titles, or data point symbols, etc. The axes are scaled over the intervals spanned by
the vectors X & Y.
However, there are parameters within the plot command as well as additional commands whose
purpose is to change the format of the graph. A graph can be formatted interactively within the
Figure Window, as well.
For plotting a function, there is the command fplot(‘function’,xmin,xmax,ymin,ymax). The
function, y = f(x), is entered as a character string, as in ‘45*cos(3*x^3)’. The drawback of fplot
is that the f(x) cannot include variable names, only the dummy variable.
b.
Other plots
There are available other plotting commands that produce log graphs, bar graphs, pie charts, etc.
c.
Multiple graphs
It is possible to graph several curves on the same plot, using the Hold On and Hold Off
commands. Alternatively, it is possible to create several separate graphs on a single page with
the Subplot command.
2.
Three Dimensional Graphs
a.
Line plots (p 323)
Plot3(X,Y,Z)
13
This one is intended to plot X(t), Y(t), & Z(t) all as functions of a fourth parameter, t.
b.
Surface plots (pp 324-330)
Mesh(X,Y,Z) or Surf(X,Y,Z)
These commands plot Z(X,Y). The mesh command creates a wire-grid surface, while the surf
command adds color shading to the surface. There are variations of mesh & surf that produce
surface graphs of differing appearance—meshz, meshc, surfc, etc.
c.
Contour plots (p 330)
Contour(X,Y,Z,n) and variations.
d.
Special graphics (p 331)
Bar3(Y)
Sphere or [X,Y,Z]=Sphere(n) – produces a set of (X,Y,Z) to be used by mesh or surf to plot a
sphere.
[X,Y,Z]=Cylinder(r) – produces a set of points to be used by mesh or surf to draw a cylinder. r
is a vector that specifies the profile of the cylinder. r = some f(t)
e.
view command
The View command alters the angle at which a 3-d plot is viewed, by specifying the azimuth and
elevation angles of the view point.
View(az,el), with az and el specified in degrees, relative to the xz-plane and the xy-plane,
respectively.
D.
Programs
MatLab® has many built-in functions and computing tools. Nonetheless, it becomes necessary to
write a special-purpose solution for a specific problem. No one commercial computing package
can address every possible situation, and no one lab can have every commercial product on hand.
Previously, we have used assignment statements to carry out calculations, and plot commands to
produce graphical output. Computer programs require also statements to make decisions, to
make comparisons and to carry out repetitive operations, not to mention input and output.
1.
Branches
a.
Relational & logical operators (p.174)
Operator
Description
<
Less than
>
Greater than
<=
Less than or equal to
>=
Greater than or equal to
==
Equal to**
~=
Not equal to
*The equal to operator consists of two equal signs, with no space between them.
14
If two numbers are compared, the result is 1 (logical true) or 0 (logical false). Comparing two
scalars yields a scalar 1 or 0. Arrays are compared element-by element. The result is a logical
array of 1s and 0s. Evidently, the two arrays must be the same size if they are to be compared
with each other. Similarly, a scalar is compared with an array element-by element, and the result
is logical array of 1s and 0s. The elements of logical arrays can be used to address elements in
ordinary arrays. Since the relational comparisons produce numerical values, relational operators
can be used within mathematical expressions. In mathematical expressions, the relational
operators are evaluated after all mathematical operators.
Logical operators
Operator
&
A&B
|
A|B
~
~A
Description
A AND B
=true if both A and B are
true, false otherwise
A OR B
=true if A or B is true,
false if both are false
NOT A
=true if A is false, false
if A is true
See the order of precedence on page 178. Notice that NOT comes after exponentiation and
before multiplication, etc., but that the other logical operators (AND, OR) come last.
There are a number of built-in logical functions, described on pages 179 – 180.
b.
If (pp. 182-190)
The IF statement is used to select between two courses of action. Several IF statements may be
nested to create a binary decision tree.
The decision is based on the truth or falsity of a statement or conditional expression. A
conditional expression is an expression consisting of relational and/or logical operators. The
expression will have the value true or false.
i.
if-end  a block of commands is executed if the conditional expression is true,
skipped if it’s false.
if conditional expression
Matlab® commands
end
ii.
if-else-end  in this case, there are two blocks of MatLab® commands—one is
executed if the conditional expression is true, the other if it is false.
if conditional expression
MatLab® commands
else
Matlab® commands
15
end
iii. if-elseif-else-end  using two conditional expressions, one of three sets of Matlab®
commands is executed.
if conditional expression
MatLab® commands
elseif conditional expression
MatLab® commands
else
Matlab® commands
end
c.
Case
If we desire to select from among more than 2 or 3 cases, then it may be more convenient to use
the switch-case statement.
switch switch expression
case value1
MatLab® commands
case value2
MatLab® commands
case value3
MatLab® commands
etc.
otherwise
MatLab® commands
end
The switch expression is a scalar or string variable or an expression that can take on the values
value1, value2, value3, etc. If none of the specified values occur, then the block following the
otherwise command is executed. The otherwise command is optional.
2.
Loops (pp. 190-200)
Another thing we want a computer program to do automatically is to repeat an operation.
a.
Counting
The for-end loop executes a block of MatLab® commands a specified number of times.
for k = f:s:t
MatLab® commands
end
The loop executes for k = f, f+s, f+2s, f+3s, . . ., t. The increment, s, may be omitted in which
case it is assumed to be 1.
b.
Conditional
Alternatively, a loop may be executed as long as a conditional expression remains true.
while conditional expression
16
MatLab® commands
end
The variables in the conditional expression must have initial values assigned, and at least one of
the variables must be changed within the loop.
3.
Input/output (pp 114-118)
a.
File input
variable = xlsread(‘filename’,’sheetname’,’range’)import data from an Excel spreadsheet
b.
Import Wizard.
The Import Wizard is invoked by selecting Import Data in the File Menu.
c.
File output
fprintf--writes to a plain text disk file fprint(fid,arguments)
fid=open(‘filename’)
fclose(fid)
xlswrite(‘filename’,’sheetname’,’range’,variablename)--export to an Excel spreadsheet
17
III. Numerical Solution of Nonlinear Equations
A.
Non-Linear Equations—one at a time
There are closed form solutions for quadratic and even 3rd degree polynomial equations. Higher
degree polynomials can sometimes be factored. However, in general there is no closed form
analytical solution to non-linear equations.
1.
The Problem
a.
Roots & zeroes
We seek to find x such that f ( x)  0 or
perhaps such that f ( x)  g ( x) . In the
latter case, we merely set
h( x)  f ( x)  g ( x)  0 . We are looking
for a root of the equation f ( x)  0 or a
zero of the function f(x).
b.
Graphical solution
Plot f(x) vs. x—observe where the graph
crosses the x-axis or plot f(x) and g(x) vs.
x and observe where the two curves intersect. A graph won’t give a precise root, but we can use
the graph to choose an initial estimate of the root.
2.
Bisection
a.
Setup
For brevity, say fo = f(xo) and f1 = f(x1), etc.
Say further that x   is the desired root.
The graph shows us that f o  f1  0 because
f(x) crosses the x-axis between [xo,x1].
b.
Algorithm
Let us find the midpoint of [xo,x1], and call it
b.
x  x1
i) b  o
and then f b  f (b)
2
ii) Does f b  0 ? If so, quit ‘cause   b .
iii) If not, then
if f o  f b  0 , then set xo  b and f o  f b
or
if f b  f o  0 , then set instead x1  b and f1  f b .
18
xo  x1
.
2
v) If not, then repeat beginning with step (i).
iv) Is x1  xo   ? If so, quit and set  
It is well also to count the iterations and to place a limit on the number of iterations that will be
performed. Otherwise, the program could be trapped in an infinite loop. Also, it is well to test
for the cases f o  f b  0 and f1  f b  0 . It may be that the function does not cross the x-axis
between fo and f1, or crosses more than once.
3.
Newton’s Method or the Newton-Raphson Method
a.
Taylor’s series
Any well-behaved function can be expanded in a Taylor’s series:
f ( xo )
f ( xo )
f ( x)  f ( xo )  ( x  xo ) f ( xo )  ( x  xo ) 2
 ( x  xo ) 3
 .
2!
3!
Let’s say that x is “close” to xo and keep just the first two terms.
f ( x)  f ( xo )  ( x  xo ) f ( xo )
We want to solve for x such that f(x) = 0.
f ( xo )  ( x  xo ) f ( xo )  0
f ( xo )
f ( xo )
In effect we have approximated f(x) by a straight line; x is the intercept of that line with the xaxis. It may or may not be a good approximation for the root  .
x  xo 
b.
Algorithm
i) choose an initial estimate, xi
ii) compute f(xi) and f ( xi )
iii) compute the new estimate:
f ( xi )
xi 1  xi 
f ( xi )
iv) return to step (ii) with i = i + 1
c.
Comments
It turns out that if the initial estimate of the
root is a good one, then the method is
guaranteed to converge, and rapidly. Even if
the estimate is not so good, the method will
converge to a root—maybe not the one we
anticipated.
% Script to implement Newton's Method
del=1
x=3
k=0
func=inline('x^3-587')
deriv=inline('3*x^2')
while del >= 0.00005
k=k+1
if k <= 10
xnew=x-func(x)/deriv(x);
del=abs((xnew-x)/x);
disp(x);disp(func(x));disp(del)
x=xnew;
end
end
Also, if there is a f   0 point nearby the method can have trouble. It’s always a good thing to
graph f(x) first.
19
4.
Secant Method
a.
Finite differences
A finite difference is merely the difference between two numerical values.
x  x2  x1 or x  xi 1  xi
Derivatives are approximated by divided differences.
f ( xi 1 )  f ( xi ) f
f ( x) 

xi 1  xi
x
We may regard this divided difference as an estimate of f  at xi or at xi+1 or at the midpoint
between xi and xi+1.
b.
The Secant method
We simply replace f  by the divided difference in the Newton-Raphson formula:
xi  xi 1
.
xi 1  xi  f ( xi )
f ( xi )  f ( xi 1 )
Notice the indices: i + 1, i, i – 1. With the Secant Method, we don’t use a functional form for
f  . We do have to carry along two values of f, however.
Care must be taken that f ( xi )  f ( xi 1 ) not be too small, which would cause an overflow error
by the computer. This may occur if f ( xi )  f ( xi 1 ) due to the finite precision of the machine.
This may also give a misleading result for the convergence test of f ( xi )  f ( xi 1 ) . To avoid
that, we might use the relative deviation to test for convergence.
f ( xi )  f ( xi 1 )

f ( xi )
c.
Compare and contrast
Both the Newton-Raphson and Secant Methods locate just one root at a time.
Newton: requires evaluation of f and of f  at each step; converges rapidly.
Secant: requires evaluation only of f at each step; converges less rapidly.
5.
Hybrid Methods
A hybrid method combines the use in one program of two or more specific methods. For
instance, we might use bisection to locate a root roughly, then use the Secant Method to compute
the root more precisely. For instance, we might use bisection to locate multiple roots of an
equation, then use Newton-Raphson to refine each one.
20
B.
Systems of Nonlinear Equations
Consider a system of n nonlinear equations with n unknowns.
f1 ( x1 , x2 , x3 ,, xn )  0
f 2 ( x1 , x2 , x3 ,, xn )  0

f n ( x1 , x2 , x3 ,, xn )  0
1.
Newton-Raphson
a.
Matrix notation
Let’s write the system of equations as a matrix equation.
 f1 
 f 
f   2  0

 
 fn 
 x1 
 
  x2 
The unknowns form a column matrix also. x 
. We might write the system of equations

 
 xn 
 
compactly as f ( x )  0 .
b.
The Method
The Newton-Raphson method for simultaneous equations involves evaluating the derivative


f
matrix, F , whose elements are defined to be Fij  i . If the inverse F 1 exists, then we can
x j
generate a sequence of approximations for the roots of functions {fi}.
 



xk 1  xk  F 1 ( xk )  f ( xk )

At each step, all the partial derivatives must be evaluated and the F matrix inverted. The
iteration continues until all the f i  0 . If the inverse matrix does not exist, then the method
fails. If the number of equations, n, is more than a handful, the method becomes very
cumbersome and time consuming.
2.
Implicit Iterative Methods
The Newton-Raphson method is an iterative method in the sense that it generates a sequence of
successive approximations by repeating, or iterating, the same formula. However, the term
iterative method as commonly used refers to a particular class of algorithms which might more
descriptively be called implicit iterative methods. Such algorithms occur in many numerical
contexts as we’ll see in subsequent sections of this course. At this point, we apply the approach
to the system of simultaneous nonlinear equations.
21
a.
General form
 1 
 
 
 
  2 
Let  
be the solution matrix to the equation f ( x )  0 . I.e., f (  )  0 . Now, solve
 
 
 n 



algebraically each f i ( x )  0 for xi. This creates a new set of equations, xi  Fi ( x  ) , where x 
refers to the set of unknowns {xj} excluding xi. Algebraically, this looks funny, because each
unknown is expressed in terms of all the other unknowns, hence the term implicit. Of course,
what we really mean is
 

xk 1  F( xk ) .
Alternatively, in terms of matrix elements, the equations take the form
xi ,k 1  Fi ( x1,k , x 2,k , x n,k ) .
b.
Algorithm
In a program, the iterative method is implemented thusly:

i) choose an initial guess, x o
 

ii) compute x1  F( xo )
 
iii) test f ( x1 )  0
 
iv) if yes, set   x1 and exit
 

v) if no, compute x 2  F ( x1 ) , etc.
c.
Convergence


We hope that lim x k   . For what conditions will this be true? Consider a region R in the
k 
space of {xi} such that x j   j  h for 1  j  n and suppose that for x in R there is a positive

n
Fi ( x )

  . Then, it “can be shown” that if x o lies in R, the iterative
number  such that 
x j
j 1

method will converge. What does this mean, practically? It means that if the initial guess, x o , is


“close enough” to  , then the method will converge to  after some number, k, of iterations.
Big deal.
22
IV.
Linear Algebra
A.
Matrix Arithmetic
The use of matrix notation to represent a system of simultaneous equations was introduced in
section III-B-1 above, mainly for the sake of brevity. In solving simultaneous linear equations,
matrix operations are central. There follows, therefore, a brief review of the salient properties of
matrices. Fuller discussion of the properties of matrices may be found in various texts,
particularly Linear Algebra texts.
1.
Matrices
A matrix is an n x m array of numbers. In these notes a matrix is symbolized by a letter with a
line on top, B ; n is the number of rows and m is the number of columns. If n = m, the matrix is
said to be a square matrix. If the matrix has only one column(row) it is said to be a column(row)
matrix. The jth element in the ith row of a matrix is indicated by subscripts, bij. Mathematically,
an entity like a matrix is defined by a list of properties and operations, for instance the rules for
adding or multiplying two matrices. Also, matrices can be regarded as one way to represent
members of a group in Group Theory.
b
b13 b14 
b
 x1 
  11 12
  

B  b21 b22 b23 b24 
x   x2 
b31 b32 b33 b34 
 x3 
2.
Addition & Subtraction
a.
Definition
The addition is carried out by adding the respective matrix elements.
  
C  A B
cij  aij  bij
b.
Rules
The sum of two matrices is also a matrix. Only matrices having the same number of rows and
the same number of columns may be added. Matrix addition is commutative and associative.
   
 
 
 
A B  B A
( A B)C  A(B C )
3.
Multiplication
a.
Definition
 
C  AB
cij   aik bkj  ai1b1 j  ai 2 b2 j  ai 3b3 j  
k
b.
Rules

The product of two matrices is also a matrix. The number of elements in a row of A must equal

the number of elements in a column of B . Matrix multiplication is not commutative.
23
  
AB  B A
A matrix may be multiplied by a constant, thusly: cij  q  aij . The result is also a matrix.
4.
Inverse Matrix
a.
Unit matrix
The unit matrix is a square matrix with the diagonal elements equal to one and the off-diagonal
elements all equal to zero. Here’s a 3x3 unit matrix:
1 0 0
 
U  0 1 0
0 0 1
b.
Inverse



  
The inverse of a matrix, B , (denoted B 1 ) is a matrix such that BB 1  B 1 B  U . The inverse
of a particular matrix may not exist, in which case the matrix is said to be singular.
The solution of a system of simultaneous equations in effect is a problem of evaluating the
inverse of a square matrix.
24
B.
Simultaneous Linear Equations
1.
The Problem
a.
Simultaneous equations
We wish to solve a system of n linear equations in n unknowns.
b11x1  b12 x2  b1n xn  c1
b21x1  b22 x2  b2n xn  c2

bn1 x1  bn 2 x2  bnn xn  cn
where the {bij} and the {ci} are constants.
b.
Matrix notation
The system of equations can be written as a matrix multiplication.
 x1 
 c1 
b11
x 
c 
 
 b


Bx  c , where x   2  , c   2  and B   21

 

 

 
c n 
bn1
 xn 
When n is small ( n  40 , say) a direct or one-step method is used.
methods are preferred.
2.
 b1n 
b22  b2 n 
.
   

bn 2  bnn 
For larger systems, iterative
b12
Gaussian Elimination

In a one-step approach, we seek to evaluate the inverse of the B matrix.
 
Bx  c
    
B 1 Bx  x  B 1c
 
The solution is obtained by carrying out the matrix multiplication B 1c .
a.
Elimination
You may have seen this in high school algebra. For brevity’s sake, let’s let n = 3.
b11x1  b12 x2  b13 x3  c1
b21x1  b22 x2  b23 x3  c2
b31x1  b32 x2  b33 x3  c3
In essence, we wish to eliminate unknowns from the equations by a sequence of algebraic steps.
normalization
reduction
b21
and add to eqn. 2; replace eqn. 2.
b11
b
ii) multiply eqn 1 by  31 and add to eqn. 3; replace eqn. 3.
b11
i) multiply eqn. 1 by 
25
b11x1  b12 x2  b13 x3  c1
 x2  b23
 x3  c2
b22
 x2  b33
 x3  c3
b32

b32
and add to eqn. 3; replace eqn. 3.

b22
b11x1  b12 x2  b13 x3  c1
 x2  b23
 x3  c2
b22
 x3  c3
b33
We have eliminated x1 and x2 from eqn.3 and x1 from eqn. 2.
iii) multiply eqn. 2 by 
back substitution iv) solve eqn. 3 for x3, substitute in eqn. 2 & 1.
solve eqn. 2 for x2, substitute in eqn. 1.
solve eqn. 1 for x1.
b.
Pivoting
Due to the finite number of digits carried along by the machine, we have to worry about the
relative magnitudes of the matrix elements, especially the diagonal elements. In other words, the

inverse matrix, B 1 may be effectively singular even if not actually so. To minimize this
possibility, we commonly rearrange the set of equations to place the largest coefficients on the
diagonal, to the extent possible. This process is called pivoting.
e.g.
37x2 – 3x3 = 4
19x1 – 2x2 + 48x3 = 99
7x1 + 0.6x2 +15x3 = -9
rearrange
19x1 – 2x2 + 48x3 = 99
37x2 – 3x3 = 4
7x1 + 0.6x2 +15x3 = -9
or
7x1 + 0.6x2 +15x3 = -9
37x2 – 3x3 = 4
19x1 – 2x2 + 48x3 = 99
3.
Matrix Operations
In preparation for writing a computer program, we’ll cast the elimination and back substitution in
the form of matrix multiplications.
a.
Augmented matrix
b
b
c 
b

   11 12 13 1 
A  B : c  b21 b22 b23 c2 
b31 b32 b33 c3 


26
b.
Elementary matrices
Each single step is represented by a single matrix multiplication.
The elimination steps:
 1
  b21
S1   
 b11
 0
0 0

1 0

0 1

 1

S2   0
 b
 31
 b11

0 0
1 0

0 1


1
0
 
S3  0
1

b
0  32

b22


0
0

1

b
c
b b
     11 12 13 1 
 b23
 c2
S 3 S 2 S1 A   0 b22

 c3 
 0 0 b33
The first back substitution step:

1 0
 

Q1  0 1

0 0


0 
0 
1 

 
b33
b
b
b
c
      11 12 13 1 
 b23
 c 2 
Q1 S 3 S 2 S1 A   0 b22
 0 0 1 x3 


This completes one cycle. Next we eliminate one unknown from the second row using
0 
1 0


 
S 4  0 1  b23

0 0
1 
b
b
c
b
       11 12 13 1 
 0 c2
S 4 Q1 S 3 S 2 S1 A   0 b22

 0 0 1 x3 
1 0


1
Q 2  0

 b22
0 0
b
        11
Q2 S 4 Q1 S 3 S 2 S1 A   0
 0
27
0

0

1
b12 b13
c1 
1 0 x2 
0 1 x3 
This completes the second cycle. The final cycle is
1
b

 11
Q3   0
0


1 0 0 x1 
         
Q3 S 6 S 5 Q2 S 4 Q1 S 3 S 2 S1  0 1 0 x2 
0 0 1 x3 
1 0  b13 
 
S 5  0 1
0 
0 0
1 
1  b12

S 6  0
1
0
0
0
0
1

0 0

1 0
0 1


        

We identify the inverse matrix B 1  Q3 S 6 S 5Q2 S 4 Q1 S 3 S 2 S1 . Notice that the order of the matrix
multiplications is significant. Naturally, we want to automate this process, and generalize to n
equations.
4.
Gauss-Jordan Elimination
a.
Inverse matrix
We might multiply all the elementary matrices together before multiplying by the augmented

 
matrix. That is, carry out the evaluation of B 1 , then perform B 1 A .
b.
Algorithm
a kjk

a kjk 1
k 1
a kk
aijk  aijk 1  aikk 1  a kjk

i  k

i  k 
k  1, n
i  1, n
j  k, n  1
n = number of equations
k = index of the step or cycle
aij = elements of the original

augmented matrix, A .
For each value of k, do the i = k line
first.
% Script to implement Gauss-Jordan Elimination
b=[4 1 2;1 3 1;1 2 5];
c=[16;10;12];
a=[b c];
np=size(b);
n=np(1);
for k=1:n
for m=k+1:n+1
a(k,m)=a(k,m)/a(k,k);
end
a(k,k)=1;
for l=1:n
if l~=k
for m=k+1:n+1
a(l,m)=a(l,m)-a(l,k)*a(k,m);
end
a(l,k)=0;
end
end
end
a
28
c.
Example
n = 3 and n + 1 = 4
4 x1  x2  2 x3  16
x1  3x2  x3  10
x1  2 x2  5x3  12
4 1 2 16
 
A   1 3 1 10 
1 2 5 12 
k=0
e.g., for k = 1, i = 1, j = 1 & j = 4
1
a11

0
a11
0
a11
1
1
a14

0
a14
0
a11

16
4
4
0
0 1
a121  a 21
 a 21
a11  1  1  1  0
k=1
k=2
k=3

1
 
A  0

0

1
4
11
4
7
4
1

4
2

1
6
2

9
8

2
5 38 

 1 0 11 11 


2 24 

A   0 1
11 11 

0 0 46 46 

11 11 
1 0 0 3

A  0 1 0 2
 0 0 1 1
 3
x  2
1
29
C.
Iterative Methods
For n > about 40, the one-step methods take too long and accumulate too much round-off error.
1.
Jacobi Method
a.
Recursion formula
Each equation is solved for one of the unknowns.
x1  c1  b12 x 2  b12 x3    b1n x n 
1
b11
1
x 2  c 2  b21x1  b23 x3    b2 n x n 
b22

x n  c n  bn1 x1  bn 2 x 2    bnn1 x n 1 
1
bnn


n

 1
In short xi   ci   bij x j  , i = 1, 2, 3, . . .,n.

 bii
j 1
i j


Of course, we cannot have bii = 0 for any i. So before starting the iterative program, we may
have to reorder the equations. Further, it can be shown that if bii  bij for each i, then the
method will converge, though it may be slowly. Here’s an outline of the “showing.”



The first iteration is:
x 1   Ax 0  V

   
  
 


After several iterations, x k 1   Ax k  V k   A  A A A x 0  A  A AV  A k 1 x 0  A k V
k 1
3 2 1
 
bij
We want lim A k 1 x 0  0 , which will happen if
 1.
k 
bii
b.
Algorithm

 

We need four arrays: x k , x k 1 , B , and c .
 x10 
 
 0  x 20 
Firstly, select an initial guess (k = 0) x 
.

 0
 x n 

Secondly, compute a new x (k + 1 = 1).


n

 1
k 1
k
xi   ci   bij x j 

 bii
j 1
i

j


30
k 1
3 2
Thirdly, test for convergence.
xik 1  xik
xik
  . Notice that all the xi must pass the test.
If all the xi do not pass the test, then repeat until they do.
2.
Gauss-Seidel Method
The Gauss-Seidel Method hopes to speed up the convergence by using newly computed values
of xi at once, as soon as each is available. Thus, in computing xnew(12), for instance, the values
of xnew(1), xnew(2), . . ., xnew(11) are used on the right hand side of the formula. We still need
to keep separate sets of xnew and xold in order to perform the convergence tests.
% Script to implement Gauss-Seidel
b=[4 1 2;1 3 1;1 2 5];
c=[16;10;12];
xold=[1;1;1];
xnew=xold;
np=size(b);
n=np(1);
flag=1;
while flag > 0
for k=1:n
sum=0;
for l=1:n
if k~=l
sum=sum+b(k,l)*xnew(l);
end
end
xnew(k)=(c(k)-sum)/b(k,k);
end
for k=1:n
if abs((xnew(k)-xold(k))/xold(k)) > 0.0005
xold=xnew;
break
else
flag=0;
end
end
end
xnew
31
D.
Applications
A couple of cases in engineering that give rise to simultaneous linear equations.
1.
Electrical Circuit
(7+2+6)x1 – 2x2 – 6x3 = 300
-2x1 + (2+5+4+1)x2 – 4x3 – x4 = 0
-6x1 – 4x2 + (4+9+6)x3 – 9x4 = 0
-x2 – 9x3 + (9+1+11)x4 = 0
15
 2
A
 6

 0
 2  6 0 300
12  4  1 0 
;
 4 19  9 0 

 1  9 21 0 
32
26.5
9.35


solution: x  
13.3 


6.13
2.
Truss System
0
1 0 0
0 0 6
0

0  1  1
0

0 4
 0 1
5
A
0 0 0
1

0 0 0
0

0
0 0 0
1 0 0
0

0
0
0
0
0
0
0
0
0
0
0
0
0
3
5
4
5
0
0
0
0
0
0
0
1
0
0
0
1 0
0
0
0
1
600 
3600 
400 

0 
;
 600

0 

600 
0 
33
 600 
  1000 


 600 


  1250 
solution: x 
1083.33


 866.67 
  600 


  600 
V.
Interpolation and Curve Fitting
Suppose one has a set of data pairs:
x
f
x1
f1
x2
f2
x3
f3


xm
fm
where fi is the measured (or known) value of f(x) at xi. We would like to find a function that will
approximate f(x) for all x in a specified range. There are two basic approaches: interpolation and
curve fitting.
A.
Polynomial Interpolation
With interpolation, the approximating function passes through the data points. Commonly, the
unknown f(x) is approximated by a polynomial of degree n, pn(x), which is required to pass
through all the data points, or a subset thereof.
1.
Uniqueness
Theorem: Given {xi} and {fi}, i = 1, 2, 3, . . ., n + 1, there exists one and only one polynomial of
degree n or less which reproduces f(x) exactly at the {xi}.
Notes
i) There are many polynomials of degree > n which also reproduce the {fi}.
ii) There is no guarantee that the polynomial pn(x) will accurately reproduce f(x) for
x  xi . It will do so if f(x) is a polynomial of degree n or less.
Proof: We require that pn(x) = fi for all i = 1, 2, 3, . . ., n+1. This leads to a set of simultaneous
linear equations
ao  a1 x1  a2 x12    an x1n  f1
ao  a1 x2  a2 x22    an x2n  f 2

2
ao  a1 xn1  a2 xn1    an xnn1  f n1
which we’d solve for the {ai}. As long as no two of the {xi} are the same, the solution to such a
set of simultaneous linear equations is unique.
The significance of uniqueness is that no matter how an interpolating polynomial is derived, as
long as it passes through all the data points, it is the interpolating polynomial. There are many
methods of deriving an interpolating polynomial. Here, we’ll consider just one.
34
2.
Newton’s Divided Difference Interpolating Polynomial
a.
Divided differences
The first divided difference is defined to be (notice the use of square brackets)
f (a )  f (b)
f a, b 
, ab
ab
If f(x) is differentiable in the interval [a,b], then there exists
df ( )
 f a, b  .
at least one point between a and b at which
dx
In practice, we would take a as close to b as we can (limited
by the finite precision of the machine) and say that
f    f a, b .
Higher order differences are defined as well:
order
0
1
notation
f x1 
f x2 , x1 
2
f x3 , x2 , x1 
3
f x4 , x3 , x2 , x1 

n

f xn1 , xn ,, x2 , x1 
definition
f ( x1 )
f x2   f x1 
x2  x1
f x3 , x2   f x2 , x1 
x3  x1
f x4 , x3 , x2   f x3 , x2 , x1 
x4  x1

f xn1 , xn ,, x3 , x2   f xn , xn1 ,, x2 , x1 
xn1  x1
b.
Newton’s divided difference formula
Build the formula up step by step:
i)
two data points (x1,f1) & (x2,f2). We wish to approximate f(x) for x1 < x < x2.
As a first order approximation, we use a straight line (p1(x) so that
f x, x1   f x2 , x
f ( x)  f1 f 2  f ( x)

x  x1
x2  x
Solve for f(x)
f ( x)  f1  ( x  x1 ) f x2 , x1   p1 ( x)
ii)
Now, if f(x) is a straight line, then f(x) = p1(x). If not, there is a remainder, R1.
R1 ( x)  f ( x)  p1 ( x)  f ( x)  f1  ( x  x1 ) f x2 , x1   ( x  x1 )( x  x2 ) f x, x2 , x1 
35
We don’t know f(x), so we cannot evaluate f[x,x2,x1]. However, if we had a third data point
we could approximate f x, x2 , x1   f x3 , x2 , x1 . Then we have a quadratic
f ( x)  f1  ( x  x1 ) f x2 , x1   ( x  x1 )( x  x2 ) f x3 , x2 , x1   p2 ( x) .
iii)
If f(x) is not a quadratic polynomial, then there is still a remainder, R2.
R2 ( x)  f ( x)  p2 ( x)
To estimate R2, we need a fourth data point and the next order divided difference. . .
f x, x3 , x2 , x1   f x4 , x3 , x2 , x1 
iv)
Jump to the generalization for n + 1 data points:
f ( x)  pn ( x)  Rn ( x) , where
pn ( x)  f x1   ( x  x1 ) f x2 , x1   ( x  x1 )( x  x2 ) f x3 , x2 , x1  
  ( x  x1 )( x  x2 )( x  x3 )( x  xn ) f xn1 , xn ,, x2 , x1 
Notice that i) p3  p2  ( x  x1 )( x  x2 )( x  x3 ) f x4 , x3 , x2 , x1 , etc. and ii) the (x – xi) factors
are also those of the previous term times one more factor.
c.
Inverse interpolation
The NDDIP lends itself to inverse interpolation. That is, given f(x), approximate x. In effect, we
are solving f(x) = 0 when f(x) is in the form of a table of data. Simply reverse the roles of the {fi}
and the {xi}.
n1
i 1
i 2
j 1


x  pn ( f )   f  f1 , f 2 ,, f i  f ( x)  f j  f  f1 
Set f(x) = 0 and evaluate x = pn(0). In practice, with a Fortran program, one would just reverse
the data columns and use the same code.
d.
Example
The difference table is computed thusly:
for j=1:n+1
diff(j,1)=f(j)
end
for j=2:n+1
for i=1:n+1-j+1
diff(i,j)=( diff(i+1,j-1)-diff(i,j-1) )/(x(i+j-1)-x(i))
end
end
36
Divided Difference Table for n = 6
f[ , , ]
f[ , , , ]
f[ , , , , ]
1.667
-2.583
1.583
-3.5
2.167
-0.96
0.833
-0.233
0.1
0.367
0.017
0.4
j
x
f
f[ , ]
1
1
-1.5
0.5
2
2
-1
3
3
2.5
0.5
-0.5
4
3
0.25
0.75
5
4
1
1.3
6
4.5
1.65
1.7
7
5
2.5
The sixth degree polynomial constructed from this table is
7
i 1
i 2
j 1
f[ , , , , , ]
-0.727
0.353
f[ , , , , , , ]
0.27
p6 ( x)  f x1    f x1 , x2 ,, xi  ( x  x j ) .
Line by line, the script might look like this:
fac = ex – x(1)
p0 = diff(1,1)
p1 = p0 + fac*diff(1,2)
fac = fac*(ex-x(2))
p2 = p1 + fac*diff(1,3)
fac = fac*(ex-x(3))
p3 = p2 + fac*diff(1,4)
fac = fac*(ex-x(4))
p4 = p3 + fac*diff(1,5))
fac = fac*(ex-x(5))
p5 = p4 + fac*diff(1,6)
fac = fac*(ex-x(6))
p6 = p5 + fac*diff(1,7)
Notice that we must use a different variable name for the argument x from the name used for the
data array x(i).
Of course, it’s more general and flexible to use a loop.
fac = 1.0
p = diff(1,1)
for j=1:n
fac = fac*(ex-x(j))
p = p + fac*diff(1,j+1)
end
e.
Issues with high degree polynomials
If we have a large number of data points, 20 or 100 or 1000s, it does not pay to use the entire
data table to create a 20 or 100 or 1000th degree polynomial. The greater the degree, the more
often the pn goes up and down between the data points. Our confidence that f ( x)  pn ( x)
actually decreases. It’s better to interpolate on subsets of the data using a p3 or a p4 using data
points that surround the specified x. This process can be incorporated into the program. These
low-degree segments are sometimes called splines.
37
B.
Least Squares Fitting
Often, there are errors or uncertainties in the data values, 10.07  0.005 sec , for instance.
Perhaps forcing the approximating function to pass through the data points is not the wisest
approach.
An alternative approach is to assume a functional form for the unknown f(x) and adjust it to “best
fit” the uncertain data. A way to judge what is “best” is needed.
1.
Goodness of Fit
The method of least squares uses a particular measure of goodness of fit.
a.
Total squared error, E
First of all, never forget that the word error in this context means uncertainty. Now, let’s say
{xi,fi} are the n+1 data values and f(x) is the assumed function. Then E is defined to be
n 1
1
2
E   2  f i  f  xi 
i 1  i
The {  i } are weighting factors that depend on the nature of the uncertainties in the data {fi}.
For measured values, the  i  f i , the experimental uncertainties. Often, we just take all the
 i  1 , perhaps implying that the experimental uncertainties are all the same.. In that case,
n 1
2
E    f i  f xi  .
i 1
b.
Least squares fit
We wish to derive an f(x) which minimizes E. That means taking the derivative of E with
respect to each adjustable parameter in f(x) and setting it equal to zero. We obtain a set of
simultaneous linear equations with the adjustable parameters as the unknowns. These are called
the normal equations.
2.
Least Squares Fit to a Polynomial
Assume that f ( x)  a  bx  cx 2  dx 3 .
a.
Total squared error
n 1
2
1
E   2 f i  a  bxi  cxi2  dxi3
i 1  i
We have four adjustable parameters: a, b, c, and d. Notice that, unlike the interpolating
polynomial, there may be any number of data pairs, regardless of the number of parameters.
Let’s take all the  i  1 .


The partial derivative with respect to the adjustable parameters are
38


E
 2 f i  a  bxi  cxi2  dxi3
a
i
E
 2 xi f i  a  bxi  cxi2  dxi3
b
i
E
 2 xi2 f i  a  bxi  cxi2  dxi3
c






E
 2 xi3 f i  a  bxi  cxi2  dxi3
d
i
b.
Normal equations
Collect the like powers of xi and set the derivatives equal to zero.
 f i  a  b xi  c xi2  d  xi3
i
i
 xi f i  a xi  b
i

i
xi2 f i
i

 a
i
 a
 c
i
xi2
i
xi3 f i
i
xi2
 b
i
xi3
i
xi3
 b
i
i
xi3
 c
i
xi4
i
xi4
 c
i
 d  xi4
 d  xi5
i
xi5
i
 d  xi6
i
In terms of the matrix elements we used in solving simultaneous linear equations,
c1   f i
b11  1
c2   xi f i
c3  
b12   xi
b21   xi
xi2 f i
c4   xi3 f i
b22   xi2 , etc.
The system is solved by any standard method, Gauss-Jordan, Gauss-Seidel, even by Cramer’s
method.
c.
Accuracy of fit
We’d like to have some statistical measure of how good the fit between the {fi} and f(x) is. This
will depend on the relation between E and the {  i2 }. Let’s consider a quantity called (N = n + 1)
N
 f ( xi )  f i 2
i 1
 i2
 
2
.
If all  i  1 , then  2  E . Now, on another hand, if  i  f ( xi )  f i , then  2  N  g ,
where g is the number of adjustable parameters and N – g is the number of degrees of freedom in
2
the mathematical model for the data. We’d like to see
 1 for a “good” fit, while
Ng
39
2
 1 indicates that the quality of the fit is ambiguous (sometimes called over fitted), and
Ng
2
 1 indicates a “poor” fit.
Ng
3.
Least Squares Fit to Non-polynomial Function
The process is similar when fitting to a function that is not a polynomial. For instance, say that
f ( x)  a ln x  b cos x  ce x .
We wish to fit this function to the data shown at right. In this case,
xi
fi
N = 10 and g = 3. The adjustable parameters are a, b and c.
.24
0.23
2
10
xi
2
.65
-0.26
E     f i  a ln xi  b cos xi  ce
.95
-1.10
i 1
1.24
-0.45
The normal equations are:
1.73
0.27
2
xi
2.01
0.10
a  ln xi   b ln xi cos xi  c  ln xi e   f i ln xi
2.23
-0.29
a ln xi cos xi  b (cos xi ) 2  c cos xi e xi   f i cos xi
2.52
0.24
xi
xi
xi 2
xi
2.77
0.56
a ln xi e  b cos xi e  c (e )   f i e
2.99
1.00


6.794a  5.348b  63.259c  1.616
 5.347a  5.108b  49.009c  2.383
63.259a  49.009b  1002.506c  26.773
When solved by the Gauss-Jordan method, these yield
a = -1.041
b = -1.261
c = 0.031
f ( x)  1.041ln x  1.261cos x  0.031e x
f(x)
2
0.926

 1
Ng
7
The goodness of fit between these data and
this function is ambiguous. A glance at a
graph verifies that the fit is “iffy.” [That’s the
technical term for it.]
Least Squares Fit
data
2
1
0
-1
-2
0
1
2
x
40
fit
3
4
MatLab® Sidelight Number One
1.
Polynomials
a.
Representation
In MatLab®, polynomials are represented by a vector composed of the coefficients. Thusly,
p3 ( x)  a3 x 3  a 2 x 2  a1 x  ao  p  [a3 a 2 a1 ao ]
b.
Evaluate
A polynomial is evaluated using the command polyval. If p is the vector of coefficients and x is
the argument,
polyval(p,x) or y=polyval(p,x)
The roots command gives the roots of a polynomial, as elements of a vector.
r=roots(p)
If the roots of a polynomial are known, then the coefficient vector can be obtained by the poly
command.
p=poly(r)
c.
Add, multiply, & divide
Polynomials are added by adding the vectors of their coefficients. The shorter vector has to be
padded with zeros to make the two vectors the same length.
Multiplication of two polynomials is done with the conv command. c(x) = a(x) * b(x)
c = conv(a,b)
Division is done with the deconv command. u(x)/v(x) = q(x) + r(x)
[q,r] = deconv(u,v)
d.
derivatives
d
k ( x) 
p ( x)  k = polyder(p)
dx
d
k ( x) 
(a( x)b( x))  k=polyder(a,b)
dx
n( x)
d  u ( x) 
  [n m] = polyder(u,v) followed by [k,r] = deconv(n,m)
 
m( x) dx  v( x) 
41
2.
Curve Fitting & Interpolation
a.
Least squares
MatLab® fits data to a polynomial using the least squares method. Fitting an nth degree
polynomial to a table of (x,y) points. If the number of data points is m, then n must be m-1 or
less, and greater than 0.
p=polyfit(x,y,n)
Fitting to functions other than polynomials is done by rewriting the function in terms of a straight
line, for instance by taking the log of both sides, etc.
b.
Interactive fitting
In the Tools menu of the Figure Window is a Basic Fitting tool. This can be used to fit a
function to data interactively. See section 8.4 in the text.
c.
Interpolation
MatLab® has four interpolating techniques built-in. The command is interpl. It estimates
yi = f(xi), given a set of {x,y}.
yi = interpl(x,y,xi,’method’)
The methods available are
‘nearest’ returns the value of the nearest data point
‘linear’
carries out linear interpolation
‘spline’
carries out interpolation using a cubic polynomial based on the data points
surrounding the interpolated point
‘pchip’
carries out interpolation using a cubic Hermite polynomial.
42
VI.
Integration
b
We wish to evaluate the following definite integral:
 f ( x)dx .
a
We use numerical methods when
i) f(x) is known analytically but is too complicated to integrate analytically or
ii) f(x) is known only as a table of data.
A.
Newton-Cotes Formulæ
1.
Trapezoid Rule
a.
Graphs
Graphically, a definite integral is the area between the x-axis and the curve f(x). Areas below the
axis are negative; areas above the axis are positive.
b.
Trapezoids
The area “under” the curve might be approximated most simply by a series of trapezoids and
triangles.
43
f f
f1  f 2
x2  x1   2 3 x3  x2   
2
2
Notice that x1 = a and that x8 = b.
c.
Interpolating polynomial
In effect, we are replacing the integrand, f(x), by a straight
line between each pair of points:
f ( xi )  f ( xi 1 )
.
p1 ( x)  f ( xi 1 )  ( x  xi 1 )
xi  xi 1
This can be checked by integrating p1(x) analytically.
xi

xi 1
f  f i 1
f  f i 1 x 2
f i 1  ( x  xi 1 ) i
dx  f i 1 xi  xi 1   i
xi  xi 1
xi  xi 1 2
xi
 xi 1
xi 1
f i  f i 1 xi
x
xi  xi 1 xi 1
xi f i xi f i 1 xi 1 f i xi 1 f i 1



 xi 1 f i  xi 1 f i 1
2
2
2
2
x f
x f
x f
 i i  i 1 i  i 1 i 1
2
2
2
 xi f i 1  xi 1 f i 1 
xi f i 1
2
1
 xi  xi 1  f i  f i 1 
2

d.
check.
Implementation
N
For N data points spanning [a,b], there are N – 1 trapezoids. T   xi  xi 1 
i 2
f i  f i 1
2
If the data are uniformly spaced, then xi  xi 1  h for all i, and
 f1 f N N 1 
h N
T    f i  f i 1   h 
  f i  .
2 i 2
2
2
i 2


The lines in the MatLab® script might look like this:
n = 10
T = 0.0
for i=2:n
T = T + (x(i)-x(i-1))*(f(i)+f(i-1))/2.0
end
2.
Extension to Higher Order Formulæ
a.
Forward difference interpolating polynomial
We’ll take this opportunity to examine an alternative interpolating polynomial—the Forward
Difference Polynomial.
Imagine we have a table of data pairs (xi,fi) which are uniformly spaced, with spacing h. The
forward differences are just the familiar deltas.
first order:
f ( x1 )  f ( x2 )  f ( x1 )  f 2  f1
44
2 f ( x1 )  f ( x2 )  f ( x1 )  ( f ( x3 )  f ( x2 ))  ( f ( x2 )  f ( x1 ))
second order:
Notice that the differences f ( x1 ) and 2 f ( x1 ) are regarded as being evaluated at x = x1.
Hence the term forward difference.
Notice, too, that the forward differences are related to the divided differences simply by
multiplying by the denominators.
f ( x1 )  h  f x2 , x1 
2 f ( x1 )  2h 2  f x3 , x2 , x1 

n
 f ( x1 )  n!h n  f xn1 , xn ,, x2 , x1 
Now, let’s expand the integrand f(x) in a Taylor’s Series about x = x1. Further, to increase the
x  x1
element of confusion, let  
so that x  x1  h .
h
 (  1) 2
 (  1)(  2) 3
f ( x)  f ( x1 )  f ( x1 ) 
 f ( x1 ) 
 f ( x1 )  
2!
3!
Depending on how many terms are kept, this will give a polynomial in  or in x.
b.
Simpson’s rule
Any number of formulæ may be created by replacing the integrand, f(x), with an interpolating
polynomial of some specified degree. If f ( x)  p1 ( x)  f ( x1 )  f ( x1 ) , the Trapezoid Rule is
recovered.
Perhaps f(x) has some curvature, so a second degree interpolating polynomial may serve better.
x3
2
2
x1
0
0

 f ( x)dx  h p2 ( x1  h)d  h  f ( x1 )  f ( x1 ) 
   1
2

2 f ( x1 ) d

1


 h 2 f ( x1 )  2f ( x1 )  2 f ( x1 )
3


Expand the differences. . .
x3

1
2
1

 f ( x)dx  h2 f ( x1 )  2 f ( x2 )  2 f ( x1 )  3 f ( x3 )  3 f ( x2 )  3 f ( x1 )
x1
h
 f ( x1 )  4 f ( x2 )  f ( x3 )
3
This is Simpson’s Rule, which integrates over segments of three data points (or two intervals of
h) in one step.

c.
Implementation
x3
 f ( x)dx  3  f ( x1 )  4 f ( x2 )  f ( x3 )
h
x1
45
x5
 f ( x)dx  3  f ( x3 )  4 f ( x4 )  f ( x5 )
h
x3

xn 1
 f ( x)dx  3  f ( xn1 )  4 f ( xn )  f ( xn1 )
h
xn 1
Add ‘em up. . .


n
h  n 1

f ( xi )  4  f ( xi )  f (a )  f (b)
 f ( x)dx  3 2i
i 2
a
 i1 2

i  2


i) the data points must be uniformly spaced.
ba
ii) n + 1 must be odd, starting with 1 so that n 
is even.
h
b
Caveats:
46
B.
Numerical Integration by Random Sampling
1.
Random Sampling
a.
Pseudorandom numbers
Random numbers are a sequence of numbers,  z1 , z2 , z3 ,  , lying in the interval (0,1). There is
no pattern in the progression of the numbers, nor is any number in the sequence related to any
other number by a continuous function. There are statistical tests for randomness in a sequence
of numbers but we won’t bother with them here.
The operation of a computer is deterministic, so truly random numbers cannot be generated by a
computer program. However, sequences can be generated that appear to be random in that the
sequence passes some of the statistical tests for randomness. Such a sequence of numbers is
called pseudorandom.
Here is an algorithm for generating a sequence of pseudorandom numbers:
xi  mod a  xi 1  c, m
xi
m
where a, c and m are integers and mod( ) is the modulus function. The pseudorandom number
uniformly distributed in the interval (0,1) is zi.
zi 
In MatLab® , this looks like the following: [Using the built-in remainder function (rem).]
x = xo
for i=1:100
x1 = rem(a*x+c,em);
z = x1/em
x = x1;
end
This process generates a sequence of numbers {zi} that have some properties of random
numbers, but in fact the sequence repeats itself—it’s periodic. The exact sequence depends on
the initial value, xo, called the seed. Usually, m is a large integer, commonly a power of 2. The
numbers c and m can have no common factor (c can be zero) while a is a multiple of a prime
factor of m + 1. The period of the sequence is m, which is why m needs to be large. For
instance, we might take m  231, c = 0 and a = 16807.
On the other hand, MatLab® has built-in random number generating functions, shown in Table
3-2.
b.
Intervals
Suppose we want our pseudorandom numbers to lie in the interval (a,b) rather than (0,1). This is
easily done by scaling, or mapping onto the desired interval. Say 0  z  1, then
y  b  a  z  a will lie in the interval (a,b).
47
c.
Distributions
The example random number generator mentioned above produces numbers uniformly
distributed in (0,1). This means that is (0,1) were divided into equal subintervals, an equal
number of random numbers is expected in each of those subintervals. The probability of the next
random number in the sequence falling in a particular subinterval is the same for all the
subintervals spanning (0,1).
It is possible to form sequences of pseudorandom numbers which obey some other distribution
function, such as Poisson or Gaussian, etc. We won’t get into that here.
2.
Samples of Random Sampling
a.
Coin toss
We have two outcomes for each toss, of equal probability. We’ll generate an integer, either 1 or
2, using a pseudorandom number generator.
zi = a uniformly distributed pseudorandom number in (0,1)
j = int(2*zi) + 1 = 1 or 2
We’ll say that if j = 1, it’s heads, if j = 2 it’s tails.
b.
Roll of a die
In this case we have six outcomes, of equal probability (we hope). So we need to produce an
integer from 1 to 6.
j = int(6*zi)+1 = 1, 2, 3, 4, 5 or 6
Now, if it is known that the die is loaded, we use a different scheme, creating subintervals in
(0,1) whose lengths reflect the relative probabilities of the faces of the die coming up. For
instance, we might say that
zi
3.
0  z i  0.2
j
1
0.2  z i  0.34
2
0.34  zi  0.56
3
0.56  zi  0.72
4
0.72  zi  0.89
5
0.89  zi  1
6
Integration
Thinking again of the definite integral as an area under a curve, we envision a rectangle whose
area is equal to the total area under the curve f(x). The area of that equivalent rectangle is just
the length of the integration interval (a,b) times the average value of the integrand over that
48
interval. How to take that average? One way is to sample the integrand at randomly selected
points.
a.
One dimensional definite integrals
1

1 n
 f ( xi ) , where the {xi} form a pseudorandom sequence uniformly distributed in
n i 1
f ( x)dx 
0
b
(0,1). Over some other interval,

f ( x)dx  (b  a )
a
1 n
 f ( xi ) , where xi  a, b .
n i 1
Since we are just averaging over a list of numbers, the error is O[
1
n
], just like the deviation of
the mean.
1
example:
 sin xdx
0
1
 sin xdx  3 sin 0.00075  sin 0.01335  sin 0.3904  0.1313
1
0
1
 sin xdx  4 sin 0.00075  sin 0.01335  sin 0.3904  sin 0.8776  0.2910
1
0
1
 sin xdx  5 sin 0.00075  sin 0.01335  sin 0.3904  sin 0.8776  sin 0.0992  0.2524
1
0

The exact result is 0.460.
b.
Multi-dimension integrals
The random sampling approach is particularly useful with 2- and 3-dimensional integrals. The
other methods of numerical integration quickly become too messy to set up.
111

f ( x, y, z )dxdydz 
000
1 n
 f ( xi , y i , z i ) ,
n i 1
where (xi,yi,zi) is an ordered triple, each member uniformly distributed on (0,1).
We may use three separate sequences of pseudorandom numbers or simply take numbers from
one sequence three at a time.
c.
Alternate integration regions
bz by bx
i)



f ( x, y, z )dxdydz  bx  a x  b y  a y bz  a z 
az a y ax
49
1 n
 f ( xi , y i , z i )
n i 1
ii) Suppose the integration region is not rectangular. Then an extra step is needed, to test for and
discard random points that fall outside the integration region.
e.g., a circle—discard points for which xi2  yi2  1 , as shown in the following diagram.
Why do it this way; to ensure that the points are uniformly distributed in all directions. If points
are taken uniformly distributed in the radius, the points will be more widely spread the further
out from the center they lie, not uniformly spread over the area of the circle.
example: compute the volume of a sphere of radius R. In this situation, the integrand is 1.
R 2 
4
V     r 2 sin  d d dr    R 3
3
0 0 0
Numerically,
m
2R 3 m  m 8R 3 .
1
V  R   R R   R R   R 
1


n
n
n
i 1
xi2  yi2  zi2  R 2
Notice this: the total number of random points generated is n. However, only m of those lie
m
within the spherical volume. The spherical volume we obtain is equal to
times the volume of
n
a cube whose side is 2R. It’s interesting to see what this fraction is.
4
3
Vsphere 3   R


  0.52359  .
3
Vcube
6
8R
m
The ratio
should approach this constant as we generate more points and include them in the
n
summation.
50
m
issue is to say that f(x) = 1 when xi2  yi2  zi2  R 2 and 0 when
n
2
2
2
2
xi  yi  zi  R . Then there is no distinction between n and m, and the summation is a sum of
n – m zeros and m ones.
Another way to look at this
d.
Example
Evaluate  sin ln( x  y  1dxdy , where  is the region

2
2
1 
1

2
x   y    r .
2 
2

2

1 1
 1
I   sin ln( x  y  1dxdy   r     r  
2 2
 n


m

f ( xi , y i )  4r 2
i 1
2
2
1 
1

2
 xi    yi    r
2 
2

1
n
m
 f ( xi , y i )
i 1
2
2
1 
1

2
 xi    yi    r
2 
2

[If you want to try it, for r = 0.5, I = 0.57.]
This is equivalent to averaging the integrand over a circular area, thusly
m
2 1
f ( xi , y i ) .
 sin ln( x  y  1dxdy  r m 
i 1

2
2
1 
1

2
 xi    yi    r
2 
2

Of course, often the shape of the region of integration isn’t a simple rectangle or circle.
e. Example script
% Script to carry out 2-dim integration via random sampling
f=inline('sin(sqrt(log(x+y+1)))')
n=100;
r=0.5;
r2=r*r;
sum=0;
ax=0.5-r;
ay=0.5-r;
bx=r+0.5;
by=r+0.5;
51
m=0
for i=1:n
ex=rand*(bx-ax)+ax;
why=rand*(by-ay)+ay;
are=(ex-0.5)^2+(why-0.5)^2;
if are<=r2
sum=sum+f(ex,why);
m=m+1;
end
end
sum=sum*(by-ay)*(bx-ax)/m;
fprintf('Integral of f(x,y) over the circle = %g',sum)
52
MatLab® Sidelight Number Two
1.
Nonlinear Equations
a.
Fzero
The built-in MatLab® command for solving individual equations is named fzero.
x = fzero(‘function’,x0)
The initial guess is x0; ‘function’ is either a mathematical expression typed as a string, or the
name of a user defined function. The function has to entered in standard form: f(x) = 0.
‘function’ is the f(x). If entered as a string, the function cannot include redefined variables.
The initial guess can be entered as a single value, or as a 2-element vector such that the function
crosses the x-axis between x0(1) and x0(2). (As in the bisection or secant methods.)
b.
Maximun/minimum
A function to find the minimum of a function is fminbnd.
[x fval] = fminbnd(‘function’,x1,x2)
The command finds the minimum of the function, if any, lying in the interval (x1,x2).
2.
Integration
a.
Integrand as function
The quad command evaluates a definite integral using an elaborated version of Simpson’s Rule.
The method adjusts the step size as it goes along.
b
 f ( x)dx  q = quad(‘function’,a,b,tol)
a
The parameter tol is an optional tolerance. If tol is not specified, MatLab® assumes tol  10 6 .
The quadl command carries out the integration using another method, the adaptive Lobatto
method. That’s quad-L.
b.
Integrand as data table
When the integrand is available as a table of data pairs, MatLab® uses the Trapezoid Rule.
q = trapz(x,y)
53
c.
Random numbers
For generating uniformly distributed pseudorandom numbers, MatLab® has the command rand.
A single random number in (0,1): z = rand
A vector of n random numbers in (0,1): z =zrand(1,n)
An nxn matrix of random numbers in (0,1): z = rand(n)
An mxn matrix of random numbers in (0,1): z = rand(m,n)
A row vector with n elements consisting of a random permutation of integers 1 – n:
m = randperm(n)
54
VII. Ordinary Differential Equations
A.
Linear First Order Equations
dx
 f ( x, t ) . There are analytical methods of
dt
solution: integration, separation of variables, infinite series, etc. In practice these may not be
convenient or even possible. In such cases we resort to a numerical solution. The x(t) takes the
form of a table of data pairs {ti,xi}, rather than a function.
We seek to solve the following equation for x(t):
1.
One Step Methods
a.
Taylor’s Series
Many numerical solutions derive from the Taylor’s series expansion
dx(t o ) (t  t o ) 2 d 2 x(t o )
(t  t o ) p d p x(t o )
x(t )  x(t o )  (t  t o )




 .
dt
2!
p!
dt 2
dt p
dx
 f ( x, t ) , so we could substitute this into the series thusly:
We are given
dt
(t  t o ) 2 df ( xo , t o )
(t  t o ) p d p 1 f ( xo , t o )
x(t )  x(t o )  (t  t o ) f ( xo , t o ) 

 .
2!
dt
p!
dt p 1
df d 2 f d 3 f
,
,
, etc., we have to use the chain rule.
dt dt 2
dt 3
df f f dx


dt t x dt
2
d 2 f 2 f
2 f
f  f
f 
2  f


2
f

f
  f
2
2
2
xt
x  x
x 
dt
t
dx
It’s easy to see that this gets very messy rather quickly.
However, to obtain
b.
Euler’s Method
Let’s keep just the first two terms of the Taylor’s series: x(t )  x(t o )  (t  t o ) f ( xo , t o )  To ,
where the To is the sum of all the terms we’re dropping—call it the truncation error. In what
follows, we will have to distinguish between the correct or exact solution, x(t), and our
approximate solution, xi. We hope xi  x(t i ) .
With the Euler Method, our algorithm is [given to, x(to) = xo and f(x,t)]
x1  xo  (t1  t o ) f ( xo , t o )
x2  x1  (t 2  t1 ) f ( x1 , t1 )

xi 1  xi  (t i 1  t i ) f ( xi , t i )

55
dx
 13t , with to = 0 and xo = 4 and (t i 1  t i )  h  0.5 .
dt
The algorithm is: xi 1  xi  (t i 1  t i )13t i  .
example:
The first few steps in the numerical solution are shown in the following table.
i
t
x
0
0
4
1
.5
4
2
1
7.25
3
1.5
13.75
4
2
23.5



example: Problem 9-35
vi 1
i
0
1
2
3
4
5
6
7
2.
Error
a.
Truncation error
dv
 0.0035 v 2  3
dt
vo  83.333
i = 0, 1, 2, 3, 4, 5, . . .
t  0.1
 vi  0.1 * (0.0035 * vi2
t (sec)
0
.1
.2
.3
.4
.5
.6
.7
 3)
v (m/s)
83.3
80.6
78.0
75.6
73.3
71.1
69.0
67.1
xi 1  xi  hf ( xi , t i )  Ti 1
Not only do we not know what the exact solution is, we don’t know how far the numerical
solution deviates from the exact solution. In the case of a truncated Taylor’s series, we can
estimate the truncation error by evaluating the first term that is dropped. For Euler’s formula,
that’s the third term of the series.
h 2 df ( xi ) h 2
Ti 1 

f ( xi )
2 dt
2
Here’s a graph of both the exact (but unknown) and the numerical solutions.
56
The deviation from the exact x(t) may tend to increase as the total truncation error accumulates
from step to step, the further we get from the initial values (to,xo). The lesson is—make h small.
b.
Round-off error
Since the values are stored in finite precision, round-off error accumulates from step to step also.
Therefore, in traversing an interval o  t  b , we’d like to have as few steps as possible. In other
words, we want h to be large. Consequently, the two sources of error put competing pressure on
our choice of step size, h. If we have some knowledge of x(t), we may be able to achieve a
balance between large and small step size. Otherwise, it’s trial and error.
c.
Higher order methods
The many numerical algorithms that have been developed over the years for solving differential
equation seek to reduce the effect of truncation error by using more terms from the Taylor’s
series, or in some way correcting for the truncation error at each step. In that way, fewer, larger
steps can be used.
57
MatLab® Sidelight Number Three
1.
First Order Ordinary Differential Equations (ODE)
In standard form,
dy
 f ( x, y )
dx
We want to solve for y(x). In Physics & Engineering, of course, we often have
dy
 f ( y , t ) , in
dt
dv x
 f (v x , x, t ) , etc.
dt
MatLab® has several built-in ODE solvers, which all have the form of
[t,y]=solver_name(‘ODEfunc’,tspan,y0)
ODEfunc is the name of the function file which defines the differential equation, the f(t,y).
tspan is a vector that specifies the interval of the independent variable spanned by the
solution.
y0 is the initial value of y.
[t,y] is the output, in the form of two column vectors. Subsequently, we would plot(t,y).
which case we wish to solve for y(t); or perhaps we have
a.
Function file
dy
for given values of y & t. That is, t & y are input arguments to
dt
the function, and the value of f(t,y) is returned.
The function file calculates
b.
Solvers
Table 9-1 lists some of the MatLab® initial-value ODE solvers. Some are more sophisticated
than others; some are adapted to problems in which the solution is not smooth, or is rapidly
varying, etc. In most physical and engineering applications, things are smooth and not toorapidly varying, so most times ode45 should suffice.
[t,y] = ode45(‘function’,[0:0.1:10],100)
plot(t,y)
MatLab® also has boundary value and partial differential equation solvers, but those are not
discussed in the introductory text, nor in this class.
example: Problem 9-35
dv
 0.0035 v 2  3 m/s 2
dt
vo  300 km/hr
v0 = 300/60/60*1000
[t,v] = ode45(‘drag’,[0:0.1:15],v0)
plot(t,v)
function dvdt = drag(t,v)
dvdt = -0.0035*v*v-3;
58
B.
Second Order Ordinary Differential Equations
d 2x
dx
 x   f (t , x, )  f (t , x, x ) , with initial conditions x(0)  xo and x(0)  vo .
2
dt
dt
1.
Reduction to a System of First Order Equations
a.
New Variables
We start by introducing new variable names: z1  t ; z 2  x ; z 3  x  ; z 4  x . The first three
variables are the solutions to the following differential equations:
z1  1
z 2  x   z 3
z 3  x  z 4
These form a set of three simultaneous first order differential equations,
z1  1
z 2  z 3
z 3  z 4  f ( z1 , z 2 , z 3 )
with the initial conditions z1 (0)  0 , z 2 (0)  xo and z 3 (0)  vo respectively.
b.
Solution
Any method, such as Euler’s, may now be applied to each first order equation in turn. Thusly:
z1,i 1  z1,i  h  1
z 2,i 1  z 2,i  h  z 3,i
z 3,i 1  z 3,i  h  f i .
The MatLab® code might look like this:
z(1) = 0.0
z(2) = xo
z(3) = vo
h = 0.01
for i=1:100
z(1) = z(1) + h
z(2) = z(2) +h*z(3)
z(3) = z(3) + h*f(z(1),z(2),z(3))
end
c.
Example
x    x   9  cos(  t )
x(0)  xo , x(0)  vo
In this case, f (t , x, x )   x   9  cos(  t ) , so the algorithm looks like
z1,i 1  z1,i  h  1
z 2,i 1  z 2,i  h  z 3,i
z 3,i 1  z 3,i  h   z 3,i  9  cos(   z1,i ) .
59
2.
Difference Equations
An alternative approach to second order ordinary differential equations is to replace the
derivatives with finite differences. The differential equation is replaced by a difference equation.
a.
Difference equation
Using forward divided differences, we obtain
d 2 x 1  x  xi xi  xi 1  xi 1  2 xi  xi 1
dx xi 1  xi
and x   2   i 1
.

x 


h
h
h 
dt
h2
dt
h
Let’s say that we have the second order differential equation
x  ax  bx  ct  d .
The corresponding difference equation is
xi 1  2 xi  xi 1
 x  xi 
 a i 1
  bxi  ct i  d .
2
h
h


The next step is to solve for the “latest” x.
xi 1  2 xi  xi 1  ahxi 1  ahxi  bh 2 xi  ch 2 t i  dh 2
1  ah xi 1  2  ah  bh 2 xi  xi 1  ch 2 t i  dh 2



1
2  ah  bh 2 xi  xi 1  ch 2 t i  dh 2
1  ah
The initial conditions are applied by setting to = 0, x0 = xo and x1  xo  vo h .
xi 1 
b.
Examples
i) x    g
Here, a  b  c  0 and d = -g.
xi 1  2 xi  xi 1  gh 2
ii) x    x   g
This time, a = -1, b = 0, c = 0 and d = -g.
1
2  h xi  xi 1  gh 2
xi 1 
1 h


c.
Discretization error
Replacing continuous derivatives with finite differences introduces what is known as
discretization error. Implicitly, we are assuming a straight line between xi and xi+1 and between
xi and xi1 as well. There will always be some   xi 1  x(t i 1 ) at each step which will then
accumulate over the sequence of steps in the numerical solution.
60