Laboratory in Oceanography:
Data and Methods
Linear Algebra & Calculus Review
MAR550, Spring 2013
Miles A. Sundermeyer
Sundermeyer
MAR 550
Spring 2013
1
Linear Algebra and Calculus Review
Nomenclature:
scalar: A scalar is a variable that only has magnitude, e.g. a speed of 40 km/h,
10, a, (42 + 7), p, log10(a)
vector: A geometric entity with both length and direction; a quantity comprising
both magnitude and direction, e.g. a velocity of 40 km/h north, velocity u, position
x = (x, y, z)
array: An indexed set or group of elements, also can be used to represent
vectors, e.g.,
row vector/array:
1 1 2 3 5 8 13 21
Sundermeyer
MAR 550
Spring 2013
column vector/array:
1 
1 
 
 2
 
3
5
 
8
13 
 
21
2
Linear Algebra and Calculus Review
matrix: A rectangular table of elements (or entries), which may be numbers or,
more generally, any abstract quantities that can be added and multiplied;
effectively a generalized array or vector - a collection of numbers ordered by rows
and columns.
[2 x 3] matrix:
1 2 3
10 20 30


[m x n] matrix:
 a1,1
a
 2,1
 a3,1

 
 am,1

Sundermeyer
MAR 550
Spring 2013
a1, 2
a1,3

a2 , 2
a3,3


a1,n 


 


am,n 
3
Linear Algebra and Calculus Review
Examples (special matrices):
A square matrix has as many rows as it has columns. Matrix A is square but
matrix B is not:
 3 4 5
A   2 12 5
 1 7 0
3 4 5
B

2 12 5
A symmetric matrix is a square matrix in which xij = xji, for all i and j.
A symmetric matrix is equal to its transpose. Matrix A is symmetric; matrix B
is not.
 1 2  1
A   2 12 10 
 1 10 0 
Sundermeyer
MAR 550
Spring 2013
 1 2  1
B  10 12 2 
 1 10 0 
4
Linear Algebra and Calculus Review
A diagonal matrix is a symmetric matrix where all the off diagonal elements are
0. The matrix D is diagonal.
4 0 0
D  0 2 0
0 0 7
An identity matrix is a diagonal matrix with only 1’s on the diagonal. For any
square matrix, A, the product IA = AI = A. The identity matrix is generally
denoted as I.
1 0 0
I  0 1 0
0 0 1
1 0 0  1 2  1  1 2  1
IA = A: 0 1 0  2 12 10    2 12 10 


 

0 0 1  1 10 0   1 10 0 
Sundermeyer
MAR 550
Spring 2013
5
Linear Algebra and Calculus Review
Example (system of equations):
Suppose we have a series of measurements of stream discharge and stage,
measured at n different times.
time (day) = [0 14 28 42 56 70]
stage (m) = [0.612 0.647 0.580 0.629 0.688 0.583]
discharge (m3/s) = [0.330 0.395 0.241 0.338 0.531 0.279]
Suppose we now wish to fit a rating curve to these measurements. Let x = stage,
y = discharge, then we can write this series of measurements as:
yi = mxi + b, with i = 1:n.
 y1 
y 
 2
This in turn can be written as: y = X b, or:  y 2  
 

 y n 
[n  1] 
Sundermeyer
MAR 550
Spring 2013
 x1 1
 x 1
 2  m
 x3 1  

 b 
  
 xn 1
[n  2] [2  1]
6
Linear Algebra and Calculus Review
yi = mxi + b
y=Xb
 y1 
y 
 2
 y2  
 

 y n 
[n  1] 
Sundermeyer
MAR 550
Spring 2013
 x1 1
 x 1
 2  m
 x3 1  

 b 
  
 xn 1
[n  2] [2  1]
7
Linear Algebra and Calculus Review
Vectors:
Addition/Subtraction: Two vectors can be added/subtracted if and only if they
are of the same dimension.
 a1   b1   a1  b1 
a  b  a  b 
2
AB   2   2   2
    
    

a
b
a

b
n
 n  n  n
Example:
Sundermeyer
MAR 550
Spring 2013
2  5  2  5 7
 3    2  3  2  1 
  
 
4  3  4  3 7
    
  
1
7
1

7
    
 8 
8
Linear Algebra and Calculus Review
Scalar Multiplication: If k is a scalar and A is a n-dimensional vector, then
 a1   ka1 
a  ka 
kA  k  2    2 
   
   
an  kan 
Example:
25 2  25 50
2 20  2  20  40
 5   2  5  10 
Example:
A + B – 3C, where
Sundermeyer
MAR 550
Spring 2013
 2
1 
10
A  3, B  1, C   1 
6
2
 2 
2 1 10 2  1  30  27
A  B  3C  3  1  3 1    3  1  3    1 
6 2  2   6  2  6   2 
9
Linear Algebra and Calculus Review
Dot Product: Let
u  u1 , u 2 ,, u n 
v  v1 , v2 ,, vn 
be two vectors of length n. Then the dot product of the two vectors u and v is
defined as
n
u  v  u1v1  u 2 v 2    u n v n   u i vi
i 1
A dot product is also an inner product.
Example:
u  v  4 1 2 3 3 1 7 2  (4  3)  (11)  (2  7)  (3  2)  33
Example (divergence of a vector):

v  
 x
Sundermeyer
MAR 550
Spring 2013

y

u v w



u
v
w

 

z 
x y z
10
Linear Algebra and Calculus Review
Dot Product and Scalar Product Rules:

u·v is a scalar

u·v = v·u

u·0 = 0 = 0·u

u·u = ||u||2

(ku)·v = (k)u·v = u·(kv) for k scalar

u·(v ± w) = u·v ± u·w
Sundermeyer
MAR 550
Spring 2013
11
Linear Algebra and Calculus Review
Cross Product: Let
u  u1 , u2 , u3 
v  v1 , v2 , v3 
be two vectors of length 3. Then the cross product of the two vectors
u and v is defined as
 iˆ ˆj

u  v  det u1 u2
v v
 1 2
kˆ 
 ˆ u2 u3 
u3   i det 

v
v
3
 2
v3 

 iˆ(u2 v3  u3v2 ) 
u u
u u
ˆj det  1 3   kˆ det  1 2 
v v 
v v 
 1 2
 1 3
ˆj (u1v3  u3v1 )  kˆ(u1v2  u2 v1 )
Example:
 iˆ ˆj kˆ 


u  v  4 1 2 3 1 7  det 4 1 2  iˆ(7  2)  ˆj (28  6)  kˆ(4  3)  5iˆ  22 ˆj  1kˆ
3 1 7 


Example (curl of a vector):
ˆj kˆ 
 iˆ
      w v   w u   v u 
  
  iˆ    ˆj    kˆ  
 v  
 u v w  det 

 x y z   y z   x z   x y 
 x y z 
 u v w 
Sundermeyer
MAR 550
Spring 2013
12
Linear Algebra and Calculus Review
Cross Product Rules:
•
u × v is a vector
•
u × v is orthogonal to both u and v
•
u×0=0=0xu
•
u×u=0
•
u × v = -(v × u)
•
(ku) × v = k(u × v) = u × (kv) for any scalar k
•
u × (v + w) = (u × v) + (u × w)
•
(v + w) × u = (v × u) + (w × u)
Sundermeyer
MAR 550
Spring 2013
13
Linear Algebra and Calculus Review
NOTE: In general, for a vector A and a scalar k, kA = Ak.
 a1   ka1   a1k   a1 
a  ka  a k  a 
kA  k  2    2    2    2  k  Ak
       
       
an  kan  an k  an 
However, when computing the gradient of a scalar, the scalar product is not
commutative because  itself is not commutative, i.e.,

T  
 x

y
 T

T

 x
z 

T
y
T  T ˆ T ˆ T ˆ

i
j
k

z  x
y
z
T
Sundermeyer
MAR 550
Spring 2013
 ˆ
 ˆ

i T
j  T kˆ
x
y
z
14
Linear Algebra and Calculus Review
Matrix Algebra:
Matrix Addition: To add two matrices, they both must have the same number of
rows and the same number of columns. The elements of the two matrices are
simply added together, element by element. Matrix subtraction works in the
same way, except the elements are subtracted rather than added.
A + B:
 a1,1 a1, 2
a
 2,1 a 2, 2
 a 3,1

 
 a m,1

Sundermeyer
MAR 550
Spring 2013
a1,3

a 3, 3


a1, n   b1,1
 b
  2,1
    b3,1
 
  
a m, n   bm,1
 a1,1
a
 2,1
  a 3,1


 a m,1

b1, 2
b2 , 2
 b1,1
 b2,1
 b3,1

 bm,1
b1,3

b3,3


a1, 2  b1, 2
a 2 , 2  b2 , 2
b1, n 


 


bm, n 
a1,3  b1,3

a 3,3  b3,3


a1, n  b1, n 






a m, n  bm, n 
15
Linear Algebra and Calculus Review
Example:
 1 2 3   100 200 300   101 202 303 
10 20 30  1000 2000 3000  1010 2020 3030

 
 

Matrix Addition Rules: Let A, B and C denote arbitrary [m x n] matrices
where m and n are fixed. Let k and p denote arbitrary real numbers.
•
A+B=B+A
•
A + (B + C) = (A + B) + C
•
There is an [m x n] matrix of 0’s such that 0 + A = A for each A
•
For each A there is an [m x n] matrix –A such that A + (-A) = 0
•
k(A + B) = kA + kB
•
(k+p)A = kA + pA
•
(kp)A = k(pA)
Sundermeyer
MAR 550
Spring 2013
16
Linear Algebra and Calculus Review
Matrix Transpose: Let A and B denote matrices of the same size, and let k
denote a scalar.
AT (also denoted A’):
 a 1,1
a
 2,1
 a3,1

 
 a m,1

a 1,2
a 1,3

a 2, 2
a 3, 3


T
a 1,n 
 a 1,1

a

 1,2
    a 1,3



 
 a 1,n
a m,n 

[m x n]
Example:
a 2,1
a3,1

a 2, 2
a 3, 3


a m ,1 


 


a m,n 
[n x m]
1 10 
1 2 3
2 20

10 20 30




3 30
T
Sundermeyer
MAR 550
Spring 2013
17
Linear Algebra and Calculus Review
Matrix Transpose Rules:
If A is an [m x n] matrix, then AT is an [n x m] matrix.
• (AT)T = A
• (kA)T = kAT
• (A + B)T = AT + BT
Sundermeyer
MAR 550
Spring 2013
18
Linear Algebra and Calculus Review
Matrix Multiplication: There are several rules for matrix multiplication. The first
concerns the multiplication between a matrix and a scalar. Here, each element
in the product matrix is simply the element in the matrix multiplied by the scalar.
Scalar Multiplication sA:
 a1,1
a
 2,1
s  a3,1

 
 am,1

a1, 2
a1,3

a2 , 2
a3,3


a1,n   s a1,1
 s a
  2,1
    s a3,1
 
  
am,n   s am,1
s a1, 2
s a1,3

s a2 , 2
s a3,3


s a1,n 


 


s am,n 
Example:
 3 2  12 8 
  

4 
3
6
12
24

 

Sundermeyer
MAR 550
Spring 2013
19
Linear Algebra and Calculus Review
Matrix Product AB: This is multiplication of a matrix by another matrix. Here,
the number of columns in the first matrix must equal the number of rows in the
second matrix, e.g., [m × n][n × m] = [m × m].
 a1,1
a
 2,1
 a3,1

 
 a m,1

a1, 2
a1,3

a 2, 2
a 3, 3


a1,n   b1,1
 b
  2,1
   b3,1

 
a m,n   bn ,1
b1, 2
b1,3

b2, 2
b3,3


b1,m 


 


bn ,m 
= [m × m] matrix whose (i,j) entry is the dot product of the ith row of A and the
jth column of B.
Example (inner (dot) product):
 4
1 2 3 5  (1 4)  (2  5)  (3  6)  32
6
Sundermeyer
MAR 550
Spring 2013
[1 x 3][3 x 1] =
[1 x 1]
20
Linear Algebra and Calculus Review
Example (outer product):
 4
4  1 4  2 4  3 4 8 12
51 2 3  5  1 5  2 5  3  5 10 15
 

 

6
6  1 6  2 6  3 6 12 18
[3 x 1][1 x 3] =
[3 x 3]
Example (general matrix product):
1 10 
(110)  (2  20)  (3  30)   14 140 
 1 2 3 
   (1 1)  (2  2)  (3  3)
2
20
10 20 30 
 (10 1)  (20  2)  (30  3) (10  10)  (20  20)  (30  30)  140 1400

 3 30 
 



[2 x 3]
Sundermeyer
MAR 550
Spring 2013
[3 x 2]
[2 x 2]
21
Linear Algebra and Calculus Review
Matrix Multiplication Rules: Assume that k is an arbitrary scalar and that A, B,
and C, are matrices of sizes such that the indicated operations can be
performed.
• IA = A, BI = B
• A(BC) = (AB)C
• A(B + C) = AB + AC, A(B – C) = AB – AC
• (B + C)A = BA + CA, (B – C)A = BA – CA
• k(AB) = (kA)B = A(kB)
• (AB)T = BTAT
NOTE: In general, matrix multiplication is not commutative: AB ≠ BA
Sundermeyer
MAR 550
Spring 2013
22
Linear Algebra and Calculus Review
Matrix Division: There is no simple division operation, per se, for matrices.
This is handled more generally by left and right multiplication by a matrix inverse.
Matrix Inverse: The inverse of a matrix is defined by the following:
AB = I = BA if and only if A is the inverse of B.
We then write: AA-1 = A-1A = 1 = BB-1 = B-1B
NOTE: Consider general matrix expression::
AX=B
A-1 A X = A-1 B
A-1 A X = A-1 B
1 X = A-1 B
X = A-1 B
0 0 
Also note, not all matrices are invertible; e.g., the matrix 
 has no inverse.
1
3


Sundermeyer
MAR 550
Spring 2013
23
Linear Algebra and Calculus Review
Special Matrix Algebra Rules in Matlab:
Matrix + Scalar Addition: A + s:
 a1,1
a
 2,1
 a3,1

 
 a m,1

a1, 2
a1,3

a 2, 2
a 3, 3


a1,n 
 a1,1  s

a  s

 2,1
   s   a3,1  s



 
 a m,1  s
a m,n 

a1, 2  s
a1,3  s

a 2, 2  s
a 3, 3  s


a1,n  s 


 


a m,n  s 
Example:
1 2 3
101 102 103
10 20 30  100  110 120 130




Sundermeyer
MAR 550
Spring 2013
24
Linear Algebra and Calculus Review
Special Matrix Algebra Rules in Matlab:
Matrix times matrix ‘dot’ multiplication, A .* B (similar for ‘dot’ division A ./ B):
 a1,1
a
 2,1
 a3,1

 
 a m ,1

a1, 2
a 2, 2
a1,3

a 3, 3


a1,n   b1,1
 b
  2,1
   b3,1

 
a m ,n   bn ,1
b1, 2
b2, 2
 a1,1b1,1
a b
 2,1 2,1
  a3,1b3,1

 
 a m ,1bn ,1

Example:
Sundermeyer
MAR 550
Spring 2013
b1,3
b3,3

a1, 2 b1, 2
a 2, 2 b2, 2
b1,m 


 



bn ,m 
a1,3 b1,3


a3,3 b3,3


a1,n b1,m 


 


a m ,n bn ,m 
40
90 
 1 2 3   10 20 30   10
.
*

10 20 30 100 200 300 1000 4000 9000

 
 

25