Geology 5670/6670
Inverse Theory
21 Sep 2023
Last time: Review of WLS and Solution Appraisal;
Introduced Assignment 1 & Matlab
Read for Tues 26 Sep Feb: Menke Ch 7
© A.R. Lowry 2023
Assignment 1 is now posted on the course website:
Ordinary & Least-Squares inversion of a large data set,
including solution appraisal
Due 3 October
The Generalized Inverse:
(basically, a generalization of the pseudo-inverse)
Derives from Singular Value Decomposition:
Given our problem,
d N´1 = G N´M mM´1 + e N´1
We can create a partitioned, symmetric matrix S
SM +N´M +N
´
0 N´N
=´ T
´
´ GM ´N
GN´M
0 M ´M
´
´
´
´
Because it is real & symmetric,
1) Has real eigenvalues, positive if S is positive definite
T
( x Sx > 0 for all x ≠ 0)
2) Eigenvectors for differing, non-zero eigenvalues
are orthogonal
Solving for eigenvalues & eigenvectors:
(
)
Ax = l x can be rearranged as A- l I x = 0 ,
where I is the identity matrix.
This system of equations has a nontrivial (nonzero)
solution only when the determinant is equal to
zero:
det A- l I = 0
which turns out to be a polynomial equation in .
Once the roots of the polynomial are found, the
eigenvectors can be solved for each  by substituting
and solving.
Example: Let A be
and
. Then
This has polynomial roots  = 1, 2, 3.
(In practice, you would probably use, e.g.,
>> lambda = eig(A);
in Matlab or other similar algebraic software!…)
The eigenvalues & eigenvectors for SVD are those that solve:
Swi = li wi
Can partition wi as
So
Note there will be only p ≤ min(N, M) nonzero eigenvalues.
Eigenvectors are nonunique and we can choose them to
normalize as
vi
T
vi vi
&
ui
T
ui ui
so that
T
vi v j
= dij &
&
T
ui u j
= dij
(in which case eigenvectors are orthonormal).
We can define matrices U and V of the eigenvectors:
[
= [v
U NxN = u1 u2 ... uN
V MxM
1
v2 ... vM
]
]
T
T
U U = UU = I N´N
T
T
V V = VV = I M ´M
And a matrix 
Then
or
T
GV = UL Þ GVV = ULV
G = U LV
T
T (the singular value
decomposition of G).
Development of the Generalized
Inverse:
For Ordinary Least Squares (full rank)  p = M < N
G = U LV
T
-2 T
T
= VL M V VL M U M
G
+
An OLS matrix is “full
rank” if it has at least
M independent rows
(i.e., rows with
values that cannot
be replicated using
linear combinations
of other rows).
-1 T
= VL M U M
(Note this is just an alternative expression for something we
already know how to do!)
Some useful metrics:
Model resolution matrix:
+
RM = G G = I M for OLS
(describes how well the model parameters can be
resolved… Here, RM = I  parameters can be
uniquely solved from the data!)
+
T
=UM UM
Data resolution matrix: N = GG
¹IN
(describes how well the predictions match the data…
Here, not I because we are fitting in a least-squares
sense, which requires misfit for data with errors!)
Where SVD becomes more valuable is in the
over-parameterized case p = N < M…
The over-parameterized case p = N < M has nonunique
solution (i.e., there are  different model
parameterizations m that can exactly fit the data).
We can specify a unique solution by imposing additional
constraints, e.g. minimum structure:
Minimization of:
Substitute
T
m m is given by
T
G = U LNVN
G
+
-1 T
= V N LN U
Aside on solution length: Suppose we have a simple
over-parameterized gravity problem:
D1 D2 D3 D4
that can be solved exactly by both:
and
The shorter solution length is smoother, varies less
wildly (like what one expects in the real Earth, where
most fields are “self-similar” and vary according to
fractal statistics  properties that are spatially or
temporally “nearer together” tend to be more similar)