Linear(-ized) Inverse Problems
Linear inverse problems
- Formulation
- Some Linear Algebra
- Matrix calculation – Revision
- Illustration under(over)determined, unique case
- Examples
Linearized inverse problems
- Formulation
- Examples
Partial derivatives
Scope: Formulate linear inverse problems as a system of equations in
matrix form. Find the conditions under which solutions exist. Understand
how to linearize a non-linear system to be able to find solutions.
Linear Inverse Problems
Computational Geophysics and Data Analysis
1
Literature
Stein and Wysession: Introduction to
seismology, Chapter 7
Aki and Richards: Theoretical Seismology
(1s edition) Chapter 12.3
Shearer: Introduction to seismology,
Chapter 5
Menke, Discrete Inverse Problems
http://www.ldeo.columbia.edu/users/menke/
gdadit/index.htm
Full ppt files and matlab routines
Linear Inverse Problems
Computational Geophysics and Data Analysis
2
Formulation
Linear(-ized) inverse problems can be formulated in the following way:
d i  G ij m j
(summation convention applies)
i=1,2,...,N
j=1,2,...,M
Gij
number of data
number of model parameters
known (mxn)
We observe:
- The inverse problem has a unique solution if N=M and det(G)≠0, i.e.
the data are linearly independent
- the problem is overdetermined if N>M
- the problem is underdetermined if M>N
Linear Inverse Problems
Computational Geophysics and Data Analysis
3
Illustration – Unique Case
In this case N=M, and det(G) ≠0. Let us consider an example
1  d 1  3 m1  2 m 2
 d1   3
   
d 
 2  1
2  d 2  m1  4 m 2
2  m1 

 
4   m 2 
d  Gm
Let us check the determinant of this system: det(G)=10
G d  G Gm  m  G d
-1
-1
 m1   0 .4


 m     0 . 1
 2 
-1
 0 .2   d 1 
  
0 .3   d 2 
 m1   0 

  

 m 2   0 .5 
Linear Inverse Problems
Computational Geophysics and Data Analysis
4
Illustration – Overdetermined Case
In this case N>M, there are more data than model parameters.
Let us consider examples with M=2, an overdetermined system would
exist if N=3.
1  d 1  m1
2  d2  m2
2  d 3  m1  m 2
A physical experiment which could result in these data:
Individual Weight measurement of two masses m1 and m2
leading to the data d1 and d2 and weighing both together
leads to d3. In matrix form:
 d1   1
  
d2   0
d  1
 3 
Linear Inverse Problems
0
 m1 

1  
 m2 

1
Computational Geophysics and Data Analysis
d  Gm
5
Illustration – Overdetermined Case
Let us consider this problem graphically
1  m1
2  m2
2  m1  m 2
A common way to solve this problem is to minimize the
difference between data vector d and the predicted data
for some model m such that
2
S  d  Gm
is minimal.
Linear Inverse Problems
Computational Geophysics and Data Analysis
6
Illustration – Overdetermined Case
Using the L2-norm leads us to the
least-squares formulation of the
problem. The solution to the
minimization (and thus the inverse
problem) is given as:
best model
~  (G T G)  1 G T d
m
In our example the resulting (best) model estimation is:
 2 / 3
~

m  
5 /3
and is the model with the minimal distance to all three lines in the plot.
Linear Inverse Problems
Computational Geophysics and Data Analysis
7
Illustration – Underdetermined Case
Let us assume we made one measurement of the combined weight of
two masses:
m1  m 2  d  2
Clearly there are infinitely many solutions to this problem. A model
estimate can be defined by choosing a model that fits the data exactly
Am=d and has the smallest l2 norm ||m||. Using Lagrange multipliers
one can show that the minimum norm solution is given by
~  G T ( GG T )  1 d
m
1
~
m   
1
Linear Inverse Problems
Computational Geophysics and Data Analysis
8
Examples – Inversion of Gravity Data
Let’s go back to the problem of gravity, in 2-D the Bouguer anomaly
at point x0 with arbitrary topography is given by (e.g. Telford et al., 1990)
z
d ( x 0 )  2
  x

 ( x, z ) z
 x  z
2
0
2
dxdz
x,x0
d(x0)
To bring this into the form d=Gm we
discretize the space
r(x,z)
xj
h
h j=1
2
3
4
5
6
7
8
9
...
zj
...
Linear Inverse Problems
20
di 
M
2 h z j
j 1
( xi  x j )  z
   


mj
Computational Geophysics and Data Analysis
2
2
2
j
mj
G ij
9
Master-Event-Method
Let s assume we have have previously located an earthquake (x0,y0,z0)
at time t0 and we recorded a new event at stations 1, ..., N
1
2
 ti   
Dti
3
 
Event 1
ui
y
gi
Li Event 2
x
L cos  i

1

u
ix
 
L  ui

x  u iy y  u iz z 
This is a system of linear equations
for 4 unknowns:
z
Linear Inverse Problems
Computational Geophysics and Data Analysis
10
Master-Event-Method
Event 1
Let us put this system into the
common form d=Gm
ui
G i1  1
m2  x
Gi2  
m3  y
u ix

m4  z
G i3  

  t1   1

 
     
 t  
N 

1

di
Linear Inverse Problems
x
y
d i   ti
m1  
gi
Li Event 2
u iy
Gi4  
u iz
u1 x
u1 y




u Nx
u Ny


u1 z   
 
  x 
  
u Nz   y 
 
   z 

z

Computational Geophysics and Data Analysis
11
Vertical Seismic Profile
Let us consider a string of receivers in a borehole
v1
seismometers
v2
- We assume straigt rays
- The ground is discretized with
M layers of equal thickness dz with
velocities vi
-The seismometers (N) are located at
depths zj
Formulate the forward problem in
matrix form d=Gm! Is the problem
linear? What would happen if the rays
are not modelled as straight lines?
vM
Linear Inverse Problems
Computational Geophysics and Data Analysis
12
Linearized Inversion
Let us formalize the situation where we are able to linearize a
otherwise nonlinear problem around some model m0. In this case the
forward problem is given by
d i  F ( m1 , m 2 ,..., m M )
i  1, N
this m-dimensional function is developed around some model
m0=(m01, m02, ..., m0M) where we neglect higher-order terms:
M
d i  F i ( m 01 , m 02 ,..., m 0 M ) 
d0
 Fi
 m
j 1
( m 01 , m 02 ,..., m 0 M )( m j  m 0 )
j
Gij
Dmj
d0
synthetic data of starting model (known)
di=di-d0 data difference vector (residuals, misfit, cost ...)
mj=mj-m0
model difference vector (gradient)
Linear Inverse Problems
Computational Geophysics and Data Analysis
13
Linearized Inversion: Hypocenter location
Above a homogeneous half space we measure P wave travel times
from an earthquake that happens at time t at (x,y,z) at i receiver
locations (xi, yi ,zi). So our model vector is m=(t,x,y,z)T. The arrival
times are given by
t i  Fi ( m )  t 
1

( x
 x )  ( yi  y )  z
2
i
2

2 1/ 2
this is a nonlinear problem! Now let us assume we have a rough idea
about the time of the earthquake and its location. This is our starting
model m0= (t0, x0, y0, z0)T.
To linearize the problem we now have to find the partial
derivatives of F with respect to all model parameters
at m0.
Linear Inverse Problems
Computational Geophysics and Data Analysis
14
Hypocenter location – partial derivatives
... we obtain :
t i  Fi ( m )  t 
t i 0  Fi ( m 0 )  t 0 
G i1 
Gi2 
G i3 
Gi4 
Linear Inverse Problems
F (m 0 )
t
F (m 0 )
x
F (m 0 )
y
F (m 0 )
z
1

( x
1

( x
 x)  ( yi  y )  ( zi  z )
2
i
 x0 )  ( yi  y0 )  z0
2
i
2
2

2 1/ 2

2 1/ 2
1



xi  x0
 Ri0
Ri0
yi  y0
 Ri0
z0
 Ri0
Computational Geophysics and Data Analysis
15
Hypocenter location – partial derivatives
... let us now define a vector
ui=1/Ri0(xi-x0, yi-y0,-z0)
which is a vector pointing from the initial source location to
receiver i. We obtain:
ti  ti 0  t  t0 
di
m1
1

u
ix
( x  x 0 )  u iy ( y  y 0 )  u iz ( z  z 0 )
m2
m3

m4
which is exactly the form we obtained for the
Master-Event Method, what is the difference, however?
This approach is an iterative algorithm
Linear Inverse Problems
Computational Geophysics and Data Analysis
16
Linearized Travel-Time Inversion
We learned in seismology that for a given ray parameter p the delay
time t(p) is given by the difference of the travel time T and the
distance X(p) the ray ermerges times p
zs ( p )
 ( p )  T ( p )  pX ( p )  2
 c
2
(z)  p

2 1 / 2
dz
0
Graphically this can be interpreted as:
p=dT/dX

pX
t(p)

T
X
Linear Inverse Problems
Computational Geophysics and Data Analysis
17
Linearized Travel-Time Inversion
… the important property of t(p) is the
fact that it decreases monotonically with
increasing p so it is a function easier to
handle than the travel-times (which may
contain triplications).
t(p) is nonlinearly related to the velocity
model c(z). So in order to invert for it we
would have to linearize. We obtain
zs ( p )
 ( p )  2
 c
0
c(z)
2
(z)  p
2

1/2
p=dT/dX

pX
t(p)

T
X
zs ( p )
 ( p)  2
 c
2
(z)  p

2 1 / 2
dz
0
 c ( z ) dz
Now the perturbation in t(p) (the data residual) is linearly related
to the perturbation in the velocity model c(z). This integral can
easily be brought into the form d=Gm by subdividing the Earth into
layers (e.g. of equal thickness).
Linear Inverse Problems
Computational Geophysics and Data Analysis
18
Partial Derivatives
Let us take a closer look at the matrix Gij for linearized
problems. What useful information is contained in this matrix
(operator)? When d=g(m), then the linearization leads to
d  Gm
And the matrix Gij contains the partial derivatives
G ij 
g i
m j
The actual (relative) values of Gij determine how the model
parameters influence the data (or data difference).
Example: Gik are small for all i. This implies that the model
Parameter mk has almost no influence on the data. It can be varied
Without changing them. Therefore, its resolution is poor.
Linear Inverse Problems
Computational Geophysics and Data Analysis
19
Resolution – Hypocenter Location
Example: Earthquake hypocenter location
ti  ti 0  t  t0 
di
m1
1

u
ix
( x  x 0 )  u iy ( y  y 0 )  u iz ( z  z 0 )
m2
m3

m4
Remember the elements of Gij where the components of the unit
vector which points from the original (known) hypocenter to the
receiver. Small uiz with respect to the other ones means bad
resolution in depth:
The depth resolution of shallow earthquakes
Far away is poor.
Linear Inverse Problems
Computational Geophysics and Data Analysis
20
Linear Dependence
When two columns of Gij are linearly dependent then for all i
G ik  cG il
What are the consequences for a model perturbation in parameters k and l?
Linear dependence implies
ml  
1
c
mk
!
G ik ( m k   m k )  G il ( m l   m l )  G ik m k  G il m l
In words: Parameters mk and ml cannot be independently determined
as they compensate each other. This is called a trade-off.
Linear Inverse Problems
Computational Geophysics and Data Analysis
21
Trade-Off Fault Zone Waves
Linear Inverse Problems
Computational Geophysics and Data Analysis
22
Calculating Partial Derivatives (1)
Generally we need to calculate the partial derivatives
G ij 
 Fi
m j
( m 10 , m 20 ,..., m M 0 )
… depending on the formulation of the forward problem …
1. For explicit functions Fi, for example:
2 h z j
2
M
di 

mj
2
( xi  x j )  z j
   

2
j 1
G ij
Gravity problem
 ti   
1

u
ix
x  u iy y  u iz z 
Master-Event Method
… we can directly calculate the partial derivatives.
Linear Inverse Problems
Computational Geophysics and Data Analysis
23
Calculating Partial Derivatives (2)
2. The data di are given implicitly through
f i ( d i , m1 , m 2 ,..., m M )  0
we differentiate with respect to mj
f i
m j
G ij 

d i
m j
fi d i
d i m j

fi
m j
0
/
fi
d i
The arguments being the model and data parameter of the
starting model m0. Often the data d0 are obtained by finding the
roots of the topmost equation.
Linear Inverse Problems
Computational Geophysics and Data Analysis
24
Calculating Partial Derivatives (3)
3. In more complicated cases the partial derivatives have to be
obtained by numerical differentiation.
G ij 
d i (..., m j 0   m j ,...)  d i 0
m j
Note that for the evaluation of each element of Gij a solution
of the forward problem is necessary! In cases where the number of
model parameters is large or where the forward problem is very
involved this is impractical. But at least this method always works
(approximately).
Linear Inverse Problems
Computational Geophysics and Data Analysis
25
Summary
Most inverse problems can be formulated as discrete
linear problems either as
d i  G ij m j
… or – if the problem is linearized - …
 d i  G ij  m j
In which case the Gij contains the partial derivatives of the
problem. The elements of Gij contain useful information on the
resolution of the model parameters and linear dependence may
indicate trade-offs between model parameters.
Linear Inverse Problems
Computational Geophysics and Data Analysis
26