Inference for Stochastic Dynamics
Manfred Opper
Computer Science
Stochastic dynamics: Why interesting ?
On molecular level kinetics is stochastic: Simple autoregulatory network: X1 Number of mRNA, X2 Number of Proteins, X = (X1, X2).
Elementary processes and process rates:
X1
X1
X2
X2
→
→
→
→
X1 + 1
X1 − 1
X2 + 1
X2 − 1
:
:
:
:
h1(X)
h2(X)
h3(X)
h4(X)
=
=
=
=
α(1 − αcΘ(X2 − xc)) ,
βX1 ,
γX1 ,
δX2 ,
Simulation of process
(Noisy) observations at discrete times.
Inference of state variable.
Stochastic processes as a prior for unobserved
dynamics
ODE model for transcriptional regulation:
dxi
= −λxi(t) + Aiµ(t) + bi
dt
µ(t) = transcription factor activity (unobserved)
xi(t) = mRNA concentration of target gene i (measured).
Day 1: Basics
• Markov processes
• Simulation
• Master - and backward equations
• Diffusion and linear noise approximations
• Stochastic differential equations and Fokker - Planck equation
Markov Processes
• Discrete time (Markov chain)
P (Xn+1 ∈ A|Xn = x, Xn−1 = xn−1, . . . , X0 = x0)
= P (Xn+1 ∈ A|Xn = x)
• Continuous time
P (Xt+dt ∈ A|X0:t = x0:t) = P (Xt+dt ∈ A|Xt = xt)
where x0:t = {x(s)|0 ≤ s ≤ t} denotes the entire path.
Jump Processes
Assume that Xt jumps between discrete states.
• For t ≥ t0 define Transition kernel:
.
P (y, t|x, t0) = P (Xt = y|Xt0 = x)
for t ≥ t0. Process is homogeneous if this depends only on t − t0.
• Short time behaviour defined by transition rate W :
P (y, t + ∆t|x, t) ' δxy + W (y|x, t)∆t
for ∆t → 0. Note that W (x|x, t) =
P
z6=x W (z|x, t).
• Time to next event:
Rt
P (no event in [t0 t)|Xt0 = x) = e
t0 W (x|x,s)
ds
• Density for time t to next event (homogeneous process)
p(t) = λ(x)e−λ(x)t
with λ(x) = −W (x|x)
A basis for simulations
Assume Xt = x and homogeneous process. We are interested in the
probability that next move happens in a small time interval ∆t at a
later time t + t0 and is given by x → x0.
This is a product of two terms
• Probability of next event
P
0
0
0
W
(x|x)t
Next event ∈ [t + t , t + t + ∆t]|Xt = x = −W (x|x) e
∆t
• Conditional probability of jump x → x0
P
0
Xt+t0+∆t = x |Xt+t0+∆t =
6 x, Xt+t0 = x =
0
P Xt+t0+∆t = x |Xt+t0 = x
=
P Xt+t0+∆t 6= x|Xt+t0 = x
W (x0|x)∆t
W (x0|x)
=
P
−W (x|x)
z6=x W (z|x)∆t
Discrete event simulation
1. initialise X0.
2. Let Xt = x. Simulate waiting time t0 to next event with density
0
p(t0) = −W (x|x) eW (x|x)t .
3. Put t := t + t0
0
W (x |x)
4. Draw new state x0 with prob − W (x|x)
5. Output Xt = x0
6. If t < Tmax return to step 2.
This is the basis for the Gillespie algorithm.
Examples
Poisson Process: Xt ∈ N0 and W (x0|x) = λδx0,x+1
20
x(t)
15
10
5
0
0
2
4
6
t
8
10
Telegraph Process: Xt ∈ {0, 1}
W (x0|x) = λδx0,1−x
x(t)
1
0
0
2
4
6
t
8
10
Reaction systems
• Biochemical reaction system defined by a set of chemical equations,
u1,1X(1) + · · · + u1,dX(d) → v1,1X(1) + · · · + v1,dX(d) ,
...
...
... ,
un,1X(1) + · · · + un,dX(d) → vn,1X(1) + · · · + vn,dX(d) ,
with number of molecules X(j) for species j & stoichiometric factors ui,j and vi,j . When reaction i occurs, the number X(j) of
individuals of species is changed to
X 0(j) = X(j) + vi,j − ui,j .
(1)
• Introduce n × d-dimensional net effect reaction matrix Ri,j = vi,j −
ui,j .
X 0 = X + R>ei ,
where ei = (δi,1, δi,2, . . . , δi,n)> denotes the i-th base vector in the
n-dimensional reaction space.
• Dynamics of the system defined by hazard h = (h1, h2, . . . hn). For
mass action kinetics one assumes
hi(x) = ci
−1
d ui,j
Y
Y
(xj − k)
j=1 k=0
• Process rates for MJP
W (x0|x) =
n
X
i=1
for x0 6= x.
δx0−x,R>e hi(x)
i
Examples
• The Lotka-Volterra model consists of two interacting species,
traditionally named preys X1 and predators X2. Hazards:
h(X1 → X1 + 1) = αX1 ,
h(X1 → X1 − 1) = βX1X2 ,
h(X2 → X2 + 1) = δX1X2 , h(X2 → X2 − 1) = γX2 ,
Forward and Backward equations
• Forward (Master) equation
i
X h dP (x, t|x0, t0)
0
0
0
P x , t|x0, t0 W x|x − P (x, t|x0, t0) W x |x .
=
dt
x0 6=x
• Kolmogorov Backward equation
n
o
X
dP (x0, t0|x, t)
0
0
0
0
=
W (y|x) P (x , t |x, t) − P (x , t |y, t)
dt
y6=x
.
• Feynman - Kac: Let ψ(x, t) = E [G(XT )|Xt = x] for T > t.
Then ψ(x, t) fulfils the backward equation
X
dψ(x, t)
=
W (y|x) {ψ(x, t) − ψ(y, t)}
dt
y6=x
with ψ(x, T ) = G(x).
Example: Poisson process
Let P (x, t) ≡ P (x, t|0, 0). The Master equation is
dP (x, t)
= λ (P (x − 1, t) − P (x, t))
dt
This is solved by
(λt)x
−λt
P (x, t) = e
x!
Diffusion approximation
.
Assume that jumps ∆x = x0 − x << x. Try approximation by a continuous state Markov process
Introduce drift (1. Jump moment)
i
1 h
E Xt+∆t − Xt|Xt = x =
f (x) = lim
∆t
X
W (x0|x)(x0 − x) ,
x0 6=x
and diffusion matrix (2. jump moment)
i
1 h
>
D(x) = lim
E (Xt+∆t − Xt)(Xt+∆t − Xt) |Xt = x =
∆t
X
(x0 − x)W (x0|x)(x0 − x)> .
x0 6=x
Drift & diffusion for Lotka - Volterra
h(X1 → X1 + 1) = αX1 ,
h(X1 → X1 − 1) = βX1X2 ,
h(X2 → X2 + 1) = δX1X2 , h(X2 → X2 − 1) = γX2 ,
f (x) =
P
0
0
x0 6=x W (x |x)(x − x) =
Drift & diffusion for Lotka - Volterra
Deterministic limit
Neglect fluctuations (diffusion), i.e. assume
Xt+∆t − Xt = f (Xt)∆t
.
Hence, Xt ≈ m(t) = E[Xt]. This yields ’classical’ rate equation
dm
≈ f (m(t))
dt
Diffusion approximation
Xt+∆t − Xt = f (Xt)∆t + D1/2(Xt)
√
∆t k .
with k i.i.d. ∼ N (0, I), has the same 2 conditional moments f (x) and
D(x). Jump sizes will go → 0 and process becomes continuous!
We will write this (in the limit) as an (Ito) stochastic differential
equation
dXt = f (Xt)dt + D1/2(Xt)dWt
where Wt is the Wiener (Brownian) process, a Gaussian process with
E[W (t)W (t0)] = min(t, t0).
Simulations of Jump and corresponding diffusion
processes
protein
300
molecules
250
200
150
100
0
0,5
1
1,5
t
2
2,5
3
Short time behaviour of transition kernel
0
p x , t + ∆t|x, t =
>
1
1 0
−1
0
=
exp −
(x − x − f (x)∆t D (x) x − x − f (x)∆t
|2πD(x)|
2∆t
Comments
• Diffusions are the only Markov processes with continuous sample
paths.
• Some physicists like to write SDEs in the form
dx
= f (x) + D1/2(x)ξ(t)
dt
which resembles an ODE driven by ’white noise’. This can be
dangerous because the ordinary laws of calculus do not apply.
• Example: Ornstein - Uhlenbeck process
dXt = −γXtdt + σdWt
2
σ
2
In the stationary state E(Xt) = 0 and E(Xt ) = 2γ .
Linear noise approximation
Approximation of a jump process by an SDE is not result of a small
parameter expansion (like 1/Number of molecules. An alternative (Van
Kampen’s Ω - ) expansion yields
• to lowest order
dm
= f (m(t))
dt
• and the linear SDE (of O - U type) for the fluctuations Zt =
Xt − m(t)
dZt = A(m)Zt dt + D1/2(m)dW
∂fi
where A(m)ij = ∂x
.
j x=m
Xt becomes approximated by a Gaussian process !
Forward and backward equations for diffusion
processes
.
Let x = (x1, . . . xd) and f (x) = (f1(x), . . . , fd(x))
• Forward (Fokker - Planck) equation
∂p(x, t|x0, t0)
=
∂t
1 X ∂2 −
(fi(x)p(x, t|x0, t0)) +
Dij (x)p(x, t|x0, t0)
∂x
2
∂x
∂x
i
i j
i
ij
X ∂
• Comment on derivation of Fokker - Planck equation
(x)
2 I, where I is the unit matrix. Then
• Let fi(x) = − ∂U
.
and
D
=
σ
∂xi
a stationary solution to the FP equation is given by
− 22 U (x)
p(x) ∝ e
σ
• Backward Fokker - Planck equation
X
∂
1X
∂2
∂p(x0, t0|x, t)
0
0
=−
fi(x)
p(x , t |x, t)−
Dij (x)
p(x0, t0|x, t)
∂t
∂xi
2 ij
∂xi∂xj
i