Monday, June 27, 2011
CEDAR-GEM joint workshop: DA tutorial
1
UNDERSTANDING DATA ASSIMILATION
APPLICATIONS TO HIGH-LATITUDE IONOSPHERIC ELECTRODYNAMICS
Tomoko Matsuo
University of Colorado, Boulder
Space Weather Prediction Center, NOAA
Monday, June 27, 2011
CEDAR-GEM joint workshop: DA tutorial
2
What is data assimilation?
Combining Information
x
prior knowledge of the state of system
empirical or physical models (e.g. physical laws)
complete in space and time
observations
directly measured or retrieved quantities
incomplete in space and time
y
Monday, June 27, 2011
CEDAR-GEM joint workshop: DA tutorial
3
Bayes Theorem - Bayesian statistics provides a coherent
probabilistic framework for most of DA approaches [e.g., Lorenc, 1986]
prior
observation
likelihood
p(x ) ∼ N (x b , Pb )
x = x b + �b
p(y |x ) ∼ N (Hx , R)
x = Hx + �b
probability distribution of y when x have a given value
posterior
p(x |y ) ∝ p(y |x )p(x )
Monday, June 27, 2011
CEDAR-GEM joint workshop: DA tutorial
4
Bayes Theorem - Bayesian statistics provides a coherent
probabilistic framework for most of DA approaches [e.g., Lorenc, 1986]
prior
observation
likelihood
p(x ) ∼ N (x b , Pb )
x = x b + �b
p(y |x ) ∼ N (Hx , R)
y = Hx + �y
probability distribution of y when x have a given value
posterior
p(x |y ) ∝ p(y |x )p(x )
p(x |y ) ∼ N (x a , Pa ) where
xa = xb + K(y − Hxb )
Pa = (I − KH)Pb
K = Pb HT (HPb HT + R)−1
Monday, June 27, 2011
CEDAR-GEM joint workshop: DA tutorial
5
What is covariance? – two variables case
Bayes theorem
p(x |y ) ∝ p(y |x )p(x )
prior
p(x ) ∼ N (x b , Pb )
x b = (2.3 2.5)
Pb =
�
0.225 0.05
0.05 0.15
x =
�
observation-likelihood
p(y |x ) ∼ N (Hx , R)
H = (1 0)
x1 : observed
x2 : unobserved
Monday, June 27, 2011
6
CEDAR-GEM joint workshop: DA tutorial
1.0
0.8
0.6
0.4
0.2
0.0
0.0
0.2
0.4
0.6
0.8
1.0
What is covariance? – in spatial sense
0.0
0.2
0.4
0.6
0.8
1.0
lag distance
P1 (µ = 0.5, γ = 0.05)
0.0
0.2
0.4
0.6
0.8
1.0
lag distance
P2 (µ = 1.5, γ = 0.1)
Monday, June 27, 2011
CEDAR-GEM joint workshop: DA tutorial
0.4
0.8
0.0
0.4
0.8
ï3
ï3
ï3 ï1
ï1
ï1
0.8
0.4
0.0
1 1 2 333
ï3
ï3
ï3 ï1
ï1
ï1
0.8
0.4
0.0
1 1 2 333
0.0
x2 ∼ N (0, P2 )
0.0
0.0
0.4
ï3
ï3
ï3 ï1
ï1
ï1
0.8
x1 ∼ N (0, P1 )
1 1 2 333
What is covariance? – in spatial sense
0.4
0.8
7
Monday, June 27, 2011
CEDAR-GEM joint workshop: DA tutorial
8
0.0
0.4
0.8
ï3
ï1
0.4
0.8
1 2 3
x2 ∼ N (0, P2 )
0.0
0.0
ï3
ï1
0.4
0.8
x1 ∼ N (0, P1 )
1 2 3
What is covariance? – in spatial sense
0.0
0.4
0.8
0.4
Ÿ Ÿ
Ÿ
0.1
0.2
0.2
0.5
0.3
ï1
0.8
1 2 3
2-D Correlation Function
0.5
0.3
0.1
0.0
ï3
0.5
0.0
0.4
0.8
0.3
0.2
0.1
HP
Monday, June 27, 2011
CEDAR-GEM joint workshop: DA tutorial
9
Assimilative Mapping of Ionospheric Electrodynamics
[Richmond and Kamide, 1988]
Inverse procedure to infer maps of
!
! !
!
E, ! , I ! , J || , !B
From observations of
!
E
!
I!
!
J ||
!
!B
IS or HF radar, Satellites
IS radar
Satellite or ground-based
magnetometers
Monday, June 27, 2011
CEDAR-GEM joint workshop: DA tutorial
10
Assimilative Mapping of Ionospheric Electrodynamics
[Richmond and Kamide, 1988]
prior
observations
Monday, June 27, 2011
[Lu et al., 1998]
CEDAR-GEM joint workshop: DA tutorial
11
Monday, June 27, 2011
CEDAR-GEM joint workshop: DA tutorial
12
Monday, June 27, 2011
CEDAR-GEM joint workshop: DA tutorial
13
AMIE – relationship among electromagnetic variables
Inverse procedure to infer maps of
!
! !
!
E, ! , I ! , J || , !B
From observations of
!
E
!
I!
!
J ||
!
!B
IS or HF radar, Satellites
IS radar
Satellite or ground-based
magnetometers
linear relationship (for a given
!)
!
! !
!
F( E) = ! , I ! , J || , !B

E = −∇Φ


I⊥ = Σ ⋅ E


J|| = ∇ ⋅ I ⊥
 

I ⊥ , J|| ←⎯→ ΔB
Biot-Savart’s law
Monday, June 27, 2011
CEDAR-GEM joint workshop: DA tutorial
14
AMIE – basis functions as forward operator
functional analysis
Φ=Ψx
Ψ : spherical harmonics
x : coefficients
!
E = −∇Ψ x
forward operator
y = Hx
= F (−∇Ψ) x
linear relationship (for a given
!)
!
! !
!
F( E) = ! , I ! , J || , !B

E = −∇Φ


I⊥ = Σ ⋅ E


J|| = ∇ ⋅ I ⊥
 

I ⊥ , J|| ←⎯→ ΔB
Biot-Savart’s law
Monday, June 27, 2011
CEDAR-GEM joint workshop: DA tutorial
15
Ideas to improve AMIE: Adaptive covariance
xa = xb + K(y − Hxb )
K = Pb (α)HT (HPb (α)HT + R)−1
Maximum likelihood Method
[Dee 1995, Matsuo et al., 2005]
d = y − Hxb
d ∼ N (0, S)
where
S(α) = R + HPb (α)HT
Find alpha that maximizes the following pdf
�
T
−1
1
d S (α)d
p(d|α) = √ J �
exp −
2
2π
detS(α)
�
Monday, June 27, 2011
CEDAR-GEM joint workshop: DA tutorial
16
Ideas to improve AMIE: Adaptive covariance
xa = xb + K(y − Hxb )
For given observations
K = Pb (α)HT (HPb (α)HT + R)−1
Monday, June 27, 2011
17
CEDAR-GEM joint workshop: DA tutorial
Multi-resolution bas
Ideas to improve AMIE: Multi-resolution basis functions
Σ = WH 2 W T
Spherical Harmonics
[Nychka et al., 20
A 1-D wavelet basis.
Multi-resolution based covariance
0.6
0.8
1.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.6
0.8
1.0
0.8
0.4
1.0
1.0
30
0.2
0.8
0.000
0.000
0.0
0.6
0.015
0.004
0.4
−0.015
1.0
25
0.8
−0.004
0.6
nt
0.4
−1e−03
0e+00
8e−04
4e−04
0e+00
0.2
0.2
.6
Mother wavelet functions
1e−03
Father wavelet (scaling) functions
Covariance structure of
wavelet coefficients for
Σ(ν
0.5,
γ = 0.25).Mother wavelet functions
Mother=
wavelet
functions
0.0
Wavelets
−1e−03
0e+00
4e−04
[Nychka et al., 2003]
A 1-D wavelet basis.
0.0
0e+00
8e−04
Σ = WH 2 W T
Mother wavelet functions
1e−03
Father wavelet (scaling) functions
Monday, June 27, 2011
the perturbations are large.
Thejoint
middle
panel shows
the horizontal
CEDAR-GEM
workshop:
DA tutorial
18 vector from the s
longitude by 1° latitude resolution. The black vectors are sem
results through intervals of intense activity and use the same
Ideas to improve AMIE: what to minimize ∆B or E
How to take advantage of new space-based observations?
A National Science Founda
provide fundamental new u
our home in space
Inverse procedure to infer maps of
!
! !
!
E, ! , I ! , J || , !B
Implemented with 66 netwo
the Iridium constellation us
path for magnetic field mea
Yields the first real-time ob
the electric currents that lin
magnetosphere and ionosp
Allows the first continuous
observations sufficient to t
storm-time dynamics
From observations of
!
E
!
I!
!
J ||
!
!B
IS or HF radar, Satellites
IS radar
Satellite or ground-based
magnetometers
Science Objective: Understand the global electrodynamics of our home in space, the coupled
magnetosphere-ionosphere (M-I) system
Technical Implementation: 100-f
magnetic field data returned from
operational, Iridium satellite con
AMPERE is a new observation facility designed to
transform M-I science by providing the first continuous,
global observations of the Birkeland magnetic field-aligned
currents linking the magnetosphere and ionosphere. The M-I
system exhibits tremendous variability most clearly evident in
the enormous differences between average and storm
conditions as shown below. By providing 24/7 real-time
observations of the Birkeland currents with global coverage
throughout all ranges of geomagnetic activity, AMPERE will
be the first ever facility to track the electrodynamic state of
the M-I system throughout geomagnetic storms. AMPERE
will give essential quantitative context for regional and local
observations and provide a key touchstone to test models and
simulations of the natural system.
AMPERE’s breakthrough is achi
quantity of magnetic field data return
Iridium constellation (illustrated below
consists of 66 satellites in 778-km alti
orbits evenly distributed among si
planes. The satellites in each orbit pla
minutes along track. The avionics of
vector magnetometer that is sensitiv
magnetic perturbations of the Bi
magnetometers are sampled every 0.7
typically only one vector sample ev
satellite is transmitted to the grou
latitude spacing is >10° whereas ~1°
resolve the Birkeland currents. A
amount of data sent to the ground u
determination of the Birkeland curren
latitude resolution every nine minute
reconfiguration time is ~20 minutes
frontiers in understanding M-I system
Climate Avg
Storm
Magnetopause
11 Re (subsolar)
< 5 Re
Auroral zone
70q lat
< 50q lat
Potential
10s kV
> 100 kV
Current
~4 MA
~ 20 MA
Energy
10s GW
~ 1 TW
Breakthrough Capability: First continuous, global
Figure 2. of
Type
2 summary
observations
Birkeland
currents plots for 3 August
11:50-12:00 UT in stand
Monday, June 27, 2011
CEDAR-GEM joint workshop: DA tutorial
19
Summary
•  Bayesian statistics as an overarching framework for many
of DA methods
•  Assumptions: Gaussian distribution, Linear H…
•  Role of Covariance in spatial interpolation
•  Applications to high-latitude electrodynamics (AMIE)
•  Functional analysis of
E, Φ, I, J, ∆B
•  Kalman update of spherical hamonics coefficients
•  Current issues
•  Resolution: global function ->> compactly supported functions
•  New space-based magnetometer observations ->> E or ∆B
•  Improve conductance models
•  Adaptive Covariance