Space Science Center, University of New Hampshire,
Durham, NH
GEM Summer Workshop, Portsmouth, VA, June 19, 2014

• Why data assimilation?
• Data coverage and the daily forecast.
• A step back: subjective and objective analysis.
• Model initialization and the assimilation cycle.
• Can we use this in the magnetosphere?
• DA in regional models.
• Parameter estimation and bias correction
• Ensemble prediction.

• Weather forecasts have shown dramatic improvements over the past 60(!) years.
• These improvements are largely due to the use of data assimilation, and better data coverage.
• Yes, we would like to have that for the magnetosphere too.
From: Kalnay,
2003

Current magnetosphere models stink!
From Pulkkinen et al., 2013

• Weather forecasts use many data sources
• They need to be
“ ingested ” into
“ synoptic maps ” .
• How poor are we with magnetosphere data?
From: Kalnay, 2003

Subjective analysis:
A forecaster draws a weather map from the available observations.
Objective analysis:
Computer algorithms deal with data that are irregular in space and time and produce synoptic maps.
NWS, ~1870

• Forecast models (solving the physics of wet gases) need to be initialized.
• Use observations and previous forecast to generate initial conditions for next forecast.
• Run next forecast.
• Repeat cycle.
• The “ data assimilation ” is part of the model initialization. Note that initialization is not just based on data but also on previous model run.
• The goal is an output that minimizes the combined error of model and data.
DA can also be continuous: 4DVAR, KF:

• The data part initializes the model and “ keeps the model on track.” Data are not on the correct grid, either in space or time!
• The model fills the data gaps. Note that the dynamic equations are essentially transport equations: The model propagates information from data rich regions to data poor regions.
• The DA cycle also filters the data: there are well separated time scales of atmospheric motion: geostrophic motion (days) versus inertia-gravity (Rossby) waves, and sound waves (hours). The faster time scale is considered noise, and DA helps to filter it out.

• DA by itself is not forecasting! There are no data from the future!
• DA combines data with a dynamic model to obtain synoptic maps that are more reliable than either data or model alone.
• Data have errors. With 10 3 - 10 7 data points per day there is no efficient error control possible. Data errors can be of much worse consequence than model errors.
The best model will produce bad results with bad input data (garbage in --> garbage out).
• Data errors come in all kinds: outliers, noise, bias, …
• The overall objective of DA is to minimize the errors between the data and the model.
• In general, there is no a priori knowledge of either the data errors or the model errors.

• Direct Insertion: Replace select model values with data.
Usually gives very bad results. Assumes data are perfect.
• Optimal Interpolation: Weighed averages between data and model. Requires a priori knowledge of model/data errors.
• Variational Methods: Minimize RMS error between model and data in space (3DVAR) or space-time (4DVAR).
Requires the adjoint operator of the model advance operator. Expensive in terms of development manpower and computational power required. But fairly easy to add new data sources. Used in numerical weather prediction.

• Kalman Filter (KF): Integrate model equations together with error-covariance. Adds NxN differential equations.
Impractical for 3D systems.
• Reduced/Truncated Kalman Filter: Apply KF to reduced set of equations. Makes KF feasible, but requires compromises, and there is no procedure that would work for every system. Requires a lot of development work.
• Ensemble Kalman Filter (EnKF): Separates co-variance estimation from model using a Monte-Carlo approach.
Only requires minor modifications to the forecast model.
The tradeoff is that 10s to 100s ensemble member runs are required, which is computational expensive but
(almost) trivial to parallelize. EnKF is increasingly used for numerical weather forecast.

•The boilerplate answer: Well, we do not have that much data. Not a good answer, because we will likely get more data, sooner or later
(see NASA’s great observatory + DMSP +
GOES + …).

• The atmosphere has a lot of inertia and thus there is an “ initialization problem ” (mathematically an IVP).
• The magnetosphere is much more a “ driven dissipative ” system.
• For the magnetosphere we face a “ boundary problem ” (IBVP). The input to the system is imperfectly known from SW monitors (monitors are
never on the sun-Earth line at L1!).
• We need to deal with a lot of turbulence, in the driver (SW/IMF), and possibly with intrinsic stochasticity.
• The magnetosphere is dominated by fast wave modes, the atmosphere is dominated by convective transport. Thus, information travels differently.

• The magnetosphere has a short memory for geomagnetic activity.
• Bimodal response:
– 20 min peak: directly driven.
– 1.5h peak: substorms.
– Generally no response after 2h, but there are exceptions (long quiet periods, tail plasma loading, ~6h (Borovsky et al.).
• By comparison, the impulse response time of the atmosphere is days: today’s
Seattle rain will arrive here next Tuesday!
Bargatze et al., 1985

• A full-blown KF or 3DVAR/4DVAR global MHD based magnetosphere model is beyond the capability of any group, given the state of the field and the funding levels.
• Atmosphere model development has taken decades and $$B (not $$M!) in funding for numerical DA forecast models. And, given the nature of the magnetosphere, the approach taken may not even be the best for the magnetosphere.
• We don’t quite have the necessary magnetosphere in situ data yet.
 We need a smarter approach.

• Some regions of the magnetosphere are transport dominated and have longer time scales:
– Ring current (hours)
– Radiation belts (days to weeks)
– Plasmasphere (days)
– Plasma sheet (hours, but beware of turbulence)
• And those regions are quite well sampled.
• And they are sensitive to model initialization.
• And the models are relatively simple (1d, 2d).
• And they matter.
• Such models are already in develoment.

This slide courtesy of: S. Naehr, F. Toffoletto, A. Chan, Rice U.
Results from direct insertion of perfect data from elliptical orbiter:
• Observing points, mapped to ( μ , L ) space, denoted by “ + ” marks
• Corrections gradually diffuse across L , yield good results if system near steady state
Results from EKF with perfect data from elliptical orbit:
•Well-distributed data from an elliptical orbit provides nearly perfect results
•Caveats: the data, diffusion, and magnetic field are all perfect in this simulation

• A simple but possibly useful approach for global magnetosphere simulations might be “ ensemble prediction ” (EP).
• EP is increasingly used for long-term weather prediction.
• EP is based on many model runs with perturbed initial and/or boundary conditions.
• EP provides a “ mean forecast ” and also probabilities.
From Kalnay, 2003

• Developed by NCAR
• Would be expensive to develop from scratch.
• Requires only small modifications to the forecast model.
• Can work with any data set as long as there is a projection operator from model space to data space.
• DART is a well developed and well tested EnKF framework that has been used with more than a dozen atmospheric and oceanic models.

• Global magnetosphere models contain numerous numerical parameters that are poorly known.
• Examples: parameterized anomalous resistivity, Knight relation fudge factors to determine e- precipitation, plasma properties at inner boundary, ….
• The normal “procedure” to optimize such parameters is a “change parameter  run simulation  compare to data  use your thumb” cycle.
• This approach is very inefficient and basically fails once there are more than a few parameters.
• Model sensitivity to parameters remains unknown, biases are difficult to find, and the procedure yields no information on correlations between parameters.

Augment the state vector u (plasma, field) with vector μ of model parameters for a new state vector x:
The typical model forward step :
With EnKF will then also update the model parameter vector μ (m) t
 μ (m) t+1
.
Such parameter optimization can be done with historical data  no need for real-time feeds, can cover solar cycle.

• (Much) improved model parameter specification and model accuracy.
• Estimation of model biases. Such biases would indicate deficiencies of the physical description. Example: cross polar cap potential.
• Estimation of parameter sensitivity. Some parameters may be unimportant; others may be critical and in need for better physical understanding.
• Estimation of parameter correlations.
• Estimation of error propagation. How do errors (like
SW/IMF misspecification) propagate through the magnetosphere?
• Estimation of optimal ensembles for ensemble predictions.

• OpenGGC is a global magnetosphere-ionospherethermosphere model with long heritage and community model status at the CCMC.
• OpenGGCM is efficient enough that ensembles of 10-
100 runs can be done in real time with reasonable hardware requirements (even on PS3!).
• An increasing number of data sources is available:
– Ionosphere: radars, AMPERE, ground magnetometers, ….
– Magnetosphere: NASA GO, GOES, LANL, DMSP, ….
– Model inputs from L1 or helio prediction (ENLIL).
• An implementation of OpenGGCM – DART EnKF would be quick and affordable.
• Heliosphere prediction could be included into the DA cycle.

• Atmospheric DA methods are well developed and successful, but it took decades and $$B to get to this stage.
• The global magnetosphere DA problem is not analogous to atmospheric DA because the magnetosphere is more of a driven
dissipative system compared to the mostly inertial atmosphere.
Boundary conditions are more important than initial conditions.
• There is promise for DA in regional magnetosphere models: RC,
RB, plasmasphere.
• Ensemble prediction and EnKF may be the most promising approach for the global magnetosphere, in particular for parameter optimization.
• Suitable EnKF frameworks (DART) are already available.