The Grand Challenge of
Space Weather Prediction
Gábor Tóth
Center for Space Environment Modeling
University of Michigan
Center for Space Environment Modeling
http://csem.engin.umich.edu
Collaborators
• Tamas Gombosi, Kenneth Powell
• Ward Manchester, Ilia Roussev
• Darren De Zeeuw, Igor Sokolov
• Aaron Ridley, Kenneth Hansen
• Richard Wolf, Stanislav Sazykin
(Rice
University)
• József Kóta
(Univ. of Arizona)
Grants
DoD MURI and NASA CT Projects
Center for Space Environment Modeling
http://csem.engin.umich.edu
Outline of Talk
• What is Space Weather and Why to Predict It?
• Parallel MHD Code: BATSRUS
• Space Weather Modeling Framework (SWMF)
• Some Results
• Concluding Remarks
Center for Space Environment Modeling
http://csem.engin.umich.edu
What Space Weather
Means
Conditions on the Sun and in the solar wind,
magnetosphere, ionosphere, and thermosphere
that can influence
the performance
Space physics
that affectsand
us. reliability
of space-born and ground-based technological
systems and can endanger human life or health.
Center for Space Environment Modeling
http://csem.engin.umich.edu
Affects Earth: The Aurorae
Center for Space Environment Modeling
http://csem.engin.umich.edu
Other Effects of Space Weather
Center for Space Environment Modeling
http://csem.engin.umich.edu
MHD Code: BATSRUS
•
Block Adaptive Tree Solar-wind Roe Upwind Scheme
•
Conservative finite-volume discretization
•
Shock-capturing Total Variation Diminishing schemes
•
Parallel block-adaptive grid (Cartesian and
generalized)
•
Explicit and implicit time stepping
•
Classical and semi-relativistic MHD equations
•
Multi-species chemistry
•
Splitting the magnetic field into B0 + B1
•
Various methods to control the divergence of B
Center for Space Environment Modeling
http://csem.engin.umich.edu
MHD Equations in Conservative
vs. Non-Conservative Form
•
Conservative form is required for correct jump conditions
across shock waves.
•
Energy conservation provides proper amount of Joule
heating for reconnection even in ideal MHD.
•
Non-conservative pressure equation is preferred for
maintaining positivity.
•
Hybrid scheme: use pressure equation where possible.
Center for Space Environment Modeling
http://csem.engin.umich.edu
Splitting
the
Magnetic
Field
• The magnetic field has huge gradients near the Sun and
Earth:
– Large truncation errors.
– Pressure calculated from total energy can become
negative.
– Difficult to maintain boundary conditions.
•
Solution: split the magnetic field as B = B0 + B1 where
B0 is a divergence and curl free analytic function.
– Gradients in B1 are small.
– Total energy contains B1 only.
– Boundary condition for B1 is simple.
Center for Space Environment Modeling
http://csem.engin.umich.edu
Vastly Disparate Scales
•
•
Spatial:
•
Resolution needed at Earth: 1/4 RE
•
Resolution needed at Sun:
•
Sun-Earth distance:
•
1 AU = 215 RS = 23,456 RE
1/32 RS
1AU
Temporal:
•
CME needs 3 days to arrive at Earth.
•
Time step is limited to a fraction of a second
in some regions.
Center for Space Environment Modeling
http://csem.engin.umich.edu
Adaptive Block Structure
Each block is NxNxN
Center for Space Environment Modeling
Blocks communicate with
neighbors through “ghost” cells
http://csem.engin.umich.edu
Parallel Distribution of the Blocks
Center for Space Environment Modeling
http://csem.engin.umich.edu
Optimized Load Balancing
Center for Space Environment Modeling
http://csem.engin.umich.edu
Parallel Performance
Center for Space Environment Modeling
http://csem.engin.umich.edu
Why Explicit Time-Stepping May
Not Be Good Enough
• Explicit schemes have time step limited by CFL
condition: Δt < Δx/fastest wave speed.
• High Alfvén speeds and/or small cells may lead to
smaller time steps than required for accuracy.
• The problem is particularly acute near planets with
strong magnetic fields.
• Implicit schemes do not have Δt limited by CFL.
Center for Space Environment Modeling
http://csem.engin.umich.edu
Building a Parallel Implicit Solver
• BDF2 second-order implicit time-stepping scheme requires
solution of a large nonlinear system of equations at each time
step.
• Newton linearization allows the nonlinear system to be solved
by an iterative process in which large linear systems are solved.
• Krylov solvers (GMRES, BiCGSTAB) with preconditioning are
robust and efficient for solving large linear systems.
• Schwarz preconditioning allows the process to be done in
parallel:
• Each adaptive block preconditions using local data only
• MBILU preconditioner
Center for Space Environment Modeling
http://csem.engin.umich.edu
Getting the Best of Both
Worlds - Partial Implicit
• Fully implicit scheme has no CFL limit, but each
iteration is expensive (memory and CPU)
• Fully explicit is inexpensive for one iteration, but
CFL limit may mean a very small Δt
• Set optimal Δt limited by accuracy requirement:
•
•
•
Solve blocks with unrestrictive CFL explicitly
Solve blocks with restrictive CFL implicitly
Load balance explicit and implicit blocks
separately
Center for Space Environment Modeling
http://csem.engin.umich.edu
Timing Results for Space
Weather on Compaq
Center for Space Environment Modeling
http://csem.engin.umich.edu
Controlling the Divergence of B
•
Projection Scheme (Brackbill and Barnes)
•
•
•
Modify MHD equations for non-zero divergence so it is advected.
Simple and robust but div B is not small. Non-conservative
terms.
Diffusive Control (Dedner et al.)
•
•
•
Expensive on a block adaptive parallel grid.
8-Wave Scheme (Powell and Roe)
•
•
•
Solve a Poisson equation to remove div B after each time step.
Add terms that diffuse the divergence of the field.
Simple but it may diffuse the solution too.
Conservative Constrained Transport (Balsara, Dai, Ryu, Tóth)
•
•
Use staggered grid for the magnetic field to conserve div B
Exact but complicated. Does not allow local time stepping.
Center for Space Environment Modeling
http://csem.engin.umich.edu
Effect of Div B Control Scheme
Center for Space Environment Modeling
http://csem.engin.umich.edu
From Codes To Framework
• The Sun-Earth system consists of many different interconnecting
domains that are independently modeled.
• Each physics domain model is a separate application, which has
its own optimal mathematical and numerical representation.
• Our goal is to integrate models into a flexible software
framework.
• The framework incorporates physics models with minimal
changes.
• The framework can be extended with new components.
• The performance of a well designed framework can supercede
monolithic codes or ad hoc couplings of models.http://csem.engin.umich.edu
Center for Space Environment Modeling
Physics Domains ID
Models
• Solar Corona
SC BATSRUS
•
Eruptive Event Generator
•
Inner Heliosphere
•
Solar Energetic Particles
•
Global Magnetosphere
GM BATSRUS
•
Inner Magnetosphere
IM Rice Convection Model
•
Ionosphere Electrodynamics IE Ridley’s potential solver
•
Upper Atmosphere
Center for Space Environment Modeling
EE BATSRUS
IH BATSRUS
SP Kóta’s SEP model
UA General Ionosphere
Thermosphere Model
(GITM)
http://csem.engin.umich.edu
Space Weather Modeling Framework
Center for Space Environment Modeling
http://csem.engin.umich.edu
The SWMF Architecture
Center for Space Environment Modeling
http://csem.engin.umich.edu
Parallel Layout and
Execution
LAYOUT.in for 20 PE-s
ID
ROOT
SC/IH
GM
IM/IE
LAST STRIDE
#COMPONENTMAP
SC
0
9
1
IH
0
9
1
GM
10
17
1
IE
18
19
1
IM
19
19
1
#END
Center for Space Environment Modeling
http://csem.engin.umich.edu
Parallel Field Line Tracing
•
Stream line and field line tracing is a common problem in
space physics. Two examples:
•
Coupling inner and global magnetosphere models
•
Coupling solar energetic particle model with MHD
•
Tracing a line is an inherently serial procedure
•
Tracing many lines can be parallelized, but
•
Vector field may be distributed over many PE-s
•
Collecting the vector field onto one PE may be too
slow and it requires a lot of memory
Center for Space Environment Modeling
http://csem.engin.umich.edu
Coupling Inner and Global
Magnetosphere
Models
Pressure
Inner magnetosphere model:
needs the field line volumes,
average pressure and density
along field lines connected to
the 2D grid on the ionosphere.
Global magnetosphere model:
needs the pressure correction
along the closed field lines:
n 1
MHD
p
Center for Space Environment Modeling

p
n
MHD

t


n
pRCM  pMHD

http://csem.engin.umich.edu
Interpolated Tracing
Algorithm
1. Trace lines inside blocks
starting from faces.
2. Interpolate and
communicate mapping.
3. Repeat 2. until the mapping
is obtained for all faces.
4. Trace lines inside blocks
starting from cell centers.
5. Interpolate mapping to
cell centers.
Center for Space Environment Modeling
http://csem.engin.umich.edu
Parallel Algorithm without Interpolation
PE 1
PE 2
1. Find next local field line.
2. If there is a local field line then
2a. Integrate in local domain.
2b. If not done send to other PE.
3. Go to 1. unless time to receive.
4. Receive lines from other PE-s.
5. If received line go to 2a.
PE 3
Center for Space Environment Modeling
PE 4
6. Go to 1. unless all finished.
http://csem.engin.umich.edu
Interpolated versus No Interpolation
Interpolated
No interpolation
45
40
CPU time [sec]
35
30
25
20
15
10
5
0
8
16
30
Number of processors
Center for Space Environment Modeling
http://csem.engin.umich.edu
Modeling a Coronal Mass
Ejection
• Set B0 to a magnetogram based potential field.
• Obtain MHD steady state solution.
• Use source terms to model solar wind
acceleration and heating so that steady solution
matches observed solar wind parameters.
• Perturb this initial state with a “flux rope”.
• Follow CME propagation.
• Let CME hit the Magnetosphere of the Earth.
Center for Space Environment Modeling
http://csem.engin.umich.edu
Initial Steady State in the
Corona
•
Solar surface is
colored with the
radial magnetic
field.
•
Field lines are
colored with the
velocity.
•
Flux rope is
shown with
white field lines.
Center for Space Environment Modeling
http://csem.engin.umich.edu
Close-up of the Added Flux Rope
Center for Space Environment Modeling
http://csem.engin.umich.edu
Two Hours After Eruption in the Solar
Corona
Center for Space Environment Modeling
http://csem.engin.umich.edu
65 Hours After Eruption in the
Inner Heliosphere
Center for Space Environment Modeling
http://csem.engin.umich.edu
The Zoom Movie
Center for Space Environment Modeling
http://csem.engin.umich.edu
More Detail at Earth
Density and magnetic
field
at shock arrival time
Pressure and magnetic field
Before shock
After shock
South Turning BZ
Center for Space Environment Modeling
North Turning BZ
http://csem.engin.umich.edu
Ionosphere Electrodynamics
Current
•
Before shock hits.
•
After shock: currents
and the resulting
electric potential
increase.
•
Region-2 currents
develop.
•
Although region-1
currents are strong, the
potential decreases
due to the shielding
effect.
Center for Space Environment Modeling
Potential
http://csem.engin.umich.edu
Upper Atmosphere
•
The Hall conductance is
calculated by the Upper
Atmosphere component and
it is used by the Ionosphere
Electrodynamics.
•
After the shock hits the
conductance increases in the
polar regions due to the
electron precipitation.
•
Before shock
arrival
After shock arrival
Note that the conductance
caused by solar illumination
at low latitudes does not
change significantly.
Center for Space Environment Modeling
http://csem.engin.umich.edu
Performance of the SWMF
Center for Space Environment Modeling
http://csem.engin.umich.edu
Center for Space Environment Modeling
http://csem.engin.umich.edu
2003 Halloween Storm
Simulation with GM, IM and IE
Components
• The magnetosphere during
the solar storm associated
with an X17 solar eruption.
• Using satellite data for
solar wind parameters
•Solar wind speed: 1800
km/s.
• Time: October 29, 0730UT
• Shown are the last closed
field lines shaded with the
thermal pressure.
• The cut planes are shaded
with the values of the electric
current density.
Center for Space Environment Modeling
http://csem.engin.umich.edu
GM, IM, IE Run vs. Observations
Center for Space Environment Modeling
http://csem.engin.umich.edu
Concluding Remarks
• The Space Weather Modeling Framework (SWMF)
uses sate-of-the-art methods to achieve flexible and
efficient coupling and execution of the physics models.
• Missing pieces for space weather prediction:
• Better models for solar wind heating and
acceleration;
• Better understanding of CME initiation;
• More observational data to constrain the model;
• Even faster computers and improved algorithms.
Center for Space Environment Modeling
http://csem.engin.umich.edu