Latest Advances in “Hybrid” Codes &
their Application to Global
Magnetospheric Simulations
A New Approach to Simulations of
Complex Systems
H. Karimabadi
1,2
, N. Omidi
1,2
Collaborators on application to the magnetosphere:
1
K. B. Quest
1
H. X. Vu
1
D. Krauss-Varban
J. Raeder (UCLA)
1
2
UCSD, San Diego, CA
SciberNet, Inc.
Challenges in Global
Simulations
 Multi-scale Coupled Systems
- Spatial scales vary from centimeters to 200 RE
- Temporal scales vary from less than milliseconds to days
- kinetic effects have large-scale consequences
 Multi-physics
- electron physics: e.g., controls reconnection
- ion physics: e.g., dominates formation of boundaries and transport
- global features and dynamics: e.g., extent of the magnetotail
- coupling to the ionosphere
Ideal Code
1. Each region would evolve based on its
own spatial and temporal scales
2. Allow distinct physical models at different
spatial scales
Mixed Resolution Modeling
Code-code coupling
Standard Recipe
1. Pick one physics model (e.g., MHD or hybrid)
2. Use space and time differencing of the eqns
a. introduce spatial grid
2  
d 2 / dx2 | 
i
2[(  ) /    (  ) /   ]/(    )  
i i1
i
i1 i
b. Advance the equations based on a uniform time step
A Typical Cycle in the
Simulation
Nonuniform Mesh
Some Numerology
Assuming that:
a. Nonuniform mesh saves a factor of 50 in # of cells
b. W Dt = 0.025 (optimistic value!)
c. Can use all 1152 processors of the Blue Horizon
1 min in real time = 1500 minutes of run time (1 day)
Three Approaches to Global
Simulations
 MHD
- used extensively
- does not resolve important ion physics
- not suitable for studies of boundaries & discontinuities
 Hybrid (fluid electrons, kinetic ions)
- it is the next stage of advance in global simulations
- resolves ion spatial scales (ion inertial length) and ion
temporal scales (gyroperiod)
 Full Particle
- not feasible or necessary
Hybrid Simulations
 Formation of boundaries occur on ion
time scales
 The length scale of discontinuities is on
ion spatial scales
 Hybrid code properly captures the
scaling properties of reconnection
process
 Yields details about plasma distribution
functions
 Can be used to study different types of
magnetospheres including asteroids
Why Is the Current State-ofthe-art Not Adequate?
1. Time advance scheme is too inefficient
- Every cell is updated regardless of whether a
change has occurred
- Time step is not based on local conditions
2. Existing time advance schemes do not
allow code-code coupling
Our Roadmap
1. Software: Further develop MRM code
2. Hardware: PC cluster
3. Science:
Scaling study: Phase transition of magnetospheres as
a function of dipole strength
a) 2D global hybrid
b) 3D global hybrid – include
ionospheric model
c) Code-code coupling
MRM Methodology
Time-stepped:
Allow time step to be different in each region
Event driven:
(a) Defines a true temporal mesh where
time step can be different for each cell
(b) Quantities are updated only when a
“change” (event) has occurred
MRM
N1/N2 = 10, B1/B2 = 0.6, b1/b2 = 1/3
MRM Application to Plasma
Sails: I
Regions I and II evolve on time scales
that are different by a factor of more
than 200
MRM Application to Plasma
Sails: II
Event Driven Methodology: I
• Based on concepts from computer science,
control theory, and operations research
• Uses irregularly time-stamped events that only
updates what needs to be updated when it
needs to be updated
• Requires construction of:
a) Simulation Engine
b) Simulation Model: Objects and Events
Event Driven Methodology: II
Event Driven Methodology: III
Event Driven Methodology:
IV
Table 1: Timing of various methods
Simulation
Methodology
Timing
(seconds)
PIC (timestepped)
1237
Discrete (tol =
0.2)
129
Discrete (tol = 2)
42
Discrete (no
persistent
events)
11
PC Cluster vs Blue Horizon
“Blue Horizon is the most powerful computer available to
the U.S. academic community and currently the 8th most
powerful computer in the world.”
OUR CLUSTER
BLUE HORIZON
88
1800+ AMD
1152
375 MHz Power3
processors
2 GB
0.5 GB
2.4 TB
5.1 TB
Particle
Pushes/Sec/processor
1 million
150,000
Cost
$70,000
$50 million
# Processors
RAM /Processor
Disk
Using all of its processors, BH is faster
than our cluster by a factor of 1.96
Simulation Model
Scaling Study: Phase transition of
model magnetospheres
Omidi et al., JGR 2002
Dp = 130 c/wp
Summary
• New MRM codes are much faster
• Global hybrid simulations are now possible: We
can for the first time look at the magnetosphere
on ion scales
• 2D offers many advantages and can be used to
address many important issues
• PC cluster offers many advantages to traditional
supercomputers