Nuclear Physics at the Interface
of 21st Century Computational Science
Michael Strayer, Assoc. Dir. Of Science for
Advanced Scientific Computing Research
April 2009
Computational Science
The Early Years
Wilson’s Analogy:
Simulation Scientists
Math/CS Researchers
Computers
Experimentalists
Theorists
Experimental Apparatus
Computational Science
The Early Years
Lax report on Large Scale Computing in Science
and Engineering, 1982
“Perhaps the most significant
applications of scientific
computing come not in the
solution of old problems, but in
the discovery of new
phenomena through numerical
experimentation.”
Computational Science
The Early Years
Lax Report on Large Scale Computing in Science
and Engineering, 1982
It is in the national interest
– that access to constantly updated supercomputing facilities be provided to
scientific and engineering researchers and
– that a large and imaginative user community be trained in their uses and
capabilities.
Future significant improvements may have to come from
architectures embodying parallel processing elements –
perhaps several thousands of processors.
Research in languages, algorithms and numerical analysis will
be crucial in learning to exploit these new architectures
fully.
Computational Science
20 years later
Scientific Discovery through Advanced
Computing (SciDAC)
“SciDAC is unique in the world. There isn't any other
program like it anywhere else, and it has the
remarkable ability to do science by bringing
together physical scientists, mathematicians, applied
mathematicians, and computer scientists who
recognize that computation is not something you do
at the end, but rather it needs to be built into the
solution of the very problem that one is addressing.”
Dr Raymond Orbach, then Under Secretary for Science and
Director, Office of Science, US Department of Energy
in SciDAC Review
Computational Science:
the Third Pillar
Ken
Wilson
Experiment
Theory
Scientific Discovery
through Advanced Computing (SciDAC)
Advancing Science through large-scale
data, modeling and simulation
– Science Application and Science Applications
Partnerships: Astrophysics, Accelerator
Science, Climate, Biology, Fusion,
Petabyte data, Materials & Chemistry,
Nuclear physics, High Energy physics,
QCD, Turbulence, Groundwater
– Centers for Enabling Technology: Address
mathematical and computing systems
software issues
– Institutes: Assist Scientific Applications
teams and foster next generation
computational scientists
http://www.scidac.gov
Facilities
Then and Now
1984
2009
Innovative and Novel Computational
Impact on Theory and Experiment
Innovative and Novel Computational Impact on Theory and Experiment
(INCITE) program started in 2004.
• Small number of computationally intense, high impact projects
• Open to national and international researchers, including industry
• No requirement of DOE or Office of Science funding on topic area
• Peer and computational reviews
Nuclear Physics
2.8%
Accelerator Physics
1.5%
2009 INCITE projects
Applied Mathematics
3.4%
Nuclear Energy
0.8%
Materials Sciences
12.4%
Astrophysics
17.8%
Lattice Gauge Theory
9.8%
Geosciences
1.2%
Atomic/Molecular Physics
0.1%
Environmental Sciences
0.6%
Biological Sciences
11.2%
Fluid Turbulence
0.1%
Plasma Physics
9.8%
Engineering
2.3%
Chemical Sciences
6.3%
Combustion
6.1%
Computer Sciences
2.7%
Climate Research
11.1%
Approximately 890 million
processors awarded in 2009
INCITE Trends
Requests for allocations continue to outpace available resources
Approximately 1.3 Billion Processor hours
available for INCITE in 2010
Delivering the Science
Breakthroughs Selection Panel
Scientific Discovery and
the Role of High End Computing
Panelist
Pete Beckman, Argonne
National Laboratory
Jacqueline Chen, Sandia
National Laboratories
Area
Computer Science
Giulia Galli, University of
California-Davis
James Hack, Oak Ridge
National Laboratory
Chemical Sciences, Nano and
Materials Sciences, Physical
Chemistry
Climate Research, Environmental
Sciences, Geosciences
David Keyes, Columbia
Applied Mathematics
Doug Kothe, Oak Ridge
National Laboratory
Paul Messina, Argonne
National Laboratory
Anthony Mezzacappa,
panel chair, Oak Ridge
National Laboratory
Claudio Rebbi, Boston
University
Nagiza Samatova, North
Carolina State University
Tang, Princeton Plasma
Physics Laboratory
Katherine Yelick,
Lawrence Berkeley
National Laboratory
Computational Fluid Dynamics,
Nuclear Engineering
Computer Science
Accelerator Physics, Combustion,
Fluid Turbulence, Engineering
Physics
Astrophysics, Solar/Space
Physics
High Energy and Nuclear Physics
Biology, Life Sciences
Fusion, Fusion, Energy, Plasma
Physics
Computer Science
Top 10 Computational Science
Accomplishments
Titles in Blue – SciDAC: Titles in Black - INCITE
Rank
Title
1
Modeling the Molecular Basis of Parkinson’s Disease (Tsigelny)
2
Discovery of the Standing Accretion Shock Instability and Pulsar Birth Mechanism in
a Core-Collapse Supernova Evolution and Explosion (Blondin)
3
Prediction and Design of Macromolecular Structures and Functions (Baker)
4
Understanding How Lifted Flame Stabilized in a Hot Coflow (Yoo)
5
New Insights from LCF-enable advanced kinetic simulations of global turbulence in
fusion systems (Tang)
6
High Transition Temperature Superconductivity: A High-Temperature
Superconducting State and a Pairing Mechanism in 2-D Hubbard Model (Scalapino)
7
PETSc: Providing the Solvers for DOE High-Performance Simulations (Smith)
8
Via Lactea II, A Billion Particle Simulation of the Dark Matter Halo of the Milky Way
(Madau)
9
Probing the properties of water through advanced computing (Galli)
10
First Provably Scalable Maxwell Solver Enables Scalable Electromagnetic
Simulations (Kovel)
DCA++ Achieves 1.3 Petaflops
High Tc Superconducting in Cuprates
•
•
•
•
•
2-D Hubbard Model
Study Materials with
Disorders/Impurities
First petaflop application
Spurred community debate
Inspired SNS experiment
DCA++
•
•
Monte Carlo Method
10X Speedup by Scientific Computing
Group at OLCF through:
– Delaying memory intensive
operations (reorder barriers)
– Mixed Precision arithmetic
(move fewer bits per flop)
Doug Scalapino
Philip Anderson
2008 Gordon Bell Prize Winner
Petaflops to Exaflops
14 years Ago
“Building a computer 10 times larger
than all the networked computing
capability in the USA”
2007
“range of applications that would
be materially transformed by the
availability of exascale systems”
14
www.er.doe.gov/ASCR/ProgramDocuments/TownHall.pdf
Scientific Challenges
Workshop Series
Enabling science communities to address scientific grand challenges
through extreme scale computational science
Workshop series:
• Climate Science
• High-Energy Physics
• Nuclear Physics
• Fusion Energy Sciences
• Nuclear Energy
• Biology
• Materials Science and Chemistry
• NNSA
• CS-Math & Architectures
26-28 January 2009, Washington, DC
109 participants; DOE/NSF/NNSA reps
The Nuclear Physics Workshop defined Priority Research Directions in
• Nuclear Astrophysics
Workshop chair: Dr. Glenn Young
• Cold QCD and Nuclear Forces
Co-chairs: Dr. David Dean,
• Nuclear Structure and Reactions
Dr. Martin Savage
• Accelerator Physics
• Hot and Dense QCD
Scientific Challenges Workshop Series
Exascale computing will unify Nuclear Physics
QCD
Nuclear
Structure
Applications in astrophysics,
defense, energy, and medicine
Scientific Challenges Workshop Series
Exascale computing will unify Nuclear Physics
Cold QCD and
Nuclear Forces
Nuclear
Astrophysics
Nuclear Structure
and Reactions
Hot and Dense
QCD
HEP neutrino
Equation of state of nuclear material?
Kaon (strange meson) Condensates?
Sigma (strange) Baryons?
Is SN1987A a black hole or a neutron star?
Scientific Challenges Workshop Series
Example: Nuclei
Scientific Challenges Workshop Series
Example: Hadronic Structure
Scientific Challenges Workshop Series
Examples: Astrophysics
Exascale Townhall:
Software – Findings
“Effective use of exascale systems will
require fundamental changes in how we
develop and validate simulation codes for
these systems and how we manage and
extract knowledge from the massive
amount of data produced.”
Challenges for the Future
Path to Extreme Scale
Applied Math and Computer Science have
contributed even more than Moore’s Law
Hitting the Cliff
Extreme Scale
SciDAC X
Terascale
SciDAC 1
I climb the "Hill of Science,"
I "view the landscape o'er;"
Such transcendental prospect,
I ne'er beheld before!
-Emily Dickinson
Co
re
SciDAC 2
Re
se
ar c
h
Petascale