Best Practices Supporting Science at CSCS
Michele De Lorenzi (CSCS)
Case 1 – Simplify the usage of HPC resources
GPU Accelerated In-Situ Visualization
§  Peter Messmer (NVIDIA Co-Design Lab for
Hybrid Multicore Computing at ETH Zurich)
HPC Workflow
Workstation
Setup
Analysis,
Visualization
Supercomputer
Viz Cluster
Dump,
Checkpointing
Visualization,
Analysis
File System
Best Practices Supporting Science at CSCS
3
Traditional Workflow: Challenges
Lack of interactivity
prevents “intuition”
Workstation
Setup
Supercomputer
High-end viz
neglected due
Analysis,
Visualization to workflow
complexity
Viz Cluster
I/O becomes main
simulation bottleneck
Dump,
Checkpointing
File System
Best Practices Supporting Science at CSCS
4
Viz resources need
to scale with simulation
Visualization,
Analysis
Eliminating the IO Bottleneck: In-Situ ViZ
Workstation
Setup
Analysis,
Visualization
Supercomputer
Viz Cluster
Best Practices Supporting Science at CSCS
5
Eliminating the IO Bottleneck: In-Situ ViZ
Workstation
Setup
Analysis,
Visualization
Supercomputer
Viz Cluster
Best Practices Supporting Science at CSCS
6
Different Visualization Scenarios
§  1) Legacy workflow
§  Separate compute & Viz system
§  Communication via filesystem
Comput
e
GPU
HPC
System
§  2) In-situ partitioned or integrated
Filesyste
m
§  Different nodes for viz & compute
§  Communication via High Performance Network
Viz
GPU
Viz System
§  3) In-situ co-processing
§  Compute and visualization on same node
Comput
e
GPU
Network
HPC System
Comput
e+Viz
nodes
HPC System
Best Practices Supporting Science at CSCS
7
Viz
GPU
Typical Scientific Visualization system
§  Visualization is more than Rendering
Simulation
Filtering
Rendering
Compositing
Visualization
§  Simulation: Data as needed in numerical algorithm
§  Filtering: Conversion of simulation data into data ready for rendering
§
§
Typical operations: binning, down/up-sampling, iso-surface extraction, interpolation, coordinate transformation, subselection, ..
Sometimes embedded in simulation
§  Rendering: Conversion of shapes to pixels (Fragment processing)
§  Compositing: Combination of independently generated pixels into final frame
Best Practices Supporting Science at CSCS
8
Visualization Dataflow
Rendering
Filtering
CPU
Simulation
GPU
Best Practices Supporting Science at CSCS
9
Fully GPU accelerated Visualization
CPU
Simulation
Filteri
ng
Renderi
ng
GPU
Best Practices Supporting Science at CSCS
10
FuLLy GPU accelerated Visualization
CPU
Simulation
Filteri
ng
Renderi
ng
Speedup
GPU
Best Practices Supporting Science at CSCS
11
Bonsai With In-Situ Viz On Daint
Best Practices Supporting Science at CSCS
12
Acknowledgement
§  Bonsai development: Evghenii Gaburov (SURFsara), Jeroen Bédorf (CWI)
§  OpenGL on Daint: Gilles Fourestey (CSCS), Nina Suvanphim (Cray)
§  OpenGL on Titan: Don Maxwell (ORNL)
Thomas Schulthess (CSCS) and Jack Wells (ORNL) for providing access to Daint
and Titan for these experiments
Best Practices Supporting Science at CSCS
13
Case 2: Software as a service
Supporting the MARVEL community
14
MATERIALS’ REVOLUTION: COMPUTATIONAL
DESIGN AND DISCOVERY OF NOVEL MATERIALS
MARVEL
EPFL-ETHZ-UNIBAS-UNIFR-UNIGE-USI-UZH-IBM-CSCSEMPA-PSI
Best Practices Supporting Science at CSCS
15
Access Pattern
MARVEL Researcher
Research
Communit
y
EPFL
Repository
access
AiiDA
Server
Traditional access
(batch jobs)
Access through
AiiDa
CSCS
External Login Access (ELA)
/store
Piz Dora
Best Practices Supporting Science at CSCS
16
Scientific
Community access
Case 3: Supporting scientific communities
The Platform for Advanced Scientific Computing
(PASC)
17
Swiss Platform for Advanced Scientific Computing
§  Promote joint effort to address key scientific issues in different
domain sciences exploiting the potential of the next generation of
HPC architectures
§  Interdisciplinary collaboration between
§  domain scientists,
§  computational scientists,
§  software developers,
§  computing centres
§  hardware developers.
§  Builds on the principle of co-design involving all these actors
Best Practices Supporting Science at CSCS
18
PASC Organization
Best Practices Supporting Science at CSCS
19
PASC instruments
Organized in four pillars:
§ Application Support Network (ASN)
§ Co-design projects
§ Institutional computing system pillar supporting the procurement
of institutional computing systems
§ The training pillar supporting the Swiss Graduate Program
Foundations in Mathematics and Informatics for Computer
Simulations in Science and Engineering (FOMICS).
Best Practices Supporting Science at CSCS
20
Application Support Network (ASN)
§  ASN are intended to coordinate and support the effort in different
domain science areas to pursue the objectives of the project
§  The following Domain Science Networks are active today:
§  Climate - Climate and Atmospheric Modelling
§  Earth - Solid Earth Dynamics
§  Life Sciences - Life Sciences Across Scales
§  Materials - Materials Simulations
§  Physics - Plasma Physics, Astrophysics and Multiscale Fluid Dynamics
Best Practices Supporting Science at CSCS
21
Co-design projects
§  Interdisciplinary projects involving several (or all) actors
§  address the following priorities for optimal exploitation of new and
emerging supercomputing platforms:
§  Refactoring of key application codes and re-engineering of their algorithms.
§  Development of numerical libraries and their integration into application codes.
§  Incorporations of innovative sub-systems (e.g. I/O, data streams, etc.) into
existing simulation codes
§  Development of programming environments or components, performance
tuning and analysis/monitoring tools.
Best Practices Supporting Science at CSCS
22
Supported projects
Physics and Astrophysics:
§  Particles and fields; PI: Laurent Villard (EPFL)
§  DIAPHANE Radiative Transport in astrophysical simulations; PI:
Lucio Mayer (University of Zurich)
Material Science
§  ANSWERS: nano-device simulations; PI: Mathieu Luisier (ETH
Zurich)
§  Electronic Structure Calculations: electronic structure
calculations; PI: Jürg Hutter (University of Zurich)
§  ENVIRON A Library for Electronic-structure Simulations; PI:
Stefan Goedecker (University of Basel)
Best Practices Supporting Science at CSCS
23
Supported projects (continued)
Life Science
§  Angiogenesis in Health and Disease: In-vivo and in-silico; PI:
Petros Koumoutsakos (ETH Zurich)
§  Coupled Cardiac Simulations: HPC Framework for Coupled
Cardiac Simulations; PI: Rolf Krause (University of Lugano), Alfio
Quarteroni (EPFL)
§  Genomic Data Processing; PI: Ioannis Xenarios (University of
Lausanne)
§  HPC-ABGEM: agent-based general ecosystems models; PI:
Christoph Zollikofer (University of Zurich)
Best Practices Supporting Science at CSCS
24
Supported projects (continued)
Geo-physics and Climate
§  GeoPC: hybrid parallel smoothers for multigrid preconditioners; PI: Paul
Tackley (ETH Zurich)
§  GeoScale: A framework for multi-scale seismic modelling and inversion;
PI: Andreas Fichtner (ETH Zurich)
§  Grid Tools: Towards a library for hardware oblivious implementation of
stencil based codes; PI: Oliver Fuhrer (Meteosuisse)
Computer Science
§  Heterogeneous Compiler Platform for Advanced Scientific Codes; PI:
Torsten Hoefler (ETH Zurich)
§  Multiscale applications: Optimal deployment of multiscale applications
on a HPC infrastructure; PI: Bastien Chopard (UNIGE)
Best Practices Supporting Science at CSCS
25
PASC conference
§  PASC organizes yearly a Platform of Advanced Scientific
Computing Conference.
§  The goal is to bring together research groups within diverse
scientific domains to foster interdisciplinary collaborations and to
strengthen (HPC) knowledge exchange.
§  PASC15 – June 1-2, 2015 hosted by ETH Zurich:
http://www.pasc15.org/
Best Practices Supporting Science at CSCS
26
Best Practices Supporting Science at CSCS
27
Thank you for your attention.