High Performance Computing
Challenges and Trends
Claude TADONKI
Mines ParisTech – LAL / CNRS / INP2P3
University of Oujda (Morocco) – October 7, 2011
High Performance Computing: Challenges and Trends
Claude TADONKI
The need of competitive HPC systems
Increasing Need of HPC (range of applications and computing power)
Large-scale scientific & technical computing (numerical and non numerical)
Large-scale data mining and statistics in experimental physics
Image and signal processing
Video and 3D animations
Gaming
Molecular biology and structural genomic
Sorting and pattern matching
Meteorology, Atmospheric studies, Medical research
Scientific and technical simulation
High-standard industrial activities, services, and research investigations
And more …
Key Factors
Massively parrallel computers & systems
Dedicated architectures
Specialized processors
Processor frequency
High level integration
Memory space, latency, and bandwidth
High speed interconnection
Advances in parallel algorithm synthesis and programming language features
Powerful compilers
University of Oujda (Morocco) – October 7, 2011
High Performance Computing: Challenges and Trends
Claude TADONKI
Observations
Parallel computing easily justifies nowadays (processor frequency evolution)
• Frequency has been multiplied by 10 since 1993
• The number of transistors in Intel proc has been
multiplied by … 100 000 since 1971
• Core voltage has been reduced by 10 (1,2V , the min is
0,7V)
However, this evolution is closed to its
asymptotic threshold !!!
HPC interest covers a larger spectra of applications, hence a wider audience
Performance expectations in some specific areas are beyond the capacity of standard computers
Research Grid (Oxford University/IBM/United Devices) Oustanding achievment in large-scale molecular
Alternatives and trends Smallpox
analysis.
Multi-core processors
TLP-based parallel machines
GPU (assume a skillful use!)
Hardcoded (embedded) solutions
Reconfigurable HPC
Grid/Meta Computing
University of Oujda (Morocco) – October 7, 2011
High Performance Computing: Challenges and Trends
Claude TADONKI
The K Computer RIKEN / Fujitsu ( JAPAN )
Number one the 37th TOP500
8.162 petaflops (Linpack) 93%
68,544 energy-efficient CPUs
672 computer racks
Complete deployment in 2012
10 petaflops expected (2012)
800 computer racks (2012)
Worldwide shared use
http://www.fujitsu.com/global/news/pr/archives/month/2011/20110620-02.html
University of Oujda (Morocco) – October 7, 2011
High Performance Computing: Challenges and Trends
Claude TADONKI
The Sequoia System LLNL / IBM ( USA )
BlueGene technology
500 teraFLOPS
1.6 million of cores
96 computer racks
https://asc.llnl.gov/computing_resources/sequoia/
98,304 compute nodes
1.6 petabytes of memory
10 times faster than today’s
Most powerful system
Sequoia in 1 hour
=
6.7 billion people calculating 24h/24h during 320 years
University of Oujda (Morocco) – October 7, 2011
High Performance Computing: Challenges and Trends
About sustained performance
Claude TADONKI
Even an optimal algorithm will run at a
fraction of the peak performance !!!
Before we calculate, we need data
Time-to-CPU could be long % flops
Memory space decreases with level
Shared use of memory
slowdown
Synchronization and data exchange
Control flow (if, while, for, case, …)
University of Oujda (Morocco) – October 7, 2011
High Performance Computing: Challenges and Trends
Important considerations
Claude TADONKI
vital on embedded systems
Energy consumption and dissipation
significant heat and cause of failure
Integration ( more powerful unit/system in a smaller chip/surface )
Total cost of the system and maintenance issues
Programmability (consider the case of the IBM CELL)
Computing nodes interconnection (topology and speed)
Accessibility (remote and shared use among several entities)
Software and tools (system, programming, monitoring, profiling, …)
Lifetime and evolution of the system (extensibility, devices change/upgrade, …)
University of Oujda (Morocco) – October 7, 2011
High Performance Computing: Challenges and Trends
Claude TADONKI
Fundamental aspects
Design of efficient parallel algorithms (modelling & scheduling)
Complexity & Performance analysis
Algorithmic and programming paradigms
Code generation and transformations (automatic parallelization)
Compilation techniques
University of Oujda (Morocco) – October 7, 2011
High Performance Computing: Challenges and Trends
Claude TADONKI
Programming models
Distributed memory parallel programming (MPI)
Shared memory parallel programming (OpenMP, Pthreads)
Accelerator-based programming (GPU, FPGA, CELL, …)
Vector programming (SSE, VMX, SPU intrinsics, …)
Hybrid programming (MPI+OpenMP/Pthread, CPU+GPU, PPU+SPU, …)
University of Oujda (Morocco) – October 7, 2011
High Performance Computing: Challenges and Trends
Claude TADONKI
University of Oujda (Morocco) – October 7, 2011