INTRODUCTION
Chapter 1
Fundamentals of Parallel Processing
Evolution of Parallel Architectures
SIMD-MIMD-Computers
G. Alaghband
Fundamentals of Parallel Processing
1, Introduction
Fundamentals of Parallel Processing
Parallel Machine and Computations
• The Evolution of Parallel Architectures
– Vector or SIMD Computers
– Multiprocessors or MIMD Computers
G. Alaghband
Fundamentals of Parallel Processing
2, Introduction
Evolution of Computer Architectures Continued
“True” Vector Processors
Single-Instruction Stream Multiple-Data Stream (SIMD)
Multiple arithmetic units with single control unit.
A Typical True SIMD Computer Architecture
G. Alaghband
Fundamentals of Parallel Processing
3, Introduction
Pipelined Vector Processors
Pipelined SIMD
Pipelined arithmetic units with shared memory
A Typical Pipelined SIMD Computer Architecture
G. Alaghband
Fundamentals of Parallel Processing
4, Introduction
Multiprocessors
Multiple-Instruction Stream Multiple-Data Stream (MIMD)
Multiple-Processors/Multi-bank-Memory:
a shared-memory MIMD representation
G. Alaghband
Fundamentals of Parallel Processing
5, Introduction
Multiprocessors
Multiple-Instruction Stream Multiple-Data Stream (MIMD)
Multi-(Processor/Memory) pairs and Communication
a distributed-memory MIMD representation
G. Alaghband
Fundamentals of Parallel Processing
6, Introduction
Pipelined Multiprocessors
Pipelined MIMD
Many instruction streams issue instructions into the pipe alternately:
G. Alaghband
Fundamentals of Parallel Processing
7, Introduction
Heracles Cluster: A hybrid of Shared Memory and
Distributed MIMD & GPU cluster
 Heracles has 18 nodes (distributed)
 Each node has 24 cores that share Memory
 Plus 4 Tesla P100 GPU cluster in one compute node
The cluster model of computing provides the most
challenge in software and system development for
high performance.
For information on the machine and accounts please visit
http://pds.ucdenver.edu/
Heracles Cluster: A hybrid of Shared Memory and
Distributed MIMD & GPU cluster
Each node in the cluster has 2 x Intel Xeon E52650v4 Processors with 24 cores (12 cores per
Processor)
Intel Xeon Processor E5-2600 v4 Product Family