Scheduling and Performance Issues
for Programming using OpenMP
Alan Real
ISS, University of Leeds
Overview
• Scheduling for worksharing constructs
– STATIC.
– GUIDED.
– RUNTIME.
• Performance.
SCHEDULE clause
• Gives control over which loop iterations are executed by which
thread.
• Syntax:
– Fortran: SCHEDULE(kind[,chunksize])
– C/C++: schedule(kind[,chunksize])
• kind is one of STATIC, DYNAMIC, GUIDED or RUNTIME
• chunksize is an integer expression with positive value; e.g.:
!$OMP DO SCHEDULE(DYNAMIC,4)
• Default is implementation dependent.
STATIC schedule
• With no chunksize specified
– Iteration space is divided into (approximately) equal
chunks, and one chunk is assigned to each thread (block
schedule)
• If chunksize is specified
– Iteration space is divided into chunks, each of chunksize
iterations. The chunks are assigned cyclically to each thread
(block cyclic schedule)
STATIC schematic
T0
T1
1
T2
T3
46
SCHEDULE(STATIC)
T0
T1
T2
T3
T0
T1
T2
T3
T0
T1
T2
T3
T0
T1
T2
T3
46
1
SCHEDULE(STATIC,4)
•
Block-cyclic schedule is good for triangular matrices to avoid load
imbalance; e.g:
do i=1,n
do j=i,n
block
end do
end do
DYNAMIC schedule
• Divides the iteration space up into chunks of size
chunksize and assigns them to threads on a firstcome first-served basis.
• i.e. as a thread finishes a chunk, it is assigned the
next chunk in the list.
• When no chunksize is specified it defaults to 1.
GUIDED schedule
• Similar to DYNAMIC, but the chunks start off large
and get smaller exponentially.
• The size of the next chunk is (roughly) the number
of remaining iterations divided by the number of
threads.
• The chunksize specifies the minimum size of the
chunks
• When no chunksize is specifies it defaults to 1.
DYNAMIC and GUIDED
T0
T1
T2
1
T2
T3
T0
T1
T0
T2
T1
T3
T1
T0
T2
46
SCHEDULE(DYNAMIC,3)
T0
1
T3
T1
T2
T3 T3
SCHEDULE(GUIDED,3)
T3
T2
T3 T1
T2
T0
46
RUNTIME schedule
• Allows the choice of schedule to be deferred until
runtime
• Set by environment variable OMP_SCHEDULE:
% setenv OMP_SCHEDULE “guided,4”
• There is no chunksize associated with the RUNTIME
schedule
Choosing a schedule
• STATIC is best for balanced loops –least overhead.
• STATIC,n good for loops with mild or smooth load imbalance –
but can introduce “false sharing” (see later).
• DYNAMIC useful if iterations have widely varying loads, but
ruins data locality (can get cache hit).
• GUIDED often less expensive than DYNAMIC, but beware of
loops where first iterations are the most expensive!
• Use RUNTIME for convenient experimentation
Performance
• Easy to write OpenMP but hard to write an efficient
program!
• 5 main causes of poor performance:
–
–
–
–
–
Sequential code
Communication
Load imbalance
Synchronisation
Compiler (non-)optimisation.
Sequential code
• Amdahl’s law: Parallel code will never run faster than
the parts which can only be executed in serial.
– Limits performance.
• Need to find ways of parallelising it!
• In OpenMP, all code outside of parallel regions and
inside MASTER, SINGLE and CRITICAL directives is
sequential.
– This code should be as small as possible.
Communication
• On Shared memory machines, communication is “disguised” as
increased memory access costs.
– It takes longer to access data in main memory or another
processor’s cache than it does from local cache.
• Memory accesses are expensive!
– ~100 cycles for a main memory access compared to 1-3 cycles for
a flop.
• Unlike message passing, communication is spread throughout
the program.
– Much harder to analyse and monitor.
Caches and coherency
•
•
•
•
Shared memory programming assumes that a shared variable has a
unique value at a given time.
Caching means that multiple copies of a memory location may exist in
the hardware.
To avoid two processors caching different values of the same memory
location, caches must be kept coherent.
Coherency operations are usually performed on the cache lines in the
level of cache closest to memory
– (level 2 cache on most systems)
•
Maxima has:
– 64kb Level 1 cache, 32byte cache lines (4 d.p. words).
– 8Mb Level 2 cache, 512byte cache lines (64 d.p. words).
P
P
P
P
L1C
L1C
L1C
L1C
L2C
L2C
L2C
Interconnect
Memory
Memory hierarchy
L2C
Coherency
• Highly simplified view:
– Each cache line can exist in one of 3 states:
• Exclusive: the only valid copy in any cache.
• Read-only: a valid copy but other caches may contain it.
• Invalid: out of date and cannot be used.
– A READ on an invalid or absent cache line will be cached as readonly.
– A READ on an exclusive cache line will be changed to read only.
– A WRITE on a line not in an exclusive state will cause all other
copies to be marked invalid and the written line to be marked
exclusive.
Exclusive
W
Read-Only
R
P
Invalid
W
P
P
P
L1C
L1C
R
L1C
L2C
L1C
L2C
L2C
Interconnect
Memory
Coherency example
L2C
False sharing
• Cache lines consist of several words of data.
• What happens when two processors are both writing to
different words on the same cache line?
– Each write will invalidate the other processors copy.
– Lots of remote memory accesses.
• Symptoms:
– Poor speedup
– High, non-deterministic numbers of cache misses.
– Mild, non-deterministic, unexpected load imbalance.
False sharing example 1
integer count(maxthreads)
!$OMP PARALLEL
:
count(myid)=count(myid)+1
•
– Each thread writes a different element of the same array
– Words with consecutive addresses are more than likely in the same cache
line.
Solution:
– Pad array by dimension of cache line:
parameter (linesize=16)
integer count(linesize,maxthreads)
!$OMP PARALLEL
:
count(1,myid) = count(1,myid) + 1
False sharing example 2
• Small chunk sizes:
!OMP DO SCHEDULE(STATIC,1)
do j = 1,n
do i=1,j
b(j)=b(j)+a(i,j)
end do
end do
• Will induce false sharing on b.
Data affinity
• Data is cached on the processors which access it.
– Must reuse cached data as much as possible.
• Write code with good data affinity:
– Ensure the same thread accesses the same subset of program
data as much as possible.
• Try to make these subsets large, contiguous chunks of data.
– Will avoid false sharing and other problems.
1
2
3
4
Data affinity example
i
i
!$OMP DO PRIVATE(i)
do j=1,n
j
do i=1,n
a(i,j)= i+j
end do
end do
!$OMP DO SCHEDULE(STATIC,16)
!$OMP& PRIVATE(i)
i
do j=1,n
do i=1,j
i
b(j)=b(j)+a(i,j)
j
end do
end do
• 1st 16 j’s OK – rest are cache misses!
j
16
j
Distributed shared memory systems
• Location of data in main memory is important.
– OpenMP has no support for controlling this.
• Normally data is placed on the first processor which accesses it.
• Data placement can be controlled indirectly by parallelisation of
data initialisation.
• Not an issue on Maxima as CPUs have reasonably uniform
memory access within each machine.
Load imbalance
• Load imbalance can arise from both communication
and computation.
• Worth experimenting with different scheduling
options
– Can use SCHEDULE(RUNTIME).
• If none are appropriate, may be best to do your own
scheduling!
Synchronisation
• Barriers can be very expensive
– Typically 1000s cycles to synchronise 32 processors).
• Avoid barriers via:
– Careful use of the NOWAIT clause
– Parallelise at the outermost level possible.
• May require re-ordering of loops +/ indicies.
– Consider using point-to-point synchronisation for nearestneighbour type problems.
– Choice of CRITICAL / ATOMIC / lock routines may impact
performance.
Compiler (non-)optimisation
• Sometimes the addition of parallel directives can inhibit the
compiler from performing sequential optimisations.
• Symptoms:
– 1-thread parallel code has longer execution and higher instruction
count than sequential code.
• Can sometimes be cured by making shared data private, or
local to a routine.
Performance tuning
•
•
My code is giving me poor speedup. I don’t know why. What do I do
now?
A:
– Say “this machine/language is a heap of junk”
– Give up and go back to your workstation/PC
•
B:
– Try to classify and localise the sources of overhead.
• What type of problem is it and where in the code does it occur
– Use any available tools that can help you:
• Add timers, use profilers – Sun Performance analyser.
– Fix problems that are responsible for large overheads first.
– Iterate
•
Good Luck!