Master/Workers Model
Example
Research Computing
UNC - Chapel Hill
Instructor: Mark Reed
Email: markreed@unc.edu
Master/Workers Model
 Often embarrassingly parallel
 Master:
• decomposes the problem into small tasks
• distributes to workers
• gathers partial results to produce the final
Master
result
 Workers:
• work
• pass results back to master
• request more work (optional)
 Mapping/Load Balancing
• Static
• Dynamic
its.unc.edu
worker
worker
worker
worker
2
Master/Workers Load
Balance
 Iterations may have different and unpredictable run
times
• Systematic variance
• Algorithmic variance
 Goal is to balance load balance and overhead
Some Schemes
 Block decomposition, static chunking
 Round Robin decomposition
 Self scheduling
• assign one iteration at a time
 Guided dynamic self-scheduling
• Assign 1/P of the remaining iterations (P = # procs)
its.unc.edu
3
Bag of Tasks
 embarrassingly parallel
 master/workers model
 Comes up in context of:
• parameter studies, sweep
• Monte Carlo
 need parallel random numbers, e.g. SPRNG
• image processing
its.unc.edu
4
An Illustrative Example
 a master list with a set of tasks
• a list of filenames to read
 each task can be done independently
• read file which has one line, an index and a text
string
 master process gathers all the results and
assembles them
• order text based on index and print result
• assemble jumbled text into coherent poem
its.unc.edu
5
sample input
%more poem.masterlist
poem.100.dat
poem.101.dat
poem.102.dat
poem.103.dat
poem.104.dat
poem.105.dat
poem.106.dat
poem.107.dat
poem.108.dat ...
%more poem.*.dat
::::::::::::::
poem.100.dat
::::::::::::::
77 the
::::::::::::::
poem.101.dat
::::::::::::::
76 that
::::::::::::::
poem.102.dat
::::::::::::::
75 for
its.unc.edu
6
workerbee.c
 See code
its.unc.edu
7
Results
The Road Not Taken
Two roads diverged in a yellow wood,
And sorry I could not travel both
And be one traveler, long I stood
And looked down one as far as I could
To where it bent in the undergrowth;
Then took the other, as just as fair,
And having perhaps the better claim
Because it was grassy and wanted wear,
Though as for that the passing there
Had worn them really about the same,
its.unc.edu
8
Results
And both that morning equally lay
In leaves no step had trodden black.
Oh, I marked the first for another day!
Yet knowing how way leads on to way
I doubted if I should ever come back.
I shall be telling this with a sigh
Somewhere ages and ages hence:
Two roads diverged in a wood, and I,
I took the one less traveled by,
And that has made all the difference.
Robert Frost
its.unc.edu
9