Review Notes for Exam 2
Chapter 5
 What three types of computations could benefit from a pipelined approach?
 Draw an example of a pipeline space-time diagram, with 4 processors, and 5
instances of the application. Be careful to label the diagram completely.
Assuming that each step of the pipeline takes the same time, in your diagram, how
many time slots, and for which processors, are processors idle? How much time
does it take all 5 instances of the application to complete?
 What is the time complexity of the parallel version of insertion sort? Explain
your answer completely.
 Is the parallel version of insertion sort efficient? Explain.
Chapter 6
 What is a barrier?
 True or false: In MPI, all processes arrive to the barrier at the same time.
 True or false: In MPI, all processes leave the barrier at the same time.
 Give the pseudocode for a centralized counter implementation of a barrier with 8
processes. How many time units are required in the master process for the barrier
to complete?
 Draw a diagram of a tree barrier for 8 processes. What are the two phases? How
many time units does it take for the departure phase to complete?
 Draw a diagram of a butterfly barrier for 8 processes. How many time units does
it take for the barrier to complete?
 What is deadlock? Give an example.
 True or false: A correctly coded program in MPI will never deadlock.
 What is the forall construct? Does MPI have this call? What language does have
this call?
 Draw a diagram that shows a pipeline solution for the prefix sum problem with 4
processes.
 What is synchronous iteration? Give an example.
 Give the MPI-like pseudocode for the synchronous solution of a set of linear
equations in parallel. Use the allgather call in your code.
 Give an example of block allocation.
 Give an example of cyclic allocation.
 What happens in Jacobi iteration as you increase the number of processes
allocated to the problem without changing the problem size? Explain your answer
completely.
 Explain the concept of ghost points in the heat distribution problem.
 What is a cellular automata?
 What is the “Game of Life”?
 Give three examples of problems that can be solved in parallel using synchronous
iteration. Describe how you would set up the problem for parallel solution for
each one of these.
Chapter 10, Sorting
 What is the optimal time complexity of sorting in parallel? Justify your answer.
 We talked about two different techniques for doing compare and exchange using
message passing. Give MPI-like pseudocode for both of these. Compare and
contrast the two techniques on at least two criteria.
 We talked about two difference techniques for merging two sublists and then
having two processes retain two different halves. Give MPI-like pseudocode for
both of these. Compare and contrast the two techniques on at least two criteria.
 What is the Odd-Even Transposition sort? Give MPI-like pseudocode for this
algorithm for N processes. Assume that each process holds a single number in
your pseudocode.
 Draw a diagram that shows the execution of parallel mergesort for 8 processes,
where each process holds a single number.
 Why are mergesort and quicksort often not as efficient as other parallel sorting
algorithms? What is the general problem with all tree-based parallel algorithms?
 Starting with the sequence 8 3 4 7 9 2 1 5, show the steps of bitonic mergesort
until the list is sorted into increasing order.
 What is a hypercube?
 Starting with an array of 16 numbers (4 by 4) show the steps of shearsort.
Cluster Networks



















Name the five network layers, in order from bottom to top, that we talked about in
class.
In which layer is CSMA-CD? In which layer is HTTP?
Expand VIA. What is VIA?
What is a low-latency protocol?
What is pinned memory? Why is pinned memory important for reducing latency
in a network protocol?
Explain how a non-blocking send works in TCP. Explain how a non-blocking
send works in VIA. Are they the same?
What does it mean to poll to receive a message? When is polling good? When is
polling bad?
What special hardware characteristics are needed to support VIA?
True or false: VIA supports broadcast in hardware.
True or false: TCP is connection-oriented.
True or false: VIA is connection-oriented.
Expand CSMA-CD.
Briefly explain how CSMA-CD works.
Give two example networks that use CSMA-CD.
What is the transmission rate of Fast Ethernet?
What is the transmission rate of Gigabit Ethernet?
Name two advantages of Gigabit Ethernet for use in a cluster.
Name three disadvantages of Gigabit Ethernet for use in a cluster.
What cluster network appears most frequently in clusters in the Top 500 list?












Who sells Myrinet? (Ans: Myricom, http://www.myri.com)
When was Myrinet developed?
Give the four design goals of Myrinet.
What is the purpose of the STOP message in Myrinet? What is the purpose of the
GO message?
What is hysteresis?
Suppose that the transmission rate of my network is 1 Gbps, that the cable is 25m
long, and that the signal travels in the wire at a rate of 200m/usec. How many bits
are on the wire during communication? You must show your work for full credit.
Expand DMA. What is the purpose of DMA?
What is GM?
Describe three advantages of Myrinet. Describe two disadvantages of Myrinet.
How was the InfiniBand standard developed?
Expand HCA. Expand TCA.
What is meant by the term“OS bypass” when talking about Myrinet and
InfiniBand?