Parallel Programming - Exercises
August 13, 2010
You might nd the following exercises and control questions
useful, while learning for the exam. Please note that this is inocial material, which is not meant as a guideline for problems you might encounter
during the exam. I do not provide any guarantees regarding completeness
or correctness of the solutions.
Important note:
All Questions are marked with stars according to their approximate dicult.
*
Small control questions.
**
Those questions are comparable to those you might nd in the exam.
***
Hard or long questions. Spend at least some time thinking about them
and try to understand the solutions.
Contents
1
Exceptions
2
2
Java Threads
4
3
Synchronization
6
4
Deadlocks
12
5
Data races, visibility, volatile
13
6
Programming
16
7
Questions
17
8
Amdahl's law
17
9
OpenMP
18
10 Mutual exclusion
21
1
1
Exceptions
1.1
(*) Explain the dierence between checked and unchecked
exceptions.
1.2
(**) What is the output of the run() method in the
following examples?
1.
class
MyExceptionA
extends
Exception
MyExceptionB
extends
MyExceptionA
{
}
class
{
}
void r u n ( ) throws MyExceptionA {
try {
try {
throw new MyExceptionB ( ) ;
} catch ( MyExceptionA e ) {
S y s t e m . o u t . p r i n t l n ( "A" ) ;
}
throw new MyExceptionA ( ) ;
catch ( MyExceptionB e ) {
}
throw new
finally {
S y s t e m . o u t . p r i n t l n ( "B" ) ;
MyExceptionB ( ) ;
System . o u t . p r i n t l n ( " f i n a l l y " ) ;
throw new
IOException ( ) ;
}
System . o u t . p r i n t l n ( " h a l f w a y
}
catch
( IOException
e)
done " ) ;
{
System . o u t . p r i n t l n ( " I O E x c e p t i o n " ) ;
}
catch
( MyExceptionB
e)
{
S y s t e m . o u t . p r i n t l n ( "B2" ) ;
}
finally
{
System . o u t . p r i n t l n ( " f i n a l l y 2 " ) ;
}
System . o u t . p r i n t l n ( " end " ) ;
}
2.
void r u n ( ) throws E x c e p t i o n ( ) {
try {
throw new I O E x c e p t i o n ( ) ;
} catch ( I O E x c e p t i o n e ) {
S y s t e m . o u t . p r i n t l n ( " IO " ) ;
}
throw new I n t e r r u p t e d E x c e p t i o n ( ) ;
catch ( I n t e r r u p t e d E x c e p t i o n e ) {
}
finally
S y s t e m . o u t . p r i n t l n ( "INT" ) ;
{
System . o u t . p r i n t l n ( " done " ) ;
}
}
2
1.3
1.
(**) Will the following classes compile? If not: Why
not? (Focus on Exception related errors)
cl as s Test {
void A( b o o l a ) {
i f ( a && ! a )
throw new E x c e p t i o n ( ) ;
}
}
2.
cl as s Test {
void A( b o o l a ) throws E x c e p t i o n
throw new I O E x c e p t i o n ( ) ;
}
}
3.
cl as s Test {
void A ( ) throws E x c e p t i o n {
throw new I O E x c e p t i o n ( ) ;
}
void B ( )
try {
{
A( ) ;
}
catch
( IOException
e)
}
}
}
4.
cl as s Test {
void A ( ) throws E x c e p t i o n {
throw new I O E x c e p t i o n ( ) ;
}
void B ( )
try {
{
A( ) ;
}
finally
{
}
}
}
3
{
{
5.
cl as s Test {
void A ( ) throws E x c e p t i o n {
throw new I O E x c e p t i o n ( ) ;
}
void
B( )
throws
IOException
{
A( ) ;
}
}
6.
cl as s Test {
void A ( ) {
throw new
NullPointerException ( ) ;
}
}
7.
cl as s Test {
void A ( ) {
try {
throw new I O E x c e p t i o n ( ) ;
} catch ( I O E x c e p t i o n e ) {
throw new I n t e r r u p t e d E x c e p t i o n ( ) ;
} catch ( I n t e r r u p t e d E x c e p t i o n e ) {
}
}
}
2
Java Threads
2.1
1.
(**) Which of the following snippets do what their
comment says?
c l a s s MyRunnable extends
void r u n ( ) {
Runnable
{
System . o u t . p r i n t l n ( " h i " ) ;
}
}
// S t a r t
new
a
new
Thread (
new
thread
to
print
" hi "
MyRunnable ( ) ) . r u n ( ) ;
4
2.
c l a s s MyThread extends T h r e a d
void r u n ( S t r i n g m e s s a g e ) {
{
System . o u t . p r i n t l n ( m e s s a g e ) ;
}
}
// S t a r t
new
3.
a
new
thread
to
print
" hi "
MyThread ( ) . r u n ( " h i " ) ;
// S t a r t
10
new
// C o n t i n u e
for
(
int
if
worker
all
threads
have
i = 0 ; i < 1 0 ; i ++)
Thread
t
=
new
to
finished
progress
their
workload
in
parallel .
work .
{
WorkerThread ( i )
t . start ();
t . join ();
}
4.
c l a s s MyThread extends
void r u n ( ) {
Thread
{
System . o u t . p r i n t l n ( " h i " ) ;
}
}
// S t a r t
new
5.
a
new
Thread (
new
thread
to
print
" hi "
MyThread ( ) ) . s t a r t ( ) ;
c l a s s MyThread extends T h r e a d
void r u n ( S t r i n g m e s s a g e ) {
{
System . o u t . p r i n t l n ( " h i " ) ;
}
}
// S t a r t
new
2.2
a
new
thread
to
print
" hi "
MyThread ( ) . s t a r t ( ) ;
(*) Draw a diagram containing all thread states and
their transitions.
Hint: There are 5 states (excluding the TIMED_ state which was not covered)
5
3
Synchronization
3.1
(**) Which of the following classes is equivalent to the
rst one?
3.1.1
Set 1
c las s Test {
private int c o u n t e r = 1 0 0 ;
public int synchronized d e c r e m e n t ( int
i f ( counter − arg < 0)
throw new E x c e p t i o n ( ) ;
counter
return
−=
arg )
throws
Exception
arg ;
counter ;
}
}
1.
cl as s Test {
private int c o u n t e r = 1 0 0 ;
public int d e c r e m e n t ( int a r g ) throws
synchronized {
i f ( counter − arg < 0)
throw new E x c e p t i o n ( ) ;
counter
=
counter
−
Exception
{
Exception
{
arg ;
}
return
counter ;
}
}
2.
class Test2 {
private int c o u n t e r = 1 0 0 ;
public int d e c r e m e n t ( int a r g ) throws
int r e t ;
synchronized {
i f ( counter − arg < 0)
throw new E x c e p t i o n ( ) ;
counter
ret
=
=
counter
−
arg ;
counter ;
}
return
ret ;
}
}
6
{
3.
cl as s Test {
private int c o u n t e r = 1 0 0 ;
public int d e c r e m e n t ( int a r g ) throws
synchronized {
i f ( counter − arg < 0)
throw new E x c e p t i o n ( ) ;
counter
}
−=
Exception
{
Exception
{
Exception
{
arg ;
synchronized {
return c o u n t e r ;
}
}
}
4.
cl as s Test {
private int c o u n t e r = 1 0 0 ;
public int d e c r e m e n t ( int a r g ) throws
i f ( counter − arg < 0)
throw new E x c e p t i o n ( ) ;
synchronized {
counter
return
−=
arg ;
counter ;
}
}
}
5.
cl as s Test {
private int c o u n t e r = 1 0 0 ;
public int d e c r e m e n t ( int a r g ) throws
int tmp = c o u n t e r − a r g ;
synchronized {
i f ( tmp < 0 )
throw new E x c e p t i o n ( ) ;
counter
return
= tmp ;
counter ;
}
}
}
6.
cl as s Test {
private A t o m i c I n t e g e r c o u n t e r = new A t o m i c I n t e g e r ( 1 0 0 ) ;
public int d e c r e m e n t ( int a r g ) throws E x c e p t i o n {
i f ( counter . get ( ) − arg < 0)
throw new E x c e p t i o n ( ) ;
return c o u n t e r . addAndGet(− a r g ) ;
}
}
}
7
7.
cl as s Test {
private int
counter
=
100;
private synchronized void
counter
−=
subCounter (
int
arg )
{
arg ;
}
public int d e c r e m e n t ( int a r g ) throws
synchronized {
i f ( counter − arg < 0)
throw new E x c e p t i o n ( ) ;
Exception
{
Exception
{
subCounter ( arg ) ;
return
counter ;
}
}
}
8.
cl as s Test {
private int
private void
counter
counter
=
100;
subCounter (
−=
int
arg )
{
arg ;
}
public int d e c r e m e n t ( int a r g ) throws
synchronized {
i f ( counter − arg < 0)
throw new E x c e p t i o n ( ) ;
subCounter ( arg ) ;
return
counter ;
}
}
}
9. (***)
cl as s Test
private
{
AtomicInteger
counter
=
new
AtomicInteger ( 1 0 0 ) ;
public int d e c r e m e n t ( int a r g ) throws E x c e p t i o n {
while ( true ) {
int o l d V a l u e = c o u n t e r . g e t ( ) ;
int n e w V a l u e = o l d V a l u e − a r g ;
i f ( newValue < 0 )
throw new E x c e p t i o n ( ) ;
i f ( c o u n t e r . compareAndSet ( o l d V a l u e , newValue ) )
return n e w V a l u e ;
}
}
8
3.1.2
Set 2
c l a s s CheapCounter {
private int c o u n t e r ;
public synchronized int
return c o u n t e r ;
getValue ( )
{
}
public void synchronized
increment ( )
{
c o u n t e r ++;
}
}
1.
c l a s s CheapCounter {
private int c o u n t e r ;
public int g e t V a l u e ( )
return c o u n t e r ;
{
}
public void synchronized
increment ( )
{
c o u n t e r ++;
}
}
2.
c l a s s CheapCounter {
private v o l a t i l e int
public int g e t V a l u e ( )
return c o u n t e r ;
counter ;
{
}
public void synchronized
increment ( )
c o u n t e r ++;
}
}
3.
c l a s s CheapCounter {
private v o l a t i l e int
counter ;
public int synchronized
return c o u n t e r ;
getValue ( )
}
public void
increment ( )
{
c o u n t e r ++;
}
}
9
{
{
3.2
(**) Static methods
c las s Test {
synchronized void
A( )
{
. . .
}
s t a t i c synchronized void
B( )
{
. . .
}
}
1. What does method A lock on?
2. What does method B lock on?
3. Do the two methods achieve mutual exclusion? (Is it possible for one
thread to enter A while another thread is inside B?)
3.3
class
(**) This Buer is used to transfer objects between
multiple producers and consumers. Does it work? If
not why?
Buffer
bool
{
full ;
Object
data ;
void p u t ( O b j e c t x ) throws
synchronized ( d a t a ) {
while ( f u l l )
InterruptedException
wait ( ) ;
data
= x;
full
=
true ;
notifyAll ();
}
}
g e t ( ) throws I n t e r r u p t e d E x c e p t i o n
synchronized ( d a t a ) {
while ( ! f u l l )
Object
wait ( ) ;
full
=
false ;
notifyAll ();
return
data ;
}
}
}
10
{
{
3.3.1
3.4
(**) The student who wrote the buer class claims that a slide
from the lecture reads lock data not code and that therefore
his code is correct. How do you respond?
(**) Correct all errors you nd in the following program:
class SynchronizedBuffer {
private boolean f u l l ;
private O b j e c t o b j e c t ;
public synchronized void
while ( f u l l ) {
try {
w r i t e ( Object
obj )
{
wait ( ) ;
}
catch
( InterruptedException
e)
{
e)
{
}
}
object
full
=
=
obj ;
true ;
}
public synchronized
while ( ! f u l l ) {
try {
Object
read ( )
{
wait ( ) ;
}
catch
( InterruptedException
}
}
full
=
return
false ;
object ;
}
}
3.5
3.5.1
Monitors and Semaphores
(*) A student notes that the following code will deadlock because a lock is acquired in incrementTenTimes() and again in
increment(), when called from the loop. Is that a problem?
Why/why not?
class SharedObject {
int x = 0 ;
public void synchronized
increment ( )
{
x++;
}
public void synchronized i n c r e m e n t T e n T i m e s ( )
for ( int i = 0 ; i < 1 0 ; i ++)
increment ( ) ;
}
}
11
{
3.5.2
(**) What happens if you convert the SharedObject class to
semaphores using the technique discussed in the lecture? Does
it work now? Does it no longer work?
3.5.3
(**) How would you solve the problems encountered in the
previous two questions (if any)? (Note that both methods
belong to the public interface of the class)
4
Deadlocks
4.1
(*) What is a deadlock?
4.2
(**) Name two techniques/approaches that can be
used to deal with deadlocks.
4.3
(**) Which of the following programs could deadlock?
Give a small argument on why a deadlock can / cannot
occur.
1.
c l a s s Account {
int b a l a n c e ;
void synchronized t r a n s f e r ( A c c o u n t
synchronized ( t a r g e t ) {
balance
−=
amount ;
target . balance
+= amount ;
}
}
}
2.
Object
a
Object
b =
=
new
new
Object ( ) ;
Object ( ) ;
// T h r e a d A
synchronized
(a)
{
}
// T h r e a d B
synchronized
synchronized
(b)
{
(a)
{
}
}
12
target ,
int
amount )
{
3.
class
MultiSemaphore
Semaphore [ 1 0 ]
void
acquire (
{
semaphores
int [ ]
what )
=
new
. . .
{
A r r a y . s o r t ( what ) ;
for
(
int
w
:
what )
semaphores [w ] . a c q u i r e ( ) ;
}
void
release (
int [ ]
what )
{
A r r a y . s o r t ( what ) ;
for
(
int
w
:
what )
semaphores [w ] . r e l e a s e ( ) ;
}
}
// U s a g e
example :
ms . a c q u i r e (
. . .
ms . r e l e a s e (
// Assume
new int [ ]
{
4,
5,
2
}
);
new int [ ]
{
4,
5,
2
}
);
that
each
thread
// ( A l w a y s
calls
release ,
// ( b e f o r e
releasing ) . . . )
uses
does
the
not
semaphore
call
correctly .
acquire
twice
4. same as (3) but without Array.sort in release
5. same as (3) but without Array.sort in acquire
5
Data races, visibility, volatile
5.1
(*) What is a data race?
5.2
(*) How can data races be prevented?
5.3
(**) What could be the value of x after the following
program has nished?
int
x =
0;
// T h r e a d A :
for
(
int
i =0;
i
<
100;
i
<
50;
i ++)
x++;
// T h r e a d B :
for
(
int
i =0;
i ++)
x=x + 1 ;
13
5.4
(**) How does the situation change if x is dened as
a volatile variable?
5.5
(**) Which of the following operations are guaranteed
to be atomic?
(int i, long l, oat f, double d, volatile int vi, volatile long vl, Integer ii, Object
o,oo)
Type Atomic Not atomic
i=5
[ ]
[ ]
l=5
[ ]
[ ]
ii = 5
[ ]
[ ]
f=5
[ ]
[ ]
d=5
[ ]
[ ]
o = oo
[ ]
[ ]
vi = 5
[ ]
[ ]
vl = 5
[ ]
[ ]
vi++
[ ]
[ ]
vl++
[ ]
[ ]
5.6
(**) Visibility/Ordering
What are the possible outputs of the following programs? Do they always print
5? If not give an execution that leads to another output.
1.
int d a t a
volatile
=
0;
bool
ready ;
// T h r e a d A :
data
=
ready
5;
=
true ;
// T h r e a d B :
while
( ! ready )
;
System . o u t . p r i n t l n ( d a t a ) ;
2.
v o l a t i l e int
bool
data
=
0;
ready ;
// T h r e a d A :
data
=
ready
5;
=
true ;
// T h r e a d B :
while
( ! ready )
;
System . o u t . p r i n t l n ( d a t a ) ;
14
3.
v o l a t i l e int [ ]
data
={0 ,0};
// T h r e a d A :
data [ 0 ]
=
5;
data [ 1 ]
=
1;
// T h r e a d B :
while
( data [ 1 ]
==
0)
;
System . o u t . p r i n t l n ( d a t a [ 0 ] ) ;
4.
v o l a t i l e int [ ]
data
=
new int [ ]
// T h r e a d A :
data
=
new int [ 2 ]
{5 ,1};
// T h r e a d B :
while
( data [ 1 ]
==
0);
System . o u t . p r i n t l n
5.
( data [ 0 ] ) ;
c l a s s MyGreatClass {
public int d a t a ;
public M y G r e a t C l a s s ( int
data
d)
= d;
}
}
MyGreatClass
mgc ;
// T h r e a d A :
mgc =
new
MyGreatClass ( 5 ) ;
// T h r e a d B :
while ( mgc
==
null ) ;
System . o u t . p r i n t l n
( mgc . d a t a ) ;
15
{
{0 ,0};
6
Programming
6.1
(**) Implement a parallel method int sum(int[] data)
that sums all elements of the data array.
6.2
(**) Write a program that increases all values in an
array by 20. The update should be done in parallel:
int [ ]
data ;
. . .
6.3
(**) Implement a counter class with the following interface:
// I n i t i a l
class
value
Counter
// i n c r e m e n t
public void
// w a i t
should
be
zero .
{
stored
until
public void
value
increment ( ) ;
the
counter
reaches
waitForValue (
int
value
value ) ;
}
6.4
(***) Implement a ThreadPool
Creating threads has some overhead. If a lot of small tasks have to be started
this overhead becomes signicant. The overhead can be avoided by using a
threadpool. A threadpool starts a number of threads at startup and then accepts
a tasks in form of Runnable objects (not necessarily Threads) which are assigned
to threads in a FIFO manner. No new threads are created.
class
ThreadPool
// I n i t i a l i z e
// w o r k e r
public
// Send
//One
{
the
thread
threads .
ThreadPool (
int
a
new
Runnable
of
the
worker
// method
as
//No new
thread
public void
soon
as
is
pool
and
create
threadCount
threadCount ) ;
object
threads
to
will
the
thread
invoke
possible .
started .
addTask ( R u n n a b l e
runnable ) ;
}
16
the
pool .
run ( )
7
Questions
7.1
Fairness
7.1.1
(**) Name 4 fairness models and describe their properties.
7.1.2
(*) Which of the following constructs are fair?
any additional assumptions you make.
Type
synchronized {
sem.acquire()
notify()
notifyAll()
7.2
Not fair
[
[
[
[
]
]
]
]
Write down
fair
[
[
[
[
]
]
]
]
Performance
7.2.1
(*) Whats the dierence between CPU time and elapsed time
(wallclock time)?
7.2.2
(*) What is the k-best measurement scheme? Why is it used?
Are there disadvantages to using the scheme?
7.2.3
(*) A program is executed on two processors. To everyone's
surprise it executed 2.5 times faster than on one processor.
Give a possible explanation.
7.3
Parallelism and scaling
7.3.1
(*) Whats the dierence between task and data parallelism?
7.3.2
(*) How would you parallelize:
1. Matrix multiplication
2. Sorting an array of integers
7.3.3
8
(*) What scales better with the number of processors. Task
or data parallelism?
Amdahl's law
8.1
(**) What can you say about the parallel part of the
following programs:
1. A program takes 20 seconds on one processor and 12 seconds on two
processors.
2. A program takes 30 seconds on one processor, 10 on two and 2 seconds on
four processors.
17
8.2
(**) Whats the maximum expected speedup if 60% of
your program can run in parallel
1. On a two processor machine?
2. On a three processor machine?
3. With an innite number of processors?
8.3
(**) Two thirds of a method M can be parallelized.
How many processors are needed to...
1. ...achieve a speedup of 2
2. ...achieve a speedup of 4
8.4
9
(**) A clever optimization speeds up a method M by
a factor of ten. What is the speedup of the whole
program?
OpenMP
9.1
1.
(**) Parallelize the following programs using OpenMP.
Do not modify the programs besides adding //omp
... lines.
int
int
int
int
[ ] [ ]
data ;
result
=
0;
i ;
j ;
______________________________________
for
( i =0;
i <d a t a . l e n g t h ;
i ++)
{
______________________________________
for
( j =0;
j <d a t a [ i ] . l e n g t h ;
j ++)
______________________________________
result
+=
data [ i ] [ j ] ;
}
18
2.
int [ ] d a t a ;
int i ;
int j ;
int a ;
______________________________________
for
( i =0;
i <d a t a . l e n g t h ; i ++)
{
______________________________________
a
=
0;
______________________________________
for ( j = 0 ; j <d a t a . l e n g t h ; j ++)
i f ( i != j )
data [ j ]
a
=
+=
data [ j ]
{
data [ i ] ;
∗ i∗j
;
}
______________________________________
data [ i ]
=
a;
}
3.
int [ ]
int [ ]
int [ ]
int
int
src ;
dst ;
res ;
i ;
j ;
______________________________________
for
( i =0;
i <s r c . l e n g t h ; i ++)
______________________________________
{
______________________________________
for
( j = 0 ; j <s r c . l e n g t h ; j ++)
{
______________________________________
dst [ j ]
+=
s r c [ i ]+ j ;
}
______________________________________
for ( j = 1 ; j <s r c . l e n g t h ; j ++)
{
______________________________________
res [ j ]
+=
res [ j
−1];
}
}
19
4.
int
[ ] [ ]
int
int
int
j ;
data ;
i ;
tmp ;
______________________________________
for
( i =1;
i <d a t a . l e n g t h ;
i ++)
{
______________________________________
for
( j =0;
j <d a t a [ i ] . l e n g t h ;
j ++)
{
______________________________________
tmp =
data [ i ] [ 0 ]
+
data [ i ] [ 2 ]
+
data [ i ] [ j ] ;
______________________________________
if
( tmp <
0)
______________________________________
{
______________________________________
data
[ i ][0]
= tmp ;
}
______________________________________
if
( tmp ==
0)
______________________________________
{
______________________________________
data [ 0 ] [ 0 ] + + ;
}
______________________________________
}
}
20
10
Mutual exclusion
10.1
Lamport's bakery algorithm
A1 :
Entering_A
=
true ;
A2 :
Number_A =
1
+ Number_B ;
A6 :
Entering_A = f a l s e ;
while ( E n t e r i n g _ B ) ;
while ( true ) {
i f ( Number_B == 0 ) break ;
i f ( Number_B >= Number_A ) break ;
A7 :
// C r i t i c a l
section
A8 :
Number_A =
0;
A3 :
A4 :
A5 :
}
B1 :
Entering_B
=
true ;
B2 :
Number_B =
1
+ Number_A ;
Entering_B = f a l s e ;
while ( E n t e r i n g _ A ) ;
while ( true ) {
i f ( Number_A == 0 ) break ;
i f ( Number_A > Number_B ) break ;
B3 :
B4 :
B5 :
B6 :
// n o t e
the
difference
to
}
B7 :
// C r i t i c a l
section
B8 :
Number_B =
0;
Note: This algorithm also works for an arbitrary number of threads:
Entering [ i ]
true ;
=
// a t o m i c i t y
is
Number [ i ]
1
=
Entering [ i ]
for
( j
=
1;
required !
... ,
false ;
=
j
not
+ max ( Number [ 1 ] ,
<= NUM_THREADS;
j ++)
{
&&
i
while ( E n t e r i n g [ j ] ) ;
while ( Number [ j ] != 0 ) {
i f ( Number [ i ] < Number [ j ] )
break ;
i f ( Number [ i ] == Number [ j ]
break ;
}
}
// C r i t i c a l
Number [ i ]
section
=
Number [NUM_THREADS ] ) ;
0;
21
<
j )
A6
10.1.1
(**) Explain the asymmetry in the version for two threads.
Why is it needed? How is this problem solved in the version
that works for N threads?
10.1.2
(**) Is starvation possible with Lamport's Bakery algorithm?
Why / Why not?
10.1.3
(**) Show that the algorithm (for two threads) achieves mutual exclusion.
You can either explain (detailed) why the algorithm works or do a proof by
induction(***).
10.2
class
1:
Claudio's solution
Mutex
{
AtomicInteger
t
AtomicInteger
b =
public void
int n ;
do {
2:
n =
3:
}
while
=
lock ()
new
new
AtomicInteger ( ) ;
AtomicInteger ( ) ;
{
t . getAndIncrement ( ) ;
(n
!=
b . get ( ) ) ;
}
public void
4:
unlock ( )
{
b . increment ( ) ;
}
}
10.2.1
(**) Is starvation possible with Claudio's solution? Why /
Why not?
10.2.2
(**) Either give an execution in which two threads enter the
critical section or show that the algorithm (for two threads)
achieves mutual exclusion.
Topics not covered
• Linearizability
• JCSP
• Equality of monitors and semaphores
• Liveness and safety properties
• ...
22