Intro to multi-threaded
programming
in Java
jonas.kvarnstrom@liu.se – 2015

2
Most operating systems allow multitasking
 Multiple tasks are run in parallel (given enough processor cores)
▪ Word processor, file manager, web browser
▪ Background MP3 player, virus checker
▪ …
Tasks are ”separate programs”
with their own code, data, stack
Process 1
Process 2
No
access
Program
Memory
heap
Program
Memory
heap
jonkv@ida
Intro 1: Multitasking

3
Multi-threading (lightweight processes)
 A
▪
▪
▪
single program can have multiple threads of execution
Executed in parallel (given enough CPU cores)
Same heap, where objects are stored
Different stack, keeping track local variables + method calls …
Word Processor
Web Server
No
access
GUI Thread
Print thread
Background
spell checker
Shared
heap
Request
handler
Request
handler
Request
handler
Shared
heap
jonkv@ida
Intro 2: Threads

4
Java is fundamentally multi-threaded
 Create threads using the Thread class
 Manage access to resources using the synchronized keyword
 Handle communication between threads using wait(), notify() methods
 Threads for a standard non-GUI program (Java 7, Windows):
▪ Group: system Max Priority: 10
Reference Handler
[pri 10, Daemon] -- Handles weak references
Finalizer
[pri 8, Daemon] -- Runs finalizers
Signal Dispatcher
[pri 9, Daemon]
Daemon: Background services,
Attach Listener
[pri 5, Daemon]
won’t prevent the program from
Group: main Max Priority: 10
terminating
main [pri 5] -- The thread that calls main()
 Additional Threads for a Swing GUI program (Java 7, Windows):
▪
Java2D Disposer
[pri 10, Daemon]
AWT-Windows
[pri 6, Daemon]
jonkv@ida
Intro 3: Threads and Java

5
To create a thread, first define what it should do
 The object-oriented way: An object with a specific method to run
▪ public class DocSaver implements Runnable {
…
public void run() {
Runnable interface:
… save the document to disk …
A single method
}
}
 Second step: Create an instance of your Runnable class
▪ final Runnable saver = new DocSaver();
 Third step: Create a Thread that will execute the Runnable
▪ final Thread t1 = new Thread(saver, "Document Saver");
Gives a name to the thread
 Fourth step: Start the thread!
▪ t1.start();
Creates an operating system
thread with stack, etc.
jonkv@ida
Intro 4: Creating Threads

6
Threads can be in one of four states:
 New – created with new Thread(…), not yet started
 Runnable – has or wants the CPU
▪ The scheduler determines which runnable thread(s) will run
▪ Time allocation depends on thread priority
 Blocked – waiting for something to happen
▪ Blocking on I/O – waits for I/O to complete
▪ Called sleep()
– waits n milliseconds
▪ Called obj.wait() – waits until notified
 Dead – stopped running
▪ The Runnable's run() method has returned
▪ But the Thread object still exists (not yet garbage collected)
Thread.
start()
Done
waiting 
Returns to
runnable
jonkv@ida
Intro 5: Thread States
7
Threads often share resources
Document saving
thread
Spell checker thread
Document editing
thread
Swing thread
(events, painting)
Document object
What if a document changes while it is being saved?
Requires support for coordination,
mutual exclusion between threads, …
jonkv@ida
Topic 1: Sharing Resources
8
Threads sometimes cooperate to achieve a common purpose
Image Processing on core 1
Coordinating
thread
in
image processing
software
Image Processing on core 2
Image Processing on core 3
Image Processing on core 4
How to communicate results?
Requires support for communication between threads
jonkv@ida
Topic 2: Cooperation
When threads cooperate…

Develop a multi-threaded queue (producer/consumer)
Potential
delays
Producer thread:
Read data from disk,
append to queue
Consumer thread:
Read data from queue,
play a sound
10
jonkv@ida
Communication 1: Producer/Consumer
11
Read data from disk, append to queue
class Producer implements Runnable {
private final MsgQueue queue;
public Producer(MsgQueue q) { this.queue = q; }
public void run() {
Read from queue, decode MP3, play sound
while (true) {
Msg data = [[read]];
class Consumer implements Runnable {
queue.put(data);
private final MsgQueue queue;
}
public Consumer(MsgQueue q) { this.queue = q; }
}
public void run() {
}
while (true) {
Msg data = queue.take();
[[play sound from data]];
}
}
}
jonkv@ida
Communication 2: Producer/Consumer

12
First attempt: Let's use an ArrayList!
 public class MsgQueue {
}
List<Msg> list = new ArrayList<Msg>();
public synchronized void put(Msg msg) {
list.add(msg); // At the end
Problem!
}
public synchronized Msg take() {
If the queue is empty,
return list.remove(0);
there is no element to pop 
}
ArrayIndexOutOfBoundsException
Solution!
The consumer must wait for an element
to arrive from the producer!
Java provides obj.wait() and obj.notify()
for thread communication – tricky!
jonkv@ida
Communication 3: First Attempt

13
Fortunately, Java already provides concurrent queues!
 Usually don't need to use the low-level methods
▪ Easy to make mistakes…
 interface BlockingQueue<E> extends Collection<E>:
▪ Supports blocking and timeouts
What happens if the operation can’t be performed (now)?
Throw
exception
Return
null or false
Block (wait until
the operation can
be performed)
Time out (after a
specific interval)
Insert (queue
may be full)
add(e)
offer(e)
put(e)
offer(e, time,
unit)
Remove
(may be empty)
remove()
poll()
take()
poll(time, unit)
Examine
(may be empty)
element()
peek()
not applicable
not applicable
jonkv@ida
Communication 4: Concurrent Queues

14
Implemented by
▪
▪
▪
▪
▪
▪
▪
ArrayBlockingQueue
SynchronousQueue
LinkedBlockingQueue
LinkedBlockingDeque
DelayQueue
PriorityBlockingQueue
LinkedTransferQueue
– bounded size
– zero capacity, each insert matched with a remove
–
–
– each element is available when its delay expires
–
– where producers may wait for consumers
jonkv@ida
Concurrent (Blocking) Queues
Shared Resources
When is a Program Thread Safe?
Mutual Exclusion: Conceptual View and Java Syntax

Threads accessing the same data must not interfere!
 Simple example: ArrayList class used from multiple threads
▪ public class ArrayList<E> {
private int size = 0;
private E[] elements = (E[]) new Object[100];
public void add(E obj)
{ elements[size] = obj; size++; }
public void clear()
{ elements = (E[]) new Object[100]; size = 0; }
}
16
jonkv@ida
Thread Safety 1: Intro
With a single CPU core…

The computer quickly switches
between active threads (time slicing)

This happens even inside methods!
Timer
Swing
17
With multiple CPU cores…

Multiple threads can be running truly
simultaneously
Timer
Swing
begin update()
begin repaint()
begin update()
begin repaint()
continue
continue
continue
continue
jonkv@ida
Thread Safety 2: Thread Switching

18
jonkv@ida
Thread Safety 3: Within Methods
What happens within a method?
Java source
(text):
Hello.java
Compiler:
javac Hello.java
class HelloWorld {
public static void main(String[] args) {
System.out.println("Hello World!");
…
…
…
…
}
}
Bytecode
(binary intermediate format):
Hello.class
Standardized,
can be distributed
Method void main(java.lang.String[])
b2 00 0b getstatic #12 <Field java.io.PrintS
12 0f
ldc #15 <String "Hello, world">
b6 00 14 invokevirtual #20 <Method void
03
iconst_0
3c
istore_1
a7 00 27 goto 39
b2 00 0b getstatic #12 <Field java.io.PrintS
bb 00 17 new #23 <Class java.lang.StringBu

19
Even with one core:
 Can switch threads after any atomic (indivisible) bytecode operation
 Very fine-grained!
Thread 2 executing a[i] = j
push a onto stack
push i onto stack
push j onto stack
arraystore
store value at top of stack in size
Bytecode
instructions
Thread 1 executing size++
push size value onto stack
increment top of stack
jonkv@ida
Thread Safety 4: Fine-Grained Switching

20
jonkv@ida
Thread Safety 5: Race condition!
Suppose two threads call add(A) / add(B) concurrently
 public void add(E obj) { elements[size] = obj; size++; }
▪ Might expect: [A,B] or [B,A]. This is not guaranteed!
▪ Called a race condition: Depends on "who gets there first"
(exactly how far does one thread get before it is interrupted?)
read size: 0
store A at [0]
read size: 0
inc  1
store size: 1
read size: 1
store B at [1]
read size: 1
inc  2
[A,B] store size: 2
read size: 0
store A at [0]
read size: 0
read size: 0
store B at [0]
read size: 0
inc  1
store size: 1
inc  1
[B] store size: 1
read size: 0
store A at [0]
read size: 0
store B at [0]
read size: 0
inc  1
store size: 1
read size: 1
inc  2
store size: 2 [B,null]

21
One common requirement for thread safety:
 Parallel execution should be equivalent to some serialization
▪ The result should be as if the concurrent calls from multiple threads
had been made in some sequence from a single thread
▪ Only the result counts – if you can still do it concurrently, it's better!
If you do this…
(multiple CPUs)
or this… (one CPU)
The result should be
as if you did this…
or as if you did this.
add(a)
add(a)
add(b)
add(b)
add(a)
add(b)
add(b)
add(a)
jonkv@ida
Thread Safety 6: One Requirement

22
How is this type of thread safety achieved?
 Sometimes through very clever algorithms and data structures
▪ ConcurrentLinkedQueue – threadsafe linked queue
▪ ConcurrentLinkedDeque – threadsafe linked double-ended queue
▪ ConcurrentHashMap
– efficient multithreaded access without locking
▪ ConcurrentSkipListMap – another variation
Methods do
execute like this…
…or like this…
add(a)
add(a)
add(b)
add(b)
...but regardless of scheduling,
correct results are guaranteed!
Beyond the scope of this course!
jonkv@ida
Thread Safety 7: Cleverness

23
How is this type of thread safety achieved?
 Usually through mechanisms for mutual exclusion
Can’t add() to the same list
from two threads concurrently
Make sure
you can’t do this…
… or this …
One thread will have to wait
for the first add() to complete
You can only
do this…
…or this.
add(a)
add(a)
add(b)
add(b)
add(b)
add(a)
add(b)
add(a)
jonkv@ida
Thread Safety 8: Mutexes

24
One mechanism for mutual exclusion: Semaphore / monitor
 semaphore.acquire()
▪ Waits until no other thread has the semaphore,
then "locks" it for this thread
 semaphore.release()
▪ Releases the semaphore,
so that some other thread can use it
list.add(A):
list.semaphore.acquire();
…
list.semaphore.release();
list.add(B):
list.semaphore.acquire();
…
list.semaphore.release();
These functions
use specific low-level
machine instructions
to avoid
race conditions
This allows you to use
the semaphore
at a higher level of
abstraction…
jonkv@ida
Mutex 1: Semaphore/Monitor

Mutual exclusion and semaphores / monitors in Java:
 Every object is associated with a monitor
 Acquired and released through synchronized blocks:
▪ synchronized(myObject) { // Acquire() myObject’s monitor
…
}
// Implicitly release() it again
 Synchronized blocks provide structure to monitors
▪ Automatically release()d when you exit the block:
You can't forget to release them,
even if an exception terminates the method
25
jonkv@ida
Mutex 2: Monitors in Java

Must use a monitor in the list example
 Which one? Create a new object…
▪ public class ArrayList<E> {
private int size
= 0;
private E[] elements
= (E[]) new Object[100];
private Object monitor
= new Object();
public void add(E obj) {
synchronized(monitor) {
// acquire monitor
elements[size] = obj;
size++;
}
// release monitor
}
public void clear() {
synchronized(monitor) {
elements = (E[]) new Object[100]; size = 0;
}
}
}
26
jonkv@ida
Mutex 3: Monitors in ArrayList

27
Suppose thread A calls add(A)
 First: synchronized(monitor) acquires the list’s monitor
Conceptual view – the monitor is
not a Java object with fields!
▪ Atomic test-and-set: acquired is false, so make it true
▪ Make thread A the owner of the monitor
▪
before
synchronized(mon) { … }
acquired false
owner
lockqueue
…
…
Thread object:
Thread A, runnable
A: acquire
A: read size 0
A: store A at 0
…
within
synchronized(mon) { …}
acquired
true
owner
lockqueue
…
…
Thread A
runnable
jonkv@ida
Mutex 4: Acquiring a Monitor

28
…
A: read size 0
B: try acquire
B: block
[other threads
might run here]
If the scheduler lets B run, before A is done:
 Method add() calls synchronized(mon)
▪ Tries to acquire the monitor
▪ Atomic test-and-set: acquired is true, so we fail
▪ Thread B is blocked, placed in a wait queue!
 When thread B is blocked:
▪ The scheduler selects another runnable thread (not necessarily A!)
acquired
true
owner
lockqueue
…
…
Thread A
runnable
Thread B
blocked
Thread C
runnable
Thread D
runnable
jonkv@ida
Mutex 5: Wait Queue
29
 Eventually, thread A is scheduled to run again
▪ Finishes its work (no interference!) and releases the monitor
▪ Thread B is unblocked (runnable),
but thread A continues running
acquired
false
owner
lockqueue
…
…
Thread A
runnable
Thread B
runnable
 Eventually, thread B is scheduled to run again
▪ Thread B is allowed to acquire the monitor
acquired
true
owner
lockqueue
…
…
Thread B
runnable
…
A: read size 0
B: try acquire
B: block
[other threads run]
A: inc  1
A: store size 1
A: release
A: keep working
jonkv@ida
Mutex 6: Wait Queue

30
Be careful: Mutual exclusion is cooperative
Not like placing the object in a safe…
More like this:
ArraySet obj.
ArraySet obj.
header
header
elements: []
size: 0
elements: []
size: 0
▪ public void getSize() { return size; }
▪ No synchronized() block
 can be called in parallel with
add() or any other method!

Fields can be accessed

Methods can be called

Everyone must cooperate
by checking the note
(trying to acquire the monitor)!
jonkv@ida
Mutex 7: Cooperative

Must all methods use synchronization?
 Depends on your class…
▪ // No synchronized()  can be called in parallel with other methods
public void getSize() { return size; }
▪ // Calling unsynchronized getSize() in parallel with clear1()
// returns either the old size, or the new size 0 – OK!
public void clear1() {
synchronized(monitor) {
elements = (E[]) new Object[100]; size = 0;
}
}
▪ // Calling unsynchronized getSize() in parallel with clear2()
// may return an "intermediate" size, that the list never really had!
// (One solution: synchronization in getSize() too)
public void clear2() {
synchronized(monitor) {
while (size > 0) elements[--size] = null;
}
}
31
jonkv@ida
Mutex 8: All Methods Synchronized?

How to ensure thread safety, if mutexes are cooperative?
 Anyone can just skip synchronization, deliberately or by mistake…

Use standard public/private protection!
 Make sure all fields are private
▪ Public fields can be accessed regardless of synchronization
▪ Other threads can read intermediate values of public fields
▪ Other threads can change values while mutators are being executed
▪ But private fields are only accessed within the class – which you control!
 Make sure all methods use synchronization where necessary
32
jonkv@ida
Mutex 9: Cooperative Protection

How do you ensure complete thread safety?
 The same way you write a bug-free program…
▪ Think!
▪ Consider all possible combinations of circumstances!
▪ Do everything right!
 Requires policy / design decisions, which must be followed
 Complicated because high parallelism is desired
▪ That is the reason why we use threads in the first place
▪ Don't want too much mutual exclusion
Sorry, but there are no simple answers!
33
jonkv@ida
Conclusions: Complete Thread Safety

Synchronization has some overhead!
35
Without synchronized:
651 ms
571 ms
566 ms
553 ms
548 ms
▪ public class SynchTester {
public static void main(String[] args) {
for (int i = 0; i < 5; i++) doIt();
}
private static void doIt() {
With synchronized:
SynchTester tester = new SynchTester();
20175 ms
long start = System.currentTimeMillis();
Huge
1368
ms
int value = 1;
proportional diff:
539
ms
for (int i = 0; i < 1_000_000_000L; i++) {
The method is
545
ms
value = tester.increment(value);
trivial in itself
545
ms
}
System.out.println((System.currentTimeMillis()-start) + "ms: " + value);
}
public synchronized int increment(int var) {
In this case, the method is
return var + 1;
dynamically optimized to
}
remove the overhead.
}
Not always possible!
jonkv@ida
Synchronization Overhead

Collection classes: Often used in single-threaded context
 Three alternatives:
Don’t use synchronization

public void add(Object obj) {
// Lots of code adding an element
}
If you use from multiple threads:
Risk incorrect results
(lost elements, …)
Do use synchronization

public void add(Object obj) {
synchronized(monitor) {
// Code adding an element
}
}
Unnecessary overhead
in most cases
Provide both versions!
A lot of redundant code… right?
37
jonkv@ida
Synchronization and Collections

Solution: Do provide two versions, don’t write collections twice
Don’t use synchronization

38
public void
add(Object obj)
{
// Lots of code adding an element
}
Do use synchronization

public synchronized void
add(Object obj)
{
// Quick call to the “real” class:
// Delegation
}
jonkv@ida
Synchronization and Collections (2)

39
To achieve this: Use a synchronization wrapper
 public class SynchList implements List {
private List storage;
private Object monitor = …;
public SynchList(List storage)
public void add(Object obj) {
synchronized(monitor)
}
Implements List 
can be used ”everywhere”
{ this.storage = storage; }
{ storage.add(obj); }
}
synchList
ArrayList
"SynchList"
Object
header:
storage
(data)
Object
header:
size
elements
(data)
0
Uses the decorator
design pattern
Header (data)
length
[0]
[1]
[2]
[3]
4
jonkv@ida
Collections: Synchronization Wrappers

Synchronization wrappers exist in the Collections framework!
 List<String> synchList =
Collections.synchronizedList(new ArrayList<String>());
 …
40
jonkv@ida
Built-in Synchronization Wrappers

42
To schedule arbitrary tasks: java.util.Timer
 Starts a separate thread for each Timer – not dependent on Swing timers
▪ import java.util.Timer;
final TimerTask task = new TimerTask() {
Don’t manipulate
public void run() { System.out.println(“Running!"); }
an existing GUI
};
here:
▪ Timer t = new Timer();
// Schedule a task to run a single time, after 500 ms
t.schedule(task, 500);
Only permitted
in the Swing thread!
▪ // Schedule a task to run repeatedly, starting now,
// 500 ms from execution n ends to execution n+1 begins
t.schedule(task, new Date(), 500);
▪ // Schedule a task to run repeatedly, starting now,
// 500 ms from execution n begins to execution n+1 begins
t.scheduleAtFixedRate(task, new Date(), 500);
…
jonkv@ida
Timers