OS Support for Building
Distributed Applications:
Multithreaded Programming
using Java Threads
1
Outline








Introduction
Thread Applications
Defining Threads
Java Threads and States
Architecture of Multithreaded servers
Threads Synchronization
Thread Concurrency Models
Summary
2
Introduction


We discuss how Middleware is supported
by the Operating System (OS) facilities at
the nodes of a distributed system.
The OS facilitates:


Encapsulation and protection of resources
inside servers;
It supports invocation of mechanisms
required to access those resources including
concurrent access/processing.
3
Middleware and Network Operating
System (NOS)

Many DOS (Distributed OS) have been investigated, but
there are none in general/wide use. But NOS are in
wide use for various reasons both technical and nontechnical.



Users have much invested in their application software; they
will not adopt to new OS that will not run their applications.
The 2nd reason against the adoption of DOS is that users tend
to prefer to have a degree of autonomy of their machines, even
in a closely knit organisation.
A combination of middleware and NOSs provides an
acceptable balance between the requirement of
autonomy and network transparency.


NOS allows users to run their favorite word processor.
Middleware enables users to take advantage of services that
become available in their distributed system.
4
Operating system layers and
Middleware
Applic ations, s ervic es
Middleware
OS: kernel,
li braries &
s ervers
OS1
Proc ess es , threads,
c ommunic ati on, ...
OS2
Proc ess es , threads,
c ommunic ati on, ...
Computer &
network hardware
Computer &
network hardware
Node 1

Platform
Node 2
Unix and Windows are two examples of Network Operating Systems – have a
networking capability built into them and so can be used to access remote
resources using basic services such as rlogin, telnet.
5
Core OS components and
functionality
Proc ess manager
Communic ati on
manager
T hread manager
Memory manager
Supervis or
6
A single threaded program
class ABC
{
….
public void main(..)
{
…
..
}
begin
body
end
}
7
A Multithreaded Program
Main Thread
start
Thread A
start
start
Thread B
Thread C
Threads may switch or exchange data/results
8
Single and Multithreaded
Processes
threads are light-weight processes within a process
Single-threaded Process
Multiplethreaded Process
Threads of
Execution
Single instruction stream
Multiple instruction stream
Common
Address Space
9
Multi-Processing (clusters & grids)
and Multi-Threaded Computing
Threaded Libraries, Multi-threaded I/O
Application
Application
Application
Application
CPU
CPU
CPU
Better Response Times in
Multiple Application
Environments
CPU
CPU
CPU
Higher Throughput for
Parallelizeable Applications
10
Web/Internet Applications:
Serving Many Users Simultaneously
PC client
Internet
Server
Local Area Network
PD
A
11
Multithreaded Server: For Serving
Multiple Clients Concurrently
Server Process
Client 1 Process
Server
Threads

Internet
Client 2 Process
12
Modern Applications need Threads (ex1):
Editing and Printing documents in background.
Printing Thread
Editing Thread
13
Multithreaded/Parallel File Copy
reader()
{
- - - - - - - - lock(buff[i]);
read(src,buff[i]);
unlock(buff[i]);
- - - - - - - - }
buff[0]
buff[1]
writer()
{
- - - - - - - - - lock(buff[i]);
write(src,buff[i]);
unlock(buff[i]);
- - - - - - - - - }
Cooperative Parallel Synchronized
Threads
14
Levels of Parallelism
Sockets/
PVM/MPI
Threads
Compilers
CPU
Task i-l
func1 ( )
{
....
....
}
a ( 0 ) =..
b ( 0 ) =..
+
Task i
func2 ( )
{
....
....
}
a ( 1 )=..
b ( 1 )=..
x
Task i+1
func3 ( )
{
....
....
}
a ( 2 )=..
b ( 2 )=..
Load
Code-Granularity
Code Item
Large grain
(task level)
Program
Medium grain
(control level)
Function (thread)
Fine grain
(data level)
Loop (Compiler)
Very fine grain
(multiple issue)
With hardware
15
Multithreading - Multiprocessors
Process Parallelism
CPU
P1
P2
CPU
P3
CPU
time
No of executing process ≥ the number of CPUs
16
Multithreading on Uni-processor

Concurrency Vs Parallelism
 Process Concurrency
P1
P2
CPU
P3
time
Number of Simultaneous execution units > number of CPUs
17
What are Threads?




A piece of code that run in concurrent with
other threads.
Each thread is a statically ordered sequence of
instructions.
Threads are being extensively used to express
concurrency on both single and multiprocessor
machines.
Programming a task having multiple threads of
control – Multithreading or Multithreaded
Programming.
18
Java Threads




Java has built in thread support for
Multithreading
Synchronization
Thread Scheduling
Inter-Thread Communication:





currentThread
yield
sleep
resume
start
run
stop
setPriority
getPriority
suspend
Java Garbage Collector is a low-priority thread.
19
Threading Mechanisms...


Create a class that extends the Thread class
Create a class that implements the Runnable
interface
20
1st method: Extending Thread
class

Threads are implemented as objects that contains a
method called run()
class MyThread extends Thread
{
public void run()
{
// thread body of execution
}
}



Create a thread:
MyThread thr1 = new MyThread();
Start Execution of threads:
thr1.start();
Create and Execute:
new MyThread().start();
21
An example
class MyThread extends Thread {
// the thread
public void run() {
System.out.println(" this thread is running ... ");
}
} // end class MyThread
class ThreadEx1 {
// a program that utilizes the thread
public static void main(String [] args ) {
MyThread t = new MyThread();
MyThread t2 = new MyThread();
// due to extending the Thread class (above)
// I can call start(), and this will call
// run(). start() is a method in class Thread.
t2.start();
t.start();
} // end main()
}
// end class ThreadEx1
22
threads programming

More about threads programming


Using interfaces
Multiple threads in a program
23
2nd method: Threads by
implementing Runnable interface
class MyThread extends ABC implements Runnable
{
.....
public void run()
{
// thread body of execution
}
}
 Creating Object:
MyThread myObject = new MyThread();
 Creating Thread Object:
Thread thr1 = new Thread( myObject );
 Start Execution:
thr1.start();
24
An example
class MyThread implements Runnable {
public void run() {
System.out.println(" this thread is running ... ");
}
} // end class MyThread
class ThreadEx2 {
public static void main(String [] args ) {
Thread t = new Thread(new MyThread());
// due to implementing the Runnable interface
// I can call start(), and this will call run().
t.start();
} // end main()
}
// end class ThreadEx2
25
Life Cycle of Thread
new
start()
wait()
sleep()
suspend()
blocked
runnable
stop()
dead
non-runnable
notify()
slept
resume()
unblocked
26
A Program with Three Java Threads

Write a program that creates 3 threads
27
Three threads example






















class A extends Thread
{
public void run()
{
for(int i=1;i<=5;i++)
{
System.out.println("\t From ThreadA: i= "+i);
}
System.out.println("Exit from A");
}
}
class B extends Thread
{
public void run()
{
for(int j=1;j<=5;j++)
{
System.out.println("\t From ThreadB: j= "+j);
}
System.out.println("Exit from B");
}
}
28
Three threads example








class C extends Thread
{
public void run()
{
for(int k=1;k<=5;k++)
{
System.out.println("\t From ThreadC: k= "+k);
}












}
}
System.out.println("Exit from C");
class ThreadTest
{
public static void main(String args[])
{
new A().start();
new B().start();
new C().start();
}
}
29
Run 1
[user@speedy] threads [1:76] java ThreadTest
From ThreadA: i= 1
From ThreadA: i= 2
From ThreadA: i= 3
From ThreadA: i= 4
From ThreadA: i= 5
Exit from A
From ThreadC: k= 1
From ThreadC: k= 2
From ThreadC: k= 3
From ThreadC: k= 4
From ThreadC: k= 5
Exit from C
From ThreadB: j= 1
From ThreadB: j= 2
From ThreadB: j= 3
From ThreadB: j= 4
From ThreadB: j= 5
Exit from B

30
Run2
[user@speedy] threads [1:77] java ThreadTest
From ThreadA: i= 1
From ThreadA: i= 2
From ThreadA: i= 3
From ThreadA: i= 4
From ThreadA: i= 5
From ThreadC: k= 1
From ThreadC: k= 2
From ThreadC: k= 3
From ThreadC: k= 4
From ThreadC: k= 5
Exit from C
From ThreadB: j= 1
From ThreadB: j= 2
From ThreadB: j= 3
From ThreadB: j= 4
From ThreadB: j= 5
Exit from B
Exit from A

31
Multithreaded Server
Multithreaded Server
Client Process
Server Process
Server
Threads
Client Process
User Mode
Kernel Mode
Message Passing
Facility
32
Architecture for Multithread Servers


Multithreading enables servers to maximize
their throughput, measured as the number of
requests processed per second.
Threads may need to treat requests with
varying priorities:


A corporate server could prioritize request
processing according to class of customers.
Architectures:




Worker pool
Thread-per-request
Thread-per-connection
Thread-per-object
33
Client and server with threads
Thread 2 makes
requests to server
Thread 1
generates
results
Input-output
Receipt &
queuing
T1
Requests
N threads
Client
Server


In worker-pool architecture, the server creates a fixed pool of worker
threads to process requests.
The module “receipt and queuing” receives requests from sockets/ports
and places them on a shared request queue for retrieval by the workers.
34
Alternative server threading
architectures
workers
I/O
remote
objects
a. T hread-per-reques t
per-connec tion threads
remote
objects
b. T hread-per-c onnecti on
per-objec t threads
I/O
remote
objects
c . T hread-per-object
35
Application 1: Multithreaded Dictionary Server –
Demonstrate the use of Sockets and Threads
A Client Program
What is the Meaning of (“Love”)?
“Love”
“A deep, tender, ineffable feeling of
affection and solicitude toward a person ”
Multithreaded
Dictionary Server
(try, dictionary.com)
“Solicitude”
A Client Program
What is the Meaning of (“Love”)?
“care or concern, as for the
well-being of another”
A Client
Program in “C”
A Client
Program in “C++”
36
Synchronisation & Priorities


Thread Synchronisation
Thread Priorities
37
Accessing Shared Resources

Applications Access to Shared Resources need
to be coordinated.



Printer (two person jobs cannot be printed at the
same time)
Simultaneous operations on your bank account.
Can the following operations be done at the same
time on the same account?



Deposit()
Withdraw()
Enquire()
38
Online Bank: Serving Many Customers
and Operations
PC client
Internet Bank
Server
Local Area Network
Bank
Database
PD
A
39
Shared Resources



If one thread tries to read the data and other
thread tries to update the same data, it leads to
inconsistent state.
This can be prevented by synchronising access
to the data.
Use “Synchronized” method:


public synchronized void update()
{


…
}
40
the driver: 3 Threads sharing the
same object
class InternetBankingSystem {
public static void main(String [] args ) {
Account accountObject = new Account ();
Thread t1 = new Thread(new MyThread(accountObject));
Thread t2 = new Thread(new YourThread(accountObject));
Thread t3 = new Thread(new HerThread(accountObject));
t1.start();
t2.start();
t3.start();
// DO some other operation
} // end main()
}
41
Shared account object
between 3 threads
class MyThread implements Runnable {
Account account;
public MyThread (Account s) { account = s;}
public void run() { account.deposit(); }
} // end class MyThread
class YourThread implements Runnable {
Account account;
public YourThread (Account s) { account = s;}
public void run() { account.withdraw(); }
} // end class YourThread
class HerThread implements Runnable {
Account account;
public HerThread (Account s) { account = s; }
public void run() {account.enquire(); }
} // end class HerThread
accoun
t
(shared
object)
42
Monitor (shared object access):
serializes operation on shared object
class Account { // the 'monitor'
int balance;
// if 'synchronized' is removed, the outcome is unpredictable
public synchronized void deposit( ) {
// METHOD BODY : balance += deposit_amount;
}
}
public synchronized void withdraw( ) {
// METHOD BODY: balance -= deposit_amount;
}
public synchronized void enquire( ) {
// METHOD BODY: display balance.
}
43
Thread Priority

In Java, each thread is assigned priority, which
affects the order in which it is scheduled for
running. The threads so far had same default
priority (NORM_PRIORITY) and they are served
using FCFS policy.

Java allows users to change priority:

ThreadName.setPriority(intNumber)



MIN_PRIORITY = 1
NORM_PRIORITY=5
MAX_PRIORITY=10
44
Thread Priority Example
class A extends Thread
{
public void run()
{
System.out.println("Thread A started");
for(int i=1;i<=4;i++)
{
System.out.println("\t From ThreadA: i= "+i);
}
System.out.println("Exit from A");
}
}
class B extends Thread
{
public void run()
{
System.out.println("Thread B started");
for(int j=1;j<=4;j++)
{
System.out.println("\t From ThreadB: j= "+j);
}
System.out.println("Exit from B");
}
}
45
Thread Priority Example
class C extends Thread
{
public void run()
{
System.out.println("Thread C started");
for(int k=1;k<=4;k++)
{
System.out.println("\t From ThreadC: k= "+k);
}
System.out.println("Exit from C");
}
}
class ThreadPriority
{
public static void main(String args[])
{
A threadA=new A();
B threadB=new B();
C threadC=new C();
threadC.setPriority(Thread.MAX_PRIORITY);
threadB.setPriority(threadA.getPriority()+1);
threadA.setPriority(Thread.MIN_PRIORITY);
System.out.println("Started Thread A");
threadA.start();
System.out.println("Started Thread B");
threadB.start();
System.out.println("Started Thread C");
threadC.start();
System.out.println("End of main thread");
}
}
46
Thread Programming models
models
Thread concurrency/operation



The master/worker model
The peer model
A thread pipeline
47
The master/worker model
Program
Resources
Workers
taskX
Files
Databases
Master
taskY
Input (Stream)
main ( )
Disks
taskZ
Special
Devices
48
Example























main() /* the master */
{
forever {
get a request;
switch( request )
case X: pthread_create(....,taskX);
case Y: pthread_create(....,taskY);
....
}
}
taskX() /* worker */
{
perform the task, sync if accessing shared resources
}
taskY() /* worker */
{
perform the task, sync if accessing shared resources
}
....
--Above runtime overhead of creating thread can be solved by thread pool
* the master thread creates all worker thread at program initialization
and each worker thread suspends itself immediately for a wakeup call
from the master
49
The peer model
Program
Input
Resources
Workers
taskX
Files
Databases
taskY
Disks
taskZ
Special
Devices
50
Example




















main()
{
pthread_create(....,thread1...taskX);
pthread_create(....,thread2...taskY);
....
signal all workers to start
wait for all workers to finish
do any cleanup
}
}
taskX() /* worker */
{
wait for start
perform the task, sync if accessing shared resources
}
taskY() /* worker */
{
wait for start
perform the task, sync if accessing shared resources
}
51
A thread pipeline
A thread pipeline
Program
Filter Threads
Stage 1
Stage 2
Stage 3
Input (Stream)
Resources
Files
Files
Files
Databases
Databases
Databases
Disks
Special Devices
Disks
Special Devices
Disks
Special Devices
52
Example
main()
{
pthread_create(....,stage1);
pthread_create(....,stage2);
....
wait for all pipeline threads to finish
do any cleanup
}
stage1() {
get next input for the program
do stage 1 processing of the input
pass result to next thread in pipeline
}
stage2(){
get input from previous thread in pipeline
do stage 2 processing of the input
pass result to next thread in pipeline
}
stageN()
{
get input from previous thread in pipeline
do stage N processing of the input
pass result to program output.
}
53
Java thread constructor and
management methods
Thread(ThreadGroup group, Runnable target, String name)
Creates a new thread in the SUSPENDED state, which will belong to group and be
identified as name; the thread will execute the run() method of target.
setPriority(int newPriority), getPriority()
Set and return the thread’s priority.
run()
A thread executes the run() method of its target object, if it has one, and otherwise
its own run() method (Thread implements Runnable).
start()
Change the state of the thread from SUSPENDED to RUNNABLE.
sleep(int millisecs)
Cause the thread to enter the SUSPENDED state for the specified time.
yield()
Enter the READY state and invoke the scheduler.
destroy()
Destroy the thread.
54
Java thread synchronization calls
thread.join(int millisecs)
Blocks the calling thread for up to the specified time until thread has terminated.
thread.interrupt()
Interrupts thread: causes it to return from a blocking method call such as sleep().
object.wait(long millisecs, int nanosecs)
Blocks the calling thread until a call made to notify() or notifyAll() on object
wakes the thread, or the thread is interrupted, or the specified time has elapsed.
object.notify(), object.notifyAll()
Wakes, respectively, one or all of any threads that have called wait() on object.
55
RPC delay against parameter size
RPC del ay
Requested data
s ize (bytes )
0
1000
2000
Pac ket
s ize
56
A lightweight remote procedure
call
Client
Server
A s tack
4. Execute proc edure
and copy results
1. Copy args
User
A
s tub
s tub
Kernel
2. T rap to Kernel
3. Upc al l
5. Return (trap)
57
Times for serialized and
concurrent invocations
Seri al ised invoc ations
proc es s args
marshal
Send
Rec ei ve
unmarshal
proc es s results
proc es s args
marshal
Send
Conc urrent i nvoc ati ons
proc es s args
marshal
Send
transmis si on
proc es s args
marshal
Send
Rec ei ve
unmarshal
exec ute reques t
marshal
Send
Rec ei ve
unmarshal
proc es s results
Rec ei ve
unmarshal
exec ute reques t
marshal
Send
Rec ei ve
unmarshal
exec ute reques t
marshal
Send
Rec ei ve
unmarshal
proc es s results
Rec ei ve
unmarshal
exec ute reques t
marshal
Send
ti me
Rec ei ve
unmarshal
proc es s results
Client
Server
Client
Server
58
Summary

Operating system provides various types of facilities to support
middleware for distributed system:




Multithreading enables servers to maximize their throughput,
measured as the number of requests processed per second.
Threads support treating of requests with varying priorities.
Various types of architectures can be used in current processing:






encapsulation, protection, and concurrent access and management of
node resources.
Worker pool
Thread-per-request
Thread-per-connection
Thread-per-object
Threads need to be synchronized when accessing and
manipulating shared resources.
New OS designs provide flexibility in terms of separating
mechanisms from policies.
59