Concurrent Servers
Process, fork & threads
ECE 297
ECE 297
Process-based
server
How do you handle
cache updates?
How do you handle
cache invalidation?
Cache
Keep it simple
Cache
Cache
Cache
ECE 297
Cache
Process-based
server
How do you handle
concurrent access to
files?
Careful with writing
to the same file
in different processes!
file
ECE 297
Process versus thread I
Process
• Unit of resource ownership with respect to the execution
of a single program
• Can encompass more than one thread of execution
– E.g., Web browser: More than one thread (process) per
window/tab, GUI, rendering engine etc.
– E.g., Web server: More than one thread for handling
requests
Thread
• Unit of execution
• Belongs to a process
• Can be traced (i.e., list the sequence of instructions)
ECE 297
Process versus thread II
• A.k.a. lightweight process (LWP), threads, multithreaded processes
ECE 297
Process versus thread III
Per process items
• Address space
• Global variables
• Open files
• Child processes
• Pending alarms
• Signal and signal
handlers
• Accounting
information
Per thread items
• Program counter
• Registers
• Stack
ECE 297
Process 1
Process 2
Process 3
Process
Threads
Threads
OS
OS
Use
• Threads are part of the
same “job” and are
actively and closely
cooperating
Use
• Processes are largely
independent and often
compete for resources
ECE 297
Threads
Process
Thread 1’s
stack
Threads
OS
ECE 297
Thread-based server
• Server design alternatives
– Thread-per-request
– Thread-per-client
– Thread-per connection
• The new thread can access all resources
held by the process that created it
• For example, the cache, open data files,
global variables are all available to the threads
– Unlike for process-based servers
p is for POSIX
pthreads API overview
• pthread_create(…): creates a thread
• pthread_wait(…): waits for a specific
thread to exit
• pthread_exit(…): terminates the
calling thread
• pthread_yield(…): calling thread
passes control voluntarily to another
thread
ECE 297
p is for POSIX
pthreads API I
#include <pthread.h>
Thread ID
Thread priority,
initial stack size, …;
NULL for defaults
pthread_create(pthread_t *tid,
const pthread_attr_t *attr,
void *(*func) (void *),
void *arg);
Pointer to
argument for
function
Function to
execute; the
actual “thread”
Returns 0, if OK, positive Exx on error
ECE 297
p is for POSIX
pthreads API IV
• pthread_self(void)
– Returns thread ID to caller
• pthread_detach(pthread_t thread)
– Indicates to system that storage for thread can be
reclaimed
• There are many other pthread API calls, the
above should suffice for our purposes
ECE 297
Thread-based server
void *thread(void *vargp); int *connfdp;
int main(int argc, char **argv) {
…
pthread_t tid;
…
listenfd = socket(…); … listen(listenfd, …)
// main server loop
for( ; ; ) {
connfdp = malloc(sizeof(int));
…
*connfdp = accept(listenfd,
(struct sockaddr *) &clientaddr, &clientlen);
pthread_create(&tid, NULL, thread,
(void *) connfdp);
} // for
} // main
ECE 297
We create the
thread to handle the
connected client.
The actual thread to handle the client
void *thread(void *vargp) {
int connfd;
// detached to avoid a memory leak
pthread_detach(pthread_self());
connfd = *((int *)vargp);
free(vargp);
// do the work, service the client
close(connfd);
return NULL;
This is where the client
gets serviced
}
ECE 297
listenfd = socket(AF_INET, SOCK_STREAM, 0)
…
bind(listenfd, …)
listen(listenfd, …)
Concurrent server
template
for( ; ; ){ …
connfd = accept(listenfd, …);
…
if ( (childPID = fork()) == 0 ){// The Child!
close(listenfd);
//Close listening socket
do the work
//Process the request
exit(0);
}
…
close(connfd);
//Parent closes connfd
}
ECE 297
Issues with thread-based servers
• Must be careful to avoid unintended sharing of
variables
• For example, what happens if we pass the
address of connfd to the thread routine?
pthread_create(&tid, NULL, thread,
(void *)&connfd);
• Must protect access to
intentionally shared data
Would be a shared
variable
– Here, we got around this by creating a new
variable, but in general …
ECE 297
Complications
!
• Imaging a global variable counter in the
process
– For example the storage server in-memory
cache (more complex structure)
– Or the connfd variable
Let’s dissect
the issue in
detail
Shared data & synchronization
Table
Table
Table
What happens if multiple threads concurrently access
shared process state (i.e., memory)?
ECE 297
Concurrently manipulating shared data
• Two threads execute concurrently as part of the
same process
• Shared variable (e.g., global variable)
– counter = 5
• Thread 1 executes
counter
– counter++
• Thread 2 executes
– counter—
• What are the possible values of
counter after Thread 1 and Thread 2
executed?
ECE 297
Machine-level implementation
•
•
Implementation of “counter++”
register1 =
counter
register1 =
register1 + 1
counter =
register1
Implementation of “counter--”
register2 =
counter
register2 =
register2 – 1
counter =
register2
ECE 297
Possible execution sequences
counter++
Context Switch
counter--
Context Switch
Context Switch
Context Switch
ECE 297
counter--
Context Switch
counter++
Interleaved execution
• Assume counter is 5 and interleaved
execution of counter++ (P) and counter– (C)
T1: r1
= counter (register1 = 5)
T1: r1
= r1 + 1
(register1 = 6)
T2 : r2
= counter (register2 = 5)
context
T2 : r2
= r2– 1
(register2 = 4)
switch
T1 : counter = r1
(counter = 6)
T2 : counter = r2
(counter = 4)
• The value of counter may be either 4 or 6,
where the correct result should be 5.
ECE 297
Race condition
• Race condition:
– Several threads manipulate shared data
concurrently. The final value of the data
depends upon which thread finishes last.
• In our example (interleaved execution) for
c++ last, result would be 6, and for c-- last,
result would be 4 (correct result should be 5)
• To prevent race conditions, concurrent
processes must be synchronized.
ECE 297
The moral of this story
• The statements
counter++;
counter--;
must each be executed atomically.
• Atomic operation means an operation that
completes in its entirety without interruption.
• This is achieved through synchronization
primitives (semaphores, locks, condition
variables, monitors, disabling of IRPs …).
ECE297
Synchronization primitives
•
•
•
•
Semaphore (cf. ECE344)
Monitor (cf. ECE344)
Condition variable (cf. ECE344)
Lock
– Prevent data inconsistencies due to race
conditions
– A.k.a. mutex (mutual exclusion)
– Use to protect shared data within a process
– Can not be used across processes
• Need to use semaphore instead
ECE 297
Mutex: Mutual exclusion
pthread_mutex_lock(pthread_mutex_t *mtpr)
pthread_mutex_unlock(pthread_mutex_t *mtpr)
Returns 0, if OK, positive Exx on error
• There are other abstractions, but the mutex should
suffice for us
• NB: In ECE344 we learn how to implement locks.
ECE 297
The pthreads mutex (lock)
pthread_mutex_t my_cnt_lock =
PTHREAD_MUTEX_INITIALIZER;
int counter=0;
pthread_mutex_lock( & my_cnt_lock );
counter++;
pthread_mutex_unlock( & my_cnt_lock );
…
ECE 297
Mutex is for mutual exclusion
pthread_mutex_t my_cnt_lock =
PTHREAD_MUTEX_INITIALIZER
For
statically
allocated
mutexes.
pthread_mutex_lock(& my_cnt_lock);
Guaranteed to execute
counter--;
atomically
pthread_mutex_unlock(&
my_cnt_lock);
pthread_mutex_lock(& my_cnt_lock);
Guaranteed to execute
atomically
counter++;
pthread_mutex_unlock(& my_cnt_lock);
ECE 297
Possible execution sequences
lock
lock
lock
counter++
lock
counter-unlock
unlock
Context Switch
Context Switch
lock
lock
lock
counter--
counter++
lock
unlock
unlock
ECE 297
Watch out for I
• For all shared data access you must use a
synchronization mechanism
• For Milestone 4 based on threads, you can
get by with the mutexes
• Other useful mechanisms in pthreads are
– pthread_join(…)
– pthread_cond_wait(…) & pthread_cond_signal()
• Bugs due to race conditions are extremely
difficult to track down
– Non-deterministic behaviour of code
ECE 297
Watch out for II
• You can not make any assumption about thread
execution order or relative speed
• Threaded code must use thread-safe functions
– Functions that use no static variables, no global
variables, don’t return pointers to static variables
• Otherwise need to protect call to non-thread-safe code with
mutexes
• Non-thread-safe code also called non-reentrant code
– Function local data is allocated on the stack
• Deadlocks
– Code halts, as threads may wait indefinitely on locks
– Cause is programmer error or poorly written code
ECE 297
Pros & cons of threads-based servers
• Probably the simplest option
–No zombies, no signal handling, no onerous data
structures
• “Easy” to share data structures between
threads
–Logging information, data files, cache, …
• Thread creation is more efficient than process
creation
• Enables concurrent processing of requests
from multiple clients
ECE 297
Pros & cons cont.’d
• Unintentional sharing can introduce subtle and
hard to reproduce race conditions
• malloc an argument (struct) for each thread and
pass pointer to variable to thread and free after use
• Keep global variables to a minimum
• If a thread references a global variable
• protect it with a mutex or
• think carefully about whether unprotected
variable is safe
–e.g., one writer thread vs. multiple readers is
OK.
ECE 297