CS 2200 Presentation 19 Threads Questions? Our Road Map Processor Memory Hierarchy I/O Subsystem Parallel Systems Networking Recall • Process – Program Counter – Registers – Stack – Code (Text) Memory – Data – Page Table – etc. • Processes must be protected from one another. Recall • Context Switching – Requires considerable work • What about a single users application? – Is there a way to make it more efficient – In effect, allow the user to have multiple processes executing in the same space? – Yes, solution: Threads or Multithreading What is Multithreading? • Technique allowing program to do multiple tasks – Example: Java GUI's Java Event Handling Our event handling code import java.awt.*; public class HelloGUI { public static void main (String[ ] arg) { System.out.println (“About to make GUI”); Frame f = new Frame (“Hello GUIs”); f.setSize( 200, 200 ); f.show(); System.out.println (“Finished making GUI”); }// main }// class HelloGUI callback The code trapping this event appears in the graphics thread What is Multithreading? • Technique allowing program to do multiple tasks – Example: Java GUI's • Is it a new technique? – has existed since the 70’s (concurrent Pascal, Ada tasks, etc.) • Why now? – “Time has come for this technology” – Emergence of SMP’s in particular SMP? • What is an SMP? – Multiple CPUs in a single box sharing all the resources such as memory and I/O • Is a dual-processor SMP more cost effective than two uniprocessor boxes? – Yes, (roughly 20% more for a dual processor SMP compared to a uniprocessor). – Modest speedup for a program on a dualprocessor SMP over a uniprocessor will make it worthwhile. What is a Thread? • Basic unit of CPU utilization • A lightweight process (LWP) • Consists of – Program Counter – Register Set – Stack Space • Shares with peer threads – Code – Data – OS Resources • Open files • Signals Threads • Can be context switched more easily – Registers and PC – Not memory management • Can run on different processors concurrently in an SMP • Share CPU in a uniprocessor • May (Will) require concurrency control programming like mutex locks. Threads in a Uniprocessor? Process active • Allows concurrency between I/O and user processing even in a uniprocessor box Threads and OS Traditional OS • DOS – Memory layout user kernel Program Data DOS Code Data – Protection between user and kernel? Threads and OS • Unix - memory layout User Kernel P1 P2 Process code and data Process code and data PCB PCB Kernel code and data – Protection between user and kernel? Threads and OS • Programs in a traditional OS are single threaded – One PC per program (process), one stack, one set of CPU registers – If a process blocks (say disk I/O, network communication, etc.) then no progress for the program as a whole MultiThreaded Operating Systems • How widespread is support for threads in OS? – Digital Unix, Sun Solaris, Win9x, Win NT, Win2k, Linux, Free BSD, etc • Process vs. Thread? – In a single threaded program, the state of the executing program is contained in a process – In a multithreaded program, the state of the executing program is contained in several ‘concurrent’ threads What is this? 1. 2. 3. 4. Crack in slide Hair stuck on lens Symbol for thread San Andreas Fault Process Vs. Thread P1 t1 t2 t3 P2 t1 user kernel code data code data PCB PCB Kernel code and data – Computational state (PC, regs, …) for each thread – How different from process state? Memory Layout Static Static Heap Heap stack1 stk1 stk2 stk3 Code Code ST program MT program ... • Multithreaded program has a per-thread stack • Heap, static, and code are common to all threads Threads • Threaded code different from nonthreaded? – Protection for data shared among threads • Mutex – Synchronization among threads • Way for threads to talk (signal) to one another – Thread-safe libraries NOTES strtok is unsafe for multi-thread applications. strtok_r is MT-Safe and should be used instead. Typical Operation • Main programs creates (or spawns) threads • Threads may – perform one task and die – last for duration of program • Threads must – be able to synchronize activity – communicate with one another Programming Support for Threads • Creation – pthread_create(top-level procedure, args) • Termination – return to top-level procedure – explicit cancel • Rendezvous – creator can wait for children • pthread_join(child_tid) • Synchronization – pthread_mutex_... – condition variables (pthread_cond_...) Programming with Threads • Synchronization – For coordination of the threads • Communication – For inter-thread sharing of data – Threads can be in different processors – How to achieve sharing in SMP? • Software: accomplished by keeping all threads in the same address space by the OS • Hardware: accomplished by hardware shared memory and coherent caches Synchronization Primitives • lock and unlock – mutual exclusion among threads – busy-waiting vs. blocking – pthread_mutex_trylock: no blocking (rare) – pthread_mutex_lock: blocking – pthread_mutex_unlock • condition variables – pthread_cond_wait: block for a signal – pthread_cond_signal: signal one waiting thread – pthread_cond_broadcast: signal all waiting threads Initially mutex is unlocked resource_state is FREE Example lock(mutex); while (resource_state == BUSY) //spin; resource_state = BUSY; unlock(mutex); use resource; Will this work? 1 - Yes 2 - No 3 - Maybe lock(mutex); resource_state = FREE; unlock(mutex); Example lock(mutex); while (resource_state == BUSY) //spin; resource_state = BUSY; unlock(mutex); Thread 1 use resource; lock(mutex); resource_state = FREE; unlock(mutex); Example Thread 2 Thread 1 lock(mutex); while (resource_state == BUSY) //spin; resource_state = BUSY; unlock(mutex); use resource; lock(mutex); resource_state = FREE; unlock(mutex); Example Thread 2 lock(mutex); while (resource_state == BUSY) //spin; resource_state = BUSY; unlock(mutex); use resource; Thread 1 lock(mutex); resource_state = FREE; unlock(mutex); Example with cond-var lock(mutex) while(resource_state == BUSY) wait(cond_var); /* implicitly give up mutex */ /* implicitly re-acquire mutex */ resource_state = BUSY unlock(mutex) /* use resource */ lock(mutex) resource_state = FREE; unlock(mutex) signal(cond_var); Example with cond-var lock(mutex) while(resource_state == BUSY) wait(cond_var); /* implicitly give up mutex */ /* implicitly re-acquire mutex */ T1 resource_state = BUSY unlock(mutex) /* use resource */ lock(mutex) resource_state = FREE; unlock(mutex) signal(cond_var); Example with cond-var lock(mutex) while(resource_state == BUSY) T2 wait(cond_var); /* implicitly give up mutex */ /* implicitly re-acquire mutex */ T1 resource_state = BUSY unlock(mutex) /* use resource */ lock(mutex) resource_state = FREE; unlock(mutex) signal(cond_var); pthreads • Mutex – Must create mutex variables pthread_mutex_t padlock; – Must initialize mutex variable pthread_mutex_init(&padlock, NULL); • Condition Variable (used for signaling) – Must create condition variables pthread_cond_t non_full; – Must initialize condition variables pthread_cond_init(&non_full, NULL); full empty buffer ... Classic CS Problem Producer Consumer • Producer If (! full) Add item to buffer empty = FALSE if(buffer_is_full) full = TRUE • Consumer If (! empty) Remove item from buffer full = FALSE if(buffer_is_empty) empty = TRUE Example Producer Threads Program while(forever){ // produce item pthread_mutex_lock(padlock); while (full) pthread_cond_wait(non_full, padlock); // add item to buffer; buffercount++; if (buffercount == BUFFERSIZE) full = TRUE; empty = FALSE; pthread_mutex_unlock(padlock); pthread_cond_signal(non_empty); } Example Consumer Threads Program while(forever) { pthread_mutex_lock(padlock); while (empty) pthread_cond_wait (non_empty, padlock); // remove item from buffer; buffercount--; full = false; if (buffercount == 0) empty = true; pthread_mutex_unlock(padlock); pthread_cond_signal(non_full); // consume_item; } // Producer while(forever){ // produce item pthread_mutex_lock(padlock); while (full) pthread_cond_wait(non_full, padlock); // add item to buffer; buffercount++; if (buffercount == BUFFERSIZE) full = TRUE; empty = FALSE; pthread_mutex_unlock(padlock); pthread_cond_signal(non_empty); } // Consumer while(forever) { pthread_mutex_lock(padlock); while (empty) pthread_cond_wait (non_empty, padlock); // remove item from buffer; buffercount--; full = false; if (buffercount == 0) empty = true; pthread_mutex_unlock(padlock); pthread_cond_signal(non_full); // consume_item; } Using Threads Servers • dispatcher model dispatcher Workers Mailbox • Team model Mailbox • Pipelined model Mailbox Threads Implementation • User level threads – OS independent – Scheduler is part of the runtime system – Thread switch is cheap (save PC, SP, regs) – Scheduling customizable, i.e., more application control – Blocking call by thread blocks process • Solution to blocking problem in user level threads – Non-blocking version of all system calls • Switching among user level threads – Yield voluntarily – How to make preemptive? • Timer interrupt from kernel to switch • Kernel Level – Expensive thread switch – Makes sense for blocking calls by threads – Kernel becomes complicated: process vs. threads scheduling – Thread packages become non-portable • Problems common to user and kernel level threads – Libraries – Solution is to have thread-safe wrappers to such library calls Questions? Solaris Threads • Three kinds – user, lwp, kernel • User: Any number can be created and attached to lwp’s • One to one mapping between lwp and kernel threads • Kernel threads known to the OS scheduler • If a kernel thread blocks, associated lwp, and user level threads block as well Solaris Terminology More Conventional Terminology Processes P1 P2 P3 Thread kernel thread (user-level view) (Inside the kernel) Questions? • • • • Does kernel see user threads? Are all threads from P3 on same CPU? Are all threads in kernel attached to lwp’s? If the left-most thread of P3 issues a blocking system call does P3 block? • Who schedules lwp’s? • Who schedules threads? Kernel Threads vs. User Threads • Advantages of kernel threads – Can be scheduled on multiple CPU’s – Can be preempted by CPU – Kernel scheduler knows their relative priorities • Advantages of user threads – (Unknown to kernel) – Extremely lightweight: No system call to needed to change threads. Questions? Things to know? 1. The reason threads are around? 2. Benefits of increased concurrency? 3. Why do we need software controlled "locks" (mutexes) of shared data? 4. How can we avoid potential deadlocks/race conditions. 5. What is meant by producer/consumer thread synchronization/communication using pthreads? 6. Why use a "while" loop around a pthread_cond_wait() call? Things to know? 7. Why should we minimize lock scope (minimize the extent of code within a lock/unlock block)? 8. Do you have any control over thread scheduling? Questions?