Kernel Synchronization
David Ferry, Chris Gill
CSE 522S - Advanced Operating Systems
Washington University in St. Louis
St. Louis, MO 63143
1
Linux is a Preemptive Kernel
The Linux kernel executes concurrently with
itself:
• Multiple processors (CONFIG_SMP)
• Interrupts
• Kernel Threads
Many data structures can be accessed
concurrently from different contexts, which
may cause concurrency bugs.
CSE 522S – Advanced Operating Systems
2
Single Variable Race Condition
Suppose two concurrent processes are accessing a
variable X with initial value of 1, what is the final
value of X?:
Thread 1:
load X
Thread 2:
load X
X = X + 1
X = X + 1
store X
store X
CSE 522S – Advanced Operating Systems
3
Data Structure Race Condition
Suppose two threads are accessing a list:
Node A Next
Node B Next
Deleter:
Node* = A->Next
A->Next = B->Next
Kfree(Node);
Node C Next
Reader:
Node* = A->Next
Value = Node->val
CSE 522S – Advanced Operating Systems
4
Shared Data Synchronization
Accessing shared data requires appropriate
synchronization
Protect data, not code!
A region of code that accesses shared data
is called a critical section
CSE 522S – Advanced Operating Systems
5
Basic Kernel Synchronization
The Linux kernel supports most of the same
synchronization primitives as userspace.
1.
2.
3.
4.
5.
Atomic variables
Spin locks (non-sleepable locks)
Mutexes (sleepable locks)
Reader-writer locks
Semaphores
CSE 522S – Advanced Operating Systems
6
Locking Example
Node A Next
Node B Next
Deleter:
lock( list_lock );
Node* = A->Next;
A->Next = B->Next;
Kfree(Node);
Unlock(list_lock);
Node C Next
Reader:
lock( list_lock );
Node* = A->Next;
Value = Node->val;
unlock( list_lock );
CSE 522S – Advanced Operating Systems
7
When to Use Different Types of Locks
Spin locks - very short lock durations and/or
very low overhead requirements
Mutexes - Long lock duration and/or process
needs to sleep while holding lock
Atomics - When shared data fits inside a
single word of memory
Other locks - special cases
CSE 522S – Advanced Operating Systems
8
Sleeping With Locks
Sleeping with a lock is generally a bad idea
unless absolutely necessary
• Delays other processes
• Deadlock risk: is it guaranteed to
eventually wake up and release the lock?
A process cannot sleep holding a spinlock.
• Restructure code to back out of lock
• Use a mutex instead
CSE 522S – Advanced Operating Systems
9
Deadlock
Deadlock occurs when a process can never
progress while waiting for a lock, e.g.:
Thread 1:
lock( A );
lock( B );
//critical section
unlock( A );
unlock( B );
Lock A
Lock B
Thread 2:
lock( B );
lock( A );
//critical section
unlock( B );
unlock( A );
CSE 522S – Advanced Operating Systems
10
Deadlock
Deadlock occurs when a process can never
progress while waiting for a lock, e.g.:
Thread 1:
lock( A );
lock( B );
//critical section
unlock( A );
unlock( B );
Lock A
Lock B
Thread 2:
lock( B );
lock( A );
//critical section
unlock( B );
unlock( A );
CSE 522S – Advanced Operating Systems
11
Deadlock
Deadlock occurs when a process can never
progress while waiting for a lock, e.g.:
Thread 1:
lock( A );
lock( B );
//critical section
unlock( A );
unlock( B );
Lock A
Lock B
Thread 2:
lock( B );
lock( A );
//critical section
unlock( B );
unlock( A );
CSE 522S – Advanced Operating Systems
12
Deadlock
Deadlock occurs when a process can never
progress while waiting for a lock, e.g.:
Thread 1:
lock( A );
lock( B );
//critical section
unlock( A );
unlock( B );
Lock A
Deadlock!
Lock B
Thread 2:
lock( B );
lock( A );
//critical section
unlock( B );
unlock( A );
CSE 522S – Advanced Operating Systems
13
Lock Ordering
Always acquire and release locks in the
same order! (Does not solve all deadlocks,
but it’s a good place to start.)
Thread 1:
lock( A );
lock( B );
//critical section
unlock( B );
unlock( A );
Lock A
Lock B
Thread 2:
lock( A );
lock( B );
//critical section
unlock( B );
unlock( A );
CSE 522S – Advanced Operating Systems
14
Lock Ordering
Always acquire and release locks in the
same order! (Does not solve all deadlocks,
but it’s a good place to start.)
Thread 1:
lock( A );
lock( B );
//critical section
unlock( B );
unlock( A );
Lock A
Lock B
Thread 2:
lock( A );
lock( B );
//critical section
unlock( B );
unlock( A );
CSE 522S – Advanced Operating Systems
15
Lock Ordering
Always acquire and release locks in the
same order! (Does not solve all deadlocks,
but it’s a good place to start.)
Thread 1:
lock( A );
lock( B );
//critical section
unlock( B );
unlock( A );
Lock A
Lock B
Thread 2:
lock( A );
lock( B );
//critical section
unlock( B );
unlock( A );
CSE 522S – Advanced Operating Systems
16
Lock Ordering
Always acquire and release locks in the
same order! (Does not solve all deadlocks,
but it’s a good place to start.)
Thread 1:
lock( A );
lock( B );
//critical section
unlock( B );
unlock( A );
Lock A
Lock B
Thread 2:
lock( A );
lock( B );
//critical section
unlock( B );
unlock( A );
CSE 522S – Advanced Operating Systems
17
Lock Ordering
Always acquire and release locks in the
same order! (Does not solve all deadlocks,
but it’s a good place to start.)
Thread 1:
lock( A );
lock( B );
//critical section
unlock( B );
unlock( A );
Lock A
Lock B
Thread 2:
lock( A );
lock( B );
//critical section
unlock( B );
unlock( A );
CSE 522S – Advanced Operating Systems
18
The BKL is No Longer Used
The Big Kernel Lock (BKL) was the first lock
introduced to the kernel
• Locks the entire kernel
• Provides correctness, but very slow
• New uses are prohibited
• Gradually replaced with finer-grained
locking schemes
CSE 522S – Advanced Operating Systems
19