Scheduler activations
Landon Cox
March 23, 2016
What is a process?
• Informal
• A program in execution
• Running code + things it can read/write
• Process ≠ program
• Formal
• ≥ 1 threads in their own address space
Parts of a process
• Thread
• Sequence of executing instructions
• Active: does things
• Address space
• Data the process uses as it runs
• Passive: acted upon by threads
Threads that aren’t running
• What is a non-running thread?
• thread=“sequence of executing instructions”
• non-running thread=“paused execution”
• Must save thread’s private state
• To re-run, re-load private state
• Want thread to start where it left off
Private vs global thread state
• What state is private to each thread?
• PC (where actor is in his/her script)
• Stack, SP (actor’s mindset)
• What state is shared?
• Global vars, heap (props)
• Code (same script)
Thread control block (TCB)
• What needs to access threads’ private data?
• The CPU
• This info is stored in the PC, SP, other registers
• The OS needs pointers to non-running threads’
data
• Thread control block (TCB)
• Container for non-running threads’ private data
• Values of PC, code, SP, stack, registers
Thread control block
Address Space
TCB1
TCB2
TCB3
PC
Ready
queue
SP
registers
PC
SP
registers
PC
SP
registers
Code
Code
Code
Stack
Stack
Stack
Thread 1
running
PC
SP
registers
CPU
Thread control block
Address Space
Ready
queue
TCB2
TCB3
PC
SP
registers
PC
SP
registers
Stack
Stack
Code
Stack
Thread 1
running
PC
SP
registers
CPU
Switching threads
• What needs to happen to switch threads?
1. Thread returns control to scheduler
• For example, via timer interrupt
2. Scheduler chooses next thread to run
3. Scheduler saves state of current thread
• To its thread control block
4. Scheduler loads context of next thread
• From its thread control block
5. Jump to PC in next thread’s TCB
Kernel vs user threads
• Kernel threads
• Scheduler and queues reside in the kernel
• User threads
• Scheduler and queues reside in user space
32-bit address space
4GB
Kernel data
(same for all
processes)
3GB
(0xc0000000)
User data
(different for
every process)
0GB
Virtual
memory
Switching threads
4GB
3GB
Where is the
interrupt
handler code?
In the kernel
0GB
Virtual
memory
Switching threads
4GB
SP
PC
Interrupthandler
code
3GB
On which
stack is the
IRQ handled?
A kernel stack
0GB
Virtual
memory
Switching threads
4GB
SP
PC
Interrupthandler
code
3GB
Where are the
threads’
stacks?
In user space
0GB
Virtual
memory
Switching threads
4GB
SP
PC
Interrupthandler
code
3GB
So far,
everything is
the same for
user-level and
kernel
threads
0GB
Virtual
memory
Kernel threads
Scheduler
code
For kernel
threads,
the thread
scheduler and
TCB queues
are in the
kernel
Handler
Interruptcode
handler
code
TCB
TCB
TCB
Kernel threads
Scheduler
code
Where is the
lock/cv code?
Handler
Interruptcode
handler
code
TCB
TCB
TCB
Kernel threads
Where is the
lock/cv code?
Handler
Interruptcode
handler
code
Scheduler
code Synchron.
code
TCB
TCB
TCB
Also, in the
kernel
User-level threads
For user-level
threads,
the thread
scheduler and
queues are in
user space
Handler
Interruptcode
handler
code
TCB
TCB
TCB
Scheduler
code
User-level threads
Where is the
lock/cv code?
Handler
Interruptcode
handler
code
Also, in the
user space
TCB
TCB
TCB
Scheduler
code
Synchron.
code
Switching to a new user-level thread
User-level threads
SP
Handler
Interruptcode
handler
code
P
C
TCB
Thread
running user
code
TCB
Program
code
Scheduler
code
Synchron.
code
User-level threads
SP
PC
Handler
Interruptcode
handler
code
TCB
Timer
interrupt!
TCB
Program
code
Scheduler
code
Synchron.
code
User-level threads
SP
PC
Handler
Interruptcode
handler
code
Does the
kernel know
where user
left off?
Yes, CPU
stores this on
kernel stack
TCB
TCB
Program
code
Scheduler
code
Synchron.
code
User-level threads
SP
PC
Handler
Interruptcode
handler
code
What does
kernel do
next?
Depends on
meaning of
the timer …
TCB
TCB
Program
code
Scheduler
code
Synchron.
code
User-level threads
SP
PC
Handler
Interruptcode
handler
code
Assuming it’s
a signal for
the process?
Kernel
delivers
signal to
process
TCB
TCB
Program
code
Scheduler
code
Synchron.
code
User-level threads
SP
PC
To deliver a
signal to
user code:
make it look
like a forced
function call
to signal
handler.
Handler
Interruptcode
handler
code
TCB
TCB
Program
code
Scheduler
code
Synchron.
code
User-level threads
SP
PC
Handler
Interruptcode
handler
code
Where is the
process’s
timer signal
handler?
In scheduler
code
(e.g., yield)
TCB
TCB
Program
code
Scheduler
code
Synchron.
code
User-level threads
SP
PC
On what stack
should the
scheduler
code run?
Stack of
interrupted
thread
Handler
Interruptcode
handler
code
TCB
TCB
yield
Program
code
Scheduler
code
Synchron.
code
User-level threads
PC
SP
On what stack
should the
scheduler
code run?
Stack of
interrupted
thread
Handler
Interruptcode
handler
code
TCB
TCB
yield
Program
code
Scheduler
code
Synchron.
code
User-level threads
PC
SP
Could the
kernel transfer
control to any
other thread?
No, it’s only
aware of
interrupted
one
Handler
Interruptcode
handler
code
TCB
TCB
yield
Program
code
Scheduler
code
Synchron.
code
Switching to a new kernel thread
Kernel threads
SP
Handler
Interruptcode
handler
code
P
C
Scheduler
code Synchron.
code
TCB
TCB
TCB
Thread
running user
code
Program
code
Kernel threads
SP
PC
Handler
Interruptcode
handler
code
Scheduler
code Synchron.
code
TCB
TCB
TCB
Timer
interrupt!
Program
code
Kernel threads
SP
PC
Handler
Interruptcode
handler
code
Scheduler
code Synchron.
code
TCB
TCB
TCB
Which thread
could run
next?
Any thread.
Kernel aware
of all of them
Program
code
Advantages of user-level
threads
• Synchronization primitives are just a function call
• Why are kernel threads more expensive?
• Accessing thread management must cross kernel boundary
• CPU protection checks, argument checks, handler dispatch, etc.
• Switching back to thread also crosses kernel boundary
• Easy to tune scheduling policy to application
• Why is this harder for kernel threads?
• All processes share the same scheduler
• Per-process policies require wider, more complex API
Advantage of kernel threads
SP
PC
Handler
Interruptcode
handler
code
Which thread
could run
next?
Only the one
that was
interrupted
TCB
TCB
Program
code
Scheduler
code
Synchron.
code
Advantage of kernel threads
SP
PC
Handler
Interruptcode
handler
code
Why might we
be unable to run
the interrupted
thread?
Thread could
have made a
blocking
system call
TCB
TCB
Program
code
Scheduler
code
Synchron.
code
Advantage of kernel threads
SP
PC
Handler
Interruptcode
handler
code
TCB
Why is this
wasteful?
There are other
threads that
could run! Kernel
doesn’t know
they exist
TCB
Program
code
Scheduler
code
Synchron.
code
Scheduler activations
• Problem with user-level threads
• Kernel may not find a thread on which to run scheduler
• Happens when thread blocks waiting for I/O
• This is exactly when you want to run other threads!
• Main ideas
•
•
•
•
•
Kernel assigns user process N virtual processors
A virtual processor corresponds to a kernel thread
Each kernel thread contains a scheduler activation (initial upcall)
Kernel invokes scheduler activation when running a thread
Scheduler activations (user-level code) schedule user-level
threads
• Kernel tells user-level code about blocked and pre-empted
threads
Blocking I/O example
T1: two virtual
processors, run
user scheduler
on each stack
Blocking I/O example
T2: thread 1
blocks in kernel,
create new
kernel thread
notifying user
code thread 1 is
blocked. User
threads can run
in context of new
kernel thread.
Blocking I/O example
T3: I/O
completes, and
pre-empts
thread 2. kernel
creates new
kernel thread to
indicate that
thread 2 was
pre-empted and
thread 1 can
resume.
Blocking I/O example
T4: with the
remaining kernel
threads, userlevel code
decides whether
to run thread 1
or 2, or some
other threads.
Course administration
• Research projects
• Ideally, work on an ongoing project from your own research
• Otherwise, suggestions will be posted tonight
• Timeline for research projects
• Proposal due Friday, April 1
• Status report due Wednesday, April 13
• Final report due Wednesday, April 20 (plus demo/presentation)
• Exams
• Still grading them
• Will return exams on Friday