Approaches to scheduling
Landon Cox
March 25, 2016
Switching threads
• What needs to happen to switch threads?
1. Thread returns control to scheduler
• For example, via timer interrupt
2. Scheduler chooses next thread to run
3. Scheduler saves state of current thread
• To its thread control block
4. Scheduler loads context of next thread
• From its thread control block
5. Jump to PC in next thread’s TCB
Scheduling goals
1. Minimize average response time
• Elapsed time to do a job (what users care about)
• Try to maximize idle time
• Incoming jobs can then finish as fast as possible
2. Maximize throughput
• Jobs per second (what admins care about)
• Try to keep parts as busy as possible
• Minimize wasted overhead (e.g., context
switches)
Scheduling goals
3. Fairness
• Share CPU among threads equitably
• Key question: what does “fair” mean?
Job 1
Job 2
Needs 100
seconds of CPU
time
Needs 100
seconds of CPU
time
What does “fair” mean?
• How can we schedule these jobs?
• Job 1 then Job 2
• 1’s response time = 100, 2’s response time = 200
• Average response time = 150
• Alternate between 1 and 2
Job 1time = 2’s response
Job 2 time = 200
• 1’s response
• Average response time = 200
Needs 100
seconds of CPU
time
Needs 100
seconds of CPU
time
Fairness
• Fairness may come at a cost
• (in terms of average response time)
• Finding a balance can be hard
• What if you have 1 big job, 1 small job?
• Response time proportional to size?
• Trade-offs come into play
FCFS (first-come-first-served)
• FIFO ordering between jobs
• No pre-emption (run until done)
• Run thread until it blocks or yields
• No timer interrupts
• Essentially what you’ll do in 1t
• Adding interrupt_disable for safety
• (for auto-grader pre-emption)
FCFS example
• Job A (100 seconds), Job B (1 second)
A arrives and starts
(t=0)
B arrives
(t=0+)
A done
(t=100)
B done
(t=101)
• Average response time = 100.5 seconds
FCFS
• Pros
• Simplicity
• Cons?
• Short jobs stuck behind long ones
• Non-interactive
• (for long CPU job, user can’t type input)
Round-robin
• Goal
• Improve average response time for short jobs
• Solution to FCFS’s problem
• Add pre-emption!
• Pre-empt CPU from long-running jobs
• (timer interrupts)
• This is what most OSes do
Round-robin
• In what way is round robin fairer than FCFS?
• Makes response times  job length
• In what way is FCFS fairer than round robin?
• First to start will have better response time
• Like children playing with a toy
• Should they take turns?
• “She’s been playing with it for a long time.”
• Should first get toy until she’s bored?
• “I had it first.”
Round-robin example
• Job A (100 seconds), Job B (1 second)
A arrives
and
starts
A swapped
A runs
out
(t=0)
(t=1)
(t=2)
A done
(t=101)
B arrives
B done
B starts
(t=0+)
(t=2)
(t=1)
• Average response time = 51.5 seconds
Does round robin always provide lower response times than FCFS?
Round-robin example 2
• Job A (100 seconds), Job B (100
second)
A arrives
and
starts
A swapped
A runs
out
(t=0)
(t=1)
(t=2)
BBarrives
Bswapped
starts out
(t=0+)
(t=2)
(t=1)
A done
(t=199)
B done
(t=200)
• Average
response
timecounting?
= 199.5
seconds
Any
hidden costs
that we aren’t
Context
switches
Round-robin example 2
• Job A (100 seconds), Job B (100
second)
A done
(t=199)
B done
(t=200)
•What
Average
response
time = 199.5
seconds
would FCFS’s
avg response
time be?
150 seconds
Round-robin example 2
• Job A (100 seconds), Job B (100
second)
A done
(t=199)
B done
(t=200)
•Which
Average
response
time…
= 199.5 seconds
one is fairer?
It depends
Round-robin
• Pros
• Good for interactive computing
• Better than FCFS for mix of job lengths
• Cons
• Less responsive for uniform job lengths
• Hidden cost: context switch overhead
• How should we choose the time-slice?
• Typically a compromise, e.g., tens of ms
• Most threads give up CPU voluntarily much faster
STCF and STCF-P
• Idea
• Get the shortest jobs out of the way first
• Improves short job times significantly
• Little impact on big ones
1. Shortest-Time-to-Completion-First
• Run whatever has the least work left before it finishes
2. Shortest-Time-to-Completion-First (Pre-emp)
• If new job arrives with less work than current job has left
• Pre-empt and run the new job
STCF is optimal
• (among non-pre-emptive policies)
• Intuition: anyone remember bubble
sort?
Job B
Start
Job A
Finish
What happened to total time to complete A and B?
STCF is optimal
• (among non-pre-emptive policies)
• Intuition: anyone remember bubble
sort?
Job A
Start
Job B
Finish
What happened to the time to complete A?
What happened to the time to complete B?
STCF is optimal
• (among non-pre-emptive policies)
• Intuition: anyone remember bubble
sort?
Job A
Start
Job B
Finish
What happened to the average completion time?
STCF-P is also optimal
• (among pre-emptive policies)
• Job A (100 seconds), Job B (1 second)
A arrives and
starts
A runs
(t=0)(t=1)
A done
(t=101)
done
B arrives B
and
pre(t=1)
empts
(t=0+)response time = 51 seconds
• Average
I/O
• What if a program does I/O too?
while (1) {
do 1ms of CPU
do 10ms of I/O
}
• To scheduler, is this a long job or a short
job?
• Short
• Thread scheduler only cares about CPU time
STCF-P
• Pros
• Optimal average response time
• Cons?
• Can be unfair
• What happens to long jobs if short jobs keep coming?
• Legend from the olden days …
• IBM 7094 was turned off in 1973
• Found a “long-running” job from 1967 that hadn’t been
scheduled
• Requires knowledge of the future
Knowledge of the future
• You will see this a lot in CS.
• Examples?
• Cache replacement (next reference)
• Banker’s algorithm (max resources)
• How do you know how much time jobs
take?
• Ask the user (what if they lie?)
• Use the past as a predictor of the future
• Would this work for banker’s algorithm?
– No. Must know max resources for certain.
Grocery store scheduling
• How do grocery stores schedule?
• Kind of like FCFS
• Express lanes
• Make it kind of like STCF
• Allow short jobs to get through quickly
• STCF-P would probably be considered unfair
Final example
• Job A (CPU-bound)
• No blocking for I/O, runs for 1000 seconds
• Job B (CPU-bound)
• No blocking for I/O, runs for 1000 seconds
• Job C (I/O-bound)
while (1) {
do 1ms of CPU
do 10ms of I/O
}
Each job on its own
A: 100% CPU, 0% Disk
B: 100% CPU, 0% Disk
C: 1/11 CPU, 10/11 Disk
CPU
Disk
Time
Mixing jobs (FCFS)
A: 100% CPU, 0% Disk
B: 100% CPU, 0% Disk
C: 1/11 CPU, 10/11 Disk
CPU
Disk
How well would FCFS work?
Not well.
Mixing jobs (RR with 100ms)
A: 100% CPU, 0% Disk
B: 100% CPU, 0% Disk
C: 1/11 CPU, 10/11 Disk
CPU
Disk
How well would RR with 100 ms slice work?
Mixing jobs (RR with 1ms)
A: 100% CPU, 0% Disk
B: 100% CPU, 0% Disk
C: 1/11 CPU, 10/11 Disk
CPU
Disk
How well would RR with 1 ms slice work?
Good Disk utilization (~90%)
Good principle: start things that can be parallelized early
A lot of context switches though …
Mixing jobs (STCF-P)
A: 100% CPU, 0% Disk
C: 1/11 CPU, 10/11 Disk
B: 100% CPU, 0% Disk
B
?
CPU
Disk
How well would STCF-P work? (run C as soon as its I/O is done)
Good Disk utilization (~90%)
Many fewer context switches
Why not run B here? When will B run?
When A finishes
Real-time scheduling
• So far, we’ve focused on average-case
• Alternative scheduling goal
• Finish everything before its deadline
• Calls for worst-case analysis
• How do we meet all of our deadlines?
• Earliest-deadline-first (optimal)
• Used by students to complete all homework assignments
• Used by professors to juggle teaching and research
• Note: sometimes tasks get dropped (for both of
us)
Earliest-deadline-first (EDF)
• EDF
• Run the job with the earliest deadline
• If a new job comes in with earlier deadline
• Pre-empt the current job
• Start the next one
• This is optimal
• (assuming it is possible to meet all deadlines)
EDF example
• Job A
• Takes 15 seconds
• Due 20 seconds after entry
• Job B
• Takes 10 seconds
• Due 30 seconds after entry
• Job C
• Takes 5 seconds
• Due 10 seconds after entry
EDF example
A: takes 15, due in 20
B: takes 10, due in 30
C: takes 5, due in 10
A
B
C
0
5
20
30
35
40
45 50 55
Course administration
• Research projects
• Best to do your own
• Look at posted ideas, otherwise
• Mid-term exam
• Taking longer to grade than expected
Proportional-share scheduling
• We might not know much about our tasks
• Don’t know how long they’ll take to finish
• Don’t know if they’ll block on I/O
• Don’t know by when they need to finish
• We probably know which jobs are important
•
•
•
•
Could express the relative importance with priorities
Assign each process a value
High priority (-20) to low priority (19)
High priority tasks should run more often than low
Priority scheduling
Info bus manager (high priority)
Communicate with ground (medium priority)
(2)
block weather
on lock (low priority)
Sense
(1) collect data,
Mars Pathfinder bug
bus () {
acquire lock,
while (1) {
copy to bus
lock.acquire ()
dispatch local messages
lock.release ()
}
}
weather () {
while (1) {
collect sensor data
lock.acquire()
copy data to bus
lock.release()
}
}
comm () {
while (1) {
send/rcv remote messages
}
}
(4) bus is stalled,
watchdog () {
while (1) {
sleep (1s)
if (bus hasn’t run) {
panic and reboot!
}
}
}
(3) pre-empt
weather, further
stalling bus
freak out!
Priority inversion
• Exemplified by Pathfinder bug
•
•
•
•
High priority tasks stuck behind lower priorities
Issue was dependency between bus and weather
Bus was blocked waiting for weather
Weather should inherit bus’s priority
• Lottery scheduling
•
•
•
•
Allows programmers to express tasks’ ideal share of resources
Goal is proportional sharing of resource
Optimize for fair share rather than response time
Allows easy inheritance of shares
Lottery scheduling
• Tasks are assigned tickets
• Percent of total tickets represents share of resource
• Example:
• A has 75 tickets (0 through 74)
• B has 25 tickets (75 through 99)
• Goal: A gets 75% of CPU, B gets 25% of CPU
• To schedule A and B, hold a lottery for
decision
Lottery scheduling
• Choose random number between 0, 99
• Whoever holds the winning ticket runs
63 85 70 39 76 17 29 41 36 39 10 99 68 83 63 62 43 0 49 49
A B A A B A A A A A A B A B A A A A A A
A runs 16 times during the 20 time slices
B runs four times during the 20 time slice
This isn’t exactly what we
wanted, but over longer
periods, A should get CPU
75% of the time. Lottery
scheduling is most effective
over many scheduling
decision
Lottery scheduling
• Lotteries can be fast, require little state
Scheduler picks random number between 0 and 19 … 15!
Task A (10 tix)
Task B (2
tix)
Task C (5
tix)
Sum = 10
Sum < 15
Sum = 12
Sum < 15
Sum = 17
Sum >=
15
Task D (1
tix)
Task E (2
tix)
Lottery scheduling
• Tickets can be dynamically assigned
• Currencies
• Allows tasks to express importance of sub-tasks
• Lotteries always convert tickets to base currency
• Compensation
• Handles case when task doesn’t use entire slice
• Idea is to inflate tickets proportional to unused resources
• Transfers
• Allows tasks to hand off tickets to other tasks
• Very useful when dependencies arise
Currencies
Whole
system (200
tix)
User A
(100 tix)
User B
(100 tix)
Curr A
(1000 tix)
Curr B
(10 tix)
Task 1
(500
tix)
Task 2
(500
tix)
50 in base 50 in base
Task 1
(10
tix)
100 in base
Compensation
• Two tasks: A and B
• Both have 400 tickets in base currency
• Want A to run 50%, B to run 50% time
• Time slice is 100ms
• What happens if A completes after 100ms, B after 20ms?
• B will only consume 20% of its allotment
• A will consume the full 100% of its allotment
• Shares over time will not be 50-50
• How to solve this?
• Inflate B’s funds by 1/f where f is fraction used
• When B stops after 20ms, give it extra 1600 tix for next lottery
• B will have 2000 tix total for next scheduling decision
Transfers
What should
Info
bus manager (high priority)
happen to bus
Communicate
with ground (medium priority)
tix when it
Sense
weather (low priority)
blocks?
Mars Pathfinder bug
bus () {
while (1) {
lock.acquire ()
dispatch local messages
lock.release ()
}
}
weather () {
while (1) {
collect sensor data
lock.acquire()
copy data to bus
lock.release()
}
}
comm () {
while (1) {
send/rcv remote messages
}
}
watchdog () {
while (1) {
sleep (1s)
if (bus hasn’t run) {
panic and reboot!
}
}
}