CPU Scheduling
CPU Scheduler
 Performance metrics for CPU scheduling
 Methods for analyzing schedulers
 CPU scheduling algorithms
 Case study:


CPU scheduling in Solaris, Window XP, and Linux.
CPU Scheduler

A CPU scheduler, running in the dispatcher, is
responsible for selecting of the next running
process.


Based on a particular strategy
When does CPU scheduling happen?

Four cases:




A process switches from the running state to waiting state
(e.g. I/O request)
A process switches from the running state to the ready state.
A process switches from waiting state to ready state
(completion of an I/O operation)
A process terminates
Scheduling queues

CPU schedulers use various queues in the
scheduling process:

Job queue: consists of all processes



All jobs (processes), once submitted, are in the job queue.
Some processes cannot be executed (e.g. not in memory).
Ready queue



All processes that are ready and waiting for execution are in
the ready queue.
Usually, a long-term scheduler/job scheduler selects
processes from the job queue to the ready queue.
CPU scheduler/short-term scheduler selects a process from
the ready queue for execution.

Simple systems may not have a long-term job scheduler
Scheduling queues

Device queue


When a process is blocked in an I/O operation, it
is usually put in a device queue (waiting for the
device).
When the I/O operation is completed, the process
is moved from the device queue to the ready
queue.
An example scheduling queue
structure

Figure 3.6
Queueing diagram representation of
process scheduling

Figure 3.7
Performance metrics for CPU
scheduling





CPU utilization: percentage of the time that CPU
is busy.
Throughput: the number of processes completed
per unit time
Turnaround time: the interval from the time of
submission of a process to the time of completion.
Wait time: the sum of the periods spent waiting in
the ready queue
Response time: the time of submission to the
time the first response is produced
Goal of CPU scheduling
Other performance metrics:

fairness: It is important, but harder to define
quantitatively.
Goal:



Maximize CPU utilization, Throughput, and
fairness.
Minimize turnaround time, waiting time, and
response time.
Which metric is used more?
CPU utilization


Trivial in a single CPU system
Fairness


Tricky, even definition is non-trivial
Throughput, turnaround time, wait time, and
response time



Can usually be computed for a given scenario.
They may not be more important than fairness,
but they are more tractable than fairness.
Methods for evaluating CPU
scheduling algorithms
Simulation:





Get the workload from a system
Simulate the scheduling algorithm
Compute the performance metrics based on simulation
results
This is practically the best evaluation method.
Queueing models:



Analytically model the queue behavior (under some
assumptions).
A lot of math, but may not be very accurate because of
unrealistic assumptions.
Methods for evaluating CPU
scheduling algorithms
Deterministic Modeling

Take a predetermined workload
Run the scheduling algorithm manually
Find out the value of the performance metric that
you care about.



Not the best for practical uses, but can give a lot
of insides about the scheduling algorithm.


Help understand the scheduling algorithm, as well as it
strengths and weaknesses
Deterministic modeling example:


Suppose we have processes A, B, and C,
submitted at time 0
We want to know the response time, waiting
time, and turnaround time of process A
turnaround time
wait time
response time = 0
+
+
A B C A B C A C A C
Gantt chart: visualize how processes execute.
Time
Deterministic modeling example


Suppose we have processes A, B, and C,
submitted at time 0
We want to know the response time, waiting
time, and turnaround time of process B
turnaround time
wait time
response time
+
A B C A B C A C A C
Time
Deterministic modeling example


Suppose we have processes A, B, and C,
submitted at time 0
We want to know the response time, waiting
time, and turnaround time of process C
turnaround time
wait time
response time
+
+
+
A B C A B C A C A C
Time
Preemptive versus nonpreemptive
scheduling

Many CPU scheduling algorithms have both
preemptive and nonpreemptive versions:



Preemptive: schedule a new process even when the
current process does not intend to give up the CPU
Non-preemptive: only schedule a new process when the
current one does not want CPU any more.
When do we perform non-preemptive scheduling?




A process switches from the running state to waiting state
(e.g. I/O request)
A process switches from the running state to the ready state.
A process switches from waiting state to ready state
(completion of an I/O operation)
A process terminates
Scheduling Policies

FIFO (first in, first out)
Round robin
SJF (shortest job first)
Priority Scheduling
Multilevel feedback queues
Lottery scheduling

This is obviously an incomplete list





FIFO

FIFO: assigns the CPU based on the order
of requests
Nonpreemptive: A process keeps running on a
CPU until it is blocked or terminated
 Also known as FCFS (first come, first serve)
+ Simple
- Short jobs can get stuck behind long jobs

-
Turnaround time is not ideal (get an example from the
class)
Round Robin

Round Robin (RR) periodically releases the
CPU from long-running jobs



Based on timer interrupts so short jobs can get a
fair share of CPU time
Preemptive: a process can be forced to leave its
running state and replaced by another running
process
Time slice: interval between timer interrupts
More on Round Robin

If time slice is too long


Scheduling degrades to FIFO
If time slice is too short


Throughput suffers
Context switching cost dominates
FIFO vs. Round Robin

With zero-cost context switch, is RR always
better than FIFO?
FIFO vs. Round Robin

Suppose we have three jobs of equal length
turnaround time of C
turnaround time of B
turnaround time of A
A B C A B C A B C
Time
Round Robin
turnaround time of C
turnaround time of B
turnaround time of A
A
B
C
FIFO
Time
FIFO vs. Round Robin

Round Robin
+ Shorter response time
+ Fair sharing of CPU
- Not all jobs are preemptable
- Not good for jobs of the same length
-
More precisely, not good in terms of the turnaround
time.
Shortest Job First (SJF)

SJF runs whatever job puts the least demand
on the CPU, also known as STCF (shortest
time to completion first)
+ Provably optimal in terms of turn-around time
(anyone can give an informal proof?).
+ Great for short jobs
+ Small degradation for long jobs

Real life example: supermarket express
checkouts
SJF Illustrated
turnaround time of C
turnaround time of B
turnaround time of A
wait time of C
wait time of B
wait time of A = 0
response time of C
response time of B
response time of A = 0
A
B
C
Shortest Job First
Time
Shortest Remaining Time First (SRTF)

SRTF: a preemptive version of SJF



If a job arrives with a shorter time to completion,
SRTF preempts the CPU for the new job
Also known as SRTCF (shortest remaining time
to completion first)
Generally used as the base case for comparisons
SJF and SRTF vs. FIFO and Round Robin

If all jobs are the same length, SJF  FIFO


FIFO is the best you can do
If jobs have varying length

Short jobs do not get stuck behind long jobs under
SRTF
Drawbacks of Shortest Job First
- Starvation: constant arrivals of short jobs
can keep long ones from running
- There is no way to know the completion time
of jobs (most of the time)

Some solutions


Ask the user, who may not know any better
If a user cheats, the job is killed
Priority Scheduling (Multilevel Queues)





Priority scheduling: The process with the
highest priority runs first
C
Priority 0:
Priority 1: A
Priority 2:
B
Assume that low numbers represent high
priority
C
A
B
Priority Scheduling
Time
Priority Scheduling
+ Generalization of SJF

With SJF, priority = 1/requested_CPU_time
- Starvation
Multilevel Feedback Queues

Multilevel feedback queues use multiple
queues with different priorities






Round robin at each priority level
Run highest priority jobs first
Once those finish, run next highest priority, etc
Jobs start in the highest priority queue
If time slice expires, drop the job by one level
If time slice does not expire, push the job up by
one level
Multilevel Feedback Queues
time = 0



Priority 0 (time slice = 1):
Priority 1 (time slice = 2):
Priority 2 (time slice = 4):
A
B
Time
C
Multilevel Feedback Queues
time = 1



Priority 0 (time slice = 1):
Priority 1 (time slice = 2):
Priority 2 (time slice = 4):
A
B
C
A
Time
Multilevel Feedback Queues
time = 2



Priority 0 (time slice = 1):
Priority 1 (time slice = 2):
Priority 2 (time slice = 4):
A B
C
A
B
Time
Multilevel Feedback Queues
time = 3



Priority 0 (time slice = 1):
Priority 1 (time slice = 2):
Priority 2 (time slice = 4):
A B C
A
B
C
Time
Multilevel Feedback Queues
time = 3



Priority 0 (time slice = 1):
Priority 1 (time slice = 2):
Priority 2 (time slice = 4):
A
B
C
suppose process A is blocked on an I/O
A B C
Time
Multilevel Feedback Queues
time = 3



Priority 0 (time slice = 1):
Priority 1 (time slice = 2):
Priority 2 (time slice = 4):
A
B
C
suppose process A is blocked on an I/O
A B C
Time
Multilevel Feedback Queues
time = 5



Priority 0 (time slice = 1):
Priority 1 (time slice = 2):
Priority 2 (time slice = 4):
A
C
suppose process A is returned from an I/O
A B C
B
Time
Multilevel Feedback Queues
time = 6



Priority 0 (time slice = 1):
Priority 1 (time slice = 2):
Priority 2 (time slice = 4):
A B C
B
A
C
Time
Multilevel Feedback Queues
time = 8



Priority 0 (time slice = 1):
Priority 1 (time slice = 2):
Priority 2 (time slice = 4):
A B C
B
A
C
C
Time
Multilevel Feedback Queues
time = 9



Priority 0 (time slice = 1):
Priority 1 (time slice = 2):
Priority 2 (time slice = 4):
A B C
B
A
C
C
Time
Multilevel Feedback Queues

Approximates SRTF




A CPU-bound job drops like a rock
I/O-bound jobs stay near the top
Still unfair for long running jobs
Counter-measure: Aging


Increase the priority of long running jobs if they are not
serviced for a period of time
Tricky to tune aging
Lottery Scheduling

Lottery scheduling is an adaptive
scheduling approach to address the fairness
problem



Each process owns some tickets
On each time slice, a ticket is randomly picked
On average, the allocated CPU time is
proportional to the number of tickets given to each
job
Lottery Scheduling


To approximate SJF, short jobs get more
tickets
To avoid starvation, each job gets at least
one ticket
Lottery Scheduling Example


short jobs: 10 tickets each
long jobs: 1 ticket each
# short jobs/#
long jobs
1/1
0/2
2/0
10/1
1/10
% of CPU for
each short job
91%
0%
50%
10%
50%
% of CPU for
each long job
9%
50%
0%
1%
5%
Case study:

CPU scheduling in Solaris:

Priority-based scheduling




Four classes: real time, system, time sharing, interactive
(in order of priority)
Different priorities and algorithm in different classes
Default class: time sharing
Policy in the time sharing class:


Multilevel feedback queue with variable time slices
See the dispatch table
Solaris dispatch table for interactive
and time-sharing threads

Good response
time for interactive
processes and
good throughput for
CPU-bound
processes
Windows XP scheduling

A priority-based, preemptive scheduling



Similar idea for user processes – Multilevel feedback queue



Highest priority thread will always run
Also have multiple classes and priorities within classes
Lower priority when quantum runs out
Increase priority after a wait event
Some twists to improve “user perceived” performance:


Boost priority and quantum for foreground process (the window that
is currently selected).
Boost priority more for a wait on keyboard I/O (as compared to disk
I/O)
Linux scheduling

A priority-based, preemptive with global
round-robin scheduling





Each process have a priority
Processes with a larger priority also have a larger time slices
Before the time slices is used up, processes are scheduled
based on priority.
After the time slice of a process is used up, the process must wait
until all ready processes to use up their time slice (or be blocked)
– a round-robin approach.
 No starvation problem.
For a user process, its priority may + or – 5 depending whether
the process is I/O- bound or CPU-bound.
 Giving I/O bound process higher priority.
Summary for the case study

Basic idea for schedule user processes is the same
for all systems:



The scheduling in Solaris / Linux is more concerned
about fairness.


Lower priority for CPU bound process
Increase priority for I/O bound process
More popular as the OSes for servers.
The scheduling in Window XP is more concerned
about user perceived performance.

More popular as the OS for personal computers.