Using Hierarchical
Scheduling to Support Soft
Real-Time Applications in
General-Purpose
Operating Systems
John Regehr
March 20, 2001
1
Outline
 Motivation and Approach
 Guarantees
 HLS Design
 Augmented Reservations
 Conclusions and Demo
2
Overview of Contributions


A general hierarchy of soft real-time
schedulers can provide guaranteed
scheduling behavior
The Hierarchical Loadable Scheduler
(HLS) architecture…
Can be efficiently implemented in a generalpurpose OS
 Increases application performance
compared to case where scheduler and apps
are mismatched


Augmented CPU reservations increase
predictability when the OS steals CPU
time from real-time applications
3
Motivation

People use general-purpose OSs
(GPOSs) for many kinds of tasks
e.g. Unix, Windows, MacOS variants
 Compatibility, commodity, convenience


Applications have diverse scheduling
requirements
Time-sharing
 Soft real-time
 Isolation
 Co-scheduling between processors or
machines

4
The Problem


One size does not fit all
Supporting evidence: companies sell
scheduler extensions
Ensim: ServerXchange
 Sun: Solaris Resource Manager
 Aurema: Active Resource Management
 TimeSys: Linux/RT

5
Motivating Data: Unfair CPU
Allocation Between Users
N1
1
1
1
1
1

%100 N2
50.000
1
19.900
4
6.100 16
1.560 64
0.313 256
%2
50.0
80.1
93.9
98.4
99.7
User 1 gets little CPU time when
User 2 creates many threads
6
Approach


Allow a hierarchy of CPU schedulers
to control processor allocation
Thesis Statement:



It is useful and feasible to extend a
GPOS with a general, heterogeneous
scheduling hierarchy.
A heterogeneous hierarchy employs
different schedulers
A general hierarchy does not impose
a fixed scheduling model
7
Example Hierarchy
FP
H
RES
L
J
Video
Player
FP: Fixed Priority
RES: CPU Reservation
J: Join
SFQ: Start-Time Fair
Queuing
SFQ
Word
Processor
Voice
Recognition
8
Time Scales

Long:
System is used in a particular way
 Duration: usually uptime of a system or
longer


Medium:
Applications start, end, and change
requirements
 Duration: seconds, minutes, or longer


Short:
Individual scheduling decisions made by
schedulers within the hierarchy
 Duration: milliseconds or 10s of ms

9
What Microsoft Should Do


Put HLS into consumer Windows!
By default


Support interactive, batch, and
multimedia applications for a single
user
However, also include
Library of useful schedulers and API
for composing them
 API for implementing new schedulers

10
Reminder

HLS is an architecture for flexibly
composing schedulers
11
Outline
 Motivation and Approach
 Guarantees
 HLS Design
 Augmented Reservations
 Conclusions and Demo
12
Guarantee

Definition:


Ongoing lower (and possibly upper)
bound on CPU allocation
Goals:
Describe the scheduling behavior
required and provided by schedulers,
and required by applications
 Permit these behaviors to be reasoned
about


Syntax:

TYPE p1 p2 …
13
Example Guarantees

100% of a CPU:


Strictly best-effort scheduling:


ALL
NULL
Proportional share:
PS s
 PSBE s δ


CPU Reservations:
RESBS x y, RESBH x y
 RESCS x y, RESCH x y

14
CPU Reservation Guarantees

Hard / Soft:
“Hard CPU reservation” ≠ hard real-time
 Soft reservations guarantee a lower bound
 Hard reservations also guarantee an upper
bound


Basic / Continuous:
15
Guarantee Conversion
by Schedulers

Schedulers require and provide
guarantees
SFQ: PSBE → PSBE+
 Rez: ALL → RESBH+


Schedulers determine if specific
guarantees can be provided

ALL → RESBH 5 10, RESBH 25 100
 EDF-based reservation scheduler
X Naïve rate monotonic reservation
scheduler
16
Selected Conversions by
Schedulers
Scheduler
Fixed Priority
SFQ
EEVDF
Lottery, Stride
Rialto, Rialto/NT
Rez, CBS
Linux/RT
Time Sharing

Conversions
any → any, NULL+
PSBE → PSBE+, PS → PS+
ALL → PSBE+
PS → PS+
ALL → RESCS+
ALL → RESBH+
ALL → RESBS+, RESBH+
NULL → NULL+
Full table contains 23 schedulers
17
Guarantee Conversion
by Rewrite Rules


Guarantees can be converted
without a scheduler, using rewrite
rules
Trivial examples:
PSBE s δ → PS s
 RESBH x y → RESBS x y


Non-trivial examples:
RESBS x y → RESCS x (2y-x+c) for
any c ≥ 0
 RESCS x y → PSBE (x/y) x/y (y-x)

18
Partial Rewrite Rule Table
ALL
T
F
T
F
T
T
T
RESBH
F
T
T
F
T
T
T
RESBS
F
T
T
T
T
T
T
RESCH
F
T
T
T
T
T
T
RESCS
F
F
T
F
T
T
T
PSBE
F
F
T
F
T
T
T
PS
F
F
F
F
F
F
T
NULL
F
F
F
F
F
F
F
→
ALL RESBH RESBS RESCH RESCS PSBE PS
T
T
T
T
T
T
T
T
NULL
T = rewrite rule exists
F = rewrite rule does not exist
19
Another View of
Some Rewrite Rules
RESBH
RESBS
PSBE
RESCH
RESCS
PS
20
Guarantees in Action
ALL
FP
ALL
NULL
RES
RESBH 5 33
*
* RESBH 10 20
J
Video
Player
PSBE 0.1 15
Word
Processor
RESBS 10 20 →
PSBE 0.5 10
SFQ
PSBE 0.3 35
Voice
Recognition
21
Related Work for Hierarchical
Guarantees


Hierarchical start-time fair queuing
[Goyal et al. 96]
Uniformly slower processors
Open environment for real-time
applications [Deng et al. 99]
 BSS-I and PShED [Lipari et al. 00]

22
Guarantee Contributions

Created a framework for reasoning
about composition of schedulers
Derived rewrite rules
 Integrated more than 20 schedulers


Guarantees provide a model of soft realtime CPU allocation
Independent of particular scheduling
algorithms
 Developers can program to
 Users can ensure that application
requirements are met

23
Outline
 Motivation and Approach
 Guarantees
 HLS Design
 Augmented Reservations
 Conclusions and Demo
24
Inside a Non-Hierarchical
CPU Scheduler
CPU
SCHEDULER

W
W
R
T1
T2
T3
W
T4
T5
Scheduling decisions are made in
response to timers and thread state
changes
25
Inside a Hierarchical
CPU Scheduler
parent virtual
processor
VP1
SCHEDULER
R
child virtual
processors

W
W
R
R
W
VP2
VP3
VP4
VP5
VP6
Distinguishing feature of hierarchical
schedulers: revocation
26
Hierarchical Scheduler
Infrastructure (HSI) Design



Schedulers are implemented by code
libraries loaded into the kernel
Scheduler instances are active
entities in the hierarchy
Virtual Processors
Represent potential for physical
processor allocation
 Used to notify schedulers

27
Scheduler Notifications

Hierarchical infrastructure notifies a
scheduler whenever:
A child VP requests or releases a
processor
 A parent VP grants or revokes a
processor
 A timer expires



Threads implicitly notify schedulers
when they block and unblock
Notifications are cheap: virtual
function calls
28
HSI and Scheduler
Implementation

HSI runs in Windows 2000 kernel
Serialized by a spinlock
 Added ~3100 lines of code


Loadable schedulers:
Time sharing / fixed priority, CPU
reservation, proportional share, join
 A representative set of schedulers,
but not a complete one
 Implemented Rez in about two days,
PS scheduler in a few hours

29
Performance


Test machine is a 500MHz Pentium III
Most mode change operations run in
less than 40μs


Create / destroy scheduler instance, begin
/ end CPU reservation, etc.
Median context switch time
Unmodified Windows 2000: 7.1μs
 HLS time-sharing scheduler: 11.7μs



Cost of each extra level in the
scheduling hierarchy is 0.96μs
Many opportunities for optimization
30
Performance Data:
Isolating Resource Principals
Without
Isolation
(1-level)
With
Isolation
(2-level)
N1
1
1
1
1
1
1
1
1
1
1
%100 N2
50.000
1
19.900
4
6.100 16
1.560 64
0.313 256
50.000
1
50.000
4
50.100 16
50.100 64
50.200 256
%2
50.0
80.1
93.9
98.4
99.7
50.0
50.0
49.9
49.9
49.8
31
Scheduling a Real-Time, CPUBound Application

Synthetic test application
Represents a virtual environment or game
 Must provide 1 frame per 33ms (~30 FPS)
 10ms of CPU time for each frame
 More frames are acceptable and desirable



Machine also runs background work
Goals
Application should never miss a deadline
 Non-real-time work should make progress
32

Performance Data: CPU
Bound Real-Time Application
App. Guarantee
NULL (high pri.)
NULL (def. pri.)
NULL (low pri.)
RESBH 10 33
RESBS 10 33

%a
96.70
49.90
3.11
32.60
67.30
FPS Misses %b
96.70
6 3.26
49.90
290 50.00
3.11
985 96.90
32.60
0 67.30
67.30
0 32.60
30-second runs
33
Related Work for HSI

Scheduler activations


CPU inheritance scheduling


[Anderson et al. 91]
[Ford and Susarla 96]
Vassal

[Candea and Jones 98]
34
Contributions


Virtual processors are a novel
extension of scheduler activations
[Anderson et al. 91]
Permit a wide range of schedulers to
dynamically create a scheduling
hierarchy in kernel of a GPOS


First system to do this
Simplifies scheduling programming
model by hiding OS and hardware
complexities
35
Outline
 Motivation and Approach
 Guarantees
 HLS Design
 Augmented Reservations
 Conclusions and Demo
36
Problem: Stolen Time




We have assumed until now that the
OS does not interfere with guarantees
However, while processing
asynchronous data the OS may steal
CPU time from applications
Time used by bottom-half mechanisms
is not accounted for correctly
A solution:

Augmented CPU reservations
37
Time Stolen by Network
Receive Processing
38
Augmented Reservations

Strategy: accurately measure stolen
time in order to compensate threads
Rez-C: avoid “charging” threads for
stolen time
 Rez-FB: use a feedback loop to assign a
larger CPU reservation so
requirements are met even when time
is stolen


Implemented as HLS schedulers
39
Augmented Reservation
Performance
40
Related Work for
Augmented Reservations

Scheduling bottom-half activity
Mach [Rashid et al. 89]
 Nemesis [Leslie et al. 96]


Scheduling bottom-half activity in a
GPOS


Feedback-based scheduling


FreeBSD [Jeffay et al. 98]
FC-EDF [Lu et al. 99]
Moving code into scheduled contexts

Soft modems [Jones and Saroiu 01]
42
Augmented Reservation
Contributions

Rez-C and Rez-FB
6% over-reservation to eliminate most
deadline misses
 vs. 24% over-reservation for plain Rez


Quantified severity of stolen time
Windows 2000 + Rez and Linux/RT
 Network, disk, software modem, USB

43
Outline
 Motivation and Approach
 Guarantees
 HLS Design
 Augmented Reservations
 Conclusions and Demo
44
Conclusions

HLS is feasible:
Composition of soft real-time
schedulers can be reasoned about
 Implemented and performs well


HLS is useful:
Permits scheduling behavior to be
tailored to specific needs
 New schedulers can be implemented
more easily

45
HLS Demonstration
FP
H
L
RES
T
1.
2.
3.
4.
TS
L
PS
TTTTTT TTTTTT
All threads scheduled by default timesharing scheduler
Create a CPU reservation
Make proportional share scheduler the
default and move time-sharing threads
46
End CPU reservation
The End

Papers and more info at:
http://www.cs.virginia.edu/~jdr8d
47