Exploiting Gray-Box Knowledge
of Buffer Cache Management
Nathan C. Burnett, John Bent,
Andrea C. Arpaci-Dusseau,
Remzi H. Arpaci-Dusseau
University of Wisconsin - Madison
Department of Computer Sciences
Caching
• Buffer cache impacts I/O performance
– Cache hits much faster than disk reads
Without Cache Knowledge:
2 disk reads
With Cache Knowledge
1 disk read
OS
Buffer Cache
Data Blocks
2
Knowledge is Power
• Applications can use knowledge of cache
state to improve overall performance
– Web Server
– Database Management Systems
• Often no interface for finding cache state
– Abstractions hide information
3
Workload + Policy  Contents
• Cache contents determined by:
– Workload
– Replacement policy
• Algorithmic Mirroring
– Observe workload
– Simulate cache using policy knowledge
– Infer cache contents from simulation model
4
Gaining Knowledge
• Application knows workload
– Assume application dominates cache
• Cache policy is usually hidden
– Documentation can be old, vague or incorrect
– Source code may not be available
• How can we discover cache policy?
5
Policy Discovery
• Fingerprinting: automatic discovery of
algorithms or policies (e.g. replacement
policy, scheduling algorithm)
• Dust - Fingerprints buffer cache policies
– Correctly identifies many different policies
– Requires no kernel modification
– Portable across platforms
6
This Talk
• Dust
–
–
–
–
Detecting initial access order (e.g. FIFO)
Detecting recency of access (e.g. LRU)
Detecting frequency of access (e.g. LFU)
Distinguishing clock from other policies
• Fingerprints of Real Systems
– NetBSD 1.5, Linux 2.2.19, Linux 2.4.14
• Exploiting Gray-Box Knowledge
– Cache-Aware Web Server
• Conclusions & Future Work
7
Dust
• Fingerprints the buffer cache
– Determines cache size
– Determines cache policy
– Determines cache history usage
• Manipulate cache in controlled way
– open/read/seek/close
8
Replacement Policies
• Cache policies often use
– access order
– recency
– frequency
• Need access pattern to identify attributes
• Explore in simulation
– Well controlled environment
– Variety of policies
– Known implementations
9
Dust
I.
Move cache to known state
a.
b.
c.
Sets initial access order
Sets access recency
Sets frequency
II. Cause part of test data to be evicted
III. Sample data to determine cache state
•
Read a block and time it
Repeat for confidence
10
Setting Initial Access Order
Test Region
Eviction Region
for ( 0 test_region_size/read_size) {
read(read_size);
}
11
FIFO Priority
Newer Pages
Older Pages
FIFO gives latter part of file priority
12
Detecting FIFO
Out of Cache
In Cache
• FIFO evicts the first half of test region
13
Setting Recency
Test Region
Left Pointer
Eviction Region
Right Pointer
do_sequential_scan();
left = 0; right = test_region_size/2;
for ( 0 test_region_size/read_size){
seek(left); read(read_size);
seek(right); read(read_size);
right+=read_size; left+= read_size;
}
14
LRU Priority
LRU gives priority to 2nd and 4th quarters of test region
15
Detecting LRU
• LRU evicts 1st and 3rd quarters of test region
16
Setting Frequency
Test Region
2 3 4 5 6 6 5 4 3 2
Left Pointer
Eviction Region
7
Right Pointer
do_sequential_scan();
left = 0; right = test_region_size/2;
left_count = 1; right_count = 5;
for ( 0  test_region_size/read_size)
for (0  left_count) seek(left); read(read_size);
for (0  right_count) seek(right); read(read_size);
right+=read_size; left+= read_size;
17
right_count++; left_count--;
LFU Priority
LFU gives priority to center of test region
18
Detecting LFU
• LFU evicts outermost stripes
• Two stripes partially evicted
19
The Clock Algorithm
•
•
•
•
Used in place of LRU
Ref. bit set on reference
Ref. bit cleared as hand passes
Hand replaces a page with a
ref. bit that’s already clear
• On eviction, hand searches for
a clear ref. bit
Page Frame
Reference bit
20
Detecting Clock Replacement
• Two pieces of initial state
– Hand Position
– Reference Bits
• Hand position is irrelevant – circular queue
• Dust must control for reference bits
– Reference bits affect order of replacement
21
Detecting Clock Replacement
• Uniform reference bits
• Random reference bits
22
Clock - Reference Bits Matter
•
•
•
•
Two fingerprints for Clock
Ability to produce both will imply Clock
Need a way to selectively set reference bits
Dust manipulates reference bits
– To set bits, reference page
– To clear all bits, cause hand to sweep
• Details in paper
23
Dust Summary
• Determines cache size (needed to control eviction)
• Differentiates policies based on
– access order
– recency
– frequency
• Identifies many common policies
– FIFO, LRU, LFU, Clock, Segmented FIFO, Random
• Identifies history-based policies
– LRU-2, 2-Queue
24
This Talk
• Dust
– Detecting initial access order (e.g. FIFO)
– Detecting recency of access (e.g. LRU)
– Detecting frequency of access (e.g. LFU)
– Distinguishing clock from other policies
• Fingerprints of Real Systems
– NetBSD 1.5, Linux 2.2.19, Linux 2.4.14
• Exploiting Gray-Box Knowledge
– Cache-Aware Web Server
• Conclusions & Future Work
25
Fingerprinting Real Systems
• Issues:
– Data is noisy
– Policies usually more complex
– Buffer Cache/VM Integration
• Cache size might be changing
• Platform:
– Dual 550 MHz P-III Xeon, 1GB RAM, Ultra2
SCSI 10000RPM Disks
26
F
I
F
O
NetBSD 1.5
L
R
U
L
F
U
• Increased variance due to storage hierarchy
27
F
I
F
O
NetBSD 1.5
L
R
U
L
F
U
• Four distinct regions of eviction/retention
28
F
I
F
O
NetBSD 1.5
L
R
U
L
F
U
• Trying to clear reference bits makes no difference
• Conclusion: LRU
29
F
I
F
O
Linux 2.2.19
L
R
U
L
F
U
• Very noisy but looks like LRU
• Conclusion: LRU or Clock
30
F
I
F
O
Linux 2.2.19
L
R
U
L
F
U
• Clearing Reference bits changes fingerprint
• Conclusion: Clock
31
F
I
F
O
Linux 2.4.14
L
R
U
L
F
U
• Low recency areas are evicted
• Low frequency areas also evicted
• Conclusion: LRU with page aging
32
This Talk
• Dust
– Detecting initial access order (e.g. FIFO)
– Detecting recency of access (e.g. LRU)
– Detecting frequency of access (e.g. LFU)
– Distinguishing clock from other policies
• Fingerprints of Real Systems
– NetBSD 1.5, Linux 2.2.19, Linux 2.4.14
• Exploiting Gray-Box Knowledge
– Cache-Aware Web Server
• Conclusions & Future Work
33
Algorithmic Mirroring
• Model Cache Contents
– Observe inputs to cache (reads)
– Use knowledge of cache policy to simulate cache
• Use model to make application-level decisions
34
NeST
•
•
•
•
NeST - Network Storage Technology
Software based storage appliance
Supports HTTP, NFS, FTP, GridFTP, Chirp
Allows configurable number of requests to
be serviced concurrently
• Scheduling Policy: FIFO
35
Cache-Aware NeST
• Takes policy & size discovered by Dust
• Maintains algorithmic mirror of buffer cache
– Updates mirror on each request
– No double buffering
– May not be a perfect mirror
• Scheduling Policy: In-Cache-First
– Reduce latency by approximating SJF
– Improve throughput by reducing disk reads
36
Performance
144 clients randomly
requesting 200, 1MB files
Server: P-III Xeon, 128MB
Clients: 4 X P-III Xeon,
1GB
Gigabit Ethernet
Linux 2.2.19
• Improvement in response time
• Robust to inaccuracies in cache estimate
37
Summary
• Fingerprinting
– Discovers OS algorithms and policies
• Dust
– Portable, user-level cache policy fingerprinting
– Identifies FIFO, LRU, LFU, Clock, Random, 2Q, LRU-2
– Fingerprinted Linux 2.2 & 2.4, Solaris 2.7, NetBSD 1.5 &
HP-UX 11.20
• Algorithmic Mirroring
– Keep track of kernel state in user-space
– Use this information to improve performance
• Cache-Aware NeST
– Uses mirroring to improved HTTP performance
38
Future Work
• On-line, adaptive detection of cache policy
• Policy manipulation
• Make other applications cache aware
– Databases
– File servers (ftp, NFS, etc.)
• Fingerprint other OS components
– CPU scheduler
– filesystem layout
39
Questions??
• Gray-Box Systems
– http://www.cs.wisc.edu/graybox/
• Wisconsin Network Disks
– http://www.cs.wisc.edu/wind/
• NeST
– http://www.cs.wisc.edu/condor/nest/
40
F
I
F
O
Solaris 2.7
L
R
U
L
F
U
41
F
I
F
O
HP-UX 11.20 (IPF)
L
R
U
L
F
U
• Low recency areas are evicted
• Low frequency areas also evicted
• Conclusion: LRU with page aging
42
Related Work
• Gray-Box (Arpaci-Dusseau)
– Cache content detector
• Connection Scheduling (Crovella, et. al.)
• TBIT (Padhye & Floyd)
43
Clock - Uniform Reference Bits
File
Buffer Cache before test scan
Buffer Cache after test scan, before eviction scan
• After initial scan, cache state does not change
• First half of test region is evicted
44
Clock - Random Reference Bits
File
Buffer Cache before test scan
Buffer Cache after test scan, before eviction scan
• Initial Sequential Scan
• Test scan does not change cache state
45
Manipulating Reference Bits
Buffer Cache after touching all resident data
Buffer Cache after an additional small read
• Setting bits is easy
• Clear bits by causing hand to do a circuit
46