Memory Buddies: Exploiting Page
Sharing for Smart Colocation in
Virtualized Data Centers
Timothy Wood, Gabriel Tarasuk-Levin,
Prashant Shenoy, Peter Desnoyers*,
Emmanuel Cecchet, and Mark D. Corner
University of Massachusetts, Amherst
*Northeastern University
Server Placement in Data Centers
• Virtualization improves
resource utilization by
consolidating servers
• But how to determine which
servers to place together?
• Must consider many resource constraints:
– CPU, Disk, Network, Memory
Why Memory?
• CPU scheduling is fine grain
– Easily share among many users
– Work conserving, so no waste
• Memory is much less flexible
– Allocated on a large time scales
– Being wrong (paging) is disastrous
• Memory is an expensive resource
• Memory capacity is increasing slower than
CPU power
Content Based Page Sharing
• If two VMs have an
identical pages in memory,
just keep one copy
• Supported by VMware
ESX platform
– Experimental tests in Xen,
further support planned
• Potential benefits
– 33% in VMware ESX paper
– 65% with subpage sharing
(Difference Engine)
Physical
RAM
FREE
A
D
FREE
B
Hypervisor
A
D
B
VM 2
Page Table
B
A
B
C
C
VM 1
Page Table
A
1) Hypervisor detects identical
pages
2) Copy-on-Write references
created for shared pages
But what if….
• Pages change
over time,
breaking sharing
• If memory is being
overcommitted,
this can lead to
hotspots
Physical
RAM
FREE
A*
A*
A
D
B
D
VM 2
Page Table
FREE
A
B
C
A
B
C
VM 1
Page Table
What’s the problem?
• Only get a benefit if VMs on a machine actually
have pages to share!
Physical
RAM
FREE
E
D
FREE
F
Host 1
E
D
F
VM 1
Page Table
B
A
B
C
C
VM 2
Page Table
A
Physical
RAM
FREE
E
D
FREE
F
Host 2
E
D
F
VM 3
Page Table
B
A
B
C
C
VM 4
Page Table
A
Where to place a VM?
• How do you figure out which
VMs to place together?
– Meet resource constraints
– Maximize sharing
• Why placement is hard in
large data centers?
– Many applications from
different clients
– Many software stacks /
platforms
– Workloads change over time
?
Here or there?
Or there or there or there…?
Memory Buddies Goals
• Efficiently analyze the memory contents
of multiple VMs to determine sharing
potential
• Find more compact VM placement
schemes
• Respond quickly to changing conditions
to prevent memory hotspots
Bonus! Traces released at traces.cs.umass.edu
Outline
•
•
•
•
•
•
Motivation
Memory Fingerprinting & Comparison
Sharing-aware Colocation
Hotspot Mitigation
Implementation & Evaluation
Related Work & Summary
Memory Fingerprints
• Hypervisor creates hash for each page
– Check hash table to see if page is sharable
– Record these hashes to create fingerprint
A
B
• Hash lists are big
0x11223344
0x55667788
VM 1
– 32bits per 4K page = 1MB per 1GB of RAM
– Need to forward fingerprint to other hosts
• Comparisons of lists is relatively slow
Bloom Filter Fingerprints
• Bloom filter is a probabilistic data structure
– Stores keys by setting some bits to 1
– False positive chance at lookup from hash
collisions
M bits
0x11223344
0x55667788
VM 1
0 1 1 0
…
1 0 0 1
Insert(key) --> set h1(key)=1 and h2(key)=1
• Very space efficient
• Tradeoff between filter size and accuracy
Fingerprint Comparison
• Hash list comparison
– Sort each list and then step through
• Bloom Filter
– Simple method: Dot product of bit vectors
1 1 0 0 1 0 1 0 0 1
0 1 1 0 1 1 1 0 1 1
=4
– Bloom Sharing Equation
• Corrects for the expected number of false
matches in each filter
• Impressively accurate!
Eval: Fingerprinting
• 4GB RAM VMs
– Hash: 4 sec
– Sorted: 0.3 sec
– Bloom: 0.02 sec
• Bloom Fingerprint
10% the size,
still < 1% error
Bloom filters are smaller and 10 to 100 times faster
Outline
•
•
•
•
•
•
Motivation
Memory Fingerprinting & Comparison
Sharing-aware Colocation
Hotspot Mitigation
Implementation & Evaluation
Related Work & Summary
Sharing Aware Placement
•
•
•
•
•
•
Where to place a freshly started VM?
Use staging area to find initial placement
Find feasible hosts
Estimate sharing potential
1100011
Migrate VM
Host 1
Done!
Compare
Staging Host
1010101
+
1000010
1110000
= 1110010
Host 3
Host 2
Consolidation & Hotspot Mitigation
• Resource usage changes over time
– Sharing may not last forever
• Periodically consolidate servers
–
–
–
–
Identify candidates (least loaded hosts)
Match to destinations (hosts with best sharing)
Migrate VMs
Disable unnecessary servers
• Hotspot Mitigation
– Monitor memory usage to detect hotspots
– VMs may run out of memory if sharing stops
– Redistribute VMs to rebalance
Offline Planning Tool
Host resources
Resource Traces
Memory Fingerprints
Offline Planning Tool
Dynamic programming
based bin-packing tool
Host 1
Host N
Number of hosts required = X
Finds subsets of VMs that
VM
to host
can be placed
together
andmapping
Estimated
sharing per host = Y
maximize
sharing
Outline
•
•
•
•
•
•
Motivation
Memory Fingerprinting & Comparison
Sharing-aware Colocation
Hotspot Mitigation
Implementation & Evaluation
Related Work & Summary
Implementation
• Memory Tracer
– Tool used to gather data for trace study
– Runs on Linux, OS X, and Windows
– Calculates 32bit hashes for each page in
memory
– Sends either a hash list or Bloom filter to
control node
• Works on physical systems or in VMs
Implementation
• Nucleus
– Collects memory
fingerprints for
each VM
– Sends data to
control plane
• Control Plane
– Gathers VM statistics and makes migration decisions
based on sharing
– Interacts with VMware Virtual Infrastructure to
manage VMs
Eval: Trace Study
System Mix
Total Memory
MB
% Sharable
Used Mem MB
(with sharing)
Linux 2.6.9
Darwin 9.0
Windows XP
4223
13.2%
3666
Darwin 9.0
Darwin 9.4
Windows XP
5248
35.3%
3397
Darwin 9.0
Darwin 9.2
Darwin 9.4
Windows XP
6272
36.8%
3966
Darwin 9.4
(3 MacBook + iMac)
8192
40.0%
4917
Eval: App Placement
• Try to place as many VMs onto a set of 4 hosts
• Sharing Oblivious: Place on first host with sufficient
capacity
• Four app types -- data contents different for each VM
instance
Sharing Aware
Sharing Oblivious
TPC-W
OFBiz
20 VMs
17 VMs
RUBiS
SpecJBB
1
2
3 4
Host
1
2
3 4
Host
Outline
•
•
•
•
•
•
Motivation
Memory Fingerprinting & Comparison
Sharing-aware Colocation
Hotspot Mitigation
Implementation & Evaluation
Related Work & Summary
Related Work
• Waldspurger, OSDI 2002
– CBPS in VMware ESX Server
• Gupta, et al., OSDI 2008
– Increase sharing potential by looking at
parts of pages
• VM Memory provisioning
– Zhao & Wang (yesterday) has a good list!
Summary
• Hypervisors already support page sharing…
• Memory Buddies makes it more useful
– Identifies sharing opportunities across data center
– Migrates VMs to maximize sharing
– Uses efficient memory fingerprinting techniques to
scale to large data centers
• Traces will be online (soon) at:
– http://traces.cs.umass.edu
– Macbooks, Linux servers, and more!
• Questions?
twood@cs.umass.edu