COMMA: Coordinating the Migration of
Multi-tier applications
Presenter: Zhaolei (Fred) Liu
Jie Zheng*
T.S Eugene Ng*
*Rice University, USA
Kunwadee Sripanidkulchai†
Zhaolei Liu*
†NECTEC, Thailand
1
Live migration in cloud
• For cloud providers
• Hardware maintenance
• Resource relocation
• For users
• Provision services
• Price and performance
2
Multi-tier application in cloud
Presentation tier
• In cloud, a multi-tier
application runs on
multiple VMs
• The VMs hosting a
multi-tier application
need to communicate
with each other
Logic tier
Data tier
3
How to migrate a group of VMs?
• Sequential migration: migrate VMs one by one
4
How to migrate a group of VMs?
• Parallel migration: migrate all VMs at the same time
5
Problem with sequential migration
• Application performance degradation caused by
component split during migration
6
Problem with parallel migration
• VMs still may not finish migration at the same time
• Static factors: VM disk size, memory size, etc.
• Dynamic factors: network bandwidth, application workload, etc.
7
COMMA: Coordinating the Migration of
Multi-tier Applications
COMMA
Parallel
Sequential
0
200
400
600
800
1000
1200
1400
Component Split time(s)
• Formulation & objective
• System design
• Algorithms
• Implementation & evaluation
8
Formulation & Objective
• Minimizing the migration’s impact on application performance
• Define impact as the time when VMs are split
Not ideal!
• Define impact as the volume of traffic impacted by migration
• TM: traffic matrix
• ti: the migration finish time of the i th VM
Minimize
9
COMMA: Periodic adaptation and coordination
Controller
Hypervisor Messages:
• Migration progress
• Available bandwidth
• Disk dirty rate and
memory dirty rate
(Pacer*)
Controller Messages:
• Start migration
• Set migration speed
Hypervisor
Hypervisor
Hypervisor
* J. Zheng, T. S. E. Ng, K. Sripanidkulchai, and Z. Liu. Pacer: A progress management system for live virtual machine migration in cloud computing. IEEE Transactions on Network and Service Management, 10(4):369–382, Dec 2013.
10
Coordination in the first stage of pre-copy
• Coordinate pre-copy of all VMs to finish at the same time
Stage 2
Stage 1
vm1
80
30
vm2
VM1 Pre-copy
VM2 Pre-copy
vm3
VM3 Pre-copy
20
vm4
50
Communication
Graph (KBps)
VM4 Pre-copy
Migration
Start
Time
Migration
End
11
Inter-group scheduling in the second stage
of dirty iteration and memory migration
• Meet the bandwidth limit by dividing VMs into different groups
Stage 2
Stage 1
vm1
80
30
vm2
VM1 Pre-copy
VM2 Pre-copy
vm3
VM3 Pre-copy
20
vm4
50
Communication
Graph (KBps)
VM3
VM4
VM4 Pre-copy
Migration
Start
vm1
20MBps
vm3
10MBps
vm2
5MBps
vm4
20MBps
Maximal dirty rate
Time
30MBps
Available bandwidth
VM1
VM2
Migration
End
12
Intra-group scheduling
VM3
Time
Performance
Start
at the same
Degradation；
time;
Short Migration
Finish
at different
Time
time
VM1
VM2
VM3
Time
Start
at the same
No
Performance
time;
Degradation；
FinishMigration
at the same
Long
time
Time
Bandwidth
VM1
VM2
Bandwidth
Bandwidth
• Maximize bandwidth utilization and minimize performance
degradation by scheduling dirty iteration inside each group in the
second stage
VM1
VM2
VM3
Time
No
Performance
Start
at different
Degradation；
time;
Short
Finish Migration
at the same
Time
time
13
Implementation & Evaluation:
• Fully implemented COMMA on KVM
platform, QEMU version 0.12.50
• Used SpecWeb and RUBiS as the
application
• Fully Evaluated COMMA on various
scenarios
14
COMMA is able to reduce application
performance degradation
Average response latency (ms)
Migrating 3-VM RUBiS using COMMA
Time(s)
15
Compared to COMMA, sequential migration incurs
high application performance degradation
Average response latency (ms)
Migrating 3-VM RUBiS using sequential migration
Time(s)
16
COMMA is able to minimize migration impact
0
500
1000
1500
2000
2500
Migration Impact (MB)
More results: vary the VM number, placement,
workload, and migration to EC2
17
Summary & Advantages
• COMMA is able to minimize application
performance degradation during migration
• COMMA has tiny overhead
• Efficient heuristic algorithms
• Computation time less than 10 ms
• COMMA is application independent
• Run-time adaptation
• All measurements are on hypervisor level
• COMMA has great applicability
• Designed for pre-copy model (KVM, XEN)
• Easily adapt to other migration models
18