International Journal of Engineering Trends and Technology (IJETT) – Volume 23 Number 9- May 2015
Efficient Virtual Machine Memory Transfer in Datacenter with
Optimal Downtime
Thara D K#1, Girish L*2
#1
Assistant Professor, Department of ISE,
Assistant Professor, Department of CSE,
Channabasaveshwara Institute of Technology, Gubbi (T), Tumkur (D)
VTU, Belgaum, Karnataka, India
*2
Abstract— Cloud computing is a technology which relies on
sharing various computing resources instead of having local
servers to handle applications. Cloud computing is driven by
virtualization technology. Virtual machines need migration from
one host to anther due to the presence of error or over loading or
slowness in the current running host machine. Live Virtual
machine migration is the transfer of running virtual machine
from one host to another without stopping the current running
task. During this live virtual machine migration Downtime is one
of the key factors that have to be considered and assessed.
Here we present detailed survey on what are the importance of
live virtual machine migration in cloud computing technology
and various techniques to reduce the downtime during live
virtual machine migration. The flow chart showing the steps used
in Pre copy approach for VM migration. And also we present
the result of the comparison between the two virtual machine
migration environments, VMWare and Xen Server.
Keywords— Virtualization, Snapshots, Precopy, Post copy
I. INTRODUCTION
Virtualization is a technique where multiple operating
systems run concurrently on a single host operating system.
This technique is implemented since 1960‟s [1]. Each virtual
OS will have its own storage, resource sharing. Hardware
Virtualization (also called platform virtualization). Here the
virtual machine that is created works like an actual computer
with an operating system. Applications executing on these
virtual machines are separated from underlying hardware
resources. Some of the advantages of Virtualization are
decreasing costs, saving time and energy and also minimizing
risk overall, efficiency, control and move-ability.
The three important areas of IT where the virtualization is
becoming popular are network virtualization, server
virtualization and storage virtualization [2].
Network virtualization: In network virtualization all the
available resources in the network are combined by splitting
the available bandwidth into various channels, each of these
are independent from others and each one can be assigned to a
particular server.
Server virtualization: In Server virtualization, all the server
resources are masked from the server users. The server
resources include the amount and uniqueness of discrete
servers, Operating systems and also processors.
ISSN: 2231-5381
Storage virtualization: In Storage virtualization, the physical
storage from multiple network storage devices is pooled into a
single storage machine that is accomplished by a central
console. This kind of virtualization is generally used in
storage area networks (SANs).
Virtual machine is nothing but an illustration of a
physical machine by software. Each virtual machine will have
its own virtual hardware like CPU, RAM, hard disks, NIC etc.
The operating system and applications are loaded upon this
hardware. Instead of actual physical hardware, the operating
system sees only the consistent normalized hardware.
Example: The VMware virtual machines consisting of 64-bit
OS and virtual symmetric multiprocessing.
What is Virtual Machine Migration?
Virtual Machine
Cloud A
Cloud
B
V
M
V
M
Network Storage
Fig1. Virtual Machine Migration over cloud.
Virtual Machine migration is a technique that
migrates Virtual Machine from one cloud to another cloud for
some technical reasons. The snapshots of the Virtual machine
taken at one particular instance can be moved to another host
machine with its own hypervisor; if the VM is momentarily
stopped, snapshotted, relocated, and then restarted on the new
host, this is known as Virtual Machine migration. In case the
older snapshots are kept in sync repeatedly, this process can
be quite fast, and allow the VM to provide continuous service
without any interruptions while its previous physical host is,
for example, taken down for physical maintenance.
http://www.ijettjournal.org
Page 454
International Journal of Engineering Trends and Technology (IJETT) – Volume 23 Number 9- May 2015
indication that more number of jobs is loaded than it can
handle.
Types of Virtual Machine Migration
1.
2.
3.
4.
5.
6.
7.
Live Migration
Quick Migration
SAN Migration
Network Based Migration
Storage Migration Type
Storage vMotion
Quick Storage Migration
Because of the above said reasons if one network is not in a
situation to handle data and transactions, then instead of going
for redundancy which is a highly expensive solution, the work
can be transferred to another network. This transferring of
work from one network to another network can be done using
a technique called Live Migration.
What is a SNAPSHOT?
A snapshot is the current state of a virtual machine at
that exact moment, and normally its storage devices, at an
precise point in time. A snapshot allows the virtual machine's
state at the time when the snapshot to be returned in the later
stage, successfully undoing any changes that occurred
subsequently. This capability is useful as a backup technique,
for example, previous to execution of a dangerous operation.
Along with storage devices, other components of the
virtual machine can also be encompassed in a snapshot, such
as
the
contents
of
its random-access
memory (RAM), BIOS settings, or its configuration settings,
processes status in the form of process control block, register
values, intermediate results, data files and object files. "Save
state" feature in video game console emulators is an instance
of such snapshots. Reinstating the snapshot consists of
removal or ignoring all overlapping layers that are added after
that snapshot, and directing all new changes to a new overlap.
What is the Need for Migration?
Virtual machines are migrated from one cloud to another
cloud because of the following reasons:
1: Presence of error in the hardware in the present VM.
The hardware fault can be of various types. For different types
of inputs values, the output value may be same. Or for the
same input value different output might be observed at
different times. There may be some fault with the storage
devices in the network, because of which there may be
chances to lose data. In order to overcome these kinds of
problems with the hardware, it is required to periodically
diagnose all the hardware components of the network and if
any problem is detected with the hardware components then
suitable remedial steps should be taken in order to overcome
those problems.
2: The load on the present VM may be more than its
maximum limit.
This case might be observed in the situations like; the number
of users of the network might be increasing enormously. Huge
amount of data might be stored and exchanged between users
in the network.
3: Heavy load might cause the system to slow down.
When the number of users and the amount of data is increased
enormously, the time taken by the system might increase in
order to send or receive any data between the users or between
the systems. If the CPU is responding slower, it is an
ISSN: 2231-5381
II. METHODOLOGY
What is Live Migration?
Live Migration is a kind of Virtual Machine
migration. This is a very powerful type of migration for
cluster and cloud administrator. The Operating system along
with the application is migrated from one VM to another
without any loss of data. Here Memory regions are copied
incrementally from source machine to destination machine
while the virtual system is still in an executing state. Over the
time, the VM on the source is stopped and when all the
resources are copied, the VM is resumed on the destination
machine. Platforms available for Live VM Migration are:
Hyper-V (2008 R2), ESX 3.0, 3.5 [3].
Technology Used for Transfer are: Windows server 2008
Failover cluster, Hyper-V, vMotion foe ESX.For a VM to
migrate, it is essential that memory is copied completely to the
endpoint before CPU in source is disabled and CPU in
endpoint is triggered.
CPU
CPU
Memory
Memory
Source machine
Destination Machine
Fig: 2 Memory Migrations
The factors that have to be assessed during Live VM
migration are given below:
1.Time taken for VM migration, is the total time taken from
the time the VM migration is started until the time VM
migration is finished.
2. Down time during VM migration, which indicates the time
duration where both the source VM and destination VM are
inactive.
3. Preparation time is the time measure from the time VM
migration is initiated until the time the processor state is
transferred to the target VM.
4 Pages Transferred, is the total number of memory pages
transferred from source VM to destination VM, including the
dirty pages.
5. Resume time, the time taken to resume the process of the
destination VM after transferring all the data from source VM.
The transfer of main memory is the difficult task during VM
migration. There are two approaches for VM migration, Post –
http://www.ijettjournal.org
Page 455
International Journal of Engineering Trends and Technology (IJETT) – Volume 23 Number 9- May 2015
Copy and Pre-Copy Memory Migration [4]. During memory
migration, the Read only pages are transferred only once
whereas to transfer writable pages there are 3 phases that need
to be followed.
Start
Working Principle of Pre-Copy approach:
Basically the Virtual Machine Migration contains
Source machine and a target machine. Both the source and
destination machines must have Integrity VM running and
also should be able to execute the guests. They should also
conform to their respective Operating system specifications
and conditions and also must be in a position to deliver the
resources allocated to the guests. For example:
1. If the guest is making use of 4GB of memory in the source
machine, it must be in a position to use same amount of
memory on the destination machine.
2. If the source machine provides 4 CPUs to the guest, the
destination machine should also provide 4 CPUs to the guest
after migration.
Pre Copy Algorithm:
Pre copy algorithm works iteratively in order to increase the
performance of Virtual machine migration by decreasing the
dirty pages as much as possible. The memory pages which are
to be transferred during ith round are modified during (i-1)th
round. The algorithm is presented in the form of a flow chart
as shown below [12].
In order to perform the migration operation, both the source
and destination machines must be symmetrically configured.
The network and storage specifications must be same on both
the machines. The following are the characteristics of a
symmetric configuration:
In case of Pre-copy approach, the first phase is the push phase
which includes; The Warm up phase and Stop and Copy phase
[6].
Memory is divided into numerous memory pages. In usual
migration these memory pages are copied to destination while
the CPU is executing state on source host [7].
CPU
CPU
Memory Migration
Source
Memory
Memory file to be transferred M is
initialized to VM
Number of round R initialized
to 1
Copy the VM dirty pages if page reaches
threshold
Perform traditional pre copy, save changed
memory
Transfer the speed for last round and move the contents of Temp M to total
memory M
Dirty Page of current round is calculated & CPU time
allocated to VM
New execution percentage E is calculated based on the
dirty page P
Constant 1 is added to each
round
Pause the VM & remaining memory is sent via stop & copy
method.
Destinatio
n Memory
Resume VM on the
destination.
Fig 4: Copying memory pages from source to destination
Stop
Fig 3: Flow chart for Iterative Pre Copy algorithm
When memory pages are copied from source machine to
destination machine some of the memory pages in source are
changed. This is because the CPU in source machine is in
running state and using the memory. The altered memory
pages are called "dirty pages" and are highlighted in filled
color in above picture. Next stage is to recopy these dirty
pages to the endpoint. This phase is called "warm-up" phase.
ISSN: 2231-5381
http://www.ijettjournal.org
Page 456
International Journal of Engineering Trends and Technology (IJETT) – Volume 23 Number 9- May 2015
CPU
CPU
Memory Migration
Destination
Memory
Source
Memory
Fig 5: Warm-up Stage
The active pages are transferred from source machine to
destination machine. The next phase is the stop and copy
phase. In this phase the source VM is stopped, all the
remaining memory is copied to the destination VM. The
destination VM is then resumed after all the replication is
done. The last phase is the pull phase; here the destination VM
may try to access the memory pages which are not yet
migrated.
Warm-up phase will recur until the amount of recopied dirty
pages is equal to the amount of new dirty pages. During this
time the CPU in source machine will be disabled and the rest
of the dirty pages are copied to the destination. This stage is
called down-time stage because both CPU in source machine
and destination machine are deactivated.
CPU
Source
Inactive
CPU
Memory Migration
Destination
Inative
increase the speed of memory operations. During VM
migration, compared to polling DMA is the best technique to
transfer data from one host to another host.
2: Data compression, also called compaction, is the technique
of using encoding approaches to reduce the amount of data
needed for the storage or transmission. To reduce the
downtime during virtual machine live migration, the memory
data is compressed before sending it across the network. This
makes use of the spare CPU capacity in the server running
Hyper-V.
3: Instead of transferring all data at once, only the data that is
required for the currently running application can be
transferred.
4: Usage of Global memory which can be accessed by other
hosts in the network. The data files which are reserved in
global memory need not be transferred to the destination host
at the time of VM migration.
5: RDMA: Remote direct memory access performs the direct
memory access from the memory of one computer to the
memory of another computer without comprising the
operating system of either of the computer. This gives high
throughput, low-latency networking during Virtual machine
live migration. RDMA is not so widely because of the
drawback that it requires to install a different networking
infrastructure.
6: Server Consolidation: Server consolidation is a method
for the proficient usage of computer server resources in order
to reduce the whole number of servers or server locations that
an organization requires [9].
Fig 6: Down time stage
Techniques to reduce downtime during Virtual Machine
Migration [8]:
Reducing downtime during virtual machine migration is
important; this is because it will reduce the amount of memory
pages that needs to be recopied. The amount of memory
change during virtual machine migration is not same for each
instance of migration. Selecting the correct dirty page
becomes more important in order to reduce the down time.
The dirty pages which are less prone to change should be
copied first and then the dirty pages which are more prone to
change.
At the time of migration since data is present both on the
source machine as well as on the destination machine more
memory is required in order to store same data in two
machines. A delay is always incurred at the time of transfer of
data mainly due to the use of slow I/O techniques like
interrupt driven I/O or polling. Down time is the sum of data
transfer time and the time required by destination host to start
VM under migration.
1: Direct memory access (DMA) is a method that allows an
input and output device to send and receive data directly to
and from the main memory, bypassing the CPU in order to
ISSN: 2231-5381
Fig 7: Server Consolidation
Comparison and Analysis of Virtual Machine
environments VMware and Xen Server
VMWare and Xen are the Virtual machine environments that
support live migration of guest operating system. This is used
in the data centers in order to facilitate uninterrupted service
during maintenance or to move the computations/data away
from failed hosts. The factors like time taken for migration
and also the virtual machine downtime are important during
assessment because the service availability agreements might
be violated.
In Live Virtual machine migration, the downtime and total
migration time are the two important factors need to be
considered and assessed. Various algorithms have been
designed in order to reduce the downtime and total migration
time. These algorithms differ in the order of transfer of
process state, the amount of data transferred etc. [10].
In this paper we are giving the comparison between VMWare
and Xen Server using Pre Copy approach[3]. The important
steps used in the algorithm are listed below. Consider the
Source machine „X‟ and the destination machine „Y‟[11].
http://www.ijettjournal.org
Page 457
International Journal of Engineering Trends and Technology (IJETT) – Volume 23 Number 9- May 2015
1: In the first step is the Source Reservation stage, the source
machine „X‟ sends a request to the destination machine „Y‟.
First thing that is to be done is to make sure that there are
enough number of required resources in the destination
machine. If yes then the destination machine with the same
amount of resources must be reserved for that virtual machine
migration. If not the virtual machine might choose some other
machine as the destination.
2: This is the Iterative stage [5]. The memory in the source
machine „X‟ are copied to the destination machine „Y‟
iteratively. During the first iteration all memory pages are
copied from „X‟ to „Y‟, during the latter iterations the memory
pages whish are modified at the end are copied.
3: This is called the Stop and Copy stage. In this stage,
copying memory pages is stopped. The source machine X‟
which was in the running state is suspended. All the network
connections and remaining dirty pages are transferred to the
destination machine. In case the Migration process fails for
some reason, the source machine „X‟ can be restarted and
work can be continued.
4: The destination machine gives an acknowledgement to the
source machine saying that the VM image has been delivered
successfully. After receiving this acknowledgement, the
source machine „X‟ destroys the original VM.
5: In this last stage, the destination machine activates and
provides a new IP address to the migrated VM.
RESULT ANALYSIS
Table 1 shows the results of Xen Server and the VMware
virtual machine environments. Because of the space
boundaries, the report of VMware is reported only for
VMSIZE up to 4GB. The same behaviour is observed for the
VMSIZE of other values. Whereas the behaviour deviates
deliberately in case of Xen. It all depends on the hypervisor
product that is selected. This selection has got significant
impact on availability. The reason for different behaviour may
be due to the usage of different rate adaptive algorithms in the
two virtualization products. The table below shows the
varying data for VMware. The maximum mean downtime
observed is 15.27 times larger than the minimum mean
downtime. This value may go up to 22.83 according to the
observations made on other experiments.
Table 1: Data Variation. VMWare versus Xen Server [3]
Hypervisor/
Guest
Time
Mean
time
[s]
Mean
Overall
XenServer/
CentOS
Migration
88.73
8.01
8.10
Treatment
Max [s]
5.32
Overall
[s]
38.08
Downtime
Migration
6.69
29.93
2.17
1.24
2.46
1.96
0.62
6.72
1.94
6.51
VMware/
ISSN: 2231-5381
Max : Min Ratio
Standard Deviation
Linux
Downtime
2.10
15.27
22.83
0.52
0.80
IV .CONCLUSION
In this paper we have given a brief survey about the Virtual
machine migration and its importance, Snapshot, kinds of
virtual machine migration, working principle of Pre copy
Migration. We have tried to explain about the various
techniques used to reduce the downtime during virtual
machine migration. At the end we have given a small
comparison between two virtual machine environments,
VMWare and Xen Server. It has been observed that the
memory load and memory access are the two factors that
cause variation in the performance of live migration.
The future work includes the comparison of other virtual
machine environments with respect to downtime and
migration and to find the best approach to reduce the
downtime during virtual machine migration.
REFERENCES
[1] Amirreza Zarrabi, “A GENERIC PROCESS MIGRATION
ALGORITHM” , International Journal of Distributed and Parallel Systems
(IJDPS) Vol.3, No.5, September 2012.
[2] Mark Blunden Mik Berx-Debeys Daeseop Sim, Storage Networking
Virtualization What‟s it all about? Red Books.
[3] Felix Salfner, Peter Tr¨oger, Andreas Polze “Downtime Analysis of
Virtual Machine Live Migration”, Depend 2011: The fourth International
Conference on Dependability.
[4] Divya Kapil, Emmanuel S. Pilli and Ramesh C. Joshi ,” Live Virtual
Machine Migration Techniques:Survey and research Challenges” .Third IEEE
International Advance Computing Conference(IACC), 2013.
[5] Khaled Z. Ibrahim, Steven Hofmeyr, Costin Iancu, Eric Roman,
Optimized Pre-Copy Live Migration for Memory Intensive Applications.
[6] Diego Perez-Botero, “A Brief Tutorial on Live Virtual Machine Migration
From a Security Perspective”.
[7]Fereydoun Farrahi Moghaddam and Mohamed Cheriet, Senior Member,
IEEE
Synchromedia Lab, Ecole de technologie supérieure, Montreal. Decreasing
Live Virtual Machine Migration Down-Time Using a Memory Page Selection
Based on Memory Change.
[8] Dr. Abhay Kothari, Ashish Agrawal, “Techniques for Reducing the Down
Time during Virtual Machine Migration: Resource Failure & Live Migration
Circumstances” International Journal of Computer Science and Information
Technologies.
[9] National Computer Board, 7th Floor Stratton Court, La Poudriere Street,
Port Louis, GUIDELINE on SERVER CONSOLIDATION and
VIRTUALISATION.
[10] M. Richmond and M. Hitchens. A new process migration algorithm.
ACM SIGOPS Operating Systems Review, 31(1):31–42, 1997.
[11] Fuchen Sun, Kar-ann Toh, Manuel Grana Romay, Kezhi Mao, Extreme
Learning Machines 2013: Algorithms and Applications.
[12] C.Sagana, M.Geetha, Dr R.C.Suganthe Enhancing the Performance of
Live Migration of Virtual Machine‟s with
WSClock Replacement Algorithm.
http://www.ijettjournal.org
Page 458