ITRI Cloud OS & Virtual
Resource Management
Patrick Fu
System Software Division, CCMA/ITRI
patrickfu@itri.org.tw
Copyright 2010 ITRI工業技術研究院
1
Agenda
•
•
•
•
Cloud OS introduction
Physical resource provisioning
Virtual resource management
Adaptive provisioning and power management
Copyright 2010 ITRI工業技術研究院
2
What is Cloud OS ?
•
System Management Software layer
– Physical Resource Provisioning
– Virtual Resource Management
• Improve manageability of massive
Cloud Data Center
• Enhance self-provisioning
• Optimize physical resource utilization
• High Availability for any single point
of failure
• Energy management
– Highly Available Distributed Storage
Management
– Service Load Balancing
– Security
– High Speed Networking
•
Mail
Virtual
Cluster
Backup
Virtual
Cluster
HC
Virtual
Cluster
AppX
Virtual
Cluster
System
Service
daemons
Compute
Nodes
Cloud
OS
agents
Data Nodes
Service Nodes
What is it not?
– It’s not Operating System
– It’s not Virtualization Hypervisor
Physical
Node
Physical
Node
Copyright 2010 ITRI工業技術研究院
Physical
Node
Storage
Storag
Server
e
Storage
Storag
Server
e
Server
Server
3
Service/Technology Mapping
SaaS
PaaS
Applications
Automated Cloudification Technology
Cloud Application Middleware Platform
LAMP
.NET
WebSphere
WebLogic
Google App Engine
Cloud System Software Platform
Hypervisor
Virtualization Mgmt
Storage Mgmt
Security
Backup/Replication
Data Center Automation
Energy Management
IaaS
Cloud Hardware Platform
Scalable System Architecture
Servers
Storage Arrays
System Management
+
Switches
Copyright 2010 ITRI工業技術研究院
Cooling
Power Distribution
4
Software Stack for Cloud OS
Virtual DataCenter Mgmt Console
Virtual Cluster Provisioning
System/Network
Management
Physical
Cluster
Deployment
Tool
Physical Compute Servers
Distributed Main/Secondary Storage
Security
All-layer-2 Network
Intra-Virtual-Cluster
Load Balancing
Power
Management
Virtual Machine Management
Copyright 2010 ITRI工業技術研究院
5
Cloud OS Service Model
• Provisioning & Runtime monitoring of Virtual Resources
– Virtual Instance
 Hypervisor construct
 An image of a guest OS
– Virtual Cluster
 A group of VM instances providing same service, front-ended by a network
load balancer
 Configuration
-
# of virtual machines and its configuration
Storage space requirement
External network bandwidth requirement
Load balancing policy
Firewall/IDS setting
Network configuration, including DNS and DHCP
OS image and application image
– Virtual Data Center
 One or more virtual cluster working in coordination (multi-tier web services,
EMR’s, VDI’s, etc)
Copyright 2010 ITRI工業技術研究院
6
CloudOS Virtualization Level
VDC
VDC
VCluster
VCluster
…
OS
OS
OS
Copyright 2010 ITRI工業技術研究院
vm
APs
APs
OS
…
APs
APs
OS
PM
vm vm vm
OS
OS
…
vm
APs
APs
OS
APs
APs
OS
PM
vm vm vm
OS
OS
…
vm
APs
APs
OS
APs
APs
PM
vm vm vm
OS
OS
OS
APs
OS
APs
APs
…
vm
APs
vm vm vm
VCluster
PM
7
CloudOS
Resource Provisioning
• To prepare VMs with appropriate resources and make
them ready for user applications
– Allocating resources to VMs to match the workloads
• To prepare a virtual cluster with appropriate instances
and make it ready for virtual cluster computation
– Consolidating VMs onto physical servers
• Goals:
– High resource utilization
– Energy efficiency
– Low performance interference
Copyright 2010 ITRI工業技術研究院
8
Provisioning Challenges
• VM size estimation
– Static SLA model/forecasting future use
• Placement
– Deploy a VM onto physical servers (initially)
– Policy: immediate, best effort, advance reservation, etc.
• Consolidation and Load Balancing
– High consolidation ration and resource utilization – low cost of running data
center
– Statically
• Heuristic based
• Average resource utilization
– Dynamic replacement
• Measure-Forecast-Remap (MFR)
• Live migration
• Balancing overloaded and underloaded nodes
– Constrained bin packing problem w/ SLA
• Performance isolation
– Cohosting VMs on a server creates performance interference
– How to model and prevent the interference
Copyright 2010 ITRI工業技術研究院
9
RPM Static Resource Provisioning
• Statically provision from SLA
• SLA w/ historical data?
– No, conservatively allocation
– Yes, forecasting joint-VM provisioning
• Immediate provisioning model (before instantiation of
virtual machine)
• Placement policy
– Proprietary
– Virtual cluster affinity placement policy
• Performance gain from locality
• Place VMs from the same virtual cluster as possible
• Need experiments to support
Copyright 2010 ITRI工業技術研究院
10
CloudOS
Our Motivation
Data Center Monthly Cost
Servers
Networking
Equipment
5%
13%
54%
21%
Power Distribution &
Cooling
Power
8%
Other Infrastructure
Source: Cost of power in Large-Scale Data Center, James Hamilton Blog, 11/28/2008
Copyright 2010 ITRI工業技術研究院
11
Joint Provisioning via VM Multiplexing
• Dataset from a
commercial data center
– 15,897 VMs
– 1325 physical hosts
• 94% of the hosts have
more than one VM
• Joint provisioning
averagely saves 40% of
the capacity
Copyright 2010 ITRI工業技術研究院
12
Meng, X. et al. Efficient Resource Provisioning in Compute Clouds via VM Multiplexing. ICAC ‘10
Joint-VM Provisioning at Runtime
Capacity
Capacity
VM
PM
Hypervisor
Hypervisor
PM
VM
t
t
100%
100%
Capacity
Capacity
t
Under provisioning
t
Over provisioning
Copyright 2010 ITRI工業技術研究院
13
CloudOS
Load balancing and DVMM
PM reconfiguration
Reach reconfiguration point
Resource Provisioning Manager (RPM)
Utilization ratio
Consolidation manager/ DVMM
VM victim histogram
Joint-VM histogram
Over provisioning
Under provisioning
PM
PM
PM
PM
Copyright 2010 ITRI工業技術研究院
PM
PM
14
CloudOS
Cloud OS RPM Software Components
Adaptive physical resource provisioning
Utilization rate reaches threshold, sending reallocation request
Static joint-VM
provisioning
Reconfiguration map
Placement
RPM core
VM monitoring
New PM map
Consolidation
manager
Victims
Runtime
joint-VM
provisioning
Performance interference
Utilization change
Power on/off
PMs
New PM map
PRM
DVMM
Copyright 2010 ITRI工業技術研究院
15
Cloud OS
Load balancing
Copyright 2010 ITRI工業技術研究院
16
Consolidation plan
Copyright 2010 ITRI工業技術研究院
17
Migration plan
Copyright 2010 ITRI工業技術研究院
18
Runtime Reallocation
VM3
VM3
VM3
VM2
VM2
VM2
VM1
VM1
VM1
PM1
PM2
PM3
VM3
VM3
VM3
VM1
VM1
VM1
PM1
PM2
PM3
VM1
VM1
VM1
PMi
PMj
PMk
VM2
VM2
VM2
VM2
VM1
VM1
VM1
PMi
PMj
Copyright 2010 ITRI工業技術研究院
PMk
19
Adaptive Physical Resource Provisioning
Power Management
PM Pool
Provisioned
Utilization threshold
Over provisioning
Low utilization
Under provisioning
High utilization
PM reallocation algorithm
Utilization rate reaches threshold, sending reallocation request
Reconfiguration map
VM placement
DVMM live migration
Load balancer
Consolidation manager
New PM map
Utilization monitor
Power on/off PMs
PRM
Copyright 2010 ITRI工業技術研究院
CloudOS
20
Challenges
• Triggering mechanism
–
–
–
–
No workload consolidation “recently” (e.g. past hour)
No physical machine load balancing going on
No physical server was powered on “recently” (e.g. past hour)
Avoid oscillation
• Cost of migration
– Network load
– Cache effects
– Domain in suspension
• Multi-dimensional bin packing
– CPU, Memory, Network, Disk I/O
• Migration plan
– Only 1 migration per Physical server @ a time
– # of cores vs. # of VMs
Copyright 2010 ITRI工業技術研究院
21
Backup
Copyright 2010 ITRI工業技術研究院
22
Software architecture
Copyright 2010 ITRI工業技術研究院
23
Procedure of Power management in Monitoring thread
Receiving CCinstance data per 30 sec
Calculate
instMonitorThreadData->cputotal
instMonitorThreadData->memorytotal
instMonitorThreadData->count
Receive data from Dom0
Calculate the data
Workload
Trigger?
No
Check
instMonitorThreadData->state
instMonitorThreadData->pwmtime
instMonitorThreadData->lbtime
instMonitorThreadData->cputotal
instMonitorThreadData->memorytotal
instMonitorThreadData->count
Yes
Perform
Change instance state
instDvmmBloc->state=doinglb
instMonitorThreadData->state=doinglb
instDvmmBloc->destHost=resource->hostName
Change external state
Consolidation Plan
Stop receiving data from Dom0
No
Call PRM to turn on
Machine
Do Load balancing
M*K < N ?
If necessary
Yes
Do Consolidation
Done
PRM to shut down
Copyright 2010Call
ITRI工業技術研究院
Machine
Change instance state
instDvmmBloc->state=doingpwm
instMonitorThreadData->state=doingpwm
instDvmmBloc->destHost=resource->hostName
Change external state
Change instance state
instDvmmBloc->pwmtime=now
instDvmmBloc->state=pwmdone
24
instMonitorThreadData->state=pwmdone
instMonitorThreadData->pwmtime=now
Data Mining:
VM Resource Usage Patterns of each VC
•
Find VM resource usage patterns for each VC
CPU usage
Monday
High (or unpredictable)
Medium
Low L
•
Aid to predict the trend of resource usages
Copyright 2010 ITRI工業技術研究院
Time
25
Q&A
Thank you!
Copyright 2010 ITRI工業技術研究院
26