Hypervisor Selection in CloudStack 4.3
and OpenStack Havana
Understanding the choices available
virtg Deep Dive Day 2014
Tim Mackey – XenServer Community Manager and Evangelist
Building a successful cloud
What are we trying to accomplish?
Service Offerings
• Clearly define what you want to offer
ᵒ What types of applications
ᵒ Who has access, and who owns them
ᵒ What type of access
• Define how templates need to be managed
ᵒ Operating system support
ᵒ Patching requirements
• Define expectations around compliance and availability
ᵒ Who owns backup and monitoring
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Define Tenancy Requirements
• Department data local to department
ᵒ Where is the application data stored
• Data and service isolation
ᵒ VM migration and host HA
ᵒ Network services
• Encryption of PII/PCI
ᵒ Where do keys live when data location unknown
ᵒ Need encryption designed for the cloud
• Showback to stakeholders
ᵒ More than just usage, compliance and audits
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Virtualization Infrastructure
• Hypervisor defined by service offerings
ᵒ
ᵒ
ᵒ
ᵒ
Don’t select hypervisor based on “standards”
Understand true costs of virtualization
Multiple hypervisors are “OK”
Bare metal can be a hypervisor
• To “Pool” resources or not
ᵒ Is there a real requirement for pooled resources
ᵒ Can the cloud management solution do better?
ᵒ Real cost of shared storage
• Primary storage defined by hypervisor
• Template storage defined by solution
ᵒ Typically low cost options like NFS
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Choice is a good thing….
Apache CloudStack
Management
Server
• Current release: 4.2.1 (4.3 imminent)
• Highly scalable
• Monolithic architecture
• Mostly written in Java
• Multi-hypervisor support
ᵒ XenServer, KVM, OracleVM, vSphere,
Linux Containers, Bare metal
ᵒ 4.3 adds: Hyper-V
• Strong backing from Citrix,
CloudOps, Shapeblue and others
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Load
Balancer
Management
Server
Management
Server
MySQL
DB
Replication
Back Up
DB
Infrastructure
Resources
OpenStack
• Current release: Havana
• Scalable in 500 node blocks
• Distributed architecture
• Mostly written in Python
• Multi-hypervisor support
ᵒ Group A: KVM
ᵒ Group B: Hyper-V, vSphere, XenServer
ᵒ Group C: All others deprecated in Icehouse
• Strong backing from HP, IBM, RAX, RHT,
Canonical, Mirantis, Piston Cloud,
Cloudscaling
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Leading hypervisor options
XenServer 6.2
Feature
Source code model
Maximum VM Density
CloudStack VM Density
CloudStack integration
OpenStack driver
Maximum native cluster size
Maximum pRAM
Largest VM
Windows Operating System
Linux Operating Systems
Advanced features supported (CloudStack)
Advanced features supported (OpenStack)
Open Source (GPLv2)
650 (Linux)
500
Direct XAPI calls
OpenStack nova-compute domU
16
1 TB
16 vCPU/128GB
All Windows supported by Microsoft
RHEL, CentOS, Debian, Ubuntu, SLES, OEL
ovs, Storage XenMotion, DMC
ovs, Storage XenMotion
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
vSphere 5.5
Feature
Source code model
Maximum VM Density
CloudStack VM Density
CloudStack integration
OpenStack driver
Maximum native cluster size
Maximum pRAM
Largest VM
Windows Operating Systems
Linux Operating Systems
Advanced features supported (CloudStack)
Advanced features supported (OpenStack)
Proprietary
512
128
vCenter
vCenter – nova-compute node per cluster
32
4 TB
64 vCPU/1TB
DOS, All Windows Server/Client
Most
HA, DRS, vDS, Storage vMotion
HA, DRS, vMotion
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
KVM
Feature
Source code model
Maximum VM Density
CloudStack VM Density
CloudStack integration
OpenStack driver
Maximum native cluster size
Maximum pRAM
Largest VM
Windows Operating Systems
Linux Operating Systems
Advanced features supported (CloudStack)
Advanced features supported (OpenStack)
Open Source (GPLv2)
10 times the number of pCores
50
CloudStack Agent (libvirt)
libvirt driver
No native cluster support
2 TB
160 vCPU/2TB
Windows XP and higher
Varies
None
None
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Microsoft Hyper-V
Feature
Source code model
Maximum VM Density
CloudStack VM Density
CloudStack integration
OpenStack driver
Maximum native cluster Size
Maximum pRAM
Largest VM
Windows Operating Systems
Linux Operating Systems
Advanced features supported (CloudStack)
Advanced features supported (OpenStack)
Proprietary
1024
1024
CloudStack Agent (C# calling WMI)
Use Cloudbase driver
64
4 TB
64 vCPU/1TB
All Windows supported by Microsoft
RHEL, CentOS, Debian, Ubuntu, SLES, OEL
None
None
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
The CloudStack 4.3 world …
Defining the network
Flat Network – Basic Layer 3 Network
Option
XenServer
vSphere
KVM
Hyper-V
Security Groups
Yes- bridge
No
Yes
Yes
IPv6
No
No
Yes
No
Multiple IPs per NIC Yes
Yes
Yes
Yes
Nicira NVP
Yes
No
Yes
No
BigSwitch VNS
Yes
No
Yes
No
Public Network
65.11.0.0/16
Security Group 1
65.11.1.2
65.11.1.3
65.11.1.4
65.11.1.5
DHCP,
DNS
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
CloudStack
Virtual Router
Guest VM 1
Guest VM 2
Guest VM 3
Guest VM 4
Security Group 2
VLANs for Private Cloud
Option
XenServer
vSphere
KVM
Hyper-V
Max VLANs
800
254
1024
4094
IPv6
No
No
Yes
No
Multiple IPs per
NIC
Nicira NVP
Yes
Yes
Yes
Yes
Yes
No
Yes
No
BigSwitch VNS
Yes
No
Yes
No
MidoKura
No
No
Yes
No
VPC
Yes
Yes
Yes
Yes
NetScaler
Yes
Yes
Yes
Yes
F5 BigIP
Yes
Yes
Yes
Yes
Juniper SRX
No
Yes
Yes
Yes
Cisco VNMC
No
Yes
No
No
Guest Virtual Network 10.0.0.0/8
VLAN 100
Public
Network/Internet
Public IP
65.37.14.1
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
CloudStack
Virtual
Router
DHCP, DNS
NAT
Load Balancing
VPN
Gateway
10.1.1.1
10.1.1.1
10.1.1.3
10.1.1.4
10.1.1.5
Guest VM 1
Guest VM 2
Guest VM 3
Guest VM 4
Beyond the VLAN – Network Virtualization
Option
XenServer
vSphere
KVM
Hyper-V
OVS GRE tunnels
Yes
No
No
No
Nicira STT tunnel
Yes
Yes
Yes
No
MidoNet
No
No
Yes
No
VXLAN
No
Yes
Yes
No
NVGRE
No
No
No
No
Nexus 1000v
No
Yes
No
No
Juniper Contrail
Yes
No
No
No
Palo Alto
Yes
Yes
Yes
No
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Virtual Private Cloud and nTier Applications
Feature
PVLAN
XenServer
Yes - ovs
vSphere
Yes
KVM
ovs
DC2
Hyper-V
No
DC1
DC3
VLAN 1
DC4
Web
S2S VPN
Router
VLAN 2
App
Private
GW
VLAN 3
DC5
DC6
DB
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Delivering specific network services
• KVM
ᵒ IPv6
ᵒ Security groups
ᵒ Large quantity of VLANs
• vSphere
ᵒ VXLAN required vSphere Enterprise Plus
ᵒ Cisco Nexus 1000v and ASA 1000v require vSphere Enterprise Plus
• XenServer
ᵒ Security groups
ᵒ Large quantity of VLANs
ᵒ Juniper Contrail
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Instances need a home …
Storage, Storage and more Storage
Primary Storage Options
Feature
Local storage
NFS
SMB
Single path iSCSI
Multipath iSCSI
Direct array
Shared Mount
Template format
SolidFire Plugin
NetApp Plugin
Zone wide
Ceph RBD
Clustered LVM
XenServer
Yes
Yes
No
Yes
PreSetup
No
No
VHD
Yes
Yes
No
No
No
vSphere
Yes
Yes
No
Yes
No
VAAI
No
OVA
Yes
Yes
Yes
No
No
KVM
Yes
Yes
No
Yes
No
No
Yes
QCOW2
Yes
Yes
Yes
Yes
Yes
Hyper-V
Yes
No
SMB3
No
No
No
No
VHD
No
No
No
No
No
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Host
Host
Primary Storage
Cluster
Secondary Storage Options
Option
XenServer
vSphere
KVM
Hyper-V
NFS
Yes
Yes
Yes
No
Swift(1)
Yes
Yes
Yes
No
S3 compatible (2)
Yes
Yes
Yes
No
SMB
No
No
No
Yes
(1) Requires NFS staging area
(2) Can be region wide, but must not have NFS secondary storage in zone
Host
Host
Primary Storage
Cluster
Pod
Secondary Storage
Zone
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Core virtualization capabilities
The limits and features which matter
CloudStack Features
Feature
XenServer
vSphere
KVM
Hyper-V
Disk IO Statistics
Yes
No
Yes
Yes
Memory Overcommit
Yes (4x)
Yes
No
No
Dedicated resources
Yes
Not with HA/DRS
Yes
Yes
Disk IO throttling
No
No
Yes
No
Disk snapshot (running)
Yes
Yes
No
No
Disk snapshot (pluggable)
Partial
Partial
No
No
Disk snapshot (Stopped)
Yes
Yes
Yes
Yes
Memory snapshot
Yes
Yes
Yes
No
Zone wide primary storage
No
Yes
Yes
No
Resize disk
Offline
Online Grow
Online
No
High availability
CloudStack
Native
CloudStack
CloudStack
CPU sockets
6.2 and higher
Yes
Yes
Yes
Affinity groups
Yes
Yes
Yes
Yes
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Multiple Hypervisor Support in CloudStack
• Networking
ᵒ Ensure network labels match
ᵒ Topology is intersect of chosen hypervisors
• Storage
ᵒ For system VMs to specific hypervisor type
ᵒ Zone with primary storage limited
• Operations
ᵒ
ᵒ
ᵒ
ᵒ
vSphere Datacenter can not span zones
Hyper-V may not be mixed with other hypervisors
HA won’t migrate between hypervisors
Capacity planning at the cluster/pod level more difficult
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
The OpenStack Havana world …
Defining the network
The goodness of Neutron (Quantum)
Flat Network – Basic Layer 3 Network
Option
XenServer
vSphere
KVM
Hyper-V
Security Groups
Yes
Yes-NVP
Yes
No
IPv6
No
No
No
No
Multiple IPs per NIC Yes
Yes
Yes
Yes
Nicira NVP
Yes
Yes
Yes
No
Firewall rules
Yes
No
Yes
No
Routing
Yes
No
Yes
No
Public Network
65.11.0.0/16
Security Group 1
65.11.1.2
65.11.1.3
65.11.1.4
65.11.1.5
DHCP,
DNS
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
CloudStack
Virtual Router
Guest VM 1
Guest VM 2
Guest VM 3
Guest VM 4
Security Group 2
VLANs for Private Cloud
Option
XenServer
vSphere
KVM
Hyper-V
Max VLANs
800
254
1024
4094
IPv6
No
No
No
No
Multiple IPs
Yes
Yes
Yes
Yes
Guest Virtual Network 10.0.0.0/8
VLAN 100
Public
Network/Internet
Public IP
65.37.14.1
CloudStack
Virtual
Router
DHCP, DNS
NAT
Load Balancing
VPN
Gateway
10.1.1.1
10.1.1.1
10.1.1.3
10.1.1.4
10.1.1.5
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Guest VM 1
Guest VM 2
Guest VM 3
Guest VM 4
Beyond the VLAN – Network Virtualization
Option
OVS GRE tunnels
Nicira STT tunnel
VXLAN
NVGRE
Nexus 1000v
Nicira NVP
BigSwitch VNS
MidoKura
Brocade
Plumgrid
Ryu
NEC
Cisco VNMC
XenServer
Yes
Yes
No
No
No
Yes
No
No
No
No
No
No
No
vSphere
No
Yes
Yes
No
Yes
Yes
No
No
No
No
No
No
No
KVM
No
Yes
Yes
No
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Hyper-V
No
No
No
No
No
No
No
No
No
No
No
No
No
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Instances need a home …
Storage, Storage and more Storage
Instance Storage Options – Nova and Cinder
Feature
Template format
Local storage
NFS
Fiber HBA
iSCSI
iSCSI CHAP
Ceph
Gluster
ZFS
XenServer
VHD
Yes
Yes
No
Yes
Yes
No
No
No
vSphere
VMDK
Yes
Yes
No
Yes
No
No
No
No
KVM
QCOW2
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Hyper-V
VHD
Yes
Yes
No
Yes
No
No
No
No
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
OpenStack Features
Feature
XenServer
vSphere
KVM
Hyper-V
Pause VM
Yes
No
Yes
Yes
Inject networking
Guest agent
Yes
Flat (Debian)
Yes
Inject file
Yes
No
Yes
cloudbase-init
Serial console
Yes
Yes
Yes
No
VNC consoles
Yes
Yes
Yes
Yes
SPICE console
No
No
Yes
No
Snapshot
Yes
Yes
Yes
Yes
Set administrator password
Yes
No
No
cloudbase-init
Auto configure disk
Yes
No
No
No
Evacuate host
Yes
No
Yes
No
Volume swap
No
No
Yes
No
Volume rate limiting
No
No
Yes
No
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Multiple Hypervisor Support in OpenStack
• Capabilities
ᵒ
ᵒ
ᵒ
ᵒ
Multiple hypervisor support varies by distro
Most deployments are single hypervisor
Difficult to schedule instances to compute nodes
Use host aggregates
• Networking
ᵒ Topology is intersect of chosen hypervisors – ML2 helps
• Operations
ᵒ HA won’t migrate between hypervisors
ᵒ Capacity planning at the cluster/pod level more difficult
ᵒ Hyper-V does not work with all Neutron plugins
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Picking the “best one”
When to use which hypervisor…
KVM
• Primary value proposition:
ᵒ Low cost with available vendor support
ᵒ Familiar administration model
ᵒ Broad feature set with active development in both CloudStack and OpenStack
• Cloud use cases:
ᵒ Linux centric workloads
ᵒ Dev/test clouds
ᵒ Web hosting
ᵒ Tenant density which dictates SDN options
• Weaknesses:
ᵒ CloudStack: Requires use of an installed libvirt agent
ᵒ Limited native storage options
ᵒ No use of advanced native features
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Microsoft Hyper-V
• Primary value proposition:
ᵒ Unlimited Windows Server VM licenses
ᵒ Familiar Windows management paradigm
• Cloud use cases:
ᵒ Windows and Linux workloads
ᵒ Dev/test clouds
ᵒ .Net application web hosting
ᵒ Desktop as a Service clouds
• Weaknesses:
ᵒ Minimal use of advanced native features
ᵒ CloudStack: First introduced with CloudStack 4.3 (not yet released)
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
vSphere
• Primary value proposition:
ᵒ Broad application and operating system support
ᵒ Readily available pool of vSphere administration talent
ᵒ Large eco-system of vendor partners
ᵒ CloudStack: Many features are native implementations
ᵒ Direct feature integration via vCenter
• Cloud use cases:
ᵒ Private enterprise clouds
ᵒ Dev/test clouds
• Weaknesses:
ᵒ vSphere up-front license and ongoing support costs
ᵒ vCenter integration requires redundant designs
ᵒ CloudStack: Single data center per zone model
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
XenServer
• Primary value proposition:
ᵒ Low cost with available vendor support
ᵒ Broad feature set with active development in both CloudStack and OpenStack
ᵒ CloudStack: Large install base
ᵒ Direct integration via XAPI toolstack
• Cloud use cases:
ᵒ Linux centric workloads
ᵒ Dev/test clouds
ᵒ Web hosting
ᵒ Desktop as a Service clouds
ᵒ Large VM density and secure tenant isolation
• Weaknesses:
ᵒ Minimal use of advanced native features
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Tying it all Together
1. Define success criteria
2. Select a topology which works
3. Decide on storage options
4. Define supported configurations
5. Select preferred hypervisor(s)
6. Validate matrix
7. Build your Cloud
© Citrix 2014. More information at xenserver.org and follow me on twitter @XenServerArmy
Work better. Live better.