Software Defined Networking in
Apache CloudStack
Chiradeep Vittal
CloudStack Committer
@chiradeep
Agenda
•
•
•
•
•
•
•
•
Introduction to CloudStack and IAAS
What is SDN
Why SDN and IAAS?
CloudStack’s Network Model
Extensible Networking in CloudStack
SDN integrations in CloudStack
CloudStack’s native SDN approach
Future
Apache CloudStack
• History
• Incubating in the Apache
Software Foundation since
April 2012
Build your cloud the way the
world’s most successful
clouds are built
• Open Source since May
2010
• In production since 2009
• Tons of deployments, including
large-scale commercial ones
How did Amazon build its cloud?
Amazon eCommerce Platform
AWS API (EC2, S3, …)
Amazon Orchestration Software
Open Source Xen Hypervisor
Networking
Commodity
Servers
Commodity
Storage
How can YOU build a cloud?
AmazonOptional
eCommerce
PortalPlatform
AWS API (EC2,
S3, API
…)
CloudStack
or AWS
CloudStack
OrchestrationSoftware
Software
Amazon Orchestration
Hypervisor
(Xen/KVM/VMW/)
Open
Source
Xen Hypervisor
Networking
Servers
Storage
SDN Definition
• Separation of Control Plane from the hardware
performing the forwarding function
• Control plane is logically centralized
SDN Advantages
• Centralized control makes it easier to
configure, troubleshoot and maintain
• Eliminates ‘box’ mode of configuration
• Enables control at a high level
Related to SDN
• API layer over a collection of ‘boxes’
– API layer communicates with boxes using box-level
APIs / ssh / telnet
• OpenFlow
– Standard protocol for the centralized control plane to
talk to the forwarding elements.
• Tunnels / overlays
– SDN is valuable for virtual topologies
– Initial target of SDN implementation
Centralized control plane
API
Controller Cluster
MySQL/NoSQL
Openflow/ssh/netconf/other
Boxes
Defining Cloud Computing (IAAS)
• Agility
– Re-provision complex infrastructure topologies in
minutes, not days
• API
– Automate complex infrastructure tasks
• Virtualization
– Enables workload mobility and load sharing
• Multi-tenancy
– Share resources and costs
Defining Cloud Computing (IAAS)
• Scalability
– Ability to consume resources limited by budget, not by
infrastructure
• Elasticity
– Scale up and down on demand
– Reduce need to engineer for peak load
• Self-service
– No IT assistance
Cloud Networking Requirements
• Agile
– Complex networking topologies created by nonnetwork engineers
• API
– Language to talk with the network infrastructure layer
(not CLI)
• Virtualization
– Hypervisor-level switches work together with physical
infrastructure
Cloud Networking Requirements
• Scalability
– Usually means L3 in the physical infrastructure
• Elasticity
– Release resources when not in use
– Introduce new resources on demand
• Self-service
– Novices deploying, maintaining, troubleshooting
virtual networks
IAAS + SDN – made for each other
• SDN enables agility
– API to controller enables easy changes to
networks
• SDN works with virtualization / vSwitches
– Typical of most SDN controllers
• SDN controllers are designed for large scale
• SDN enables virtual networking
– The illusion of isolated networks on top of shared
physical infrastructure
SDN issues
• Discovery of virtual address -> physical
address mapping
– VxLAN = multicast
– GRE = programmed by control plane
– L3 isolation = no mapping, no discovery
SDN issues
• State maintenance
– Large number of endpoints + flows
– High arrival rate of new flows
– Needs fast and scalable storage and
processing
– Differentiator between vendors
SDN issues
• L4-L7
– Service insertion and orchestration
– How do endpoints get services such as
• Firewall
• Load balancers
• IDS/IPS
– Service levels and performance
– Service Chaining
Network Virtualization in IAAS
Tenant 1 Virtual Network 10.1.1.0/24
Gateway
address 10.1.1.1
Internet
Tenant
1 VM 1
10.1.1.2
Tenant
1 VM 2
10.1.1.3
Tenant
1 VM 3
Tenant
1 VM 4
10.1.1.4
10.1.1.5
Network Virtualization in IAAS
Tenant 1 Virtual Network 10.1.1.0/24
Public
Network
Internet
Public IP
address
65.37.141.11
65.37.141.36
Gateway
address 10.1.1.1
Tenant 1
Edge
Services
Appliance(s)
NAT
DHCP
FW
Tenant
1 VM 1
10.1.1.2
Tenant
1 VM 2
10.1.1.3
Tenant
1 VM 3
Tenant
1 VM 4
10.1.1.4
10.1.1.5
Network Virtualization in IAAS
Tenant 1 Virtual Network 10.1.1.0/24
Public
Network
Internet
Public IP
address
65.37.141.11
65.37.141.36
Gateway
address 10.1.1.1
Tenant 1
Tenant 1
Edge
Edge
Services
Services
Appliance(s)
NAT
Appliance(s)
DHCP
FW
Load
Balancing
VPN
Tenant
1 VM 1
10.1.1.2
Tenant
1 VM 2
10.1.1.3
Tenant
1 VM 3
Tenant
1 VM 4
10.1.1.4
10.1.1.5
Network Virtualization in IAAS
Tenant 1 Virtual Network 10.1.1.0/24
Public
Network
Public IP
address
65.37.141.11
65.37.141.36
Gateway
address 10.1.1.1
Tenant 1
Tenant 1
Edge
Edge
Services
Services
Appliance(s)
NAT
Appliance(s)
Internet
Tenant
1 VM 1
10.1.1.2
Tenant
1 VM 2
10.1.1.3
Tenant
1 VM 3
DHCP
FW
Load
Balancing
Tenant
1 VM 4
10.1.1.4
10.1.1.5
Tenant 2 Virtual Network 10.1.1.0/24
Public IP
address
65.37.141.24
65.37.141.80
Gateway
address
10.1.1.1
Tenant 2
Edge
Services
Appliance
VPN
NAT
DHCP
Tenant
2 VM 1
10.1.1.2
Tenant
2 VM 2
10.1.1.3
Tenant
2 VM 3
10.1.1.4
CloudStack Network Model
Public
Network
Tenant 1 Virtual Network
10.1.1.0/24
Public IP
Tenant
10.1.1.2
address
1 VM
Gateway
65.37.141.11
1
address
65.37.141.36
10.1.1.1
Tenant 1
Tenant 10.1.1.3
Tenant
1
Edge
1 VM
Edge
Services
2
Services
Appliance(s)
NAT
Appliance(s)
Tenant 10.1.1.4
DHCP
1 VM
FW
Load
3
Balancing
Tenant 10.1.1.5
1 VM
4
Tenant 2 Virtual Network
Public IP
10.1.1.0/24
Tenant
address
Gateway
10.1.1.2
•
Map
virtual
networks
to
physical
2 VM
65.37.141.24
address
1
65.37.141.80
10.1.1.1
infrastructure
2
Tenant
10.1.1.3
•Tenant
Define
and
provision
network
services in
Edge
2 VM
Services
2
virtual networks
VPN
•Appliance
Manage elasticity andTenant
scale of10.1.1.4
network
NAT
2 VM
DHC services
3
P
CloudStack Network Model:
Network Services
Network
Services
• L2
connectivity
• IPAM
• DNS
• Routing
• ACL
• Firewall
• NAT
• VPN
• LB
• IDS
• IPS
CloudStack Network Model:
Network Services
Network
Services
• L2
connectivity
• IPAM
• DNS
• Routing
• ACL
• Firewall
• NAT
• VPN
• LB
• IDS
• IPS
Service
Providers
 Virtual
appliances
 Hardware
firewalls
 LB
appliances
 SDN
controllers
 IDS /IPS
appliances
 VRF
 Hypervisor
CloudStack Network Model:
Network Services
Network
Services
• L2
connectivity
• IPAM
• DNS
• Routing
• ACL
• Firewall
• NAT
• VPN
• LB
• IDS
• IPS
Service
Providers
 Virtual
appliances
 Hardware
firewalls
 LB
appliances
 SDN
controllers
 IDS /IPS
appliances
 VRF
 Hypervisor
Network
Isolation
• No isolation
• VLAN
isolation
• Overlays
• L3 isolation
Service Catalog
• Cloud users are not exposed to the nature of the
service provider
• Cloud operator designs a service catalog and offers
them to end users.
– Gold = {LB + FW, using virtual appliances}
– Platinum = {LB + FW + VPN, using hardware
appliances}
– Silver = {FW using virtual appliances, 10Mbps}
Service Catalog examples
L2 network with software appliances
L2 network with hardware appliances
10.1.1.0/24
VLAN 100
10.1.1.0/24
VLAN 100
65.37.141.1
11
65.37.141.1
12
10.1.1.
2
CS
Virtual
Router
DHCP, DNS
NAT
Load
Balancing
VPN
VM 1
65.37.141.11
Juniper
1
SRX
Firewall
10.1.1.1
10.1.1.
3
10.1.1.4
10.1.1.5
VM 2
10.1.1.1
10.1.1.2
NAT,
VPN
10.1.1.3
65.37.141.11
2
Netscaler
Load
Balancer
VM 3
VM 4
VM 2
10.1.1.112
10.1.1.4
10.1.1.
5
CS
Upgrade
VM 1
DHCP Virtual
Router
, DNS
VM 3
VM 4
Multi-tier virtual networking
Internet
IPSec or SSL site-to-site VPN
Loadbalancer
(virtual or HW)
Customer
Premises
Virtual appliance/
Hardware Devices
MPLS VLAN
Network Services
• IPAM
• DNS
• LB [intra]
• S-2-S VPN
• Static Routes
• ACLs
• NAT, PF
• FW [ingress & egress]
App VM
1
Web VM
1
App VM
2
Web VM
2
Web VM
3
VLAN 2724
VLAN 353
DB VM
1
Web VM
4
Web subnet
10.1.1.0/24 VLAN
101
App subnet
10.1.2.0/24
DB Subnet
10.1.3.0/24
Orchestration
• Orchestration describes the automated
arrangement, coordination, and management of
complex computer systems, middleware and
services
– Wikipedia
CloudStack Architecture
Hypervisor
Hypervisor
Plugins
Plugins
Plugin
Framework
Orchestration Core
Network
Network
Plugins
Plugins
Allocator
Allocator
Plugins
Plugins
Storage Plugins
CloudStack Architecture
XenServer
•VMWare
•KVM
•OracleVM
•
Hypervisor
Hypervisor
Plugins
Plugins
Plugin
Framework
Orchestration Core
Nicira
•Netscaler
•Brocade
•MidoNet
•
Network
Network
Plugins
Plugins
Allocator
Allocator
Plugins
Plugins
Random
•Userconcentrated
•Intel TXT
•Affinity
•
CloudStack Orchestration
5
4
1
API
API
API
Plugin
Framew
ork
Hypervis
Hypervis
or Plugins
or Plugins
6
Orchestration Core
2
8
3
7
Network
Network
Plugins
Plugins
Allocator
Storage
Plugins
Plugins
9
Hypervisor
Hypervisor
Resource
Resource
Network
Network
Resource
Resource
Storage
Storage
Resource
Resource
Allocator
Allocator
Plugins
Plugins
Physical Resources
Orchestration steps can be executed in parallel or in sequence
CloudStack and SDN
5
4
1
API
API
API
Plugin
Framew
ork
Hypervis
Hypervis
or Plugins
or Plugins
6
Orchestration core
2
8
3
7
Network
Network
Plugins
Plugins
Allocator
Storage
Plugins
Plugins
9
Hypervisor
Hypervisor
Resource
Resource
Network
SDN
Resource
controller
Storage
Storage
Resource
Resource
Allocator
Allocator
Plugins
Plugins
Physical Resources
Network plugin is the glue that understands the SDN controller’s API
CloudStack SDN Integration
• Nicira NVP
– L2 (STT) isolation in 4.0
– Source NAT / Logical Router in 4.2
• BigSwitch
– VLAN isolation in 4.1
– VNS in 4.2
• Midokura
– L2-L4 network virtualization
– Coming in 4.2
• CloudStack Native
– Tech preview (since 4.0)
– Requires XenServer
VM Orchestration Example
Hypervisor
Hypervisor
Resource
Resource
Call Hypervisor APIs
Hypervisor
Hypervisor
Plugins
Plugins
Plugin
Framew
ork
AP
I AP
I
API
Network
Resource
SDN controller
Network
Network
Plugins
Plugins
Orchestration
core
Allocator
Storage
Plugins
Plugins
Start 3 VMs
Storage
Storage
Resource
Resource
Allocator
Allocator
Plugins
Plugins
Allocate
hypervisors
VM
1
VM
2
Host 1
Host 2
Host 3
VM
3
VR
Host 4
Built-in (native) controller
Create Full Mesh of GRE
tunnels (if they don't
already exist) between
hosts on which VMs are
deployed
CloudStack
SDN
Controller
Host 1 (Pod 2)
OVS
Host 3 (Pod 3)
CloudStack SDN
controller programs the
Open vSwitch (OVS) on
XenServer to configure
GRE tunnels
VM
1
GRE Tunnel
Host 2 (Pod 4)
VM
2
GRE Tunnel
OVS
Host 4 (Pod 2)
VM
3
OVS
VR
GRE Tunnel
Built-in controller
Assign
'Tenant' key
for isolation
Tenant1
Tenant2
Host 1
VM
1
New tenants
can share the
established
GRE tunnels
with separate
tenant keys
Host 3
VM
1
VM
3
VR
GRE Tunnel
Host 2
VM
2
GRE Tunnel
Host 4
VM
2
VM
3
VR
GRE Tunnel
What makes it different
• Purpose built for IAAS
– Not general purpose SDN solution
• Proactive model
– Deny all flows except the ones programmed by the
end-user API
– Scaling problem is manageable
• Part of CloudStack
– ASF project
• Uses Virtual Router to provide L3-L7 network
services
– Could change
Futures
• AWS VPC semantics
– Support security groups, ACL
• Optimize ARP & DHCP responses
• Cross-zone networks
– Optimize inter-subnet routing