OpenFlow Hands-on Tutorial
OFELIA Meeting
Feb 7, 2011 at Berlin
Srini Seetharaman, DT R&D Lab, USA
With help from:
Dan Levin, TU Berlin
Nadi Sarrar, TU Berlin
Andi Wundsam, TU Berlin
1
Welcome! Getting Started
• Copy to your hard disk from Flash Drive/:
– Corresponding files (terminal, X server, Vmware) for your platform
(Win/Mac/Linux)
– OpenFlowTutorial-20110131.vmwarevm.zip
• unzip OpenFlowTutorial-20110131.vmwarevm.zip
• Instructions:
http://www.openflow.org/wk/index.php/OpenFlowTutoria
lOFELIA2011
• VMWare Key (if needed):
– 4J09J-3U2E5-58C32-08AK0-8DH42
• Online attendees: Download file from:
• http://www.net.t-labs.tuberlin.de/research/routerlab/oftutorial.zip
2
Introductions
3
Agenda
Time
Description
10:00-11:00
Introduction: How OpenFlow works, Potential,
Limitations, Current vendors, SDN/OPEN
11:00-12:15
Begin Hands-on Portion (learn tools, self-exploration)
Lunch break
13:00-14:00
Intro to Slicing, Details about GENI deployments
14:00-14:30
Continue Hands-on Portion (Slice using FlowVisor, use
Expedient)
Coffee break
15:00-15:45
Vision and long-term directions of OpenFlow
(Nick McKeown)
15:45-16:30
Q&A; Discussions
4
Goals of this Tutorial
• By the end, everyone should know:
–
–
–
–
–
what OpenFlow is
how it’s used and how you can use it
where it’s going
how OpenFlow compares to other platforms
how OpenFlow fits in the Software-Defined
Networking (SDN) spectrum
• Present a useful mix of hands-on and lecturebased content
• Have fun
5
Why OpenFlow?
6
The Ossified Network
Routing, management, mobility management,
access control, VPNs, …
Feature
Feature
Operating
System
Specialized Packet
Forwarding Hardware
Million of lines
of source code
5400 RFCs
Barrier to entry
Billions of gates
Bloated
Power Hungry
Many complex functions baked into the infrastructure
OSPF, BGP, multicast, differentiated services,
Traffic Engineering, NAT, firewalls, MPLS, redundant layers, …
An industry with a “mainframe-mentality”, reluctant to change
7
Research Stagnation
• Lots of deployed innovation in other areas
– OS: filesystems, schedulers, virtualization
– DS: DHTs, CDNs, MapReduce
– Compilers: JITs, vectorization
• Networks are largely the same as years ago
– Ethernet, IP, WiFi
• Rate of change of the network seems slower in
comparison
– Need better tools and abstractions to demonstrate
and deploy
8
Closed Systems (Vendor Hardware)
•
•
•
•
Stuck with interfaces (CLI, SNMP, etc)
Hard to meaningfully collaborate
Vendors starting to open up, but not usefully
Need a fully open system – a Linux equivalent
9
Open Systems
Performance Scale
Fidelity
Real User
Traffic?
Complexity
Open
Simulation
medium
medium
no
medium
yes
Emulation
medium
low
no
medium
yes
Software
Switches
poor
low
yes
medium
yes
NetFPGA
high
low
yes
high
yes
Network
Processors
high
medium
yes
high
yes
Vendor
Switches
high
high
yes
low
no
gap in the tool space
none have all the desired attributes!
10
OpenFlow: a pragmatic compromise
• + Speed, scale, fidelity of vendor hardware
• + Flexibility and control of software and
simulation
• Vendors don’t need to expose implementation
• Leverages hardware inside most switches
today (ACL tables)
12
How does OpenFlow work?
13
Ethernet Switch
14
Control Path (Software)
Data Path (Hardware)
15
OpenFlow Controller
OpenFlow Protocol (SSL/TCP)
Control Path
OpenFlow
Data Path (Hardware)
16
OpenFlow Example
Software
Layer
Controller
PC
OpenFlow Client
Flow Table
Hardware
Layer
MAC
src
MAC
dst
IP
Src
IP
Dst
TCP
TCP
Action
sport dport
*
*
*
5.6.7.8
*
port 1
5.6.7.8
port 2
*
port 3
port 1
port 4
1.2.3.4
17
OpenFlow Basics
Flow Table Entries
Rule
Action
Stats
Packet + byte counters
1.
2.
3.
4.
5.
Switch VLAN
Port
ID
Forward packet to zero or more ports
Encapsulate and forward to controller
Send to normal processing pipeline
Modify Fields
Any extensions you add!
VLAN MAC
pcp src
MAC
dst
Eth
type
IP
Src
IP
Dst
IP
L4
IP
ToS Prot sport
L4
dport
+ mask what fields to match
18
Examples
Switching
Switch MAC
Port src
*
MAC Eth
dst
type
00:1f:.. *
*
VLAN IP
ID
Src
IP
Dst
IP
Prot
TCP
TCP
Action
sport dport
*
*
*
*
IP
Dst
IP
Prot
TCP
TCP
Action
sport dport
*
*
port6
Flow Switching
Switch MAC
Port src
MAC Eth
dst
type
port3 00:20.. 00:1f.. 0800
VLAN IP
ID
Src
vlan1 1.2.3.4 5.6.7.8
4
17264 80
port6
Firewall
Switch MAC
Port src
*
*
MAC Eth
dst
type
*
*
VLAN IP
ID
Src
IP
Dst
IP
Prot
TCP
TCP
Action
sport dport
*
*
*
*
*
22
drop
19
Examples
Routing
Switch MAC
Port src
*
*
MAC Eth
dst
type
*
*
VLAN IP
ID
Src
IP
Dst
*
5.6.7.8 *
*
VLAN IP
ID
Src
IP
Dst
IP
Prot
vlan1 *
*
*
TCP
TCP
Action
sport dport
port6,
port7,
*
*
port9
*
IP
Prot
TCP
TCP
Action
sport dport
*
port6
VLAN Switching
Switch MAC
Port src
*
*
MAC Eth
dst
type
00:1f.. *
20
Centralized vs Distributed Control
Both models are possible with OpenFlow
Centralized Control
Controller
OpenFlow
Switch
Distributed Control
Controller
OpenFlow
Switch
Controller
OpenFlow
Switch
OpenFlow
Switch
OpenFlow
Switch
Controller
OpenFlow
Switch
21
Flow Routing vs. Aggregation
Both models are possible with OpenFlow
Aggregated
Flow-Based
•
•
•
•
Every flow is individually
set up by controller
Exact-match flow entries
Flow table contains one
entry per flow
Good for fine grain
control, e.g. campus
networks
•
•
•
•
One flow entry covers large
groups of flows
Wildcard flow entries
Flow table contains one
entry per category of flows
Good for large number of
flows, e.g. backbone
22
Reactive vs. Proactive (pre-populated)
Both models are possible with OpenFlow
Reactive
Proactive
•
•
•
•
•
First packet of flow
triggers controller to insert
flow entries
Efficient use of flow table
Every flow incurs small
additional flow setup time
If control connection lost,
switch has limited utility
•
•
•
Controller pre-populates
flow table in switch
Zero additional flow setup
time
Loss of control connection
does not disrupt traffic
Essentially requires
aggregated (wildcard) rules
23
Usage examples
• Alice’s code:
– Simple learning switch
– Per Flow switching
– Network access
control/firewall
– Static “VLANs”
– Her own new routing protocol:
unicast, multicast, multipath
– Home network manager
– Packet processor (in
controller)
– IPvAlice
–
–
–
–
–
VM migration
Server Load balancing
Mobility manager
Power management
Network monitoring
and visualization
– Network debugging
– Network slicing
… and much more you can create!
Quiz Time
• How do I provide control connectivity? Is it really clean slate?
• Why aren’t users complaining about time to setup flows over
OpenFlow? (Hint: What is the predominant traffic today?)
• Considering switch CPU is the major limit, how can one take
down an OpenFlow network?
• How to perform topology discovery over OpenFlow-enabled
switches?
• What happens when you have a non-OpenFlow switch
inbetween?
• What if there are two islands connected to same controller?
• How scalable is OpenFlow? How does one scale deployments?
25
What can you not do with OpenFlow ver1.0
• Non-flow-based (per-packet) networking
– ex. Per-packet next-hop selection (in wireless mesh)
– yes, this is a fundamental limitation
– BUT OpenFlow can provide the plumbing to connect these
systems
• Use all tables on switch chips
– yes, a major limitation (cross-product issue)
– BUT an upcoming OF version will expose these
26
What can you not do with OpenFlow ver1.0
• New forwarding primitives
– BUT provides a nice way to integrate them through
extensions
• New packet formats/field definitions
– BUT a generalized OpenFlow (2.0) is on the horizon
• Optical Circuits
– BUT efforts underway to apply OpenFlow model to circuits
• Low-setup-time individual flows
– BUT can push down flows proactively to avoid delays
Where it’s going
• OF v1.1: Extensions for WAN, spring 2011
– multiple tables: leverage additional tables
– tags and tunnels
– multipath forwarding
• OF v2+
– generalized matching and actions: an “instruction
set” for networking
28
OpenFlow Implementations
(Switch and Controller)
29
OpenFlow building blocks
oftrace
oflops
Monitoring/
debugging tools
openseer
Stanford Provided
ENVI (GUI)
NOX
LAVI
Beacon
FlowVisor
Console
n-Casting
Helios
Applications
SNAC
Controller
Maestro
Slicing
Software
FlowVisor
Stanford Provided
Commercial Switches
HP, NEC, Pronto,
Juniper.. and many
more
Expedient
Software
Ref. Switch
NetFPGA
Broadcom
Ref. Switch
OpenWRT
PCEngine
WiFi AP
OpenVSwitch
OpenFlow
Switches
30
Current SDN hardware
Juniper MX-series
NEC IP8800
WiMax (NEC)
HP Procurve 5400
Netgear 7324
PC Engines
Pronto 3240/3290
Ciena Coredirector
More coming soon...
31
Commercial Switch Vendors
Model
Virtualize
Notes
HP Procurve 5400zl or
6600
1 OF
instance
per VLAN
-LACP, VLAN and STP processing
before OpenFlow
-Wildcard rules or non-IP pkts
processed in s/w
-Header rewriting in s/w
-CPU protects mgmt during loop
NEC IP8800
1 OF
instance
per VLAN
-OpenFlow takes precedence
-Most actions processed in
hardware
-MAC header rewriting in h/w
Pronto 3240 or 3290
with Pica8 or Indigo
firmware
1 OF
instance
per switch
-No legacy protocols (like VLAN
and STP)
-Most actions processed in
hardware
-MAC header rewriting in h/w
32
Controller Vendors
Vendor
Notes
Vendor
Notes
Nicira’s
NOX
•Open-source GPL
•C++ and Python
•Researcher friendly
Stanford’s
Beacon
•Open-source
•Researcher friendly
•Java-based
Nicira’s
ONIX
•Closed-source
•Datacenter networks
BigSwitch
controller
•Closed source
•Based on Beacon
•Enterprise network
SNAC
•Open-source GPL
•Code based on NOX0.4
•Enterprise network
•C++, Python and Javascript
•Currently used by campuses
Maestro (from
Rice Univ)
•Open-source
•Based on Java
NEC’s Helios
•Open-source
•Written in C
33
Growing Community
Vendors and start-ups
More...
Providers and business-unit
More...
Note: Level of interest varies
34
Software-Defined
Networking (SDN)
35
Current Internet
Closed to Innovations in the Infrastructure
Closed
Ap
p
Ap
p
Ap
p
Operating
System
Ap
p
Specialized Packet
Forwarding Hardware
Ap
p
Ap
p
Ap
p
Ap
p
Operating
System
Ap
p
Specialized Packet
Forwarding Hardware
Operating
System
Ap
p
Specialized Packet
Forwarding Hardware
Ap
p
Ap
p
Operating
System
Ap
p
Ap
p
Ap
p
Specialized Packet
Forwarding Hardware
Operating
System
Specialized Packet
Forwarding Hardware
36
“Software Defined Networking” approach
to open it
App
App
App
Network Operating System
Ap
p
Ap
p
Ap
p
Operating
System
Ap
p
Specialized Packet
Forwarding Hardware
Ap
p
Ap
p
Ap
p
Ap
p
Operating
System
Ap
p
Specialized Packet
Forwarding Hardware
Operating
System
Ap
p
Specialized Packet
Forwarding Hardware
Ap
p
Ap
p
Operating
System
Ap
p
Ap
p
Ap
p
Specialized Packet
Forwarding Hardware
Operating
System
Specialized Packet
Forwarding Hardware
37
The “Software-defined Network”
2. At least one good operating system
Extensible, possibly open-source
3. Well-defined open API
App
App
App
Network Operating System
1. Open interface to hardware
Simple Packet
Forwarding
Hardware
Simple Packet
Forwarding
Hardware
Simple Packet
Forwarding
Hardware
Simple Packet
Forwarding
Hardware
Simple Packet
Forwarding
Hardware
38
Hands-on Session Flow
Resume talk
Resume lecture
setup VM,
play with mininet,
use tools
Run hub, turn
hub into
an Ethernet
switch
Slicing the
network
with FlowVisor
and Expedient
On your own
add multiple
switch
support
modify
switch to
handle IP
forwarding
39
Stuff you’ll use
•
•
•
•
•
•
NOX
Reference Controller/Switch
OpenvSwitch
Mininet
cbench
flowvisor
• iperf
• tcpdump
• Wireshark
40
Tutorial Setup
c0
Controller
port6633
loopback
(127.0.0.1:6633)
OpenFlow Tutorial
3hosts-1switch
topology
s1
OpenFlow Switch
s1-eth0
h1-eth0
s1-eth1
h3-eth0
loopback
(127.0.0.1:6634)
s1-eth2
h4-eth0
h2
h3
h4
10.0.0.2
10.0.0.3
10.0.0.4
virtual hosts
dpctl
(user space
process)
41
Hands-on Tutorial
Next presentation starts at 13:00
www.openflow.org/wk/index.php/OpenFlowTutorialOFELIA2011
•Hope you got the files from the DVD or USB keys
•VMWare Key (if needed):
–4J09J-3U2E5-58C32-08AK0-8DH42
42
Virtualizing OpenFlow
43
Trend
App
App
App
Windows
Windows
Windows
(OS)
(OS)
(OS)
Linux
Linux
Linux
App
App
App
Mac
Mac
Mac
OS
OS
OS
Virtualization layer
x86
(Computer)
Computer Industry
Controller11
NOX
Controller
(Network OS)
Controller
Controller
Network
OS
22
Virtualization or “Slicing”
OpenFlow
Network Industry
Isolated “slices”
App
App
Network
Operating
System 1
Many operating systems, or
Many versions
App
App
Network
Operating
System 2
App
App
App
Network
Operating
System 3
App
Network
Operating
System 4
Open interface to hardware
Virtualization or “Slicing” Layer
Open interface to hardware
Simple Packet
Forwarding Hardware
Simple Packet
Forwarding Hardware
Simple Packet
Forwarding Hardware
Simple Packet
Forwarding Hardware
Simple Packet
Forwarding Hardware
45
Switch Based Virtualization
Exists for NEC, HP switches but not flexible enough
Research VLAN 2
Flow Table
Controller
Research VLAN 1
Flow Table
Controller
Production VLANs
Normal L2/L3 Processing
46
FlowVisor-based Virtualization
Heidi’s
Controller
Aaron’s
Controller
Craig’s
Controller
Topology
discovery is
per slice
OpenFlow
Protocol
OpenFlow
Switch
OpenFlow FlowVisor
& Policy Control
OpenFlow
Protocol
OpenFlow
Switch
OpenFlow
Switch
47
FlowVisor-based Virtualization
Separation not only
by VLANs, but any
L1-L4 pattern
Multicast
Broadcast
OpenFlow
Protocol
dl_dst=FFFFFFFFFFFF
OpenFlow
Switch
http
Load-balancer
tp_src=80, or
tp_dst=80
OpenFlow
FlowVisor & Policy Control
OpenFlow
Protocol
OpenFlow
Switch
OpenFlow
Switch
48
FlowSpace: Maps Packets to Slices
FlowVisor Message Handling
Alice
Controller
Bob
Controller
Cathy
Controller
OpenFlow
Policy Check:
Is this rule
allowed?
Policy Check:
Who controls
this packet?
FlowVisor
OpenFlow
Full Line Rate
Forwarding
Packet
Packet
OpenFlow
Firmware
Data Path
Rule
Exception
Use Case: New CDN - Turbo Coral ++
Basic Idea: Build a CDN where you control the entire network
–
–
–
–
All traffic to or from Coral IP space controlled by Experimenter
All other traffic controlled by default routing
Topology is entire network
End hosts are automatically added (no opt-in)
Switch MAC
Port src
MAC Eth
dst
type
VLAN IP
ID
Src
IP
Dst
IP
Prot
TCP
TCP
sport dport
*
*
*
*
*
*
*
*
84.65.* *
*
*
*
*
*
*
84.65.* *
*
*
*
*
*
*
*
*
*
*
*
*
51
Use Case: Aaron’s IP
• A new layer 3 protocol
• Replaces IP
• Defined by a new Ether Type
Switch MAC
Port src
MAC Eth
dst
type
VLAN IP
ID
Src
IP
Dst
IP
Prot
TCP
TCP
sport dport
*
*
*
AaIP
*
*
*
*
*
*
*
*
*
!AaIP *
*
*
*
*
*
52
OpenFlow/GENI
Deployments
53
OpenFlow Deployment at Stanford
Switches (23)
APs (50)
WiMax (1)
54
Live Stanford
Deployment Statistics
http://yuba.stanford.edu/ofhallway/wide-ofv1.html
55
How did we get there?
Staged Deployment of OpenFlow
1.
2.
3.
Add experimental VLAN
Enable OpenFlow for Exptl VLAN
Configure Controller for new network
Verify correctness and performance
4.
5.
Add new Production subnetwork
Gradually add/move users to new subnet
Verify reachability
6.
7.
Enable OpenFlow for new subnet
Slice the network
GENI OpenFlow deployment (2010)
10 institutions and 2 National Research Backbones
Kansas State
57
GENI Network Evolution
National
Lambda
Rail
GENI Integration
• FlowVisor
Expedient3
– Slicing control
• Expedient
– Experimenter’s portal
for slice management
GENI API
API X
Expedient1
API X
Opt-in Mgr1
Expedient2
API X
Opt-in Mgr2
FlowVisor API
FlowVisor API
• Opt-in Manager
– Network admins’ portal
to approve/ deny expt
requests for traffic
FlowVisor1
OpenFlow
Substrate
1
FlowVisor2
OpenFlow
Substrate
2
Mistakes we made
• OpenFlow over Q-in-Q
– OpenFlow routing is unaware and sends traffic with same MAC
address in both direction, causing perpetual learning and CPU inflation
• Moving uplinks for 1 switch, while being pointed to the same
controller (causing two islands)
– Causes controller learning to oscillate between the 2 uplinks
• Bad interaction with legacy protocols
– LLDP and STP are treated differently with different switches
• Loop in OpenFlow network being exposed to non-OF side
– Miscommunication between the aggregate operator and the
experimenter during testing phase
• Loop across backbones
– Same campus connected over NLR and Internet2
Next steps for GENI
• Remove duct-tape
– Fix any issues that we learned about during demos
• Be better prepared
– Higher stability and better isolation
– Test bandwidth slicing
• Grow topology
– Add more switches and hosts
• Wean off
– Each campus takes charge of control framework
Tutorial Setup so far
c0
Controller
port6633
loopback
(127.0.0.1:6633)
OpenFlow Tutorial
3hosts-1switch
topology
s1
OpenFlow Switch
s1-eth0
h1-eth0
s1-eth1
h3-eth0
loopback
(127.0.0.1:6634)
s1-eth2
h4-eth0
h2
h3
h4
10.0.0.2
10.0.0.3
10.0.0.4
virtual hosts
dpctl
(user space
process)
62
Virtualized Network Setup
Controller
port 7000
c0
FlowVisor
port 6633
OpenFlow Tutorial
3hosts-1switch
topology
s1
OpenFlow Switch
s1-eth0
h1-eth0
s1-eth1
h3-eth0
loopback
(127.0.0.1:7000)
loopback
(127.0.0.1:6634)
loopback
(127.0.0.1:6634)
s1-eth2
h4-eth0
h2
h3
h4
10.0.0.2
10.0.0.3
10.0.0.4
virtual hosts
dpctl
(user space
process)
63
FlowVisor Usage
• <flowvisor dir>/script/fvctl.sh
• listDevices
– list of all OpenFlow switches’ datapath ID
• getLinks
– list of all links (port # and datapath ID of both end switches)
• createSlice
– creating slice (specifying controller’s URL and slice name)
• listSlices
• addFlowSpace
– add flow space to slices
• listFlowSpace
– show current flow space
64
Concluding Remarks
65
Highlights of Deployments
• Stanford deployment
– McKeown group for 1.5 years: production and experiments
– To scale later this year to entire building (~500 users)
• Nation-wide trials and deployments
– 7 other universities and BBN deploying now
– GEC9 in Nov, 2010 showcased nation-wide OF
– Internet 2 and NLR starting to serve as the GENI Backbone
• Global trials
– Over 60 organizations experimenting
2011 likely to be a big year for OpenFlow
66
Current Trials
• 68 trials/deployments spanning 13 countries
67
Get involved!
• Ask and answer questions on mailing lists:
– openflow-discuss
– openflow-spec
• Share and update wiki content
• Submit bug-reports and/or patches to OF
reference implementation
• Release open-source applications
• Write a controller!
68
Are you innovating in your network?
69
SDN Team at Stanford