Why can’t I innovate
in my wiring closet?
MIT, April 17, 2008
Nick McKeown
nickm@stanford.edu
The Stanford Clean Slate Program
http://cleanslate.stanford.edu
Funded by: Cisco, DT
DoCoMo, NEC, Xilinx
Outline
2

Routing doesn’t belong in routers

Substrates to enable innovation

OpenFlow as a streamlined substrate
Internet Innovation
End to end arguments led to the principle:
Move function “up” out of the network to the end-points
Led to streamlined substrate of links and
routers
Even congestion control done at end-points
Led to enormous innovation on top
Email, WWW, VOIP, P2P, Social networks, …
3
A fixed substrate
Fierce protection against overloading
network, unless necessary
Change happens slowly or surreptitiously
IPv6, Multicast, NAT
An era of fixed substrates
IP, Intel instruction set, ms-dos
• Stable fixed substrates
• Standards that bred innovation
It’s hard to imagine it having been any other way….
4
How streamlined is the substrate?
Feature lists as long as your arm
Routers with >10M lines of source code,
datapath with 100M gates & 1Gbyte RAM.
Many complex functions in the infrastructure
OSPF, BGP, multicast, differentiated services,
Traffic Engineering, NAT, firewalls, MPLS, redundant layers, …
Claim
• Most complexity is about routing: picking paths,
managing location, identity and access.
• The current network controls the routing.
• Routing doesn’t belong in the boxes.
• If we control routing, we can innovate.
Uh-oh, looks like another GENI talk…
5
Routing inside routers
It made sense at the time

Early on, seemed necessary for scalability & robustness

Routing doesn’t need to be in the network
Trivial example: source routing

Routing doesn’t belong in the network
Aside: Electricity grid of lines, transformers and switches



Stakeholders should control routing and innovation
Why can’t I add my own routing protocol? Or mobility
management protocol?
Why do you and I have to run the same routing protocol?
4D [Zhang], RCP [Rexford] and others
6
Local equilibrium
 Inelegant
closed design (like a bad API)
leads to complexity, and fragility. Hard to
innovate.
Resistant to change
Inelegance
Industry, IETF, …
High
Barrier to
Entry
Add complexity
 Not
7
a conspiracy, just a local equilibrium
Why should we care?
There are basic things the Internet can’t do well
Mobility, access control/security, management, …
Why not just add more features to fix it?
An uneasy feeling that we are piling more and more
complexity into the network, making it overloaded,
fragile, less elegant and harder to innovate.
We need a simple, reliable, constant substrate
Oh my god – it really does look like another GENI talk…
8
Current substrate
On a PC substrate, we can choose an OS.
And now we have virtualization too.
In the network world, your switch/router comes
bundled and you have no choice.
So…
1. Alternative OS on your switch/router?
2. Virtualization of your switch/router?
9
Outline

Routing doesn’t belong in routers

Substrates to enable innovation
1. Open OS’s on switches/routers
2. Virtualization of switches/routers

10
OpenFlow as a streamlined substrate
Innovation by ripping off the lid
XORP (and Zebra; or just Linux)
Open-source router code for Linux, FreeBSD, etc.
Flexible, easy to use
Software-based; limited performance
Point solution
NetFPGA
Small (4x1GE) “router”; open-source hardware and software
Flexible, low-cost, medium ease of use
Hardware-based; line-rate performance
Point solution, limited port count (4-8)
OpenWRT
Embedded Linux for wireless APs and routers
11
Open source networking hardware
CPU
PCI
FPGA
Memory
 Hardware available from 3rd party ~$500 (universities)
PC with NetFPGA
 Line-rate forwarding
 PCI board, or pre-built system (desktop or rack-mount)
 500
1GEboards, 15 countries; expect 2,000 this year
1GE
 Free reference designs, gateware and courseware
1GE
Memory
Rack of 1U servers
96 x 1GE ports
1GE
NetFPGA Board
12
Outline

Routing doesn’t belong in routers

Substrates to enable innovation
1. Open OS’s on switches/routers
2. Virtualization of switches/routers

13
OpenFlow as a streamlined substrate
Innovation by changing the substrate
Diverse applications
Diverse applications
Diverse transport layers
Diverse transport layers
IP X Y Z
IP
Virtualization layer
Diverse link layers
Diverse link layers
Diverse physical layers
Diverse physical layers
This is the GENI approach: Innovation from the “top down”
e.g. WUSTL SPP, VINI, …
Proposed approach: OpenFlow … innovation from the “bottom up”
14
OpenFlow Model
Allow lots of innovative research experiments
Diverse applications
Routing,
Mobility,
Diverse
transport layers
Naming/Addressing,
Ethernet
IP X Y Z
Access Control,
Flow layer
Management,
Monitoring…
Collapse difference; ease use of optical circuits
Packet switching and circuit switching
Diverse link layers
Diverse physical layers
15
OpenFlow Switching
A way to innovate in the networks we use everyday.
A “pragmatic” compromise
Allow researchers to run experiments in their network…
…without requiring vendors to expose internal workings.
1.
2.
3.
Work with switch and AP vendors to add OpenFlow to
their products
Deploy on university campuses
Stand back and watch students innovate…
Basics
An Ethernet switch (e.g. 128-ports of 1GE)
Use flow-table already in every switch and chipset
An open protocol to remotely add/remove flow entries
16
What we’d like…
Isolation: Regular production traffic untouched
 Virtualized and programmable: Different flows
processed in different ways
 Equipment we can trust in our wiring closet
 Open development environment for all
researchers (e.g. Linux, Verilog, etc).
 Flexible definitions of a flow

 Individual application traffic
 Aggregated flows
 Alternatives to IP running side-by-side
…
17
OpenFlow Switching
Controller
OpenFlow Switch specification
OpenFlow Switch
PC
sw Secure
Channel
hw Flow
Table
18
• Add/delete flow entry
• Encapsulated packets
• Controller discovery
Flow Table Entry
“Type 0” OpenFlow Switch
Rule
Action
Stats
Packet + byte counters
1. Forward packet to port(s)
2. Encapsulate and forward to controller
3. Drop packet
4. Send to normal processing pipeline
Switch MAC
Port
src
+ mask
19
MAC
dst
Eth
type
VLAN
ID
IP
Src
IP
Dst
IP
Prot
TCP
sport
TCP
dport
OpenFlow “Type 1”
(future)
More flexible header
 Allow arbitrary matching of first few bytes
Additional actions
 Rewrite headers (e.g. NAT, or address
obfuscation)
 Map to queue/class
 Encrypt
Support multiple controllers
 Load-balancing and robustness
20
OpenFlow Consortium
http://OpenFlowSwitch.org
Goal: Evangelize OpenFlow to vendors
Free membership for all researchers
Whitepaper, OpenFlow Switch Specification,
Reference Designs
Licensing: Free for research and commercial use
21
OpenFlow: Status
Commercial Ethernet switches and routers
 Working with six vendors to add to existing products
 Expect OpenFlow “Type 0” to be available in 2008-09
Reference switches
 Software: Linux and OpenWRT (for access points)
 Hardware: NetFPGA (line-rate 1GE; available soon)
 Working on low-cost 48-port 1GE switch based on
Quanta switch (Broadcom chips)
Reference controller
 Simple test controller
 NOX controller (available soon)
22
Deployment at Stanford
Stanford Computer Science Department
Gates Building
~1,000 network users
23 wiring closets
Stanford Center for Integrated Systems (EE)
Allen Building
~200 network users
6 wiring closets
Working with HP Labs and Cisco on deployment
23
Deployment in Internet2
NetFPGA routers deployed
24
OpenFlow Usage Models
1.
Experiments at the flow level








2.
25
• Experiment-specific controllers
• Static or dynamic flow-entries
Experiments at the packet level



3.
Layer 2 or Layer 3 (same!)
User-defined routing protocols
Admission control
Network access control
Network management
Energy management
VOIP mobility and handoff
…
Slow: Controller handles packet processing
Fast: Redirect flows through programmable hardware
Modified routers, firewalls, NAT, congestion control…
Alternatives to IP
OpenFlow Switch
Usage example: Production Traffic Unchanged
Policy Rule
Commercial
Switch or AP
Hari: Use production network
User Space
Open API
“Hari”
Controller
Llinux kernel
26
sw
Normal
Software
Secure
Channel
hw
Normal
datapath
Flow
Table
Linux PC
Hari
OpenFlow Switch
Usage example: New Protocols Isolated, But Alongside
Policy Rule
Commercial
Switch or AP
Dina: Use Dina’s protocol
Open API
User Space
“Dina”
Controller
Llinux kernel
27
sw
Normal
Software
Secure
Channel
hw
Normal
datapath
Flow
Table
Linux PC
Dina
Example Experiment at the flow level
Mobility
Lots of interesting questions
• Management of flows
• Control of switches
• Access control of users and devices
• Tracking user location and motion
28
Innovation in the datapath
Layerless plumbing
Line-rate packet processing
1. Encryption
2. IP++
3. Packet inspection
4. Congestion control
(XCP etc.)
5. …?
OpenFlow-enabled
Commercial Switch
Normal
Software
Normal
Datapath
Secure
Channel
Flow
Table
Laboratory
NetFPGA
29
Controller
PC
Controller Innovation
Controller Innovation Layer
Static
Controllers
Experiment
Specific
Controllers
4D
NOX
OpenFlow Substrate Layer
30
IP
Ethernet
Alongside
Normal routing,
mgmt, inside box
Alongside
Spanning tree,
VLANs, mgmt,
inside box
TDM/WDM
Circuits
Alongside
circuit routing,
mgmt, inside box
Controller Open Questions
 Scalability
of controller
 Load-balanced and redundant controllers
 Aggregation of flows
31
Conclusion
Networking has to work harder than most fields
to enable innovation.
 The good news is that industry is open to the
idea.
 And government agencies are on our side.
 Let’s work together to do it… let’s deploy
OpenFlow widely in universities.
 If you are interested, please contact me

nickm@stanford.edu
http://OpenFlowSwitch.org
32