GENI Engineering Conference March 4, 2008 A couple of platforms (Or: “Why can’t I innovate in my wiring closet?”) Nick McKeown nickm@stanford.edu The Stanford Clean Slate Program http://cleanslate.stanford.edu GENI: How to try new network architectures in my local network? PlanetLab Suitable for distributed systems using overlays Not suitable for radical network architectures Performance is limited XORP Open-source router code for Linux, FreeBSD, etc. Performance is limited Point solution OpenWRT Linux for wireless access points and routers VINI, WUSTL SPP, … Designed for large nationwide backbone infrastructure The Stanford Clean Slate Program http://cleanslate.stanford.edu Outline NetFPGA ¾ Open source networking hardware ¾ 4-ports of 1GE ¾ Operates at line-rate ¾ Available for $500 (university) OpenFlow ¾ Enabling innovation on campus ¾ Standard way to control flow-tables in commercial switches and routers ¾ Being deployed at Stanford ¾ Consider deploying it at your campus too The Stanford Clean Slate Program http://cleanslate.stanford.edu Open source networking hardware CPU Memory PC with NetFPGA PCI 1GE FPGA 1GE 1GE Memory 1GE NetFPGA Board Rack of 1U servers 96 x 1GE ports 9 Hardware available from 3rd party ~$500 (universities) 9 PCI board, or pre-built system (desktop or rack-mount) The Stanford Clean Slate Program http://cleanslate.stanford.edu NetFPGA: Usage Models Types of user 1. 2. 3. 4. Teachers Students Researchers Commercial development Supported Use Models 1. Router Kit 2. Supported Modular Reference Designs 3. Free Design (or 3rd party design) All software and designs available under BSD-like license The Stanford Clean Slate Program http://cleanslate.stanford.edu a Us # e g 1 NetFPGA “Router Kit” User-space development, 4x1GE line-rate forwarding OSPF CPU BGP Memory My Protocol user kernel Routing Table PCI “Mirror” Fwding Table Packet Buffer 1GE FPGA IPv4 Router 1GE Memory 1GE The Stanford Clean Slate Program 1GE http://cleanslate.stanford.edu a Us # e g 2 NetFPGA “Reference Router” Modules in hardware PW-OSPF CPU Java GUI Front Panel (Extensible) PCI NetFPGA Driver 1GE FPGA 1GE 1GE Memory Verilog EDA Tools (Xilinx, Mentor, etc.) Memory 1GE L3 Parse L2 Parse 1. 1. Design Design 2. 2. Simulate Simulate 1GE 3. 3. QSynthesize Synthesize In 4. 4. Download Download Mgmt 1GE IP Lookup My Block Out Q Mgmt 1GE 1GE Verilog modules interconnected by FIFO interfaces The Stanford Clean Slate Program http://cleanslate.stanford.edu a Us # e g 3 NetFPGA Free Design CPU Memory PCI 1GE FPGA 1GE Verilog EDA Tools (Xilinx, Mentor, etc.) 1. 1. Design Design NetFPGA 2. Simulate 2. Driver Simulate 3. 3. Synthesize Synthesize 4. 4. Download Download My Design 1GE Memory 1GE The Stanford Clean Slate Program 1GE 1GE 1GE (1GE MAC is soft/replaceable) 1GE http://cleanslate.stanford.edu Resources http://NetFPGA.org Available ¾ ¾ ¾ ¾ NetFPGA hardware Linux Driver Java GUI Test & regression software Courseware ¾ Undergraduate EE: Ethernet switch ¾ Masters EE/CS: IPv4 router hardware and software Tutorials ¾ 6 tutorials worldwide in 2008 ¾ 1 week Stanford summer camp The Stanford Clean Slate Program http://cleanslate.stanford.edu Resources (2) http://NetFPGA.org Reference Designs ¾ 4x1GE NIC ¾ 4x1GE IPv4 Router ¾ 4x1GE OpenFlow (soon) User-contributed ¾ NAT ¾ IEEE 1588 Timing Sync ¾ Real-time buffer monitoring ¾ RCP (Rate Control Protocol) The Stanford Clean Slate Program http://cleanslate.stanford.edu Funding http://NetFPGA.org Launched with funds from NSF EIA Program The Stanford Clean Slate Program http://cleanslate.stanford.edu Outline NetFPGA – – – – Open source networking hardware 4-ports of 1GE Operates at line-rate Available for $500 (university) OpenFlow – Enabling innovation on campus – Standard way to control flow-tables in commercial switches and routers – Being deployed at Stanford – Consider deploying it at your campus too The Stanford Clean Slate Program http://cleanslate.stanford.edu Innovations in campus wiring closets Experiments we’d like to do ¾ Mobility management ¾ Network-wide energy management ¾ New naming/addressing schemes ¾ Network access control Problem with our network ¾ Paths are fixed (by the network) ¾ IP-only ¾ Addresses dictated by DNS, DHCP, etc ¾ No means to add our own processing The Stanford Clean Slate Program http://cleanslate.stanford.edu OpenFlow Switching 1. A way to run experiments in the networks we use everyday. 2. Bring GENI to college campuses. A “pragmatic” compromise Allow researchers to run experiments in their network… …without requiring vendors to expose internal workings. Basics An Ethernet switch (e.g. 128-ports of 1GE) An open protocol to remotely add/remove flow entries The Stanford Clean Slate Program http://cleanslate.stanford.edu Experimenter’s Dream (Vendor’s Nightmare) Standard sw Network hw Processing The Stanford Clean Slate Program Userdefined Processing Experimenter writes experimental code on switch/router http://cleanslate.stanford.edu No obvious way Commercial vendor won’t open software and hardware development environment ¾ Complexity of support ¾ Market protection and barrier to entry Hard to build my own ¾ Prototypes are flakey ¾ Software only: Too slow ¾ Hardware/software: Fanout too small (need >100 ports for wiring closet) The Stanford Clean Slate Program http://cleanslate.stanford.edu Furthermore, we want… Isolation: Regular production traffic untouched Virtualized and programmable: Different flows processed in different ways Equipment we can trust in our wiring closet Open development environment for all researchers (e.g. Linux, Verilog, etc). Flexible definitions of a flow ¾ Individual application traffic ¾ Aggregated flows ¾ Alternatives to IP running side-by-side ¾… The Stanford Clean Slate Program http://cleanslate.stanford.edu OpenFlow Switching Controller OpenFlow Switch specification OpenFlow Switch sw Secure Channel low F n Ope ocol t Pro SSL PC hw Flow Table The Stanford Clean Slate Program http://cleanslate.stanford.edu Flow Table Entry “Type 0” OpenFlow Switch Rule Action Stats Packet + byte counters 1. 2. 3. 4. Switch MAC Port src + mask MAC dst The Stanford Clean Slate Program Forward packet to port(s) Encapsulate and forward to controller Drop packet Send to normal processing pipeline Eth type VLAN ID IP Src IP Dst IP Prot TCP sport TCP dport http://cleanslate.stanford.edu OpenFlow “Type 1” Definition in progress Additional actions ¾Rewrite headers ¾Map to queue/class ¾Encrypt More flexible header ¾Allow arbitrary matching of first few bytes Support multiple controllers ¾Load-balancing and reliability The Stanford Clean Slate Program http://cleanslate.stanford.edu Secure Channel SSL Connection, site-specific key Controller discovery protocol Encapsulate packets for controller Send link/port state to controller The Stanford Clean Slate Program http://cleanslate.stanford.edu Server room OpenFlow OpenFlow Access Point Controller PC OpenFlow The Stanford Clean Slate Program OpenFlow-enabled Commercial Switch Normal Software Channel Channel Normal Datapath Flow Flow Table Table OpenFlow Secure Secure http://cleanslate.stanford.edu OpenFlow Usage Models 1. Experiments at the flow level 2. • Experiment-specific controllers • Static or dynamic flow-entries Experiments at the packet level 3. User-defined routing protocols Admission control Network access control Network management Energy management VOIP mobility and handoff … Slow: Controller handles packet processing Fast: Redirect flows through programmable hardware Modified routers, firewalls, NAT, congestion control… Alternatives to IP The Stanford Clean Slate Program http://cleanslate.stanford.edu Example Experiment at the flow level Mobility Lots of interesting questions • Management of flows • Control of switches • Access control of users and devices • Tracking user location and motion The Stanford Clean Slate Program http://cleanslate.stanford.edu Experiments at the packet level Controller OpenFlow-enabled Commercial Switch Normal Software Channel Channel Normal Datapath Flow Flow Table Table PC Secure Secure Laboratory NetFPGA The Stanford Clean Slate Program http://cleanslate.stanford.edu OpenFlow Usage Models 2. Experiments at the flow level Experiments at the packet level 3. Alternatives to IP 1. Flow-table is Layer-2 based e.g. new naming and addressing schemes … The Stanford Clean Slate Program http://cleanslate.stanford.edu OpenFlow Consortium http://OpenFlowSwitch.org Goal: Evangelize OpenFlow to vendors Free membership for all researchers Whitepaper, OpenFlow Switch Specification, Reference Designs Licensing: Free for research and commercial use The Stanford Clean Slate Program http://cleanslate.stanford.edu OpenFlow: Status Commercial Ethernet switches and routers ¾ Working with six vendors to add to existing products ¾ Expect OpenFlow “Type 0” to be available in 2008-09 Reference switches ¾ Software: Linux and OpenWRT (for access points) ¾ Hardware: NetFPGA (line-rate 1GE; available soon) ¾ Working on low-cost 48-port 1GE switch based on Broadcom reference design Reference controller ¾ Simple test controller ¾ NOX controller (Martin Casado; available soon) The Stanford Clean Slate Program http://cleanslate.stanford.edu Deployment at Stanford Stanford Computer Science Department Gates Building ~1,000 network users 23 wiring closets Stanford Center for Integrated Systems (EE) Paul Allen Building ~200 network users 6 wiring closets Working with HP Labs and Cisco on deployment The Stanford Clean Slate Program http://cleanslate.stanford.edu If you are interested in deploying OpenFlow on your campus… Please contact me! nickm@stanford.edu http://OpenFlowSwitch.org