OpenFlow Hands-on Tutorial OFELIA Meeting Feb 7, 2011 at Berlin Srini Seetharaman, DT R&D Lab, USA With help from: Dan Levin, TU Berlin Nadi Sarrar, TU Berlin Andi Wundsam, TU Berlin 1 Welcome! Getting Started • Copy to your hard disk from Flash Drive/: – Corresponding files (terminal, X server, Vmware) for your platform (Win/Mac/Linux) – OpenFlowTutorial-20110131.vmwarevm.zip • unzip OpenFlowTutorial-20110131.vmwarevm.zip • Instructions: http://www.openflow.org/wk/index.php/OpenFlowTutoria lOFELIA2011 • VMWare Key (if needed): – 4J09J-3U2E5-58C32-08AK0-8DH42 • Online attendees: Download file from: • http://www.net.t-labs.tuberlin.de/research/routerlab/oftutorial.zip 2 Introductions 3 Agenda Time Description 10:00-11:00 Introduction: How OpenFlow works, Potential, Limitations, Current vendors, SDN/OPEN 11:00-12:15 Begin Hands-on Portion (learn tools, self-exploration) Lunch break 13:00-14:00 Intro to Slicing, Details about GENI deployments 14:00-14:30 Continue Hands-on Portion (Slice using FlowVisor, use Expedient) Coffee break 15:00-15:45 Vision and long-term directions of OpenFlow (Nick McKeown) 15:45-16:30 Q&A; Discussions 4 Goals of this Tutorial • By the end, everyone should know: – – – – – what OpenFlow is how it’s used and how you can use it where it’s going how OpenFlow compares to other platforms how OpenFlow fits in the Software-Defined Networking (SDN) spectrum • Present a useful mix of hands-on and lecturebased content • Have fun 5 Why OpenFlow? 6 The Ossified Network Routing, management, mobility management, access control, VPNs, … Feature Feature Operating System Specialized Packet Forwarding Hardware Million of lines of source code 5400 RFCs Barrier to entry Billions of gates Bloated Power Hungry Many complex functions baked into the infrastructure OSPF, BGP, multicast, differentiated services, Traffic Engineering, NAT, firewalls, MPLS, redundant layers, … An industry with a “mainframe-mentality”, reluctant to change 7 Research Stagnation • Lots of deployed innovation in other areas – OS: filesystems, schedulers, virtualization – DS: DHTs, CDNs, MapReduce – Compilers: JITs, vectorization • Networks are largely the same as years ago – Ethernet, IP, WiFi • Rate of change of the network seems slower in comparison – Need better tools and abstractions to demonstrate and deploy 8 Closed Systems (Vendor Hardware) • • • • Stuck with interfaces (CLI, SNMP, etc) Hard to meaningfully collaborate Vendors starting to open up, but not usefully Need a fully open system – a Linux equivalent 9 Open Systems Performance Scale Fidelity Real User Traffic? Complexity Open Simulation medium medium no medium yes Emulation medium low no medium yes Software Switches poor low yes medium yes NetFPGA high low yes high yes Network Processors high medium yes high yes Vendor Switches high high yes low no gap in the tool space none have all the desired attributes! 10 OpenFlow: a pragmatic compromise • + Speed, scale, fidelity of vendor hardware • + Flexibility and control of software and simulation • Vendors don’t need to expose implementation • Leverages hardware inside most switches today (ACL tables) 12 How does OpenFlow work? 13 Ethernet Switch 14 Control Path (Software) Data Path (Hardware) 15 OpenFlow Controller OpenFlow Protocol (SSL/TCP) Control Path OpenFlow Data Path (Hardware) 16 OpenFlow Example Software Layer Controller PC OpenFlow Client Flow Table Hardware Layer MAC src MAC dst IP Src IP Dst TCP TCP Action sport dport * * * 5.6.7.8 * port 1 5.6.7.8 port 2 * port 3 port 1 port 4 1.2.3.4 17 OpenFlow Basics Flow Table Entries Rule Action Stats Packet + byte counters 1. 2. 3. 4. 5. Switch VLAN Port ID Forward packet to zero or more ports Encapsulate and forward to controller Send to normal processing pipeline Modify Fields Any extensions you add! VLAN MAC pcp src MAC dst Eth type IP Src IP Dst IP L4 IP ToS Prot sport L4 dport + mask what fields to match 18 Examples Switching Switch MAC Port src * MAC Eth dst type 00:1f:.. * * VLAN IP ID Src IP Dst IP Prot TCP TCP Action sport dport * * * * IP Dst IP Prot TCP TCP Action sport dport * * port6 Flow Switching Switch MAC Port src MAC Eth dst type port3 00:20.. 00:1f.. 0800 VLAN IP ID Src vlan1 1.2.3.4 5.6.7.8 4 17264 80 port6 Firewall Switch MAC Port src * * MAC Eth dst type * * VLAN IP ID Src IP Dst IP Prot TCP TCP Action sport dport * * * * * 22 drop 19 Examples Routing Switch MAC Port src * * MAC Eth dst type * * VLAN IP ID Src IP Dst * 5.6.7.8 * * VLAN IP ID Src IP Dst IP Prot vlan1 * * * TCP TCP Action sport dport port6, port7, * * port9 * IP Prot TCP TCP Action sport dport * port6 VLAN Switching Switch MAC Port src * * MAC Eth dst type 00:1f.. * 20 Centralized vs Distributed Control Both models are possible with OpenFlow Centralized Control Controller OpenFlow Switch Distributed Control Controller OpenFlow Switch Controller OpenFlow Switch OpenFlow Switch OpenFlow Switch Controller OpenFlow Switch 21 Flow Routing vs. Aggregation Both models are possible with OpenFlow Aggregated Flow-Based • • • • Every flow is individually set up by controller Exact-match flow entries Flow table contains one entry per flow Good for fine grain control, e.g. campus networks • • • • One flow entry covers large groups of flows Wildcard flow entries Flow table contains one entry per category of flows Good for large number of flows, e.g. backbone 22 Reactive vs. Proactive (pre-populated) Both models are possible with OpenFlow Reactive Proactive • • • • • First packet of flow triggers controller to insert flow entries Efficient use of flow table Every flow incurs small additional flow setup time If control connection lost, switch has limited utility • • • Controller pre-populates flow table in switch Zero additional flow setup time Loss of control connection does not disrupt traffic Essentially requires aggregated (wildcard) rules 23 Usage examples • Alice’s code: – Simple learning switch – Per Flow switching – Network access control/firewall – Static “VLANs” – Her own new routing protocol: unicast, multicast, multipath – Home network manager – Packet processor (in controller) – IPvAlice – – – – – VM migration Server Load balancing Mobility manager Power management Network monitoring and visualization – Network debugging – Network slicing … and much more you can create! Quiz Time • How do I provide control connectivity? Is it really clean slate? • Why aren’t users complaining about time to setup flows over OpenFlow? (Hint: What is the predominant traffic today?) • Considering switch CPU is the major limit, how can one take down an OpenFlow network? • How to perform topology discovery over OpenFlow-enabled switches? • What happens when you have a non-OpenFlow switch inbetween? • What if there are two islands connected to same controller? • How scalable is OpenFlow? How does one scale deployments? 25 What can you not do with OpenFlow ver1.0 • Non-flow-based (per-packet) networking – ex. Per-packet next-hop selection (in wireless mesh) – yes, this is a fundamental limitation – BUT OpenFlow can provide the plumbing to connect these systems • Use all tables on switch chips – yes, a major limitation (cross-product issue) – BUT an upcoming OF version will expose these 26 What can you not do with OpenFlow ver1.0 • New forwarding primitives – BUT provides a nice way to integrate them through extensions • New packet formats/field definitions – BUT a generalized OpenFlow (2.0) is on the horizon • Optical Circuits – BUT efforts underway to apply OpenFlow model to circuits • Low-setup-time individual flows – BUT can push down flows proactively to avoid delays Where it’s going • OF v1.1: Extensions for WAN, spring 2011 – multiple tables: leverage additional tables – tags and tunnels – multipath forwarding • OF v2+ – generalized matching and actions: an “instruction set” for networking 28 OpenFlow Implementations (Switch and Controller) 29 OpenFlow building blocks oftrace oflops Monitoring/ debugging tools openseer Stanford Provided ENVI (GUI) NOX LAVI Beacon FlowVisor Console n-Casting Helios Applications SNAC Controller Maestro Slicing Software FlowVisor Stanford Provided Commercial Switches HP, NEC, Pronto, Juniper.. and many more Expedient Software Ref. Switch NetFPGA Broadcom Ref. Switch OpenWRT PCEngine WiFi AP OpenVSwitch OpenFlow Switches 30 Current SDN hardware Juniper MX-series NEC IP8800 WiMax (NEC) HP Procurve 5400 Netgear 7324 PC Engines Pronto 3240/3290 Ciena Coredirector More coming soon... 31 Commercial Switch Vendors Model Virtualize Notes HP Procurve 5400zl or 6600 1 OF instance per VLAN -LACP, VLAN and STP processing before OpenFlow -Wildcard rules or non-IP pkts processed in s/w -Header rewriting in s/w -CPU protects mgmt during loop NEC IP8800 1 OF instance per VLAN -OpenFlow takes precedence -Most actions processed in hardware -MAC header rewriting in h/w Pronto 3240 or 3290 with Pica8 or Indigo firmware 1 OF instance per switch -No legacy protocols (like VLAN and STP) -Most actions processed in hardware -MAC header rewriting in h/w 32 Controller Vendors Vendor Notes Vendor Notes Nicira’s NOX •Open-source GPL •C++ and Python •Researcher friendly Stanford’s Beacon •Open-source •Researcher friendly •Java-based Nicira’s ONIX •Closed-source •Datacenter networks BigSwitch controller •Closed source •Based on Beacon •Enterprise network SNAC •Open-source GPL •Code based on NOX0.4 •Enterprise network •C++, Python and Javascript •Currently used by campuses Maestro (from Rice Univ) •Open-source •Based on Java NEC’s Helios •Open-source •Written in C 33 Growing Community Vendors and start-ups More... Providers and business-unit More... Note: Level of interest varies 34 Software-Defined Networking (SDN) 35 Current Internet Closed to Innovations in the Infrastructure Closed Ap p Ap p Ap p Operating System Ap p Specialized Packet Forwarding Hardware Ap p Ap p Ap p Ap p Operating System Ap p Specialized Packet Forwarding Hardware Operating System Ap p Specialized Packet Forwarding Hardware Ap p Ap p Operating System Ap p Ap p Ap p Specialized Packet Forwarding Hardware Operating System Specialized Packet Forwarding Hardware 36 “Software Defined Networking” approach to open it App App App Network Operating System Ap p Ap p Ap p Operating System Ap p Specialized Packet Forwarding Hardware Ap p Ap p Ap p Ap p Operating System Ap p Specialized Packet Forwarding Hardware Operating System Ap p Specialized Packet Forwarding Hardware Ap p Ap p Operating System Ap p Ap p Ap p Specialized Packet Forwarding Hardware Operating System Specialized Packet Forwarding Hardware 37 The “Software-defined Network” 2. At least one good operating system Extensible, possibly open-source 3. Well-defined open API App App App Network Operating System 1. Open interface to hardware Simple Packet Forwarding Hardware Simple Packet Forwarding Hardware Simple Packet Forwarding Hardware Simple Packet Forwarding Hardware Simple Packet Forwarding Hardware 38 Hands-on Session Flow Resume talk Resume lecture setup VM, play with mininet, use tools Run hub, turn hub into an Ethernet switch Slicing the network with FlowVisor and Expedient On your own add multiple switch support modify switch to handle IP forwarding 39 Stuff you’ll use • • • • • • NOX Reference Controller/Switch OpenvSwitch Mininet cbench flowvisor • iperf • tcpdump • Wireshark 40 Tutorial Setup c0 Controller port6633 loopback (127.0.0.1:6633) OpenFlow Tutorial 3hosts-1switch topology s1 OpenFlow Switch s1-eth0 h1-eth0 s1-eth1 h3-eth0 loopback (127.0.0.1:6634) s1-eth2 h4-eth0 h2 h3 h4 10.0.0.2 10.0.0.3 10.0.0.4 virtual hosts dpctl (user space process) 41 Hands-on Tutorial Next presentation starts at 13:00 www.openflow.org/wk/index.php/OpenFlowTutorialOFELIA2011 •Hope you got the files from the DVD or USB keys •VMWare Key (if needed): –4J09J-3U2E5-58C32-08AK0-8DH42 42 Virtualizing OpenFlow 43 Trend App App App Windows Windows Windows (OS) (OS) (OS) Linux Linux Linux App App App Mac Mac Mac OS OS OS Virtualization layer x86 (Computer) Computer Industry Controller11 NOX Controller (Network OS) Controller Controller Network OS 22 Virtualization or “Slicing” OpenFlow Network Industry Isolated “slices” App App Network Operating System 1 Many operating systems, or Many versions App App Network Operating System 2 App App App Network Operating System 3 App Network Operating System 4 Open interface to hardware Virtualization or “Slicing” Layer Open interface to hardware Simple Packet Forwarding Hardware Simple Packet Forwarding Hardware Simple Packet Forwarding Hardware Simple Packet Forwarding Hardware Simple Packet Forwarding Hardware 45 Switch Based Virtualization Exists for NEC, HP switches but not flexible enough Research VLAN 2 Flow Table Controller Research VLAN 1 Flow Table Controller Production VLANs Normal L2/L3 Processing 46 FlowVisor-based Virtualization Heidi’s Controller Aaron’s Controller Craig’s Controller Topology discovery is per slice OpenFlow Protocol OpenFlow Switch OpenFlow FlowVisor & Policy Control OpenFlow Protocol OpenFlow Switch OpenFlow Switch 47 FlowVisor-based Virtualization Separation not only by VLANs, but any L1-L4 pattern Multicast Broadcast OpenFlow Protocol dl_dst=FFFFFFFFFFFF OpenFlow Switch http Load-balancer tp_src=80, or tp_dst=80 OpenFlow FlowVisor & Policy Control OpenFlow Protocol OpenFlow Switch OpenFlow Switch 48 FlowSpace: Maps Packets to Slices FlowVisor Message Handling Alice Controller Bob Controller Cathy Controller OpenFlow Policy Check: Is this rule allowed? Policy Check: Who controls this packet? FlowVisor OpenFlow Full Line Rate Forwarding Packet Packet OpenFlow Firmware Data Path Rule Exception Use Case: New CDN - Turbo Coral ++ Basic Idea: Build a CDN where you control the entire network – – – – All traffic to or from Coral IP space controlled by Experimenter All other traffic controlled by default routing Topology is entire network End hosts are automatically added (no opt-in) Switch MAC Port src MAC Eth dst type VLAN IP ID Src IP Dst IP Prot TCP TCP sport dport * * * * * * * * 84.65.* * * * * * * * 84.65.* * * * * * * * * * * * * * 51 Use Case: Aaron’s IP • A new layer 3 protocol • Replaces IP • Defined by a new Ether Type Switch MAC Port src MAC Eth dst type VLAN IP ID Src IP Dst IP Prot TCP TCP sport dport * * * AaIP * * * * * * * * * !AaIP * * * * * * 52 OpenFlow/GENI Deployments 53 OpenFlow Deployment at Stanford Switches (23) APs (50) WiMax (1) 54 Live Stanford Deployment Statistics http://yuba.stanford.edu/ofhallway/wide-ofv1.html 55 How did we get there? Staged Deployment of OpenFlow 1. 2. 3. Add experimental VLAN Enable OpenFlow for Exptl VLAN Configure Controller for new network Verify correctness and performance 4. 5. Add new Production subnetwork Gradually add/move users to new subnet Verify reachability 6. 7. Enable OpenFlow for new subnet Slice the network GENI OpenFlow deployment (2010) 10 institutions and 2 National Research Backbones Kansas State 57 GENI Network Evolution National Lambda Rail GENI Integration • FlowVisor Expedient3 – Slicing control • Expedient – Experimenter’s portal for slice management GENI API API X Expedient1 API X Opt-in Mgr1 Expedient2 API X Opt-in Mgr2 FlowVisor API FlowVisor API • Opt-in Manager – Network admins’ portal to approve/ deny expt requests for traffic FlowVisor1 OpenFlow Substrate 1 FlowVisor2 OpenFlow Substrate 2 Mistakes we made • OpenFlow over Q-in-Q – OpenFlow routing is unaware and sends traffic with same MAC address in both direction, causing perpetual learning and CPU inflation • Moving uplinks for 1 switch, while being pointed to the same controller (causing two islands) – Causes controller learning to oscillate between the 2 uplinks • Bad interaction with legacy protocols – LLDP and STP are treated differently with different switches • Loop in OpenFlow network being exposed to non-OF side – Miscommunication between the aggregate operator and the experimenter during testing phase • Loop across backbones – Same campus connected over NLR and Internet2 Next steps for GENI • Remove duct-tape – Fix any issues that we learned about during demos • Be better prepared – Higher stability and better isolation – Test bandwidth slicing • Grow topology – Add more switches and hosts • Wean off – Each campus takes charge of control framework Tutorial Setup so far c0 Controller port6633 loopback (127.0.0.1:6633) OpenFlow Tutorial 3hosts-1switch topology s1 OpenFlow Switch s1-eth0 h1-eth0 s1-eth1 h3-eth0 loopback (127.0.0.1:6634) s1-eth2 h4-eth0 h2 h3 h4 10.0.0.2 10.0.0.3 10.0.0.4 virtual hosts dpctl (user space process) 62 Virtualized Network Setup Controller port 7000 c0 FlowVisor port 6633 OpenFlow Tutorial 3hosts-1switch topology s1 OpenFlow Switch s1-eth0 h1-eth0 s1-eth1 h3-eth0 loopback (127.0.0.1:7000) loopback (127.0.0.1:6634) loopback (127.0.0.1:6634) s1-eth2 h4-eth0 h2 h3 h4 10.0.0.2 10.0.0.3 10.0.0.4 virtual hosts dpctl (user space process) 63 FlowVisor Usage • <flowvisor dir>/script/fvctl.sh • listDevices – list of all OpenFlow switches’ datapath ID • getLinks – list of all links (port # and datapath ID of both end switches) • createSlice – creating slice (specifying controller’s URL and slice name) • listSlices • addFlowSpace – add flow space to slices • listFlowSpace – show current flow space 64 Concluding Remarks 65 Highlights of Deployments • Stanford deployment – McKeown group for 1.5 years: production and experiments – To scale later this year to entire building (~500 users) • Nation-wide trials and deployments – 7 other universities and BBN deploying now – GEC9 in Nov, 2010 showcased nation-wide OF – Internet 2 and NLR starting to serve as the GENI Backbone • Global trials – Over 60 organizations experimenting 2011 likely to be a big year for OpenFlow 66 Current Trials • 68 trials/deployments spanning 13 countries 67 Get involved! • Ask and answer questions on mailing lists: – openflow-discuss – openflow-spec • Share and update wiki content • Submit bug-reports and/or patches to OF reference implementation • Release open-source applications • Write a controller! 68 Are you innovating in your network? 69 SDN Team at Stanford