Tutorial: OpenFlow in GENI with
Instrumentation and Monitoring
Divya Bhat, Umass Amherst
Niky Riga, GENI Project Office
IC2E 2014
Sponsored by the National Science Foundation
“The current Internet is at an impasse
because new architecture cannot be
deployed or even adequately evaluated”
[PST04]
[PST04]: Overcoming the Internet Impasse through Virtualization, Larry Peterson, Scott Shenker, Jonothan Turner
Hotnets 2004
Modified slide from: http://cenic2012.cenic.org/program/slides/CenicOpenFlow-3-9-12-submit.pdf
Sponsored by the National Science Foundation
March 10 2014
2
OpenFlow…
• Enables innovation in networking
• Changes practice of networking
Google’s SDN WAN
Sponsored by the National Science Foundation
March 10 2014
3
OpenFlow’s basic idea
Sponsored by the National Science Foundation
March 10 2014
4
OpenFlow’s basic idea
Sponsored by the National Science Foundation
March 10 2014
5
Network Devices
DHCP
access
router
point
DNS
proxy
VPN
firewall
gateway
NAT
switch
software
Any network device can be OpenFlow enabled
Sponsored by the National Science Foundation
March 10 2014
6
Network Types
Campus
Multiple buildings, heterogeneous IT, groups of users, campus
backbone
Enterprise Data Centers
Security, various sizes, storage, WAN optimizations
Data Centers – Clouds
Multi-tenant, virutalization, disaster recovery, VM mobility
WAN
Diversity, multiple domains/carriers/users
Sponsored by the National Science Foundation
March 10 2014
7
Deployment Stories
Google global private WAN [1]
Connects dozens of datacenters worldwide with a long-term average
of 70% utilization over all links
Stanford Campus deployment
Part of Stanford campus
migrated to OpenFlow
NTT’s BGP Free Edge
Internet 2 - AL2S
Can build Layer 2 circuits between https://www.ntt-review.jp/archive/ntttechnical.php?contents=ntr201310fa3.html
any Internet 2 end-points
[1] B4: Experience with a Globally-Deployed Software Defined WAN, SIGCOMM’13, Jain et al.
Sponsored by the National Science Foundation
March 10 2014
9
GENI Programmable Network
• Key GENI concept: slices & deep programmability
– Internet: open innovation in application programs
– GENI: open innovation deep into the network
Good
old
Slice 0
Internet
Slice 1
Slice 1
OpenFlow switches one of
the ways GENI is providing
deep programmability
Sponsored by the National Science Foundation
March 10 2014
Slice 2
Slice 3
Slice 4
10
Racks and Campuses
• GENI Rack projects are
expanding available GENI
infrastructure in the US.
• Racks provide reservable,
sliceable compute and
network resources using
Aggregate Managers.
• GENI AM API compliance
Sponsored by the National Science Foundation
March 10 2014
11
GENI Rack Campuses
Funds
in hand
Needs
funding
Oct. 24, 2012
• 43 racks planned this year
• Each rack has an OpenFlow-enabled switch
Sponsored by the National Science Foundation
March 10 2014
12
Core Networks
Internet2 SDN networks
•
•
•
NLR committed to 2013 meso-scale expansion following reorganization
Internet2 adding 10GbE paths to Advanced Layer 2 Services (AL2S) at 4 of 5
OpenFlow meso-scale/ProtoGENI Pops
GENI Aggregate Manager in Internet2 AL2S and dynamic stitching with GENI
coming in Spiral 5
Sponsored by the National Science Foundation
March 10 2014
13
OpenFlow Experiments
Debugging OpenFlow experiments is hard:
– Network configuration debugging requires coordination
– Many networking elements in play
– No console access to the switch
Before deploying your OpenFlow experiment
test your controller.
http://mininet.github.com/
Sponsored by the National Science Foundation
March 10 2014
http://openvswitch.org/
14
OpenFlow common PitFalls
• Controller is responsible for all traffic, not just your
application!
– ARP, DHCP, LLDP
• Reactive controllers
– UDP
• Performance in hardware switches
– Not all actions are supported in hardware
• No STP
– Broadcast storms
Sponsored by the National Science Foundation
March 10 2014
15
LabWiki
The complete Experiment
LifeCycle in a Tool
Divya Bhat, Umass Amherst
Sponsored by the National Science Foundation
Experiment Workflow
Sponsored by the National Science Foundation
March 10 2014
17
The “Experiment Cycle”
Design
Notes
Experiment
Description
Setup
Runs
Code
Measurements
Execute
Sponsored by the National Science Foundation
March 10 2014
18
The “Successful Experiment Cycle”
Publish
Design
Notes
Experiment
Description
Setup
Runs
Code
Measurements
Execute
Sponsored by the National Science Foundation
March 10 2014
19
“Experiment Cycle” in a Tool: LabWiki
Sponsored by the National Science Foundation
March 10 2014
20
Collect and Archive
Experimenter
6.Obtain
1.Instrument
iRODS
5.Save
LabWiki
4.Plot
0.Reserve
OML
Server
2.Run
GENI
TestBed
3.Collect
OML Client
Sponsored by the National Science Foundation
March 10 2014
21
Search
Wiki
Sponsored by the National Science Foundation
Edit
March 10 2014
Run
22
• Status:
– Open-source MIT License
– Code:
• https://github.com/mytestbed/labwiki
– Bug reports & documentation (hahaha):
• http://omf.mytestbed.net/projects/labwiki
– Plugin example:
• https://github.com/mytestbed/labwiki_topology_plugin
Sponsored by the National Science Foundation
March 10 2014
23
• Part I: Design/Setup
– Network Load Balancing and OpenFlow
– Design your experiment
– Obtain Resources
• Part II: Execute
– Configure and Initialize Services
– Execute Experiment
– Collect Measurements
• Part III: Finish
– Archive results
– Teardown Experiment
Sponsored by the National Science Foundation
March 10 2014
24
Balanced Service
• Balance load between
two or more server
providers
• In networks
Balance traffic between two
or more paths
Need to run load balancer
in each decision point, i.e.
within the network
Sponsored by the National Science Foundation
March 10 2014
Picture from : www.inetu.ne
25
OpenFlow
Any Host
OpenFlow Controller
OpenFlow Protocol (SSL/TCP)
Switch
Control Path
OpenFlow
• The controller is
responsible for
populating forwarding
table of the switch
• Controller can get
statistics directly from
each switch:
– per port
– per flow
– per table
Data Path (Hardware)
Modified slide from : http://www.deutsche-telekom-laboratories.de/~robert/GENI-Experimenters-Workshop.ppt
Sponsored by the National Science Foundation
March 10 2014
26
Today’s Experiment
• Two paths from source to destination
• Balance traffic between the two paths so that:
– Both paths are equally utilized
– All TCP flows have similar performance
Picture from : www.spacegamejunkie.com
Sponsored by the National Science Foundation
March 10 2014
27
• Part I: Design/Setup
– Network Load Balancing and OpenFlow
– Design your experiment
– Obtain Resources
• Part II: Execute
– Configure and Initialize Services
– Execute Experiment
– Collect Measurements
• Part III: Finish
– Archive results
– Teardown Experiment
Sponsored by the National Science Foundation
March 10 2014
28
Design Your Experiment
S
LB
SW
VM
OF
Ctl
VM
VM
R
• Start OpenFlow Load
Balancing Controller
• Connect OpenFlow
Switch to Controller
• Start several TCP
flows from Sender to
Receiver
Modified slide from : http://www.deutsche-telekom-laboratories.de/~robert/GENI-Experimenters-Workshop.ppt
Sponsored by the National Science Foundation
March 10 2014
29
Monitor Your Experiment
Monitor
S
SW
LB
VM
OF
Ctl
VM
VM
R
Modified slide from : http://www.deutsche-telekom-laboratories.de/~robert/GENI-Experimenters-Workshop.ppt
Sponsored by the National Science Foundation
March 10 2014
30
• Part I: Design/Setup
– Network Load Balancing and OpenFlow
– Design your experiment
– Obtain Resources
• Part II: Execute
– Configure and Initialize Services
– Execute Experiment
– Collect Measurements
• Part III: Finish
– Archive results
– Teardown Experiment
Sponsored by the National Science Foundation
March 10 2014
31
Resources Reserved!
Sponsored by the National Science Foundation
March 10 2014
32
• Part I: Design/Setup
– Network Load Balancing and OpenFlow
– Design your experiment
– Obtain Resources
• Part II: Execute
– Configure and Initialize Services
– Execute Experiment
– Collect Measurements
• Part III: Finish
– Archive results
– Teardown Experiment
Sponsored by the National Science Foundation
March 10 2014
33
Execute Experiment
• Follow Instructions on Tutorial page to execute your
experiment on LabWiki
• Pay attention to the output of your OpenFlow
Controller:
Sponsored by the National Science Foundation
March 10 2014
34
Trema
• Trema is NOT an OpenFlow Controller
• Trema is a Programming Framework for
implementing OpenFlow Controllers
• Trema is a Platform to run OpenFlow Controllers
• You need to:
– Write your own controller using Trema API
– Run your controller with Trema
Sponsored by the National Science Foundation
March 10 2014
35
Trema API
• API for both C and Ruby
• Event-triggered Functions:
–
–
–
–
start() is called when controller starts up
switch_ready() is called when switch connects in
packet_in() is called when a packet is received
stats_reply() is called when a flow stats reply message
is received
– ......
• More information about Trema API:
– http://rubydoc.info/github/trema/trema/master/frames
Sponsored by the National Science Foundation
March 10 2014
36
• Part I: Design/Setup
– Network Load Balancing and OpenFlow
– Design your experiment
– Obtain Resources
• Part II: Execute
– Configure and Initialize Services
– Execute Experiment
– Collect Measurements
• Part III: Finish
– Archive results
– Teardown Experiment
Sponsored by the National Science Foundation
March 10 2014
37
• Part I: Design/Setup
– Network Load Balancing and OpenFlow
– Design your experiment
– Obtain Resources
• Part II: Execute
– Configure and Initialize Services
– Execute Experiment
– Collect Measurements
• Part III: Finish
– Archive results
– Teardown Experiment
Sponsored by the National Science Foundation
March 10 2014
38
Collect and Archive
Experimenter
6.Obtain
1.Instrument
iRODS
5.Save
LabWiki
4.Plot
0.Reserve
OML
Server
2.Run
GENI
TestBed
3.Collect
OML Client
Sponsored by the National Science Foundation
March 10 2014
39
• Part I: Design/Setup
– Network Load Balancing and OpenFlow
– Design your experiment
– Obtain Resources
• Part II: Execute
– Configure and Initialize Services
– Execute Experiment
– Collect Measurements
• Part III: Finish
– Archive results
– Teardown Experiment
Sponsored by the National Science Foundation
March 10 2014
40
Part III: Finish Experiment
AM API
aggregate
slice
When your experiment is done, you should always
release your resources.
– Normally this is when you would archive your data
– Delete your slivers at each aggregate
Sponsored by the National Science Foundation
March 10 2014
41