GENI as a Virtual Laboratory for
Networking and Distributed Systems
Classes
Workshop 7 at SIGCSE ‘14
Sarah Edwards
GENI Project Office
sedwards@bbn.com
Sponsored by the National Science Foundation
Agenda
Presentation and Demo: An Introduction to GENI (45min)
Hands-on: Getting Started with GENI (45min)
Break (10min)
Hands-on: IPv4 Routing Assignment (50min)
Resources for Instructors (15min)
Miscellaneous GENI Topics & Wrap-Up (15min)
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
2
GENI
Exploring Networks of the Future
www.geni.net
Sponsored by the National Science Foundation
Outline
•
•
•
•
•
•
GENI – Exploring future internets at scale
The GENI Concept
Building GENI
Experimental and Classroom use of GENI
What’s next for GENI?
GENI: An experimenter’s view
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
4
Global networks are creating
extremely important new challenges
Science Issues
We cannot currently
understand or predict the
behavior of complex,
large-scale networks
Innovation Issues
Substantial barriers to
at-scale experimentation with
new architectures, services,
and technologies
Society Issues
Credit: MONET Group at UIUC
We increasingly
rely on
the Internet but are unsure
we can trust its security,
privacy or resilience
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
5
GENI: Infrastructure for Experimentation
GENI provides compute resources that can be connected in
experimenter specified Layer 2 topologies.
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
6
GENI: Infrastructure for Experimentation
GENI provides compute resources that can be connected in
experimenter specified Layer 2 topologies.
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
7
Multiple GENI Experiments run Concurrently
Infrastructure
can be shared
between slices
Experiments live in isolated “slices”
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
8
GENI is “Deeply Programmable”
I install software I want
throughout my network slice (into
routers, switches, …) or control
switches using OpenFlow
Experimenters can set up custom topologies, protocols and
switching of flows
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
9
GENI Compute Resources
GENI Wireless
compute nodes
GENI Racks
Existing Testbeds
(e.g. Emulab)
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
10
GENI Networking Resources
Networking within a Rack
National Research Backbones
(e.g. Internet2)
Regional Networks
(e.g. CENIC)
WiMAX Base Stations
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
11
Outline
•
•
•
•
•
GENI – Exploring future internets at scale
The GENI Concept
Building GENI
Experimental and Classroom use of GENI
GENI: An experimenter’s view
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
12
A bright idea
“I have a great idea.”
“That will never work.”
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
13
Let’s try it out!
My new architecture worked great in the
lab, so now I’m going to try a larger
experiment for a few months.
He uses a modest slice of GENI, sharing its
infrastructure with many other concurrent experiments.
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
14
It turns into a really good idea
This service looks very useful
His slice of GENI keeps growing, but GENI is still running many
other concurrent experiments.
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
15
Attracts real users
“Looks like an app to me.”
“It’s my very own GENI slice.”
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
16
“Boy did I learn a lot!”
“What a cool service.”
(I wonder how it works.)
“I always said it was
a great idea.”
(But way too conservative.)
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
17
??
If you have a great idea, check out the
NSF CISE research programs for current opportunities.
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
18
Moral of this story
GENI is meant to enable . . .
– At-scale experiments
– Internet-incompatible experiments
– Both repeatable and “in the wild”
experiments
– ‘Opt in’ for real users
– Instrumentation and measurement
tools
GENI creates a huge opportunity for ambitious research!
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
19
Outline
•
•
•
•
•
GENI – Exploring future internets at scale
The GENI Concept
Building GENI
Experimental and Classroom use of GENI
GENI: An experimenter’s view
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
20
Growing GENI’s footprint
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
21
Federation
GENI grows by GENI-enabling heterogeneous infrastructure
My experiment runs across
the evolving GENI federation.
GENI Rack
Commercial
Clouds
Backbone #1
Campus
My GENI Slice
Access
#1
Research
Testbed
Corporate
GENI suites
Regional
Non-US
Testbeds
This approach looks
remarkably familiar . . .
GENI
Rack
Avoid technology “lock in” and grow quickly by
incorporating existing infrastructure
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
22
Federation Extends the Reach of
GENI and International Peer Testbeds
Initial plan to federate testbeds on five continents
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
23
Build GENI at sufficient scale
Infeasible to build a testbed as big as the Internet
HP ProCurve 5400 Switch
NEC WiMAX Base Station
GENI-enabled
equipment
GENI-enabled campuses,
students as early adopters
“At scale” GENI prototype
GENI-enable testbeds, commercial equipment, campuses, regional and
backbone networks
Campus photo by Vonbloompasha
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
24
GENI architecture
UNI VERSI T Y
ISP
Internet
Metro
Research
Backbones
g
GENI-enabled
hardware
Legend
Layer 2
Data Plane
Layer 3
Control Plane
g
UNI VERSI T Y
Regional Networks
Campus
g
• Flexible network / cloud research • Support “hybrid circuit” model plus
infrastructure
much more (OpenFlow)
• Also suitable for physics,
• Distributed cloud (racks) for content
genomics, other domain science
caching, acceleration, etc.
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
25
Creating and deploying GENI racks
Ilia Baldine
RENCI
More resources / rack,
fewer racks
Rick McGeer
Fewer resources / rack,
more racks
GENI Rack
Installed at GPO – Feb 22, 2012
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
26
GENI WiMAX 2013
• 26 Wimax Base
Stations in 13 Sites
• Sliced, virtualized
and interconnected
Sponsored by the National Science Foundation
On the Air
Not On the Air
• Researcher-owned,
• researcher-operated
• 4G cellular systems
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
27
GENI WiMAX Agreements
• Agreement with Clearwire
– Clearwire and Rutgers University have signed a master agreement
– encompassing all WiMAX sites, to ensure operation in the EBS Band.
– An emergency stop procedure, in case of interference with Clearwire
service, has been agreed upon.
• GENI Mobile Virtual Network Operator (MVNO)
- Partner with Sprint and Arterra (a Sprint partner) to create and
operate an (MVNO) that serves the academic research community
- The effort is led by Jim Martin, Clemson Univ, and is underway with
a 1 year NSF EAGER
Agreements in place to
broadcast on our own frequency and to support roaming
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
28
GENI Operations
GMOC: GENI Meta-operation Center
• Keeps track of outages
• Notification system for resource reservation
• Monitors most GENI Aggregates
GMOC Google Calendar keeps track of
reservations/outages
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
29
Outline
•
•
•
•
•
GENI – Exploring future internets at scale
The GENI Concept
Building GENI
Experimental and Classroom use of GENI
GENI: An experimenter’s view
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
30
How is GENI being Used?
Research
• Future Internet
Architectures
• Software defined
networking
• Large scale evaluation of
protocols
Education
• Networking and
Distributed systems
classes
• Cloud computing classes
• WiMAX classes
As of October 2013, GENI had over 1200 users!
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
31
Three FIA Teams have Slices on GENI
NDN (demo at GEC 13)
XIA (demo at GEC15)
MobilityFirst (demo at GEC 12 & GEC18)
GENI is the only testbed that can support these teams.
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
32
Virtual Desktop Cloud
Prasad Calyam, University
of Missouri, Columbia
Program realtime loadbalancing functionality
deep into the network to
improve QoE
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
33
GENI in the Classroom – A great success!
Fall 2013:
Prasad Calyam (U. of Missouri)
Zongming Fei (U. of KY)
John Geske (Kettering U.)
Deniz Gurkan (U. of Houston)
Christos Papadopoulos (Col. State)
Violet Syrotiuk (Arizona State U.)
Zhi-Li Zhang (U. of MN)
Jeannie Albrecht (Williams College) with
students from her Spring 2012 Distributed
Systems class
Spring 2013:
Jay Aikat (U. of NC)
Rudra Dutta (NCSU)
Khaled Harfoush (NCSU)
Jelena Marasevic (Columbia U)
Parmesh Ramanathan (U. Wisc)
Violet Syrotiuk (Arizona State U.)
KC Wang (Clemson)
Michael Zink (U. of MA)
Spring 2014:
Jeannie Albrecht (Williams College)
Suman Banerjee (U. of Wisconsin)
Deniz Gurkan (U. of Houston)
Thanasis Korakis (NYU Poly)
Yaoqing Liu (Clarkson U)
Shivendra Panwar (NYU Poly)
Robert Ricci (Utah)
Fall 2012:
Violet Syrotiuk (Arizona State U)
Rudra Dutta (NCSU)
Bing Wang (U. of Conn)
Zongming Fei (U. of KY)
KC Wang (Clemson)
Fraida Fund (NY Poly)
Vasillis Maglaris (NTUA Greece)
Kaiqi Xiong (RIT)
Gaia Maselli (Sapienza University of Rome – Italy)
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
34
Upcoming GENI Events
Tutorials at Conferences
IC2E ‘14, etc
GENI Summer Camp
June ‘14
GEC19
Georgia Tech, Atlanta
March 17-19, 2014
GENI Engineering Conferences, held three times a year
Planning & discussion for experimenters, software, infrastructure
Tutorials and workshops
Travel grants to US academics for participant diversity
Sponsored by the National Science Foundation
Train-the-TA
Offered online at the start
of each semester
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
35
Outline
•
•
•
•
•
GENI – Exploring future internets at scale
The GENI Concept
Building GENI
Experimental and Classroom use of GENI
GENI: An experimenter’s view
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
36
Slice
Slice
Container for a collection of resources capable of
running experiments
– An experiment uses
resources in a slice
– Slices isolate
experiments
– Experimenters are
responsible for their
slices
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
37
Clearinghouse and Aggregates
Create & Register Slice
Slice credentials
users
projects
slice
s
clearinghouse
Aggregate Manager API
Researcher
Tool
- listResources
- createSliver
…
Aggregate
Manager
Aggregate Resources
• Clearinghouse: Manages users, projects and slices
– Standard credentials shared via custom API or new Common CH API
– GENI supported accounts: GENI Portal/CH, PlanetLab CH, ProtoGENI CH
• Aggregate: Provides resources to GENI experimenters
– Typically owned and managed by an organization
– Speaks the GENI AM API
– Examples: PlanetLab, Emulab, GENI Racks on various campuses
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
38
Resource Specifications (Rspecs)
• RSpecs: Lingua franca for describing and requesting
resources
– “Machine language” for negotiating resources between experiment
and aggregate
– Experimenter tools eliminate the need for most experimenters to
write or read RSpec
<?xml version="1.0" encoding="UTF-8"?>
<rspec xmlns="http://www.protogeni.net/resources/rspec/2"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.protogeni.net/resources/rspec/2
http://www.protogeni.net/resources/rspec/2/request.xsd"
type="request" >
<node client_id="my-node"
exclusive="true">
<sliver_type name="raw-pc" />
</node>
RSpec for requesting a single node
</rspec>
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
39
Reserving Resources using RSpecs and the AM API
•
•
•
•
Experimenter tools and aggregates talk to each other using resource
specifications (RSpecs) and the GENI Aggregate Manager API (GENI AM
API)
Advertisement RSpec: What does an aggregate have?
Request RSpec: What does the experimenter want?
Manifest RSpec: What does the experimenter have?
What do
you
have?
Experimenter
Tool
I would
like …
ListResources(…)
Advertisement RSpec
CreateSliver(Request RSpec, …)
What do I
have?
You have
Aggregate
… Manager
Manifest RSpec
ListResources(SliceName, …)
Manifest RSpec
Sponsored by the National Science Foundation
I have …
GENI Introduction – SIGCSE ‘14 – 5 March 2014
You have
…
www.geni.net
40
Reserve resources across
many aggregates in the same slice
Campus
#3
My slice contains resources
from many aggregates.
Commercial
Clouds
Backbone #1
Campus
My GENI Slice
Access
#1
Research
Testbed
Corporate
GENI suites
Backbone #2
Other-Nation
Projects
Campus
#2
One slice can contain: bare metal machines, virtual
machines, VLANs, OpenFlow resources, etc
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
41
Putting it all Together: Demo
• Demo
– Login to the GENI Portal
– Create a slice
– Create a sliver at one
aggregate
server
(VMs)
client
(VMs)
• Two computers (VMs),
connected by a LAN
– Install and run software
on the machines
– View output of software
– Delete sliver
• Experimenter tool: Flack
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
42
Start Demo
•
•
•
•
•
•
Login to GENI Experimenter Portal
Create slice
Launch Flack
Draw topology
Create sliver
Verify sliver creation was successful
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
43
The Demo Experiment in Flack
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
44
The Request RSpec
<rspec type="request” xsi:schemaLocation=“http://www.geni.net/resources/rspec/3 http://www.geni.net/resources/rspec/3/request.xsd”
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.geni.net/resources/rspec/3">
<node client_id="server" component_manager_id="urn:publicid:IDN+emulab.net+authority+cm" exclusive="true">
<sliver_type name="raw-pc">
<disk_image name="urn:publicid:IDN+emulab.net+image+emulab-ops//FEDORA10-STD"/>
</sliver_type>
<services>
<execute command="sudo /local/install-script.sh" shell="sh"/>
<install install_path="/local" url="http://www.gpolab.bbn.com/experiment-support/HelloGENI//hellogeni-install.tar.gz"/>
</services>
<interface client_id="server:if0” />
</node>
<node client_id="client" component_manager_id="urn:publicid:IDN+emulab.net+authority+cm" exclusive="false">
<sliver_type name="emulab-openvz"/>
<services>
<execute command="sudo /local/install-script.sh" shell="sh"/>
<install install_path="/local" url="http://www.gpolab.bbn.com/experiment-support/HelloGENI//hellogeni-install.tar.gz"/>
</services>
<interface client_id="client:if0” />
</node>
<link client_id="Lan">
<component_manager name="urn:publicid:IDN+emulab.net+authority+cm"/>
<interface_ref client_id="server:if0"/>
<interface_ref client_id="client:if0"/>
<property source_id="server:if0" dest_id="client:if0"/>
<property source_id="client:if0" dest_id="server:if0"/>
</link>
</rspec>
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
45
The Manifest RSpec
<rspec type="manifest" …>
<node client_id="server" component_manager_id="urn:publicid:IDN+emulab.net+authority+cm" exclusive="true"
component_id="urn:publicid:IDN+emulab.net+node+pc554" sliver_id="urn:publicid:IDN+emulab.net+sliver+95506">
<sliver_type name="raw-pc">
<disk_image name="urn:publicid:IDN+emulab.net+image+emulab-ops//FEDORA10-STD"/>
</sliver_type>
<services>
<execute command="sudo /local/install-script.sh" shell="sh"/>
<install install_path="/local" url="http://www.gpolab.bbn.com/experiment-support/HelloGENI//hellogeni-install.tar.gz"/>
<login authentication="ssh-keys" hostname="pc554.emulab.net" port="22" username="vthomas"/>
</services>
<interface client_id="server:if0" component_id="urn:publicid:IDN+emulab.net+interface+pc554:eth2"
sliver_id="urn:publicid:IDN+emulab.net+sliver+95509" mac_address="0024e87a46fb">
<ip address="10.10.1.1" type="ipv4"/>
</interface>
</node>
<node client_id="client" component_manager_id="urn:publicid:IDN+emulab.net+authority+cm" exclusive="false"
component_id="urn:publicid:IDN+emulab.net+node+pc533" sliver_id="urn:publicid:IDN+emulab.net+sliver+95505">
<sliver_type name="emulab-openvz"/>
<services>
<execute command="sudo /local/install-script.sh" shell="sh"/>
<install install_path="/local" url="http://www.gpolab.bbn.com/experiment-support/HelloGENI//hellogeni-install.tar.gz"/>
<login authentication="ssh-keys" hostname="pc533.emulab.net" port="37178" username="vthomas"/>
</services>
<interface client_id="client:if0" component_id="urn:publicid:IDN+emulab.net+interface+pc533:eth2"
sliver_id="urn:publicid:IDN+emulab.net+sliver+95510" mac_address="0262331adfd4">
<ip address="10.10.1.2" type="ipv4"/>
</interface>
</node>
<link client_id="Lan" sliver_id="urn:publicid:IDN+emulab.net+sliver+95508" vlantag="310">
<interface_ref client_id="server:if0" component_id="urn:publicid:IDN+emulab.net+interface+pc554:eth2"
sliver_id="urn:publicid:IDN+emulab.net+sliver+95509"/>
<interface_ref client_id="client:if0" component_id="urn:publicid:IDN+emulab.net+interface+pc533:eth2"
sliver_id="urn:publicid:IDN+emulab.net+sliver+95510"/>
<property
source_id="server:if0" dest_id="client:if0"/> <property
source_id="client:if0" dest_id="server:if0"/>
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
46
QUESTIONS?
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
48
Agenda
Presentation and Demo: An Introduction to GENI (45min)
Hands-on: Getting Started with GENI (45min)
Break (10min)
Hands-on: IPv4 Routing Assignment (50min)
Resources for Instructors (15min)
Miscellaneous GENI Topics & Wrap-Up (15min)
Sponsored by the National Science Foundation
GENI Introduction – SIGCSE ‘14 – 5 March 2014
www.geni.net
49