HOPI / Dynamic Services Update
Rick Summerhill, Internet2
Director, Network Research, Architecture, and Technologies
Tom Lehman, ISI East
Jerry Sobieski, Mid Atlantic Crossroads
John Vollbrecht, Internet2
Spring Member Meeting
April 24, 2007
Alexandria, VA
Introduction
• Dynamic Circuits work
• Intra-domain work focuses on Ciena
CoreDirectors
• Inter-domain work and collaborations with the
International Community
• Panel
• John Vollbrecht - Summary of Overall Status
• Tom Lehman - Control Plane discussion and demo
• Jerry Sobieski - DRAGON/HOPI status and
Workshops
HOPI and Dynamic Circuit
Services - Status Summary
John Vollbrecht jrv@internet2.edu
Session Structure
• Present status of Internet2 Dynamic Circuit
Capabilities and collaborations with other
infrastructure providers
• Demonstrate operation of new DCS services
in “prototypical” multidomain environment
• Describe a way that RONS, campuses and
others can participate in multidomain
infrastructure
Status of Internet2 DCS capabilities
• Internet2 has two dynamic circuit infrastructures
• HOPI - Hybrid Optical/Packet Infrastructure
• DCS - Dynamic Circuit Services being deployed
• Both are planned to be maintained for different
purposes for the intermediate term
• Both use control software “DRAGON” that has been
developed by the HOPI Testbed Support Center
•
Adapted by ISI-East to work with Ciena Infrastructure
• Both use Infinera/Level3 infrastructure to provide
connectivity between Access Points
DCS Infrastructure - status
Global Dynamic Circuit Infrastructure
Many organizations are developing Circuit Services sometimes called Waves or Lambdas
• Internet2, ESnet /SDN, NLR, GEANT, CANARIE,
JGN2, others
Internet2 collaborates with several group
managing/describing dynamic circuit services - DCS
• DICE [Dante(GEANT), Internet2, CANARIE and
Esnet
• GLIF [Global Lambda Integrated Facility]
GEANT
February 15, 2007
Outline of JGN2 Network
Sapporo
USA (Chicago)
Sendai
Thailand
(Bangkok)
Kanazawa
Kitakyushu
Hiroshima
Fukuoka
Naha
Osaka
Okayama
Koch
i
Nagano
NICT
Koganei
Headquarters
NICT Keihanna Nagoy
a
Branch
NICT Tsukuba
Research
Center
Singapore
Akihabara
Toky
o
*IX: Internet eXchange
AP: Access Point
GLIF MAP August 2005
Standards Bodies
•
•
•
•
OGF Open Grid Forum
IETF (CCAMP)
IEEE (PTB)
OIF (ASON)
Dynamic Circuit Exchange Points
• International networks interconnect
• Currently Exchange Points provide ability to switch
connections from one provider to another
• In future the expectation is that these will provide
the ability to switch under program control
• For Exchange points that do L1 or L2 GLIF has
coined the name GOLE
• At Exchange points where switching (which may
be single interconnection) providers must
exchange information that allows interconnection
under program control
GOLES listed on GLIF web page
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
* AMPATH - Miami
* CERN - Geneva
* CzechLight - Prague
* HKOEP - Hong Kong
* KRLight - Daejoen
* MAN LAN - New York
* MoscowLight - Moscow
* NetherLight - Amsterdam
* NGIX-East - Washington D.C.
* NorthernLight - Stockholm
* Pacific Wave (Los Angeles) - Los Angeles
* Pacific Wave (Seattle) - Seattle
* Pacific Wave (Sunnyvale) - Sunnyvale
* StarLight - Chicago
* T-LEX - Tokyo
* UKLight - London
Interernet2 Dynamic Circuit
Connectors
• HOPI and DCS provide
access to Dynamic Circuits
in logically identical ways
• Access ports allow circuits to
be multiplexed over
backbone to other access
circuits
• Access may include control
plane interaction
• Connector locations are
shown on map in previous
slide
Global Dynamic Circuits
Specific Projects
• Implement basic DRAGON control
plane to run on Ciena infrastructure
• See demo
• Testing of Ethernet paths
• Testing with Spirent has been completed,
will be documented in next few weeks
• Working to test ethernet between different
hardware, at 10G and 1G.
Projects
• Work with ESnet to create common module that runs
with both ESnet and DRAGON to support
authentication of users and trust between domain
controllers
• Authentication, Authorization, Scheduling
• Status• demonstrated interoperability at control level
• Demonstrated ability to make Ethernet path that extends
through HOPI(dynamic) and ESNet(static)
• Expect to be entirely dynamic in next week or so
• Plan is to integrate this into DRAGON
• Hope to make this code generally available
OSCARS
SC07 protocol demo plans
• University of Amsterdam
• Plan to collaborate on developing specific
capabilities in DRAGON
• Will establish permanent connection
between HOPI and UvA
• Will do SC07 demo together
• Token signaling
• Topology sharing
Collaborations
DICE
• “Stitching” project to describe data layer
interconnections between segments of a PTP
path
• Topology exchange
• Reviewing schemas
GLIF
• Collaborating on developing control plane
interoperation between domains
DVTS
TeraPaths
• TeraPath sites use QoS
within site
• Between Sites they may
create special path for some
flows
• MPLS path added over IP
• Or create dynamic ckt
between routers
• Looking into how TeraPaths
controllers can configure
routers to send specific flows
over newly created ckt
Issues in finding and authorizing
Segments for dynamic interdomain
PTP circuits
• Networked topology
• Topology exchange, path computation
• Types of exchange - OSPF/BGP
• Grid/ VO approach
• Resource allocation that includes computation,
storage and networking
Implementation approach is to create
“InterDomain Contoller” that can participate in
either approach
Status Summary
• Making progress in developing and deploying core
infrastructure
• Collaborating with dynamic circuit community on how it will
develop
• Working to get users / RONS/ Campuses connected
to core
• Currently we are at the start of an operational global
infrastructure
•
•
future is being worked out
users and user needs will shape future development
Dynamic Circuit Services
Control Plane Overview
April 24, 2007
Internet2 Member Meeting
Arlington, Virginia
Tom Lehman
University of Southern California
Information Sciences Institute (USC/ISI)
Chris Tracy
University of Maryland
Mid-Atlantic Crossroads (MAX)
Outline
• Internet 2 Dynamic Circuit Services
Architecture
• Control Plane Overview
• Control Plane Messaging Example
• I2 DCS Demonstration
I2 DCS Control Plane Objectives
• Multi-Service, Multi-Domain, Multi-Layer,
Multi-Vendor Provisioning
• Basic capability is the provision of a “circuit” in
above environment
• In addition, need control plane features for:
• AAA
• Scheduling
• Easy APIs which combine multiple individual
control plane actions into an application specific
configuration (i.e., application specific
topologies)
Multi-Domain Control Plane
The (near-term) big picture
•
•
•
•
•
Multi-Domain Provisioning
Interdomain ENNI (Web Service and OIF/GMPLS)
Multi-domain, multi-stage path computation process
AAA
Scheduling
GEANT
TDM
Internet2 Network
RON
RON
Dynamic Ethernet
ESNet
Domain Controller
Ctrl Element
Ethernet
SONET Switch
Router
Dynamic Ethernet
TDM
Data Plane
Control Plane Adjacency
LSP
IP Network (MPLS, L2VPN)
Internet2 Dynamic Circuit Services
(DCS)
I2 HOPI: Force10 E600
10 Gigabit Ethernet
10 Gigabit Ethernet
1 Gigabit Ethernet
I2 DCS: Ciena CoreDirector
10 Gigabit Ethernet
1 Gigabit Ethernet
or SONET/SDH
OC192 SONET/SDH
DCS Demonstration
Actual Topology
HOPI East
Internet2 DCS
HOPI
Central
NEWY
CHIC
CHIC
NEWY
CLEV
Internet2
Office
Ann
Arbor
WASH
DRAGON
PITT
PHIL
MCLN
ARLG
WASH
•
•
HOPI Network Partitioned to mimic RONS connected to edge of Internet2 DCS
Provisioning across subset of currently deployed Ciena CoreDirectors
Force10 E600 HOPI Ethernet Switch
Ciena Core Director SONET Switch
Raptor ER-1010 Ethernet Switch
Client “Service” View
IntraDomain
Service Request
Dynamically Provisioned Dedicated
Resource Path (“Circuit”)
Source Address
Destination Address
Bandwidth (50 Mbps increments)
VLAN TAG (None | Any | Number)
User Identification (certificate)
Schedule
CSA can run on the
client or in a
separate machine
(proxy mode)
Domain
Controller
b
1
csa
2
csa
Client A
a
Client B
Ethernet Mapped SONET
or
SONET Circuits
Internet2 DCS
•Items 1,2 represent service
request/approval
•Items a,b represent service
instantiation (signaling)
VLSR
Domain
Controller
Switch
Fabric
What is the Internet2 DCS Service?
• Physical Connection:
• 1 or 10 Gigabit Ethernet
• OC192 SONET
• Circuit Service:
• Point to Point Ethernet (VLAN) Framed SONET Circuit
• Point to Point SONET Circuit
• Bandwidth provisioning available in 50 Mbps increments
• How do Clients Request?
• Client must specify [VLAN ID|ANY ID|Untagged], SRC Address, DST
Address, Bandwidth
• Request mechanism options are GMPLS Peer Mode, GMPLS UNI
Mode, Web Services, phone call, email
• Application Specific Topology is an XML request for one or more
individual circuits
• What is the definition of a Client?
• Anyone who connects to an ethernet or SONET port on an Ciena Core
Director; could be RONS, GIgaPops, other wide area networks, end
systems
InterDomain
•From a client perspective, an InterDomain provisioning
is no different than IntraDomain
•However, additional work for Domain Controllers
Domain
Controller
Domain
Controller
Domain
Controller
CSA
CSA
RON Dynamic Infrastructure
Ethernet VLAN
RON Dynamic Infrastructure
Ethernet VLAN
Internet2 DCS
Ethernet Mapped SONET
Provisioning Flow
GUI
AST
Domain
Controller
Domain
Controller
Domain
Controller
AAA
AAA
AAA
A
XML
A
A
A
Need more
work on AAA,
Scheduling
Flexible Edge
Mappings
(port(s), tag, untag)
3
3
1
2
4
NARB
5
VLSR
RON Dynamic Infrastructure
Ethernet VLAN
RON Dynamic Infrastructure
Ethernet VLAN
Internet2 DCS
Ethernet Mapped SONET
1. Service Request
A. Abstracted topology exchange
2. Path Computation Request
3. Recursive Per Domain Path Computation/Scheduling Processing
4. Path Computation/Scheduling Response (loose hop route object returned)
5. Service Instantiation (Signaling)
(includes loose hop expansion at domain boundaries)
VLSR
(Virtual Label Switching Router)
• GMPLS Proxy
• (OSPF-TE, RSVP-TE)
• Local control channel
• CLI,TL1, SNMP, others
• Used primarily for ethernet
switches
• Provisioning
requests via CLI,
XML, or ASTB
CLI Interface
One NARB per Domain
Integration Core Director Domain into the Endto-End Signaling
VLSR
uni-subnet
LSR
upstream
signaling flow
data flow
CoreDirector
LSR
downstream
CoreDirector
Ciena Region
CD_a
•
CD_z
Signaling is performed in contiguous mode.
•
•
•
subnet signaling flow
Single RSVP signaling session (main session) for end-to-end circuit.
Subnet path is created via a separate RSVP-UNI session (subnet session),
similar to using SNMP/CLI to create VLAN on an Ethernet switch.
The simplest case: one VLSR covers the whole UNI subnet.
•
•
•
VLSR is both the source and destination UNI clients.
This VLSR is control-plane ‘home VLSR’ for both CD_a and CD_z.
UNI client is implemented as embedded module using KOM-RSVP API.
DCS Demonstration
Logical Topology
Ann Arbor
RON Central
Internet2 DCS
TDM Switch
Ethernet Switch
End System
RON East
DRAGON
Dedicated Layer 2 Network
Site to Site
Ann Arbor
RON Central
Internet2 DCS
RON East
DRAGON
• Dynamically set up Site to Site dedicated layer 2
networks
• End Sites attachment is flexible:
• One Port (untagged or tagged)
• Multiple Ports (untagged or tagged)
Dedicated Layer 2 Network
System to System Service Connections
Ann Arbor
RON Central
Internet2 DCS
RON East
DRAGON
• Dynamically set up dedicated layer 2 host to
host connection
• End System termination point is flexible:
• One “circuit” (untagged or tagged)
• Multiple “circuits” (tagged)
• reflected as multiple virtual interfaces on the end system
Application Specific Topology Example
Ann Arbor
RON Central
Internet2 DCS
RON East
DRAGON
• Application specific topologies refer to the:
• automatic set up of multiple provisioned paths and
• coordinated end system application control
• The above example show three systems connecting to a single
“server/processing node” as might be required for:
• data repository access
• content distribution infrastructure
• data streaming to a centralized processing center
Demo
• Graphical User Interface
• Ciena Core Director
• Monitoring and Control
• “NodeManager”
Timeslot Map
Network Utilization Monitor
DCS Demonstration
Actual Topology
HOPI East
Internet2 DCS
HOPI
Central
NEWY
CHIC
CHIC
NEWY
CLEV
Internet2
Office
Ann
Arbor
WASH
DRAGON
PITT
PHIL
MCLN
ARLG
WASH
•
•
HOPI Network Partitioned to mimic RONS connected to edge of Internet2 DCS
Provisioning across subset of currently deployed Ciena CoreDirectors
Force10 E600 HOPI Ethernet Switch
Ciena Core Director SONET Switch
Raptor ER-1010 Ethernet Switch
Dedicated Layer 2 Network
Site to Site
Ann Arbor
RON Central
Internet2 DCS
RON East
DRAGON
• Dynamically set up Site to Site dedicated layer 2
networks
• End Sites attachment is flexible:
• One Port (untagged or tagged)
• Multiple Ports (untagged or tagged)
Site to Site Provision Request
DRAGON ARLG to Ann Arbor
Thank You
extras
DRAGON Control Plane
Key Components
• Network Aware Resource Broker – NARB
• Intradomain listener, Path Computation, Interdomain Routing
• Virtual Label Swapping Router – VLSR
• Open source protocols running on PC act as GMPLS network
element (OSPF-TE, RSVP-TE)
• Control PCs participate in protocol exchanges and provisions
covered switch according to protocol events (PATH setup, PATH
tear down, state query, etc)
• Client System Agent – CSA
• End system or client software for signaling into network (UNI or
peer mode)
• Application Specific Topology Builder – ASTB
• User Interface and processing which build topologies on behalf
of users
• Topologies are a user specific configuration of multiple LSPs
Key Control Plane Features
(for Connection Control)
• Routing
• distribution of "data" between networks. The data that needs to
be distributed includes reachability information, resource usages,
etc
• Path computation
• the processing of information received via routing data to
determining how to provision an end-to-end path. This is
typically a Constrained Shortest Path First (CSPF) type
algorithm for the GMPLS control planes. Web services based
exchanges might employ a modified version of this technique or
something entirely different.
• Signaling
• the exchange of messages to instantiate specific provisioning
requests based upon the above routing and path computation
functions. This is typically a RVSP-TE exchange for the GMPLS
control planes. Web services based exchanges might employ a
modified version of this technique or something entirely different.
Key Control Plane Key Capabilities
• Domain Summarization
• Ability to generate abstract representations of your domain for making
available to others
• The type and amount of information (constraints) needed to be included
in this abstraction requires discussion.
• Ability to quickly update this representation based on provisioning
actions and other changes
• Multi-layer “Techniques”
• Stitching: some network elements will need to map one layer into
others, i.e., multi-layer adaptation
• In this context the layers are: PSC, L2SC, TDM, LSC, FSC
• Hierarchical techniques. Provision a circuit at one layer, then treat it as
a resource at another layer. (i.e., Forward Adjacency concept)
• Multi-Layer, Multi-Domain Path Computation Algorithms
• Algorithms which allow processing on network graphs with multiple
constraints
• Coordination between per domain Path Computation Elements
Inter-Domain Topology
Summarization
Full Topology
Semi-topo (edge nodes only)
Maximum Summarization
- User defined summarization level maintains privacy
- Summarization impacts optimal path computation but allows
the domain to choose (and reserve) an internal path
Interdomain Path Computation A Hierarchical
Architecture
Summarized/Abstract InterDomain Topoloy (A single link state flooding area)
NARB
w/RCE
NARB
w/RCE
NARB
w/RCE
IntraDomain Topoloy - Area 2
IntraDomain Topoloy - Area 1
•
•
•
IntraDomain Topoloy - Area 3
NARB summarizes individual domain topology and advertise it globally using link-state
routing protocol, generating an abstract topology.
RCE computes partial paths by combining the abstract global topology and detailed local
topology.
NARB’s assemble the partial paths into a full path by speaking to one another across
domains.