GMPLS optical networks
Malathi Veeraraghavan
Professor
Charles L. Brown Dept. of Electrical & Computer Engineering
University of Virginia
mvee@virginia.edu
ETRI, Korea
Feb. 2009
GMPLS: Generalized MultiProtocol Label Switched networks
(MPLS, SONET, WDM, SDM, VLAN)
1
Outline
• Telcom “transport network”
• Cheetah vs. Dragon Approach
– Theoretical concepts
• GMPLS networks
– Technologies, off-the-shelf switches, control-plane
protocols
• State of the art on different applications
& networks
– Commercial
– Research-and-Education (REN) networks
2
Spectrum of services
Leased lines are used to connect IP routers.
Network that offers leased line service is called
Leased line “transport network” by telcom industry
IP
Circuit technologies: time/frequency division multiplexing
PDH: T1, T3
switch: Digital
Cross Connect
(DCS)
SONET/SDH:
OC3-OC768
Switch: SONET/SDH
crossconnects
DWDM: OTU1-OTU3
Switch: optical WDM
crossconnects
Packet technologies: virtual circuit switches
ATM
MPLS
Carrier-grade Ethernet
All the above: Data-plane technologies
3
IP and leased line
service deployment
Leased line
Circuit or virtual circuit (VC) switch
Internet service provider or
enterprise owns IP routers
Telco service provider
(transport network) owns
circuit/VC switches
IP Router
Management plane
(in transport network)
(1) Admins use
Web interface
to request leased
line creation
(2) NMS computes path with
available bandwidth
Network
management
system
Customer
edge device
(3) NMS sends
provisioning signals to
each switch on path
using SNMP/CLI/TL1
Customer
edge device
Customer
edge device
Customer
edge device
Customer
edge device
switch controller
has minimal software
(SNMP agent, CLI/TL1
parser)
Spectrum of services
New service: rapid provisioning
Leased line
Verizon Bandwidth-on-Demand (BoD)
IP
6
Management plane
+ control plane
Network
management
system
(1) Admins use
Web interface
to request leased
line creation
(3) TL1/CLI to
edge node
(2) NMS still computes path
with available bandwidth
Customer
edge device
Customer
edge device
(4) hop-by-hop
distributed signaling
for circuit/VC
provisioning
Customer
edge device
Customer
edge device
Customer
edge device
switch controllers
have RSVP-TE software
Progress made in
telcom industry
• Data-plane progress
– Excellent: interesting new switching technologies being
invented for transport networks
• Control-plane
– Switch controllers implement RSVP-TE capable of
distributed route computation and admission control
– But only provisioning phase is distributed
• Requests for circuits/VCs are still handled
through management plane with involvement of
administrators even in “Dynamic” scenarios
• Why is this an issue?
– Limits access to “transport” circuit/VC network
8
Difference with R&E thinking
Scheduler
(1) application software
running at end host
initiates request for
circuit/VC
(3) TL1/CLI to
edge node
external
controller
(2) scheduler computes path
with available bandwidth
(4) hop-by-hop
distributed signaling
for circuit/VC
provisioning
(3a)
Enterprise
switch controllers
have RSVP-TE software
(3a) configure router to filter
packets for long flow on to circuit/VC
Effect of opening up access to
circuit/VC “transport” network
• Application software running on end hosts deep
inside enterprises can access dynamic circuit/VC
services of the backbone transport network
• Circuit network reach does not need to extend all
the way to the desktop
• With additional high-speed line from enterprise
edge router into transport network, high-speed
access can be enabled for short durations
• High call volume of setup/release: automatic
generation of calls by software
• New applications!
10
Spectrum of services
New services
Leased line
Verizon BoD
Book-ahead (BA) mode
• call duration specified
Current solution:
• centralized per-domain path
computation/admission control
Low call handling volume
OSCARS/DRAGON
eScience
10G POTS
IP
Plain Old Telephone Service (64kbps)
Immediate-Request (IR) mode
• unspecified call duration
Low call setup overhead
( holding times can be shorter)
Distributed path computation/admission
control
High call handling volume
CHEETAH
11
Outline
• Telcom “transport network”
 Cheetah vs. Dragon Approach
– Theoretical concepts
• GMPLS networks
– Technologies, off-the-shelf switches, control-plane
protocols
• State of the art on different applications
& networks
– Commercial
– Research-and-Education (REN) networks
12
Observations
• "Many e-science experiments ... are
optimized to provide maximum throughput
to a few facilities, as opposed to moderate
throughput to millions of users, which is
the raison d'etre for commercial
networks."
• Networks should be scalable:
– Metcalfe's statement: Value of a network
increases exponentially with the number of
users
13
Key difference between
DRAGON and CHEETAH
• DRAGON focus:
– For eScience
• Small number of users
• High throughput to a few facilities
– Transfer technology to Internet2
• Implement and deploy software for book-ahead reservations
and circuit provisionining by teaming with ESNet and DANTE
• CHEETAH focus:
– General-purpose commercial network goal to bring GMPLS
services to millions of users
– But not with just moderate throughput, but also high-rate
– Analyze GMPLS network bandwidth sharing modes (BA + IR)
– Implementation: IR
14
Background
• Types of switches
• Types of bandwidth-sharing modes
– IP networks vs connection-oriented
(GMPLS) networks
• Tradeoffs in GMPLS network modes
– Immediate-request mode (e.g., Plain Old
Telephone Service)
– Book-ahead (advance-reservation)
15
Types of switches
Multiplexing technique on
data-plane links
Admission
control in
control plane?
Circuit
Packet switch (PS)
switch (CS) - header based
- position
based
(port, time,
lambda)
Connectionless (CL)
- no admission control
Not an
option
Connection-oriented (CO)
- admission control
e.g.,
Virtual-circuit
telephone
e.g., MPLS, ATM,
SONET
PBBTE
WDM, SDM
GMPLS network
switches
e.g., Ethernet
16
Difference between
bandwidth (BW)-sharing modes
• In connectionless networks (e.g., IP)
– Pre-1988 IP network:
• Just send data without reservations or any mechanism to adjust
rates  congestion collapses in the Internet in the 80s!
– Van Jacobson's 1988 contribution:
• Added congestion control to TCP
• Sending TCP adjusts rate
– TCP congestion-control pros and cons:
• Pros: Proportional fairness and high utilization
• Cons: No rate guarantees & No temporal fairness (job seniority)
• In connection-oriented networks (e.g., GMPLS)
– Key: Admission control
17
Bandwidth sharing modes
in GMPLS networks
•
Can execute admission control in two ways:
– Bufferless (immediate-request)
– With buffers (book-ahead is effectively the same as having buffers to hold
calls to start in the future)
•
Immediate-request: M/G/m/m model
– m: number of channels on a link (servers)
– if all channels are occupied, reject call
•
Book-ahead: M/G/m/p model
–
–
–
–
•
p: max number in system: advance-reservation window K = p/m timeslots
waiting time and call blocking
K cannot be : need to block calls if per-server traffic intensity can be > 1
Or engineer the system so per-server traffic intensity ≤ 1
Difference:
– Not as the names suggest: IR calls need bandwidth immediately
•
Misconception: BA with book-ahead time of “now”  IR  NOT TRUE
– Instead, call duration needs to be specified to support BA mode
– For IR mode, applications do not need to specify duration
18
IR mode: M/G/m/m
ErlangB formula



Pb 
: offered traffic load in Erlangs
: call arrival rate
1/: mean call holding time
/m: per-server traffic intensity
m: number of circuits
Pb: call blocking probability
ub: utilization
 m / m!
m
k

 / k!
k 0
(1  Pb )  
ub 
m
For a 1% call blocking probability, i.e., Pb = 0.01

1
10
100
m
ua
4
17
117
24.8%
58.2%
84.6%
If m is small, high
utilization can only be
achieved along with high
call blocking probability
19
Comparison of Immediate-Request (IR)
and Book-Ahead (BA) schemes
• Example
– To achieve a 90% utilization
with a call blocking probability
less than 10%
• BA-First schemes are needed
when m < 59
– To achieve a 90% utilization
with a call blocking probability
less than 20%
• BA-First schemes are needed
when m < 32
U: utilization
K: number of time periods in
advance-reservation window
IR m=10, U = 80%: PB = 23.6%
m=100, U = 80%: PB = 0.4%
Link capacity C = 10Gbps
m = 10 if per-call allocation = 1Gbps
BA m=10, K=10, U = 80%: PB = 0.4%
20
Bandwidth sharing mechanisms
in GMPLS networks
Needed if per-call
Bandwidth sharing mechanisms
circuit rate is a large
fraction of link capacity
(e.g., 1Gbps circuits on a
10Gbps link, m = 10)
Book-ahead
Immediate-request
call duration specified
BA-n/BA-First
session-type requests: BW, duration
BA-n
Users specify a set of
n call-initiation time
options
unspecified call duration
VBDS
(Varying-Bandwidth Delayed Start)
BA-First
Users are given first
available timeslot
data-type requests: file size
(can assign any rate, even vary
rate in different time ranges)
X. Zhu, Ph.D. Thesis, UVA, http://www.ece.virginia.edu/mv/html-files/students.html
21
Relate BW sharing modes to
network types
Bandwidthsharing
mechanisms
Book-Ahead (BA)
(high rate per call)
Immediate-Request (IR)
(moderate rate per call)
eScience
networks
(small number of
users)
Very large (TB, PB) file
transfers need high-BW and
long holding time + remote viz.
need to reserve other
resources such as displays.
Centralized control-plane
solution sufficient, since call
durations are high
(OSCARS+DRAGON)
What applications?
Centralized control-plane
(DRAGON)
general-purpose
networks
(large number of
users)
To assign 1Gb/s on 10Gb/s per
file transfer, m=10, need BA
mode. Need distributed
control-plane solution: small
durations implies high call
arrival rate at same util (load)
Moderately large (100MB, GB)
file transfers assigned
moderate-BW (100-300Mbps)
(CHEETAH)
22
References on bandwidth sharing modes
• IR mode for file transfers with moderate-BW allocation
(100Mbps on 10Gbps link)
– X. Fang and M. Veeraraghavan, “On using a hybrid architecture for file
transfers,” acceptedto IEEE Transactions on Parallel and Distributed
Systems, 2009.
– X. Fang and M. Veeraraghavan, On using circuit-switched networks for
file transfers,” in IEEE Globecom, New Orleans, LA, Nov. 2008.
– X. Zhu, X. Zheng, and M. Veeraraghavan, "Experiences in implementing
an experimental wide-area GMPLS network," IEEE Journal on Selected
Areas in Communications (JSAC), Apr. 2007.
– M. Veeraraghavan, X. Fang, and X. Zheng, “On the suitability of
applications for GMPLS networks,” in IEEE Globecom, San Francisco,
CA, Nov. 2006.
• Large-scale deployment of BA mode: (mean waiting time,
blocking rate)
– X. Zhu and M. Veeraraghavan, "Analysis and Design of Book-ahead
Bandwidth-Sharing Mechanisms," IEEE Transactions on
Communications, Dec. 08.
– X. Zhu, M. E. McGinley, T. Li, and M. Veeraraghavan, "An Analytical
Model for a Book-ahead Bandwidth Scheduler," in IEEE Globecom
Washington, DC, Nov. 2007.
Heterogeneous rate allocation
23
Is an opportunity being missed if distributed IR
bandwidth sharing mode is not explored?
• Yes. Four reasons:
1. Increase end-to-end rate relative to IP service; possible in the presence
of admission control (programmable patch panels to share ports)
2. Enable the creation of large-scale circuit/VC networks with moderaterate circuits that can support a brand new class of applications
• economic value for the networking industry
3. A "reservations-oriented" mode of networking to complement today's
connectionless Internet
• analogy: airlines complement roadways
4. Alternative pricing models for bandwidth
•
•
•
Leased lines and IP service are at two extremes
Usage based pricing
Dedicated (moderately high) bandwidth for short durations instead of low bandwidth for
all time
24
To increase end-to-end rate
• Problem:
– WDM allows 40Gbps/channel with 80 channels/port
– But, end-to-end rate is still on the order of tens of Mbps
– Why? Access link rates: both for enterprises and residences
• Inter-domain link cost:
– Internet2 charges $250K/year for a 1Gbps Ethernet connection
– Why so high? High router port cost and no sharing
• Router port cost:
– One-port 10Gbps or ten-port 1Gbps interface card costs $150-200K
• 2007 data for local access links in US:
– 1.5M T1, 183K T3, 44K OC3, 21K OC12, 2K OC48 and 2.5K OC192
• Add leased lines to terminate on a space-division switch for moderate rate, connect to sub-Gbps ports
– With admission control for ports, connect high-speed link for short
duration for single flows based on request from file-transfer apps.
25
What "brand new class of applications?"
• Moderate-bandwidth
– Video: “Harry Potter” application,
multiple-cameras/automated cameraman
for video-tel/conf, distance-learning,
virtual reality
– Cloud computing, gaming
– Teleoperations, telemedicine
• High-bandwidth, short-held calls
– Web, P2P, storage, CDN file transfers
26
Outline
• Cheetah vs. Dragon Approach
– Theoretical concepts
GMPLS networks
– Technologies, off-the-shelf switches, controlplane protocols
• State of the art on different
applications & networks
– Commercial
– Research-and-Education (REN) networks
27
GMPLS related technologies
• GMPLS networks
– Data-(user-) plane protocols
• packet-switched: MPLS, VLAN Ethernet (PBBTE)
• circuit-switched: SONET/SDH, WDM, SDM (space div. mux)
– Control-plane protocols:
• RSVP-TE: signaling protocol
• OSPF-TE: routing protocol
• LMP: link management protocol
• Internetworking: Ethernet-over SONET/MPLS/WDM
– GFP, VCAT, LCAS for SONET/SDH
– PWE3 for MPLS networks
– Digital wrapper for OTN
28
Why internetworking?
• GMPLS networks do not exist as standalone
entities as data-sourcing end hosts do not have
MPLS, SONET, WDM NICs
• Instead they need to be internetworked with
Ethernet interface cards:
– Common usage: IP layer internetworking
• IP routers with Packet-over-SONET (PoS) interfaces
– Newer usage: Ethernet layer internetworking
• Ethernet over MPLS/SONET/WDM/SDM
– Port-mapped
– VLAN-mapped (probably not supported with SDM)
• Ethernet interface could be on hosts or routers
29
Off-the-shelf GMPLS switches
Vendor/system
Data-plane
Control-plane
Cisco 12000 series
MPLS switching;
PWE3 Ethernet-over-MPLS
RSVP-TE, OSPF-TE
Juniper T640
MPLS switching;
PWE3 Ethernet-over-MPLS
RSVP-TE, OSPF-TE
Sycamore SN16000
SONET switching;
GFP/VCAT Ethernet-overSONET (EoS)
RSVP-TE, OSPF-TE
for SONET circuits;
no support for EoS
Ciena CDCI
SONET switching;
GFP/VCAT EoS
Proprietary signaling/routing
protocols
Movaz (now Adva)
RayExpress
WDM switching;
G.709 Eth-over-WDM
RSVP-TE, OSPF-TE
Calient
SDM switching;
Ethernet-over-fiber
RSVP-TE, OSPF-TE (?)
Force10 E600
Ethernet VLAN switching
None
30
GMPLS control-plane scope
• RSVP-TE and OSPF-TE do not have parameters to
support admission control for BA calls
– e.g., call duration, optional desired call-initiation time
• Strengths:
– Distributed routing and call setup/release functions for
high-call volume IR calls
– OSPF-TE (in each switch controller)
• Loading conditions shared only intra-area
• Link-state + Distance vector (even basic OSPF)
– RSVP-TE (in each switch controller)
• Route computation and admission control
– CSPF can be done only intra-area by ingress switch
– Any switch could be an ingress switch – hence highly scalable
• Switch fabric configuration (i.e., provisioning)
31
Control-plane for BA calls
• Run an external scheduler to perform
– path computation and admission control for future start time
– add authentication and authorization
• Centralized scheduler - one per domain
• Inter-domain scheduler-to-scheduler protocol:
– Abstracted topology exchange
– Reservation phase (path computation + admission control)
– Signaling phase (not clear why RSVP-TE is not used interdomain)
• Intradomain
– Provisioning phase: RSVP-TE is used
– OSPF-TE data is read out from switch controllers by scheduler for intradomain path computation
• Not a scalable solution to support short-duration, high-BW calls
32
Outline
• Cheetah vs. Dragon Approach
– Theoretical concepts
• GMPLS networks
– Technologies, off-the-shelf switches, controlplane protocols
State of the art on different
applications & networks
– Commercial
– Research-and-Education (REN) networks
33
Spectrum of services
New services
Leased line
Verizon BoD
eScience
10G POTS
IP
34
Commercial uses
• Semi-permanent MPLS virtual circuits
– Traffic engineering
– Voice over IP
• QoS concerns: telephony has a 150ms oneway delay requirement (with echo cancellers)
– Business or service provider interconnect
• interconnecting geographically distributed
campuses of an enterprise
• interconnecting wide-area routers of an ISP
service provider
35
Traffic engineering (TE)
• Since BGP and OSPF routing protocols mainly
spread reachability information, routing tables are
such that some links become heavily congested
while others are lightly loaded
• MPLS virtual circuits are used to alleviate this
problem
– e.g., NY to SF traffic could be directed to take an MPLS
virtual circuit on a lightly loaded route avoiding all paths
on which more local traffic may compete
• This is an application of MPLS VCs without
bandwidth allocation
36
Business or service provider
interconnect (leased lines)
• Multiple options:
– TDM circuits (traditional private line, T1, T3,
OC3, OC12, etc.)
– Ethernet private line
• point-to-point (Ethernet over MPLS/SONET/WDM)
• VPNs (called Virtual private LAN service)
– MPLS VPNs
– WDM lightpaths
– Dark fiber
37
Dynamic circuits/virtual circuit
(GMPLS control-plane)
• Commercial:
– fast restoration
• circuit/VC setup delay significant
– rapid provisioning
• Verizon: Bandwidth on Demand (Just-in-Time
Provisioning)
• AT&T: Shared mesh networks
– Customer Applications for dynamic network configuration
» Key industries: Financial, Media & Entertainment
» Corporate Utility Backbone Networks (e.g. reconfigure
for disaster recovery)
» Distribution of real-time content (e.g., Video)
• Level3: Vyvx service
38
Spectrum of services
New services
Leased line
Verizon BoD
eScience
10G POTS
IP
Book-ahead (BA) mode
• call duration specifie d
Current solution:
• centralized per-domain path
computation/admission control
Low call handling volume
OSCARS/DRAGON
39
Research & Education
(G)MPLS networks
•
•
•
•
Internet2’s Dynamic Circuit network
NSF-funded DRAGON
DOE's ESnet - Science Data Network
DOE's Ultra Science Network (USN)
40
Internet2 DWDM network
Infinera
DWDM system
http://events.internet2.edu/speakers/speakers.php?go=people&id=178
Rick Summerhill talk (10/11/2007)
41
Internet2
Dynamic Circuit (DC) network
Ciena CD-CI
Eth-SONET
switch
http://events.internet2.edu/speakers/speakers.php?go=people&id=178
Rick Summerhill talk (10/11/2007)
42
Internet2 IP-routed network
IP-router-to-router links on one wavelength
SONET switch-to-switch links on another wavelength
Ciena CD-CI
Eth-SONET
switch
Juniper
T640 IP router
http://events.internet2.edu/speakers/speakers.php?go=people&id=178
Rick Summerhill talk (10/11/2007)
43
Equipment at each PoP
http://events.internet2.edu/speakers/speakers.php?go=people&id=178
Rick Summerhill talk (10/11/2007)
44
Control-plane software
(for DC network)
• OSCARS implemented in InterDomain
Controller (IDC) - one per domain
– Abstracted topology exchange
– Interdomain scheduling
– Interdomain signaling (for provisioning)
• DRAGON (intradomain control-plane)
– Used in Internet2’s DC network
– Intradomain routing, path computation,
signaling (for provisioning)
45
OSCARS
• On-demand Secure Circuits and Advance Reservation
System (OSCARS)
• DOE Office of Science and ESnet project
• Co-development with Internet2
• Web Service based provisioning infrastructure, which
includes scheduling, AAA architecture using X.509
certificates
– Extended to include the DICE IDCP
– Reservations held in SQL database
• Recall no support for book-ahead in GMPLS control protocols
• http://www.es.net/oscars/index.html
http://www.csm.ornl.gov/workshops/NetworkingResearchChallenges/agenda.html
Talk by Tom Lehman, Sep. 28, 2008
46
DRAGON
•
Washington DC metro-area network:
– Adva (old Movaz) WDM switches and Ethernet switches (G.709)
•
Control-plane software:
– Network Aware Resource Broker – NARB
• Intradomain listener, Path Computation
– Virtual Label Swapping Router – VLSR
• Implements OSPF-TE, RSVP-TE
• Run on control PCs external to switches (since not all switches implement
these GMPLS control-plane protocols)
• Communicates with switches via SNMP, TL1, CLI to configure circuits.
– Client System Agent – CSA
• End system software for signaling into network (UNI or peer mode)
– Application Specific Topology Builder – ASTB
• User Interface and processing which build topologies on behalf of users
• Topologies are a user specific configuration of multiple LSPs
http://dragon.east.isi.edu
47
Open Source
DCN Software Suite
• OSCARS (IDC)
– Open source project maintained by ESNet and Internet2
– Uses WDSL, XML, SQL database to store reservations
– Reservations accepted with 1 minute granularity
• DRAGON (DC)
– NSF-funded Open source project maintained by USC ISI
EASTand MAX
• Version 0.4 of DCNSS current deployed release
– https://wiki.internet2.edu/confluence/display/DCNSS
• DCN workshops offered for training:
– http://www.internet2.edu/workshops/dcn/index.html
http://www.csm.ornl.gov/workshops/NetworkingResearchChallenges/agenda.html
Talk by Tom Lehman, Sep. 28, 2008
48
DICE IDCP
•
•
•
•
Dante, Internet2, CANARIE, ESNet
http://www.controlplane.net
IDCP: InterDomain Controller Protocol
wsdl - web service definition of message
types and formats
• xsd – definition of schemas used for
network topology descriptions and path
definitions
http://www.csm.ornl.gov/workshops/NetworkingResearchChallenges/agenda.html
Talk by Tom Lehman, Sep. 28, 2008
49
InterDomain Controller (IDC)
Protocol (IDCP)
•
The following organizations have implemented/deployed systems which are
compatible with this IDCP
–
–
–
–
–
–
–
–
–
–
–
–
•
Internet2 Dynamic Circuit Network (DCN)
ESNet Science Data Network (SDN)
GÉANT2 AutoBahn System
Nortel (via a wrapper on top of their commercial DRAC System)
Surfnet (via use of above Nortel solution)
LHCNet (use of I2 DCN Software Suite)
Nysernet (use of I2 DCN Software Suite)
LEARN (use of I2 DCN Software Suite)
LONI (use of I2 DCN Software Suite)
Northrop Grumman (use of I2 DCN Software Suite)
University of Amsterdam (use of I2 DCN Software Suite)
DRAGON Network
The following "higher level service applications" have adapted their existing
systems to communicate via the user request side of the IDCP:
–
–
–
LambdaStation (FermiLab) – CMS project on Large Hadron Collider
TeraPaths (Brookhaven) - ATLAS project on Large Hadron Collider
Phoebus
http://www.csm.ornl.gov/workshops/NetworkingResearchChallenges/agenda.html
Talk by Tom Lehman, Sep. 28, 2008
50
Heterogeneous Network Technologies
Complex End to End Paths
Example: DRAGON
AS 1
Example: Internet2 DC
Example: ESNet SDN
AS 2
IP Control Plane
IP Control Plane
AS 3
IP Control Plane
VLSR
VLSR
Ethernet over WDM
End
System
Ethernet Segment
VLSR Established VLAN
Ethernet over
SONET
Ethernet
Lambda Switch
SONET Switch
Router MPLS LSP
End
System
Ethernet Segment
VLSR Established VLAN
Router
http://events.internet2.edu/speakers/speakers.php?go=people&id=178
Rick Summerhill talk (10/11/2007)
51
IDCP operation
Route selection,
admission control
centralized per
domain at IDC
•
•
•
Advance reservation request and circuit provisioning at scheduled time:
•
End user signals IDC with a reservation request
•
Authenticate requester and check authorization
•
Request reservation (create time, bandwidth, VLAN tag)
•
Signaling: creation of circuit (automatic or in response to message to IDC)
Topology exchange: interdomain (abstracted topology information)
Monitoring
52
http://hpn.east.isi.edu/dice-idcp/dice-idcp-v1.0/idc-protocol-specification-may302008.doc
Intra-domain operations
• Using DRAGON in Internet2 DCN
– NARB does intra-domain path computation after
collecting routing information by listening to OSPF-TE
exchanges between VLSRs
– These intradomain paths are provided to IDC for use
during resource scheduling (upto 3 path options are
considered)
– 5 VLSRs serve 22 CD-CIs: “subnets of CD-CIs”
– In Signaling phase, VLSR sends TL1 command to edge CDCI, which initiates proprietary hop-by-hop signaling to
configure circuit through subnet
53
GOLE: GLIF open lightpath exchange
54
DOE networks
• ESnet and Science Data Network (SDN)
– OSCARS: an advance-reservation system
– Science Data Network: MPLS network
• UltraScience Network
–
–
–
–
Research network for DoE labs
GbE and SONET (Ciena CD-CI)
Centralized scheduler for advance-reservation calls
5-PoP network: ORNL, Atlanta, Chicago, Seattle,
Sunnyvale
– Connections to Fermi Lab, PNNL, SLAC, CalTech
• Lambdastation: CMS project
– Between Fermi Lab and Univ. of Nebraska
55
Spectrum of services
New services
Leased line
Verizon BoD
eScience
10G POTS
IP
Plain Old Telephone Service (64kbps)
Immediate-Request (IR) mode
• unspecified call duration
Low call setup overhead
( holding times can be shorter)
Distributed path computation/admission
control
High call handling volume
CHEETAH
56
NSF-funded CHEETAH network
GbEthernet and SONET
UVa
TN PoP
SN16000
CUNY
GbE
GbE
OC192 Control GbE/
10GbE
card
card
card
NCSU
End hosts
GbEs
GbE
OC-192
NC PoP
GA PoP
SN16000
End
GbE GbE/
Control OC192
10GbE card
hosts
cards
card
ORNL
GbE
SN16000
OC192 Control GbE/ GbE
10GbE
card
card
End
card
OC-192
hosts
GbE
57
Sycamore SN16000
SONET switch with GbE/10GbE interfaces
GaTech
Networking software
• Sycamore switch comes with built-in GMPLS
control-plane protocols:
– RSVP-TE and OSPF-TE
• We developed CHEETAH software for Linux
end hosts:
– circuit-requestor
• allows users and applications to issue RSVP-TE
call setup and release messages asking for
dedicated circuits to remote end hosts
– CircuitTCP (CTCP) code
http://www.ece.virginia.edu/cheetah/
58
CHEETAH network usage
End Host
CHEETAH
software
IP-routed
network
DNS client
RSVP-TE module
Application
End Host
CHEETAH
software
DNS client
SONET circuitswitched network
RSVP-TE module
TCP/IP
Application
TCP/IP
NIC 1
CTCP/IP
NIC 2
Circuit
Gateway
Circuit
Gateway
NIC 1
NIC 2
CTCP/IP
• Bandwidth-sharing mode:
•
•
Immediate-request mode (blocked calls fall back to IP path)
Heterogeneous rate allocation under high loads:
• higher BW for large files than for small files
• Applications:
•
Common file transfers (web, P2P, CDN, storage)
•
•
attempts circuits for large files (if blocked, use IP-routed path)
use IP-routed path for small files
59
End-to-end call setup delay
measurements
•
Delays incurred in setting up a circuit between host zelda1 (in Atlanta, GA) and
host wuneng (in Raleigh, NC) across the CHEETAH network
Circuit type
End-to-end
circuit setup
delay (s)
Processing delay for
Path message at
the NC SN16000 (s)
Processing delay for
Resv message at
the NC SN16000 (s)
OC-1
0.166103
0.091119
0.008689
OC-3
0.165450
0.090852
0.008650
1Gb/s EoS
1.645673
1.566932
0.008697
Round-trip signaling message propagation plus emission delay between GA SN16000 and NC SN16000:
0.025s
•
Observations:
–
–
–
Setup delays for SONET circuits (OC1, OC3) are small (166ms)
Setup delays for Ethernet-over-SONET (EoS) hybrid circuits are much higher (1.6s)
(no standard; proprietary implementation)
Signaling message processing delays dominate end-to-end circuit setup delays
60
Conclusions
•
•
•
•
•
•
•
Need BA service if the per-call bandwidth allocation is a significant
fraction of link capacity (1Gbps on a 10Gbps link)
Key differentiator between BA and IR: BA calls specify call
duration
GMPLS control-plane protocols are designed for distributed
scalable implementation of IR service
GMPLS control-plane protocols do not have parameters to support
BA service (e.g., call duration in RSVP-TE)
BA service with centralized schedulers per domain suitable for long
call-duration eScience applications (small number of users)
To support BA service for general-purpose applications, e.g., large
file transfers in Web, P2P, storage, CDN, with short call durations,
need to design scalable control-plane solution for BA calls
Four reasons to develop an IR service with moderate per-BW calls
61
Item 7: Related Items on
Future Internet
• US National Science Foundation (NSF) interest
– CyberPhysical Systems to create an "Internet of Things“
– "Network Science"
– Ty Znati (Director of Computer Network Systems
division in the NSF's CISE directorate):
http://www.csm.ornl.gov/workshops/NetworkingResearch
Challenges/agenda.html
• GENI effort to build a global network for
research:
– http://www.geni.net/
62