Connect. Communicate. Collaborate
Pan-European BW-on-Demand:
Dream Pipes or Pipe Dream?
Michael Enrico, DANTE
Network Engineering & Planning Manager
ESLEA Technical Collaboration Meeting
e-Science Institute, Edinburgh, 20-21 June 2006
Agenda
•
•
•
•
•
•
Connect. Communicate. Collaborate
A bit of a retrospective (aka some ramblings on BoD)
GÉANT2 – the network
GÉANT2 – the service portfolio
GÉANT2 – the R&D activities (mainly JRA3)
Other BoD-related R&D activities (briefly)
Conclusions (will BoD remain in our dreams?)
Ramblings on BoD 1
Connect. Communicate. Collaborate
• BoD is already with us!
• The PSTN is a successful BoD service today and has been
for many years
• Limited BW (64kbps, analogue UNI) (nightmare pipes!?)
• Evolved into (narrowband) ISDN
• Good signaling, more versatile BW (1 B-channel up to 2M
PRI)…
• …but still not broadband!
Ramblings on BoD 2
Connect. Communicate. Collaborate
• Remember the Broadband ISDN (B-ISDN)?
• ATM was considered the most likely technology candidate
• Lots of nice features…
–
–
–
–
versatile BW capabilities
versatile QoS (in principle)
well-defined signalling & routing
nice OAM
• …BUT complex, proved too expensive, missed the >622M
boat, etc
• The B-ISDN never really delivered
• However, a lot of the signalling and routing aspects live
on…in (G)MPLS (more precisely G.ASON)
Ramblings on BoD 3
•
•
•
•
•
•
•
•
Connect. Communicate. Collaborate
IP networks provide BoD!
How…? Open a socket!
OK, network needs to be over provisioned or…
…DiffServ capable to deliver BW guarantees
Over provisioned BE IP may still be jittery
Again DiffServ helps out here
But DiffServ needs careful management – i.e. Premium IP
QoS can be controlled but what about IP addressing,
routing and security?
• Points to VPNs, or better still real PNs!
Ramblings on BoD 4
Connect. Communicate. Collaborate
• GMPLS/G.ASON are standards bodies’ approaches to
provision of BW (circuits) on demand
• (or at least under streamlined management control)
• Mostly re-use tried and tested Internet protocols for routing
(e.g. OSPF-TE) and signalling (e.g. RSVP-TE/CR-LDP)
– except G.7713.1 (based on PNNI)
• BUT they do not address (advance) reservation!
The 7 Generations of
European R&E Networks
Connect. Communicate. Collaborate
Services
Underlying
Technology
Datagram
&
IP
PDH
circuits
IP
&
ATM
ATM
over
SDH
IP only
waves
hybrid
fibre
IXI (1991-93)
EuropaNET (1993-96)
EuropaNET 2 (1996-97)
TEN-34 (1997-98)
TEN-155 (1998-2001)
GEANT (2001-05)
GEANT 2 (2006-??)
0.01
0.1
1
10
100
Link Speed Mbps
1000
10000
100000
GÉANT (in 2005)
Connect. Communicate. Collaborate
Some Facts & Figures…
Typical GÉANT POP
1 or 2 x E3 to STM-64/10GE
Offers:
Many IP services
Emulated L2 services
To NREN
Primary
IP access
Leased lambdas
(2.5 or 10G)
or
SDH circuits
(STM-1 or STM-4)
to neighbouring
POPs
Backup
IP access
•
M160
Switched
GE/FE
LAN
WS WS
•
•
•
•
•
•
TS
ISDN/PSTN
•
•
•
•
•
•
•
21 POPs
serves 29 European NRENs
16 x 10G lambda
13 x 2.5G “lambda”
+ some lower speed links
Juniper M160 & M40 routers
in POPs with
NREN accesses at up to
10Gbps (+ backup)
12G to North America
POP in NY
connections to R&E
networks in:
USA, Canada, Japan & RSA
AND Mediterranean
AND Latin America
AND (soon) South East Asia
GÉANT Services
Connect. Communicate. Collaborate
• Best effort (unicast) IPv4 and IPv6 transit
– Between European NRENs
– To/from R&E backbones in other world regions
• Multicast IPv4
• Premium IP (destination aware DiffServ)
• Less-than-Best-Effort IPv4
• Multicast IPv6 (almost there)
• L2VPN (P2P EoMPLS)
• Distributed (routed) testbed
Connect. Communicate. Collaborate
So, what’s new (for GÉANT)?
GÉANT(1) → GÉANT2
Connect. Communicate. Collaborate
Some New Facts & Figures…
•
•
•
•
•
•
•
•
•
•
•
•
•
25 POPs (+4)
to serve 30 NRENs (+1)
11600 km of fibre
140+ ILA sites
50+ x (own) 10G lambdas
9 x (leased) 10G lambda
8 x 2.5G (leased) “lambda”
+ some lower speed links
Juniper T640, M160 & M40
routers in POPs with
NREN accesses at up to 10Gbps
(+ backup) + P2P
4 x 10G to North America
POP in NY
connections to other R&E
networks as before…
Abilene, ESnet, CA*net4, SINET,
TENET, EUMEDCONNECT,
RedCLARA, TEIN2 (coming)
GÉANT2
(not just a network!)
Connect. Communicate. Collaborate
But also an INTEGRATED INFRASTRUCTURE INITIATIVE - I3
NETWORKING ACTIVITIES (NAs)
SERVICE ACTIVITIES (SAs)
(Human networks)
(focus on services, not infrastructure)
Management
Network Service Provisioning
Dissemination and outreach
Network Operation and Support
User support (NRENs)
JOINT RESEARCH ACTIVITIES (JRAs)
Focus on academic, technological research
GÉANT2 Project Partners
vice
r
e
S
and
Joint Research Activities
Connect. Communicate. Collaborate
JRA1 (perfSONAR) – Advanced Network Monitoring
•
•
•
JRA2 – Security
Versatile web services based framework for the
JRA3 – Bandwidth Allocation & Reservation
exchange
of network monitoring information
• Of the network
JRA4
– Technology & Service Testing
MP
and
MA
services
• Developing
best practices
CSIRTs
• Develop &a promoting
connection-oriented,
E2E,fornon-contended,
Started
at JRA5
L3,toolkit
now
adding lower
layers
– Mobility
AAI
• Develop
P2P
Ethernet
(EPL) &
service
• Build distributed testbed for use by NRENs and other
• Looking
atSA3
using
netflow
data QoS
– End-to-end
• Multi-domain
(hence multi-technology)
JRAs
• advance
Developreservation
and promote roaming access infrastructure • With
• Undertake
technology testing programme
eduGAIN
• More
on
this
later…
• mopped
Make
“quality”
moreaspects
ubiquitous
• •Also
up technical
use(such
of
Develop all-in-one
AAI for use bysurrounding
new services
• Develop
PIP provisioning
system (AMPS) and
so-called
Cross-Border
Fibre (CBF)
as monitoring,
PIP provisioning,
BoD, etc)
(encourage) roll-out across many NREN domains
• Operate enhanced monitoring infrastructure as
“production” service
• Develop and operate PERT
GÉANT2:
A hybrid infrastructure
n × 10 Gbps
10 Gbps
Primary IP
Peering
Connect. Communicate. Collaborate
Backup IP
Peering
To NREN
GÉANT2
POP
A
IP Router
n × lambda
to neighbouring
POP
Switch
GÉANT2
POP B
To NREN
n × lambda
to neighbouring
POP
Transport Equipment
Connect. Communicate. Collaborate
Alcatel 1626 LM (Light Manager)
Alcatel 1678 MCC (Metro Core Connect)
GÉANT2 Services
•
•
•
•
Connect. Communicate. Collaborate
Continue to support usual IP services
– BE, LBE, PIP, mcast (v4), v6, mcast (v6)
Enhance these with:
– implementation of new features as required and available
• e.g. embedded RP for IPv6mcast (now done)
– improved performance monitoring (JRA1 & SA3) via perfSONAR
– more automated PIP provisioning (SA3) via AMPS
– network security best practice (JRA2)
Continue to support emulated L2 P2P services
– L2VPNs for lower capacity [“virtual lightpath”] support
Add portfolio of new L1 and L2 P2P services
– cost-effective support for high-capacity [“lightpath”] services
– up to 10G (for now)
– emphasis placed on support for Ethernet framing
– try to provide translational P2P service support
• e.g. native gigE to GFP VC-4-7v over SDH
– explore various protection/restoration modes
Services over GÉANT2
Connect. Communicate. Collaborate
More resilient IP service
POP C
POP A
Primary IP access
Backup IP access
GÉANT2
POP B
Features:
• Ethernet
• Ethernet VLAN
• SDH
POP D
Services over GÉANT2
Connect. Communicate. Collaborate
Point-to-point GE (GE access)
POP C
POP A
Essentially an
implementation of
ITU-T G.8011.1
EPL service
Type 1
Features:
• uses GFP/VCAT
• GE port per instance
• more dynamic
• sub 1G possible
GÉANT2
POP B
POP D
Services over GÉANT2
Connect. Communicate. Collaborate
Point-to-point GE (10GE access)
POP C
POP A
VLAN X
VLAN Y
Essentially an
implementation of
ITU-T G.8011.2
EVPL service
Type 1
Features:
• uses GFP/VCAT
• 10GE port (supporting
multiple instances)
• use 802.1Q VLAN tags as IDs
• sub (or >) 1G possible
GÉANT2
POP B
POP D
Services over GÉANT2
Connect. Communicate. Collaborate
Point-to-point GE (10G SDH access)
POP C
POP A
VCG X
VCG Y
GÉANT2
POP B
Features:
• uses GFP/VCAT
• 10G SDH port
• GFP done in NREN
• sub 1G possible
POP D
Services over GÉANT2
Connect. Communicate. Collaborate
Managed wavelength service
POP C
POP A
GÉANT2
POP B
Features:
• 10G only
• SONET/SDH or
10GE LAN PHY
• static
• 10GE is “full-rate”
POP D
Connect. Communicate. Collaborate
Rollout update…
From → To
Connect. Communicate. Collaborate
0 km of dark fiber
→
~12,000 km of dark fiber
22 sites
[main PoP stations]
→
200 sites
[including amplifier sites]
~100 systems
→
~400 systems to manage
less than 40 e2e
services
→
hundreds of them
[including optical e2e paths]
GÉANT2 as of 9th June 2006
Connect. Communicate. Collaborate
Connect. Communicate. Collaborate
North Atlantic Connectivity
The JRA3 Activity of GN2
•
•
Connect. Communicate. Collaborate
A ‘Joint Research Activity’ investigating the provision of ‘Bandwidth on
Demand’ services to the NREN community
The environment:
– Multi-domain
– Multiple technologies
• GFP over SDH, L2 MPLS VPN, Native Ethernet
– Requirements for:
• end-to-end non-contended capacity
• a standardized interface for service requests at end-points
• service level indication to end-users
• advance reservation (scheduled)
JRA3 approach
•
•
•
Connect. Communicate. Collaborate
The goal is to streamline the inter-domain setup of end-to-end paths
– shorten the provisioning time
– reduce the amount of human intervention
– using existing (NREN/aggregation) networks by an overarching method
– automate the process step-by-step; focus on inter-domain coordination
process
Service specification
– End-to-end, connection oriented service for provisioning non-contended
capacity
– Layer 1, 2 technologies
– AAI, policies
– Single point of entry for users/applications
PROTOTYPE: focus on provisioning of a deterministic non-contended
bandwidth pipe between two 1Gigabit Ethernet access ports over multiple
domains that employ different technologies
JRA3 architecture
•
•
•
Connect. Communicate. Collaborate
Inter-Domain manager (IDM) - Domain manager (DM) - Standardized interfaces
JRA3 will provide:
– The IDM module
– Reference implementation(s) for the DM (human NOC, Ethernet-based, etc.)
Each domain participating in BoD service provisioning needs to operate an IDM and honor the IDM-DM
and IDM-IDM interfaces
Distributed approach
Connect. Communicate. Collaborate
(1)
(6)
(4)
(10)
(3)
(9)
Inter-domain
path-finding
(2)
(5)
(8)
(7)
Inter Domain Manager
User interface
JRA3 BoD system
User interface
Inter-Domain Manager
User access
module
Resource
modelling
Connect. Communicate. Collaborate
AAI
Policy
module
Request
handling logic
Inter-domain
pathfinder
Domain Manager
User interface
Inter-Domain Manager
User access
module
Resource
modelling
AAI
Policy
module
Request
handling logic
Inter-domain
pathfinder
Domain Manager
DM pathfinding
DM pathfinding
Technology Proxy
Resource
modelling
AAI
Request
handling logic
Policy
module
Inter-domain
pathfinder
Domain Manager
DM pathfinding
Technology Proxy
NMS
Data plane
Inter-Domain Manager
User access
module
GE domain
Technology Proxy
GMPLS signalling
SDH domain
IP domain
Client
equipment
L2 MPLS VPN
•
•
Native Ethernet
GFP over SDH
Client
equipment
A set of automated procedures for the required non-technology
specific inter-domain negotiations
Can be considered as a ‘bandwidth broker’, but it is more than this
Why an
Inter-Domain Manager?
Connect. Communicate. Collaborate
• The effort to provision end-to-end Bandwidth on Demand
services in the European scenario requires specific
developments in inter-domain collaboration
• Splitting intra-domain management functionalities from
inter-domain ones in separate modules, allows multidomain R&D to proceed autonomously and focus on this
less standardized area
• At the same time, it allows to leverage existing interdomain managers through wrappers and interfaces,
exploiting a modular approach
• This effort can provide solid experience for brokering
services other than Bandwidth on Demand
IDM multi-domain issues
•
Connect. Communicate. Collaborate
The IDM faces a number of challenges related to its multi-domain scope:
– domain independence for resource usage policies and technological
choices
– a service and network abstraction schema to describe implementation
over very different networks
• a schema which allows to clearly specify which type of service is
requested
• a network abstraction which allows inter-domain information
exchange independently of the underlying technologies
– advance reservation
– multi-domain path finding procedure
– monitoring
– Authentication and Authorization
IDM: Overall functionality
•
•
•
•
•
•
•
Connect. Communicate. Collaborate
Ingress point to the BoD system
– It receives and processes BoD reservation requests from users or from other
IDMs of neighboring domains
Selection of the chain of domains to be involved in each end-to-end path
establishment for serving a reservation request
– Inter-domain path-finding
– Based on topology and traffic engineering information
Pre-reservation and commit process between all IDMs along the end-to-end path
Interaction with the AAI service, to authenticate the identity of BoD service
requestors and authorize access to the BoD service
Credit management sub-module for the controlled
allocation of bandwidth resources
Accounting and logging sub-module
Policies for allocation of BoD resources and for
management of the BoD service
IDM internals
Connect. Communicate. Collaborate
IDM Prototype implementation Connect. Communicate. Collaborate
• Objectives
– to validate design and architectural assumptions
– to define potential risk points and bottlenecks
– to test IDM reservation procedures and communication
schemas
• Modular implementation
• Web-services’ technology
IDM Prototype
Connect. Communicate. Collaborate
• Minimal functionality
– IDM prototype is designed to provide minimal required
functionality for test purposes. Therefore, the following
limitations are introduced:
• Simplified network abstraction schema
• Pathfinder and DM processing are pre-defined
manually
• Authentication is based on X.509 certificates for SSL
connections and authorization is based on „always
allow” policy
• Data life-time is limited to application run-time
IDM Prototype
Connect. Communicate. Collaborate
•
IDM
User Access
&
Main Request
Handling Module
End User
Resource
Modeling &
Pathfinder
Access
IDM
AAI
XML
paths
XML
domain
data
Network engineer
DM
IDM prototype features
– accepts UNI service request
(request, cancel, status)
– NNI communication is implemented,
so domains can agree on
reservation parameters and
schedule resources’ booking
– performs reservation process at
inter-domain level (inter-domain link
capacity check, VLAN numbers,
path costs validation)
– the pathfinder supports IDM with
manually pre-defined inter-domain
paths
– DM supports IDM with manually predefined information about domain
topology
IDM Prototype
Connect. Communicate. Collaborate
• Future development after prototype tests:
– design and implementation of DM functionality (may
include manual provisioning)
– design of network resources’ representation at the IDM
and DM level
– extensions to the current transaction mechanism (data
life-time will exceed application run-time)
– full implementation of pathfinder functionality
– AAI extensions, incorporating the federated model of
JRA5 activity in GN2 project
Intra-domain provisioning
Connect. Communicate. Collaborate
• Manual intra-domain configurations and provisioning for
the establishment of the intra-domain segments of the endto-end path
• Intra-domain provisioning design to accommodate
– Domains that have a G.ASON/GMPLS CP “out of the
box” e.g. Generic MPLS Routing Engine (distributed
control plane in their Alcatel 1678 MCC OXC)
– Domains operated via NMS
– Domains that may decide to adopt proprietary
Bandwidth Brokers
Domain Manager
Connect. Communicate. Collaborate
• Intra-domain modules, implemented in later phases, will
comprise the so-called BoD service Domain Manager (DM)
– Processes intra-domain provisioning requests from the
IDM wrt technology-specific issues
– Provides to the IDM intra-domain topology updates
– Includes one or more technology proxy sub-modules for
the configuration of the network elements/interaction
with the local NMS/interaction with the local control
plane
Technology Stitching
Why is it needed?
•
•
•
•
Connect. Communicate. Collaborate
Different network technologies exist across NRENs and this is not
expected to change in the near future
Need to provide a homogenous method to interconnect domains
The technology stitching sub activity starts with
determining/collecting (manual) procedures how to stitch
technologies between two domains
Automated Technology Stitching is the aim
Technology Stitching
Network Technology Types
•
•
Connect. Communicate. Collaborate
Based on existing NREN technologies
– SONET/SDH
– Ethernet based:
• Native Ethernet
• L2 MPLS VPN
– DiffServ technologies
• PIP
• IP MPLS QoS
14 different interconnection scenarios in total identified
Technology Stitching
The testing process
•
•
•
•
Connect. Communicate. Collaborate
Simple manual stitching testing between one technological domain and another
1 GE circuit implemented over an SDH
SAMPLE CASE
network using GFP and VCAT, and across
an Ethernet domain using trunk connections
implemented as a VLAN
Pre-Conditions
– Sufficient BW available on the SDH
network to create the full-rate
connection
– Host A and B have an IP address in the
same (sub) network range
– Link connectivity is present
Actions
– Configure the circuit on the SDH
network as a VC4-7v or a VC4-8c.
– Configure the trunk circuit on the
Ethernet network
– Testing connectivity: ping hosts
– Testing BW achieved: use a test set
and loopbacks on either end of the
circuit
JRA3 thinks it is also
fundamental to work on …..
Connect. Communicate. Collaborate
• Looking into developments in standardization bodies (OIF, IETF)
• Collaboration with Internet2, CANARIE and ESnet (already ongoing for 2 years)
• Liaison with projects: MUPBED, NOBEL, VIOLA, DRAGON,
HOPI, UCLPv2, ...
• Specifying requirements for a pan-European scale test-bed to
test JRA3 prototypes and modules
• General information at:
http://www.geant2.net/server/show/nav.756 (to be updated)
• Collecting user/application requirements on BoD service
– Please send your feedback to: sevasti@grnet.gr
Afterthoughts
Connect. Communicate. Collaborate
• Today is the summer solstice!
• So, is all of the above (at least the BoD part) just a
Midsummer Night’s Dream?
The reality of BoD today
Connect. Communicate. Collaborate
Afterthoughts
•
•
•
•
•
•
•
Connect. Communicate. Collaborate
Today is the summer solstice!
So, is all of the above (the BoD part) just a
Midsummer Night’s Dream?
Answer: Hopefully not! (Although there are many sceptics)
But remember the R in JRA3…
Efforts are underway to reduce lead times (TL)
1 day < TL < 1 week should be possible by the end of GN2 (Aug 08)
– provided the NRENs do their part
– and assuming ends (beyond the NREN) are “aligned”
However, regional & campus will still be problematic
– E2E may well mean edge-to-edge for some time to come
Connect. Communicate. Collaborate
Thank you for listening
Any questions?