SuperJANET5 for
Science and Research
David Salmon
UKERNA
SuperJANET5 slides from Jeremy Sharp
Copyright JNT Association 2005
SuperJANET5 for Science and Research
1
Overview
• Requirements background
– Major factors
– Experiences with UKLight
• SuperJANET5
– Architecture
– Services
– UKLight Integration
Copyright JNT Association 2005
SuperJANET5 for Science and Research
2
Requirements
• Wide consultation with community
• IP service
– Greater reliability & resilience
– Higher capacity
• Dedicated capacity
– “data intensive” applications
– Network research
– Point-to-point circuits - UKLight in SuperJANET4
Copyright JNT Association 2005
SuperJANET5 for Science and Research
3
SuperJANET5 Spade Work
SuperJANET5 Architecture
Requirements
Reliability
improve by building in more
resilience
requirements to be served
Scalability
ability to increase bandwidth at
controllable cost
Separability
protection of interests of teaching
& learning and research sectors
Flexibility
responsiveness to additional
network service requirements
Visibility
controlled access to network
monitoring and measurement
information by end users
commodity use
IP production network
IP production network
special purpose
special
purpose
bandwidth
bandwidth
Core
Network
RN
RN
ba tel
nd co.
wid p
th rovi
ser des
vic
es
wa
ve telco
len . p
gth rov
ser ides
vic
es
tel
c
UK o . m
E
a
of RNA nag
the ha es
tra s co fibre
ns
mi ntro
ssi l
UK
on
ER
ma NA
na
l
e
ge ase
s
s
tra it an fibre
ns d t
mi
ssi he
UK
on
ER
NA
ow
ns
ma , ligh
na
ge ts an
sf
i br d
e
Collector Arc
Spectrum of options
RN
Copyright JNT Association 2005
testtestbed(s)
bed(s)
Procurement Options
RN
RN
testtestbed(s)
bed(s)
service
development
flexible transmission platform
flexible transmission platform
Service Model
RN
network
R&D
e-science
SuperJANET5 for Science and Research
4
SuperJANET5 –
Geographic view
Copyright JNT Association 2005
SuperJANET5 for Science and Research
5
SuperJANET5 Fibre Infrastructure
(simplified)
FaTMAN
UHI
Glasgow
NIRAN
Clydenet
EaStMAN
C&NLMAN
AbMAN
NorMAN
Dublin
(HEANET)
YHMAN
NNW
Leeds
Warrington
EastNet
MidMAN
EMMAN
T-House
T-City
London
Reading
TVN
LMN
WREN
Kentish
MAN
LeNSE
SWERN
Bristol
Copyright JNT Association 2005
SuperJANET5 for Science and Research
6
SuperJANET5 Equipment
• Transmission layer
• DWDM (Multiple
wavelength)
equipment
– Ciena CoreStream
• Up to 96 waves per
fibre
– Ciena 4200
• Up to 32 waves per
fibre
Copyright JNT Association 2005
SuperJANET5 for Science and Research
7
SuperJANET5 services
• JANET IP service
• Circuits
– (aka wavelengths /
lightpaths / lambdas…)
– UKLight style
– Optical layer
• 1GE
• 2.5 Gb/s (STM-16)
• 10Gb/s
JANET IP
UKLight - SDH
Transmission
OpticalTransmission
(DWDM)
– STM-64
– 10GE WAN/LAN PHY
Copyright JNT Association 2005
SuperJANET5 for Science and Research
10
Core Routers
• Open procurement launched in Sept
2005
• Contract signed with Lucent Technology
• Juniper T-640 routers
– Carrier Class, 40G capable
– >1000 in service world wide
Copyright JNT Association 2005
SuperJANET5 for Science and Research
11
Juniper T-640
Copyright JNT Association 2005
SuperJANET5 for Science and Research
12
JANET IP service
• Basic services as before, but higher
capacities & increased resilience
• IPv4 & IPv6 in core (dual stack)
• Multicast (including IPv6)
• Diffserv QoS – still in trial stage
– LBE, BE & EF classes
Copyright JNT Association 2005
SuperJANET5 for Science and Research
13
IP architecture
• No more BARs - Backbone Access
Routers
• Core to regional connections
established at transmission layer (SDH
for management features – alarms etc.
• RN routers peer directly with core
routers
• Large cost savings
Copyright JNT Association 2005
SuperJANET5 for Science and Research
14
IP Architecture - SJ4/5
difference – no BARs
SJ4
Copyright JNT Association 2005
SJ5
SuperJANET5 for Science and Research
15
40Gbit/s channel testbed
• 40Gbit/s channel operation envisaged
in year 3 of SJ5
• Implement 40Gbit/s testbed channel in
year 1
• Gain Operational Experience
• linked at 40Gbit/s to the IP routers
Copyright JNT Association 2005
SuperJANET5 for Science and Research
16
Operational management
• The supplier will manage SuperJANET5 initially
• UKERNA will be considering whether this model
continues to meet the needs of the communities
served by JANET in SuperJANET6 and beyond
• Review during SuperJANET5 and keep our options
open, i.e.
– UKERNA has the ability for a phased change to the
operational model during SuperJANET5
– This would result in more direct management by UKERNA
Copyright JNT Association 2005
SuperJANET5 for Science and Research
17
Research Capacity
• 10 Gb/s for each RN
• Delivered to ONE of the Regional
Network Entry Points (RNEPs)
• Model is still for UNPROTECTED
capacity – single paths.
• Unlike the JANET IP service which is
protected - dual paths
Copyright JNT Association 2005
SuperJANET5 for Science and Research
18
FaTMAN
SuperJANET5 Research capacity
UHI
Glasgow
NIRAN
Clydenet
EaStMAN
C&NLMAN
AbMAN
NorMAN
Dublin
(HEANET)
YHMAN
NNW
Leeds
Warrington
MidMAN
Europe
USA
StarLight GEANT2 &
NetherLight
EastNet
EMMAN
T-House
T-City
London
Reading
TVN
LMN
WREN
Kentish
MAN
LeNSE
SWERN
Bristol
Copyright JNT Association 2005
SuperJANET5 for Science and Research
19
SuperJANET5 summary
• Telco grade transmission infrastructure dedicated to
JANET – NO sharing !
• Suppliers: Verizon Business (was MCI) & Ciena
• Fibre everywhere
• Designed for resilience
– Dual entry points to RNs
– All RN requirements met
• Bandwidth model that we can grow into
• Ability to add additional bandwidth at marginal costs
Copyright JNT Association 2005
SuperJANET5 for Science and Research
20
UKLight and SuperJANET5
Structuring the research capacity
Copyright JNT Association 2005
SuperJANET5 for Science and Research
21
Model for Extension of the SuperJANET Development Network
Northern
Ireland
Clydenet
S
NNW
T
UHI Network
FaTMAN
T
C&NL
MAN
C
T
S
Leeds
T
Reading
London
C
TVN
C
South Wales S
MAN
Copyright JNT Association 2005
S
SWERN
Portsmouth
C
T
LeNSE
SuperJANET5 for Science and Research
EMMAN
T
C
Bristol
NorMAN
Backbone Access Router
R-PoP
T YHMAN
C
Warrington
T
Core Point of Presence
C-PoP
Edinburgh
C
MidMAN
EaStMAN
C
Glasgow
North Wales
MAN
AbMAN
EastNet
Kentish MAN
T
LMN
Key for components.
C-PoP
C
Extension to 2.5G BAR
S
Extension to 10G BAR
T
22
UKLight Equipment
• SDH transmission
systems
• ~50Mb/s granularity
• Edge interfaces:
– 155Mb/s to 10Gb/s
– Fast ethernet to
10GE
Copyright JNT Association 2005
SuperJANET5 for Science and Research
23
ClydeNET
Glasgow
EastMAN
Edinburgh
Glasgow
C-PoP
Edinburgh
C-PoP
C&NLMAN
Lancaster
Warrington
C-PoP
NNW
Manchester
UKLight and the JANET
development network
Leeds
C-PoP
YHMAN
Leeds
MidMAN
Birmingham
Reading
C-PoP
CLRC-RAL
EastNet
Cambridge
London
C-PoP
10G
10G
10G
10G
ULCC
StarLight
Chicago
NetherLight
Amsterdam
10G
UCL
E-Research Bandwidth
ClydeNet
Glasgow
CCLRC DL
10G
C&NL MAN
Lancaster
NNW
Manchester
MidMAN
Birmingham
5G
EaStMAN
Edinburgh
7.5G
7.5G
YHMAN
Leeds
10G
10G
Glasgow Edinburgh
Leeds
Warrington
Reading
London
10G
CCLRC
RAL
7.5G
Cambridge
7.5G
10G
Bristol
Portsmouth
Chicago
20G
ULCC
UCL
2.5G
Imperial
Copyright JNT Association 2005
SuperJANET5 for Science and Research
10G
10G
Amsterdam
25
C&NLMAN
Lancaster
ClydeNET
Glasgow
EastMAN
Edinburgh
UKLight on SJ5
Glasgow
C-PoP
Edinburgh
C-PoP
Before extension to
all RNs
Warrington
C-PoP
Leeds
C-PoP
YHMAN
Leeds
NNW
Manchester
MidMAN
Birmingham
London
C-PoP
EastNet
Cambridge
CLRC-RAL
Reading
C-PoP
LMN
London
GEANT SDH
10G
StarLight
Chicago
Bristol
C-PoP
Internet2 HOPI ?
NetherLight
Amsterdam
SuperJANET5 and Research
Copyright JNT Association 2005
SuperJANET5 for Science and Research
27
Supporting Research
Dark-fibre for R&D
• Emerging requirement for access to dark fibre for
R&D
– E.g. UKLight – dark fibre for photonics research
• UKERNA has the option of asking for additional dark
fibre within SJ5 contract – untested as yet
Dedicated, flexible bandwidth to support research
• 10G wavelength to all RNs as standard
• UKLight will be migrated onto SuperJANET5
• Direct links into Geant2 ‘light-path’ service
• Ability to configure additional bandwidth swiftly
(contractually within 2 to 3 weeks)
Copyright JNT Association 2005
SuperJANET5 for Science and Research
28
Access to network data
• Traffic statistics – data rates etc
• Access to network traffic itself
– Packet headers & selected “content”
• Plans to install optical taps near busy IP routers
• Framework for access to network data in place
– Legal issues
• Regulation of Investigatory Powers Act (RIPA)
• Data Protection Act (DPA)
– Projects will need to sign contracts
Copyright JNT Association 2005
SuperJANET5 for Science and Research
29
“Big” projects
Capacity at the Optical Transmission Layer
Copyright JNT Association 2005
SuperJANET5 for Science and Research
30
LCG T0/T1 (T2)
• Dedicated 10GE path from RAL to
CERN
– RAL / TVN / JANET / GEANT / CERN
– Wavelength segments at transmission
level
• No special requirements T2 to T1
– JANET IP service
Copyright JNT Association 2005
SuperJANET5 for Science and Research
31
DEISA
• European distributed HPC facility
• HPCx at DL will connect to Europe
(Frankfurt ?)
• 10GE path across:
– DL / NNW / JANET / GEANT / WIN? /
Frankfurt
Copyright JNT Association 2005
SuperJANET5 for Science and Research
32
Radio Astronomy VLBI
• Requirements being discussed
• Jodrell bank to Mainland Europe at
10Gb/s
• Probably 2007
Copyright JNT Association 2005
SuperJANET5 for Science and Research
33
Capacity & Cost Model
• JISC committee for Networking has just
agreed an approach
• UKLight capacity as common pool for
scheduled circuits in to 50Mb/s to 1Gb/s
region
• Dedicated cross-JANET capacity
(wavelenghts) for persistent static
connections may be charged to projects
Copyright JNT Association 2005
SuperJANET5 for Science and Research
34
Summary
Copyright JNT Association 2005
SuperJANET5 for Science and Research
35
Rollout timetable
Rollout planning
4Q 05-1Q 06
Delivery of the SuperJANET5 core
End Apr 06
IP Routers installation complete
End May 06
Delivery of circuits to RNs
End Jun 06
Transition from SJ4 to SJ5
2H 06
SuperJANET5 rollout complete
End Dec 06
Copyright JNT Association 2005
SuperJANET5 for Science and Research
36
From requirements to
solution
Scalability
• minimise single points of failure
• reduce components and complexity
• Ability to add greater capacity when needed
• with tight control of costs
Separability configure parallel purpose-built networks via
Flexibility
Copyright JNT Association 2005
3
3
control at transmission-level
3
ability quickly to change configuration of
network when needed
3
SuperJANET5 for Science and Research
end-to-end across JANET
Reliability
37
Basic SuperJANET5 services
• JANET IP
– pervasive
• Point-to-point capacity
(circuits/lightpaths…)
– 50Mb/s -> n x 1Gb/s (UKLight / SDH layer)
– n x (1, 2.5, 10)Gb/s (Optical Transmission
layer)
Copyright JNT Association 2005
SuperJANET5 for Science and Research
38
SuperJANET5 Summary
• Flexible, High-Capacity transmission
infrastructure supporting:
– Highly Reliable and Resilient IP backbone
– Point-to-point circuit/wavelength services
• Direct on to the transmission Layer
• Finer grained through the SDH (UKlight) layer
• Foundation for JANET growth and
evolution over the next 5/7 years
Copyright JNT Association 2005
SuperJANET5 for Science and Research
39