From UKLight to SuperJANET5
David Salmon
UKERNA
Copyright JNT Association 2005
Optional
Optional
1
Topics
• UKLight Background
– Current implementation in SuperJANET4
• SuperJANET5
– Research capacity & UKLight
• Connection issues
• Some open questions
Copyright JNT Association 2005
Optional
Optional
2
UKLight Background
Copyright JNT Association 2005
Optional
Optional
3
What UKLight provides
• Dedicated Network capacity for research
projects and demanding applications
• Infrastructure provides point to point circuits
(aka “Lightpaths”)
– RN to RN within UK
– Internationally to peer facilities in Europe and
USA
– Telco type equipment (SDH) which provides
switched circuits
Copyright JNT Association 2005
Optional
Optional
4
UKLight II
• Separate from JANET production
network
• Projects can undertake potentially
disruptive work without impact on the
production IP service
– Applications with large data-rates (eg 1Gb/s and
above)
– Work with new equipment, protocols etc
• NB not an IP network!
Copyright JNT Association 2005
Optional
Optional
5
Funding
• Funding – SRIF - £6.5M total
– From HEFCE (Higher Education Funding Council for
England)
– Via JISC (Joint Information Systems Committee)
• Sub-committee for Support of Research (JCSR)
• International Point of Access
– Circuits, equipment, housing, NMS, staff…
• UK Access and photonics infrastructure
– Equipment, housing…
Copyright JNT Association 2005
Optional
Optional
6
Local Research
Equipment
UK Researchers
International
Point-of-Access
Extended JANET
Development Network
Existing
connections
Proposed
connections
CA*net
StarLight
Chicago
UKLight
London
10Gb/s
10Gb/s
Abilene
10Gb/s
CERN
10Gb/s
10Gb/s
NorthernLight
10Gb/s
NetherLight
Amsterdam
UKLight – showing connections to
selected International peer facilities
Copyright JNT Association 2005
Optional
CzechLight
GEANT
Optional
7
UK National access to
UKLight
Copyright JNT Association 2005
Optional
Optional
8
Model for Extension of the SuperJANET Development Network
Northern
Ireland
Clydenet
S
NNW
T
UHI Network
FaTMAN
T
C&NL
MAN
C
Glasgow
North Wales
MAN
AbMAN
T
Edinburgh
Leeds
London
C
C
South Wales S
MAN
Copyright JNT Association 2005
Portsmouth
C
S
T
SWERN
LeNSE
Optional
EMMAN
T
C
Bristol
NorMAN
Backbone Access Router
R-PoP
T YHMAN
T
Reading
TVN
S
C
Warrington
T
Core Point of Presence
C-PoP
C
C
MidMAN
EaStMAN
EastNet
Kentish MAN
T
LMN
Key for components.
C-PoP
C
Extension to 2.5G BAR
S
Extension to 10G BAR
T
Optional
9
Development Network
Extension Model
• Uncommitted backbone bandwidth in MCI
contract for SuperJANET4
– 10G for current production backbone
– Additional 10G available but uncommitted
• Parallel to existing core
– Capacity upgrades to Regional Network access
circuits also included
Copyright JNT Association 2005
Optional
Optional
10
Principles – transmission
infrastructure
• Operating our own transmission equipment
• Use SDH switches/multiplexers to provide
circuits across the network
– e.g. 10Gb/s Æ 4 x 2.5Gb/s (or 64 x 155Mb/s)
• Projects will run ethernet / VLANs / IP etc.
over their circuits
• Links to concepts for SuperJANET5
Copyright JNT Association 2005
Optional
Optional
11
Ciena Core Director
• CDci (the small one)
• Backbone / high
density device
• Current Interface
Options
– STM-64 (10Gb/s) (16
max)
– Gigabit Ethernet
• Switch matrix with VC3
granularity (~50Mb/s)
Copyright JNT Association 2005
Optional
Optional
12
Ciena Metro Director
• Used as an edge device
• Interface options
–
–
–
–
–
–
STM-64 (10Gb/s) (max 2)
Gigabit Ethernet
10/100 Ethernet
Local ethernet switch
802.1q VLANs
Lower order SDH (below
155Mb/s)
• Switch matrix with VC3
granularity (~50Mb/s)
Copyright JNT Association 2005
Optional
Optional
13
CN3616
• Similar to K2
• Better scalability, max
7 10G SDH slots
• Flexible config
– Both resilient and nonresilient options
(economy)
• GMPLS control plane
• Development path (K2
less so)
Copyright JNT Association 2005
Optional
Optional
14
Core
Director
Copyright JNT Association 2005
Optional
Optional
15
Metro
Director
Copyright JNT Association 2005
Optional
Optional
16
ClydeNET
Glasgow
EastMAN
Edinburgh
Glasgow
C-PoP
Edinburgh
C-PoP
C&NLMAN
Lancaster
Warrington
C-PoP
NNW
Manchester
UKLight and the JANET
development network
Leeds
C-PoP
YHMAN
Leeds
MidMAN
Birmingham
Reading
C-PoP
CLRC-RAL
EastNet
Cambridge
London
C-PoP
10G
10G
10G
10G
ULCC
StarLight
Chicago
NetherLight
Amsterdam
10G
UCL
E-Research Bandwidth
ClydeNet
Glasgow
CCLRC DL
10G
C&NL MAN
Lancaster
NNW
Manchester
MidMAN
Birmingham
5G
EaStMAN
Edinburgh
7.5G
7.5G
YHMAN
Leeds
10G
10G
Glasgow Edinburgh
Leeds
Warrington
Reading
London
10G
CCLRC
RAL
Copyright JNT Association 2005
7.5G
Cambridge
7.5G
10G
Bristol
Portsmouth
Optional
Chicago
20G
ULCC
UCL
2.5G
Imperial
10G
10G
Optional
18
Amsterdam
Use by projects
• Projects expected to be funded by Research Council
or other programmes
• Central funding covers equipment interface at
backbone edge
• Projects discuss options with UKLight project
manager & submit written connection proposal
Copyright JNT Association 2005
Optional
Optional
19
Futures
• Sustainability through 2006
– JISC committees have agreed in principle
– Funding arrangements are being made
• Later this year - SuperJANET5 !
– Incorporate these concepts from ground up
– Flexible transmission platform
• Fibre / wavelength service based
• Case being made for continued funding through
lifetime of SJ5 – support (and develop) the
switched infrastructure
Copyright JNT Association 2005
Optional
Optional
20
Futures - 2
•
•
•
•
Move Ciena equipment into SJ5
Full separation from production IP service
International circuits
Level 3 contract has about 9 months to run
– Current view thereafter:
– Use GEANT path service & relinquish direct
Netherlight connection
– Retain/re-procure US Starlight connection
Copyright JNT Association 2005
Optional
Optional
21
Some key points
• Infrastructure provides dedicated pointto-point paths/channels for projects
• NOT an IP network
– Typical use is to carry ethernet traffic
(containing IP packets) from LAN to LAN
• Co-operation with local network
support staff essential – both on site
and in the regional network
Copyright JNT Association 2005
Optional
Optional
22
UKLight and SuperJANET5
SuperJANET5 details from
Jeremy Sharp at UKERNA
Copyright JNT Association 2005
Optional
Optional
23
SuperJANET5
Jeremy Sharp
Copyright JNT Association 2005
Optional
Optional
24
SuperJANET5 Spade Work
SuperJANET5 Architecture
Requirements
Reliability
improve by building in more
resilience
requirements to be served
Scalability
ability to increase bandwidth at
controllable cost
Separability
protection of interests of teaching
& learning and research sectors
Flexibility
responsiveness to additional
network service requirements
Visibility
controlled access to network
monitoring and measurement
information by end users
commodity use
IP production network
IP production network
special purpose
special
purpose
bandwidth
bandwidth
ba tel
nd co.
wid p
th rovi
ser des
vic
es
wa
ve telco
len . p
gth rov
ser ides
vic
es
tel
c
UK o . m
E
a
of RNA nag
the ha es
tra s co fibre
ns
mi ntro
ssi l
UK
on
ER
ma NA
na
l
e
ge ase
s
s
tra it an fibre
ns d t
mi
ssi he
UK
on
ER
NA
ow
ns
ma , ligh
na
ge ts an
sf
i br d
e
Collector Arc
Spectrum of options
RN
Copyright JNT Association 2005
testtestbed(s)
bed(s)
Procurement Options
RN
RN
testtestbed(s)
bed(s)
service
development
flexible transmission platform
flexible transmission platform
Service Model
RN
network
R&D
e-science
Core
Network
RN
RN
Optional
Optional
25
SuperJANET5: light-paths & PoPs
UHI
(off-net
SDH)
FaTMAN
ClydeNET
Glasgow
EaStMAN
Scolocate
AbMAN
NIRAN
NNW
NorMAN
C&NLMAN
YHMAN
Dublin
Warrington
Leeds
(optional)
MidMAN
WREN
Reading
EMMAN
TVN
LMN
Telehouse
Telecity
EastNET
London
Kentish
SWERN
Core PoP
LeNSE
Bristol
Other PoP
Core path
Regional path
Copyright JNT Association 2005
Optional
Optional
26
SuperJANET5 –
Geographic view
Copyright JNT Association 2005
Optional
Optional
27
SuperJANET5 summary
• Telco grade infrastructure, but dedicated to
JANET
• Fibre everywhere
• Designed for resilience
– Dual entry points to RNs
– All RN requirements met
• Bandwidth model that we can grow into
• Ability to add additional bandwidth at
marginal costs
Copyright JNT Association 2005
Optional
Optional
28
40Gbit/s channel testbed
• 40Gbit/s channel operation envisaged
in year 3 of SJ5
• Implement 40Gbit/s testbed channel in
year 1
• Operational Experience
• To be of use will be linked at 40Gbit/s
to the IP routers
Copyright JNT Association 2005
Optional
Optional
29
Operational management
• The supplier will manage SuperJANET5 initially
• UKERNA will be considering whether this model
continues to meet the needs of the communities
served by JANET in SuperJANET6 and beyond
• Review during SuperJANET5 and keep our options
open, i.e.
– UKERNA has the ability for a phased change to the
operational model during SuperJANET5
– This would result in more direct management by UKERNA
Copyright JNT Association 2005
Optional
Optional
30
From requirements to
solution
Scalability
• minimise single points of failure
• reduce components and complexity
3
control of costs of adding large amounts of
bandwidth when needed
3
Separability configure parallel purpose-built networks via
Flexibility
Copyright JNT Association 2005
control at transmission-level
3
ability quickly to change configuration of
network when needed
3
Optional
end-to-end across JANET
Reliability
Optional
31
Routers
• Open procurement launched in Sept
2005
• Contract signed with Lucent Technology
• Juniper T-640 routers
– Carrier Class, 40G capable
– >1000 in service world wide
Copyright JNT Association 2005
Optional
Optional
32
Juniper T-640
Copyright JNT Association 2005
Optional
Optional
33
Next steps and rollout
Rollout planning
4Q 05-1Q 06
Delivery of the SuperJANET5 core
End Apr 06
IP Routers installation complete
End May 06
Delivery of circuits to RNs
End Jun 06
Transition from SJ4 to SJ5
2H 06
SuperJANET5 rollout complete
End Dec 06
Copyright JNT Association 2005
Optional
Optional
34
SuperJANET5
• Ciena CoreStream
– Up to 96 waves per fibre
• Ciena 4200
– Up to 32 waves per fibre
• Tractable cost model for additional
wavelengths beyond the base provision
Copyright JNT Association 2005
Optional
Optional
35
SuperJANET5 Fibre Infrastructure
(simplified)
FaTMAN
UHI
Glasgow
NIRAN
Clydenet
EaStMAN
C&NLMAN
AbMAN
NorMAN
Dublin
(HEANET)
YHMAN
NNW
Leeds
Warrington
EastNet
MidMAN
EMMAN
T-House
T-City
London
Reading
TVN
LMN
WREN
Kentish
MAN
LeNSE
SWERN
Bristol
Copyright JNT Association 2005
Optional
Optional
36
FaTMAN
SuperJANET5 Research capacity
UHI
Glasgow
NIRAN
Clydenet
EaStMAN
C&NLMAN
AbMAN
NorMAN
Dublin
(HEANET)
YHMAN
NNW
Leeds
Warrington
MidMAN
Europe
USA
StarLight GEANT2 &
NetherLight
EastNet
EMMAN
T-House
T-City
London
Reading
TVN
LMN
WREN
Kentish
MAN
LeNSE
SWERN
Bristol
Copyright JNT Association 2005
Optional
Optional
37
Supporting Research
Dark-fibre for R&D
• Emerging requirement for access to dark fibre for
R&D
– E.g. UKLight – dark fibre for photonics research
• UKERNA Has the option of asking for dark fibre
Dedicated, flexible bandwidth to support research
• 10G wavelength to all RN’s as standard
• UKLight will be migrated onto SuperJANET5
• Direct links into Geant2 ‘light-path’ service
• Ability to configure additional bandwidth swiftly
Access to the network for network behaviour data
• Framework for access to network data in place
• Plans to install optical taps near busy IP routers
Copyright JNT Association 2005
Optional
Optional
38
Research Capacity
• 10 Gb/s for each RN
• Delivered to ONE of the Regional
Network Entry Points (RNEPs)
• Model is still for UNPROTECTED
capacity – single paths.
• Unlike the JANET IP service which is
protected - dual paths
Copyright JNT Association 2005
Optional
Optional
39
Edge Services
•
•
•
•
Implemented through Ciena CN4200
1Gb/s
2.5Gb/s (STM-16)
10Gb/s
– STM-64
– 10GE WAN PHY
– 10GE LAN PHY
• Others…
Copyright JNT Association 2005
Optional
Optional
40
Connecting
Copyright JNT Association 2005
Optional
Optional
41
Connecting to JANET
• Policy permits connections for
organisations involved in bona-fide
research and collaboration with
• Details can be found on the JANET
web site
– www.ja.net/services/connections/connecti
ng/index.html
Copyright JNT Association 2005
Optional
Optional
42
Connecting to UKLight
•
•
•
•
•
Brief project overview (2 sides A4 max)
Locations (sites)
Network capacities (&possible scheduling)
Project contacts
Network contacts
– Site
– Regional network
Copyright JNT Association 2005
Optional
Optional
43
Open Issues
Copyright JNT Association 2005
Optional
Optional
44
Open Issues
• Extending Research capacity to RNs which
have not had access to UKLight in SJ4
• Connection options
– GE (10GE) direct to the SJ5 transmission
equipment
– Install additional MSPPs (SDH switches) at the
new locations
– A mixture of the two
• Approach is being considered
Copyright JNT Association 2005
Optional
Optional
45
Ongoing Issues
• Edge integration/separation with campus IP
network/s
– LCG have detailed proposals being considered
– DEISA has a similar model (star overlay topology)
• Longer term
– Control plane – can we let clients request paths ?
• Tensions – spectrum of
requirements/expectations
– paths as short term vs persistent resources
– Paths engineered for economy or production
quality ?
Copyright JNT Association 2005
Optional
Optional
46
A plea !
• Encourage projects to speak to their
local network managers !
• Institution
• Regional Network
Copyright JNT Association 2005
Optional
Optional
47
Further Information
• UKLight
– www.uklight.ac.uk
– D.Salmon@ukerna.ac.uk
• SuperJANET5
– www.ja.net/SJ5/
Copyright JNT Association 2005
Optional
Optional
48