TransPAC-Pacific Wave 100G
Jennifer M. Schopf, PhD
Director, International Networks
Indiana University;
PI NSF TransPAC IRNC award
jmschopf@iu.edu
Professor Ron Johnson
CEO, Pacific Northwest Gigapop,
Professor, University of Washington
ronj@uw.edu
Professor Jun Murai
Dean and Professor
Keio University,
Founder, WIDE
Supported by the National Science Foundation
IRNC funded Backbones and Exchange Points
ACE
Pacific Wave
Pacific Wave
Backplane
Starlight
Pacific Wave
TransPAC
SXTransport
Pacific Islands
Atlantic Wave
AmLight Exp
TransPAC-Pacific Wave 100G
• Dedicated 100Gb/s wavelength between
Pacific Wave in Seattle and Tokyo, Japan
• Network fabric provided by Pacific Northwest
Gigapop
• Points of Presence in Tokyo at WIDE/T-REX
and at Tata
• Pacific Wave Points of Presence in USA are in
Seattle, Los Angeles & Chicago (@ StarLight)
• 5 year agreement
4
Seattle
Equinix Bldg
PW
Brocade
T
A
T
A
TransPAC/Pacific Wave Circuit
T
A
T
A
(provided by Pacifi c NorthWest GigaPoP)
TP2-SEAT-TP-TOKY-100GE-01522
PW
Brocade
Tata Space - not installed yet
100G Span
100G Peering
Huawei
Huawei
metro
TransPAC
MLXe-4
TransPAC
Seattle
Netflow
2x10g
2x10g
perf
Tokyo
PW
Brocade
100G
DENV
SUNN
WIDE
Huawei
ne5000
STAR
100G
AS22388 BGP Peering
Huawei
100G
Otemachi Bldg
Pacific Wave
Network
ALBU
MX480
10G
perf
100G
100G
APAN
JGN-X
TP-LOSA-TOKY-10GE-1
IU
10
G
10G
MLXe-4
TransPAC LA
LOSA
100G
100G
ELPA
Endpoint: Seattle
• TransPAC/Pacific Wave is connected at 100g to Pacific Wave >100g
distributed peering fabric which has major peering points in Seattle
(Westin) and Los Angeles (‘One Wilshire’ complex), and has other
peering capabilities in Sunnyvale & Chicago (@ StarLight)
• Pacific Wave’s production 100g distributed peering fabric includes:
– 100g connectivity to USA DoE ESnet & Internet2 at multiple locations, as
well as to major cloud providers and all 100g Pacific Rim R&E nets.
– 100g and/or 10g to nearly all international Pacific Rim R&E Networks
– the full range of major .com peerings and ‘Tier 1’ ISP connectivity
• Pacific Wave also includes:
– a dedicated 100g Science DMZ backplane between Seattle, Sunnyvale
and Los Angeles with extensions to Denver, Albuquerque and Chicago.
– the backbone for the NSF funded ‘Pacific Research Platform’ (PRP) DMZ
– Dedicated SDN fabric spanning Seattle, Sunnyvale & LA
– Dedicated SDX fabric spanning Seattle, Sunnyvale & LA
As of Dec 2015
Endpoint: Tokyo
NTT OTEMACHI 7F
TY2
4F
UNIV
TOKYO
100GE
3/0/0
100
L3SW
NE40E
WIDE
100
NAO
100GE
3/1/0
3/2
100
L3SW
MLX38
PWave
3/1
100
OTN
OTN
OSN9800
OSN9800
WIDE
WIDE
100
TATA TGN-Pacific
100G
2/0/0
(H)OSN1800
WIDE
Westin Building
Seattle WA
10
T-LEX L3 SW
MLXe8
WIDE
10
10
10
JGN-X
SINET
100
100
Router
MX2020
NII/SINET
L3SW
(B)MLX38
PWave
3F
100
NTT OTEMACHI ANNEX 9F
100GE
Japan US Cable Network (JUS)
Router
T640
APAN-JP
KDDI OTEMACHI 5F EAST
LA
ONE-WILSHIRE
Operational Apr 2016
Endpoint: Tokyo
to CANARIE
Pacific Wave/Transpac4
100G
to UK, EU
to Internet2
Seattle
PW/RC
100G (?)
Tokyo
New York
100G (April 2016)
SINET
100G (April 2016)
to Internet2
to South Amrica
Los
Angels
to APAN
TransPAC3
10G
Hawaii
CN/RC
100G (?)
TREX/WIDE
CENIC
SINET
SINET
TransPAC
APAN
Note: All dotted lines are PLAN or Uncertain
Internet2
Schedule
• Used experimentally at SC’15
• Tested clean Dec 12, 2015
– Now fully operational as a production network
– WIDE Tokyo access & peering point now
operational
– Interconnects to/at the Pacific Wave USA
distributed peering facility now fully operational
– Tata-Tokyo facility router access and peering point
planned to be in production by May 1 2016
Users
• Usage Policy – all R&E traffic allowed
– TransPAC/Pacific Wave is technically AUP-free
– Pacific Wave is AUP-Free
– Peering/routing to shift gradually, starting at 2015
years end, discussion at APAN in January
• TransPAC is supporting end-user engagement
to increase efficient use of the networks
• Example application from SC’15:
250-300ms
Pacific Wave
BackPlane
500-600ms
University of Tokyo TCP Experiment at SC’15
University of Tokyo Test over 100G
ultra long distance links
• Investigating what happens to TCP
performance over ultra long, ultra large
circuits
– Cubic-TCP
– Jumbo Frames
• CPU Load
• Limitations of the TCP Protocol
– 1GB Max buffer (Linux kernel limitation)
– 32 bit sequence numbers (RFC 793)
Connected Networks/Topologies
• Via Pacific Wave, Internet2 & ESnet etc, in USA
– All major R&E networks including multiple 100g
connections to Internet2 and ESnet
– Nearly all Pacific Rim R&E Networks
– StarLight and all European Networks attached
– Ampath and all South American Networks attached
– Peering with most major .com’s
– Connectivity to most major Tier 1 ISP’s
– Peering (often at 1 or more 100g) to major Clouds
– Peering with other major peering sites
Connected Networks/Topologies
• Via Tokyo
– All WIDE peerings & interconnects
– Any other connections/peering via WIDE/T-REX
– Any other connections/peering via TransPACPacific Wave router at Tata
– Plans to peer with APAN, TEIN, and prior
collaborators on TransPAC
TransPAC plans for
2016 additional circuit & peerings
• Had planned to partner on the Singaren-I2
100g
– Collaboration but won’t own part of it
– Interconnect in Tokyo as well as LA-Pacific Wave?!
• Exploring alternative options away from our
traditional paths
– Guam to ?
• Peer in Tokyo and/or Seattle with all other
willing R&E nets or peering points
17
Acknowledgements
• IN@IU is funded by
– US NSF award #0962968 for TransPAC3
– US NSF award #1450904 for TransPAC4
• The TransPAC PacificWave 100gb/s network
fabric is provided by Pacific Northwest
GigaPop
• Pacific Wave is partially funded by a NSF IRNC
award to CENIC & Pacific NorthWest Gigapop
18