StarLight, the Facility
Joe Mambretti, Tom DeFanti, Maxine Brown
Alan Verlo, Linda Winkler
StarLight: A 1 Gigabit and 10 Gigabit Exchange
StarLight hosts electronic
switching and routing for United
States national and international
Research and Education
networks
Since November 2003, StarLight
has been optically switching
wavelengths between Chicago
and Amsterdam
Abbott Hall, Northwestern University’s
Chicago downtown campus
StarLight History
NSF-funded support of STAR TAP (1997-2000) and STAR TAP2/ StarLight
(2000-2005), and the High Performance International Internet Services
program (Euro-Link, TransPAC, MIRnet and AMPATH).
StarLight, the Facility
• *Flexible* facility for scientists by scientists; input from:
–
–
–
–
–
E-science and C-science researchers
Technical leaders of NSF cyberinfrastructure
Academic national and international R&E networks
Next-generation Federal networks
Metro/regional efforts
• A GE and 10 GE exchange for R&E Networks
• A MEMS-switched Optical Research Network Exchange
• A specialized co-location space for new implementations
and integrations of infrastructure
• Home to fiber and circuits from SBC, Qwest, AT&T, Global
Crossing, Level3, T-Systems, Looking Glass, RCN, and IWIRE, custom integrators, like MCI for DREN
StarLight Facility-Connected
US and International Networks
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
US National
• NIH
•
Abilene/HOPI
• OMNInet2
DREN (DOD)
• TeraGrid
•
ESnet (DOE)
• TRECC
•
NLR
• UltraLight (HEP) via NLR •
NASA GSFC
• USGS
•
NISN (NASA)
• WiscWave (Wisconsin)
NREN (NASA)
UltraScience Net (DOE) • ASnet (Taiwan)
• CA*net4 (Canada)
•
US Other
• CERNET (China)
CAVEwave
• GLORIAD-CSTNet
•
FermiWave
(China) via CA*net4
I-Light (Indiana)
• GLORIAD-Russia via
•
I-WIRE (Illinois)
CA*net4
•
LONI (Louisiana) via NLR
MiLR (Michigan LambdaRail)
MREN (Midwest)
GLORIAD/KREONet2
(Korea) via CA*net4
HARNET (Hong Kong)
JGN-II (Japan)
SINET (Japan)
SURFnet (Netherlands),
which also carries
NORDUnet and
CESNET
TaiwanLight (Taiwan)
via CA*net4
TransLight/StarLight
IRNC
UKLight (UK)
US LHCnet (CERN)
E-Science and C-Science
Experiments in StarLight Co-Lo Space
•
OMNInet2 – Joe Mambretti,
Nortel/SBC
•
NSF National Center for Data
Mining Cluster – Bob Grossman
•
DARPA DWDM-RAM – Joe Mambretti
and Nortel Networks
•
•
NSF/DOE Caltech/CERN Cluster for
Data Grid development (GriPhyN,
PPDG, iVDGL, EU DataGrid) –
Harvey Newman
NSF OptIPuter visualization
cluster – Tom DeFanti and Jason
Leigh
•
NSF MEMS Optical Switching –
Tom DeFanti and Jason Leigh
•
University of Amsterdam Itanium
for Optical switching; Generic
AAA – Cees de Laat
•
USGS map storage
•
•
NSF Distributed Optical Testbed
(DOT) – Joe Mambretti and Valerie
Taylor
NSF Logistical Networking – Micah
Beck and Yotta Yotta
StarLight Provided Major iGrid 2005 Support
OptIPuter: Major NSF-funded Research
Initiative Utilizing StarLight Capabilities
The OptIPuter Project –
Removing Bandwidth as an Obstacle In Data Intensive Sciences
• An NSF-funded award that focuses on developing technology to
enable the real-time collaboration and visualization of very-large timevarying volumetric datasets for the earth sciences and the biosciences
• OptIPuter is examining a new model of computing whereby ultra-highspeed networks form the backplane of a global computer
NIH Biomedical Informatics
Research Network
NSF EarthScope
and ORION
http://ncmir.ucsd.edu/gallery.html
siovizcenter.ucsd.edu/library/gallery/shoot1/index.shtml
OptIPuter is a Global Virtual Computer
• Hardware: clusters of computers that act as giant storage, compute or
visualization peripherals, in which each node of each cluster is
attached at 1 or 10GigE to a backplane of ultra-high-speed networks
• Software: Advanced middleware and application toolkits are being
developed for lightpath management, data management and mining,
visualization, and collaboration
Commodity GigE Switch
Fibers or Lambdas
The Scalable Adaptive Graphics Environment
To Manage Visual Content on Scalable Tiled Displays
SAGE runs on LambdaVision, a display-rich collaborative work
environment (11x5 LCDs)
Moving Data Over these Networks:
Ultra-High-Speed Transport Protocols
• Reliable Blast UDP designed for ultra-high-speed bulk
data delivery applications
•
•
Developed originally in 2000
Achieved 18Gb/s out of 20Gb/s available bandwidth on TeraGrid in
2003.
• LambdaStream designed for ultra-high-speed
streaming applications that need high throughput but
low latency and low jitter (such as streaming graphics)
•
•
Developed in 2003/04
LambdaStream achieves this by adaptive rate adjustment AND
attempting to predict the CAUSE of loss using techniques from
Mobile Adhoc Networks (MANETs).
10GE OptIPuter CAVEwave
on the National LambdaRail
EVL
Source: Tom DeFanti, OptIPuter co-PI
Calient DiamondWave Switches at StarLight
and NetherLight
• 128x128 Calient O-O-O 3D
MEMS switch at StarLight
• 64x64 Calient switch at
NetherLight
• 1% the cost of routing
Photonic Interdomain Negotiator (PIN)
Application-Centric Scheduling of Photonic Networks
Correlation / Filtering
StarLight (Chicago)
Data Access UvA
(Amsterdam)
Cluster
Cluster
Cluster
Visualization
EVL (Chicago)
Cluster
Calient
Photonic
Switch
Calient
Photonic
Switch
Glimmerglass
Photonic Switch
Muxed & DeMuxed DWDM
OC-192
PPBAC
PIN
PDC
PDC
PIN
PIN
Joe Mambretti (NU), Eric He, Cees de Laat (UvA), Oliver Yu (UIC)
Bring Us Your Lambdas
www.startap.net/starlight
Thanks to our Sponsors and Collaborators
• StarLight planning, research, collaborations, and outreach
efforts have been made possible, in major part, by funding
from:
– US National Science Foundation (NSF) awards SCI-9980480, SCI9730202, CNS-9802090, CNS-9871058, SCI-0225642, and CNS0115809
– State of Illinois I-WIRE Program, and major UIC cost sharing
– Northwestern University for facility space, engineering and
management
• US NSF/CISE and US DoE/Argonne National Laboratory for StarLight
and I-WIRE network engineering and design
• Bill St. Arnaud of CANARIE and Kees Neggers of SURFnet for
networking leadership
• Larry Smarr of Calit2 for I-WIRE and OptIPuter leadership