Key Aspects of eVLBI Network Design with Emphasis Between BoSSNET’s POP at ISI-E and the GGAO at GSFC Prepared by J. Patrick Gary Network Projects Leader Earth and Space Data Computing Division/Code 930 NASA Goddard Space Flight Center pat.gary@gsfc.nasa.gov 301-286-9539 September 25, 2001 J. P. Gary 9/25/01 1 Key Aspects of eVLBI Network Design with Emphasis Between BoSSNET’s POP at ISI-E and the GGAO at GSFC Preface ● ● ● ● All info is essentially DRAFT; much more detailed planning is required This info makes many assumptions few of which are yet adequately described but all need to be Further planning needs to be based better on a clearer understanding of this project’s requirements, timetable, and actual funding support All references to UMCP-led Mid-Atlantic Crossroads (MAX) cooperation is assumed, but no MAX commitments have yet been negotiated; we just haven’t yet had time to discuss this more thoroughly with them J. P. Gary 9/25/01 2 Key Aspects of eVLBI Network Design with Emphasis Between BoSSNET’s POP at ISI-E and the GGAO at GSFC Proposed eVLBI Phase 1 “Shared GE” Test Configuration ● Achieves approximately 1 Gigabit per second (GB ˚b ps) end-to-end but can only be used for time-limited demos as it uses shared MAX and GSFC/High End Computer Network (HECN) project network infrastructure between ISI-E and GGAO Potential eVLBI Phase 1 “Dedicated GE” Test Configuration ● Builds on technical approach validated in Phase 1 replacing shared MAX and GSFC/HECN network infrastructure with eVLBI-dedicated network infrastructure J. P. Gary 9/25/01 3 Key Aspects of eVLBI Network Design with Emphasis Between BoSSNET’s POP at ISI-E and the GGAO at GSFC Westford Actual as of 9/24/01 (independent of eVLBI) Mark 5 ISI-E w/ MAX POP Haystack MAX POP at UMCP HECN Testbed GGAO at GSFC in GSFC Building 28 Building 202 Correlator GE Sw Test Wkst BoSS NET GE Sw Juniper M160 router MAX other GE Juniper networks M40 router Lincoln Labs Juniper M160 router GE Sw other GE networks GE Sw other GE networks FORE ASX1000 FORE ASX4000 GE Sw Juniper M20 router Mark 4 WDM HSCC other GE networks FORE ASX4000 Cisco GSR12016 router GGAO LAN GSFC SEN GSFC CNE WDM Abilene other NGI networks Legend NTON 8x10Gbps/OC-192c 4x2.5Gbps/OC-48c 2.5Gbps/OC-48c ATDNet/ MONET 1 Gbps GE JPG 9/25/01 622 Mbps/OC-12c 100 Mbps FE Key Aspects of eVLBI Network Design with Emphasis Between BoSSNET’s POP at ISI-E and the GGAO at GSFC Westford Expected as of 11/01/01 (independent of eVLBI) Mark 5 ISI-E w/ MAX POP Haystack MAX POP at UMCP HECN Testbed GGAO at GSFC in GSFC Building 28 Building 202 Correlator GE Sw Test Wkst BoSS NET GE Sw Juniper M160 router MAX other GE Juniper networks M40 router Lincoln Labs WDM GE Sw other GE networks other GE networks FORE ASX1000 FORE ASX4000 GE Sw Juniper M20 router Juniper M160 router Mark 4 WDM GE Sw other GE networks FORE ASX4000 Cisco GSR12016 router GGAO LAN GSFC SEN GSFC CNE WDM HSCC Abilene other NGI networks Legend NTON 8x10Gbps/OC-192c 4x2.5Gbps/OC-48c 2.5Gbps/OC-48c ATDNet/ MONET 1 Gbps GE JPG 9/25/01 622 Mbps/OC-12c 100 Mbps FE Key Aspects of eVLBI Network Design with Emphasis Between BoSSNET’s POP at ISI-E and the GGAO at GSFC Westford Proposed eVLBI Phase 1 “Shared GE” Test Config. Mark 5 ISI-E w/ MAX POP Haystack MAX POP at UMCP HECN Testbed GGAO at GSFC in GSFC Building 28 Building 202 Correlator GE Sw Test Wkst BoSS NET GE Sw Juniper M160 router MAX other GE Juniper networks M40 router Lincoln Labs WDM GE Sw other GE networks other GE networks FORE ASX1000 FORE ASX4000 GE Sw Juniper M20 router Juniper M160 router Mark 5 WDM GE Sw other GE networks FORE ASX4000 Cisco GSR12016 router fiber pair between B28, B10, B200 GE Sw Test Wkst GGAO LAN WDM HSCC Abilene other NGI networks GSFC SEN GSFC CNE Legend NTON 8x10Gbps/OC-192c 4x2.5Gbps/OC-48c 2.5Gbps/OC-48c ATDNet/ MONET 1 Gbps GE JPG 9/25/01 622 Mbps/OC-12c 100 Mbps FE Key Aspects of eVLBI Network Design with Emphasis Between BoSSNET’s POP at ISI-E and the GGAO at GSFC Westford Potential eVLBI Phase 1 “Dedicated GE” Test Config. Mark 5 ISI-E w/ MAX POP Haystack MAX POP at UMCP HECN Testbed GGAO at GSFC in GSFC Building 28 Building 202 Correlator GE Sw Test Wkst BoSS NET GE Sw Juniper M160 router MAX other GE Juniper networks M40 router Juniper M160 router other GE networks Juniper M20 router other GE networks FORE ASX1000 FORE ASX4000 GE Sw WDM GE Sw Mark 5 WDM GE Sw other GE networks FORE ASX4000 Cisco GSR12016 router fiber pair between B28, B10, B200 GE Sw Test Wkst GGAO LAN WDM HSCC Abilene other NGI networks GSFC SEN GSFC CNE Legend NTON 8x10Gbps/OC-192c 4x2.5Gbps/OC-48c 2.5Gbps/OC-48c ATDNet/ MONET 1 Gbps GE JPG 9/25/01 622 Mbps/OC-12c 100 Mbps FE Key Aspects of eVLBI Network Design with Emphasis Between BoSSNET’s POP at ISI-E and the GGAO at GSFC Phase 1 Notes ● At Haystack and MIT/LL » Info source is Haystack’s Alan Whitney and MIT/LL Steven Bernstein’s 5/1/01 “Lincoln Optical Fiber Network Infrastructure” ppt » Assumes a VLBI Mark 5 unit or a VLBI Correlator and at least one suitable workstation are Gigabit Ethernet (GE) connected to a Haystack local GE switch which is connected through MIT/LL (details not important here but are available from MIT/LL or Haystack) to BoSSNET » Workstation will be used for end-to-end checkout testing with a comparable workstation unit at GGAO » Questions/issues: – TBD J. P. Gary 9/25/01 8 Key Aspects of eVLBI Network Design with Emphasis Between BoSSNET’s POP at ISI-E and the GGAO at GSFC Phase 1 Notes ● At ISI-E » Info source is ISI-E’s Tom Lehman » Leverages BoSSNET’s existing termination/CPE user interface which presently is essentially a GE to a local GE switch (and is planned to be replaced with a 2.4 Gbps POS user interface to the ISI-E’s M40 router) » Assumes Supernet/BoSSNet IP peering with MAX is prearranged (to be resolved between DARPA and MAX) » Assumes “simple” physical GE switch interconnection among BoSSNET, ISI-E GE switch, and MAX’s M160 router (until replaced by 2.4 Gbps POS interconnection among BoSSNET, ISI-E M40 router, and MAX’s M160 router) » Questions/issues: – Maximum MTU size of M160 supported frames - 4470 bytes? J. P. Gary 9/25/01 9 Key Aspects of eVLBI Network Design with Emphasis Between BoSSNET’s POP at ISI-E and the GGAO at GSFC Phase 1 Notes ● At MAX’s UMCP site » Info sources are GSFC’s Bill Fink and MAX’s Jerry Sobieski and Dan Magorian » Leverages GSFC/HECN’s existing and planned GE interfaces to MAX’s M160 and GE switch over today’s existing (only) one fiber pair*. One GE link exists today to/from a Cisco 3508. GSFC/HECN-acquired Course WDM with two GE lambdas for this fiber pair is expected to be installed before 11/01/01. – *One other fiber pair exists but is dedicated to a semi-operational function » Questions/issues: – What is the UMCP host (M160 or GE switch) for each of the GE lambdas? – Jumbo frames are not presently supported on existing GE switch, but MAX may be planning an upgrade soon. This need more investigation and discussion. J. P. Gary 9/25/01 10 Key Aspects of eVLBI Network Design with Emphasis Between BoSSNET’s POP at ISI-E and the GGAO at GSFC Phase 1 Notes ● At GSFC/HECN’s Building 28 site » Info sources are GSFC’s Bill Fink and Paul Lang » Leverages GSFC/HECN’s existing and planned GE interfaces to MAX’s M160 and GE switch over today’s existing (only) one fiber pair*. One GE link exists today to/from a Cisco 3508. GSFC/HECN-acquired Course WDM with two GE lambdas for this fiber pair is expected to be installed before 11/01/01. – *One other fiber pair exists but is dedicated to a semi-operational function » Questions/issues: – What is the HECN host (GSR 12016 or GE switch) for each of the GE lambdas? – Jumbo frames are not presently supported on existing GE switch, but an Extreme Network Summit 5i with jumbo frame support should be installed before 11/01/01. – Need one LX/1000BaseT converter for the Summit 5i ~$ 2K J. P. Gary 9/25/01 11 Key Aspects of eVLBI Network Design with Emphasis Between BoSSNET’s POP at ISI-E and the GGAO at GSFC Phase 1 Notes ● At GSFC/GGAO’s Trailer near Building 202 site » Info sources are GSFC’s Lee Foster, Herb Durbeck and Scott Douglas » Need fiber pair between B28/W220 (via B10) and GGAO trailer near B202 – Douglas’ 7/2/01 email identified cost of a fiber pair between B28 and Area 200 ~$10K » Need GE switch with jumbo frame support at GGAO – Recommend Extreme Network Summit 1i with one LX and two 1000BaseT ports ~$ 6K » Proposes Mac G4 with 1000BaseT GE NIC as test workstation – Similar to Fink’s recent acquisitions ~$ 3K » Assumes a VLBI Mark 5 unit with 1000BaseT GE NIC » Questions/issues: – Need to confirm Mark 5”s GE NIC is 1000BaseT J. P. Gary 9/25/01 12 Key Aspects of eVLBI Network Design with Emphasis Between BoSSNET’s POP at ISI-E and the GGAO at GSFC Potential eVLBI Phase 1 “Dedicated GE” Test Configuration (Same as Phase 1 “Shared GE” Test Config except were noted) ● At Haystack and MIT/LL: no change ● At ISI-E: » Need one additional GE port on M160 ● Between MAX/ISI-E and MAX/UMCP: » Need additional fiber pair or lease a lambda from MAX ● TBD At MAX/UMCP: » Need two additional GE ports on M160 ● ~$13K ~$25K Between UMCP and GSFC/HECN » Need additional fiber pair or add a lambda » Need additional LX GE port on HECN GE switch ~$20K ~$5K J. P. Gary 9/25/01 13 Key Aspects of eVLBI Network Design with Emphasis Between BoSSNET’s POP at ISI-E and the GGAO at GSFC Backup Charts J. P. Gary 9/25/01 14 GSFC High End Computer Network (HECN) Testbed External Interfaces circa 9/24/01 Legend HSCC NTON BoSSNET ATDNet/ MONET HECN Testbed in Building 28 SAN Cisco 3508 GE 2 Sun Cisco 4000 GE Sun other NGI networks MAX/ NGIX-DC/ UMCP NREN 4 Sun NISN Building 1 Lucent WDM Cisco LS-1010 ATM Cisco GSR 12016 router Abilene 8x10Gbps/OC-192c 2.5Gbps/OC-48c 622 Mbps/OC-12c 155 Mbps/OC-3c 100 Mbps FE & FDDI 1 Gbps GE FORE ASX-4000 ATM 1 Sun FORE ASX-1000 ATM 2 Sun ATM IsoLAN Cisco GSR 12012 router CAN and SAN FE/ FDDI IsoLAN WAN FW router CNE 9 Sun Cisco 7500 router NILE FE & FDDI JPG 9/24/01 MAX Fiber Infrastructure Plant 2 NIH UMBC HHMI WDST FG NLM CB ATT / ETC L3 NOAA QW / 2 ADEL QW / 2 UMCP FG / 6 ISIE NCSA / 2 QW QWST / 2 QW FG / 6 GSFC FG CENS FG NWS Target Core Optical Network ATM OC48c POS WDST Router NGIX Router VBNS UMCP NISN OADM Router Router ADEL QWST OC48c POS OC192c POS (future) Qwest ISP Abilene ISIE Router OC48c POS