MASSACHUSETTS INSTITUTE OF TECHNOLOGY
HAYSTACK OBSERVATORY
WESTFORD, MASSACHUSETTS 01886
Telephone:
Fax:
978-692-4764
781-981-0590
10 August 2002
TO:
Distribution
FROM:
Alan R. Whitney
SUBJECT:
9 August 2002 e-VLBI telecon summary
Attendees:
Lee Foster, Bill Fink, Pat Gary, Paul Lang – GSFC
Steve Bernstein, Lorraine Prior, Peter Schultz – Lincoln Laboratory
Tom Lehman – ISI-E
Richard Crowley, Hans Hinteregger, Arthur Niell, Mike Titus, Alan Whitney – Haystack
Observatory
This telecon is one of an ongoing series of telecons to prepare for gigabit/sec e-VLBI demonstrations
between NASA GSFC and MIT Haystack Observatory using a combination of network facilities
including all or part of Glownet, Bossnet, ISI-E, SuperNet, Max and GSFC/HECN.
ACTION ITEMS ARE HIGHLIGHTED IN RED.
Status Reports
The attached figures of the e-VLBI path have been updated to reflect current status and are pretty
much self-explanatory; critical status items are indicated in red. In addition, the following comments
are relevant:
− Peter reported that Bossnet will be down 1-2 hours on 21 August as LL tries to isolate the
problem causing the OC-48 link to work improperly. Peter will notify everyone of exact
down times when that is known. The Bossnet problem appears to be somewhere in the path
between New York and Newark.
− Tom has received some 10 Gbps equipment (Nortel Passport 8500 with beta software) that he
would like to try on Bossnet over the next month or so. He will send equipment to LL. This
equipment was purchased by DARPA; Tom does not know cost, but will look into it. Since
no one yet has a 10Gbps source, testing will take place using multiple simultaneous GigE
streams up to at least 4-5 Gbps.
− Lee reported he has found information on the web about a 10 Gbps host; Lee will distribute
reference via e-mail.
− Bill reported that the asymmetric performance results observed by Kevin some weeks ago
were due to an MTU mismatch between the ISI-E Juniper M40 (G10), which was set at 4448,
and the MAX Juniper M160 (H3), which was set at 4470. Kevin had noted that the
minutes16.doc 8/10/2002
1
asymmetry disappeared when he set the MTU size down to 4448 and things worked well.
− Bill reported that Abilene is planning to go to 9K MTU size, so we should plan to upgrade to
9K as well if possible.
− Bill reported that, prior to the near-Gbps performance tests that Kevin did on 18 July, he had
discovered a high number of retries on link H6 in the direction from UMCP to ISI-E which
limited aggregate throughput to ~150 Mbps. Reseating connections on UMCP equipment
raised speed to ~700Mbps, but not to ~950Mbps expected. Temporary fix was to re-route
UMCP-to-ISI traffic the other (long) way around the MAX ring; a subsequent sustained 2-hr
test by Bill showed 931Mbps UMCP-to-LL and 906Mbps in reverse direction (MTU=4448).
Kevin’s tests on 18 July were carried out with this asymmetric configuration. The reason for
the problem in the direct link from UMCP to ISI-E is not clear and needs further
investigation; Bill, Tom, Jerry and Dan will pursue this.
− Pat reported nuttcp package has now been ported to Windows. Lee’s testing showed 860880Mbps from ‘xly’ (K6A) to Pluto (L6).
− Pat suggested that we plan to convene a face-to-face ‘team’ meeting of e-VLBIL project team,
perhaps next spring, in advance in the international e-VLBI meeting to be held in The
Netherlands in spring 2003. All agreed this is a good idea, and would be an opportunity for
planning and organizing future e-VLBI activities. Alan will pick up the ball on this and begin
to get organized; Haystack will be happy to host such a meeting, but some people waxed less
than enthusiastic for Boston in January or February; we will keep this in mind!
− Alan reported that Haystack, in collaboration with MIT LL and MIT Lab for Computer
Science, has been awarded an NSF grant under the Strategic Technology for Internet (STI)
program to develop an IP protocol specifically designed for e-VLBI and similar applications.
This project will attempt to make best use of unused (‘scavenged’) bandwidth in a
‘background’ mode which does not interfere with higher priority ‘foreground’ usesr. The
project is funded for 3 years and includes demonstrations of high-speed e-VLBI connections
both with the U.S. and to Europe and Japan. Alan will distribute the text of the proposal to
the group. Pat expressed a strong interest in trying to obtain GSFC support to evolve GSFC
capabilities in parallel with the activities under the NSF grant.
− Alan reported that Haystack and Bill Wildes are working to organize e-VLB connections from
Kokee Park in Hawaii to Haystack and from Univ. of Regensburg in Germany to Haystack,
both probably at OC-3. Excellent cooperation is being received from PMRF (Pacific Missile
Range Facility) people on Kauai and from the Germans in working to establish these links.
The goal is have demonstrate both of these with the next couple of months. Haystack will
contact Steve Corbato (corbato@internet2.edu), director of Abilene, to coordinate the use of
peered connection to Abilene at ISI-E.
− Alan will give a short presentation on U.S. e-VLBI activities at the URSI meeting in The
Netherlands later this month and a longer presentation in November in Korea. He will
prepare some slides to share with everyone.
− Haystack is planning to spruce up the e-VLBI web page at the Haystack web site in the near
future. Some recent work has been done on the Mark 5 web page and it is looking a lot better
(http://web.haystack.mit.edu/mark5/Mark5.htm).
minutes16.doc 8/10/2002
2
− Lee suggested everyone should take at look at the 10GigE Alliance web site
(http://www.10gea.org/) for up-to-date information on 10GigE stuff.
− Alan reported that Haystack is planning to send two Mark 5 systems down to GGAO by ~16
Aug. The plan is to use two Mark 5 systems in parallel at both the Westford and GGAO
antennas to achieve a near-Gbps aggregrate data rate. An additional copper GigE port is
available on the switch (L2) at GGAO for the second Mark 5 (first Mark 5 will be also be
copper). It appears that an additional SX port is available at Westford (first Mark 5 will also
be SX). Lorraine and Kevin will check and Haystack will procure necessary NIC cards.
Control of the Mark 5’s will be on a different network with minimal traffic; a VLAN might be
set up for this purpose.
Performance Testing
Kevin did a 10-hour performance test on 18 July that showed an average of ~960Mbps GGAO-toHaystack and ~900Mbps Haystack-to-GGAO (see attached Figure). Kevin will document these
results in a formal memo.
As reported above, a sustained 2-hr test by Bill (immediately followed Kevin’s 18 July test) showed
931Mbps UMCP-to-LL and 906Mbps in reverse direction (MTU=4448).
Tom reported he had done some performance testing between ISI-E and LL and found numbers
similar to what Kevin and Bill reported in tests between Haystack and GGAO.
As reported above, Lee’s testing showed 860-880Mbps from ‘xly’ (K6A, a Windows machine) to
Pluto (L6).
Next telecon
Next telecon will be Mon, 26 Aug 2002 at 2 pm.
xc:
Steve Bernstein, LL
Jim Calvin, LL
Lorraine Prior, LL
Leslie Weiner, LL
Herbert Durbeck, GSFC
Lee Foster, GSFC
Pat Gary, GSFC
Chuck Kodak, GSFC
Kevin Kranacs, GSFC
Paul Lang, GSFC
Aruna Muppalla, GSFC
Bill Wildes, GSFC
Dan Magorian, UMCP
Tom Lehman, ISI
Jerry Sobieski, Max
Richard Crowley, Haystack
Kevin Dudevoir, Haystack
Hans Hinteregger, Haystack
Arthur Niell, Haystack
Joe Salah, Haystack
minutes16.doc 8/10/2002
3
Westford Antenna
Millstone RPE Bldg (Lorraine)
Mark 5
A1
SX
SX
A1A
A2
Dedicated no scheduling
1
Summit
1i
GigE
A2A
Mark 5
GigE
8
Alpine
A4
3804
A3
Dedicated no scheduling
SX
28-1A
14
B1
Summit
15
13
WS/
Mark 5
100 Mb
B8
B4
B6
B7
Hub
B10
B9
To HS switch
B3
Corr
Cntl
Cmptr
GlowNet
Congress St
Boston
(~50 km)
Amp
E6
E3
GigE
E2
E4A
Test
WS
3l (2 GigE + 1 OC-48)
GlowNet
Alpine
3804
E4
GigE
Test
WS
Bossnet (Steve/Jim)
ISI-E
Scheduling on
Bossnet calendar
GigE
Juniper
M20
1310
E7
E8
GigE
E5
(~100 km)
D1
Shared with MIT traffic
Cisco
1500
E1
D2
Lincoln Lab (Lorraine/Steve/Jim)
Schedule to be
coordinated with LL
Cisco
1500
GigE
B11
D3
WDM
E5A
WDM
GigE
5i
SX
28-1B
(Shared with LL on
scheduled basis)
C3
16
GigE
GigE
Dell Poweredge 2500
(’evlibhay’)
GigE
B5
B2
LL
C2
Haystack Correlator
WS/
Mark 5
Schedule to be
coordinated with LL
C1
MIT
Campus
WDM
F1
E9
Other users (no control)
To be upgraded to OC-48
155x
Amp
Amp
F2
Bossnet currently running GigE
pending investigation and repair
(’superglide’)
(’evlbill’)
From ISI-E
Dell Poweredge 500SC
(provided by Haystack)
Figure 1: e-VLBI Path - Haystack to ISI-E
evlbi011.cdr
10 Aug 02
ISI-E (Tom)
Test
WS
Scheduling on
Bossnet calendar
Test
WS
G5
G7B
G6
GE
G6B
(’kame’)
G8
OC-48
WDM
Summit
5i
(’omega’)
Tuner
Bossnet
GE
G7
MAX @ ISI-E (Jerry)
G6C
ponder
G1
OC-48
Switch
GigE
G3
H5
H2
GigE
G11
OC-48
Juniper
M160
GigE
H8
GigE
H9
Primary e-VLBI path
J3
Test
WS?
J2
Borrowed OC-48
interface - may not
be able to keep
H3
Needs configuration
H6
Other users ( no control)
Cisco
4912
Switch
GigE
Summit
5i
H10
GigE
J5
J4
WDM
GigE
on loan from GSFC
H11
2 GigE l’s
LuxN
J6
J1
Need details
OC-48
Juniper
M160
MAX/UMCP (Jerry/Dan)
No jumbo frames
No scheduling;
sufficient capacity
for e-VLBI
H4
Frame relay encapsulation
G10
H7
MAX performance node;
need details
Test
WS?
Backup plan
Juniper
M40
G9
H1
G12
GigE
GigE
Trans-
G2
Summit
5i
G4
LX (~14km)
GSFC/Bldg 28 (Pat/Paul/Kevin/Bill F.)
K4
K1
WDM
Will try to schedule
either K2 or K3
exclusively for
e-VLBI experiment
G4 Mac with
Yellowdog Linux
(’clifford’)
GGAO/Bldg 201 (Pat/Chuck)
GigE
K2
GigE LX (~7.25km)
1310
K7
Summit
5i
K3
K5A
K5
Test
WS
K6
Test
WS
K6A
K5B
Cisco
12016
K6B
GGAO Antenna Trailer (Bill W./Chuck)
L4
L2
Dedicated - no
scheduling necessary
(~500m)
GigE Summit L3
GigE
L1
5i
L5
G4 Mac
(’pluto’)
RJ-45
Test
WS
RJ-45
L3A
RJ-45
Mark 5
L4A
Mark 5
L6
Windows PC
(’xly’)
evlbi012.cdr
Figure 2: e-VLBI Path - ISI-E to GSFC/GGAO
10 Aug 02