MASSACHUSETTS INSTITUTE OF TECHNOLOGY
HAYSTACK OBSERVATORY
WESTFORD, MASSACHUSETTS 01886
Telephone:
Fax:
978-692-4764
781-981-0590
24 June 2002
TO:
Distribution
FROM:
Alan R. Whitney
SUBJECT:
24 June 2002 e-VLBI telecon summary
Attendees:
Lee Foster, Pat Gary, Chuck Kodak, Kevin Kranacs – GSFC
Steve Bernstein, Lorraine Prior, Peter Schultz – Lincoln Laboratory
Tom Lehman – ISI-E
Richard Crowley, Kevin Dudevoir, Hans Hinteregger, Arthur Niell, Mike Titus, Alan Whitney –
Haystack Observatory
This telecon is one of an ongoing series of telecons to prepare for gigabit/sec e-VLBI demonstrations
between NASA GSFC and MIT Haystack Observatory using a combination of network facilities
including all or part of Glownet, Bossnet, ISI-E, SuperNet, Max and GSFC/HECN.
Status Reports
The attached figures of the e-VLBI path have been updated to reflect current status and are pretty
much self-explanatory; critical status items are indicated in red. In addition, the following comments
are relevant:
− Tom and Peter reported that there is still a problem with the OC-48 Bossnet service. It has
temporarily been changed to GigE, which seems to work properly. A service person will be
dispatched to try to diagnose and fix the problem in the next few days; the problem appears to
be in an optical amplifier in the Newark, NJ area.
− Routers at G10, H3 and H7 need configuration updates to support the e-VLBI path. Pat and
Tom will consult with Jerry Sobieski and/or Dan Magorian to accomplish this. Bill Fink can
assist with this task if Jerry/Dan are not able to do it. Goal is to have this work done by the
end of this week.
− Pat reported that, as a result of the creation of several VLAN’s for e-VLBI purposes,
workstation L6 (pluto at GGAO) has had an IP change to 206.196.178.53.
− Pat reported that another Summit 5i (designated K4A) has been added between K4 (Bldg 28)
and L2 (GGAO). Additionally, a ‘one-armed’ Cisco GSR 12000/16 router is attached to K4A
to connect the GGAO and Bldg 28 nets. Pat does not believe this additional equipment
should affect net performance. All equipment in the e-VLBI path should support 9KB jumbo
frames.
minutes14.doc 6/24/2002
1
− G7B IP address has been changed to 140.173.174.22.
Performance Testing
After the conversion of Bossnet to GigE, Tom achieved >900 Mbps single stream TCP in both
directions.
Kevin D. conducted four 30-minute tests, achieving ~965 Mbps average from ISI to Haystack, and
~940 Mbps average from Haystack to ISI, based on 1-sec averaging using iperf. Kevin’s plots are
attached (top two plots of each test are transmitter performance; bottom two plots are receiver
performance). The bandwidth was measured using iperf-1.1.1 with 1 sec averaging. The cpu idle
time was measured using vmstat with 1 sec averaging. No information on packet loss rates or retransmissions. The tests were done with the following parameters:
mtu = 4470 Bytes
mss = 4418 Bytes
buf = 3976200 Bytes
Quick summary of the results:
1] test 1: ISI(kame) -> Haystack(evlbihay)
avg. bandwidth = 965991626 Mb/s,
tx cpu idle time = 40%
rx cpu idle time = 37%
2] test 2: Haystack(evlbihay) -> ISI(kame)
avg. bandwidth = 934996557 Mb/s
tx cpu idle time = 12%
rx cpu idle time = 40%
3] test 3: ISI(kame) -> Haystack(evlbihay)
avg. bandwidth = 969819409 Mb/s
tx cpu idle time = 37%
rx cpu idle time = 41%
4] test 4: Haystack(evlbihay) -> ISI(kame)
avg. bandwidth = 951017228 Mb/s
tx cpu idle time = 13%
rx cpu idle time = 40%
Kevin D. reported that application performance with two Mark 5 systems back-to-back is currently
limited to ~450 Mbps due to an apparent bottleneck from memory to/from disk. He plans to further
investigate, perhaps trying another motherboard with a faster CPU (current CPU is 1GHz P3) and
PCI bus.
Kevin D. reported that he had difficulty measuring round-trip ping time from ISI to Haystack and
back to ISI; varies from 4msec (impossible) to ~14msec. Round-trip ping time from Haystack to ISI,
however, is consistent at ~14.5msec. Reason for this inconsistency is unknown.
Pat reported that Bill F. has done testing between GGAO (L6) and GSFC Bldg 28 (K6), achieving
~992 Mbps TCP single stream (likely in both directions) using nuttcp. Connection seems to be very
solid.
minutes14.doc 6/24/2002
2
Action Items
Alan: Distribute updated workstation configuration information to group.
Steve: Keep group updated on Bossnet status.
Pat/Tom/Bill F.: Work with Jerry Sobieski/Dan Magorian to properly configure routers at G10, H3
and H7. To be done by end of the week (28 June 2002).
Bill F: Distribute information on nuttcp routine.
Bill F.: Set up reverse mapping so GGAO test workstation can be managed from Bldg 28.
Next telecon
Next telecon will be Thursday, 11 July 2002 at 2 pm.
xc:
Steve Bernstein, LL
Jim Calvin, LL
Lorraine Prior, LL
Leslie Weiner, LL
Herbert Durbeck, GSFC
Lee Foster, GSFC
Pat Gary, GSFC
Chuck Kodak, GSFC
Kevin Kranacs, GSFC
Paul Lang, GSFC
Aruna Muppalla, GSFC
Bill Wildes, GSFC
Dan Magorian, UMCP
Tom Lehman, ISI
Jerry Sobieski, Max
Richard Crowley, Haystack
Kevin Dudevoir, Haystack
Hans Hinteregger, Haystack
Arthur Niell, Haystack
Joe Salah, Haystack
minutes14.doc 6/24/2002
3
Westford Antenna
Millstone RPE Bldg (Lorraine)
Dedicated no scheduling
GigE
WS/
Mark 5
Summit
1i
1
A2
A1
GigE
8
Alpine
A4
3804
A3
Dedicated no scheduling
28-1A
14
B1
Summit
15
WS/
Mark 5
100 Mb
28-1B
B8
B4
B6
B7
Hub
B10
B9
To HS switch
B3
Corr
Cntl
Cmptr
GlowNet
Congress St
Boston
(~50 km)
Amp
Cisco
1500
Connection made for e-VLBI
E3
GigE
Alpine
3804
E2
E1
D2
Lincoln Lab (Lorraine/Steve/Jim)
Schedule to be
coordinated with LL
E4A
Test
WS
From ISI-E
(’Superglide’)
GigE
E6
E4
GigE
Test
WS
3l (2 GigE + 1 OC-48)
GlowNet
(~100 km)
D1
Bossnet (Steve/Jim)
ISI-E
Scheduling on
Bossnet calendar
E7
Juniper OC-48
1310
M20
E8
GigE
E5
Cisco
1500
GigE
B11
D3
WDM
E5A
WDM
GigE
5i
13
(Shared with LL on
scheduled basis)
C3
16
GigE
GigE
Dell Poweredge 2500
GigE
B5
B2
LL
C2
Haystack Correlator
WS/
Mark 5
Schedule to be
coordinated with LL
C1
MIT
Campus
WDM
F1
E9
Other users (no control)
To be upgraded to OC-48
155x
Amp
Amp
F2
Bossnet currently running GigE
pending investigation and repair
Dell Poweredge 500SC
(provided by Haystack)
Figure 1: e-VLBI Path - Haystack to ISI-E
evlbi011.cdr
24 June 02
ISI-E (Tom)
Test
WS
Scheduling on
Bossnet calendar
GE
Summit
5i
Test
WS
G5
G7B
G6
G7
G8
OC-48
WDM
GE
G6B
Summit
5i
G4
G6C
Tuner
Bossnet
MAX @ ISI-E (Jerry)
ponder
G1
OC-48
GigE
H2
GigE
G3
G11
OC-48
Borrowed OC-48
interface - may not
be able to keep
G10 Needs configuration
H4
No jumbo frames
Juniper
M160
GigE
H8
GigE
H9
J3
Test
WS?
Primary e-VLBI path
Needs configuration
J2
H3
Needs configuration
H6
Other users ( no control)
Cisco
4912
Switch
GigE
Summit
5i
H10
GigE
J5
J4
WDM
GigE
on loan from GSFC
H11
2 GigE l’s
LuxN
J6
J1
Need details
OC-48
Juniper
M160
MAX/UMCP (Jerry/Dan)
H7
No scheduling;
sufficient capacity
for e-VLBI
Backup plan
Juniper
M40
G9
MAX performance node;
need details
Test
WS?
H5
H1
G12
GigE
GigE
Trans-
G2
Switch
LX (~14km)
GSFC/Bldg 28 (Pat/Paul/Kevin/Bill F.)
K4
K1
WDM
Will try to schedule
either K2 or K3
exclusively for
e-VLBI experiment
G4 Mac with
Yellowdog Linux
GigE
K2
K4A
Summit
Summit
5i
5i
K4B
K3
K5A
K4
Test
WS
Test
WS
K6
K6A
GGAO/Bldg 201 (Pat/Chuck)
GigE LX (~7.25km)
1310
K7
GGAO Antenna Trailer (Bill W./Chuck)
Dedicated - no
scheduling necessary
(~500m)
GigE Summit
L1
5i
K5B
Cisco
12000/16
K6A
Windows PC
(Lee Foster)
L2
GigE
L3
L5
G4 Mac
SX
Mark 5
L4
Test
WS
L6
evlbi012.cdr
Figure 2: e-VLBI Path - ISI-E to GSFC/GGAO
24 June 02