MASSACHUSETTS INSTITUTE OF TECHNOLOGY
HAYSTACK OBSERVATORY
WESTFORD, MASSACHUSETTS 01886
Telephone:
Fax:
978-692-4764
781-981-0590
12 July 2002
TO:
Distribution
FROM:
Alan R. Whitney
SUBJECT:
11 July 2002 e-VLBI telecon summary
Attendees:
Lee Foster, Bill Fink, Pat Gary, Kevin Kranacs, Paul Lang – GSFC
Steve Bernstein, Lorraine Prior, Peter Schultz – Lincoln Laboratory
Tom Lehman – ISI-E
Richard Crowley, Hans Hinteregger, Arthur Niell, Mike Titus, Alan Whitney – Haystack
Observatory
This telecon is one of an ongoing series of telecons to prepare for gigabit/sec e-VLBI demonstrations
between NASA GSFC and MIT Haystack Observatory using a combination of network facilities
including all or part of Glownet, Bossnet, ISI-E, SuperNet, Max and GSFC/HECN.
Status Reports
The attached figures of the e-VLBI path have been updated to reflect current status and are pretty
much self-explanatory; critical status items are indicated in red. In addition, the following comments
are relevant:
− Peter reported that there is still a problem with the OC-48 Bossnet service and LL is short of
personnel to address the problem immediate. As a result, Bossnet will be maintained as a
GigE link until at least August. The GigE link seems to be working properly and is suitable
for the e-VLBI demonstration; however, the Bossnet GigE link is shared with MIT campus, so
Bossnet use for e-VLBI needs to be scheduled (via Bossnet scheduling calendar) so that
campus activity can be suspended while e-VLBI testing takes place. The Bossnet problem
appears to be somewhere in the path between New York and Newark, but is not yet isolated.
− Router configurations at G10, H3 and H7 have been updated, as promised at the last meeting,
for e-VLBI use.
− Test workstation at K6 is now named ‘clifford’ (not ‘dawg’). Alan will update on workstation
specification list.
− Lorraine reported that another LL demonstation using Glownet is scheduled for 22 July,
during which time Glownet will be unavailable for e-VLBI.
− Haystack target date is tentatively August for full demonstration.
minutes15.doc 7/12/2002
1
Performance Testing
Bill F. requested that all test workstations install a nuttcp server to facilitate ease of testing. He will
distribute a note to everyone with details of the installation procedure (has been done).
Kevin D. conducted several short tests from Haystack (‘evlibhay’) to GSFC/GGAO (‘pluto’),
achieving ~350 Mbps throughput. Below is a summary of his results:
Configuration MIT(evlbihay):
ip
140.173.174.4
mtu
4470 (9000 max)
mss
4418
/proc/sys/net/core/wmem_max
8388608
/proc/sys/net/core/rmem_max
8388608
/proc/sys/net/ipv4/tcp_timestamps 1
/proc/sys/net/ipv4/tcp_sack
1
/proc/sys/net/ipv4/tcp_wmem
"4096 1048576 8388608"
/proc/sys/net/ipv4/tcp_rmem
"4096 1048576 8388608"
nuttcp
v3.1.9
Configuration GGAO(pluto):
ip
206.196.178.53
nuttcp
v3.2.1
Rtt:
~ 15 msec
Throughput:
~ 350 Mb/s (Haystack to GGAO)
Note: Only very short duration (1-15 secs) tests were done.
The following was typical output:
[kad@evlbihay bin]$ ./nuttcp -t -l4418 -n16384 -w2048 -ws2048 206.196.178.53
nuttcp-t: v3.1.9: socket
nuttcp-t: buflen=4418, nbuf=16384, nstream=1, port=5001 tcp -> 206.196.178.53
nuttcp-t: connect to 206.196.178.53
nuttcp-t: send window size = 4194304, receive window size = 1048576
nuttcp-t: 69.031 MB in 1.64 real seconds = 43007.87 KB/sec = 352.3205 Mb/s
nuttcp-t: 16384 I/O calls, msec/call = 0.10, calls/sec = 9968.33
nuttcp-t: 0.0user 0.3sys 0.01real 21% 0i+0d 0maxrss 0+0pf 0+0csw
nuttcp-r: v3.2.1: socket
nuttcp-r: buflen=4418, nbuf=16384, nstream=1, port=5001 tcp
nuttcp-r: accept from 140.173.174.4
nuttcp-r: send window size = 16384, receive window size = 2097152
nuttcp-r: 69.031 MB in 1.68 real seconds = 41996.25 KB/sec = 344.0333 Mb/s
nuttcp-r: 69.031 MB in 0.22 CPU seconds = 321309.09 KB/cpu sec
nuttcp-r: 16467 I/O calls, msec/call = 0.10, calls/sec = 9783.16
nuttcp-r: 0.0user 0.2sys 0:01real 13% 0i+0d 0maxrss 1+1pf 0+0csw
Kevin D. feels that there may some tuning issues on ‘pluto’, but he was unable to get a password to
make tuning changes. Bill F. will work with Kevin on ‘pluto’ tuning issues. Tom also suggested
trying a multi-stream TCP process to improve aggregate throughput rate (easily done with nuttcp).
nuttcp is not yet ported to Windows. Lee reports that a summer student is currently working on this
project.
Tom will investigate possibility of putting nuttcp on MAX test workstations, but warned that these
machines are not particularly high-performance machines so probably may not be able to draw useful
performance conclusions.
minutes15.doc 7/12/2002
2
Tom and Bill will coordinate increasing mtu size on paths between ISI-E and MAX. The issue is
complicated by the use of ‘frame relay encapsulation’ on some of these links and links to Abilene.
Action Items
Bill F: Distribute information on nuttcp routine (has been done).
Tom/Bill F.: Will coordinate installation of nuttcp on ISI-E test machines (Tom will be on vacation
for next two weeks, but will be available via e-mail occasionally during this time). Bill will then do
testing on ISI-E/GSFC link and report to group.
Tom/Bill F.: Will coordinate mtu change on ISI-E/MAX links.
Tom: Will investigate possibility of putting nuttcp on MAX test workstations.
Kevin D./Bill F.: Will coordinate to allow Kevin access to ‘pluto’ (requires Kerberos account).
Next telecon
Next telecon will be Wed, 7 Aug 2002 at 2 pm.
xc:
Steve Bernstein, LL
Jim Calvin, LL
Lorraine Prior, LL
Leslie Weiner, LL
Herbert Durbeck, GSFC
Lee Foster, GSFC
Pat Gary, GSFC
Chuck Kodak, GSFC
Kevin Kranacs, GSFC
Paul Lang, GSFC
Aruna Muppalla, GSFC
Bill Wildes, GSFC
Dan Magorian, UMCP
Tom Lehman, ISI
Jerry Sobieski, Max
Richard Crowley, Haystack
Kevin Dudevoir, Haystack
Hans Hinteregger, Haystack
Arthur Niell, Haystack
Joe Salah, Haystack
minutes15.doc 7/12/2002
3
Westford Antenna
Millstone RPE Bldg (Lorraine)
Dedicated no scheduling
GigE
WS/
Mark 5
Summit
1i
1
A2
A1
GigE
8
Alpine
A4
3804
A3
Dedicated no scheduling
28-1A
14
B1
Summit
15
WS/
Mark 5
100 Mb
28-1B
B8
B4
B6
B7
Hub
B10
B9
To HS switch
B3
Corr
Cntl
Cmptr
GlowNet
Congress St
Boston
(~50 km)
Amp
E6
E3
GigE
E2
E4A
Test
WS
3l (2 GigE + 1 OC-48)
GlowNet
Alpine
3804
E4
GigE
Test
WS
Bossnet (Steve/Jim)
ISI-E
Scheduling on
Bossnet calendar
GigE
Juniper
M20
1310
E7
E8
GigE
E5
(~100 km)
D1
Shared with MIT traffic
Cisco
1500
E1
D2
Lincoln Lab (Lorraine/Steve/Jim)
Schedule to be
coordinated with LL
Cisco
1500
GigE
B11
D3
WDM
E5A
WDM
GigE
5i
13
(Shared with LL on
scheduled basis)
C3
16
GigE
GigE
Dell Poweredge 2500
(’evlibhay’)
GigE
B5
B2
LL
C2
Haystack Correlator
WS/
Mark 5
Schedule to be
coordinated with LL
C1
MIT
Campus
WDM
F1
E9
Other users (no control)
To be upgraded to OC-48
155x
Amp
Amp
F2
Bossnet currently running GigE
pending investigation and repair
(’superglide’)
(’evlbill’)
From ISI-E
Dell Poweredge 500SC
(provided by Haystack)
Figure 1: e-VLBI Path - Haystack to ISI-E
evlbi011.cdr
12 July 02
ISI-E (Tom)
Test
WS
Scheduling on
Bossnet calendar
Test
WS
G5
G7B
G6
GE
G6B
(’kame’)
G8
OC-48
WDM
Summit
5i
(’omega’)
Tuner
Bossnet
GE
G7
MAX @ ISI-E (Jerry)
G6C
ponder
G1
OC-48
Switch
GigE
H2
GigE
G3
G11
OC-48
Frame relay encapsulation
H4
H3
Juniper
M160
GigE
H8
GigE
H9
Primary e-VLBI path
Needs configuration
J3
Test
WS?
J2
Other users ( no control)
Cisco
4912
Switch
GigE
Summit
5i
H10
GigE
J5
J4
WDM
GigE
on loan from GSFC
H11
2 GigE l’s
LuxN
J6
J1
Need details
Needs configuration
H6
MAX/UMCP (Jerry/Dan)
No jumbo frames
OC-48
Juniper
M160
Borrowed OC-48
interface - may not
be able to keep
G10 Needs configuration
H7
No scheduling;
sufficient capacity
for e-VLBI
Backup plan
Juniper
M40
G9
MAX performance node;
need details
Test
WS?
H5
H1
G12
GigE
GigE
Trans-
G2
Summit
5i
G4
LX (~14km)
GSFC/Bldg 28 (Pat/Paul/Kevin/Bill F.)
K4
K1
WDM
Will try to schedule
either K2 or K3
exclusively for
e-VLBI experiment
G4 Mac with
Yellowdog Linux
(’clifford’)
GGAO/Bldg 201 (Pat/Chuck)
GigE
K2
GigE LX (~7.25km)
1310
K7
Summit
5i
K3
K5A
K5
Test
WS
Test
WS
K6
K6A
GGAO Antenna Trailer (Bill W./Chuck)
Dedicated - no
scheduling necessary
(~500m)
GigE Summit
L1
5i
K5B
Cisco
12016
K6B
L2
GigE
L3
L5
G4 Mac
(’pluto’)
SX
Mark 5
L4
Test
WS
L6
Windows PC
(’xly’)
evlbi012.cdr
Figure 2: e-VLBI Path - ISI-E to GSFC/GGAO
12 July 02