Mark 5 and e-VLBI
Alan Whitney
MIT Haystack Observatory
Goals of Mark 5 VLBI Data System
•
•
•
•
•
•
•
•
•
•
Low cost
Based primarily on unmodified COTS components
Modular, easily upgradeable
Robust operation, low maintenance cost
Easy transportability
Conformance to VSI specification
Compatibility with existing VLBI systems during transition
e-VLBI support
Minimum of 1 Gbps data rate
24-hour unattended operation at 1 Gbps
Tape vs. Disc Price Comparison
7
6
5
Disc Drive
Street Prices
log($/GB)
4
1998 IBM
Disc Projection
3
c
Computer Tape
Media
2
1998 NSIC
Disc Projection
1
LTO Media
Projection
Mark IV/VLBA/K4 Media
0
S2, S3 Media
Disc industry
Projections
-1
-2
1980
1985
1990
1995
Year
2000
2005
2010
Mark 5 VLBI Demonstration System
Mark 5 Development Plan
•
•
•
Funding support is being provided by international consortium of
organization – BKG, JIVE, KVN,MPI, NASA, NRAO & USNO.
Contracted with Boulder Instruments (aka Conduant) to provide
upgraded 1 Gbps version of StreamStor card with external interface
Development program in three stages:
- Mark 5P (‘prototype’) –32 tracks (up to 512 Mbps) from Mark4/VLBA
formatter. In ~May 2002, plan deployment of ~20 systems.
- Mark 5A – 8/16/32/64 tracks (up to 1024 Mbps) from Mark4/VLBA
formatter; parity stripped, reconstructed; direct replacement for Mark
4/VLBA tape drives; expect deployment summer/fall 2002.
- Mark 5B – VSI-compliant, up to 1024 Mbps; will be backwards compatible
to existing Mark4/VLBA correlator systems; fall 2003.
Mark 5 Block Diagram
General Mark 5 Characteristics
•
•
•
•
•
•
•
Uses low-cost IDE/ATA consumer discs
With sixteen 120 GB discs (1.9 TB), will record continuous
1024 Mbps for 4.2 hrs (3 VLBA tapes equal to ~2 TB)
Random data access; fast synchronization to correlator
Data output clock rate can be set internally or driven by external clock
Linux OS
VSI-like command set for Mark 5P/A; VSI-S command set for Mk5B
Cost - ~$22K for initially-deployed systems;
expect DIY system cost <$15K
Other Mark 5 Attributes
•
•
•
•
•
Failsoft: Data is equally spread over many discs; loss or substandard
performance of a disc on record is compensated for automatically;
loss of a disc on playback will lose only fractional data equally over
all ‘tracks’
Operable with 1 to 16 discs, depending on data rate and capacity
requirements
Systems are cascadable for increased capacity from single data source
Can arbitrarily mix recording modes and data rates on same set of
discs
StreamStor card will support 200 MB transfer to discs for general use
In progress
•
•
•
•
•
Full integration into NASA/GSFC FS underway.
Integration into Mark 4 correlator ~90% complete; routine
correlation now possible.
Individual disc-performance statistics to weed out marginal or
failing discs
‘Bank-switching’: Discs organized into two 8-disc ‘banks’ which
can operate in a ‘ping-pong mode’ for uninterrupted datarecord/playback; inactive disc-bank may be replaced at leisure. Will
significantly simplify disc management as disc capacities evolve and
different stations have different disc capacities.
StreamStor files will appear as standard Linux files
Mark 5 Prototype Unit
Mk4 Fringe Plot
3C371.plnfsl, 081-1903_mark5, EV
WESTFORD - WETTZELL, fgroup X, pol RR
Fringe quality 9
SNR
31.9
PFD
0.0e+00
Intg.time187.559
Amp
5.085
Phase -165.3
Sbdelay (us)
-0.003717
Mbdelay (us)
0.004114
Fr. rate (Hz)
-0.003022
Ref freq (MHz)
8182.9900
AP (sec) 3.000
Exp.
I02081
Exper # 2944
Yr:day 2002:081
Start 190303.00
Stop 190621.00
FRT 190443.00
Corr. date:
2002:094:152325
Fourfit date:
2002:094:155038
Position (J2000)
18h06m50.6807s
+69Ê49'28.109"
Amp. and Phase vs. time for each freq., 22 segs, 3 APs / seg (9.00 sec / seg.), time ticks 10 sec
U
Validity
L
Parity
U
Bias
L
Level
U
L
E (SU 5)
V (SU 1)
EV
EV
EV
8862.99
8842.99
8782.99
8682.99
8562.99
8422.99
8222.99
8182.99
Freq (MHz)
-165.5
-154.1
-172.5
-165.7
-163.0
-177.7
-161.4
-162.1
Phase
5.1
4.4
5.0
5.4
4.8
5.1
5.2
6.1
Ampl.
33.0
32.9
32.5
32.8
33.1
33.0
33.4
32.9
Sbd box
64/0
64/0
64/0
64/0
64/0
64/0
64/0
U/L 64/0
APs used
10:10
10:10
10:10
10:10
10:10
10:10
10:10
E:V 10:10
PC freqs
66:45
82:-156
-12:108
9:-153
-67:-79
-147:36
-83:169
E:V 7:41
PC phase
0:0
0:0
0:0
0:0
0:0
0:0
0:0
E:V 0:0
Manl PC
64:64
63:60
70:70
72:64
66:57
61:63
63:63
E:V 70:61
PC amp
X8U
X7U
X6U
X5U
X4U
X3U
X2U
Chan ids
E X1U
3,5
26,28
22,24
18,20
14,16
10,12
6,8
2,4
Tracks
X8U
X7U
X6U
X5U
X4U
X3U
X2U
X1U
Chan ids
V 2,4
3,5
26,28
22,24
18,20
14,16
10,12
6,8
Tracks
Group delay (usec) -2.85655692329E+02 Apriori delay (usec) -2.85659805834E+02 Resid mbdelay (usec) 4.11350E-03 +/-1.8576000E+01 Resid sbdelay (usec) -3.71666E-03 +/Sband delay (usec) -2.85663522493E+02 Apriori clock (usec)
0.0000000E+00 Resid phdelay (usec) -5.61253E-05 +/Phase delay (usec) -2.85659861959E+02 Apriori clockrate (us/s)
-3.76839E-07 +/-4.92299670642E-01 Resid rate (us/s)
-4.92300047481E-01 Apriori rate (us/s)
Delay rate (us/s)
-165.3 +/Apriori accel (us/s/s) -6.99713916328E-06 Resid phase (deg)
-285.8
Total phase (deg)
RMS Theor.
Amplitude
5.085 +/- 0.159
Pcal mode: NORMAL, NORMAL
ph/seg (deg) 9.4
8.1
Search (256X256)
4.989
Pcal rate: 4.474E-08, 3.726E-08 (us/s)
amp/seg (%) 14.6 14.1
Interp.
5.012
Bits/sample: 1
ph/frq (deg) 6.7
4.8
Inc. seg. avg.
5.097
Sample rate(MSamp/s): 8
amp/frq (%) 8.9
8.3
Inc. frq. avg.
5.099
Control file: ../cf_2944 Input file: /data1/2944/081-1903_mark5/EV..plnfsl Output file: Suppressed by test mode
All
-165.3
5.1
32.9
2.0E-05
4.5E-03
1.2E-06
1.1E-08
3.7
First Mark 5
‘Operational’ Fringes
Mark 5 e-VLBI Connectivity
•
Mark 5 supports a triangle of connectivity for e-VLBI requirements
Disc array
Data Port/FPDP
PCI bus/Network
(64bit/66MHz)
Mark 5 can support several possible e-VLBI modes:
• e-VLBI data buffer (first to Disc Array, then to Network); vice versa
• Direct e-VLBI (Data Port directly to Network); vice versa
• Data Port simultaneously to Disc Array and Network at ~800 Mbps
e-VLBI Demo Experiment w/Mark 5
•
•
•
•
•
•
Goal to demonstrate ~1 Gbps near-real-time and real-time VLBI
between Westford antenna in MA and NASA/GSFC antenna in MD
Funded by DARPA
Data will be transmitted to Haystack correlator for correlation
Significant portion of route is through shared networks
Mark 5A systems at all sites
Uses Gigabit Ethernet through standard network connections
Westford
Haystack
(correlator)
USNO
(correlator)
NASA/GSFC
GLOWNET: Gigabit Lincoln Optical WDM Network
Millstone
2
Lincoln
Laboratory
GLOWNET
4
MIT
4
6
230 Congress St.
Bossnet to Washington D.C.
Bossnet: Boston South Optical Network
Bossnet Route
Boston
Stamford
White Plains
Framingham
New York City
Providence
Newark
Bridgeport
Trenton
Philadelphia
Washington DC
Wilmington
Perryville
Baltimore
Greenhill
New London
New Haven
Details of path from Haystack to ISI-E – work in progress!
Westford Antenna
GigE 1 Summit
WS/
Mark 5
A2
Millstone RPE Bldg (Lorraine)
Dedicated no scheduling
GigE
8
1i
A1
Summit
A4
5i
A3
Haystack Correlator
C2
GigE
Dedicated no scheduling
WS/ 28-1A
Mark 5 B2
14
B1
13
B6
B5
GlowNet
WDM
Amp
D3
D2
Lincoln Lab (Lorraine/Steve)
Schedule to be
coordinated with LL
To be replaced by
Juniper M20 Jan 02 with
Cisco
OC-48 to Bossnet
E3
E5
GlowNet
D1
Bossnet(Steve/Jim)
ISI-E
E7
GigE
E4
To be provided
by Haystack
To be upgraded from
2 to 3λ − ϑαν 2002
Scheduling on
Bossnet calendar
GigE Summit GigE Summit GigE
1310
5i
5i
E2
E6
E1
Cisco
1500
GigE
Congress St
Boston
B3
1500
(Shared with LL on
scheduled basis)
WDM
GigE
LL
Plan install by 1 Jan 02
C3
16
GigE Summit
5i
GigE
15
WS/ 28-1B
Mark 5 B4
Schedule to be
coordinated with LL
C1
Test
WS
MIT
Campus
WDM
F1
1510
Amp
Amp
F2
E8
Other users
(no control)
To be updated
to OC48 Jan02
evlbi011.drw
Figure 1: e-VLBI Path - Haystack to ISI-E
20 Dec 01
Details of path from ISI-E to GSFC Antenna – work in progress
ISI-E (Tom)
Scheduling on
Bossnet calendar Plan to update to
OC-48 ~Jan 02 Plan mid Jan 02
GigE
Trans- GigE G4
G2
ponder
G1
OC-48
G3
GigE
H9
GSFC/Bldg 28 (Pat)
GigE
K2
WDM
Will try to schedule
either K2 or K3
exclusively for
e-VLBI experiment
K1
K3
Mac with
Yellowdog Linux;
need details
J2
Summit
5i
H7
OC-48
Borrowed OC48
interface - may not
be able to keep
J3
Cisco
3508
Switch
J4
GigE
H3
J5
WDM
GigE
J6
LuxN
GigE
L1
GGAO Antenna Trailer (Bill)
On order
Summit
5i
K6
1600 ft of fiber
to be installed
Jan 02
WS/
L3
Mark 5
Dell Linux currently
at Haystack will
also be available here
Figure 2: e-VLBI Path - ISI-E to GSFC/GGAO
MAC on order
due Jan-Feb 02
GigE
L2
K5
Test
WS
OC-48
Juniper
M160
Other users ( no control)
Dedicated - no
scheduling necessary
K7
No scheduling;
sufficient capacity
for e-VLBI
H4
H6
GGAO/Bldg 202 (Pat)
GigE
1310
H2
GigE
Need details
Scheduling necessary
if this path used
Preferred e-VLBI path,
but may not be in place.
Pat working with Jerry and Dan.
Other users
(no control)
K4
H1
Backup plan
G11
G10
No jumbo frames
J1
H8
GigE
G12
GigE
H5
Test
WS?
UMCP (Jerry or Dan)
Test
WS?
H7
Switch
Need details
G9
Max @ ISI-E (Jerry)Need details
Juniper
M160
G7
Test
WS?
G6
Summit
GigE G8 Juniper
5i
M40
G5
Tuner
Bossnet WDM
Max @ ISI-E (Jerry)
L4
Eventually replaced
with Mark 5 unit
evlbi011.drw
20 Dec 01
e-VLBI Status
•
•
•
•
•
Detailed examination of all paths in progress; equipment being upgraded
Testing of GigE connection now in progress; ~980 Mbps sustained
TCP/IP to LL (~70 km fiber); tests now in progress to Washington, D.C.
area show ~900 Mpbs possible with some packet loss – under
investigation
Excellent cooperation from Lincoln Labs, ISI, Supernet, GSFC
Staged program:
• Mark 5-to-Mark 5 file transfer
• Mark 5-DAS direct to recording Mark 5 at correlator
• Full real-time DAS to correlator
Expect 1 Gbps demo in late spring 2002
Future Plans
•
•
•
•
Looking at possibility of added dedicated 8 on Haystack-to-LL link
Exploring possibilities to extend path to Europe and Japan using SurfNet
and TransPAC links; possible demonstration with Europe this fall
Examining various options for real-time data-transmission protocols best
suited to VLBI requirements
Add additional GigE interface to Mark 5 and/or emerging 10GigE
interface for higher e-VLBI rates