e-VLBI Development at Haystack Observatory
Alan R. Whitney
Kevin Dudevoir
Chester Ruszczyk
Jason SooHoo
5th Annual e-VLBI Workshop
Haystack Observatory
20 September 2006
Current Projects at Haystack Observatory
•
Network interfacing equipment for e-VLBI
–
•
•
Standardization (VSI-E)
Intelligent Applications
–
–
–
•
•
Automation of e-VLBI transfers an ongoing process
Development of optimization-based algorithms for intelligent
applications ongoing (EGAE)
Intelligent optically-switched networks (DRAGON)
e-VLBI test experiments
Production e-VLBI
–
•
Mark 5 VLBI data system
Put e-VLBI into routine use – progressing well in limited venues
Support e-VLBI development for VLBI2010 initiative
Scorecard of Antenna/Correlator Connectivity
• JIVE Correlator (6 x 1 Gbps)
• Haystack (2.5 Gbps; plans to expand to 10Gbps)
• Westford, MA (10 Gbps to Haystack; 1 Gbps to outside world)
• Kashima, Japan (2.5 Gbps)
• Usuda, Japan (2.5 Gbps)
• Nobeyama, Japan (2.5 Gbps)
• Koganei, Japan (2.5 Gbps)
• Metsahovi, Finland (1 Gbps)
• MPI (1Gbps)
• Tsukuba, Japan (2.5 Gbps)
• GGAO, MD (1 Gbps)
• Onsala, Sweden (1 Gbps)
• Torun, Poland (1 Gbps)
• Westerbork, The Netherlands (1 Gbps)
• Medicina (1 Gbps)
• NyAlesund – (~80 Mbps)
• Jodrell Bank (1 Gbps)
• Arecibo, PR (155 Mbps)
• Wettzell, Germany (~30 Mbps)
• Kokee Park, HA (nominally ~30 Mbps, but currently disconnected)
• TIGO (~2 Mbps)
• Svetloe (speed unknown)
In progress:
• Hobart – expect fiber to station by Feb 2007
• Forteleza – funds secured for fiber connection at 2.5Gbps;
contract has been signed; expect completion Feb 2007
• Zelenchukskaya – details unknown
• Badary – details unknown
VSI-E
•
Goals:
–
–
–
–
–
–
Efficient transport mechanism
Standard protocols
Internet-friendly transport
Scalable to 100Gbps
Suitable for real-time, quasi-real-time, and post-real-time data
transmission
Ability to make use of multicasting to transport data and/or
control information in an efficient manner
•
could be used in the future for support of distributed correlation
VSI-E Status
• Beta version of VSI-E is being tested in transfers from Kashima to
Haystack and NyAlesund to Haystack (has been somewhat delayed by
network availability at Haystack); expect result soon
• Awaiting successful demonstration before final ratification; plan
submission to Internet Engineering Task Force (IETF) as international
standard
• Working to make fast, efficient VSI-E implementation on Mark 5B
DRAGON Collaboration
• Collaborative development of API for control of dynamically optically
switched networks
• e-VLBI experiments using DRAGON optically-switched networks
over part of the path
• 512Mbps intercontinental real-time demonstration at SC05 in Seattle
Bossnet Route
Real-time e-VLBI SC05 Demo
Nov 2005
Real-time transmission and processing of data from
antennas in Westford, MA, Greenbelt, MD, and
Onsala, Sweden at 512 Mbps/antenna
All except Kashima equipped with Mark 5 data
systems; Kashima uses Japanese K5, included via
VSI-E
Correlation results displayed in
real-time at SC05 meeting
Progress towards routine e-VLBI
•
•
•
•
•
April 2005
–
Starting ~June 2005
–
–
Automated regular e-VLBI UT1 Intensive data transfers from Wettzell to
ISI-E (disks hand-carried to USNO for correlation)
A few start-up problems, but now operating fairly smoothly
Spring 2005
–
Commitment to connect Hobart via optical fiber (schedule unknown)
September 2005
–
–
All CONT05 data from Tsukuba transferred to Haystack via e-VLBI
Also – all Kashima and Syowa data now transferred via e-VLBI from Japan
to Haystack
November 2005
–
–
•
Start routine e-VLBI transfer from Kashima and Tsukuba (>200Mbps)
Project initiated to connect NyAlesund to Haystack through NASA/GSFC
at up to 100Mbps
Data now transmitted routinely from NyAlesund, but slow data rate of
~80Mbps does not allow all data to be transferred; decision by NMA is
pending whether to continue, upgrade, etc.
December 2006
–
Funds secured to connect Fortaleza at 2.5 Gbps;
will in place by Feb 2007
e-VLBI Production Transfer
4500
4000
3500
Ny Alesund
2500
Tsukuba
2000
Kashima
1500
1000
500
8/5/06
6/5/06
4/5/06
2/5/06
12/5/05
10/5/05
8/5/05
6/5/05
4/5/05
2/5/05
12/5/04
0
10/5/04
GB
3000
Total data transferred:
Tsukuba ~66TB
Kashima/Syowa ~8TB
NyAlesund ~6TB
EGAE Progress
•
•
•
‘Production’ e-VLBI facility has been established at Haystack to
support routine e-VLBI transfers
EGAE is now supporting routine non-real-time e-VLBI data transfers
from Tsukuba
EGAE will soon be used for routine e-VLBI transfers from Wettzell
and NyAlesund
File-naming for Mark 5B/e-VLBI
For disk2file operations, Mark 5B has adopted the internationally agreed
e-VLBI file-naming convention that uniquely identifies the data and
carries all information necessary to satisfy VSI-E requirements, namely:
<exp name>_<stn code>_<scan name>_bm=<bit-stream mask>.m5b
Example: ‘exp103_ef_scan001_bm=0x0000ffff.m5b’
A file2disk operation will parse the file name and preserve all relevant
information.
Universal adoption of the internationally agreed conventions will result in
easier e-VLBI data transfers with fewer errors.
Future directions
• Continued e-VLBI performance improvements, particularly with Mark 5B
• Continuing development and support of routine e-VLBI transfers;
distribution of EGAE software package to others (MPI, etc)
• e-VLBI with 10GigE connections; motherboard and NIC performance
evaluations
• Development with community of high-speed serial specification for
VSI-E (10GigE, etc)
• Investigation of the use and value of Network Attached Storage as
temporary buffer for e-VLBI data
• Collaboration with NROA/VLBA in e-VLBI development and (hopefully)
deployment
• Continued close association with national and international community for
global e-VLBI development and advocacy