e-VLBI progress in the South
“Towards e-VLBI”  “e-VLBI”
Tasso Tzioumis
ATNF, CSIRO
September 2006
Outline
• Overview - details in other presentations/posters
• Current LBA systems/developments
–
–
–
–
Tapes - phasing out
DAS - digital filters
Disks - LBADR
Software Correlators
• Network developments
• “Real” e-VLBI tests
• Plans & the Future
LBA
x NTD
x New Norcia
Long baseline Array (LBA)
•
Telescopes in Australia
– ATNF, U.Tasmania, NASA - “core” Oz VLBI network
•
•
+ South Africa
+ Telescopes in Asia Pacific region
– Japan, China, Hawaii, VLBA,…
 Asia Pacific Telescope (APT) (VSOP support)
•
•
Open network (EVN model of operations)
http://www.atnf.csiro.au/vlbi
•
(+ Tests with NZ - fringes ok - no large antenna)
LBA -S2 tape system
• Canadian S2 VCR-based recorder
– 8 VCR recorders  128 Mbps max data rates
– S2 maintenance issues (no external support)
• 7-station S2 LBA correlator at ATNF Sydney
– XF architecture (ATCA correlator design)
– Difficult to upgrade correlator
• The main production-VLBI system for > 12 years
• 2005 workshop: Phase-out S2 within ~3-4 years?
• 2006 workshop: Decommission S2 in 2006-7!!
• Move to disk-systems + software correlation
– Correlation contracted to Swinburne - 1 October 2006
LBA - DAS
• DAS: Based on Digital Filters (> 10 years old!)
–
–
–
–
–
–
–
2 input IFs @ 64 MHz BW (2 pols or 2 frequencies)
Sampling (10 bits) + Digital filters (ideal passbands)
Bands: 64MHz  125 KHz
S2 mode: 2 x 16 MHz ( 128 Mbps to match S2)
Huygens mode: 4 x 16 MHz (256 Mbps)
Maximum: 2 x 64 MHz (512 Mbps) - tested in 2006
Multibit mode: 2 x 16 MHz x 8(10) bits  1 Gbps
• Interference excision?
• ATNF stations: 2 DAS units
– 512 Mbps for Huygens (8 x 16 MHz)
– 1 Gbps by 4 x 64 MHz channels - developed & tested in 2006
• Remains the main DAS for LBA system(s)
– BUT no more units available!!
• Move to new VLBI dBBC units? (Haystack) - timescales?
LBADR - Disk recorders
• (See C. Phillips poster for details)
• PCEVN-Metsahovi VSI interface system
– VSIC (ECL-LVDS) & VSIB(computer)
– Connects to LBA DAS
– All LBA antennas equipped
• COTS computers
– New Tyan “Thunder K8WE” motherboards (2005)
• Multiple PCI bus systems
• Internal disks; IDE removable disks; X-RAID on fibre channel
• (Replaced test systems on Dell 1600SC servers)
• Apple Xserve hardware RAID systems (7 TB)
Software correlators
• See poster by S. Tingay & A. Deller
• FX software correlator(s)
• Swinburne Supercomputer
– 300 Dell computers in cluster
– Real time correlation (“normal” modes) - 30-50 nodes
• Smaller clusters at telescopes
– (Parkes: 32 nodes; ATCA: 16 nodes)
• Cray XD-1 supercomputer in WA + FPGAs
– Under development
• Very versatile & robust (can run on most systems)
• Only correlators in Australia for disk systems!
“Real-time” fringe testing
• Disk-systems
– Can run parallel to S2 VCRs (S2 C2 output port)
– Short segment (~0.1s) data or use full recorded
data
– Transfer to Swinburne - (slow) ftp or fast links
• Correlate and post on web
– Start correlating immediately (not all data needed)
– Fringe in a few secs; web post within minutes
• Semi-automatic operation
• Routine for all LBA observations
– Setup fringe tests for all observations
– http://pentane.ssi.swin.edu.au/~cphillip/
Increased LBA reliability and robustness
Network connectivity -1
• See talk by Shaun Amy
• AARNet3 Regional Network
• Fibre tails to ATNF antennas in 2005-6
–
–
–
–
$2M project, funded by CSIRO corporate
Fibre “tails” owned by CSIRO
“e-vlbi” tests in March 2006 - Official start June 2006!
Legal contract problems!!
• 2 x 1 Gbps connectivity
– 1 operating now; 2nd circuit to be connected
• 10 Gbps (x multiple wavelengths) backbone
• Good international connectivity
AARNet3 Regional Network
Network connectivity -2
• Swinburne: 1 Gbps link(s) constructed
– Operation delayed by legal problems!! (again)
– (10 Gbps link from Parkes?)
– Enables software correlator connectivity
• Hobart: Fibre links funded and in construction
(2006)
– 1 Gbps - telescope ---> University
– 10 Gbps to AARNet
– (BUT Bass-link limited to ~0.5 Gbps?)
• WA Connectivity (Cray @ UWA)
– CeNTIE: 3 x 1 Gbps BUT closes in 2006!
– AARNet3 links planned
– Impact on x-NTD?
• (Tidbinbilla - NASA: 2 Mbps commercial links
First Oz real “e-VLBI”
•
•
•
•
•
•
•
+
Date:
24 August 2006
Antenna:
Parkes, Mopra, ATCA
Data rate:
128 Mbps (per station)
Bandwidth:
2x16 MHz (per station)
Processor:
Dual AMD server (ie only 2 CPUs)
No disk buffering --> “e-VLBI”
Time:
10minutes
• 1-bas test Parkes-ATCA at 256 Mbps/station
Plans - Near term
• Transition production VLBI to disk systems +
software correlation (2006-7)
– Investigate remote-disk recording
– Acquire ~$0.5M of disks
• Extensive “real” e-VLBI tests
– To Swinburne cluster correlator (in 2006)
– To JIVE correlator under EXPReS (2006-7)
• Data disk formatting issues (LBA - Mk5a) - VSI-E??
– Tests to Haystack/Japan/Global - VSI-E??
• DAS replacement?
Special “e-VLBI” Instrument?
• Parkes-Mopra- Narrabri
– 1 Gbps links installed
– 1 Gbps data from 2 x DAS systems
– Software correlation (Parkes or ATCA cluster)
 Real-time VLBI instrument at 1 Gbps
– Cf PTI instrument (Parkes -Tid @ 10 MHz BW)
• Very popular with non-VLBI specialists
– Fast detection observations/surveys
– May operate in parallel with disk recording
• E.g. correlate 2nd 512 Mbps stream from ATNF antennas
Plans - Longer term
• CABB - Broadband upgrade of ATCA
– 4 Gsamples x (8) bits  32 Gbps streams
– 4 IFs per antenna; 8 antennas
– Digital Polyphase filters -- > CABB Correlator
• ATCA operation in 2007-8
– VLBI capability designed-in (2008?)
• Need for VLBI operation:
– n x 10 Gbps connections to ATCA (Narrabri)
– Broadbanding of Rx systems (Mopra, Parkes)
• Narrabri polyphase filters?
– Broadband and Fast links from non-ATNF antennas
 Multiple 10 Gbps inputs & e-VLBI
A Wideband Upgrade for the Australia Telescope Compact
Array
Luneburg Lens Phased Array
Luneburg Lens Phased Array
6 * 22m antenna AT Compact Array
Basic Configuration
Simple Zoom
x 0(m)
x 1(m)
x ρ (m)
xM-1(m)
X0(m)
po(m)
0
p1(m)
1
pρ(m)
ρ
p M-1(m)
M-1
xn
FPGA hardware
may be completely
reprogrammed to
produce many
different filterbank
configurations, as
different
observations may
require.
X1(m)
M-Point
DFT
via
FFT
FFT
FIR
FFT
Multiple Zoom (>2 possible)
Xk(m)
XM-1(m)
DMUX
A 2GHz bandwidth polyphase digital
filterbank with 4096 channels ...
RF, IF and Baseband Signal
Path
FIR
Filterbank
Fringe
Rotators
Correlator
s
... will fit into four XC2V6000 FPGAs
Filterbank Architecture
Compound Zoom
Summary
• S2 tape LBA system to be decommissioned 2006-7
• LBADR - disk recorder developed
• Software correlation - “real time” speeds
– Very versatile, robust and portable
• LBA transition: disk recording + software correlation
• Digital DAS - need to augment/replace
• 1 Gbps links to ATNF antennas operating
– Links to correlator and other telescopes in construction
• First “real” e-VLBI tests successful
• Plans for eVLBI instrument (Pks-Mop-Nar)
• Future: 10 Gbps links + CABB correlator