IP Data Transfer System
for Real-time VLBI
Sotetsu Iwamura, Hisao Uose, Kazunari Irie
(NTT Information Sharing Platform Laboratories)
Hitoshi Kiuchi, Shin-ichi Nakagawa
(Communications Research Laboratory)
e-VLBI Workshop@Haystack
1
Outline
NTT’s IP Transfer System for Real-time VLBI in GALAXY
project
• What is GALAXY ?
– Project introduction and activity
– Testbed Network (asynchronous transfer mode)
• IP transfer for real-time VLBI in GALAXY
– Constructed system
– Experiments
• Summary and future plans
e-VLBI Workshop@Haystack
2
GALAXY Project
• A joint research project for real-time VLBI over very
high-speed network (NTT’s 2.4 Gbps testbed), since
1998
• Member: NTT, Communications Research Laboratory
(CRL), National Astronomical Observatory of Japan
(NAOJ), and The Institute of Space and Astronautical
Science (ISAS)
• Prominent experimental results in radio astronomy and
geodesy by GALAXY’s real-time VLBI observation at
256 Mbps and 1Gbps (the world record) of native
asynchronous transfer mode (ATM)
e-VLBI Workshop@Haystack
3
GALAXY Testbed Network
NTT Musashino
R&D Center
Usuda
(ISAS)
Ibaraki
Kashima
(CRL)
Tokyo
Nobeyama
(NAOJ)
Koganei
(CRL)
Kanagawa
Mitaka
(NAOJ)
Chiba
NTT optical fiber
over 200 km
e-VLBI Workshop@Haystack
4
Real-time VLBI System in GALAXY
(native ATM Based)
digital data (ID1):
analog data: 8MHz x 16ch
analog signal
down converter
16Mbps x 16ch, with time stamp
ATM transmitter
sampler
・A/D conversion
・time-stamp
imprint
ATM cells with
sequence numbers at
the top of each cell
VP1
・data conversion to ATM cells
・unique VPI to identify each antenna site
・ extracting data from ATM cells
cross correlator
・sequence number confirmation
・inserting random data to control lost data
results
ATM receiver
digital signal (ID1):
GALAXTY
network
16Mbps x 16ch, with time stamp
Data flow （Max 256Mbps）
e-VLBI Workshop@Haystack
from other
antenna
VP2
5
Why IP for Real-time VLBI ?
Advantages
• Improvement of interconnectivity with other observation
sites
– longer baseline
– available antenna selection
• Easiness to introduce distributed processing schemes
• Utilization of high-performance but low-cost IP equipment
– Gigabit routers, Layer 2 switches, etc...
Disadvantage
• Hard to guarantee quality of service
e-VLBI Workshop@Haystack
6
IP Data Transfer System
for Real-time VLBI
Overview
• Transparent VLBI data transfer over IP; no need to care
about contents and structure of data
• standard ID1 I/F support; to make use of the existing
equipment (sampler, correlator)
ID1 over IP
e-VLBI Workshop@Haystack
7
GALAXY’s IP-based Real-time VLBI System
digital data (ID1):
analog data: 8MHz x 16ch
16Mbps x 16ch, with time stamp
ATM cells with
sequence numbers at
the top of each cell
analog signal
down converter
sampler
cross correlator
ATM transmitter
IP transfer
results
GALAXTY
network
ATM receiver
digital signal (ID1): 16Mbps
x 16ch, with time stamp
existing flow （data rate：
IP transfer256Mbps）
e-VLBI Workshop@Haystack
from other
antenna
8
GALAXY’s IP Data Transfer System
for Real-time VLBI
Technique
• PC-based transfer with parallel IP streams
– to achieve high-speed data transfer with bandwidth scalability
– to examine feasibility of multi-purpose PC for real-time VLBI
• IEEE 1394 for high-speed serial I/F on PC
– Low cost
– Isochronous transfer function suitable to real-time transfer
Basic component of the system
•
•
•
•
ID1 parallelizer: ID1stream --> multiple IEEE 1394 streams
ID1 serializer: multiple IEEE 1394 streams --> ID1 stream
IP-transmitting PC: IEEE1394 stream --> IP stream
IP-receiving PC: IP stream --> IEEE1394 stream
e-VLBI Workshop@Haystack
9
System Configuration
IP-transmitting PCs
(max. 16 PCs)
down converter
sampler
IEEE1394
digital data (ID1): 16Mbps x
16ch, with time stamp
IP
ID1
parallelizer
IP-receiving PCs
(max. 16 PCs)
GALAXY
network
IP
cross correlator
digital data (ID1): 16Mbps x
16ch, with time stamp
IEEE1394
ID1 serializer
ID1 serializer
e-VLBI Workshop@Haystack
10
ID1 Parallelizer/Serializer
parallel transfer to
PCs (max. 16 PCs)
input data rate: 16,32,64,128,256 Mbps
(ID1 clock: 2,4,8,16,32 MHz)
ID1 I/F
IP-transmitting PCs
fixed-length data block
IP
clock source: hydrogen
maser
round robin
IEEE1394
ID1 parallelizer
parallel receive from
PCs (max. 16 PCs)
fixed-length data block
ID1 I/F
to cross correlator
IP
IEEE1394
round robin, with the same
scanning order with ID1
ID1 serializer parallelizer
e-VLBI Workshop@Haystack
IP-receiving PCs
11
Specifications of ID1 Parallelizer / Serializer
ID1 parallelizer
ID1 serializer
Input I/F
ID1 x1
IEEE1394 x16
Output I/F
IEEE1394 x16
ID1 x1
Maximum PCs to connect
16
16
Internal data-block size (KB)
32, 64, 128, 256, 512, 1024
(manually selected)
←
ID1 clock (MHz)
Automatic synchronization to
input ID1 I/F
2, 4, 8, 16, 32
(manually selected)
Maximum throughput (Mbps)
256
256
e-VLBI Workshop@Haystack
12
Appearances of ID1 parallelizer / serializer
ID1 parallelizer
ID1 serializer
IEEE 1394 I/F
ID1 I/F
e-VLBI Workshop@Haystack
13
PCs for Transmitting and Receiving Data
•
•
•
•
IP-transmitting PC: IEEE1394 --> IP
IP-receiving PC: IP --> IEEE1394
Software-based: coded in C language
Protocol between IP-transmitting and IP-receiving PC
– UDP/IP with sequence number inserted in payload
– packet loss detection by counting sequence number
– packet loss control: dummy data insertion at IP-receiving PC
• Platform (for both IP-transmitting and receiving)
– Pentium 2/400
– Linux 2.4.6
e-VLBI Workshop@Haystack
14
Experiments
• VLBI observation experiment
– Data rate: 128 Mbps (32 Mbps x4 streams)
– 4 PCs: Two PCs for each data transmission and receiving
– Hybrid data transfer: IP from Usuda and ATM from Kashima
• Local IP transfer experiment
– Data rate: 256 Mbps (32 Mbps x8 streams)
– 8 PCs: Four PCs for each data transmission and receiving
– IP-transmitting and receiving PCs connected with ether SW
e-VLBI Workshop@Haystack
15
VLBI Observation Experiment with IP Transfer
--The First Real-time VLBI using IP Transfer in The World -J0136+47
Kashima (CRL)
Usuda (ISAS)
Successful !
(2002. 1/17)
Musashino (NTT)
down conv.
128Mbps
down conv.
128Mbps
cross
correlator
sampler
ID1 I/F
ID1 I/F
128Mbps
128Mbps
sampler
IEEE1394
ATM transmitter
IP data
transfer system
ID1 parallelyzer
ID1 serialyzer
IEEE1394
PC
PC
PC
100BaseT 32Mbps×4
IP
ID1 I/F
ATM receiver
PC
100BaseT
GALAXY
GALAXYnetwork
network
e-VLBI Workshop@Haystack
ATM
16
Local IP Transfer Experiment
signal
generator
sampler (time stamping)
ID1 I/F
time code display
256Mbps
ID1 I/F
ID1 parallelyzer
ID1 parallelyzer
IEEE1394
PC
PC
PC
256Mbps
IEEE1394
PC
PC
PC
PC
PC
100BaseT
Ether SW
32Mbps×8
e-VLBI Workshop@Haystack
17
Local IP Transfer Experiment
IP-transmitting PCs
IP-receiving PCs and ID1 serializer
sampler and ID1 parallelizer
e-VLBI Workshop@Haystack
18
Summary
We developed IP data transfer system for real-time VLBI
in GALAXY project
• Implementation
– multi-purpose PC based
– Parallel IP transfer with bandwidth scalability
– ID1 parallelizer and serializer H/W
• Experiments succeeded
– VLBI observation (128Mbps, 4 PCs)
– Local IP transfer (256 Mbps, 8PCs)
e-VLBI Workshop@Haystack
19
Future Plans
• Experiment
– 256 Mbps or higher transfer in VLBI observation
– trial in connecting to overseas sites
• Applying to The Internet:
– advanced error correction
– delay compensation
– dynamic adaptation to network quality of service
• Distributed system
– distributed cross correlation
– distributed data storage, management, ...
e-VLBI Workshop@Haystack
20