Particle Physics and Radio Astronomy
Nicola Pezzi

• CDF
• eVLBI
9 th Sept 2004 UKLight Town Meeting 2

• Collider Detector at Fermilab
• PP experiment based at FermiLab
• Involves ~800 physicists (~50 in UK)
• Data stored in the US
– 1000TB per year
– 2000TB stored so far
– 10,000TB predicted for 2008
9 th Sept 2004 UKLight Town Meeting 3

• Get local copy datasets (0.5-10TB)
• re-process with better calibration
• Share data/results with other UK physicists and CDF physicist worldwide
• UK sites include: UCL, Oxford,
Liverpool, Glasgow
9 th Sept 2004 UKLight Town Meeting 4

• Send data from FNAL to multiple UK sites using StarLight/UKLight
• Reprocess data in multiple UK sites and have results transparently visible to all CDF users
• Transfer data back to FNAL and update catalogue
9 th Sept 2004 UKLight Town Meeting 5

• Reliable high speed UK-US connection: currently 1GE between
FermiLab and UCL using UKLight
• Reliable high speed links inside UK
• Access to large CPU clusters:
– Disk storage (~10TB)
– Software installation
9 th Sept 2004 UKLight Town Meeting 6

• Standard protocols for secure file transfer to satisfy local sites and FNAL security restrictions(Kerberos5)
• A transparent file catalogue system to keep track of different copies of the same datasets
• Support for new Grid protocols (EU vs. US standards, Grid3 vs. LCG) since it affects sw for job submission and file catalogue
9 th Sept 2004 UKLight Town Meeting 7

• Use of UCL PC farm and later on
GridPP/Atlas Tier-2 centres
• Modification to CDF software to be more Grid enabled (e.g. file transfer using GridFTP)
• Develop an interface to LCG to make use of UK CPU resources
9 th Sept 2004 UKLight Town Meeting 8

• Inside Fermilab: 64Mbit/s
• Fermilab -> UCL: 5Mbit/s (JANET)
• Fermilab -> UCL: n.a. (UKLight)
• Issues:
– Routing to end hosts OK
– Accounts at FermiLab OK
– Routing to Kerberos servers In progress
9 th Sept 2004 UKLight Town Meeting 9

9 th Sept 2004
1 Gbit/s sustained using new
UDP protocol (UDT)
UKLight Town Meeting 10

• Join Institute for Very Long Baseline
Interferometry in Europe
• http://www.jive.nl
• Benefits:
– Sees through the earth’s atmosphere
– Sees through interstellar dust
– Higher resolution than other wavebands
9 th Sept 2004 UKLight Town Meeting 11

9 th Sept 2004 UKLight Town Meeting 12

9 th Sept 2004 UKLight Town Meeting 13

9 th Sept 2004 UKLight Town Meeting 14

9 th Sept 2004 UKLight Town Meeting 15

• Scientists submit proposals to a review committee.
• About twenty experiments selected.
• Three sessions per year, two weeks of continuous observations
• Up to 20 telescopes per experiment
9 th Sept 2004 UKLight Town Meeting 16

Radio Astronomy – VLBI Storage
• Signals from space are sampled and digitised at up to 1Gbit/s
• Magnetic tape (up to 1TB each) are shipped to data processor for correlation
Hundreds of tapes per session
9 th Sept 2004 UKLight Town Meeting 17

Radio Astronomy – VLBI Problems
• Tape recorders use a 1980s technology and cost $100.000 each
• Each tape costs $1200 and can be used only 30 times
• Record/replay heads cost $9000 each and wear out in 1 or 2 years
• No further increase in performance feasible.
9 th Sept 2004 UKLight Town Meeting 18

Radio Astronomy – VLBI
Correlator
• Highly dedicated super-computer
• 16 thousand billion operations per second
• 1024 correlator chips working in parallel
• Capable of processing 16 X 1Gbit/s data streams
9 th Sept 2004 UKLight Town Meeting 19

• Connecting to the Telescopes is not easy.
• They are deliberately built far from cities so it’s expensive
• The principle must be demonstrated first eVLBI will connect up to 6 Telescope to
JIVE in real-time for 2 years as a proof of concept
9 th Sept 2004 UKLight Town Meeting 20

Radio Astronomy – eVLBI partners
•DANTE/GÉANT
•SURFnet
•GARR
•UKERNA
•PSNC
•DFN
•KTHNOC/NORDUnet
•Manchester University
•JIVE
•Westerbork telescope
•Onsala Space Observatory
•MRO
•MPIfR
•Jodrell Bank
•TCfA
•CNR IRA
Pan-European Network
Dutch NREN
Italian NREN
UK NREN
Polish NREN
German NREN
Nordic NREN
Network application software
EVN Correlator
Netherlands
Sweden
Finland
Germany
UK
Poland
Italy
9 th Sept 2004 UKLight Town Meeting 21

Radio Astronomy – eVLBI Targets
(1/2)
• Evaluate feasibility of eVLBI:
– Interfaces: network, correlator
– Network installation cost and timescale
– Data rates
– Improved reliability
• Standards and new capabilities:
– Network protocols
– Higher data rates
– Parameter tuning for high data rates
9 th Sept 2004 UKLight Town Meeting 22

Radio Astronomy – eVLBI Targets
(2/2)
• To see significant network usage with multiple Gbit streams converging on JIVE from diverse European sources
• Must be seen to enable new science and not just solve an existing data-transport problem
• Real-time operation is seen as the ultimate aim, buffered operation accepted only as a development stage.
9 th Sept 2004 UKLight Town Meeting 23

Radio Astronomy – eVLBI Today
• 6 telescopes connected to correlator
• Tape recorders being replaced by PCs with multiple hard disks
• UDP is used for the PoC
• US partners are developing a version of RTP for eVLBI
9 th Sept 2004 UKLight Town Meeting 24

Radio Astronomy – eVLBI early success
• On 28 th April first real-time elaboration without passing via disks
• 2 hours of full real-time computation were performed from 6 sites (Jodrell Bank in the UK)
• Data rate utilised was 32Mbit/s (JANET)
• Hoping for 512Mbit/s by the end of year
9 th Sept 2004 UKLight Town Meeting 25