Integrating Timepix(3)
Szymon Kulis, Mathieu Benoit, Bas van der Heijden, Frans Schreuder,
Henk Boterenbrood, MvB
and the Timepix3 designers
Xavi Llopart, Tuomas Poikela, Vladimir Gromov, Francesco Zappon
Massimiliano de Gaspari
and others
10 December 2013
Timepix versus Timepix3
Timepix
Timepix3
55x55 um2 pixels
55x55 um2 pixels
256x256 matrix = 1.4x1.4 cm2
256x256 matrix = 1.4x1.4 cm2
ToA or ToT measurement
ToA and ToT simultaneously
frame based readout
data driven readout
zero-suppressed frame readout possible
max. 120 frames/sec,
with 100 tracks/frame
= ~10k tracks/s (more is possible)
(tens of) millions of tracks/s
theoretical max of 80 Mhits/s
dead time due to read out of single
frame: 8+ ms
dead time less
not radiation hard
radiation hard; several hundred Mrad but
not SEU robust
FitPix or Relaxd readout
SPIDR or FitPix readout
Martin van Beuzekom
Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013
2
Timepix integration in common DAQ





Detailed talk given by Mathieu Benoit in WP9.3 meeting on Nov. 21
Lots of info in that talk, will not repeat it here. Please look at
https://indico.desy.de/conferenceDisplay.py?confId=8849
Main bottle-neck: Timepix is either in acquisition mode, or in readout
mode
Readout time of 8 (or more) ms per frame -> long dead time
Operation of FitPix and Relaxd is similar
Martin van Beuzekom
Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013
3
Our (Nikhef/LHCb) focus is on Timepix3



In view of the LHCb VeloPix developments (ASIC submission aimed for Q3 2014)
VeloPix which has to cope with 600 Mhits/s
But we also have a complete and working Timepix/Relaxd telescope
Martin van Beuzekom
Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013
4
SPIDR
Speedy PIxel Detector Readout
Readout system for Timepix3 and MPX3.1/RX
over 10 Gb or 1 Gb Ethernet
Martin van Beuzekom
Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013
5
SPIDR

Firmware properties:




Vendor independent and highly configurable
1 Timepix3 at full (80 Mhits/s) speed, or multiple Timepix3 chips at
lower speed
LFSR lookup tables in FPGA
Pixel data over UDP/IP, slow control over TCP/IP

Currently running on a development system (Xilinx VC707)

Multiple setups running at Nikhef and CERN
Development of Compact SPIDR ongoing (but at a slower pace)

+/- 400V Bias supply I2C
FMC
PHY SFP+ cage 1G/10G
FGPA
USB
DC/DC
1V2
DC/DC
1V5
DC/DC
2V5
12V
Trigger/busy
Ext bias
Expansion header
Martin van Beuzekom
Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013
6
Multiple TPX3 hardware configurations
(Thanks to Jerome Alozy)
Nikhef
Chipboard
Nikhef
Chipboard
Nikhef
Chipboard
FMC
extender
cable
50cm
FMC
extender
cable
50cm
FMC
extender
cable
50cm
Xilinx VC707
development board
Martin van Beuzekom
Nikhef
Compact
readout
Cern
Cern
Chipboard
Chipboard
FMC 2
VHDCI
Nikhef
Compact
readout
Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013
FitPix
7
Data acquisition

To run at high speed, each SPIDR needs its own DAQ PC



Data is streamed to disk, without looking at the data





Not possible to evaluate each packet at maximum speed
Separate monitoring stream which samples (copies) the main data stream
DAQ Format: header of several kBytes (settings etc. ) followed by a
stream of pixel packets, 8 bytes each


A single timepix3 at max. speed produces 5.12 Gbits/s
Including the extended time-stamp and overhead this fills ~70% of a
10 GbE link
Header format not yet frozen
No trigger/event number or alike
Pixels packets are unique by their time-stamp
Synchronisation and checking of different SPIDR data streams is
important
Martin van Beuzekom
Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013
8
Data acquisition II
63
60 59
hdr 4b
44 43
PixAddr 16b
20 19
30 29
ToA 14b
ToT 10b
FTOA
4b
33
SPIDR time 16b




0
SPIDR time 16b
18 17
ToA 14b
4
3
FTOA
4b
0
time stamp
30 bits, 25ns resolution
+ 4 bits, 1.56 ns resolution
range: 26.844 sec
However 26 seconds is too short for a normal run
Timepix3 has an internal 48 bit counter, which is reset with the T0-sync


16 15
resolution 25 ns, range 81.44 days
This timer can be read on request, and has unique packet header
-> timer data will end up in the data-stream
Hence to cover a larger time range, the (Leon) processor in the SPIDR
FPGA will request readout of this timer every second.
Martin van Beuzekom
Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013
9
Running with multiple SPIDRs
repeater
repeater
SPIDR
SPIDR
SPIDR
busy
TLU
clock, T0-sync, shutter



Required TLU functionality can be very basic
Provide clock, shutter, T0-sync and combine busy signals (OR)
Somewhat more advanced functionality required for high speed
monitoring (next slides)
Martin van Beuzekom
Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013
10
Interface to TLU, sync time-stamp

Synchronising time-stamps



Disable shutter
Send T0-sync, min. 25 ns
Enable shutter
shutter (ext)
tpx3-reset (int, optional)
25 ns (or more)
T0-sync (ext)
> 1.6 us
time-stamp
data-out
w.o reset
data-out
with reset
Martin van Beuzekom
> 25 ns
n-2 n-1 n
0
1
2
3
4
5
6
7
8
no data packets
no data packets
Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013
11
TLU interface, Busy



Data flow is controlled/halted via shutter/busy mechanism
Busy is tied to ‘almost full flag’ of SPIDR ethernet buffer
DAQ PC signals busy/overflow by sending pause-frames to SPIDR


In addition DAQ PC can send a ‘halt’ command to Leon processor


-> SPIDR buffers fill up, leading to the SPIDR pulling the busy
This will pull also pull the busy line
Will be implemented soon
shutter (ext-in)
busy-0 (out)
busy-1 (out)
data-out0
data-out1
Martin van Beuzekom
no data packets
no data packets
Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013
12
Monitoring of data streams




Monitoring of data streams by copying a fraction of the data
Send data to a dedicated monitoring/slow control PC
Two sampling methods
Software sampling




Each DAQ PC takes a snapshot at a given time (CPU timer triggered, or via run
control?)
However DAQ streams are not synchronised
-> need a large snapshot to guarantee overlap between streams
Hardware sampling





Dedicated ethernet packets (different destination IP address, and/or port number)
are generated by SPIDR
Copies data from a range of timestamps to this dedicated monitoring packet
Start of monitoring sample could be controlled by TLU
Length of monitoring sample defined by SPIDR setting
This is foreseen, but not yet implemented
Martin van Beuzekom
Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013
13
Summary & Plans



Development of Timepix3 readout is actively ongoing (Nikhef + CERN)
But building a high speed readout is not trivial (so we have to be
patient)
We (LHCb) are building a high speed Timepix3 telescope




Timepix3 is good exercise for VeloPix which produces 8x more hits/tracks
VeloPix has to be submitted Q3 next year
Integrating Timepix3 into an AIDA telescope seems simpler than
integrating Timepix
Matching of data from different SPIDR streams using time-stamps


Checking of synchronicity is key
Separate monitoring data stream
Martin van Beuzekom
Timepix(3) integration, AIDA common DAQ, 10 Dec. 2013
14