News on nSYNC and nODE
for the LHCB muon upgrade
S. Cadeddu, A. Cardini, L. Casu, A. Lai, A. Loi – INFN Cagliari
P. Ciambrone – INFN LNF
nSYNC architecture
48 Input channels:
•
•
•
•
Fully digital TDC with programmable resolution (8-32
slices; 3,125 – 0,78 ns) @ 40MHz.
Nominal resolution: 16 slices
Programmable pipeline to synchronize different
channels.
Histogram block for each channel
BX tagging:
•
12 bits BXcnt, programmable length, preloaded with
12 bit programmable value when BCres signal is
asserted.
Frame builder and TDC Zero suppression:
•
•
•
Frame fixed will be implemented
Header will be 16 bits wide
Full Hit Map NZS + TDC data ZS according to the Hit
Map (starting from channel number 0).
GBTx interface via e-LINK
TFC interface decoder
I2C Interface
L. Casu: News on nSYNC and nODE - LHCb Week
07/03/2016
2
VTTx
– Slave GBT
GBTx
•
1 GBTx forTFC/ECS
– MasterGBT
GBTx
•
1 GBT-SCA
•
2 VTTx
•
1 VTRx
VTTx
GBTx
VTRx
L. Casu: News on nSYNC and nODE - LHCb Week
07/03/2016
GBTx
ECS
nSYNC
48Input ch
4 GBTx for hit+ TDC data
nSYNC
48Input ch
•
GBTx
nSYNC
48Input ch
4 nSYNC @ 48 channels
nSYNC
48Input ch
•
TDC out Hit map TDC out Hit map TDC out Hit map TDC out Hit map
nODE
GBT
SCA
3
TDC architecture and test results
CHANNEL BLOCKS
L. Casu: News on nSYNC and nODE - LHCb Week
07/03/2016
4
TDC introduction
48 Input channels:
•
One TDC for each channel
TDC:
•
Fully digital TDC with programmable resolution (8-32
slices; 3,125 – 0,78 ns) @ 40MHz.
•
Resolution is the number of slices the master clock is
divided
•
Nominal resolution: 16 slices
•
Measure the incoming signal phase with respect to
the 40MHZ master clock.
•
Every clock cycle two info are output:
–
–
a flag giving the simple binary info Hit/NoHit
a 5 bit-wide word with the measured phase
L. Casu: News on nSYNC and nODE - LHCb Week
07/03/2016
5
TDC architecture
•
Uses the Giordano-DCO developed for ALLDIGITALL and
for which we have a patent accepted in Italy and we are
extending in Europe.
•
The DCO is required to generate a fast clock with a period
equal to (25/res) ns.
•
TDC measurement is done with a fast counter running
with the clock DCO generated.
•
TDC has an automatic calibration tool that can be
activated on request.
•
•
TDC uses a dithering system to minimize the systematic
errors due to discrete delay unit.
•
•
TDC is active only when a signal arrives, to minimize the
switching power consumption.
•
•
•
L. Casu: News on nSYNC and nODE - LHCb Week
07/03/2016
DCDL Based on a high-fan-out network (HFN)
feeding a multi-input delay chain
HFN can be easily managed with standard tools
to minimize skew between in and the input of
each delay (t0)
tpd is the propagation delay of each mux
Thermometric encoder determines the number
of muxes crossed from in to out => the total
delay Dt=ntpd+t0
Full automatic implementation with standard
CAD tools
6
TDC Layout
•
•
•
•
Size: (90 x 171) mm2
Working Power consumption: <100mW
UMC130 nm Technology
It was submitted in February 2015 in the
new ADV2 chip, with two identical TDC
integrated.
L. Casu: News on nSYNC and nODE - LHCb Week
07/03/2016
7
TDC Calibration: Test Results
•
Digital words controlling the DCO and Dithering are output from the TDC and can be used to verify
the TDC Calibration results.
•
The plot below shows the comparison between the DCO control word values from Post Layout
Simulation vs values from the TDC’s under test
LHCb
L. Casu: News on nSYNC and nODE - LHCb Week
07/03/2016
8
TDC Phase Meas.: Results
•
•
This test is performed comparing the phase measured by
the Scope and the TDC under test read-back.
Below are reported the results for the resolution of 16 (that
will be used @ LHCb)
TDC 0
L. Casu: News on nSYNC and nODE - LHCb Week
TDC 1
07/03/2016
9
•
TDC Phase Meas.: Results
•
This test is performed comparing the phase measured by
the Scope and the TDC under test read-back.
Below are reported the results for the resolution of 16 (that
will be used @ LHCb)
TDC 0
TDC 1
L. Casu: News on nSYNC and nODE - LHCb Week
07/03/2016
10
TDC Results: Sigma plots
•
•
Below are some plots comparing the error sigma plots vs resolution. The LSB/2 and the Theoretical
sigma curves are plotted as reference
The Sigma curves obtained from TDC’s under test are shown.
LHCb
L. Casu: News on nSYNC and nODE - LHCb Week
07/03/2016
11
Histogram and pipeline
CHANNEL BLOCKS
L. Casu: News on nSYNC and nODE - LHCb Week
07/03/2016
12
Histogram block
•
Each input channel is equipped with a TDC and a Histogram facility block.
– Build phase histogram of the incoming input signal.
•
The block consist of 16 counters 24-bit wide each one.
– When a Hit occurs, the corresponding counter is increased by one
– When one of the counters become full, all the block is frozen until a reset occurs.
•
TDC has a programmable resolution ranging from 8 to 32.
– When TDC res > 16, it is possible to configure the histogram block in “extended mode”.
– The 16 counters are divided by two becoming 32 counter 12-bit wide equivalent.
•
Histogram block can also build histogram w.r.t the assigned BXid.
– Both normal and extended mode can be used.
– BXid is evaluate as whole 12-bit wide, in order to histogram only the Hits arriving in the first
16/32 bunches.
L. Casu: News on nSYNC and nODE - LHCb Week
07/03/2016
13
BXID SYNCHRONIZATION
L. Casu: News on nSYNC and nODE - LHCb Week
07/03/2016
14
Bunch Counter block
•
The Bunch Crossing counter is a 12-bit wide counter increased @ 40MHz.
•
When it reaches the maximum value it wraps around.
–
–
•
The wrapping around point can be programmed via ECS.
Default value is 3653 (means 3654 bunches per orbit starting from zero).
The synchronization is achieved by means the signal BCres received through the TFC
interface.
– Value is loaded into the Bunch Crossing counter when BCres is asserted.
– Value to be loaded is programmable via ECS (full 12 bits).
•
The counter is hamming protected and the correct value is refreshed at every master
clock cycle
L. Casu: News on nSYNC and nODE - LHCb Week
07/03/2016
15
FRAME FORMAT
L. Casu: News on nSYNC and nODE - LHCb Week
07/03/2016
16
Fixed Frame – Our choice
Header
Header
•
Data Frame: 112 bits
48 Hit bits
4 4 4 4 4 4 4 4 4 4
48 Hit bits
5 5 5 5 5 5 5 5
8 edac
8 edac
Frame using the 8 bits Hamming/EDAC, hits NZS, two possibility for TDC data (depending
on the resolution), 16 bits for the Header
– 4 TDC bits: max 10 channels (20,8%), “full frame”
– 5 TDC bits: max 8 channels (16,6%), “full frame”
Header
Header
•
Data Frame: 112 bits
48 Hit bits
4 4 4 4 4 4 4 4 4 4 4 4
48 Hit bits
5 5 5 5 5 5 5 5 5
Frame using no protection, hits NZS, two possibility for TDC data (depending on the
resolution), 16 bits for the Header
– 4 TDC bits: max 12 channels (25%), “full frame”
– 5 TDC bits: max 9 channels (18,7%), three bits “empty”
L. Casu: News on nSYNC and nODE - LHCb Week
07/03/2016
17
TDC res=4 bits
WOI No Hamming Hamming
0
TDC ZS
TDC ZS
1 TDC ch 0…11 TDC ch 0…9
2 TDC ch 8…19 TDC ch 8…17
3 TDC ch 16…27 TDC ch 16…25
4 TDC ch 24…35 TDC ch 24…33
5 TDC ch 32…43 TDC ch 32…41
6 TDC ch 40…47 TDC ch 40…47
7
TDC ZS
TDC ZS
NZS TDC data transmission
•
TDC res=5 bits
No Hamming Hamming
TDC ZS
TDC ZS
TDC ch 0…8 TDC ch 0…7
TDC ch 8…16 TDC ch 8…15
TDC ch 16…24 TDC ch 16…23
TDC ch 24…32 TDC ch 24…31
TDC ch 32…40 TDC ch 32…39
TDC ch 40…47 TDC ch 40…47
TDC ZS
TDC ZS
System based on “Window Of Interest” (WOI)
–
–
–
–
–
–
–
TDC are grouped in “window” of at least 8 TDC channel
Via ECS we can select the WOI that we want, bypassing the ZS algorithm.
TDC values transmitted into the WOI do not depend on priority or other stuff.
They can be monitored in NZS mode.
WOI are set via ECS
Setting a bit in configuration the WOI is activated to be transmitted continuously.
When a NZS command is asserted by the TFC, the nSYNC will transmit the corresponding WOI set via ECS
L. Casu: News on nSYNC and nODE - LHCb Week
07/03/2016
18
Frame patterns
•
In case Sync TFC command is received, the frame will be:
– Header (12 BXid bits) + sync_pattern (16 bits) + all zero’s
– The Hamming bits will be placed at the same position
– The sync_pattern is set via ECS
•
A 16-bit wide fixed pattern can be set via ECS.
– Whole frame is composed by the fixed pattern replicated up to fill the frame
•
LSFR pseudo-random pattern.
– Programmable seed
•
TFC command used as pattern
– Header (16 bits) + TFC command + zero’s
L. Casu: News on nSYNC and nODE - LHCb Week
07/03/2016
19
Output Frame
•
Data frame is output @ 320MHz
– 320MHz clock is generated by an internal PLL starting from the 40MHz master clock coming
from TFC GBT master.
– Frame is splitted into 8 parts
•
40 and 320 MHz domains are interfaced with an asynchronous FIFO.
•
There is a 16 steps programmable output pipeline in order to allow the correct alignment
from the receiver side (GBTX)
Signal
gbt_out[0]
gbt_out[1]
gbt_out[2]
gbt_out[3]
gbt_out[4]
gbt_out[5]
gbt_out[6]
gbt_out[7]
gbt_out[8]
gbt_out[9]
gbt_out[10]
gbt_out[11]
gbt_out[12]
gbt_out[13]
L. Casu: News on nSYNC and nODE - LHCb Week
07/03/2016
Elink
dlO[1]
dlO[5]
dlO[9]
dlO[13]
dln[0]
dln[4]
dln[8]
dln[12]
dln[16]
dln[20]
dln[24]
dln[28]
dln[32]
dln[36]
Frame
FRMUP[7:0]
FRMUP[15:8]
FRMUP[23:16]
FRMUP[31:24]
FRMUP[39:32]
FRMUP[47:40]
FRMUP[55:48]
FRMUP[63:56]
FRMUP[71:64]
FRMUP[79:72]
FRMUP[87:80]
FRMUP[95:88]
FRMUP[103:96]
FRMUP[111:104]
20
TFC INTERFACE
L. Casu: News on nSYNC and nODE - LHCb Week
07/03/2016
21
TFC interface
13
Synch
11
9..7
5
3
NZS
Snapshot Calibration Type(1,0)
FE Reset
Mode
1
BXID
Reset
From the TFC (via GBTx master) we receive the 40 MHz master clock and several commands
General remarks:
– The TFC Command Time alignment will be managed through a pipeline programmable via ECS
• 16 steps programmable pipeline
– After this alignment the TFC commands will be executed as they exit from the pipeline
BXid reset:
– Received every LHC orbit
– When asserted, the BXid counter will be loaded with the value set via ECS
FE reset:
– Reset all the logic involved in the data acquisition. Are excluded:
• BXid counter
• Status and monitor counters
• Configuration registers
L. Casu: News on nSYNC and nODE - LHCb Week
07/03/2016
22
TFC Interface
13
Synch
11
9..7
5
3
NZS
Snapshot Calibration Type(1,0)
FE Reset
Mode
1
BXID
Reset
Calibration Type:
– TDC calibration (Calib_type[0])
– Error register Reset (Calib_type[1])
NZS:
– We transmit the WOI TDC when a NZS command is received.
SYNCH:
– Our Frame Header becomes 12 bits wide instead of 16, we transmit a pattern 16 bits wide
– It is protected using Hamming code according to the ECS settings
Snapshot:
– It latches a set of registers: command counters, error counters, FIFO status.
L. Casu: News on nSYNC and nODE - LHCb Week
07/03/2016
23
13
TFC: Command counters
Synch
•
11
9..7
5
3
NZS
Snapshot Calibration Type(1,0)
FE Reset
Mode
1
BXID
Reset
We have implemented command counters for monitoring:
–
–
–
–
–
–
One for each command.
They counts the number of clock cycles the TFC commands are active.
They are 32-bit wide.
“Snapshot” command latches these counters into other counters accessible via ECS.
They are reset by “Snapshot” command
The maximum time they can monitor depends on the commands rate.
Max rate
(% of clock cycle the command is active)
100%
10%
1%
0.03%
L. Casu: News on nSYNC and nODE - LHCb Week
Time accumulated
(aprox)
107 s
18 min
Almost 3h
More than 4 days
07/03/2016
Notes
BCres frequency
24
ECS INTERFACE
L. Casu: News on nSYNC and nODE - LHCb Week
07/03/2016
25
ECS
•
All the nSYNC functionalities are controlled and configured using the I2C protocol:
– Reading of read-only register by mean a sub-addressing protocol
– Setting write/read configuration registers
– Execute such kind of commands (e.g. calibration, histogram reset, …)
•
An equivalent TCF commands set is implemented and triggered by ECS.
• Status and counters registers are available on the ECS bus in three different modes :
– The live value to see the functioning of the counters and the system
– the latched value triggered by the snapshot command.
– the latched value triggered by an error or abnormal condition.
L. Casu: News on nSYNC and nODE - LHCb Week
07/03/2016
26
LAYOUT AND PACKAGE
L. Casu: News on nSYNC and nODE - LHCb Week
07/03/2016
27
Layout
Size: about 4.4 x 3.8 mm2
Input signals
Technology: UMC130 nm
TFC inputs
Voltage Supply:
I/O ring: 3.3V
Core: 1.2V
I2C and global signals
Pin count: 125
Package: CQFP 160
GBT data out
L. Casu: News on nSYNC and nODE - LHCb Week
07/03/2016
28
Conclusion
•
TDC prototype was designed, produced and tested
–
–
•
All the main blocks has been designed and simulated
–
–
–
–
–
–
–
•
Test shows good performance, in agreement with the simulation.
Performances fit the needs for nSYNC project
Histograms.
BXid synchronization
Frame builder
TDC Zero Suppression
GBT interface
TFC interface
ECS
nSYNC layout design has been completed.
The nSYNC prototype for the LHCb muon upgrade has been submitted on
19/02/2016.
L. Casu: News on nSYNC and nODE - LHCb Week
07/03/2016
29
SPARE
L. Casu: News on nSYNC and nODE - LHCb Week
07/03/2016
30
SYNC 1.0 vs nSYNC
8 vs 48 input channels
Programmable TDC res.
Histo on each channel
No Memories
Also TDC data transmitted
@40MHz
TDC data ZS implemented
directly on nSYNC
97 vs 125 pins
L. Casu: News on nSYNC and nODE - LHCb Week
07/03/2016
31
TDC Calibration
•
Calibration is performed launching a TDC measure with a calibration signal at the
beginning of the clock cycle. The TDC count is compared with the resolution requested.
For better results we integrate the counts for 8 master clock cycles.
•
Calibration is divided in 4 different phases:
1. The best 6 bit control code is searched using a SAR (Successive Approximation Register)
approach.
2. Refinement of previous found code moving it by +/- 1 and verifying if the result is better.
3. Determining the direction of the dithering
4. Determining how long, in term of DCO generated clock cycles, the dithering must be applied to
minimize the systematic error
•
Phase 3 and 4 can be skipped if the dithering is disabled or if the previous phases
detected the dithering is not needed
•
Digital words controlling the DCO and Dithering are output from the TDC and can be
used to verify the TDC Calibration results.
L. Casu: News on nSYNC and nODE - LHCb Week
07/03/2016
32
TDC Phase Meas.: Results
•
Below are reported some results for resolutions different
from 16 (that will be used @ LHCb).
RES=8
RES=11
RES=13
L. Casu: News on nSYNC and nODE - LHCb Week
07/03/2016
33
TDC Phase Meas.: Results
•
Below are reported some results for resolutions different
from 16 (that will be used @ LHCb).
RES=20
RES=26
RES=32
L. Casu: News on nSYNC and nODE - LHCb Week
07/03/2016
34
TDC Phase Meas.: Results
•
Below are reported error distributions for all the
resolutions.
RES=8
RES=16
RES=22
RES=26
L. Casu: News on nSYNC and nODE - LHCb Week
07/03/2016
35
TDC Results: Sigma plots
Around res of 26 something happens.
•
Calib algorithm fails to discriminate
these small differences
– Between res 26 and 27 there is a
difference of about 40ps/period
– Much less then the DCO step.
– Integration time could be not enough
LHCb
L. Casu: News on nSYNC and nODE - LHCb Week
07/03/2016
36