CMX  L1 Topo fiber data format and
protocol proposal
Wojtek Fedorko
Version 0.1, Dated 23 May 2013
Introduction
In this document the format of the data sent from CMX to L1Topo processor
is proposed. The format builds on trigger object (TO) formats previously proposed
by S. Hillier. It also specifies line encoding, positioning of the TO data within the
transmission, provisions for data alignment and error checking.
Each CMX provides up to 24 fibers outputs. The current working assumption
is that each type of CMX (Jet, EM/Tau and energy sum) provides 6 fiber outputs to a
given L1Topo processor and this document follows this 6-fiber assumption. Within
this limit up to 24 Jet TOs and up to 30 EM/Tau TOs can be sent per event. The CMX
provides 0 suppression to this amount of TOs from a greater number sent on the
backplane and signals backplane or zero-suppression overflow to L1Topo. At the
time of writing preliminary studies suggest that limiting number of TOs to the
number mentioned may jeopardize physics performance. If these findings are
confirmed extending the number of fiber links between CMX and L1Topo following
the patterns presented in this document will be straightforward. Duplication of the
data stream (for instance to two L1Topo processors or L1Topo and topo function
CMX) may be provided if required by duplicating data from fiber 0 on fiber 6, data
from fiber 1 on fiber 7 etc.
Line rate and encoding
The line rate on the CMX -> Topo serial links is 6.4 Gbps. The data will be
encoded using 8b/10b encoding providing sufficient number of transitions for PLL
lock on the RX side and DC balancing of the lines. Virex 6 and 7 FPGAs provide
hardware support for 8b/10b encoding making the implementation very
convenient. Given the line rate and the encoding the data rate on a single serial link
is 5.12 Gbps providing 128 bits of information per fiber per event. In the rest of this
document this unencoded (or ‘parallel’) data will be discussed as this is the data
available for processing in the FPGA logic
1
TO formats
The TO Formats were presented previously. The proposal is reproduced in
Fig. 1 with a modification that the overflow bit has been removed from the JET and
EM/Tau TO
TO Type: Electron or Tau
4b
3b
3b Local
CPM Num CP Chip Coords
Length: 23
5b isolation
8b electron energy
TO Type: Jet
4b
JEM Num
Length: 28
3b RoI
Frame
2b
LC
9b jet energy (small size)
10b jet energy (large size)
Local Coordinates
TO Type: Energy
Exactly 6 Energy RoI words (in order shown) Length: 96
X
X
15b Signed Ex (restricted eta range)
15b Signed Ex
O
O
Y
Y
15b Signed Ey (restricted eta range)
15b Signed Ey
O
O
T
T
15b Unsigned Sum E (reweighted by JEM)
or
15b Unsigned Sum Et
Sum Et with restricted eta range
O
O
Ex, Ey, ET Overflows
CPM Local Coordinates
JEM Frame
2
3
6
7
0,1,0 0,1,1 1,1,0 1,1,1
3
7
0,1,1 1,1,1
JEM Local Coordinates
2 1,0 3 1,1
0
1
4
5
0,0,0 0,0,1 1,0,0 1,0,1
2
6
0,1,0 1,1,0
0 0,0 1 0,1
1
5
0,0,1 1,0,1
0
4
0,0,0 1,0,0
2013-05-15
Figure 1.
Proposed Trigger Object formats
Trigger Object, alignment data, overflow bit and CRC checksum
placement within the 128 bit transmission
The maximum number of EM/Tau trigger objects possible to transmit on a
single serial link is five per event taking 115 bits of the 128 available. Maximally
2
four jet TO’s can be sent per fiber using 112 bits. A single bit signaling presence of
backplane or zero-suppression overflow also needs to be sent on at least one of the
channels leaving 12 bits on any given channel for error checksum. Based on the
available bandwidth for error checking a Cyclic Redundancy Check of length 12 was
chosen for signaling transmission errors. The error-detection properties and the
computation of the 12 bit CRC checksum are discussed in the next section. The
formats of the Jet and EM CMX to L1 Topo data transmission are shown in Fig. 2 and
3.
3
63
56
55
28
Jet TO 2
27
0
Jet TO 1
127
116 115 114
CRC
63
Ov
56
Jet TO 0
112 111
84
Reserved 0
28
27
0
Jet TO 1
127
116 115
CRC
63
Jet TO 0
112 111
84
Reserved 0
56
83
Jet TO 2
55
28
27
0
Jet TO 1
116 115
CRC
Jet TO 0
112 111
Reserved 0
64
Jet TO 3
Jet TO 2
127
64
Jet TO 2
55
Jet TO 2
83
Jet TO 3
84
Jet TO 3
83
64
Jet TO 2
}
}
}
fiber 0
fiber 1
fiber 5
Figure 2.
Data format of the Jet CMX to L1 Topo transmission. Data shown before the 8b/10b encoding. 16 lowest significant bits of each
channel are loaded first into the megabit transceiver data interface, next bits 31-16 etc. Overflow bit is indicated by ‘Ov’. Reserved
bits are set to 0.
4
63
46
45
23
EM TO 2
127
116 115 114
CRC
92
46
23
92
46
0
69
23
R0
64
EM TO 2
22
0
EM TO 1
116 115 114
68
EM TO 3
45
EM TO 2
EM TO 2
91
EM TO 4
63
64
EM TO 0
112 111
R0
68
22
EM TO 1
116 115 114
CRC
69
EM TO 3
45
EM TO 2
127
91
EM TO 4
63
CRC
0
EM TO 0
112 111
Ov
127
22
EM TO 1
EM TO 0
112 111
92
EM TO 4
91
69
EM TO 3
68
64
EM TO 2
}
}
}
fiber 0
fiber 1
fiber 5
Figure 3.
Data format of the EM CMX to L1 Topo transmission. Data shown before the 8b/10b encoding. 16 lowest significant bits of each
channel are loaded first into the megabit transceiver data interface, next bits 31-16 etc. Overflow bit is indicated by ‘Ov’ and the
reserved bits (set to 0) by ‘R 0’.
5
In the (likely) case that there are less than the maximum number of TOs to be sent in
a given event the existing TOs are front loaded onto the serial links starting with the
first channel and all fields for the non-existent TOs are set to 0. For example in case
there are only two jet TOs to be sent to Topo from a given CMX the first jet TO will
appear as ‘Jet TO 0’ on fiber 0 and the second jet to will appear as ‘Jet TO 0’ on fiber
1. For the Jet CMX, in case that the Jet TOs 2 and 3 on a given fiber are not present
and for the EM CMX, in case EM TOs 2, 3 and 4 are not present on a given fiber, bits
64 to 83 are replaced by a 20-bit alignment word containing K28.5 character and 12
bit BCID. The K character is needed for RX byte alignment and monitoring. The fixed
position of the K character (always on byte 8) (if it appears) gives the receiver
necessary information for initial alignment and subsequent monitoring of the event
boundaries within the data stream, while inclusion of the BCID counter allows for
aligning the information to the same event and subsequent monitoring. The
alignment word is shown in Fig 4.
19
8
BCID
7
0
K28.5
Figure 4. The alignment word
The energy sum information takes up 96 bits including the 6 overflow bits. The 128
bit per event bandwidth allows for fitting the 96 bits of physics information together
with the alignment word and the CRC sum to be sent on every event. However the
constant position of the alignment word demands partitioning of the energy sum TO
as shown on Figure 5.
6
63
X
Y
62
48
Signed Ey (restricted eta range)
127
CRC
47
Y
O
46
32
Signed Ey
116 115 114
100
T
Unsigned Sum E (reweighted by JEM) or Sum Et with restricted eta range
O
99
T
O
31
X
O
30
16
Signed Ex (restricted eta range)
98
84
Unsigned Sum Et
15
X
O
14
0
Signed Ex
83
72
BCID
71
64
K28.5
Figure 5.
Energy TO transmission. Sixteen lowest significant bits are sent first then nits 31-16 etc. ‘ XO’ ‘YO’ and ‘TO’ designate the
overflow bits for the energy sums immediately preceding them.
7
CRC calculation and performance
The cyclic redundancy check provides varying level of transmission error
detection depending on the length of the transmitted data (‘message’) and the length
of the checksum and the choice of so called ‘generator polynomial’. The CRC is
especially well suited for detecting errors common to communication channels. In
particular, provided that the generator polynomial is well chosen the CRC will detect
any transmission error where odd number of bits are flipped and any burst error1 of
length up to the length of the checksum. Protection against arbitrary bit flips
scattered through the transmission is highly dependent on the generator polynomial
choice. Note that since the data is 8b/10b encoded protection against burst errors is
highly desired. If a bit flip occurs in the serialized (10b) data the resulting ten bit
pattern may not be in the in the encoding table, or it is in the table and therefore is
interpreted by the receiver as a different eight bit payload. The megabit transceiver
hardware checks for the first possibility – user logic must monitor the appropriate
port (RXNOTINTABLE) of the transceiver. Error resulting in a valid 10-bit character
will be detected by the CRC performed on the decoded data. A failure resulting in
dark fiber will also be caught by RXNOTINTABLE as within the 8b/10b encoding
long strings of 0’s or 1’s are not permitted. Alternatively the receiver PLL will lose
synchronization. The CRC check does not need to protect against this failure mode.
Since the maximum length of the ‘message’ (or physics+alignment data)
present in the system is 116 bits and the transmission length is 128 we are left with
12 bits for the CRC check. Following recommendations in [1] a generating
polynomial 0x80f has been chosen (in notation with implied leading 1).
Performance of the error check provided by this choice of CRC has been tested using
a software model. The results are:
 All burst errors of length up to 12 bits are caught.
 99.97% of burst errors of length up to 32 bits (inclusive) are caught.
 All errors where two arbitrary bits are flipped in the transmission are caught.
 99.93% of errors where arbitrary four bits are flipped within the
transmission are caught.
 99.95% of errors where arbitrary six bits are flipped within the transmission
are caught.
Detection of more complicated errors was not tested, however it may be desirable to
test against possible single bit flips giving valid 10-bit characters resulting in more
than one length 8 burst errors scattered through the transmission.
8