EDMS Number:
ATLAS
EDMS Id:
1
ATLAS Level-1 Calorimeter Trigger
2
FOX Demonstrator
3
4
Design document
5
6
Document Version:
Draft 0.4
7
Document Date:
2 June 2015
8
9
Prepared by:
Yuri Ermoline1, Murrough Landon2, Philippe Laurens1,
Reinhard Schwienhorst1,3
1
10
11
12
Michigan State University, East Lansing, MI, USA
Queen Mary, University of London, London, UK
3
LPSC Grenoble, FR
2
13
14
Document Change Record
Version
0
0
0
0
0
Issue
0
1
2
3
4
Date
20 March 2015
03 April 2015
10 April 2015
20 May 2015
2 June 2015
Comment
Initial document layout
Comments from PDR, Reinhard, Philippe
MTP connector fibre numbering, sliding rails
Chapters 2.1.2, 2.2.4, 2.3.2
Clarifications: 2.1.1, Fig.9, 2.3.2
FOX Demonstrator
page 1
ATLAS Level-1 Calorimeter Trigger
Design document
FOX Demonstrator
Version 0.4
15
TABLE OF CONTENTS
16
1.
17
1.1.
FOX DEMONSTRATOR OVERVIEW
3
18
1.2.
CONVENTIONS
4
19
1.3.
RELATED PROJECTS
4
20
1.4.
REFERENCE MATERIALS
4
21
2.
22
2.1.
INTRODUCTION
3
FOX DEMONSTRATOR INPUTS, OUTPUTS, MAPPING AND PARTITIONING
5
INPUTS TO THE DEMONSTRATOR
7
23
2.1.1.
From CMX module
7
24
2.1.2.
From FTM module
7
25
2.1.3.
From LAr (LDPS)
7
26
2.1.4.
From Tile (TREX)
7
27
OUTPUTS FROM THE DEMONSTRATOR
7
28
2.2.1.
To eFEX module
7
29
2.2.2.
To jFEX module
7
30
2.2.3.
To gFEX module
8
31
2.2.4.
Outputs to CMX and FTM
8
32
2.2.
2.3.
FIBER MAPPING AND DEMONSTRATOR PARTITIONING
8
33
2.3.1.
Fiber mapping
8
34
2.3.2.
Demonstrator partitioning
8
35
3.
36
3.1.
TRUNK CABLES TO/FROM FOX DEMONSTRATOR
10
37
3.2.
HOUSING MECHANICS
11
38
3.2.1.
1U Boxes
11
39
3.2.2.
Feed through
12
40
3.3.
FOX DEMONSTRATOR COMPONENTS
10
FIBERS MAPPING
12
41
3.3.1.
Fan-out split cable
12
42
3.3.2.
LC-LC couplers
12
43
3.3.3.
Mapping by fusion splicing
12
44
3.3.4.
Fiber passive splitting
12
45
4.
MEASUREMENT TOOLS AND TEST PROCEDURE
46
4.1.
MINIPOD LIGHT LEVEL MONITORING
13
47
4.2.
BIT ERROR RATIO TESTER (BERT)
13
48
4.3.
OPTICAL POWER METER
13
49
4.4.
OPTICAL OSCILLOSCOPE
13
50
APPENDIX A. INITIAL STUDIES
14
51
52
page 2
FOX Demonstrator
13
Design document
ATLAS Level-1 Calorimeter Trigger
Version 0.4
FOX Demonstrator
53
1. INTRODUCTION
54
1.1. FOX DEMONSTRATOR OVERVIEW
55
56
57
58
59
60
61
62
63
This document describes the Fiber Optics eXchange (FOX) demonstrator for integration tests in late
2015 and for future integration and studies. The demonstrator pursues two main goals:
 Provide the light path between the transmitter MiniPODs of the FEX Test Module (FTM), Liquid
Argon (Lar) and Tile Calorimeter (Tile) Front-Ends and the receiver MiniPODs of the Feature
Extractor (FEX) modules of l1calo.
 Provide mechanical building blocks necessary to construct an overall physical plant providing the
required management and mapping of all the fibres and its installation in USA15.
The initial proposal for the FOX demonstrator was presented in the FOX Project Specification [1] and
shown in Figure 1:
64
65
66
Figure 1: Draft diagram of the FOX Optical Demonstrator.
67
68
69
70
This drawing shows the light path between transmitting and receiving MiniPODs. The input side is
defined as a 48-fiber MTP connector(s) (LAr and Tile side) or a 24-fiber MTP connector(s) (FTM, not
shown on the initial proposal); the output side is defined as a 48-fiber (eFEX side) or 72-fiber MTP
connector(s) (jFEX and gFEX side).
FOX Demonstrator
page 3
ATLAS Level-1 Calorimeter Trigger
Design document
FOX Demonstrator
Version 0.4
71
72
The FOX demonstrator will map each of the input fibre to a specific FEX destination. It will also
provide passive duplication (optical splitting) of some of the fibres.
73
74
75
76
Reminder: the initial idea of the FOX partitioning (see Figure 2) is that it separated into five sets of
modules by mapping functionality. The two input module sets are the LArFox and the TileFox which
organize the fibres by destination. The three output module sets are eFox, jFox and gFox, which
provide the final fibre ribbon by fibre ribbon mapping and provide fibre duplication as required.
eFox
LAr supercells
Tile eFEX t owers
eFEX
jFox
LAr t rigger t owers
Tile jFEX t owers
jFEX
LAr supercells
LAr LAr t rigger t owers
DPS
LAr gTowers
LArFox
Tile eFEX t owers
JEP
Tile jFEX t owers
TileFox
Tile gTowers
LAr gTowers
gFox
Tile gTowers
gFEX
77
78
Figure 2: Overview of optical plant partitioning.
79
1.2. CONVENTIONS
80
81
82
83
84
85
86
The following conventions are used in this document:
 The term “FOX” is used to refer to the Phase-I L1Calo Optical Plant – Fex Optics eXchange or
Fiber Optics eXchange (FOX). Alternate names are “fiber plant” or “optical plant” or “FEX
optical plant”.
 eFEX – electron Feature EXtractor.
 jFEX – jet Feature EXtractor.
 gFEX – global Feature EXtractor.
87
88
89
1.3. RELATED PROJECTS
[1] FOX Project Specification, v0.14, 11 November 2014.
https://edms.cern.ch/file/1430017/1/FOX_PDR_v0.14_11.11.2014.pdf
90
91
[2] ATLAS Liquid Argon Phase 1 Technical Design Report, CERN-LHCC-2013-017,
92
93
[3] ATLAS Tile Calorimeter, http://atlas.web.cern.ch/Atlas/SUB_DETECTORS/TILE/
[4] ATLAS L1Calo Jet-PPM LCD Daughterboard (nLCD)
[5] Electromagnetic Feature Extractor (eFEX) Prototype, v0.2, 6 February 2014,
94
95
96
97
98
99
https://cds.cern.ch/record/1602230
https://twiki.cern.ch/twiki/pub/Atlas/LevelOneCaloUpgradeModules/eFEX_spec_v0.2.pdf
[6] Jet Feature Extractor (jFEX) Prototype, v0.2, 14 July 2014,
http://www.staff.uni-mainz.de/rave/jFEX_PDR/jFEX_spec_v0.2.pdf
[7] Global Feature Extractor (gFEX) Prototype, v0.3, 16 October 2014,
https://edms.cern.ch/file/1425502/1/gFEX.pdf
100
101
[8] High-Speed Demonstrator (HSD), v1.5, 18 July 2011,
102
103
[9] FEX Test Module (FTM), v0.0, 18 July 2014,
104
1.4. REFERENCE MATERIALS
105
The Fiber Optic Association Guide: http://www.thefoa.org/tech/ref/contents.html
https://twiki.cern.ch/twiki/bin/view/Atlas/LevelOneCaloUpgradeModules
http://epweb2.ph.bham.ac.uk/user/staley/ATLAS_Phase1/FTM_Spec.pdf
106
page 4
FOX Demonstrator
Design document
ATLAS Level-1 Calorimeter Trigger
Version 0.4
FOX Demonstrator
107
2. FOX DEMONSTRATOR INPUTS, OUTPUTS, MAPPING AND PARTITIONING
108
109
110
111
The inputs and outputs to/from the FOX demonstrator are optical signal carried by multi-fiber ribbon
cables with parallel Multi-fiber Termination Push-On (MTP) connectors – improved version of the
MPO connector (known as multi-fiber push-on and also as multi-path push-on) – a multi-fiber
connector defined according to IEC 61754-7 and TIA/EIA 604-5 that can accommodate12-72 fibers:
112
113
114
115
116
117
Figure 3: MPO cable female connector for accommodating 24 fibers.
The following convention will apply to the optical connectors:
 All optical connectors on the FOX demonstrator modules are MALE,
 All optical connectors on the cables, connected to FOX demonstrators, are FEMALE.
118
119
120
Figure 4: Convention for optical connectors.
The fibre numbering within a connector and fibre colours:
121
122
123
Figure 5: Fibre numbering in 12-fibers MTP.
FOX Demonstrator
page 5
ATLAS Level-1 Calorimeter Trigger
Design document
FOX Demonstrator
Version 0.4
124
125
Figure 6: Fibre numbering in 48-fibers MTP.
126
127
128
Figure 7: Fibre numbering in 72-fibers MTP.
129
page 6
FOX Demonstrator
Design document
ATLAS Level-1 Calorimeter Trigger
Version 0.4
FOX Demonstrator
130
2.1. INPUTS TO THE DEMONSTRATOR
131
132
The FOX demonstrator will provide a set of different input connectors for several possible data
sources – from L1Calo modules and from calorimeter electronics.
133
2.1.1. From CMX module
134
The contact persons from CMX to discuss issues: Wojtek Fedorko, wojtek.fedorko@gmail.com
135
136
137
The CMX module may be the only possible data source/destination at the beginning of the
demonstrator integration studies. It is based on the Virtex 6 FPGAs, has both – 12-fiber transmitters
and 12-fiber receivers – but can only test a transmission speed up of 6.4 Gbps.
138
The FOX demonstrator will provide one 12-fiber MTP male input connector from the CMX.
139
2.1.2. From FTM module
140
The contact persons from FTM to discuss issues: Richard Staley, r.j.staley@bham.ac.uk
141
142
The eFEX/jFEX Tester Module (FTM) must be capable of running high-speed links with bit rates up
to at least 9.6 Gbps with parts specified for operation up to 12.8 Gbps.
143
144
The FTM provides two connectors with 48 fibers each of transmitters, and one connector with 24
receivers.
145
The FOX demonstrator will provide one 48-fiber MTP male input connector from the FTM.
146
2.1.3. From LAr (LDPS)
147
The contact persons from Lar to discuss issues:
148
149
150
The trigger information from the entire LAr calorimeter to the three FEX systems will be sent by the
LAr Digital Processor System (LDPS). The LDPS is a set of about 30 ATCA modules called LAr
Digital Processor Blades (LDPBs) housed in three ATCA shelves (crates).
151
152
153
154
Each LDPB acts as a carrier board for four mezzanine cards (AMCs) each of which has a single FPGA
with 48 output optical fibres providing data to the FEXes over 48-fiber MTP connector. One of these
fibres will contain gTower information, 4 to 8 will contain trigger tower information, 24 to 32 fibres
will contain super cell information, and the rest are spares.
155
The FOX demonstrator will provide one 48-fiber MTP male input connector from the LDPS.
156
2.1.4. From Tile (TREX)
157
The contact persons from TREX to discuss issues: Victor Andrei, andrei@kip.uni-heidelberg.de
158
159
The Tile calorimeter data will be sent to the FOX demonstrator from the existing L1Calo PreProcessor modules (PPMs) via new rear transition cards - the TREX board.
160
The FOX demonstrator will provide one 48-fiber MTP male input connector from the TREX.
161
2.2. OUTPUTS FROM THE DEMONSTRATOR
162
2.2.1. To eFEX module
163
The contact persons from L1Calo to discuss issues: Weiming Qian, Weiming.Qian@cern.ch
164
165
Each eFEX module receives three cables of four ribbons with 12 fibres, i.e. the eFEX has three 48fiber MTP eFEX input connectors.
166
The FOX demonstrator will provide one 48-fiber MTP male output connector to the eFEX module.
167
2.2.2. To jFEX module
168
The contact persons from L1Calo to discuss issues: Ulrich Schäfer, uschaefe@uni-mainz.de
FOX Demonstrator
page 7
ATLAS Level-1 Calorimeter Trigger
Design document
FOX Demonstrator
Version 0.4
169
170
Each jFEX module receives four cables of six ribbons with 12 fibres, i.e. the jFEX has four 72-fiber
MTP input connectors.
171
The FOX demonstrator will provide one 72-fiber MTP male output connector to the jFEX module.
172
2.2.3. To gFEX module
173
The contact persons from L1Calo to discuss issues: Michael Begel, michael.begel@cern.ch
174
175
The gFEX module receives four cables of six ribbons with 12 fibres, i.e. the gFEX has four 72-fiber
MTP input connectors.
176
The FOX demonstrator will provide one 72-fiber MTP male output connector to the gFEX module.
177
2.2.4. Outputs to CMX and FTM
178
179
The FOX demonstrator will provide output connectors to the CMX and the FTM modules in order to
test the demonstrator with 6.4 Gbps and 9.6 Gbps before getting access to the other modules.
180
181
The FOX demonstrator will provide one 24-fiber MTP male output connector to the FTM module and
one 12-fiber MTP male output connector to the CMX module.
182
2.3. FIBER MAPPING AND DEMONSTRATOR PARTITIONING
183
2.3.1. Fiber mapping
184
185
186
For the integration tests of the FOX demonstrator, the Lar and TREX teams in agreement with the
eFEX, jFEX and gFEX teams shall provide for every fibre in their input connector a possible
destination in the output connectors:
187
Input Connector Fibre №
Destination(s) (eFEX, jFEX, gFEX, spare)
Output Connector Fibre №
188
189
Write down which fibres in each group of 12 are unused.
190
2.3.2. Demonstrator partitioning
191
192
193
194
195
196
Follow the initial proposal for the FOX demonstrator (Figure 1) and the initial idea of the FOX
partitioning (Figure 2), the demonstrator will be implemented in two logical/physical parts:
 First part will represent the LArFox and TileFox in Figure 2.
 Second part will represent eFox, jFox and gFox in Figure 2.
However, the FOX demonstrator, discussed in this document will have more inputs and outputs,
compare to the initial proposal.
197
First part - LArFox / TileFox demonstrator
198
199
200
For the final design this part may be implemented using a custom build commercial mapping module,
which redistributes the input signals to output connectors, as described in 3.2.4 of the FOX Project
Specification [1] or by connecting fibres by fusion splicing , as described in 3.2.3 of [1] .
201
202
203
For the FOX demonstrator this part will initially implement mapping by LC connectors, as described
in 3.2.2 of [1] and at the later stage also by fusion splicing upon availability of the fusion-splicing
machine. Input to output connection by male-male trunk cable is also possible.
204
205
206
The LArFox / TileFox demonstrator part will have feed-troughs for 4 input male connectors, as
described in 2.1 and one 48-fiber MTP output male connector, as shown on Figure 8. Mechanically
this part may be implemented in a separate box.
207
208
Internal mapping of the input-output fibres will be provided by the MTP fan-out split cables (or by
trunk cable without mapping) – male MTP to male LC connectors.
page 8
FOX Demonstrator
Design document
ATLAS Level-1 Calorimeter Trigger
Version 0.4
FOX Demonstrator
209
210
Figure 8: LArFox / TileFox part for the FOX demonstrator.
211
212
Second part - eFox/jFox/gFox demonstrator
213
214
215
216
217
218
219
220
221
222
223
224
225
For the FOX demonstrator as for the final design the eFox/jFox/gFox part will provide both - fibre
mapping and splitting:
 Internal mapping of the input-output fibres will be provided by the MTP fan-out split cables (or by
trunk cable without mapping) – male MTP to male LC connectors.
 For the data path, which require passive splitting, we will use the connectorized passive splitters
with the LC connectors on both end, as described in 3.3 of [1] . It will be inserted in-situ upon
need. We would also like to have 3 passive splitters, connected directly to the 48-fiber MTP input
male connector with LC connectors on output fibers. The input fan-out cable therefore will be
48MTP(M)-45LC(F)+ 3 splitters, connected directly to the MTP (on the other side of the splitters
– 2 LC(F).
On the input side, the eFox/jFox/gFox part will have feed-trough for one 48-fiber MTP input male
connector and on the output side - four feed-trough for MTP output male connectors - 12-fiber, 24fiber, 48-fiber and 72-fiber, as described in 2.2. (72-fiber MTP will serve both – jFEX and gFEX).
226
Mechanically this part may be implemented in a separate box or in the same as the first part.
227
228
Figure 9: eFox/jFox/gFox part for the FOX demonstrator.
229
FOX Demonstrator
page 9
ATLAS Level-1 Calorimeter Trigger
Design document
FOX Demonstrator
Version 0.4
230
3. FOX DEMONSTRATOR COMPONENTS
231
3.1. TRUNK CABLES TO/FROM FOX DEMONSTRATOR
232
Trunk cables with female MTP connectors to connect all systems:
233
N fibres
N cables
Length
System
Company
Availability
12
1 (2?)
?
CMX
Sylex
2 (2 m)
Fiberstore
2 (? m)
24
1 (2?)
?
FTM
48
3
?
LDPS, TREX, eFEX
Sylex
2 ordered (1 m)
72
2
?
jFEX, gFEX
Sylex
2 ordered (1 m)
234
235
Here may go a short technical description of different cables:
236
12-fibers / 24-fibers
237
1x12f MTP(F) to 1x12f MTP(F) 12-fiber OM3 trunk cable, Polarity A – 2 m long:
238
239
48-fibers / 72-fibers
page 10
FOX Demonstrator
Design document
ATLAS Level-1 Calorimeter Trigger
Version 0.4
FOX Demonstrator
240
241
Figure 10: 48-fibers / 72-fibers trunk cable.
242
3.2. HOUSING MECHANICS
243
The contact persons from L1Calo to discuss issues:
244
245
246
For the integration tests with other components of the L1Calo, the FOX demonstrator will be mounted
in existing 19-inch rack infrastructure in USA15 in a 1U rack mounted box(s) – possibly . The housing
will provide the MTP feed-through for the patch cables connections.
247
248
249
250
251
252
253
The current proposal is to split the FOX demonstrator into two separate parts and therefore have two
separate 1U boxes:
 First box: 1U box with feed-trough for 4 input connectors and one output connector on the front
panel, as described in 2.3.2 and shown on Figure 8.
 Second box: 1U box with feed-trough for 1 input connectors and 2 output connector on the front
panel, as described in 2.3.2.
Do we need access to internal connections for light measurements?
254
3.2.1. 1U Boxes
255
256
Figure 11: 1U 19” case.
FOX Demonstrator
page 11
ATLAS Level-1 Calorimeter Trigger
Design document
FOX Demonstrator
257
Version 0.4
3.2.2. Feed through
258
259
Figure 12: Shuttered Adapter compatible with all MTP connectors in fibre counts ranging from 4 to 72.
260
3.3. FIBERS MAPPING
261
262
263
264
265
266
267
268
269
The FOX demonstrator will map each of the fibre on an input MTP connector to a fibre on an input
MTP connector. In order to achieve this, the input and output parallel fibre ribbons break out in
individual fibres with MTP fan-out split cables. Connecting two segments of optical fibres may be
done in several ways:
 Through optical LC connectors on the end of the fibres and a connector coupler,
 By fibres fusion splicing.
For the fibres that go to two destinations and therefore require splitting, a passive optical splitter with
the even split ration (50/50) can be used. The splitter may be connected to the input/output fibres by
LC connectors or by fusion splicing.
270
3.3.1. Fan-out split cable
271
TBD…
272
3.3.2. LC-LC couplers
273
TBD…
274
3.3.3. Mapping by fusion splicing
275
TBD…
276
3.3.4. Fiber passive splitting
277
TBD…
278
page 12
FOX Demonstrator
Design document
ATLAS Level-1 Calorimeter Trigger
Version 0.4
FOX Demonstrator
279
4. MEASUREMENT TOOLS AND TEST PROCEDURE
280
4.1. MINIPOD LIGHT LEVEL MONITORING
281
4.2. BIT ERROR RATIO TESTER (BERT)
282
4.3. OPTICAL POWER METER
283
4.4. OPTICAL OSCILLOSCOPE
284
FOX Demonstrator
page 13
ATLAS Level-1 Calorimeter Trigger
Design document
FOX Demonstrator
Version 0.4
285
APPENDIX A. INITIAL STUDIES
286
APPENDIX A.A. STUDY WITH THE CMX MODULE
287
Initial study is done with the CMX based test setup in the L1Calo test rig.
288
289
290
The CMX SN01 (in slot 03) has 2 Transmitters (MiniPODs MP1, MP2), connected to the “BASE”
FPGA and 3 Receivers (MiniPODs MP3, MP4, MP5), connected to the “TOPO” FPGA. The CMX
module is based on the Virtex 6 FPGAs and can only test a transmission speed up of 6.4 Gbps.
291
292
293
The firmware in these 2 FPGAs consists of Integrated Bit Error Ratio Tester (IBERT) core design
from XILINX. Interaction with the IBERT is done by using ChipScope™ Pro Analyzer. Design was
provided by W. Fedorko.
294
295
A cmxDiagnostics program, written by A. Chegwidden, allows to read transmitters light output and
receivers light input in µW and dBm.
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
The 12-fifres trunk cable connects MiniPODs MP2 and MP3. First measurement is shown below:
***********************************************
MiniPod 2 Internal Monitors (CMX0)
Thu Jan 1 01:00:00 1970
***********************************************
Channel 0 TX Light Output [µW (dBm)]: 1030.8 (0.132)
Channel 1 TX Light Output [µW (dBm)]: 1044.9 (0.191)
Channel 2 TX Light Output [µW (dBm)]: 1005.9 (0.026)
Channel 3 TX Light Output [µW (dBm)]: 920.5 (-0.360)
Channel 4 TX Light Output [µW (dBm)]: 1067.5 (0.284)
Channel 5 TX Light Output [µW (dBm)]: 1008.6 (0.037)
Channel 6 TX Light Output [µW (dBm)]: 1068.2 (0.287)
Channel 7 TX Light Output [µW (dBm)]: 1047.9 (0.203)
Channel 8 TX Light Output [µW (dBm)]: 1140.9 (0.572)
Channel 9 TX Light Output [µW (dBm)]: 1096.9 (0.402)
Channel 10 TX Light Output [µW (dBm)]: 1073.8 (0.309)
Channel 11 TX Light Output [µW (dBm)]: 1048.7 (0.207)
***********************************************
MiniPod 3 Internal Monitors (CMX0)
Thu Jan 1 01:00:00 1970
***********************************************
Channel 0 RX Light Input [µW (dBm)]: 799.1 (-0.974)
Channel 1 RX Light Input [µW (dBm)]: 980.0 (-0.088)
Channel 2 RX Light Input [µW (dBm)]: 943.8 (-0.251)
Channel 3 RX Light Input [µW (dBm)]: 936.0 (-0.287)
Channel 4 RX Light Input [µW (dBm)]: 841.0 (-0.752)
Channel 5 RX Light Input [µW (dBm)]: 1000.0 (0.000)
Channel 6 RX Light Input [µW (dBm)]: 950.0 (-0.223)
Channel 7 RX Light Input [µW (dBm)]: 967.7 (-0.143)
Channel 8 RX Light Input [µW (dBm)]: 956.3 (-0.194)
Channel 9 RX Light Input [µW (dBm)]: 915.4 (-0.384)
Channel 10 RX Light Input [µW (dBm)]: 953.4 (-0.207)
Channel 11 RX Light Input [µW (dBm)]: 1017.3 (0.074)
These masurements can be translated to light loss, e.g., for the channel 0:
0.132 + (1-0.974) = 0.352 dBm –> 2 connectors, ~0.18 dBm/connector (12-fibre MPT)
page 14
FOX Demonstrator