Overview of
ATLAS
ATLAS Liquid Argon
Frontend Crate Electronics
John Parsons
Nevis Labs, Columbia University
LAr Electronics ASSO Review, June 17 2002
Overview of ATLAS Liquid Argon (LAr) Calorimeter Readout
J. Parsons, LAr Electronics ASSO, June 2002
Requirements of ATLAS LAr Frontend Crate Electronics





read out  170k channels of calorimeter
dynamic range  16 bits
measure signals at bunch crossing frequency of 40 MHz (ie. every 25 ns)
store signals during L1 trigger latency of up to 2.5 s (100 bunch crossings)
digitize and read out 5 samples/channel at a max. L1 rate of 100 kHz
 measure deposited energies with resolution < 0.25%
 measure times of energy depositions with resolution << 25 ns




high density (128 channels per board)
low power ( 0.8 W/channel)
high reliability over expected lifetime of > 10 years
must tolerate expected radiation levels (10 yrs LHC, no safety factors) of:
 TID 5 kRad
 NIEL 1.6E12 n/cm2 (1 MeV eq.)
 SEU 7.7E11 h/cm2 (> 20 MeV)
J. Parsons, LAr Electronics ASSO, June 2002
ATLAS LAr Frontend Crate Electronics Overview
 On-detector electronics

Boards tested functionally on Mod 0
 ATLAS rad-tol boards being finalized
Calibration :
116 boards @ 128 ch
Front End Board (FEB) :
1524 boards @ 128 ch
Controller :
116 boards
Tower builder (TBB) :
120 boards @ 32 ch
J. Parsons, LAr Electronics ASSO, June 2002
Module 0 Electronics Experience
 Full functionality “Module 0” CALIB and FEB boards were developed

Provided verification of the electronics design concepts
 Used in testbeam runs with Module 0 and production calorimeter modules
 Several CALIB boards and  6000 channels of FEBs were produced

Have been operating reliably in testbeam for past several years
 Performance meets or exceed ATLAS specifications
 Prototype Controller board was evaluated in dedicated test
 Due to schedule, Mod0 boards developed with some “short cuts”:

Did not use final control signal (TTC and SPAC) distribution
 FEB used Cu output cables instead of optical links
 Did not pay strict attention to radiation tolerance requirements
 the main task remaining in the development of the final ATLAS
boards was to radiation harden the designs, and in particular to
replace several FPGAs and other COTs with custom rad-tol ICs
J. Parsons, LAr Electronics ASSO, June 2002
On-detector electronics control interfaces
 LAr on-detector electronics uses common control signal distribution:
TTC – for 40 MHz CLK, L1Accept and other “fast” signals
 SPAC – serial protocol for downloading/reading back configuration parameters

 Controller Board (one per 1/2 crate) is connected optically to external TTC
system and SPAC Master, and fans signals out electrically to each of the
boards within its 1/2 crate
 Redundancy has been implemented to increase reliability, with no logic
switching components required on-detector
TTC and SPAC Master  Slave (MS) signals sent on two separate fibers each,
with off-detector selection of which will be active
 On-detector electronics configured to drive both of the SPAC Slave  Master
(SM) links
 SPAC MS and SM redundancy preserved on SPAC bus (though Controller is
common and must therefore be VERY reliable)

J. Parsons, LAr Electronics ASSO, June 2002
On-detector electronics control interfaces (cont’d)
 TTC


TTC fanount within crate is point-to-point over halogen-free USB2.0 mini-B cables
connected through the boards’ front panels
Each board has TTCRx chip to receive 80 MHz encoded TTC signals
 SPAC





Manchester-encoded serial protocol combining CLK and data with 20 MHz edges
Implementation uses separate (ie. uni-directional) MS and SM lines
SPAC signals are fanned out via SPAC bus, implemented as a PCB along the side of the
crate with through comb connectors to boards
Each board has SPAC Slave to receive/transmit SPAC data using either 8-bit parallel
interface or one of two I2C interfaces provided by SPAC Slave
Final SPAC Slave version includes separate address and data partial checksums for better
fault and SEU tolerance
 Some concerns:



Cabling of TTC cables on crate, while still allowing extraction of single boards
Robustness/fault tolerance of SPAC bus (will be tested in system crate test)
Robustness/fault tolerance of Controller
J. Parsons, LAr Electronics ASSO, June 2002
Controller Board Overview
(LPNHE-Paris)
J. Parsons, LAr Electronics ASSO, June 2002
Connectors on Controller
Optical Part for TTC and SPAC
Up to 22 TTC connectors
(by pairs on the 2 faces)
TTC (USB mini-B) connector pinout
1
2
3
4
5
=
=
=
=
=
GND
(D-)TTC(D+)TTC+
not connected
GND
+ shield connected to GND
J. Parsons, LAr Electronics ASSO, June 2002
Controller Board Status
 Test board produced to evaluate fanout issues
 Design of full prototype board underway; delivery to BNL by September
 Radiation tests of SPAC Slave V3 and optical receiver by end June
J. Parsons, LAr Electronics ASSO, June 2002
Controller Board Responsibility Chart
J. Parsons, LAr Electronics ASSO, June 2002
Calibration Board Overview
(Annecy, Mainz, Orsay)
 Generate 0.1% precision calibration pulses
 Rise time < 1 ns
 Current pulse amplitude from 200 nA up to 10 mA
 Delay programmable from 0 to 24 ns in 1 ns steps
 Number of current pulsers per CALIB board is 128
J. Parsons, LAr Electronics ASSO, June 2002
Overview of Main CALIB Components
DMILL
AMS
COTS
5Ώ
0.1%
Enable
Spac
VDAC
128
Output
signals
128 opamp
10 μV offset
1 DAC
16 bits
IDAC
6 CALogic
1 SPAC
TTC
CMD
50 Ώ
0.1%
10 uH128
HF switch
16 driver
2 delay
1 TTCRx
4 pos. Vreg and 1 (non-essential?) neg. Vreg
J. Parsons, LAr Electronics ASSO, June 2002
CALIB Control Interfaces
 SPAC slave used only as I2C master, with one I2C link used for TTCRx and
Delay chips, and the other I2C link used for all the other chips
J. Parsons, LAr Electronics ASSO, June 2002
Digital CALIB Testboard
 Developed to test integration of digital ASICs on CALIB board

Tests with TTCRx, SPAC Slave, 2 Delay chips, 6 CALogic all OK
J. Parsons, LAr Electronics ASSO, June 2002
8 Channel CALIB Prototype
 Include digital control plus analog chain for 8 channels
 3 boards received in April 02
SPAC2
CALlogic
TTCRx
Delay
8 outputs
Opamps & switch
DAC
 Design of full-sized 128 channel board is underway; delivery to BNL by Oct.
J. Parsons, LAr Electronics ASSO, June 2002
CALIB Responsibility Chart
J. Parsons, LAr Electronics ASSO, June 2002
Frontend Board Overview
(Nevis)
 functionality includes:
 receive input signals from calorimeter
 amplify and shape them
 store signals in analog form while awaiting L1 trigger
 digitize signals for triggered events
 transmit output data bit-serially over optical link off detector
 provide analog sums to L1 trigger sum tree
J. Parsons, LAr Electronics ASSO, June 2002
Overview of main FEB components
128
input
signals
Analog
sums
to TBB
32 0T
32 Shaper
2 LSB
14 pos. Vregs
+6 neg. Vregs
32 SCA
2 SCAC
2 DCU
16 ADC
DMILL
AMS
DSM
COTS
8 GainSel
1 Config.
1 SPAC
1 MUX
1 GLink
7 CLKFO
1 TTCRx
1 fiber
to
ROD
TTC,
SPAC
signals
 10 different custom rad-tol ASICs, relatively few COTs
J. Parsons, LAr Electronics ASSO, June 2002
FEB Analog Interfaces
 Input signals

128 signals per FEB, using the two outer rows of the two 96-pin VME-style
DIN connectors
 Middle row of 32 pins of each signal connector are shared Ground pins (ie.
one GND pin per 2 signal pins)
 Signal connectors equipped with custom shields and additional grounding
contacts, with mating springs mounted on crate baseplane
• Mod 0 demonstrated this is necessary to achieve coherent noise specification
 Output analog trigger sums

First level of summing provided by shaper chips, configured via switches to
enable/disable individual channels
 Next level of summing provided by plug-in Layer Sum Boards (LSB), the
output stages of which drive the sums through the middle FEB connector to
the baseplane, where they are routed to the TBB (or TDB for HEC)
 LSB sums depend on calorimeter region, with up to 32 separate O/P sums
 Remaining 64 pins are GND pins, with GNDs separating adjacent signals
J. Parsons, LAr Electronics ASSO, June 2002
FEB Control Interfaces
Shaper 0
…..
Shaper 1
Left Gain selector 0
Shaper 31
Left Gain selector 1
TTCRx
23
Left Gain selector 2
5 MHz
I2C_0
Left Gain selector 3
4
data[7:0]
SPAC slave
Right Gain selector 0
subadd[6:0]
n_read
n_write
4
Right Gain selector 1
I2C_1
Right Gain selector 2
8 8
4
Right Gain selector 3
4
DCU 0
DCU 1
Regulator 0
…
Regulator 19
Left SCA controller
Right SCA controller
J. Parsons, LAr Electronics ASSO, June 2002
FEB Optical Links
 one GLink output link per FEB, with rate of 1.6 Gbps
 link collaboration includes SMU, ISN,
Stockholm, and Taiwan
 FEB integration complete (ROD integ. underway)
1.6 Gb/s
J. Parsons, LAr Electronics ASSO, June 2002
1/4 Digital FEB
 Test setup for FEB digital ASICs

Complete operation of SCAC, GainSel, Config, SPAC, TTCRx, MUX, GLink
Optical
Tx
Agilent
GLink
DMILL
MUX
DSM/DMILL
Gain
selector
DMILL
TTCRx
DMILL
SPAC
DMILL
Config
controller
DSM/DMILL
SCA Controller
 Allowed quick testing/integration of new ASICs as they became available
• Integration was carried over to FEB design, which worked properly first time
J. Parsons, LAr Electronics ASSO, June 2002
FEB Prototype
 128 channels/FEB
 large 10 layer PCB
 components on both
sides to achieve density
 layout, with STm
Vregs, done Oct. 01
 first FEB assembled
and successfully
tested Nov. 01
 pos. STm Vregs
added Dec. 01
 2nd FEB assembled/tested and shipped to Orsay, Mar. 02
 Need neg. Vregs before launching 20 FEB preproduction
J. Parsons, LAr Electronics ASSO, June 2002
FEB Responsibility Chart
J. Parsons, LAr Electronics ASSO, June 2002
Organizational Issues
 Meeting the specifications, particularly radiation tolerance, has required
development of a LARGE number of custom ASICs and other devices
 Final PCB designs, while the responsibility of one (sometimes 2) labs, must
integrate custom devices developed by a number of different institutions
 Due to radiation concerns with COTs, custom devices need to work together
without resorting to “glue logic”
 For this integration to work as well as it has required careful attention to
interfaces during ASIC design and development:





Occasional workshops, including engineers, to develop and agree on architecture
Many meetings and discussions
Design reviews for each individual device
Documentation and agreement on interfaces
In limited (critical) cases, simulation of ASICs together (eg. SPAC and Config)
 As we move toward production, the complexity of the distribution of
responsibilities will present us with a new set of challenges
J. Parsons, LAr Electronics ASSO, June 2002
Moving Toward Production
 Final prototypes of each of the various boards in the frontend crate are
currently being developed and/or tested individually
 However, a “system crate test” using all the various boards together is
necessary to validate the designs before launching production:

Are boards functionally compatible with each other? (eg. TTC and SPAC)
 Does crate performance meet specifications? (eg. coherent noise)
 Are “environmental” issues under control? (eg. power, cooling)
 Test will be performed at BNL, with first boards delivered for Sept. 02
 Some concerns:

Coupling of schedules increases schedule risk
 Continued problems with radtol negative Vreg are causing a significant delay
 Since we cannot wait too long to produce the radtol ASICs, most of the
components will be submitted for production before completion of the system
crate test
J. Parsons, LAr Electronics ASSO, June 2002
Component Production
 Lab which developed a given component (eg. ASIC or plug-in) is responsible
for the production and delivery of that component
 Separate PRRs to be held for all major custom components

So far, have been held for preamps, HEC preshapers, shapers, LSBs, SCAs
 To achieve cost savings, many radtol ASICs will be produced together on
shared wafers:
DMILL Analog Reticle – opamp, DAC, BiMUX
 DMILL Digital Reticle – SPAC, CALogic, Config, MUX
 DSM Digital Reticle – SCA Controller, GainSelector, CLKFO


PRRs for the ASICs on these shared wafers are expected by end 2002
 Some concerns:

Sharing of ASIC wafers for production increases schedule risk
 Careful tracking required during ASIC packaging and delivery
J. Parsons, LAr Electronics ASSO, June 2002
Component Quality Assurance and Tracking
 QA plan developed and presented as part of PRR; to be archived in EDMS

For LSB example, see B. Cleland’s “Trigger Sums” presentation

For plug-in assemblies (preamps, preshapers, LSBs) this includes:
•
•
•
•

Initial inspection
Pre-test
Burn-in at elevated temperature
Re-test
Custom ASICs will be individually tested, but not burned in
 Serial numbers are to be printed on the components
 Results of component testing and QA procedures are to be entered in
database and tracked via serial number
J. Parsons, LAr Electronics ASSO, June 2002
Example of Component Tracking
 BNL document for following preamp acceptance procedures
IO-823 PREAMPLIFIERS
MANUFACTURING & QUALITY ASSURANCE TRAVELER
Drawing No.
IO-823-1
Rev
E
Serial Numbers
OP.
OPERATION
NO.
05
15
20
40
42
45
50
60
65
REVIEW VENDOR DATA
FILL OUT TRAVELER
INCOMING INSPECTION
BURN-IN START TIME
BURN-IN FINISH TIME
TEST EQUIPMENT CHECK
TEST
FINAL INSPECTION
FINISH STOCK ROOM
APPLICABLE
DOCUMENTS
BNL PO 30705
TBD
IO-823 Comp. Outline
TBD
TBD
TBD
TBD
TBD
DOC.
REV.
IN
QTY
REJ
OUT
OPERATOR
DATE
N/R
Serial Numbers Rejected
J. Parsons, LAr Electronics ASSO, June 2002
Example of Component Quality Results
(Final sample of 50000 channels 50W/1 mA preamps
from INFN-Milano)
Gain
Peaking time (5%-100%)
Equivalent Noise Current
Acceptance limits
J. Parsons, LAr Electronics ASSO, June 2002
Component Qualification and Testing
 In most cases, lab which developed a given component (eg. ASIC or plug-in)
is responsible for radiation qualification and
for functional testing of production parts
prior to delivery for assembly onto PCBs
 For shapers and SCAs, where > 50000 chips
needed, ISN Grenoble developed robotic tester

Shaper testing completed (last week)
 SCA testing to start soon
 Remaining ASICS must be tested manually
 Some concerns:

PCB complexity makes re-work difficult, requiring excellent pre-testing and
rejection of faulty parts
 Achieving the necessary very high quality of automated PCB assembly process
requires no damage to pins
 Most of the critical ASIC testing is being done long before the start of PCB
production, meaning there is no chance for “feedback” to the testing procedures
J. Parsons, LAr Electronics ASSO, June 2002
PCB Design/Testing Issues
 Each PCB includes many custom components developed by other labs
 Responsibility charts have been created for all PCBs, identifying individuals
responsible for delivery of each of the major components (see earlier slides)
 Steps to ensure successful design and integration





Usually, dedicated jigs developed first to test integration of custom ASICs
All final schematics made available via WWW
Relevant contact person asked to verify schematic related to their ASIC
Board design documentation and reviews (eg. later this week)
System crate test before launching board production
 All final PCBs will be fully tested before delivery to CERN
 Test procedures still being developed

Take FEB as example, since it is by far the PCB of largest quantity
 Nevis and LAL are working on developing and defining the final FEB test
procedure (see next slide)
J. Parsons, LAr Electronics ASSO, June 2002
FEB Testing Procedures
 FEB test steps as currently envisioned include:

Receipt and visual inspection of assembled PCBs at Nevis
• Boards will be fully assembled in industry, but NOT including preamp/LSB plug-ins

Labelling of FEBs with serial numbers
 Initial test without preamps/LSB
• Full testing/debugging of digital part
• Partial testing/debugging of analog part (from shaper outputs)

Limited burn-in (eg. overnight? room temperature?)
• Does not appear practical to perform a long burn-in on the full set of FEBs









Re-test
In parallel, full testing (ie. with preamps/LSBs) at Nevis of sample (5-10%) of
FEBs to allow quick feedback to Assembler
Shipment to LAL of FEBs which passed Nevis tests
Installation of preamps and LSBs of appropriate combinations
Full testing/debugging/calibration of analog part
Shipment to CERN of FEBs which passed LAL tests
Installation and testing of cooling plates
Functional re-test
Ready for installation/commissioning in the pit
J. Parsons, LAr Electronics ASSO, June 2002
Radiation Tolerant Voltage Regulators
 CERN-driven development of rad-tol Vregs by STm has long been identified
as a critical item, particularly for the FEB development

Each FEB requires 14 positive Vregs and 6 negative Vregs
 As one backup possibility, we asked MDI in early 2001 to consider an SBIR to
develop a hybrid radtol Vreg; they did so, but it was NOT approved by DoE
 We irradiated with protons samples of 9 different commercial Vregs, including
several advertised as radtol (but only wrt ionizing radiation)
• Only the Intersil HS9S-117RH positive Vreg survived as a backup ( $75 unit cost)

We worked on designing our own radtol Vreg with COTs, but with little in the
way of encouraging results
 STm delivered first positive Vreg prototypes in Dec. 01, which we have since then
been using successfully on the FEB  +ve Vreg issue would appear to be resolved
 In May 02, STm announced they were still having stability problems with
the neg. Vreg design, and an additional design iteration was required

STm has not yet announced a new schedule for the negative radtol Vreg
 Past experience would imply a delay of order 6 months (assuming no further
problems are encountered!)
J. Parsons, LAr Electronics ASSO, June 2002
Radiation Tolerant Voltage Regulators (cont’d)
 Intersil has recently announced a new neg. Vregulator, radtol to 300 kRad

We acquired a few samples ($85 each)
 On May 24, we irradiated 2 samples at HCL  results look encouraging
 STm remains the most attractive solution (assuming availability)
Cheaper ($10 unit cost versus  $50 - 70??, about 10k devices needed for FEBs)
 Higher current (3 Amp versus 1 Amp)
 Better voltage range (we need –1.7 V, while Intersil rated for < –2.5 V)
 Same control logic as STm positive Vreg

 Next steps

Complete evaluation of Intersil devices
• Further statistics needed for irradiation qualification (& HCL no longer available!)
• Produce jig to interface to allow FEB electrical measurements
• Discuss procurement/financial aspects with Intersil

Keep pestering STm for updated delivery schedule for their development
 In September, re-visit situation and decide how to proceed
 Dates of PRRs will be fixed in September once this issue is resolved
(given the Vreg delay, FEB PRR will not be before Summer 2003)
J. Parsons, LAr Electronics ASSO, June 2002
Summary
 Building on the success of the Module 0 electronics development, we are
finalizing the designs of the final ATLAS LAr readout boards

The system crate test is the remaining major milestone before moving to
production
 Prototypes of the large number of required custom ASICs have been
successfully developed and integrated

Most devices will move into production before the end of 2002
 Continued problems with STm negative radtol Vreg are now dominating
schedule

We are aggressively pursuing a backup, and working to minimize delay
 As we move toward production, we are working to formalize the
documentation, QA and tracking procedures that will be required to
sucessfully manage the complex production process
J. Parsons, LAr Electronics ASSO, June 2002