System Architecture of Detector Control and Safety for
the ATLAS Inner Detector Upgrade
Didier Ferrère, DPNC Université de Genève
Susanne Kersten, Wuppertal University
New Inner Detector for ATLAS Upgrade
Under consideration:
• Current silicon tracker is expected to smoothly die with an integrated luminosity (< 600
fb-1) : Radiation damages and Inefficiencies at high rate.
• A new ID is foreseen at phase 2 Upgrade of the machine and for data taking in ~2020
• The electrical services between the counting rooms and the cavern remain a constraint
for the new detector
• All the hardware for the FE electronics, the DAQ, the power supply and the DCS will
have to be renewed
• The cooling is one of the critical challenges for the future silicon tracker with requested
operational temperature down to -40°C
• All the development has to fit in the framework of the existing ATLAS DAQ, DCS,
Trigger with some restrictions under study
One ID layout under consideration
4 pixel layers
3 Short-strip layers
2 Long-strip layers
D. Ferrère
HSTD7, Aug 29th Sept 1st, 2009 in Hiroshima
2
ATLAS Detector Control System to be Updated
Pixel & Strip to
be renewed
Cooling and ID
environment to
be renewed
The new DCS structure has to fit in the ATLAS GCS
D. Ferrère
HSTD7, Aug 29th Sept 1st, 2009 in Hiroshima
3
DCS Motivations
Up to 180 kW to control and monitor during the operation
in term of power, cooling, and environment across the all ID.
The DCS has to be defined as early as possible such it is integrated into
the readout architecture together with the powering and the services.
Towards the specifications:
• Define the needs  Use cases (close to current tracker)
• The detector safety and the interlock to be considered as the 1st requirement
• Monitoring sources to be well evaluated
• Minimize as much as possible the material and the services
• Optimize the development effort
• Limit access installation in the cavern  DCS hardware in the counting room
• Try to define a common Pixel and Strip DCS architecture if possible
• Search for adequate sensors: Humidity, others?
• Identify all the topics where early resources may be necessary
• Define the prototypes for DCS investigations like power monitoring and control
D. Ferrère
HSTD7, Aug 29th Sept 1st, 2009 in Hiroshima
4
Short Strip Barrel Stave Layout
Cooling In
TTC, Data &
DCS fibers
Module #1
Module #2
Module #12
Opto
GBT
DCS env. IN
DCS
interlock
Cooling Out
PS cable
SMC
Hybrid
MCC
MCC
* the GBT (Giga Bit Transmission) is a chip for
data transmission developed by the CERN PH-ESE group
MCC
MCC
Service bus
Beam Axis
MCC
MCC
IP
Comments:
• SMC is located at the stave end and steer 24 hybrids. In total and per stave there are 2 SMCs:
one per side.
• SMC (Super Module Controller) is a hybrid board which integrates the opto, the GBT, the DCS
and some power regulations
• 1 or 2 MCC (Module Controller Chip) steer the data of the 20 chips on a hybrid
• The service bus is running below the Si-wafers and the front and back detector bias are
separated.
•There are 2 SMCs par Stave/SM electrically separated from the top and bottom side
D. Ferrère
HSTD7, Aug 29th Sept 1st, 2009 in Hiroshima
5
Pixel Barrel Stave Layout
2 single sided staves sandwiched together back to back
Module 7
Module 0
FE FE
FE FE
FE FE
FE FE
DCS
FE FE
FE FE
FE FE
FE FE
GBT*
End of Stave Card
GBT* DCS
FE FE
FE FE
FE FE
FE FE
FE FE
FE FE
FE FE
FE FE
half stave
IP
Beam Axis
* the GBT (Giga Bit Transmission) is a chip for
data transmission developed by the CERN PH-ESE group
Comments:
• The construction of the staves varies between the different layers 24 - 32 modules/stave
There are 4 Front End chips/detector module
• The DCS relevant building blocks are half stave/disk sector: up to 16 modules + 1 EoS card
• For both Pixel and strip two options are considered for the powering:
- Serial powering
- Parallel powering with 2 DC-DC stages
D. Ferrère
HSTD7, Aug 29th Sept 1st, 2009 in Hiroshima
6
ID Upgrade DCS Architecture
ID DCS
Strip DCS
Not defined yet!
Env DCS
Strip Stave
Power Supply
Elect.
Opto
at PS
Lines from
staves to
counting room
FE DCS
@ « BBIM »
@ PS card
or
@ « BBIM »
‘Cooling Interlock’
DCS Chip
@ « SMC »
Opto
Env DCS
Elect.
Elect.
SPI or I2C
DCS Data with
RO Data
FE DCS
‘SMC Interlock’
‘Strip Module Interlock’
D. Ferrère
Pixel Stave
DCS at PS
DCS split at
BOC/ROD
DCS Data with
RO Data
Cool DCS
ID DCS Gen?
Pixel DCS
DCS Chip
Lines from
staves to
counting room
@ « BBIM »
‘Pixel Module Interlock’
HSTD7, Aug 29th Sept 1st, 2009 in Hiroshima
7
Strip Architecture Overview
Option 2
Option 1/3
LAN
SCT
DCS
CAN Bus
Matrix
SCT
DAQ
CAN Bus CAN Bus
Global Interlock
Elmb
Ibox
ROD
SPI bus or
LVDS lines
daisy chain
Elmb
BOC
5 GB/s optical link
TTC - DCS
BBIM
PS Crate
Type 2 
Type4 cables
Stave
EoS Card
Cooling
Temp
Env.
Structure
FE DCS
NTC
Hybrid Temp
RH
Hybrid Power
Channel Interlock
RH
PS Type 2  Type 4 cables
D. Ferrère
Detector
HSTD7, Aug 29th Sept 1st, 2009 in Hiroshima
8
Pixel Architecture Overview
~100
Control
room
Readout
Crate
Pixel DAQ
Control –
feedback
diagnostics
safety
Interlock
Circuit
DCS Master
~50
Opto Board DCS
DCS
DCS
Distance from
interaction
point [m]
Power
Supplies
Pixel DCS
EoS
Controller
End of Stave
Card
D. Ferrère
Environment
cable bundle
from half stave
half staves
half staves
symmetric to both sides
Detector volume
HSTD7, Aug 29th Sept 1st, 2009 in Hiroshima
9
Safety Interlock and Monitoring Sources
NTCs / Stave
Cooling Interlock
Justification
Action
NTCs / Stave
Justification
Module Interlock
Stave Card Interlock
Strip
Pixel
2
-
Protection against cooling
failure
Power cut of the
corresponding LV & HV stave
-
48
16  4 lines
Protection against any heat
excess. Reasons: Runaway,
Cooling failure, contact
problem
Protection against any heat
excess. Reasons: Cooling
failure, runaway, contact
problem
Power cut of the
corresponding LV & HV
stave
Action
Either Disable LV locally or at
power supply
NTCs / Stave
2
1
Protection against excess
heating of EoS Card
Disable the LV power on all
the stave
Protection against excess
heating of EoS Card
Disable the LV power on all
the stave
Justification
Action
Monitoring sources: NTCs, RH sensors, power supplies, FE temp & power.
D. Ferrère
HSTD7, Aug 29th Sept 1st, 2009 in Hiroshima
10
Interlock and Monitoring of Module NTCs for Strip
Interlock steered on the stave card
IC on SMC HYBRID
1 interlock signal/ half stave
OR
Stave side – 24 hybrids
REF
OVER HEAT
REGISTER
TRIPLE VOTE LOGIC
INTERLOCK
+V
D F/F
-
Enable DAC with
Triple vote logic
THRESHOLD
from DAC with
Triple vote logic
NTC
NTC
NTC
NTC
GND
1
2
3
23 or 24
Option 3
+V
CLEAR
GBT
E-port?
+
ADC
1 ADC 12bits
multiplexed
GND
RESISTORS
Asic either with GBT SCA or separated
D. Ferrère
REF
GND GND
GND
HSTD7, Aug 29th Sept 1st, 2009 in Hiroshima
11
Combining Interlocks at PS crate – Strip Serial Powering
Top-side Stave
Power-DCS cable
PS Crate
NTC-Mod (option3)
NTC-Cooling
NTCSMC
I-box
DCS
I-Cooling/Mod
I-SMC-Bot
I-SMC-Top
LV Mod PS Top
LV Mod PS Bot
I-Modue Top
LV SMC PS Top
LV SMC PS Bot
I-Modue Bot
HV PS Top X 9-12
HV PS Bot X 9-12
Bottom-side Stave
Power-DCS cable
D. Ferrère
HSTD7, Aug 29th Sept 1st, 2009 in Hiroshima
12
Interlock and Monitoring of Module NTCs for Pixel
Interlock steered in the counting room
A
DCS
chip
Interlock
NTC
Off detector
On detector
• Temp. measurement of detector modules: NTCs are supplied from outside
• In case of environmental temp. measurement, the NTCs will be supplied
by the DCS chip
D. Ferrère
HSTD7, Aug 29th Sept 1st, 2009 in Hiroshima
13
Towards a Common Radiation Hard DCS Chip
Motivations:
- Unification of some DCS hardware across the ATLAS ID
- Optimize development effort and cost
- Chip should be as flexible as possible in term of use in the all ID volume
VSupply VRef
Main features:
a1
• Radiation hardness up to
1 MeV
2
neq/cm
• Have to work with two protocols: I2C/SPI
(low frequency), and GBT e-port (40 MHz)
• Low power when running at low frequency
with I2C or SPI: < 0.1W
• About 32 analog input needed
• Some DAC for power control and up 17
dig. Out (Bypass for pixel SP + reset)
• Interlock function based on FSM with
programmable temp limits (Strip)
• SEU protection for all the relevant parts
• Interlock decision need to be sure
• Power supply reference for NTCs
Vref
1.3x1016
a32
Analog
Mux
32
ADC
10bit
FSM
Alarm flag
DAC
1 bit
dout
I2C
SPI
SCK
CE
SDA SCL
Din Dout
GBT
E-link for
Slow Control
NB: A DCS IC submitted by Wuppertal to study I2C and SPI
D. Ferrère
HSTD7, Aug 29th Sept 1st, 2009 in Hiroshima
14
GBT-SCA – Slow Control in a Radiation Hard DCS IC
From A. Marchioro
On-Detector
Custom Electronics & Packaging
Radiation Hard
Monitoring
ADC
Ch1
Ch2
…
Ch16
16 x I2C Buses
4 x PP[0:7]
SCL
SDATA
D[0:7] A[0:15]
Off-Detector
Commercial Off-The-Shelf (COTS)
Custom Protocol
JTAG
Master
Monitoring
ADC
The “GBT Project” is part of the
“Radiation Hard Optical Link Project”
which aims at developing a radiation
hard bi-directional optical link for use
in the LHC upgrade programs
Clock
Generation
PIA
I2C Master
SCA
Controller
e-Link
Controller
E-Link
I2C Master
Ext Reset*
Memory
Interface
D. Ferrère
HSTD7, Aug 29th Sept 1st, 2009 in Hiroshima
15
Conclusions
• The Inner Tracker Upgrade has to be renewed and is under development &
prototyping construction based on silicon pixel and strip
• The DCS has to be designed as a new system to fit with the detector
requirements in term of safety and operation as well as with the ATLAS Global
Control System
• The key parameters of such a system are the control of the power and of the
temperature of the complete tracker
• Controllable power is needed at the detector parts using either serial powering
or DC-DC conversion
• New features are proposed for detector diagnostics like including DCS into FE
chips
• Strip and Pixel detectors have different requirements but need to unify system,
effort and some hardware parts
• A single radiation hard DCS IC is one of the illustration of it with some common
specifications that are under investigation. GBT-SCA is a possible option now
considered
• Still a lot to investigate in term of steering DCS interlock and information at the
counting room and linked to the Power Supply units
D. Ferrère
HSTD7, Aug 29th Sept 1st, 2009 in Hiroshima
16
Back-up slides
D. Ferrère
HSTD7, Aug 29th Sept 1st, 2009 in Hiroshima
17
Back-up slides
Pixel - Disabling of serial powered modules
charge pump:
DCS chip produces pulses (between 0 and 3.3 V)
C1 is charged to U_GS (2.5 V due to inefficiency)
Bonn-Wuppertal Stave emulator system
DCS chip is simulated by COBOLT (DCS board with microprocessor)
D. Ferrère
HSTD7, Aug 29th Sept 1st, 2009 in Hiroshima
18
Back-up slides
Pixel - Power Up Sequence
DCS chips are running permanently
check that there is no temp. interlock
check temp of opto board
turn on cooling of opto boards
turn on power of opto board + monitor its power
check temp of modules and EOS
turn on cooling of modules
turn on power of EOS controller and modules +
monitor their power consumption
initialize opto board
configure modules
power consumption → successful configuration
start tuning, calibration, etc … of data taking chain
D. Ferrère
HSTD7, Aug 29th Sept 1st, 2009 in Hiroshima
19
Back-up slides
Strip – DCS Operational Sequences
DCS Cooling Survey
A)
Pre-operation
B)
C)
SMC ramp & Opto com
D)
Module ramp
Operation
Detector Cold
Cooling Interlock Active, Env. data accessible, PS
Low data volume
> 5x104 data/day/stave
@ 0.1-1Hz
Module Interlock Active, Module temp, local PS accessible
Medium data volume
> 4.3x105 data/day/stave
@ 0.1-1Hz
FE DCS, Hybrid power accessible
On
High data volume
Possible but not desired request
D. Ferrère
HSTD7, Aug 29th Sept 1st, 2009 in Hiroshima
20
Back-up slides
Strip – DCS Operational Sequences
Operational phase
Option 1 & 3
Option 2
Access to Env DCS and monitor
cooling Interlock NTCs (+ 1
Module side for option 3)
Access to Env DCS and monitor
cooling interlock NTCs &
module temps  Module
Interlock active
Power ramp-up on SMC for
Opto settings and com. DCS ON
– Monitor module temp + Env
 Module Interlock active
Power ramp-up on SMC for Opto
settings and com. DCS ON –
Monitor module temp + Env 
Module interlock still active
Power ramp-up on FE hybrids.
Active protection + Bypass (SP)
or Power En/Dis (DC-DC)
DCS diagnostics accessible
Power ramp-up on FE hybrids.
Active protection + Bypass (SP)
or Power En/Dis (DC-DC)
DCS diagnostics accessible
Active protection + Bypass (SP)
or Power En/Dis (DC-DC)
DCS diagnostics accessible
Active protection + Bypass (SP)
or Power En/Dis (DC-DC)
DCS diagnostics accessible
E) Module temp failure
Cooling ON
SP: FE LV + HV inhibit (Case B)
DC-DC: Module power disable
(Case C-D)
SP: FE LV + HV inhibit (Case B)
DC-DC: Module power dsiable
(Case C-D)
F) Cooling failure
Cooling OFF
All power interlocked! (Case A)
All power interlocked! (Case A)
Pre-operation
Cooling OFF
SMC ramp & Opto com
Cooling ON
Module ramp
Cooling ON
Operation
Cooling ON
D. Ferrère
HSTD7, Aug 29th Sept 1st, 2009 in Hiroshima
21
Back-up slides
DCS Data & Interlock versus the Construction Phase & Test
Hybrid &
Module QA
Stave QA
Integration &
Tests
Barrel
commissioning
ID
commissioning
Cooling type
Water
Water or
evaporative
No cooling
Evaporative
Evaporative
Power type
Local power
SMC power
Module power
SMC power
(quick tests)
Final PS crate
SMC and Module
power
Final PS crate
SMC and Module
power
Type of test
FE Readout
FE readout
chain
GBT-Opt com
Conn. tests
Full readout
Full readout
PS DCS
Custom survey
Custom survey
NO
Final system –
PS Mon
Final system - PS
Mon
Environmental
survey
No
No
Connectivity test
Yes
Yes
SMC DCS
No
Yes
Yes for com test
Yes
Yes
Cooling
interlock
No
No
No
Yes (Warm cool)
Yes (Possibly
warm cool)
Module
interlock
Home made
Home made
No
Yes
Yes
SMC interlock
No
Home made
No
Yes
Yes
NB: Modules are never tested un-cooled either individually or on the stave
D. Ferrère
HSTD7, Aug 29th Sept 1st, 2009 in Hiroshima
22